[Corpora-List] CLEF QA Track - Entrance Exams - 1st Call for Participation

[Anselmo Peñas]

> **** Apologies for cross posting *****
> 
> ---------------------------------------------
> 
> Entrance Exams 2014
> http://nlp.uned.es/entrance-exams/
> 
> ---------------------------------------------
> CLEF Question Answering Track (CLEF QA Track)
> a CLEF 2014 Lab
> http://nlp.uned.es/clef-qa/
> ---------------------------------------------
> 
> Summary
> Japanese University Entrance Exams include questions formulated at 
> various levels of complexity and test a wide range of capabilities. The 
> challenge of "Entrance Exams" aims at evaluating systems under the same 
> conditions humans are evaluated to enter the University. Exams are 
> created by the Japanese National Center for University Admissions Tests. 
> The "Entrance Exams" corpus is provided by NII's Todai Robot Project and 
> NTCIR RITE.
> 
> Task Description -----------------------------------------------------
> Participant systems are asked to read a given document and answer 
> questions. Questions are given in multiple-choice format, with four 
> options from which a single answer must be selected. Questions are taken 
> from reading tests of real university entrance exams, and may include a 
> variety of question types that require a wide range of inference. 
> Systems have to answer questions by referring to "common sense 
> knowledge" that high school students who aim to enter the university are 
> expected to have. Another important difference with previous QA 
> exercises is that we do not intend to restrict question types. Any types 
> of reading comprehension questions in real entrance exams will be 
> included in the test data.
> 
> Language --------------------------------------------------------------
> Test documents, questions, and options will be made available in 
> English, French and Spanish. Japanese, Chinese, German and Russian are 
> under consideration. Please contact the organization for expressions of 
> interest in any particular language.
> 
> Evaluation ------------------------------------------------------------
> Each test will receive an evaluation score between 0 and 1 using c at 1. 
> This measure, used in previous CLEF QA Tracks, encourages systems to 
> reduce the number of incorrect answers while maintaining the number of 
> correct ones by leaving some questions unanswered.
> 
> Important Dates ------------------------------------------------------
> Send the agreement form            by April 30, 2014
> Test set release            May 6, 2014
> Run submissions                May 15, 2014
> Individual results to participants    May 20, 2014
> Submission of Working Notes Papers    June 15, 2014
> CLEF Workshop in Sheffield, UK        September 15-18, 2014
> 
> Contact ---------------------------------------------------------------
> Anselmo Peñas (anselmo at lsi.uned.es)
> Yusuke Miyao (yusuke at nii.ac.jp)
> 
> Organizing Committee -------------------------------------------------
> Anselmo Peñas (UNED, Spain)
> Eduard Hovy (Carnegie Mellon University, USA)
> Pamela Forner (CELCT, Italy)
> Noriko Kando (National Institute of Informatics, Japan)
> Teruko Mitamura (Carnegie Mellon University, USA)
> Yusuke Miyao (National Institute of Informatics, Japan)
> 
> 
> -- 
> Dr. Anselmo Peñas Padilla
> 
> Dpto. Lenguajes y Sistemas Informáticos, UNED
> c./Juan del Rosal, 16
> 28040 Madrid, Spain
> 
> Correo-e: anselmo at lsi.uned.es
> Pág. web: http://nlp.uned.es/~anselmo

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora