[Corpora-List] CLEF QA Track - Entrance Exams - 1st Call for Participation

[Anselmo Peñas]

**** Apologies for cross posting *****

---------------------------------------------

Entrance Exams 2014
http://nlp.uned.es/entrance-exams/

---------------------------------------------
CLEF Question Answering Track (CLEF QA Track)
a CLEF 2014 Lab
http://nlp.uned.es/clef-qa/
---------------------------------------------

Summary
Japanese University Entrance Exams include questions formulated at 
various levels of complexity and test a wide range of capabilities. The 
challenge of "Entrance Exams" aims at evaluating systems under the same 
conditions humans are evaluated to enter the University. Exams are 
created by the Japanese National Center for University Admissions Tests. 
The "Entrance Exams" corpus is provided by NII's Todai Robot Project and 
NTCIR RITE.

Task Description -----------------------------------------------------
Participant systems are asked to read a given document and answer 
questions. Questions are given in multiple-choice format, with four 
options from which a single answer must be selected. Questions are taken 
from reading tests of real university entrance exams, and may include a 
variety of question types that require a wide range of inference. 
Systems have to answer questions by referring to "common sense 
knowledge" that high school students who aim to enter the university are 
expected to have. Another important difference with previous QA 
exercises is that we do not intend to restrict question types. Any types 
of reading comprehension questions in real entrance exams will be 
included in the test data.

Language --------------------------------------------------------------
Test documents, questions, and options will be made available in 
English, French and Spanish. Japanese, Chinese, German and Russian are 
under consideration. Please contact the organization for expressions of 
interest in any particular language.

Evaluation ------------------------------------------------------------
Each test will receive an evaluation score between 0 and 1 using c at 1. 
This measure, used in previous CLEF QA Tracks, encourages systems to 
reduce the number of incorrect answers while maintaining the number of 
correct ones by leaving some questions unanswered.

Important Dates ------------------------------------------------------
Send the agreement form			by April 30, 2014
Test set release			May 6, 2014
Run submissions				May 15, 2014
Individual results to participants	May 20, 2014
Submission of Working Notes Papers	June 15, 2014
CLEF Workshop in Sheffield, UK		September 15-18, 2014

Contact ---------------------------------------------------------------
Anselmo Peñas (anselmo at lsi.uned.es)
Yusuke Miyao (yusuke at nii.ac.jp)

Organizing Committee -------------------------------------------------
Anselmo Peñas (UNED, Spain)
Eduard Hovy (Carnegie Mellon University, USA)
Pamela Forner (CELCT, Italy)
Noriko Kando (National Institute of Informatics, Japan)
Teruko Mitamura (Carnegie Mellon University, USA)
Yusuke Miyao (National Institute of Informatics, Japan)


-- 
Dr. Anselmo Peñas Padilla

Dpto. Lenguajes y Sistemas Informáticos, UNED
c./Juan del Rosal, 16
28040 Madrid, Spain

Correo-e: anselmo at lsi.uned.es
Pág. web: http://nlp.uned.es/~anselmo
Teléfono: +34 91 398 77 50

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora