[Corpora-List] Call for participation: WSD-QA task at CLEF2008

[Eneko Agirre]

Apologies for cross-postings



			Call for participation

			   QA-WSD at CLEF2008

		      An evaluation exercise on

		      Word Sense Disambiguation
		for (Cross-Lingual) Question Answering

		     http://ixa2.si.ehu.es/qawsd



The QA-WSD task will bring semantic and retrieval evaluation
together. The participants will be offered the same queries and
document collections from the main QA exercise which have been
annotated by systems for word sense disambiguation (WSD). The goal of
the task is to test whether WSD can be used beneficially for Question
Answering. Note that there will be another related task on WSD and
CLIR (http://ixa2.si.ehu.es/clirwsd).

The exercise scenario is event-targeted QA on a news document
collection. In the QA-WSD track only English monolingual and Spanish
to English bilingual tasks are offered, i.e. English will be the only
target language, and queries will be available on both English and
Spanish. The queries will be the same as for the main QA exercise, and
the participation will follow the same process, except for the use of
the sense-annotated data.

A large number of questions will be topic-related, i.e. clusters of
questions which are related to the same topic and possibly contain
anaphoric references between one question and the other
questions. Contrary to the main QA task, Wikipedia articles are not
included, and thus systems need to reply to the questions that have an
answer in the news document collection (more details below).

The goal of this task is to evaluate whether word sense information
can help in certain queries. For this reason, participants are
required to send two runs for each of the monolingual/bilingual tasks
where they participate: one which does not use sense annotations and
another one which does use sense annotations. Whenever possible, the
only difference between the two runs should be solely the use or not
of the sense information. Participants which send a single run will be
discarded from the evaluation.

The WSD data is based on WordNet version 1.6 and will be supplemented
with data from the English and Spanish WordNets in order to test
different expansion strategies. Several leading WSD experts will run
their systems, and provide those WSD results for the participants to
use.

Time Schedule
  Registration Opens - 15 February 2008 (closes 28 April 2008)
  Release of documents with annotations - from 1 March 2008
  Question Release - 5 May 2008
  Submission of Runs by Participants - 9 May 2008
  Release of Relevance Assessments and Individual Results - 1 July 2008
  Submission of Paper for Working Notes - 13 August 2008
  Workshop - 17-19 September 2008

Contact
  Eneko Agirre, University of the Basque Country, e.agirre at ehu es
  Anselmo Peñas, UNED, anselmo at lsi uned es

For more details please visit
    http://ixa2.si.ehu.es/qawsd


Please join the mailing list:
   http://groups.google.com/group/qawsd





_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora