<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 5.50.4522.1800" name=GENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
<DIV><FONT face=Arial size=2>APOLOGIES FOR MULTIPLE POSTINGS]
<DIV> </DIV>
<DIV> </DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<BR>
MULTILINGUAL QUESTION ANSWERING TRACK AT CLEF-2004<BR>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>
------------------------------------------------------------------<BR>
First Announcement and Call for
Participation<BR>
------------------------------------------------------------------</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>We are glad to announce that the second CLEF
Question Answering evaluation exercise is starting. <BR>For information about
the track and instructions for participants visit the QA@CLEF website
at:</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV><FONT face=Arial size=2>
http://clef-qa.itc.it</FONT></DIV>
<DIV><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify><FONT face=Arial size=2>Question Answering (QA) systems are
of great interest to the Language Engineering community because they combine
Information Retrieval and Natural Language Processing within the same task. QA
systems receive natural language queries (and not keywords) as input, process
large unstructured document collections, and return precise answers (and not
entire documents) as output.<BR></FONT><FONT face=Arial size=2>Within the
framework of the Cross Language Evaluation Forum (CLEF), a pilot evaluation
exercise for non-English and cross-language QA systems was successfully carried
out in 2003. Three monolingual tasks (with Dutch, Italian and Spanish questions)
and five bilingual tasks (where Dutch, French, German, Italian and Spanish
queries searched for an answer in an English target corpus) were proposed. Eight
groups tested their systems. The results showed that multilingual QA is a
promising field, and the experience was encouraging in terms of participation
and future perspectives. The CLEF-2003 campaign culminated in a workshop held in
Trondheim last August: proceedings are available at the CLEF website
(http://clef.iei.pi.cnr.it:2002/)</FONT></DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify><FONT face=Arial size=2> CLEF-2004 QA
TRACK</FONT></DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify><FONT face=Arial size=2>A new, challenging evaluation
exercise is planned for 2004, with six main tasks. </FONT><FONT face=Arial
size=2>Each main task is identified by a target language and is
divided into several sub-tasks. We plan to have Dutch, French, German, Italian,
Spanish and English as target languages.</FONT></DIV>
<DIV align=justify><FONT face=Arial size=2>In the monolingual tasks, queries,
document collection and responses are formulated in the same language. In the
cross-language tasks the document collection and the queries are written in two
different languages, and responses are due in the language of the target corpus.
For instance, in the “Dutch => Spanish” sub-task, participants are provided
with Dutch queries whose answers must be retrieved in a Spanish document
collection.<BR>Questions will be mostly factoid, but the test sets could include
also definition queries (like “Who/What is X”) and questions that do not have a
known answer in the target corpora.<BR>Participants will be allowed to submit
only one exact answer per question, and up to two runs per
sub-task.<BR></DIV></FONT>
<DIV align=justify><FONT face=Arial size=2> </DIV></FONT>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify><FONT face=Arial size=2>
SCHEDULE</FONT></DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV>
<DIV align=justify><FONT face=Arial size=2>Registration Open: January 15,
2004<BR>Corpora Release: February 2004<BR>Track Guidelines
Available: by February 2004<BR>Trial data: March 2004<BR>Test Sets
Release: May 10, 2004<BR>Submissions of Runs by Participants: May 17,
2004<BR>Release of Individual Results: from July 15, 2004<BR>Submission of
Papers for Working Notes: August 15, 2004<BR>CLEF Workshop (in Bath,
UK, after ECDL): 16-17 September 2004</FONT></DIV>
<DIV align=justify><FONT face=Arial size=2></FONT> </DIV><FONT face=Arial
size=2>
<DIV align=justify> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify>You are all encouraged to participate!<BR>To register, please
contact Carol Peters (carol.peters@isti.cnr.it), who is in charge of the
general co-ordination of the CLEF campaign. Registration will open on the
15th of January, 2004, via the CLEF website at
www.clef-campaign.org.<BR></DIV>
<DIV align=justify>A QA@CLEF mailing-list (clef-qa@itc.it) has been established:
to be included and for further information, please contact Bernardo Magnini
(magnini@itc.it) or Alessandro Vallin (vallin@itc.it). </DIV>
<DIV align=justify> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify> </DIV>
<DIV align=justify>--------------------------------------------------</DIV>
<DIV align=justify> ORGANIZING COMMITTEE</DIV>
<DIV align=justify><BR>ITC-irst</DIV>
<DIV align=justify>Centro per la Ricerca Scientifica e Tecnologica, Trento -
Italy<BR>Bernardo Magnini, Simone Romagnoli, Alessandro Vallin<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>UNED</DIV>
<DIV align=justify>Spanish Distance Learning University, Madrid - Spain
<BR>Felisa Verdejo, Anselmo Peñas, Jesús Herrera<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>ILLC</DIV>
<DIV align=justify>Language and Inference Technology Group, University of
Amsterdam - The Netherlands<BR>Maarten de Rijke<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>DFKI</DIV>
<DIV align=justify>German Research Center for Artificial Intelligence,
Saarbruecken - Germany <BR>Hans Uszkoreit<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>ELDA/ELRA</DIV>
<DIV align=justify>Evaluations and Language Resources Distribution Agency, Paris
- France<BR>Khalid Choukri<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>University of Limerick - Ireland<BR>Richard
Sutcliffe<BR></DIV>
<DIV align=justify>ISTI-CNR</DIV>
<DIV align=justify>Istituto di Scienza e Tecnologie dell’Informazione “A.
Faedo”, Pisa - Italy <BR>Carol Peters<BR></DIV>
<DIV align=justify> </DIV>
<DIV align=justify>NIST</DIV>
<DIV align=justify>National Institute of Standards and Technology, Gaithersburg,
Md. - United States of America<BR>Donna Harman</DIV>
<DIV
align=justify>-------------------------------------------------</DIV></FONT></FONT></DIV></BODY></HTML>