19.1476, FYI: Call for Participation: INFILE

LINGUIST Network linguist at LINGUISTLIST.ORG
Fri May 2 17:33:47 UTC 2008


LINGUIST List: Vol-19-1476. Fri May 02 2008. ISSN: 1068 - 4875.

Subject: 19.1476, FYI: Call for Participation: INFILE

Moderators: Anthony Aristar, Eastern Michigan U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews: Randall Eggert, U of Utah  
         <reviews at linguistlist.org> 

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, 
and donations from subscribers and publishers.

Editor for this issue: Matthew Lahrman <matt at linguistlist.org>
================================================================  

To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.

===========================Directory==============================  

1)
Date: 02-May-2008
From: Helene Mazo < mazo at elda.org >
Subject: Call for Participation: INFILE

 

	
-------------------------Message 1 ---------------------------------- 
Date: Fri, 02 May 2008 13:29:45
From: Helene Mazo [mazo at elda.org]
Subject: Call for Participation: INFILE
E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=19-1476.html&submissionid=177203&topicid=6&msgnumber=1  


Apologies for cross-postings


        Call for participation

             INFILE at CLEF2008

     Information, Filtering, Evaluation
     
          http://www.infile.org


INFILE welcomes participation of any institution to its first evaluation
campaign. This participation is free of charge and
participants can keep and use the development and evaluation data for free
after the evaluations for research and development purposes.

INFILE (INformation, Filtrage, Evaluation) is a cross-language adaptive
filtering evaluation campaign jointly organized by CEA,
Université de Lille 3 and ELDA. It is organized as a pilot track in CLEF
2008 and is supported by NIST TREC.

INFILE extends the last filtering track of TREC 2002 in the following ways:
-INFILE is crosslingual (English, French and Arabic); a corpus of  100,000
comparable news-wire stories from Agence France Presse (AFP)
 for each language is used for the evaluations.

-Evaluation will be performed using an automatic interrogation of test 
systems with a simulated user feedback. Each system will be able to
 use the feedback at any time to increase performance.

The participant systems will have to provide a Boolean decision for each
document according to each filtering  profile. A curve of the evolution of
efficiency will be computed.
Although cross-lingual systems are encouraged, the campaign is also open to
monolingual systems.


Tasks and languages
-------------------
Two tasks and three languages are considered. The first task is Information
Filtering on general news and events. For this task, participants will have
to classify each transmitted news-wire into
zero, one or more different profiles. 30 general profiles will be made
available in 3 languages (Arabic, English and French).

The second task is Information Filtering on science and technology domain.
Participants will have to associate to each news-wire zero, one or more
science and technology profiles.
A total of 20 profiles will be available in 3 languages (Arabic, English
and French).

For each task, participants are free to register to monolingual filtering
(e.g. information filtering using profiles and news-wires in the same
language) or to crosslingual filtering (e.g. information
filtering according to profiles in one language and news-wires in another
language).

Corpus
------
The corpus consists of 300,000 news-wires in Arabic, English and French
from the news agency Agence France Presse covering the 2004-2006 period.
The news-wires are related to general news and events information and are
comparable between Arabic, English and French.


Protocol description
---------------------
General information about the domain of profiles is given to each
participant. 15 days afterwards, 50 profiles are given to participants (30
general
profiles and 20 profiles related to science and technology). Profiles are
composed of a list of keywords (simple and complex noun phrases) and up to 3
documents illustrating each profile.
 
Then, news-wires are transmitted by the organizer to an automated 
interface of each participating system. The interface returns a
 Boolean response for each profile. After reception of this response, and
if requested by the participant, the organizer sends a feedback
 consisting of expected profile assignments for each document  submitted.
Participants may adapt their system at any time using this feedback.


Important Dates
---------------
    Registration Opens - Feb 11th, 2008
    Dry Run - June 2nd to June 14th, 2008
    Evaluation Run June 30th - July 19th
    Release of Human Assessments and Individual Results - August 4th, 2008
    Submission of Paper for Working Notes - 15 August 2008
    Workshop - 17-19 September 2008 CLEF Workshop

Contact
-------
    info at infile.org
    http://www.infile.org 



Linguistic Field(s): Computational Linguistics





 






-----------------------------------------------------------
LINGUIST List: Vol-19-1476	

	



More information about the LINGUIST mailing list