[Corpora-List] Call for Participation: MWE 2008 Shared Task on Multiword Ranking

[Stefan Evert]

################################################################

  CALL FOR SHARED TASK PARTICIPATION

     LREC2008 - Towards a Shared Task for Multiword Expressions (MWE  
2008)

  endorsed by the ACL Special Interest Group on the Lexicon (SIGLEX)


Date: Sunday, 1 June 2008
Location: Marrakech, Morocco
Deadline: Friday, 14 March 2008 (Shared Task)

Workshop web page: http://multiword.sf.net/mwe2008/

#################################################################

The MWE 2008 workshop is primarily intended as a forum for  
collecting, sharing and exploiting MWE evaluation resources.  
Following an earlier call, numerous contributors have submitted data  
sets of various kinds, which can now be downloaded from the Multiword  
homepage: http://multiword.sourceforge.net/

With this call, we invite teams to participate in a SHARED TASK on  
MULTIWORD EXTRACTION, based on a subset of the new resources. For  
this first and exploratory evaluation campaign, we decided to focus  
on the simplest and most straightforward task -- ranking MWE  
candidates -- in order to make it easier for participating teams to  
adapt their existing systems before the deadline on March 14th.

Participating teams have to apply a MWE candidate ranking algorithm  
to at least two different data sets, provided in the form of a list  
of MWE candidates that have been manually annotated as true positives  
(TP) and false positives (FP). Evaluation is carried out by the teams  
themselves in terms of n-best precision and precision-recall graphs  
(sample R scripts for the evaluation will be provided on the shared  
task homepage soon). Further analysis and experiments are strongly  
encouraged (e.g. on different subsets of the candidates, or for  
different subtypes of MWE). Baseline results for standard association  
measures will be presented by the workshop organisers.

More information and downloads can be found on the shared task homepage:

   http://multiword.sourceforge.net/mwe2008/shared_task.html

Teams planning to participate in the shared task should also join the  
multiword mailing list for up-to-date information and discussions:

   https://lists.sourceforge.net/lists/listinfo/multiword-expressions

We expect a full-fledged competition with a more varied range of  
tasks to be organised in the near future, making use of the full  
collection of data sets from this workshop.


SUBMISSION INFORMATION

Shared task participants have to submit a short paper with complete  
evaluation results for their ranking algorithm on at least three  
different data sets. Submissions are not anonymous and will be  
briefly reviewed by the workshop organisers for complicance with the  
shared task requirements.

Papers must adhere to the format of LREC proceedings (preferably  
using the style files provided on the conference Web site) and must  
not exceed four (4) pages, including references. Only submissions in  
PDF format will be considered.

The papers must be submitted no later than 23:59 GMT on March 14, 2008.

Please submit your paper here: https://www.softconf.com/LREC2008/ 
MWE2008/submit.html

In addition, participants should make available their full candidate  
rankings and, if possible, an implementation of their ranking  
algorithm, to encourage further experiments. Details will be  
discussed on the multiword mailing list.


IMPORTANT DATES

8-page regular paper submission deadline: February 29, 2008
4-page resource paper submission deadline: February 29, 2008
4-page shared task paper submission deadline: March 14, 2008
Notification of acceptance: March 28, 2008
Camera ready papers due: April 4, 2008
Workshop date: June 1, 2008


WORKSHOP CHAIRS

Nicole Grégoire
University of Utrecht, The Netherlands

Stefan Evert
University of Osnabrueck, Germany

Brigitte Krenn
Austrian Research Institute for Artificial Intelligence (ÖFAI), Austria


CONTACT

For any inquiries regarding the workshop please contact Nicole  
Grégoire (Nicole.Gregoire at let.uu.nl).
_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora