[Corpora-List] CoNLL Shared Task 2011: Call for Proposals

[Erik Tjong Kim Sang]

SIGNLL (ACL's Special Interest Group on Natural Language Learning)
invites proposals for the CoNLL Shared Task 2011. Since 1999, CoNLL
(the Conference on Computational Natural Language Learning) has
included a shared task in which training and test data are provided by
the organizers, which allows participating systems to be evaluated and
compared in a systematic way. Descriptions of the participating
systems and an evaluation of their performance are presented both at
the conference and in the proceedings. Previous shared tasks have been
devoted to chunking (1999, 2000), clausing (2001), named entity
recognition (2002, 2003), semantic role labeling (2004, 2005),
dependency parsing (2006, 2007), joint parsing of syntactic and
semantic dependencies (2008, 2009), and hedge detection (2010).  For
CoNLL 2011, to be held in conjunction with ACL in Portland, Oregon,
June 23-24, proposals are invited for any shared task involving
natural language learning, including not only natural language
processing tasks and applications but also the modeling of human
language learning.

Proposals should contain the following information:

- Names and affiliations of the members of the organizing committee
      (including a designated contact person)
- Description of the shared task, including its relevance for the
      study of computational natural language learning
- Description of the process for selecting, acquiring and preparing
      the data for training and evaluation, including conditions of
      availability
- Description of the procedure for evaluating the performance of
      participating systems, including the availability of evaluation
      software
- Any other relevant information

The expected timeline for the CoNLL shared task 2011 is as follows:

- November 1, 2010: general announcement (made by SIGNLL)
- December 1, 2010: call for interest in participation
- January 15, 2011: task website (first version with general information,
                     task definition, examples and calendar)
- January  15, 2011: trial datasets (plus documentation and scorer)
- February 1, 2011: registration for the task (plus corpora licenses)
- February 15, 2011: training and development sets available
- April 15, 2011: test set available
- April 20, 2011: systems' outputs collected

Proposals should make clear that a similar schedule can be implemented,
especially for the preparation of resources (trial/training/development/
test data sets).

Proposals should be in PDF format, maximum 2 pages, and sent by e-mail
to conll11st at stp.lingfil.uu.se no later than September 21. Proposals
will be reviewed by the SIGNLL steering committee and proposers will
be notified about the selection before October 15. Questions regarding
the proposals can be sent to conll11st at stp.lingfil.uu.se

_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora