[Corpora-List] CoNLL Shared Task 2015: Final Call for Proposals

[Erik TKS]

CoNLL Shared Task 2015: Final Call for Proposals

SIGNLL (ACL's Special Interest Group on Natural Language Learning)
invites proposals for the CoNLL Shared Task 2015. Since 1999, CoNLL
(the Conference on Computational Natural Language Learning) has
included a shared task in which training and test data are provided
by the organizers, which allows participating systems to be evaluated
and compared in a systematic way. Descriptions of the participating
systems and an evaluation of their performance are presented both at
the conference and in the proceedings. Previous shared tasks have
been devoted to chunking (1999, 2000), clausing (2001), named entity
recognition (2002, 2003), semantic role labeling (2004, 2005),
dependency parsing (2006, 2007), joint parsing of syntactic and
semantic dependencies (2008, 2009), hedge detection (2010),
co-reference resolution (2011, 2012), and grammatical error
correction (2013, 2014). For CoNLL-2015 proposals are invited for
any shared task involving natural language learning, including not
only natural language processing tasks and applications but also
the modeling of human language learning.

Proposals should contain the following information:

- Names and affiliations of the members of the organizing committee
   (including a designated contact person)
- Description of the shared task, including its relevance for the
   study of computational natural language learning
- Description of the process for selecting, acquiring and preparing the
   data for training and evaluation, including conditions of
   availability
- Description of the procedure for evaluating the performance of
   participating systems, including the availability of evaluation
   software
- Any other relevant information

A tentative timeline for the CoNLL Shared Task 2015 is as follows:

- November 15, 2014: general announcement (made by SIGNLL)
- December 17, 2014: call for interest in participation
- February 1, 2015: task website (first version with general
   information, task definition, examples and calendar)
- February 1, 2015: trial datasets (plus documentation and scorer)
- February 15, 2015: registration for the task (plus corpora licenses)
- March 1, 2015: training and development sets available
- May 5, 2015: test set available
- May 10, 2013: systems' outputs collected

Proposals should make clear that a similar schedule can be
implemented, especially for the preparation of resources
(trial/training/development/test data sets).

The deadline to submit a proposal is *** September 15, 2014 ***.
Proposals should be in PDF format, 2 to 3 pages long, even though
we encourage proposers to include annexes with examples that help
understanding the data and type of evaluation. Proposals should
be submitted by e-mail to proposals at signll.org, from
the email of the contact person, with email subject “CoNLL-2015
Shared Task Proposal”. Proposals will be reviewed by the SIGNLL
steering committee and proposers will be notified about the
selection before October 31. Proposers might be contacted
during the reviewing period to provide further information.

We ask potential proposers to indicate the intention to
submit by emailing proposals at signll.org. The
same email address can be used for questions about the
process, or to solicit early feedback for a proposal.
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora