[Corpora-List] SemEval-2010 Call for Task Proposals

[Katrin Erk]

SemEval-2010: 5th International Workshop on Semantic Evaluations

CALL FOR TASK PROPOSALS

The Senseval Committee invites proposals for tasks to be run as part
of SemEval-2010.  Following the spirit of the past SemEval-2007,
the nature of the tasks in previous Senseval editions has evolved
to include semantic analysis tasks outside of word sense disambiguation.

Thus we welcome proposals for any tasks that can test an automatic system
for semantic analysis of text, be it application dependent or
independent. We especially encourage tasks for different languages,
cross-lingual tasks, and tasks that are relevant to particular NLP
applications such as machine translation, information retrieval and
information extraction.

It will help us if tasks are designed in such that they can be run and
scored automatically from a central website (as in Senseval-3 and
SemEval-2007).
We'll be happy to help with task design, data formatting, and so on.

Information on the design of previous Senseval-1/2/3 tasks and SemEval-2007
can be found at:

http://nlp.cs.swarthmore.edu/semeval/index.php
http://www.senseval.org/senseval3/
http://193.133.140.102/senseval2/
http://www.itri.brighton.ac.uk/events/senseval/ARCHIVE/index.html

The time period for SemEval-2010 has not yet been finalised, but it will be
held over a 1-2 month period in the first part of 2010.

SUBMISSION DETAILS

Proposals for tasks will ideally contain:

 - A description of the task (about 1 page)
 - How the training/testing data will be built and/or procured
 - The evaluation methodology to be used including clear evaluation
   criteria
 - The availability of the resources to the participants (copyright,
   costs, etc)
 - The resources required to prepare the task (time, money, and human)

If you are not yet at a point to provide outlines of all of these,
that's fine, but please give some thought to each, and present an overview
of your first ideas.  We will gladly give feedback.

Please submit proposals as soon as possible, preferably by electronic
mail in plain ASCII text to the SemEval-2010 chairs:

Katrin Erk, University of Texas at Austin (katrin.erk at mail.utexas.edu)
Carlo Strapparava, FBK-irst, Italy (strappa at fbk.eu)

IMPORTANT DATES

Sep 21, 2008  Intention to submit (send a short-paragraph description)
Oct 19, 2008  Submission deadline for task proposals
Nov 16, 2008  Notification of acceptance

In SemEval-2010 we are going to give plenty of time both in tasks
preparation phase,
especially regarding the tuning of data sets, and to task participants
to prepare for the task and to tune their systems. Along this
rationale, we strongly
recommend that each task organizers come up with the presentation of
trial data set,
well in advance (roughly, summer 2009). Please consider this point in
designing the task schedule.

_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora