[Corpora-List] SemEval-2010 Second Call for Task Proposals

[Katrin Erk]

[apologies if you receive multiple copies of this announcement]

SemEval-2010: 5th International Workshop on Semantic Evaluations

Second CALL FOR TASK PROPOSALS

The Senseval Committee invites proposals for tasks to be run as part
of SemEval-2010.  Following the spirit of the past SemEval-2007, the
nature of the tasks in previous Senseval editions has evolved to
include semantic analysis tasks outside of word sense disambiguation.

Thus we welcome proposals for any tasks that can test an automatic
system for semantic analysis of text, be it application dependent or
independent. We especially encourage tasks for different languages,
cross-lingual tasks, and tasks that are relevant to particular NLP
applications such as machine translation, information retrieval and
information extraction.

It will help us if tasks are designed in such that they can be run and
scored automatically from a central website (as in Senseval-3 and
SemEval-2007).
We'll be happy to help with task design, data formatting, and so on.

Information on the design of previous Senseval-1/2/3 tasks and
SemEval-2007 can be found at:

http://nlp.cs.swarthmore.edu/semeval/index.php
http://www.senseval.org/senseval3/
http://193.133.140.102/senseval2/
http://www.itri.brighton.ac.uk/events/senseval/ARCHIVE/index.html

The time period for SemEval-2010 has not yet been finalised, but it
will be held over a 1-2 month period in the first part of 2010.

SUBMISSION DETAILS

Proposals for tasks will ideally contain:

 - A description of the task (about 1 page)
 - How the training/testing data will be built and/or procured
 - The evaluation methodology to be used including clear evaluation
  criteria
 - The availability of the resources to the participants (copyright,
  costs, etc)
 - The resources required to prepare the task (time, money, and human)

If you are not yet at a point to provide outlines of all of these,
that's fine, but please give some thought to each, and present an
overview of your first ideas.  We will gladly give feedback.

Please submit proposals as soon as possible, preferably by electronic
mail in plain ASCII text to the SemEval-2010 chairs:

Katrin Erk, University of Texas at Austin (katrin.erk at mail.utexas.edu)
Carlo Strapparava, FBK-irst, Italy (strappa at fbk.eu)

IMPORTANT DATES

Sep 21, 2008  Intention to submit (send a short-paragraph description)
Oct 19, 2008  Submission deadline for task proposals
Nov 16, 2008  Notification of acceptance

In SemEval-2010 we are going to give plenty of time both in tasks
preparation phase, especially regarding the tuning of data sets, and
to task participants to prepare for the task and to tune their
systems. Along this rationale, we strongly recommend that each task
organizers come up with the presentation of trial data set, well in
advance (roughly, summer 2009). Please consider this point in
designing the task schedule.

_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora