17.1577, FYI: SemEval-1 / SensEval-4: Call for Task Proposals

[linguist at LINGUISTLIST.ORG]

LINGUIST List: Vol-17-1577. Wed May 24 2006. ISSN: 1068 - 4875.

Subject: 17.1577, FYI: SemEval-1 / SensEval-4: Call for Task Proposals

Moderators: Anthony Aristar, Wayne State U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews (reviews at linguistlist.org) 
        Sheila Dooley, U of Arizona  
        Terry Langendoen, U of Arizona  

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.

Editor for this issue: Svetlana Aksenova <svetlana at linguistlist.org>
================================================================  

To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.


===========================Directory==============================  

1)
Date: 23-May-2006
From: Eneko Agirre < e.agirre at ehu.es >
Subject: SemEval-1 / SensEval-4: Call for Task Proposals 

	
-------------------------Message 1 ---------------------------------- 
Date: Wed, 24 May 2006 09:58:01
From: Eneko Agirre < e.agirre at ehu.es >
Subject: SemEval-1 / SensEval-4: Call for Task Proposals 
 


SEMEVAL-1 / SENSEVAL-4

CALL FOR TASK PROPOSALS

The Senseval Committee invites proposals for tasks to be run as part
of SemEval-1 and Senseval-4. As the nature of the tasks in Senseval
has evolved to include semantic analysis tasks outside of word sense
disambiguation, the Senseval Committee is pleased to announce the
debut of ''SemEval: Evaluating Semantics''. In 2007, the SemEval
workshop will retain the Senseval name to help ease the transition to
the new name.

We welcome proposals for any tasks that can test an automatic system
for semantic analysis of text, be it application dependent or
independent. We especially encourage tasks for different languages,
cross-lingual tasks, and tasks that are relevant to particular NLP
applications such as machine translation, information retrieval and
information extraction.

It will help us if tasks are designed in such a way that they can be
run and scored automatically from a central website (as in
Senseval-3). We will be happy to help with task design, data
formatting, and so on.

Information on the design of tasks from Senseval-1, -2, and -3 can be
found at:
http://www.senseval.org/senseval3/
http://193.133.140.102/senseval2/
http://www.itri.brighton.ac.uk/events/senseval/ARCHIVE/index.html

The time period for SemEval-1/Senseval-4 has not yet been finalised,
but it will likely be held over a 45-day period sometime during the
first quarter of 2007.

SUBMISSION DETAILS

Proposals for tasks will ideally contain:

- A description of the task (max 1 page)
- How the training/testing data will be built and/or procured
- The evaluation methodology to be used, including clear evaluation
  criteria
- The anticipated availability of necessary resources to the
  participants (copyright, etc)
- The resources required to prepare the task (computation and annotation
  time, etc)

If you are not yet at a point to provide outlines of all of these,
that is acceptable, but please give some thought to each, and present
a sketch of your first ideas. We will gladly give feedback.

Please submit proposals as soon as possible, preferably by electronic
mail in plain ASCII text to the SemEval-1 / Senseval-4 email address:
semeval at nlp.cs.swarthmore.edu

IMPORTANT DATES

July 1, 2006  Submission deadline for task proposals
July 15, 2006 Notification of acceptance

TENTATIVE SCHEDULE

November 1, 2006     Preliminary call for participation
January 1, 2007      Call for participation, release of trial data
March 2007           Estimated Evaluation Period (45 days in total)
Summer 2007          Workshop, possibly co-located with ACL or NAACL

SEMEVAL-1 / SENSEVAL-4 CHAIRS

Richard Wicentowski, Swarthmore College
Lluís Màrquez, Technical University of Catalonia (UPC)
Eneko Agirre, University of the Basque Country (UPV-EHU)

SEMEVAL-1 / SENSEVAL-4 WEBSITE

http://nlp.cs.swarthmore.edu/semeval/ 



Linguistic Field(s): Computational Linguistics
                     Semantics





 




-----------------------------------------------------------
LINGUIST List: Vol-17-1577