21.525, Qs: Call for Participants: SemEval Shared Task 10

Tue Feb 2 18:12:21 UTC 2010

LINGUIST List: Vol-21-525. Tue Feb 02 2010. ISSN: 1068 - 4875.

Subject: 21.525, Qs: Call for Participants: SemEval Shared Task 10

Moderators: Anthony Aristar, Eastern Michigan U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>

Reviews: Monica Macaulay, U of Wisconsin-Madison  
Eric Raimy, U of Wisconsin-Madison  
Joseph Salmons, U of Wisconsin-Madison  
Anja Wanner, U of Wisconsin-Madison  
       <reviews at linguistlist.org> 

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, 
and donations from subscribers and publishers.

Editor for this issue: Danielle St. Jean <danielle at linguistlist.org>
================================================================  

We'd like to remind readers that the responses to queries are usually
best posted to the individual asking the question. That individual is
then strongly encouraged to post a summary to the list. This policy was
instituted to help control the huge volume of mail on LINGUIST; so we
would appreciate your cooperating with it whenever it seems appropriate.

In addition to posting a summary, we'd like to remind people that it
is usually a good idea to personally thank those individuals who have
taken the trouble to respond to the query.

To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.

===========================Directory==============================  

1)
Date: 25-Jan-2010
From: Josef Ruppenhofer < josefr at coli.uni-sb.de >
Subject: Call for Participants: SemEval Shared Task 10

-------------------------Message 1 ---------------------------------- 
Date: Tue, 02 Feb 2010 13:09:06
From: Josef Ruppenhofer [josefr at coli.uni-sb.de]
Subject: Call for Participants: SemEval Shared Task 10

E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=21-525.html&submissionid=2607775&topicid=8&msgnumber=1

Call for Participation
SemEval-2010 Shared Task #10:

Linking Events and Their Participants in Discourse  

http://www.coli.uni-saarland.de/projects/semeval2010_FG/

Training Data now available

This shared task should appeal to researchers working on:
- Semantic role labelling (FrameNet or PropBank)
- Co-reference resolution, or 
- Information extraction.

Background:
Semantic role labelling (SRL) has traditionally been viewed as 
sentence-internal problem. However, it is clear that there is an interplay
between local semantic argument structure and the surrounding 
discourse. In this shared task, we would like to take SRL beyond the 
domain of isolated sentences by linking local semantic argument 
structures to the wider discourse context. In particular, we aim to find 
fillers for roles which are left unfilled in the local context (null 
instantiations, NIs). An example is given below, where the 'charges' 
role ('arg2' in PropBank) of "cleared" is left empty but can be linked to 
"murder" in the previous sentence: 

"In a lengthy court case the defendant was tried for murder." 
"In the end, he was cleared." 

Short Task Description:
Participants can tackle either (or both) of the following two tasks, which
will be evaluated independently: 

Full Task:
The test set will be annotated with GOLD Standard frames. For each 
lexical unit annotated with a frame, participants have to:
- find the semantic arguments (role recognition)
- label them with the correct role (role labelling)
- find links between null instantiated roles and their fillers in the
wider discourse context if such links exist (NI linking)

NIs-only Task:
The test set will be annotated with GOLD Standard local semantic 
argument structure; only the fillers of null instantiations have to be 
found. 

Note: We provide the annotated data in both FrameNet and PropBank 
style. Participants can choose with which framework they want to work. 
The two frameworks are evaluated separately.

For more details see the task web site
(http://www.coli.uni-saarland.de/projects/semeval2010_FG/) and the
following paper: 

Josef Ruppenhofer, Caroline Sporleder, Roser Morante, Collin Baker 
and Martha Palmer. 'SemEval-2010 Task 10: Linking Events and Their 
Participants in Discourse'. The NAACL-HLT 2009 Workshop on 
Semantic Evaluations: Recent Achievements and Future Directions 
(SEW-09), Boulder, Colorado, USA, June 4, 2009.  

http://www.coli.uni-saarland.de/~csporled/papers/sew09.pdf

Data Sets:
We provide training and (later) test data of running text from the fiction
domain. The data sets are freely available and downloadable from the
SemEval website (http://semeval2.fbk.eu/semeval2.php). The training 
set for both tasks is annotated with:
- GOLD Standard semantic argument structure (FrameNet and 
PropBank style)
- linking information for null instantiations
- co-reference chains (for evaluation purposes)

Participants are also free to make use of external resources, such as 
the FrameNet (FN) or PropBank/NomBank releases. (We distribute the 
FN-style training set with an intermediate FrameNet release (1.4 
alpha). 

Task Organisers:
- Josef Ruppenhofer, Saarland University, Germany
- Caroline Sporleder, Saarland University, Germany
- Roser Morante, University of Antwerp, Belgium
- Collin Baker, ICSI, Berkeley, USA
- Martha Palmer, University of Colorado, Boulder, USA

Contact:
If you're interested in participating in the shared task, please email 
Josef Ruppenhofer to inquire about joining our Google group: 
http://groups.google.com/group/semeval2010-task10

Task #10 website: 
http://www.coli.uni-saarland.de/projects/semeval2010_FG/

SemEval 2010 website: 
http://semeval2.fbk.eu/semeval2.php 

Linguistic Field(s): Computational Linguistics
                     Text/Corpus Linguistics

-----------------------------------------------------------
LINGUIST List: Vol-21-525	