28.3781, FYI: Call for Participation: Machine Comprehension using Commonsense Knowledge

The LINGUIST List linguist at listserv.linguistlist.org
Thu Sep 14 15:58:38 UTC 2017


LINGUIST List: Vol-28-3781. Thu Sep 14 2017. ISSN: 1069 - 4875.

Subject: 28.3781, FYI: Call for Participation: Machine Comprehension using Commonsense Knowledge

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Helen Aristar-Dry, Robert Coté,
                                   Michael Czerniakowski)
Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Kenneth Steimel <ken at linguistlist.org>
================================================================


Date: Thu, 14 Sep 2017 11:58:30
From: Simon Ostermann [simono at coli.uni-saarland.de]
Subject: Call for Participation: Machine Comprehension using Commonsense Knowledge

 
Call for Participation
 
SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge

Web: https://competitions.codalab.org/competitions/17184

Description:

In this task, systems will be presented with narrative texts about everyday
activities and are required to answer multiple-choice questions based on this
text.
  
  This task assesses how the inclusion of commonsense knowledge in the form of
script knowledge would benefit machine comprehension systems. Script knowledge
is defined as the knowledge about everyday activities, i.e. sequences of
events describing stereotypical human activities (also called scenarios), for
example baking a cake, taking a bus, etc. In addition to what is mentioned in
the text, a substantial number of questions require inference using script
knowledge about different scenarios, i.e. answering the questions requires
knowledge beyond the facts mentioned in the text. 
  
  Each question is associated with a set of two or four answers. Answers are
short and limited to a few words. The texts used in this task cover more than
100 everyday scenarios, hence include a wide variety of human activities. The
trial data set can be downloaded from the task webpage.
  
  Participating systems should be able to answer such questions, using common
sense knowledge or, more specifically, script knowledge. We encourage
participants to make use of existing resources for script knowledge, such as
DeScript, RKP or OMCS; script knowledge representations such as narrative
chains, event embeddings or event paraphrase sets; and other knowledge sources
such as Wikipedia. Links to the resources are provided on the task webpage. 
  
  We do not put constraints on the form of common sense knowledge that is
used, i.e. participants are encouraged to use any external resources that
could improve their systems.

Important Dates: 

Aug 14, 2017  Trial Data Release, Practise Phase starts 
Sep 22, 2017  Training and Development Data Release 
Jan 9, 2018, 0:00 UTC  Test Data Release, Evaluation Phase starts 
Jan 28, 2018, 23:59 UTC  End of Evaluation Phase 

Contact:

Google Group/Mailing List:
https://groups.google.com/forum/#!forum/semeval2018-task11

Organizers:

Simon Ostermann, Saarland University, Germany 
http://www.coli.uni-saarland.de/~simono/
mail: simono (at) coli.uni-sb.de
   
Ashutosh Modi, Saarland University, Germany
http://www.coli.uni-saarland.de/~ashutosh/
mail: ashutosh (at) coli.uni-sb.de
   
Michael Roth, Saarland University, Germany
http://www.coli.uni-saarland.de/~mroth/
mail: mroth (at) coli.uni-sb.de
   
Stefan Thater, Saarland University, Germany
http://www.coli.uni-saarland.de/~stth/
mail: stth (at) coli.uni-sb.de
   
Manfred Pinkal, Saarland University, Germany
http://www.coli.uni-saarland.de/~pinkal/
mail: pinkal (at) coli.uni-sb.de
 



Linguistic Field(s): Computational Linguistics
                     Semantics





 



------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-28-3781	
----------------------------------------------------------






More information about the LINGUIST mailing list