26.4047, Calls: English, Spanish, Cognitive Sci, Computational Ling, Semantics, Translation/Germany

Mon Sep 14 21:06:04 UTC 2015

LINGUIST List: Vol-26-4047. Mon Sep 14 2015. ISSN: 1069 - 4875.

Subject: 26.4047, Calls: English, Spanish, Cognitive Sci, Computational Ling, Semantics, Translation/Germany

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Anthony Aristar, Helen Aristar-Dry, Sara Couture)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
              http://funddrive.linguistlist.org/donate/

Editor for this issue: Anna White <awhite at linguistlist.org>
================================================================

Date: Mon, 14 Sep 2015 17:05:40
From: Daniel Cer [danielcer at acm.org]
Subject: SemEval 2016 Task 1: Semantic Textual Similarity

Full Title: SemEval 2016 Task 1: Semantic Textual Similarity 
Short Title: STS 

Date: 10-Jan-2016 - 12-Aug-2016
Location: Berlin, Germany 
Contact Person: Daniel Cer
Meeting Email: danielcer at acm.org
Web Site: http://alt.qcri.org/semeval2016/task1/ 

Linguistic Field(s): Cognitive Science; Computational Linguistics; Semantics; Translation 

Subject Language(s): English (eng)
                     Spanish (spa)

Call Deadline: 10-Jan-2015 

Meeting Description:

SemEval 2016: Task 1
Semantic Textual Similarity (STS)

Semantic Textual Similarity (STS) measures the degree of equivalence in the underlying semantics of paired sentences. While making such an assessment is trivial for humans, constructing models that mimic human level performance represents a difficult computational semantics problem.

To stimulate research in this area and encourage the development of creative new approaches to modeling sentence level semantics, the STS shared task has been held annually since 2012, as part of the SemEval/*SEM family of workshops.

Task Definition:

Given two sentences, participant models are asked to produce a similarity score on a scale from 0 to 5, with 0 indicating semantic independence and 5 signifying semantic equivalence. Performance is assessed by the Pearson correlation between model assigned semantic similarity scores and human judgements. 

STS 2016 offers two tracks: STS Core and Cross-lingual STS. The former is a traditional STS task with paired monolingual sentences drawn from English data sources. The latter involves assessing paired English and Spanish sentences. 

2016 Data:

[STS Core]

This year's shared task includes five distinct evaluation sets:

* News Article Headlines
* Plagiarism Detection (new)
* Post-Edited Machine Translations (new)
* Q&A Answer-Answer
* Q&Q Question-Question (new)

[Cross-lingual STS]

We are excited to announce a new subtask involving cross-lingual pairings of English and Spanish sentences. The data will be sourced from: (i) comparable literary works in English and Spanish; (ii) data from the monolingual Core STS Task but with one sentence within each pair translated into Spanish. 

Organizers:

[STS Core] 	
Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab,
Aitor Gonzalez-Agirre, Weiwei Guo

[Cross-lingual STS] 	
Carmen Banea, Daniel Cer, Rada Mihalcea, Janyce Wiebe

Call for Participation:

Participation:

To register, please complete the following form: 

https://docs.google.com/forms/d/1HTRtP7B94gqdW5YuRfRh5pEBhukuRIh5hXR1nOEib90/viewform?usp=send_form

STS 2016 Website: http://alt.qcri.org/semeval2016/task1/

Mailing List: http://groups.google.com/group/STS-semeval.

Important Dates:

Evaluation start: January 10, 2016
Evaluation end: January 31, 2016
Paper submission due: February 28, 2016
Paper reviews due: March 31, 2016
Camera ready due: April 30, 2016
SemEval workshop: Summer 2016

References:

E. Agirre, C. Banea, C. Cardie, D. Cer, M. Diab, A. Gonzalez-Agirre, W. Guo, I. Lopez-Gazpio, M. Maritxalar, R. Mihalcea, G. Rigau, L. Uria and J. Wiebe. SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability. SemEval 2015.

E. Agirre, C. Banea, C. Cardie, D. Cer, M. Diab, A. Gonzalez-Agirre, W. Guo, R. Mihalcea, G. Rigau and J. Wiebe. SemEval-2014 Task 10: Multilingual Semantic Textual Similarity. SemEval 2014.

E. Agirre, D. Cer, M. Diab, A. Gonzalez-Agirre and W. Guo. *SEM 2013 shared task: Semantic Textual Similarity. *SEM 2013.

E. Agirre, D. Cer, M. Diab and A. Gonzalez-Agirre. SemEval-2012 Task 6: A Pilot on Semantic Textual Similarity. SemEval 2012.

----------------------------------------------------------
LINGUIST List: Vol-26-4047	
----------------------------------------------------------