<div dir="ltr"><p style="text-align:left;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-family:Arial;vertical-align:baseline;white-space:pre-wrap">(sorry for cross-posting)</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center">
<span style="font-size:16px;font-family:Arial;font-weight:bold;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center">
<span style="font-size:16px;font-family:Arial;font-weight:bold;vertical-align:baseline;white-space:pre-wrap">Call For Participation</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center"><span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span><span style="font-size:16px;font-family:Arial;font-weight:bold;vertical-align:baseline;white-space:pre-wrap">Cross-Level Semantic Similarity</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center"><span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">SemEval 2014 - Task 3</span></p>
<br><font size="3" style="font-weight:bold"><span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center">
<a href="http://alt.qcri.org/semeval2014/task3/" style="text-decoration:none" target="_blank"><span style="font-size:16px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">http://alt.qcri.org/semeval2014/task3/</span></a><span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"></span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center"><span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt;text-align:center">
<span style="font-size:16px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;font-family:Arial">To what extent is the meaning of the sentence “do u know where i can watch free older movies online without download?” preserved in the phrase “streaming vintage movies for free”? Or, how similar is “circumscribe” to the phrases “beating around the bush” and “avoiding the topic”? The aim of Task 3 is to evaluate semantic similarity when comparing items of different syntactic and semantic scales, such as paragraphs. sentences, phrases, words, and senses.</span></p>
<p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-weight:normal;font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">INTRODUCTION</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><u><br></u></span><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:normal">Semantic similarity is an essential component of many applications in Natural Language Processing (NLP). Task 3 provides an evaluation for semantic similarity across different types of text, which we refer to as lexical levels. Unlike prior SemEval tasks on textual similarity that have focused on comparing similar-sized texts, this task evaluates the case where a smaller text or sense is compared to the meaning of a larger text. Specifically, this task encompasses four types of semantic similarity comparisons:</span></p>
<br><font style="font-weight:bold"><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:normal"></span></font><ol style="font-weight:normal;margin-top:0pt;margin-bottom:0pt">
<li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:13px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">paragraph to sentence,</span></p>
</li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:13px;background-color:transparent;font-family:Arial"><p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">sentence to phrase,</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:13px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">phrase to word, and</span></p></li><li dir="ltr" style="vertical-align:baseline;list-style-type:decimal;font-size:13px;background-color:transparent;font-family:Arial">
<p dir="ltr" style="line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">word to sense.</span></p></li></ol><br><font style="font-weight:bold"><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial;font-weight:normal"></span></font><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Task 3 unifies multiple objectives from different areas of NLP under a single task, e.g., Paraphrasing, Summarization, and Compositional Semantics. </span><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">One of the major motivations of this task is to produce systems that handle all comparison types, thereby freeing downstream NLP applications from needing to consider the type of text being compared.</span><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial"></span></p>
<div style="font-weight:normal"><b style="font-weight:normal"><br></b></div><br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">TASK DETAILS</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial">Task participants are provided with pairs of each comparison type and asked to rate to what extent does the smaller item preserve the meaning of the larger item. For example, given a sentence and a paragraph, a system would assess how similar is the meaning of the sentence to the meaning of the paragraph. Ideally, a high-similarity sentence would summarize overall meaning of the paragraph. Similarly, in the case of word to sense, we assess how well the semantics a word is captured by a WordNet 3.0 sense.</span><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"></span></p>
<div style="font-weight:normal"><b style="font-weight:normal"><br></b></div><br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">PARTICIPATING</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Teams are free to participate in one, some, or all comparison types. Given the unified setting of the task, we especially encourage systems that handle more than one comparison type. However, we also allow specialized systems that target only a single comparison type. </span></p>
<br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Interested teams are encouraged to join the task’s </span><a href="https://groups.google.com/d/forum/semeval-2014-clss" style="text-decoration:none" target="_blank"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">mailing list</span></a><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> for discussion and announcements.</span></p>
<div style="font-weight:normal"><b style="font-weight:normal"><br></b></div><br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">EVALUATION</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Systems will be evaluated against human similarity scores using both rank-based and score-based comparisons. See the task’s </span><a href="http://alt.qcri.org/semeval2014/task3/index.php?id=evaluation" style="text-decoration:none" target="_blank"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">Evaluation</span></a><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> page for further details.</span></p>
<br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">DATASETS</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">The Task 3 training data set is now available and contains 500 scored pairs for each comparison type to use in building and training systems. Please see the task’s </span><a href="http://alt.qcri.org/semeval2014/task3/index.php?id=data-and-tools" style="text-decoration:none" target="_blank"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">Data</span></a><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> page for further details.</span></p>
<br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">IMPORTANT DATES</span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">
</span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Test data ready: March 10, 2014</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Evaluation start: March 15, 2014</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Evaluation end: March 30, 2014</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Paper submission due: April 30, 2014 [TBC]</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Paper reviews due: May 30, 2014</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• Camera ready due: June 30, 2014</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">• </span><span style="vertical-align:baseline;font-size:13px;white-space:pre-wrap;background-color:transparent;font-family:Arial">SemEval workshop August 23-24, 2014, co-located with COLING and *SEM in Dublin, Ireland.</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">MORE INFORMATION</span></p>
<p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">The Semeval-2014 Task 3 website includes details on the training data, evaluation, and examples of the comparison types:</span></p>
<br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> </span><a href="http://alt.qcri.org/semeval2014/task3/" style="text-decoration:none" target="_blank"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">http://alt.qcri.org/semeval2014/task3/</span></a><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br>
</span><span style="font-family:Arial;font-size:13px;white-space:pre-wrap;line-height:1.15">If interested in the task, please join our mailing list for updates:</span></p>
<br><font style="font-weight:bold"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap;font-weight:normal"></span></font><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"> </span><a href="https://groups.google.com/d/forum/semeval-2014-clss" style="text-decoration:none" target="_blank"><span style="font-size:13px;font-family:Arial;text-decoration:underline;vertical-align:baseline;white-space:pre-wrap">https://groups.google.com/d/forum/semeval-2014-clss</span></a><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"></span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">ORGANIZERS</span></p>
<p dir="ltr" style="font-weight:bold;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br></span></p><p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt">
<span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">David Jurgens (</span><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">jurgens@uniroma</span><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">), Sapienza University of Rome, Italy</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Mohammad Taher Pilehvar (</span><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">pilehvar@uniroma</span><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">), Sapienza University of Rome, Italy</span></p>
<p dir="ltr" style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">Roberto Navigli (</span><span style="font-size:13px;font-family:Arial;color:rgb(17,85,204);vertical-align:baseline;white-space:pre-wrap">navigli@uniroma</span><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">), Sapienza University of Rome, Italy</span><span style="font-size:16px;font-family:Arial;font-weight:bold;vertical-align:baseline;white-space:pre-wrap"></span></p>
<p style="font-weight:normal;line-height:1.15;margin-top:0pt;margin-bottom:0pt"><font color="#000000"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap">(uniroma=><a href="http://di.uniroma1.it">di.uniroma1.it</a></span></font><span style="font-family:Arial;font-size:13px;white-space:pre-wrap;line-height:1.15">)</span></p>
<p></p><p style="font-weight:bold"></p><p style="font-weight:normal"></p><div style="font-weight:normal"><span style="font-size:13px;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><br>
</span></div></div>