[Corpora-List] Call for Participation: EVALUATING PHRASAL SEMANTICS (SemEval 2013, Task 5)

[Lushan Han]

Hi,

Regarding to the subtask A, I wonder if WordNet can be used in developing
the semantic measure? In the homepage of the task, I saw that dictionaries
are prohibited.

Thanks,

Lushan

On Fri, Jan 11, 2013 at 7:25 PM, Eugenie Giesbrecht <eugeniegiz at gmx.de>wrote:

>                                 ** CALL FOR PARTICIPATION **
>
>                         ** EVALUATING PHRASAL SEMANTICS **
>
>                                                 -- as part of --
>
>                                 ** SemEval-2013, TASK 5 **
>
>                         http://www.cs.york.ac.uk/semeval-2013/task5/
>
>
> Numerous tasks have focused on leveraging the meaning of words (noun
> categorization, TOEFL test), or of words in context (WSD, metonymy
> resolution, lexical substitution). These tasks have enjoyed a lot of
> success. A natural further step is to build up and evaluate models that can
> perform similar tasks in the face of multiword expressions and complex
> compositional structures.
>
> We suggest two subtasks that are designed to evaluate phrasal models.
> Participating systems may attempt any or all of the subtasks, in any or all
> of the languages provided in the datasets. The sub-tasks are the following:
>
> a) Semantic similarity of words and compositional phrases
>
> b) Evaluating the compositionality of phrases in context
>
>
> All subtasks are based on items drawn from the large-scale, freely
> available WaCky corpora (Baroni et al., 2009).
>
> ** Training data as well as further details are available ** @
> http://www.cs.york.ac.uk/semeval-2013/task5/
>
>
> (a) Semantic Similarity of Words and Compositional Phrases
>
> The aim of this subtask is to evaluate how well systems can judge the
> semantic similarity of a word and a short sequence of (two or more) words.
> For example, in the word-sequence pair: <contact, close interaction> the
> meaning of the sequence as a whole is semantically close to the meaning of
> the word. Contrarily, in the sequence-word pair: <megalomania, great
> madness> the meaning of the word is semantically different from the meaning
> of the sequence, although it is not entirely unrelated. This task is
> offered for English, German and Italian.
>
> == Subtask Organizers ==
>
> - Yannis Korkontzelos (National Centre for Text Mining, University of
> Manchester, UK)
> - Fabio Massimo Zanzotto (University of Rome "Tor Vergata", Italy)
>
>
>  (b) Semantic Compositionality in Context
>
> An interesting sub-problem of semantic compositionality is to decide
> whether a phrase is used in its literal or figurative meaning in a **given
> context**. For example, “big picture” may be used literally as in “Click
> here for a bigger picture”, or figuratively as in “To solve this problem,
> you have to look at the bigger picture”.
>
> Another example is “old school” which can also be used literally (“During
> the 1970's the hall of the old school was converted into the library”) or
> figuratively (“He will go down in history as one of the old school, a true
> gentlemen”).
>
> Participants of this subtask are provided with a list of target phrases
> together with real usage examples sampled from WaCky (Baroni et al., 2009)
> corpora. For each usage example, the task is to make a binary decision
> whether the target phrase is used literally or figuratively in this
> context. There may be cases where a binary decision is not possible, then
> the classification as “both” is allowed.
>
> For English, we define two different subsets that will be scored
> separately and in combination. One subset of target phrases is accompanied
> by a large number of contexts, which allows learning single classifiers for
> each phrase. The validation/test sets will only contain phrases that occur
> in the training set. This is comparable to the lexical sample task, which
> encourages the participation of supervised systems.
>
> The other subset contains target phrases together with a smaller number of
> usage contexts, probably favoring unsupervised approaches. The
> validation/test sets will contain new target phrases. This is comparable to
> the all-word task. Here, systems must grasp a notion of literal vs.
> idiomatic use in general, without training classifiers for each phrase.
>
> For German, only the second setup is offered.
>
>
> == Subtask Organizers ==
>
> - Chris Biemann, UKP Lab, Technische Universität Darmstadt, Germany
> - Eugenie Giesbrecht, Karlsruhe Institute of Technology, Germany
> - Torsten Zesch, UKP Lab, Technische Universität Darmstadt, Germany
>
>
> SemEval-2013 schedule
>
>         • February 15, 2013 - Registration Deadline [for Task Participants]
>         • March 1, 2013 onwards - Start of evaluation period [Task
> Dependent]
>         • March 15, 2013 - End of evaluation period
>         • April 9, 2013 - Paper submission deadline [TBC]
>         • April 23, 2013 - Reviews Due [TBC]
>         • May 4, 2013 - Camera ready Due [TBC]
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20130211/924aa196/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora