[Corpora-List] SemEval-2010 Task 12: Parser Evaluation using Textual Entailments (PETE)

[Deniz Yuret]

-- Apologies for multiple postings of this CFP --

    SemEval-2010 Shared Task #12
    Parser Evaluation using Textual Entailments (PETE)
    (http://pete.yuret.com)

The purpose of this e-mail is to encourage participation in the task
"Parser Evaluation using Textual Entailments" in the 5th International
Workshop on Semantic Evaluations, SemEval-2010
(http://semeval2.fbk.eu/semeval2.php) collocated with ACL-2010, July
15-16, Uppsala.

This shared task should be of interest to researchers working on
* parsing
* semantic relation extraction
* recognizing textual entailments

DESCRIPTION:

Parser Evaluation using Textual Entailments is a shared task in the
SemEval-2010 Evaluation Exercises on Semantic Evaluation.  The task
involves recognizing textual entailments (RTE) based solely on
syntactic information:

* The man with the hat was tired.
* The man was tired. (YES)
* The hat was tired. (NO)

Our goals in introducing this task are:

* To focus parser evaluation on semantically relevant phenomena.
* To introduce a parser evaluation scheme that is formalism independent.
* To introduce a targeted textual entailment task focused on a single
linguistic competence.
* To be able to collect high quality evaluation data from untrained annotators.

The following criteria were used when constructing the entailments:

* They should be decidable using only syntactic inference.
* They should be easy to decide by untrained annotators.
* They should be challenging for state of the art parsers.

You can find more details about the entailment generation process at
the task website (http://pete.yuret.com).  The trial dataset can be
downloaded from the task website or the SemEval website
(http://semeval2.fbk.eu).  There will be no training data.  The
evaluation will be similar to the past RTE tasks.  There is a Google
group (http://groups.google.com/group/semeval-pete) for task related
messages.

INSTRUCTIONS:

* join the mailing list (http://groups.google.com/group/semeval-pete)
* register in SemEval website (http://semeval2.fbk.eu)
* download the trial data from task website (http://pete.yuret.com) or
SemEval website (http://semeval2.fbk.eu)
* download task guide from task website (http://pete.yuret.com/guide)
* download test data from SemEval website (http://semeval2.fbk.eu)
* upload results to SemEval website (http://semeval2.fbk.eu)

IMPORTANT DATES:

* February 19 - the development data available.
* March 26 - the test data available.
* April 2 - end of submission period for the task.
* April 17 - Submission of description papers.
* May 6 - Notification of acceptance.
* July 15-16 - Workshop at ACL 2010, Uppsala.

CONTACT:
* Deniz Yuret  dyuret at ku.edu.tr

_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora