<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 2.0cm 2.0cm;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=IT link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span lang=EN-US>[Apologies for cross-postings]<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>4TH TEXTUAL ENTAILMENT CHALLENGE AT TAC
2008 - SECOND ANNOUNCEMENT<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>(http://www.nist.gov/tac/tracks/2008/rte/index.html)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>The three Recognising Textual Entailment
(RTE) challenges (see RTE websites,
http://www.pascal-network.org/Challenges/RTE/;
http://www.pascal-network.org/Challenges/RTE2/;
http://www.pascal-network.org/Challenges/RTE3/) held so far have shown that the
interest for Textual Entailment research has steadily grown in years. The RTE
organizing committee is now glad to announce the 4th round of the Recognizing
Textual Entailment (RTE) Challenge, organized as a track within NIST’s
new Textual Analysis Conference (TAC, http://www.nist.gov/tac/).<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>WHAT IS NEW IN RTE4<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Although RTE4 will maintain the basic
structure of the previous challenges, some important changes will be introduced
in order to make the task more stimulating and bring the research in textual
entailment to the next level. In particular:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>- the task will include the 3-way
classification task piloted in RTE3 (see http://nlp.stanford.edu/RTE3-pilot/),
allowing the systems to make a further distinction between hypotheses unknown
on the basis of the texts and hypotheses contradicted or proved false by the texts
(submitting 2-way classifications will still be possible, and 2-way results
will be announced as well).<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>- the number of pairs will be increased to
300 in two of the application settings, namely Information Extraction and
Information Retrieval, as they have proven to be more difficult from the
analysis of the results in previous challenges. The test set will be made of
1000 pairs (300 each for IE and IR, 200 each for SUM and QA).<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>- there will be no development data set for
RTE4, and the participants are invited to use the past RTE data for training.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>- in 2008 the RTE challenge will be carried
out for the first time as a track at the Textual Analysis Conference, organised
by NIST (more details at http://www.nist.gov/tac/). Registration is open until
June 1, 2008, at the TAC 2008 RTE Track website
((http://www.nist.gov/tac/tracks/2008/rte/index.html)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>TASK AND DATA DESCRIPTION<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>RTE is the task of recognizing that the
meaning of one text, termed H(ypothesis), can be inferred by the content of
another, termed T(ext). Given a set of pairs of T's and H's as input, the
systems must recognise whether each T entails the corresponding H, deciding
whether:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-T entails H <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-T contradicts H, or shows it false<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-the veracity of H is unknown on the basis
of T. <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>System results will be compared to a
human-annotated gold-standard test corpus. Examples of three-way judgments
taken from last year's pilot task are given at the bottom of this message.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>As in previous challenges, the test data
sets will be based on multiple data sources, intended to be representative of
typical problems encountered by applied systems. Specifically, data types
corresponding to the following application areas will be used:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-Question Answering (QA): simulating a QA
scenario in which the hypothesized answer has to be inferred from the candidate
text passage<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-Information Retrieval (IR): choosing
propositional queries as hypotheses, and proposing relevant and irrelevant sentences
retrieved by IR systems as texts<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-Information Extraction/Relation Extraction
(IE): generating T-H pairs, picking positive and negative examples of typical
outputs of IE systems <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>-Summarization (SUM): converting sentence
pairs produced by multi-document text summarization systems into T-H pairs<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>More details can be found at the RTE-3
website (http://www.pascal-network.org/Challenges/RTE3/). The guidelines for
participants will be available at the track website shortly.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>THE RTE RESOURCE POOL AT ACLwiki<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>(http://aclweb.org/aclwiki/index.php?title=Recognizing_Textual_Entailment)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>The RTE Resource Pool, set up for the first
time during RTE3, serves as a portal and forum for publicizing and tracking
resources, and reporting on their use. RTE participants and other members of
the NLP community who develop or use relevant resources are encouraged to
contribute to this important resource.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>TENTATIVE SCHEDULE<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Registration deadline: EXTENDED
to 27 June 2008<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Test Set Release: 2
September 2008<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Submissions: 9
September 2008<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Release of individual evaluated results: 12
September 2008 <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Workshop: 17-18
November 2008, at TAC 2008<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>TRACK COORDINATORS AND ORGANIZERS:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal>Danilo Giampiccolo, CELCT (Trento), Italy (Coordinator,
giampiccolo@celct.it)<o:p></o:p></p>
<p class=MsoNormal>Hoa Dang, NIST, USA (Coordinator, hoa.dang@nist.gov)<o:p></o:p></p>
<p class=MsoNormal>Ido Dagan, Bar Ilan University, Israel<o:p></o:p></p>
<p class=MsoNormal>Bill Dolan, Microsoft Research, USA<o:p></o:p></p>
<p class=MsoNormal>Bernardo Magnini, FBK-irst (Trento), Italy <o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><span lang=EN-US>SCIENTIFIC COMMITTEE:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Johan Bos, University of Rome "La
Sapienza", Italy <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Christopher Manning, Stanford, USA<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Dan Moldovan, University of Texas at
Dallas, USA<o:p></o:p></span></p>
<p class=MsoNormal>Dan Roth, UIUC, USA<o:p></o:p></p>
<p class=MsoNormal>Annie Zaenen, Palo Alto Research Center, USA<o:p></o:p></p>
<p class=MsoNormal>Fabio Massimo Zanzotto, University of Rome "Tor
Vergata", Italy<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><span lang=EN-US>----------<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Examples of three-way judgments taken from
last year's pilot task:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: After his release, the clean-shaven
Magdy el-Nashar told reporters outside his home that he had nothing to do with
the July 7 transit attacks, which killed 52 people and the four
bombers.</t> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H:52 people and four bombers were killed on
July 7.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Entailment: YES<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: Mrs. Bush's approval ratings have
remained very high, above 80%, even as her husband's have recently dropped
below 50%.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H: 80% approve of Mr. Bush.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Entailment: NO<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: Recent Dakosaurus research comes from a
complete skull found in Argentina in 1996, studied by Diego Pol of Ohio State
University, Zulma Gasparini of Argentinas National University of La Plata, and
their colleagues.</t> <o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H: A complete Dakosaurus was discovered by
Diego Pol.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Entailment: UNKNOWN<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: The British tabloids portrayed Nicholas
Leeson as a working-class villain who single-handedly brought down Barings PLC,
a 233-year-old London merchant bank that helped finance the Napoleonic wars.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H: Barings was Britain's oldest merchant
bank.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Entailment: UNKOWN<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: The floods were exceptional since they
affected an extensive area across Europe from the UK to Spain and as far east
as the Black Sea coast. Economic losses amounted to EUR 9.2 bn in Germany, EUR
2.9 bn in Austria and EUR 2.3 bn in the Czech Republic. Total economic damage
exceeds EUR 15 bn.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H: Flooding in Europe causes major economic
losses.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>Entailment: YES<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US>T: Oscar-winning director Franco Zeffirelli
has been awarded an honorary knighthood for his "valuable services to
British performing arts".<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US>H: Italian director is awarded an honorary
Oscar.<o:p></o:p></span></p>
<p class=MsoNormal>Entailment: NO<o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>