<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"
xmlns:ns0="urn:schemas-microsoft-com:office:smarttags">
<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:Consolas;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Tahoma","sans-serif";
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;}
p.stile6, li.stile6, div.stile6
{mso-style-name:stile6;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.apple-style-span
{mso-style-name:apple-style-span;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 2.0cm 2.0cm;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=IT link=blue vlink=purple>
<div class=Section1>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>CLEF 2010 – Evaluation LABs<o:p></o:p></span></p>
<p class=MsoPlainText><o:p> </o:p></p>
<p class=MsoPlainText><a href="http://www.clef2010.org/"><span lang=EN-US>http://www.clef2010.org/</span></a><span
lang=EN-US><o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:12.0pt;font-family:"Tahoma","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Call for Participation<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:12.0pt;font-family:"Tahoma","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoPlainText><b><span lang=EN-US>CLEF 2010 </span></b><span
lang=EN-US>will take the form of an independent, peer-reviewed <b>Conference</b>
organized in conjunction with a set of <b>Evaluation Labs<o:p></o:p></b></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Two different forms of labs are offered:
benchmarking activities which are very similar to the CLEF "tracks"
evaluation campaigns, and workshop-style labs that explore issues of
information access evaluation and related fields.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The results will be presented in a
series of workshops to be held in conjunction with the CLEF2010 conference: <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>20–23 September 2010, Padua, Italy<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><b><span lang=EN-GB style='font-size:10.0pt;font-family:
"Segoe UI","sans-serif"'><o:p> </o:p></span></b></p>
<p class=MsoPlainText><span lang=EN-US>Benchmarking activities<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>CLEF-IP<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>This year, CLEF-IP puts to use a
collection of almost 2 million patent documents in XML format with content in
English, German, and French. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The lab offers a Prior Art Candidate
Search task and a Classification task.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The first task will ask participants to
retrieve documents that are potential prior art to a given document. Topics
will be chosen as to stimulate multilingual retrieval. The second task will ask
participants to classify documents according to the International Patent
Classification scheme. Training data for both tasks will be available prior to
the topic sets release. Relevance assessment will be done using patent
citations for the first task and current patent classifications for the second
task. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Coordinator: Information Retrieval
Facility (AT). See <a
href="http://www.ir-facility.org/research/evaluation/clef-ip-10"><span lang=IT
style='color:windowtext;text-decoration:none'>http://www.ir-facility.org/research/evaluation/clef-ip-10</span></a><o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Cross-Language Image Retrieval
(ImageCLEF)<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>This track evaluates retrieval from
visual collections; both text and visual retrieval techniques are exploitable.
Four challenging tasks are foreseen: 1) retrieval from a Wikipedia collection
containing images and structured information in several languages; 2) medical
image retrieval with visual, semantic and mixed topics in several languages
with a data collection from the scientific literature; 3) detection of semantic
categories from robotic images (non-annotated collection, concepts to be
detected); 4) a photo annotation task that investigates automated semantic
annotation based on visual information with approaches based on Flickr user
tags and multimodal approaches. Track coordinators are U. of Applied Sciences
Western Switzerland (CH), Oregon Health and Science U. (US), CWI (NL), TELECOM
Bretagne (FR), Leiden University (NL), U. of Geneva (CH), Fraunhofer Society
(DE), IDIAP (CH). See also <a href="http://www.imageclef.org/"><span lang=IT
style='color:windowtext;text-decoration:none'>http://www.imageclef.org/</span></a>.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Uncovering Plagiarism, Authorship, and
Wikipedia Vandalism (PAN) - New this year<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>PAN @ CLEF 2010 divides into two tasks:<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>- Plagiarism Detection. Today's
plagiarism detection systems are faced with intricate situations, such as
obfuscated plagiarism or plagiarism within and across languages. Moreover, the
source of a plagiarism case may be hidden in a large collection of documents,
or it may not be available at all. Following the success of the 2009 campaign
on plagiarism detection, we will provide a revised evaluation corpus consisting
of artificial and simulated plagiarism.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>- Wikipedia Vandalism Detection.
Vandalism has always been one of Wikipedia's biggest problems. However, the
detection of vandalism is done mostly manually by volunteers, and research on
automatic vandalism detection is still in its infancy. Hence, solutions are to
be developed which aid Wikipedians in their efforts. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The lab is organized by the Bauhaus-Universität
Weimar, the Universidad Politécnica de Valencia, the University of the Aegean,
and the Bar-Ilan University. See http://pan.webis.de for more details.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>ResPubliQA<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Two separate tasks are proposed for the
ResPubliQA 2010 evaluation campaign which allow both passages and exact answers
(smallest exact demarcation) to be returned as the type of answer in response
to the same 200 input questions. Systems can also return NOA if they are not
confident of their answer.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The focus is on the direct comparison of
systems' performances among languages, a goal which is enabled by the adoption
of the multilingual parallel paragraph-aligned document collections (JRC-Acquis
and Europarl) of EU legislative documents, available in 9 languages, (i.e:
Bulgarian, Dutch, English, French, German, Italian, Portuguese, Romanian and
Spanish). <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The Lab is jointly coordinated by UNED,
CELCT and the University of Limerick. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>For more details visit the website http://celct.isti.cnr.it/ResPubliQA/<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>WePS - New this year<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>WePS-3 is a competitive evaluation
campaign which consists of two tasks concerning the Web Entity Search Problem: <br>
- Task 1 is related to Web People Search, and focuses on person name ambiguity
and person attribute extraction on Web pages. Given a set of web search results
for a person name, the tasks consists of clustering the pages according to the
different people sharing the name and extract certain biographical attributes
for each person.<br>
- Task 2 is related to Online Reputation Management for organizations, and
focuses on the problem of ambiguity for organization names and the relevance of
Web data for reputation management purposes. Given a set of Twitter entries
containing an (ambiguous) company name, and given the home page of the company,
the tasks consists of discriminating entries that do not refer to the company.<br>
WePS 3 is coordinated by three universities (UNED, New York University,
the University of Illinois at Chicago) and two corporate stakeholders: Intelius
Corp. and Llorente & Cuenca. More information at http://nlp.uned.es/weps.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Workshops<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Cross-lingual Expert Search - Bridging
CLIR and Social Media (CriES) - New this year<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>This workshop addresses the problem of
multi-lingual expert search in social media environments. The main topics are
multi-lingual expert retrieval methods, social media analysis with respect to
expert search, selection of datasets and evaluation of expert search results. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>In addition to the workshop we also
organize a pilot challenge:<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>- Workshop: We expect submissions
addressing the main topics including user characterization in multi-lingual
social media, community analysis for retrieval scenarios, user-centric
recommender algorithms, proposals of new social media datasets and evaluation
of cross-lingual expert search.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>- Pilot Challenge: The challenge is
based on a dataset from Yahoo!Answers, consisting of multi-lingual questions,
answers and user relations. Given a set of multi-lingual questions the task is
to retrieve relevant users that will most likely be able to answer the
questions. <o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Coordinators are KIT, U. of Koblenz and
U. of Bielefeld (DE). See http://www.multipla-project.org/cries for details.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>LogCLEF<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>The goal of LogCLEF is the analysis and
classification of queries in order to understand search behavior in
multilingual contexts and ultimately to improve search systems.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>A common data set will be distributed to
the participants. In coordination with the organizers, participating groups
will be devoted to different tasks in exploring and understanding the data.
Tasks will include the identification of the language of a query,
identification of sessions with more than one language, user clustering,
labelling named entities (esp. person names and geographic names) and linking
them to these entities (e.g. Wikipedia pages). Both search log and HTTP logs
for the 2007/2008 (the same period used this year), plus the search log of 2009
will be (most likely) available from The European Library.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>At the workshop, participants are
required to present their algorithms, their results and discuss what the
results tell about user behavior. The workshop will be the basis for a
definition of a set of competitive tasks for future studies on log analysis at
LogCLEF.<o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US>Coordinators are: University of
Hildesheim and University of Padua. See <a
href="http://www.uni-hildesheim.de/logclef/"><span lang=IT style='color:windowtext;
text-decoration:none'>http://www.uni-hildesheim.de/logclef/</span></a><o:p></o:p></span></p>
<p class=MsoPlainText><span lang=EN-US><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:12.0pt;font-family:"Tahoma","sans-serif"'><o:p> </o:p></span></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
lang=EN-US style='font-size:12.0pt;font-family:"Tahoma","sans-serif";
color:navy'>================================</span><span lang=EN-US
style='font-size:10.0pt;font-family:"Times New Roman","serif"'><o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>Pamela Forner<o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>CELCT (web: </span><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:blue'><a href="http://www.celct.it/" title="http://www.celct.it"><span
lang=EN-GB style='color:blue'>www.celct.it</span></a></span><span lang=EN-GB
style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>)<o:p></o:p></span></p>
<p class=MsoNormal><i><span lang=EN-GB style='font-size:10.0pt;font-family:
"Tahoma","sans-serif";color:navy'>Center for the Evaluation of Language and Communication
Technologies<o:p></o:p></span></i></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>Via alla Cascata 56/c </span><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>38100 Povo – TRENTO –</span><span lang=EN-GB
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'><ns0:country-region><ns0:place><span
lang=IT style='color:navy'>Italy</span></ns0:place></ns0:country-region></span><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>email: </span><span style='font-size:10.0pt'><a
href="mailto:forner@celct.it" title="mailto:fordyce@celct.it"><span lang=EN-GB
style='font-family:"Tahoma","sans-serif";color:blue'>forner@celct.it</span></a></span><span
lang=EN-GB style='font-size:10.0pt'><br>
</span><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>tel.: +39 0461 314 804</span><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Roman","serif"'><o:p></o:p></span></p>
<p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>fax: +39 0461 314 846</span><o:p></o:p></p>
<p class=MsoNormal> <o:p></o:p></p>
<p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:navy'>Secretary Phone: +39 0461 314 870</span><o:p></o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>