<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:p="urn:schemas-microsoft-com:office:powerpoint" xmlns:a="urn:schemas-microsoft-com:office:access" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:s="uuid:BDC6E3F0-6DA3-11d1-A2A3-00AA00C14882" xmlns:rs="urn:schemas-microsoft-com:rowset" xmlns:z="#RowsetSchema" xmlns:b="urn:schemas-microsoft-com:office:publisher" xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" xmlns:c="urn:schemas-microsoft-com:office:component:spreadsheet" xmlns:odc="urn:schemas-microsoft-com:office:odc" xmlns:oa="urn:schemas-microsoft-com:office:activation" xmlns:html="http://www.w3.org/TR/REC-html40" xmlns:q="http://schemas.xmlsoap.org/soap/envelope/" xmlns:rtc="http://microsoft.com/officenet/conferencing" xmlns:D="DAV:" xmlns:Repl="http://schemas.microsoft.com/repl/" xmlns:mt="http://schemas.microsoft.com/sharepoint/soap/meetings/" xmlns:x2="http://schemas.microsoft.com/office/excel/2003/xml" xmlns:ppda="http://www.passport.com/NameSpace.xsd" xmlns:ois="http://schemas.microsoft.com/sharepoint/soap/ois/" xmlns:dir="http://schemas.microsoft.com/sharepoint/soap/directory/" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:dsp="http://schemas.microsoft.com/sharepoint/dsp" xmlns:udc="http://schemas.microsoft.com/data/udc" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sub="http://schemas.microsoft.com/sharepoint/soap/2002/1/alerts/" xmlns:ec="http://www.w3.org/2001/04/xmlenc#" xmlns:sp="http://schemas.microsoft.com/sharepoint/" xmlns:sps="http://schemas.microsoft.com/sharepoint/soap/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:udcs="http://schemas.microsoft.com/data/udc/soap" xmlns:udcxf="http://schemas.microsoft.com/data/udc/xmlfile" xmlns:udcp2p="http://schemas.microsoft.com/data/udc/parttopart" xmlns:wf="http://schemas.microsoft.com/sharepoint/soap/workflow/" xmlns:dsss="http://schemas.microsoft.com/office/2006/digsig-setup" xmlns:dssi="http://schemas.microsoft.com/office/2006/digsig" xmlns:mdssi="http://schemas.openxmlformats.org/package/2006/digital-signature" xmlns:mver="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns:mrels="http://schemas.openxmlformats.org/package/2006/relationships" xmlns:spwp="http://microsoft.com/sharepoint/webpartpages" xmlns:ex12t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:ex12m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:pptsl="http://schemas.microsoft.com/sharepoint/soap/SlideLibrary/" xmlns:spsl="http://microsoft.com/webservices/SharePointPortalServer/PublishedLinksService" xmlns:Z="urn:schemas-microsoft-com:" xmlns:st="" xmlns="http://www.w3.org/TR/REC-html40" xmlns:ns0="urn:schemas-microsoft-com:office:smarttags"><head><meta http-equiv=Content-Type content="text/html; charset=iso-8859-1"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Tahoma","sans-serif";
color:blue;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 2.0cm 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=IT link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>*****************************<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Apologies for multiple postings<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>******************************<o:p></o:p></span></p><p class=MsoNormal align=center style='text-align:center'><span lang=EN-US style='font-size:16.0pt'>CLEF 2011 Labs - Call for Participation</span><span lang=EN-US style='font-size:12.0pt'> -<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal align=center style='text-align:center'><span lang=EN-US style='font-size:14.0pt'>http://clef2011.org/<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>The CLEF 2011 conference is the continuation of the popular CLEF campaigns and workshops that ran 2000-2009 (http://www.clef-campaign.org/). In 2010, CLEF was organized in a different way: the CLEF 2010 Conference (http://clef2010.org) presenting related research papers followed by a series of “Labs” covering a broad range of issues from fields of multilingual and multimodal information access evaluation. CLEF 2011 will be organized in a similar way. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Two different forms of Labs are offered:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- Labs that are a “campaign-style” evaluation for specific information access problems (during the twelve month period preceding the conference), similar in nature to the traditional CLEF campaign “tracks”<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- Labs that follow a more classical “workshop” pattern, exploring issues of evaluation methodology, metrics, processes etc. in information access and closely related fields.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Labs Description<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Below is a brief summary of the Labs proposed this year. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Cross-Language Image Retrieval (ImageCLEF)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>----------------------------------------- <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>This track evaluates retrieval from visual collections; both textual and visual retrieval techniques are exploitable. Four challenging tasks are foreseen: 1) retrieval from a collection of Wikipedia images with textual annotations and topics in several languages; 2) medical image retrieval with visual, semantic and mixed topics in several languages with a data collection from the scientific literature; 3) visual classification of leaf images for the identification of plant species; 4) a photo annotation task that investigates automated semantic annotation based on visual information with approaches based on Flickr user tags and multimodal approaches. In addition, a practical showcase is planned at the conference for the real-time evaluation of interactive image search systems. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Track coordinators are U. of Applied Sciences Western Switzerland (CH), Oregon Health and Science U. (US), CEA LIST (FR), U. of Geneva (CH), Fraunhofer Society (DE), University of North Texas (US), INRIA (FR). See also http://www.imageclef.org/.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Uncovering Plagiarism, Authorship, and Wikipedia Vandalism (PAN)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>----------------------------------------------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>PAN @ CLEF 2011 divides into three tasks:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- Plagiarism Detection. Today's plagiarism detectors face intricate situations, such as paraphrased plagiarism within and across languages. Moreover, the source of a plagiarism case may be hidden within a large collection of documents such as the Web, or it may not be available at all. Building on the successful evaluation framework developed in the last two years, we continue to add new challenges this year.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- Author Identification. Throughout history and especially today, many texts are written anonymously or under false names, so that readers may not be certain of a text's alleged author. Author identification is the task determining the true author of a text, and one of the main challenges is to automatically attribute a text to a set of known authors. For the purpose of the evaluation, we have developed a new authorship evaluation corpus. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- Wikipedia Vandalism Detection. Vandalism has always been one of Wikipedia's biggest problems. However, the detection of vandalism is done mostly manually by volunteers, and research on automatic vandalism detection is still in its infancy. Hence, solutions are to be developed which aid Wikipedians in their efforts.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>PAN is organized by the Bauhaus-Universität Weimar, the Universidad Politécnica de Valencia, the University of the Aegean, the Bar-Ilan University, the Illinois Institute of Technology, and the Duquesne University.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>LogCLEF<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>---------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>The research goal of LogCLEF is the analysis and classification of queries in order to understand search behaviour in multilingual contexts, and it consists of three tasks: 1) Language identification task; 2) Query classification, 3) Success of a query. In coordination with the organizers, participating groups will be devoted to different tasks in exploring and understanding the data. For some of these tasks, test sets of human labelled queries (a subset from the log files) should be assembled and released to the participants. In order to create an interesting test and training set, participants will be required to submit both manual and automatic annotations. At least two large log datasets will be made available: The European Library action logs; the Sogou chinese search engine logs.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Lab Coordinators: University of Padova, Dublin City University, University of Hildesheim.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>For further information: http://www.promise-noe.eu/<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>QA4MRE - Question Answering for Machine Reading Evaluation<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>-----------------------------------------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>QA4MRE is a new task of the Question Answering (QA) Track. The goal is to evaluate Machine Reading abilities through Question Answering and Reading Comprehension Tests. The focus is on capturing knowledge from given text collections and using it to answer questions. The task consists of the reading of single documents, where correct answers require some inference and previously-acquired background knowledge. Like in Reading Comprehension Tests, questions will be in the form of multiple choice questions. The participating systems will be required to answer the questions by choosing in each case one answer from the alternatives proposed.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Coordination: UNED (Spain), ISI (USA), CELCT (Italy), University of Limerick (Ireland). </span><span style='font-size:12.0pt'>For more information see: http://celct.fbk.eu/QA4MRE/<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>CLEF-IP: IR in the IP domain<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>----------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>In 2011, CLEF-IP puts to use a collection of more than 2 million patent documents in XML format with content in English, German, and French. The collection will also include patent images. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>During this benchmarking activity, several tasks will be organised: <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- The Prior Art Candidate Search task requires the participants to retrieve documents that are potential prior art to a given document. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- The Image-based Document Retrieval task is a pilot task where participants are asked to find patent documents or images relevant to a given patent document containing images.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- The Classification task will ask participants to classify documents according to the International Patent Classification scheme up to the subclass level. An optional Refined Classification task will be available, where participants are asked to classify patent documents up the group and subgroups levels when the subclass is given.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>- The Image-based Classification task asks participants to categorise given patent images into pre-defined categories of images (flow-charts, hand drawings, technical drawings, chemical structures, etc.)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Coordinators: Information Retrieval Facility (AT), max.recall (AT). Website: http://www.ir-facility.org/clef-ip<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>MusiClef<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>---------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Currently, music search is still mostly text-based. The goal of MUSICLEF is to promote the development of novel methodologies for music access and retrieval, which can combine content-based information, automatically extracted from music files, with contextual information, provided by users through tags, comments, reviews, possibly in different languages. The combination of these two sources of information is still under-investigated in music retrieval. To this end, MUSICLEF aims at developing and maintaining a test collection of both music files and relevant desciptors, to be distributed to participants in the form of feature vectors (e.g. MFCCs, Chroma-vectors, rhythmic features, prominent pitch) and links to public available music excerpts.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>CHiC 2011 – Cultural Heritage in CLEF: From Use Cases to Evaluation in Practice for Multilingual Information Access to Cultural Heritage<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>------------------------------------------------------------------------------------------------------------<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>The CHiC2011 workshop at CLEF 2011 aims at moving towards a systematic and large-scale evaluation of cultural heritage digital libraries and information access systems by surveying all the evaluation efforts in the cultural heritage field as well as defining user scenarios and identifying possible relevant metrics.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Workshop participants are asked to introduce ideas for evaluation scenarios derived from use cases, user studies and previous evaluation efforts and should focus on describing possible scenarios from the CH domain that are applicable for various systems and projects.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>Coordinators are: University of Padua, Humboldt-Universität zu Berlin and University of Sheffield.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>For more information see: http://svrims.dei.unipd.it:8082/web/guest/use-cases/unlocking-culture/chic-2011<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:12.0pt'>If you would like to participate in one or more of the above Labs, please register online at the CLEF website (available soon)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";color:blue'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";color:blue'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-family:"Tahoma","sans-serif";color:blue'><o:p> </o:p></span></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span lang=EN-US style='font-size:12.0pt;font-family:"Tahoma","sans-serif";color:navy'>================================</span><span lang=EN-US style='font-size:10.0pt;font-family:"Times New Roman","serif";color:blue'><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>Pamela Forner<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>CELCT (web: </span><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:blue'><a href="http://www.celct.it/" title="http://www.celct.it"><span lang=EN-GB style='color:blue'>www.celct.it</span></a></span><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>)<o:p></o:p></span></p><p class=MsoNormal><i><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>Center for the Evaluation of Language and Communication Technologies<o:p></o:p></span></i></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>Via alla Cascata 56/c </span><span style='font-size:12.0pt;font-family:"Times New Roman","serif";color:blue'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>38100 Povo – TRENTO –</span><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'><ns0:country-region><ns0:place><span lang=IT style='color:navy'>Italy</span></ns0:place></ns0:country-region></span><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>email: </span><span style='font-size:10.0pt;color:blue'><a href="mailto:forner@celct.it" title="mailto:fordyce@celct.it"><span lang=EN-GB style='font-family:"Tahoma","sans-serif";color:blue'>forner@celct.it</span></a></span><span lang=EN-GB style='font-size:10.0pt;color:blue'><br></span><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>tel.: +39 0461 314 804</span><span lang=EN-US style='font-size:12.0pt;font-family:"Times New Roman","serif";color:blue'><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>fax: +39 0461 314 846</span><span style='color:blue'><o:p></o:p></span></p><p class=MsoNormal><span style='color:blue'> <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-GB style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:navy'>Secretary Phone: +39 0461 314 870</span><span style='color:blue'><o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>