27.647, FYI: EVALITA 2016: Call for Tasks

[The LINGUIST List via LINGUIST]

LINGUIST List: Vol-27-647. Wed Feb 03 2016. ISSN: 1069 - 4875.

Subject: 27.647, FYI: EVALITA 2016: Call for Tasks

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Anthony Aristar, Helen Aristar-Dry, Sara Couture)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
                   25 years of LINGUIST List!
Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Ashley Parker <ashley at linguistlist.org>
================================================================


Date: Wed, 03 Feb 2016 10:23:49
From: Malvina Nissim [m.nissim at rug.nl]
Subject: EVALITA 2016: Call for Tasks

 
EVALITA 2016: CALL for TASKS
The Submission deadline is Sunday, 21 February 2016

The 5th evaluation campaign EVALITA 2016 will be held in Naples, as a
satellite event of the 3rd Italian Conference on Computational Linguistics -
CLiC-it 2016 (December 5-7, 2016).
EVALITA 2016 is an initiative of AILC (Associazione Italiana di Linguistica
Computazionale, http://www.ai-lc.it/).

As in the previous editions (http://www.evalita.it/), EVALITA 2016 will be
organized along few selected tasks, which provide participants with
opportunities to discuss and explore both emerging and traditional areas of
Natural Language Processing and Speech for Italian. The participation is
encouraged for teams working both in academic institutions and industrial
organizations.

TASK PROPOSAL SUBMISSION:

Tasks proposals should be no longer than 4 pages and should include:

- task title and acronym
- names and affiliation of the organizers (minimum 2 organizers)
- brief task description, including motivations and state of the art showing
the international relevance of the task
- description and examples of the data needed for the task, including
information about their availability and development stage
- expected number of participants and attendees
- names and contact information of the organizers

In submitting your proposal, please bear in mind that we encourage:

- the re-annotation of datasets from previous years with new annotation types
or texts from publicly available corpora
- application-oriented tasks, that is tasks that have a clearly defined
end-user application showcasing
- both new tasks and re-runs. For new tasks, you will have to specify in the
proposal why this would attract a reasonable number of participants, and why
is needed. For a re-run, you should defend in your proposal the need for
having such task again (new metrics, new types of data, new domains, growing
interest, etc.)

The organizers of the accepted tasks should take care of planning, according
to the scheduled deadlines (see below):

- the development and distribution of data sets needed for the contest, i.e.
data for training and development, and data for testing; the scorer to be used
to evaluate the submitted systems should be included in the release of
development data
- the development of task guidelines, where all the instructions for the
participation are made clear together with a detailed description of data and
evaluation metrics applied for the evaluation of the participant results; the
collection of participants results
- the evaluation of participants results according to standard metrics
- the solicitation of submissions 
- the reviewing process of the papers describing the participants approach and
results (according to the template to be made available by the EVALITA-2016
chairs)
- the production of a paper describing the task (according to the template to
be made available by the EVALITA-2016 chairs)

Email your proposal in PDF format to evalita2016 at gmail.com with ''Evalita 2016
TASK Proposal'' as the subject line by the submission deadline (21st February
2016). 
Please feel free to contact the EVALITA-2016 chairs at evalita2016 at gmail.com
in case of any questions or suggestions.

IMPORTANT DATES:

- 21 February 2016: submission of task proposals
- 7 March 2016: notification of task proposal acceptance; call for Interest
opens
- 14 March 2016: on-line registration opens
- 30 May 2016: development data available to participants
- 12 September 2016: test data available, registration closes
- 19 September 2016: system results due to organizers
- 26 September 2016: assessment returned to participants
- 31 October 2016: technical reports due to organizers
- 5-7 December 2016: final workshop (one day during CLIC-2016)

EVALITA-2016 CHAIRS
Pierpaolo Basile (Università degli Studi di Bari Aldo Moro)
Franco Cutugno (Università degli Studi di Napoli Federico II)
Malvina Nissim (Center for Language and Cognition Groningen, Rijksuniversiteit
Groningen)
Viviana Patti (University of Torino)
Rachele Sprugnoli (Digital Humanities Group, Fondazione Bruno Kessler)

CONTACT:
evalita2016 at gmail.com
 



Linguistic Field(s): Computational Linguistics





 



------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-27-647	
----------------------------------------------------------