[Corpora-List] GREC'10 Call for Participation (Shared Task for Summarisation and NLG researchers)

[Anja Belz]

FIRST CALL FOR PARTICIPATION

GENERATION OF REFERENCES IN CONTEXT (GREC) TASKS 2010
-----------------------------------------------------

Part of Generation Challenges 2010, in conjunction with INLG 2010.

Generation Challenges is a shared-task evaluation initiative for
research that involves the generation of language, in the widest
sense.  Generation Challenges 2010 is the fourth event in this
intiative, and aims to continue to provide a common forum for a number
of different Shared Tasks (see
http://www.nltg.brighton.ac.uk/research/genchal10/).

As part of Generation Challenges 2010, we are organising three GREC
Shared Task Competitions.  The first is the GREC-NEG (Named Entity
Generation) Task which uses the GREC-People Corpus of 1,000 Wikipedia
introduction sections about people in which single and plural
references to all people mentioned in the text have been annotated;
the task in GREC-NEG is to select appropriate referential expressions
for all mentions (singular and plural) of people from a given list.
The second task, GREC-NER, is a straightforward Named Entity
Recognition task, using the new GREC-People-II extension to the
GREC-People Corpus. The third task is GREC-FULL which requires
participating systems to combine NER and NEG in order to improve
references to people in texts, and this also uses the GREC-People-II
Corpus.

Submissions to all three tasks will be evaluated using a range of
intrinsic and extrinsic measures, some assessed automatically, some
manually.  Submitted systems and evaluation results will be presented
in a special session at INLG'10 in Dublin in July 2010, and published
in the INLG'10 proceedings.


1. Background
--------------

There has been increasing interest recently among text summarisation
researchers in postprocessing techniques to improve the referential
clarity and coherence of extractive summaries, and among language
generation researchers in generating referential expressions in
context.  The GREC tasks are aimed at researchers in both of these
groups, and the objective is the development of methods for generating
chains of referential expressions for discourse entities in the
context of a written discourse, as is useful e.g. for postprocessing
extractive summaries and repeatedly edited texts (such as Wikipedia
articles).


2. Data
--------

The GREC data resources consist of introduction sections collected
from Wikipedia articles in which several broad syntactic categories of
reference to named entities have been annotated. The annotations
include features encoding syntactic and semantic information.

The GREC-People and GREC-People-II corpora consist of 1,000 texts each
in one domain, Wikipedia articles about people. Here, all references,
singular and plural, to all people mentioned in a text have been
annotated.  Each text in these corpora therefore includes explicit
coreference annotation for at least one coreference chain.

For GREC-People and GREC-People-II we have test sets of 100 texts each,
where referential expressions have been selected by participants in an
elicitation experiment. In these test sets, there are three versions
for each corpus text, in each of which the referential expressions
have been manually selected by a single participant in the experiment.

Full details of the evaluation methods for GREC'10 will be given in
the Participants' Pack that will be distributed to registered
participants.


6. Participation
----------------

Registration is now open at the GREC'10 homepage
(http://www.nltg.brighton.ac.uk/research/genchal10/grec).  Once
registered, participants in the GREC-NEG Task will receive the
complete training and development set, evaluation software and
detailed documentation (collectively known as the Participants' Pack)
for this task.  Participants in the other two tasks will first receive
a sample of the training and development data, to enable them to start
building systems; they will receive the complete Participants' Pack
for GREC-NER/FULL by the end of 2009.


7. Proceedings and Presentations
--------------------------------

The Generation Challenges 2010 meeting will be held as a special
session at INLG 2010 in July 2010 in Dublin. The session will include
overviews of all the shared tasks, including the GREC'10 Tasks. The
participating systems will additionally be presented as papers in the
INLG'10 proceedings, and as posters during the INLG'10 poster session.

GREC'10 papers will not undergo a selection procedure with multiple
reviews, but the organisers reserve the right to reject material which
is not appropriate given the participation guidelines.


8. Important Dates
------------------

Oct 2009          First Call for Participation in GREC'10 Tasks;
                  GREC-NEG Participants' Pack available;
                  GREC-NER/FULL Data Sample available
Dec 31, 2009      GREC-NER/FULL Participants Pack available
Apr 5-May 2, 2010 Test data submission period; 4-step submission process:
                  1. Fill in submission form (available on website from
                     Apr 5).
                  2. Upload 2-page paper describing approach and reporting
                     development set results.
                  3. Download test data (inputs only).
                  4. Submit test data outputs at the latest 48 hours after
                     download, but in any case no later than May 2.
May 2, 2010       Final deadline for submission of test data outputs.
May 2-23, 2010    GREC'10 Evaluation period
Jul 7-9, 2010     Generation Challenges meeting at INLG'10


9. Organisation
---------------

Anja Belz, NLTG, University of Brighton, UK
Eric Kow, NLTG, University of Brighton, UK
Jette Viethen, Macquarie University, Australia

GREC'10 homepage: http://www.nltg.brighton.ac.uk/research/genchal10/grec
Generation Challenges homepage:
http://www.nltg.brighton.ac.uk/research/genchal10
Generation Challenges email: nlg-stec at itri.brighton.ac.uk


_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora