[Corpora-List] INLG 2006 Special Session on Sharing Data and Comparative Evaluation, at Coling-ACL 2006 -- Second Call for Contributions

[Anja Belz]

INLG 2006 Special Session on Sharing Data and Comparative Evaluation
--------------------------------------------------------------------

Second Call for Contributions
-----------------------------

WHAT THIS IS ABOUT

Unlike many other areas within natural language processing, Natural
Language Generation (NLG) does not have a tradition of sharing data
resources, and no tradition of comparative evaluation of alternative
techniques.  We believe the introduction of a shared task for NLG
would be of benefit to the community, both internally and in terms of
making the field more visible to those outside the community.

We would like to move discussion on this topic forward by means of a
special session at INLG'06.  The focus of the session will be to move
us closer to an answer to the question: what would a shared task for
NLG look like?


BACKGROUND

Over the past year, in particular at ENLG'05 and UCNLG'05, NLG
researchers have started discussing ways in which the field might move
towards a situation where sharing data and other resources becomes
possible and comparative evaluation results can be produced.  It is
clear that the wide range of tasks and issues of interest to
researchers in NLG makes it difficult to identify a single set of data
that can be used by many, or a single task that could serve as the
basis for comparative evaluation of techniques.  For this reason we
believe it is important to see if we can establish a consensus view
amongst NLG researchers as to the content and focus of a shared task.

We invite submissions of two kinds of contributions to the INLG'06
Special Session on Sharing Data and Comparative Evaluation:

(1) Short papers for inclusion in the main INLG'06 proceedings:

    (a) Position papers that explore or make proposals regarding
        shared data and shared tasks, covering topics including (but
        not necessarily limited to) the following:

	* standardisation of NLG tasks
	* what kind of data people are interested in sharing
	* mechanisms for sharing
	* evaluation metrics
	* human evaluation methods
	* evaluation campaigns
	* lessons from evaluation campaigns in MT and document
	* summarisation

    (b) Reports on the results of experimenting with sharable data
        sets. Contributors may use any data set at all, but for anyone
	looking for a data set we recommend the SumTime Meteo data set
	(created by the SumTime project group at Aberdeen), available
        from the SIGGEN webpage.

(2) Abstracts for oral presentations during an Open Mic session on the
    topics of the special session.  We hope to involve a good number
    of NLG researchers, and particularly encourage contributions likely
    to stimulate discussion.


SUBMISSION

Paper submissions should follow the two-column format of ACL
proceedings (http://www.acl2006.org/program/style/), should be
in PDF format, and should not exceed three (3) pages, including
references.  As reviewing will be blind, please do not include the
authors' names and affiliations, and avoid revealing their identity in
the text of the paper.  Please follow the submission instructions for
the main INLG'06 session (details will be available on the Coling/ACL
web site: http://www.acl2006.org), but note that you should submit
your paper to the SPECIAL SESSION TRACK (not the short paper track or
any of the others).  The deadline for submissions is Friday April 28th
(9 days later than the deadline for the main INLG'06 session).

Abstracts for the Open Mic part of the special session should not
exceed 300 words in length and preferably be in plain text
format. Please send your abstract to Anja Belz (asb at brighton.ac.uk)
and Robert Dale (rdale at ics.mq.edu.au) by no later than Friday May
27th.


TIMETABLE

Submission of papers:                            April 28th, 2006
Notification of acceptance for papers:           May, 22nd, 2006
Submission of abstracts for Open-Mic session:    May, 27th, 2006
Submission of camera-ready copy of papers:       June 6th, 2006
Notification of acceptance for Open-Mic session: June, 16th, 2006
INLG 2006:                                       July 15-16th, 2006


Questions regarding the special session, its topics and submission
procedure, should be directed to the organisers, Anja Belz
(asb at brighton.ac.uk) and Robert Dale (rdale at ics.mq.edu.au).