12.583, Calls: NL Generation, Language/Dialogue Systems
The LINGUIST Network
linguist at linguistlist.org
Fri Mar 2 20:14:43 UTC 2001
LINGUIST List: Vol-12-583. Fri Mar 2 2001. ISSN: 1068-4875.
Subject: 12.583, Calls: NL Generation, Language/Dialogue Systems
Moderators: Anthony Aristar, Wayne State U.<aristar at linguistlist.org>
Helen Dry, Eastern Michigan U. <hdry at linguistlist.org>
Andrew Carnie, U. of Arizona <carnie at linguistlist.org>
Reviews (reviews at linguistlist.org):
Simin Karimi, U. of Arizona
Terence Langendoen, U. of Arizona
Editors (linguist at linguistlist.org):
Karen Milligan, WSU Naomi Ogasawara, EMU
Lydia Grebenyova, EMU Jody Huellmantel, WSU
James Yuells, WSU Michael Appleby, EMU
Marie Klopfenstein, WSU Ljuba Veselinova, Stockholm U.
Software: John Remmers, E. Michigan U. <remmers at emunix.emich.edu>
Gayathri Sriram, E. Michigan U. <gayatri at linguistlist.org>
Home Page: http://linguistlist.org/
The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.
Editor for this issue: Jody Huellmantel <jody at linguistlist.org>
==========================================================================
As a matter of policy, LINGUIST discourages the use of abbreviations
or acronyms in conference announcements unless they are explained in
the text.
=================================Directory=================================
1)
Date: Fri, 2 Mar 2001 11:01:53 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: 8th European Workshop on NL Generation (ACL-2001)
2)
Date: Fri, 2 Mar 2001 10:57:00 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: Evaluation for Language & Dialogue Systems (ACL-2001)
-------------------------------- Message 1 -------------------------------
Date: Fri, 2 Mar 2001 11:01:53 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: 8th European Workshop on NL Generation (ACL-2001)
ACL/EACL 2001 Workshop
8th EUROPEAN WORKSHOP ON NATURAL LANGUAGE GENERATION
6-7 July 2001
Toulouse, France
http://www.cs.unca.edu/~bruce/acl01/NLG.html
- ----------------------------------------------------------------------
Natural language generation (NLG) constitutes the production of meaningful
texts in natural languages from some underlying non-linguistic
representation of information. Accomplishing this goal may be envisioned
for a number of different purposes, including standardized and/or
multi-lingual reports, summaries, machine translation, dialog applications,
and embedding in multi-media and hypertext environments. Consequently, the
automated production of language is associated with a large number of
highly diverse tasks whose appropriate orchestration in high quality poses
a variety of theoretical and practical problems. Relevant issues include
content selection, text organization, the production of referring
expressions, aggregation, lexicalization, and surface realization, as well
as coordination with other media.
This workshop is part of a bi-annual series of workshops about natural
language generation that runs since 1987. Previous European workshops have
been held at Royaumont, Edinburgh, Judenstein, Pisa, Leiden, Duisburg, and
Toulouse. The goal of the workshop is to be an informal meeting which
facilitates the dissemination of knowledge and expertise in the field. The
workshop will focus on the following topics:
* Search methods for NLG (in content planning and realization)
There seems to be a substantial discrepancy between
application-oriented systems and principled approaches to NLG.
Accomodating a standard pipeline architecture with suitable heuristic
preferences to the intended functionality of a system stands in
contrast to several principled approaches to searching which have been
tried out so far. These include blackboard architectures, constraint
propagation and, more recently genetic algorithms and statistical
techniques. A comparison of these methods in terms of their potential
and limitations is likely to improve understanding about this issue.
Gained insights could prove fruitful for building applications in a
more general and, thus, better reusable way, especially in large-scale
applications such as summarization and machine translation.
* Differences in information organization between source and
presentation specifications (and methods to bridge between these)
Whether the generation task is to verbally express contents of some
knowledge base or to produce multi-lingual presentations from
language-neutral or similar representations, there are strong
similarities in building the target representations: In the
overwhelming number of cases, the ordering and embedding of elements
in the source representation is reflected by the ordering and
embedding of their corresponding realizations at the surface. Often,
this reflection is systematic, many times even simple. But a few cases
prove complex and involve a major restructuring of the surface
structure when compared to the source structure. A major emphasis of
this topic is on collecting such complex cases, identifying
commonalities between them and discussing restructuring techniques.
Accepted papers on these topics will be scheduled for presentation. The
majority of the time will be devoted to discussions, either in sequence or
in parallel, depending on the number of participants. We are considering
organizing a panel. For the focus topics above, we will contact a number of
competent researchers to address the topic from a specific perspective
according to their experience. In addition, we will ask some of them to
prepare material / concrete examples for discussions.
WORKSHOP CHAIRS
Helmut Horacek Univ. of the Saarland
Nicolas Nicolov IBM T.J. Watson Research Center
Leo Wanner Univ. of Stuttgart
PROGRAMME COMMITTEE
John Bateman Univ. of Bremen
Dan Cristea Univ. of Iasi
Robert Dale Macquarie University
Laurence Danlos Universite Paris 7
Marc Dymetman Xerox Research Centre Europe, Grenoble
Michael Elhadad Ben-Gurion Univ.
Kristiina Jokinen Univ. of Art and Design Helsinki
Richard Kittredge Univ. of Montreal & CoGenTex
Daniel Marcu ISI, Univ. of Southern California
Chris Mellish Univ. of Edinburgh
Sergei Nirenburg CRL, New Mexico
Owen Rambow AT&T Research
Ehud Reiter Univ. of Aberdeen
Manfred Stede Technical University of Berlin
Michael Zock LIMSI, CNRS
SUBMISSIONS
Papers describing original work in the area of NLG in particular related to
the workshop focus topics above should be submitted electronically. Papers
should be 6-8 pages long in PDF format. We recommend a A4, two-column
format like the ACL proceedings: http://acl2001.dfki.de/style/
We also invite poster submissions (free format, up to 6 page, PDF).
The submissions should be associated with a cover email containing the
following information (ASCII text):
# TITLE: <title of the paper>
# AUTHORS: <list of authors>
# EMAIL: <email of author(s) for correspondence>
# KEYWORDS: <keywords, topic sub-areas, ...>
# ABSTRACT: <abstract of the paper>
Send your submission to Leo Wanner <wannerlo at informatik.uni-stuttgart.de>.
IMPORTANT DATES
Paper submissions *** 6 April 2001 ***
Notification of acceptance 27 April 2001
Camera-ready copies due 16 May 2001
Registration deadline as ACL
Workshop dates 6-7 July 2001
REGISTRATION
The registration fee for the workshop will be posted at a later stage. The
registration fee includes attendance of the workshop and a copy of workshop
proceedings. Follow the registration instructions at the ACL site and
indicate that you would like to attend the NLG workshop.
People wishing to attend the workshop but not submitting papers should send
a notification of attendance: a 1-2 page stating interest to participate,
work done in NLG so far, and potential contributions / material for
discussions about one of the topics. This informationn will help with the
organisation of discussions and allow for an informal and highly
interactive character of the workshop. Notifications of attendance should
be sent to Helmut Horacek <horacek at cs.uni-sb.de>.
MORE INFORMATION
Check the following web site for updates about the NLG workshop:
http://www.cs.unca.edu/~bruce/acl01/NLG.html
-------------------------------- Message 2 -------------------------------
Date: Fri, 2 Mar 2001 10:57:00 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: Evaluation for Language & Dialogue Systems (ACL-2001)
Call for Papers
Workshop on Evaluation for Language and Dialogue Systems
ACL/EACL 2001
Toulouse, France
July 6-7, 2001
WORKSHOP GOALS
The aim of this two day workshop is to identify and to synthesize current
needs for language-technology evaluation.
The first day of the workshop will focus on one of the most challenging
current issues in language engineering: the evaluation of dialogue systems
and models. The second day will extend the discussion to address the problem
of evaluation in language engineering more broadly and on more theoretical
grounds.
The space of possible dialogues is enormous, even for limited domains like
travel information servers. The generalization of evaluation methodologies
across different application domains and languages is an open problem.
Review of published evaluations of dialogue models and systems suggests that
usability techniques are the standard method. Dialogue-based system are
often evaluated in terms of standard, objective usability metrics, such as
task-completion time and number of user actions. In the past, researchers
have proposed and debated theory-based methods for modifying and testing the
underlying dialogue model, but the most widely used method of evaluation is
usability testing, although more precise and empirical methods for
evaluating the effectiveness of dialogue models have been proposed. For
task-based interaction, typical measures of effectiveness are
time-to-completion and task outcome, but the evaluation should focus on user
satisfaction rather than on arbitrary effectiveness measurements.Indeed, the
problems faced in current approaches to measurement of effectiveness
dialogue models and systems include:
Direct measures are unhelpful because efficient performance on the nominal
task may not represent the most effective interaction
Indirect measures usually rely on judgment and are vulnerable to weak
relationships between the inputs and outputs
Subjective measures are unreliable and domain-specific
For its first day, the workshop organizers solicit papers on these issues,
with particular emphasis on methods that go beyond usability testing to
address the underlying dialogue model. Representative questions to be
addressed include:
o How do we deal with the combinatorial explosion
of dialogue states?
o How can satisfaction be measured with respect to
underlying dialogue models?
o Are there useful direct measures of dialogue properties
that do not depend on task efficiency?
o What is the role of agent-based simulation in
evaluation of dialogue models?
Of course, the problems faced in evaluating dialogue and system models are
found in other domains of language engineering, even for non-interactive
processes such as part-of-speech tagging, parsing, semantic disambiguation,
information extration, speech transcription, and audio document indexing. So
the issue of evaluation can be viewed at a more generic level, raising
fundamental, theoretical questions such as:
o What are the interest and benefits of evaluation
for language engineering?
o Do we really need these specific methodologies,
since a form of evaluation sould always be present
in any scientific investigation?
o If evaluation is needed in language engineering, is
it the case for all domains?
o What form should it take? Technology evaluation
(task-oriented in laboratory environment) or
field/user Evaluation (complete systems in real-life
conditions)?
We have seen before that the the evaluation of dialogue models is still
unsolved, but for domains where metrics already exists, are they
satisfactory and sufficient? How can we take into account or abstract from
the subjective factor introduced by human operators in the process?
Do similarity measures and standards offer appropriate answers to this
problem? Most of the efforts focus on evaluating process, but what about the
issue of language resources evaluation?
For its second day of work, the workshop organizers solicit papers on these
issues, with the intent to address the problem of evaluation both from a
broader perspective (including novel applications domains for evaluation,
new metrics for known tasks and resource evaluation) and a more theoretical
point of view (including formal theory of evaluation and infrastructural
needs of language engineering).
NOTE: People who would like to submit a paper on lexical semantic
disambiguation evaluation should consider the parallel workshop, on July
5-6, for the closure of the SENSEVAL-2 evaluation campaign.
- -----------------------------------------------------------
WORKSHOP ORGANIZATION
The organization of each of the two days of the workshop will reflect the
workshop's two main themes. Each day will begin with a session of
presentations of selected papers and follow with panel discussions to
synthesize and develop possible methodologies from additional selected
workshop papers.
WORKSHOP PARTICIPATION
The workshop seeks participation from people involved or interested in the
problem of evaluation in language processing and the research and industrial
communities that study and implement dialogue models for natural-language
interaction systems.
The first part of the workshop will specifically draw on the
natural-language interaction community, for instance like the one developing
at the confluence of SIGdial and SIGCHI, which will find in this workshop an
atmosphere more flavored by computational-linguistics related issues (see,
for example, the First SIGdialWorkshop on Discourse and Dialogue).
The second part of the workshop is intended to provide a forum for a broader
audience more in the spirit of the one that attended the LREC'2000 Satellite
Workshop on Evaluation (see http://www.limsi.fr/TLP/CLASS), in particular
offering an opportunity to people involved in language engineering
evaluation (e.g ., the CLASS audience) in the context of national or
transnational projects or programs, both in Europe and abroad.
- -----------------------------------------------------------
SUBMISSION DETAILS
Paper submissions should follow the two-column format of ACL proceedings and
should not exceed eight (8) pages, including references. We strongly
recommend the use of ACL LaTeX style files or Microsoft Word Style files
tailored for this year's conference. They are available from the ACL-2001
program committee Web site at http://acl2001.dfki.de/style/.
Papers should be submitted electronically, as either a LaTeX, Word or PDF
file to either:
Patrick Paroubek, pap at limsi.fr
Karen Ward, kward at cs.utep.edu
- -----------------------------------------------------------
TIMETABLE OF IMPORTANT DATES
Deadline for workshop paper submissions: April 6, 2001
Deadline for notification of workshop paper acceptance: April 27, 2001
Deadline for camera-ready workshop papers: May 16, 2001
Workshop date: July 6-7, 2001
- -----------------------------------------------------------
WORKSHOP ORGANIZING COMMITTEE
David G. Novick, UTEP
novick at cs.utep.edu
http://www.cs.utep.edu/novick
Joseph Mariani, Limsi - CNRS
mariani at limsi.fr
http://www.limsi.fr/Individu/mariani
Candy Kamm, AT&T Labs
cak at research.att.com
http://www.research.att.com/info/cak
Patrick Paroubek, Limsi - CNRS
pap at limsi.fr
http://www.limsi.fr/Individu/pap
Nils Dahlbdck, Linkvping University
nilda at ida.liu.se
http://www.ida.liu.se/~nilda/
Frankie James, NASA Ames Research Center
fjames at riacs.edu
http://www-pcd.stanford.edu/frankie/
Karen Ward, UTEP, kward at cs.utep.edu
http://www.cs.utep.edu/kward
- -----------------------------------------------------------
SCIENTIFIC COMMITTEE
David G. Novick
Joseph Mariani
Candy Kamm
Patrick Paroubek
Nils Dahlbdck
Frankie James
Karen Ward
Christian Jacquemin
Niels Ole Bernsen
Stephane Chaudiron
Khalid Choukri
Martin Rajman
Robert Gaizauskas
Donna Harman
Lynette Hirschman (tentative)
David Pallett (tentative)
Carol Peters (tentative)
Jose Pardo (tentative)
Herman Steeneken (tentative)
Oliviero Stock (tentative)
Saod Tazi
Hans Uszkoreit (tentative)
- -----------------------------------------------------------
SPONSORS
ACL 2001
CLASS
ELRA
ELSNET
We also anticipate co-sponsorship from SIGdial.
- -----------------------------------------------------------
ADDITIONAL INFORMATION
Additional information on the workshop, including accepted papers and the
workshop schedule, will be made available as needed at
http://www.limsi.fr/TLP/CLASS/eacl01.html
---------------------------------------------------------------------------
LINGUIST List: Vol-12-583
More information about the LINGUIST
mailing list