12.3161, Calls: Parsing Evaluation,Applied Linguistics
LINGUIST List
linguist at linguistlist.org
Fri Dec 21 23:21:41 UTC 2001
LINGUIST List: Vol-12-3161. Fri Dec 21 2001. ISSN: 1068-4875.
Subject: 12.3161, Calls: Parsing Evaluation,Applied Linguistics
Moderators: Anthony Aristar, Wayne State U.<aristar at linguistlist.org>
Helen Dry, Eastern Michigan U. <hdry at linguistlist.org>
Andrew Carnie, U. of Arizona <carnie at linguistlist.org>
Reviews (reviews at linguistlist.org):
Simin Karimi, U. of Arizona
Terence Langendoen, U. of Arizona
Editors (linguist at linguistlist.org):
Karen Milligan, WSU Naomi Ogasawara, EMU
Jody Huellmantel, WSU James Yuells, WSU
Michael Appleby, EMU Marie Klopfenstein, WSU
Ljuba Veselinova, Stockholm U. Heather Taylor-Loring, EMU
Dina Kapetangianni, EMU Richard Harvey, EMU
Karolina Owczarzak, EMU Renee Galvis, WSU
Software: John Remmers, E. Michigan U. <remmers at emunix.emich.edu>
Gayathri Sriram, E. Michigan U. <gayatri at linguistlist.org>
Home Page: http://linguistlist.org/
The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.
Editor for this issue: Dina Kapetangianni <dina at linguistlist.org>
==========================================================================
As a matter of policy, LINGUIST discourages the use of abbreviations
or acronyms in conference announcements unless they are explained in
the text.
=================================Directory=================================
1)
Date: Fri, 21 Dec 2001 10:55:08 +0000
From: John Carroll <johnca at cogs.susx.ac.uk>
Subject: LREC 2002 Workshop on Parsing Evaluation - Announcement and CFP
2)
Date: Fri, 21 Dec 2001 13:53:45 +0100
From: Elsa Tragant <tragant at fil.ub.es>
Subject: call for abstracts
-------------------------------- Message 1 -------------------------------
Date: Fri, 21 Dec 2001 10:55:08 +0000
From: John Carroll <johnca at cogs.susx.ac.uk>
Subject: LREC 2002 Workshop on Parsing Evaluation - Announcement and CFP
Call for Papers
Beyond PARSEVAL
-- Towards Improved Evaluation Measures for Parsing Systems --
http://let.dfki.uni-sb.de/BeyondPARSEVAL/
LREC 2002 Workshop
2nd June
Las Palmas, Canary Islands, Spain
OVERVIEW
The PARSEVAL metrics for evaluating the accuracy of parsing systems have
underpinned recent advances in stochastic parsing with grammars learned
from treebanks (most prominently the Penn Treebank of English). However,
a new generation of parsing systems is emerging based on different
underlying frameworks and covering other languages. PARSEVAL is not
appropriate for many of these approaches: the NLP community therefore
needs to come together and agree on a new set of parser evaluation
standards.
BACKGROUND AND MOTIVATION
In line with increasing interest in fine-grained syntactic and semantic
representations, stochastic parsing is currently being applied to
several high level syntactic frameworks, such as unification-based
grammars, tree-adjoining grammars and combinatory categorial grammars. A
variety of different types of training data are being used, including
dependency annotations, phrase structure trees, and unlabelled text.
Other researchers are building parsing systems using shallower
frameworks, based for example on finite-state transducers. Many of these
novel parsing approaches are using alternative evaluation measures --
based on dependencies, valencies, or exact or selective category match
- since the PARSEVAL measures (of bracketing match with respect to
atomic-labelled phrase structure trees) cannot be applied, or are
uninformative.
The field is therefore confronted with a lack of common evaluation
metrics, and also of appropriate gold standard evaluation corpora in
languages other than English. We need a new and uniform scheme for
parser evaluation that covers both shallow and deep grammars, and allows
for comparison and benchmarking across different syntactic frameworks
and different language types.
A previous LREC-hosted workshop on parser evaluation in 1998 (see
http://ceres.ugr.es/~rubio/elra/parsing.html brought together a
number of researchers advocating parser evaluation based on dependencies
or grammatical relations as a viable alternative to the PARSEVAL
measures.
The aim of this workshop is to start an initiative by bringing together
four relevant parties:
- researchers in symbolic and stochastic parsing
- builders of annotated corpora
- representatives from different syntactic frameworks
- groups with interests in and proposals for parser evaluation
The workshop will provide a forum for discussion with the aim of
defining a new parser evaluation metric; we also intend the workshop to
kick off a sustained collaborative effort into building or deriving
sufficiently large evaluation corpora, and possibly training corpora
appropriate to the new metric. To maintain the momentum of this
initiative we will work towards setting up a parsing competition based
on new standard evaluation corpora and evaluation metric.
TOPICS OF INTEREST
The workshop organisers invite papers focussing on:
- benchmarking the accuracy of individual parsing systems
- parser evaluation
- design of annotation schemes covering different languages and
grammar frameworks
- creation of high-quality evaluation corpora
Papers on the following topics will be particularly welcome:
- descriptions of experiments using alternative evaluation measures
with existing (stochastic or symbolic) parsers, focussing on
comparison and discussion of qualitative differences
- methods for creation of evaluation (or training) corpora, allowing
flexible adaptation to a new evaluation standard based on
dependencies or grammatical relations
- comparisons of existing or possible new schemes for dependency-based
evaluation (differences, similarities, problems)
MODE OF ORGANISATION
The one-day workshop will consist of (30-minute) paper presentations, a
panel session, and an extended open session at which important results
of the workshop will be summarised and discussed.
As a follow-up, we hope to arrange a half-day meeting outside the
workshop format to discuss concrete action plans, create working groups,
and plan future collaboration.
WORKSHOP ORGANISERS
John Carroll University of Sussex, UK
Anette Frank DFKI GmbH, Saarbruecken, Germany
Dekang Lin University of Alberta, Canada
Detlef Prescher DFKI GmbH, Saarbruecken, Germany
Hans Uszkoreit DFKI GmbH and Saarland University, Saarbruecken, Germany
PROGRAM COMMITTEE
Thorsten Brants Xerox PARC
Gosse Bouma Rijksuniversiteit Groningen
Ted Briscoe University of Cambridge
John Carroll University of Sussex
Jean-Pierre Chanod XRCE Grenoble
Michael Collins AT&T Labs-Research
Anette Frank DFKI Saarbruecken
Gregory Grefenstette Clairvoyance, Pittsburgh
Julia Hockenmaier University of Edinburgh
Dekang Lin University of Alberta
Detlef Prescher DFKI Saarbruecken
Khalil Sima'an University of Amsterdam
Hans Uszkoreit DFKI Saarbruecken and Saarland University
SUBMISSIONS
Abstracts for workshop contributions should not exceed two A4 pages
(excluding references). An additional title page should state: the
title; author(s); affiliation(s); and contact author's e-mail address,
as well as postal address, telephone and fax numbers.
Submission is by email, preferably in Postscript or PDF format, to:
John.Carroll at cogs.susx.ac.uk
to arrive by 1st February 2002. Abstracts will be reviewed by at least 3
members of the program committee.
Formatting instructions for the final full version of papers will be
sent to authors after notification of acceptance.
IMPORTANT DATES
1 February 2002 deadline for receipt of abstracts
22 February 2002 notification of acceptance
12 April 2002 camera-ready final version for workshop proceedings
2 June 2002 workshop
TIME AND LOCATION OF THE WORKSHOP
The workshop will take place on 2nd June, following the main LREC 2002
Conference, in the Palacio de Congreso de Canarias, Las Palmas, Canary
Islands.
WORKSHOP REGISTRATION
The registration fee for the workshop is:
If you are also attending LREC: 90 EURO
If you are not attending LREC: 140 EURO
All attendees will receive a copy of the workshop proceedings.
-------------------------------- Message 2 -------------------------------
Date: Fri, 21 Dec 2001 13:53:45 +0100
From: Elsa Tragant <tragant at fil.ub.es>
Subject: call for abstracts
Barcelona English Language and Literature Studies: NEW FORMAT
Published by the English and German Department at the University of Barcelona,
the next issue of this journal (vol. 12) will be published on-line
and will be distributed by subscription.
Bells vol. 12: Call for Abstracts
THE TEACHING OF FOREIGN LANGUAGES
IN HIGHER AND ADULT EDUCATION
An efficient program of foreign language instruction should obviously
go on beyond secondary education, both regarding the improvement of a
first foreign language as well as the introduction to a second or a
third language. To this respect, universities and language schools can
offer an appropriate context especially when the learning is powered
by: - self-directed learning - the development of autonomy - the use
of new technologies.
This topic issue of BARCELONA ENGLISH LANGUAGE AND LITERATURE STUDIES,
which will be published in 2002, is open to all those contributions
which deal with a topic related to the learning/teaching in higher and
adult education: - methodology and teaching innovations - evaluation
and planning - language acquisition - program administration, etc., as
well as to those contributions which deal with more specific issues of
self-learning, autonomy or new technologies applied to the teaching of
any foreign language.
The call is open to a number of contexts and several kinds of
contributions, which are listed below: - schools or faculties with a
language program for specific purposes (i.e., Engineering, Journalism,
Humanities, etc.) - university language schools - schools or
faculties where language specialists are trained (Linguistics,
Translation, Teacher training...) - Multimedia rooms, virtual
self-learning services, self-access centres - language schools
involved in adult aducation. Types of contributions: - formal studies
- case studies and action research - descriptions of innovative
experiences and results of surveys - conceptual and opinion articles -
interviews and book reviews.
We would like people interested in contributing to this volume to send
a one-page abstract of their work (in English, Spanish or Catalan),
together with a mailing and an e-mail address to ribe at lingua.fil.ub.es
or to the address below no later than:
February 1st, 2002 Ramón Ribé (editor) BELLS, Departament de Filologia
Anglesa i Alemanya Universitat de Barcelona Gran Via, 585 08007
Barcelona Once the abstract is received and within one month, authors
will be contacted to let them know about the evaluation of their
proposal. They will also receive the conventions to follow to write
their full-length contribution, which should be sent before the 15th
of May, 2002.
---------------------------------------------------------------------------
LINGUIST List: Vol-12-3161
More information about the LINGUIST
mailing list