12.108, Calls: Machine Translation, Network Problems
The LINGUIST Network
linguist at linguistlist.org
Thu Jan 18 21:09:31 UTC 2001
LINGUIST List: Vol-12-108. Thu Jan 18 2001. ISSN: 1068-4875.
Subject: 12.108, Calls: Machine Translation, Network Problems
Moderators: Anthony Aristar, Wayne State U.<aristar at linguistlist.org>
Helen Dry, Eastern Michigan U. <hdry at linguistlist.org>
Andrew Carnie, U. of Arizona <carnie at linguistlist.org>
Reviews: Terence Langendoen: U. of Arizona <terry at linguistlist.org>
Simin Karimi: U. of Arizona <simin at linguistlist.org>
Editors: Karen Milligan, Wayne State U. <karen at linguistlist.org>
Michael Appleby, E. Michigan U. <michael at linguistlist.org>
Lydia Grebenyova, E. Michigan U. <lydia at linguistlist.org>
Jody Huellmantel, Wayne State U. <jody at linguistlist.org>
Marie Klopfenstein, Wayne State U. <marie at linguistlist.org>
Naomi Ogasawara, E. Michigan U. <naomi at linguistlist.org>
James Yuells, Wayne State U. <james at linguistlist.org>
Ljuba Veselinova, Stockholm U. <ljuba at linguistlist.org>
Software: John Remmers, E. Michigan U. <remmers at emunix.emich.edu>
Gayathri Sriram, E. Michigan U. <gayatri at linguistlist.org>
Home Page: http://linguistlist.org/
The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.
Editor for this issue: Jody Huellmantel <jody at linguistlist.org>
==========================================================================
As a matter of policy, LINGUIST discourages the use of abbreviations
or acronyms in conference announcements unless they are explained in
the text.
=================================Directory=================================
1)
Date: Wed, 17 Jan 2001 13:38:26 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: NAACL-01 Machine Translation Workshop
2)
Date: Wed, 17 Jan 2001 18:48:16 -0600
From: Franz Rothlauf <rothlauf at illigal.ge.uiuc.edu>
Subject: ROPNET-2001 Workshop at GECCO-2001
-------------------------------- Message 1 -------------------------------
Date: Wed, 17 Jan 2001 13:38:26 EST
From: Priscilla Rasmussen <rasmusse at cs.rutgers.edu>
Subject: NAACL-01 Machine Translation Workshop
________________________________________________________________
CALL FOR PARTICIPATION
Workshop on Machine Translation Evaluation
in conjunction with NAACL-2001
WORKSHOP ON MT EVALUATION:
Hands-On Evaluation
3 June, 2001
Pittsburgh, PA
United States
MOTIVATION
Evaluation of language tools, particularly tools that generate language,
remains an interesting and general problem. Machine Translation (MT) is a
prime example. Approaches to evaluating MT are even more plentiful than
approaches to MT itself; the number of evaluations and range of variants is
confusing to anyone considering an evaluation. In an effort to systematize
MT evaluation, the NSF-funded ISLE project has created a taxonomy of
evaluation-related features and measures. Unfortunately, however, many prior
evaluations do not include an adequate specification of important aspects
such as evaluation process complexity, cost, variance of score, etc.
In an effort to drive MT evaluation to the next level, this workshop will
focus on exercising with methods of acquiring such information for several
important MT evaluation measures. The workshop thus embodies the challenge
of Hands-On Evaluation, within the context of the framework being developed
by the ISLE MT Evaluation effort. The workshop follows a workshop on MT
Evaluation held at the AMTA Conference in Cuernavaca, Mexico, in October
2000, and a subsequent workshop being planned for April 2001 in Geneva.
STRUCTURE OF THE WORKSHOP
The first part of the workshop will introduce the ISLE MT Evaluation effort,
funded by NSF and the EU, to create a general framework of characteristics
in terms of which MT evaluations, past and future, can be described and
classified. The framework, whose antecedents are the JEIDA and EAGLES
reports, consists of taxonomies of increasingly specific features, with
associated measures and pointers to systems. The discussion will review the
current state of the classification effort as well as review the MT
evaluation history from which it was drawn.
The second, principal, part of the workshop will focus on real-world
evaluation. In an effort to facilitate common ground for discussion,
participants will be given specific evaluation exercises, defined by the
taxonomy and recent MT evaluation trends. In addition, they will be given a
set of texts generated by MT systems and human reference translations. They
will be asked, during the workshop, to perform given evaluation exercises
with the given data. This common framework will give insights into the
evaluation process and useful metrics for driving the development process.
The results of the exercises will then be presented by the participants,
synthesized into a uniform description of each evaluation, and added to the
ISLE taxonomy, which has been made available on the web for future analysis
in MT evaluation. The results of the workshop will also be incorporated into
a publicly available resource and the workbook from the workshop will be
able to be used by teachers of evaluation and MT.
QUESTIONS AND ISSUES
Since this is a hands-on workshop, participants will be asked to submit an
intent to participate. At that time, they will be able to download the
relevant data for review. During the workshop, they will be given a series
of exercises and split into teams for working these exercises. The result of
the workshop will be at least one paper which addresses the following
threads of investigation within the framework:
* What is the variance inherent in an evaluation measure?
* How complex is it to employ a measure?
* What task(s) is the evaluation measure suited to?
* What kinds of tools automate the evaluation process?
* What kind of metrics are useful for users versus system developers?
* How can we use the evaluation process to speed up or improve the MT
development process?
* What kind of impacts does real-world data have?
* How can we evaluate MT when MT is a small part of the data flow?
* How independent is MT of the subsequent processing? That is, cleaning
up the data improves performance, but does it improve it enough? How do
we quantify that?
TO REGISTER
Since this is a hands-on workshop, no papers are being solicited.
Participants will be expected to take part in the exercises and report their
conclusions. They will additionally be encouraged to contribute to a summary
paper of the workshop proceedings. The data will be sent to participants in
advance of the workshop, with instructions on what to do and what to
prepare. The amount of work required should not exceed 4 hours (much less
than paper preparation).
To register an intent to participate, please send a paragraph outlining your
interest in MT, experience with MT evaluation, knowledge of either Spanish
or Arabic, and the following contact information to Flo Reeder (contact info
below):
* name
* address
* e-mail address
* knowledge of other foreign languages
* translation domain specialization
Participants will need to register for the workshop as part of their NAACL
registration.
IMPORTANT DATES
Intent to Participate: April 16, 2001
Release of Data: April 23, 2001
Workshop date: June 3, 2001
CONTACT POINTS
Florence Reeder
MITRE Corporation
1820 Dolley Madison Blvd.
McLean, VA 22102-3481
TEL: 703-883-7156
FAX: 703-883-1379
EMAIL: freeder at mitre.org
Eduard Hovy
Information Sciences Institute
University of Southern California
4676 Admiralty Way
Marina del Rey, CA 90292-6695
TEL: 310-448-8731
FAX: 310-823-6714
EMAIL: hovy at isi.edu
Workshop URL: http://www.isi.edu/natural-language/mt-eval-naacl.html
-------------------------------- Message 2 -------------------------------
Date: Wed, 17 Jan 2001 18:48:16 -0600
From: Franz Rothlauf <rothlauf at illigal.ge.uiuc.edu>
Subject: ROPNET-2001 Workshop at GECCO-2001
2nd Call for participation
ROPNET-2001
REPRESENTAITONS AND OPERATORS FOR NETWORK PROBLEMS
Bird-of-a-feather Workshop
at the
Genetic and Evolutionary Computation Conference 2001
( GECCO-2001 )
San Francisco, California, July 7 - 11, 2001 (Saturday - Wednesday)
A recombination of
the Sixth Annual Genetic Programming Conference (GP-2001) and
the International Conference on Genetic Algorithms (ICGA-2001)
organized by
Franz Rothlauf
to be held on Saturday, July 7, 2001
WORKSHOP SUMMARY
Finding good solutions for network design problems is important in
many fields such as telecommunications, computer, backbone access,
transportation and distribution networks. Over the last years genetic
algorithms have been applied with success to a wide variety of these
different problems. One of the major design issues is how the network
could be represented as an artificial chromosome and what kind of
operators could be defined on the chromosome.
The workshop is intended to give an overview over the existing
approaches and to discuss various representations and operators in the
context of genetic and evolutionary computation. It should compare
theoretical properties and empirical performance characteristics of
different representations and operators and try to find explanations
for performance differences of a genetic algorithm. The workshop will
be focused on representations and operators for network problems, but
it welcomes interesting contributions to encoding issues
that are meaningful for network representations.
PARTICIPATION
Presentations will be selected according to the submitted 10-page
papers which will be reviewed by at least two members of the
international program committee. Accepted papers will be available
in electronic form before the workshop. Abbreviated versions of
the papers will be later published in the workshop proceedings.
The length of each paper will be determined by the number of
accepted papers.
For more detailed submission guidelines and recent updates,
see the workshop pages at
http://btw6x2.oec.uni-bayreuth.de/ropnet/
IMPORTANT DATES (subject to change)
Paper submission deadline: March 1, 2001
Decisions will be mailed by: April 1, 2001
Submissions of camera-ready papers: April 20, 2001
WORKSHOP CHAIR
Franz Rothlauf
Department of Information Systems
University of Bayreuth
95445 Bayreuthn
Germany
rothlauf at uni-bayreuth.de
Tel/Fax: +49 921 55 2819, +49 921 55 2216
Illinois Genetic Algorithms Laboratory
104 S. Mathews Ave.
University of Illinois at Urbana-Champaign
Urbana, IL 61801
rothlauf at illigal.ge.uiuc.edu
ATTENDANCE
Attendance to the workshop is open to all GECCO attendees.
Further information will be posted on the workshop web pages
( http://btw6x2.oec.uni-bayreuth.de/ropnet/ ) as soon as it becomes
available.
We are looking forward to your participation at the first
workshop ROPNET-2001 which is a great opportunity to meet and
discuss the covered topics for researchers in this area of
research as well as the ones who would like to learn more
about representations and networks.
---------------------------------------------------------------------------
LINGUIST List: Vol-12-108
More information about the LINGUIST
mailing list