8.1706, Confs: Word Sense Disambiguation Evaluation
The LINGUIST List
linguist at linguistlist.org
Thu Nov 27 16:19:40 UTC 1997
LINGUIST List: Vol-8-1706. Thu Nov 27 1997. ISSN: 1068-4875.
Subject: 8.1706, Confs: Word Sense Disambiguation Evaluation
Moderators: Anthony Rodrigues Aristar: Texas A&M U. <aristar at linguistlist.org>
Helen Dry: Eastern Michigan U. <hdry at linguistlist.org>
T. Daniel Seely: Eastern Michigan U. <seely at linguistlist.org>
Review Editor: Andrew Carnie <carnie at linguistlist.org>
Associate Editor: Ljuba Veselinova <ljuba at linguistlist.org>
Assistant Editors: Martin Jacobsen <marty at linguistlist.org>
Brett Churchill <brett at linguistlist.org>
Anita Huang <anita at linguistlist.org>
Julie Wilson <julie at linguistlist.org>
Elaine Halleck <elaine at linguistlist.org>
Software development: John H. Remmers <remmers at emunix.emich.edu>
Zhiping Zheng <zzheng at online.emich.edu>
Home Page: http://linguistlist.org/
Editor for this issue: Anita Huang <anita at linguistlist.org>
==========================================================================
Please keep your conference announcement as short as you can; LINGUIST
will not post conference announcements which in our opinion are
excessively long. Also, please remember that, once posted, your
announcement will be permanently available at our website:
http://www.linguistlist.org/issues/indices/Confs1997r.html
For this reason, we discourage multiple submissions of the same
conference announcement. Thank you for your cooperation.
=================================Directory=================================
1)
Date: Thu, 27 Nov 1997 14:23:38 GMT
From: senseval-coord at itri.brighton.ac.uk
Subject: SENSEVAL WSD Evaluation Exercise
-------------------------------- Message 1 -------------------------------
Date: Thu, 27 Nov 1997 14:23:38 GMT
From: senseval-coord at itri.brighton.ac.uk
Subject: SENSEVAL WSD Evaluation Exercise
(This document is http://www.itri.brighton.ac.uk/events/senseval/cfp.txt)
****************************************
* CALL FOR PARTICIPATION *
* WSD EVALUATION EXERCISE *
* (SENSEVAL) *
* *
* WORKSHOP ANNOUNCEMENT *
* "SENSEVAL AND THE LEXICOGRAPHY LOOP" *
****************************************
Sponsored by ACL SIGLEX
There are now many automatic Word Sense Disambiguation (WSD) programs
but it is currently very hard to determine which are better, which
worse, and where the strengths and weaknesses of each lie. There is
widespread agreement that the field urgently needs an evaluation
framework. Under the auspices of ACL SIGLEX, a pilot will take place
in the course of 1998. As in ARPA evaluations, the framework comprises:
1) definition of task and scoring metric
2) preparation of a set of manually tagged correct answers
3) a dry run, with sample data distributed to participants
4) distribution of test data to participants
5) participants sense-tag and return the data
6) participants' taggings scored against correct answers
7) workshop to discuss results, lessons learned, way forward
We shall be undertaking evaluation for English, French, German,
Italian and Spanish. The workshop will be held at
Herstmonceux Castle, Sussex, UK
on
Sept 2-4 1998
If you have a working WSD program (or will have one by Summer 1998),
and would like to subject it to objective, quantitative evaluation, or
if you have skills or resources that you would like to contribute to
the exercise, first look at
http://www.itri.bton.ac.uk/events/senseval/exp-interest.html
and then mail your expression of interest to
senseval-coord at itri.bton.ac.uk
Provisional timetable
=====================
By end of
Dec 97: expressions of interest to be registered
Mar 98: 'dry run' data samples distributed to participants
May 98: test data distributed to participants, and, one week
later, returned with taggings
Jul 98: correct answers and scores made available
Sep 98: workshop
The workshop will comprise:
(1) reports/papers on the manual tagging and other aspects of
the evaluation exercise
(2) reports/papers from participants on the design and
performance of their system
(3) other related research papers
(4) working sessions on the way ahead for WSD evaluation
The deadline for "other related research papers" is
16th April 1998
* Preference will be given to papers discussing WSD, with particular
attention to evaluation issues
* Maximum submission length: 6 pages
* First page to include title, abstract, and author's name(s) and
contact details
* Electronic submission of postscript documents permitted but must be
supported by hard copy to arrive not later than 7th April
(in case of printing problems).
email: senseval-submissions at itri.bton.ac.uk
hard copy: SENSEVAL Submissions
ITRI
University of Brighton
Lewes Road
Brighton BN2 4GJ, UK
A background paper is available on
ftp://ftp.itri.bton.ac.uk/pub/reports/ITRI-97-11.ps.gz
Planning and Programme Committee
================================
Robert Amsler Science and Engineering Associates, Inc.
Susan Armstrong ISSCO, University of Geneva
Roberto Basili University of Rome, Tor Vergata
Henri B\'{e}joint University of Lyon II
Bran Boguraev Apple Computers
Nicoletta Calzolari Istituto di Linguistica Computazionale, Pisa
John Carroll University of Sussex
Steve Crowdy Addison Wesley Longman
Roger Evans ITRI, University of Brighton
Helmut Feldweg University of T\"{u}bingen
Christiane Fellbaum WordNet, Princeton University
Patrick Hanks Oxford University Press
Marti Hearst University of California, Berkeley
Lynette Hirschman MITRE Corporation
Nancy Ide Vassar College
Adam Kilgarriff (co-ordinator) ITRI, University of Brighton
Judith Klavans Columbia University
Ramesh Krishnamurthy University of Birmingham
Robert Krovetz NEC Research Institute, New Jersey
Marc Light University of Stuttgart
Dan Melamed University of Pennsylvania
Rosamund Moon Collins and University of Birmingham
Boyan Onyshkevych US Department of Defense
Martha Palmer (ACL SIGLEX chair) University of Pennsylvania
Philip Resnik University of Maryland
Michael Rundell Lexicographer
Hinrich Sch\"{u}tze Xerox Parc and Stanford University
Fr\'{e}d\'{e}rique Segond Xerox Research Centre, Grenoble
Jean V\'{e}ronis Universit\'{e} de Provence
Evelyne Viegas New Mexico State University
Clare Voss ARL (US Army Research Lab)
Piek Vossen University of Amsterdam
Jan Wiebe New Mexico State University
Yorick Wilks University of Sheffield
David Yarowsky Johns Hopkins University
Antonio Zampolli University of Pisa
(This document is http://www.itri.brighton.ac.uk/events/senseval/cfp.txt)
---------------------------------------------------------------------------
LINGUIST List: Vol-8-1706
More information about the LINGUIST
mailing list