8.1706, Confs: Word Sense Disambiguation Evaluation

The LINGUIST List linguist at linguistlist.org
Thu Nov 27 16:19:40 UTC 1997

LINGUIST List:  Vol-8-1706. Thu Nov 27 1997. ISSN: 1068-4875.

Subject: 8.1706, Confs: Word Sense Disambiguation Evaluation

Moderators: Anthony Rodrigues Aristar: Texas A&M U. <aristar at linguistlist.org>
            Helen Dry: Eastern Michigan U. <hdry at linguistlist.org>
            T. Daniel Seely: Eastern Michigan U. <seely at linguistlist.org>

Review Editor:     Andrew Carnie <carnie at linguistlist.org>

Associate Editor: Ljuba Veselinova <ljuba at linguistlist.org>

Assistant Editors:  Martin Jacobsen <marty at linguistlist.org>
                    Brett Churchill <brett at linguistlist.org>
                    Anita Huang <anita at linguistlist.org>
                    Julie Wilson <julie at linguistlist.org>
                    Elaine Halleck <elaine at linguistlist.org>

Software development: John H. Remmers <remmers at emunix.emich.edu>
                      Zhiping Zheng <zzheng at online.emich.edu>

Home Page:  http://linguistlist.org/

Editor for this issue: Anita Huang <anita at linguistlist.org>
Please keep your conference announcement as short as you can; LINGUIST
will not post conference announcements which in our opinion are
excessively long.  Also, please remember that, once posted, your
announcement will be permanently available at our website:
For this reason, we discourage multiple submissions of the same
conference announcement.  Thank you for your cooperation.


Date:  Thu, 27 Nov 1997 14:23:38 GMT
From:  senseval-coord at itri.brighton.ac.uk
Subject:  SENSEVAL WSD Evaluation Exercise

-------------------------------- Message 1 -------------------------------

Date:  Thu, 27 Nov 1997 14:23:38 GMT
From:  senseval-coord at itri.brighton.ac.uk
Subject:  SENSEVAL WSD Evaluation Exercise

(This document is http://www.itri.brighton.ac.uk/events/senseval/cfp.txt)

                 *        CALL FOR PARTICIPATION        *
                 *       WSD EVALUATION  EXERCISE       *
                 *               (SENSEVAL)             *
                 *                                      *
                 *        WORKSHOP ANNOUNCEMENT         *

                          Sponsored by ACL SIGLEX

There are now many automatic Word Sense Disambiguation (WSD) programs
but it is currently very hard to determine which are better, which
worse, and where the strengths and weaknesses of each lie.  There is
widespread agreement that the field urgently needs an evaluation
framework.  Under the auspices of ACL SIGLEX, a pilot will take place
in the course of 1998.  As in ARPA evaluations, the framework comprises:

1)  definition of task and scoring metric
2)  preparation of a set of manually tagged correct answers
3)  a dry run, with sample data distributed to participants
4)  distribution of test data to participants
5)  participants sense-tag and return the data
6)  participants' taggings scored against correct answers
7)  workshop to discuss results, lessons learned, way forward

We shall be undertaking evaluation for English, French, German,
Italian and Spanish.  The workshop will be held at

                     Herstmonceux Castle, Sussex, UK
                              Sept 2-4 1998

If you have a working WSD program (or will have one by Summer 1998),
and would like to subject it to objective, quantitative evaluation, or
if you have skills or resources that you would like to contribute to
the exercise, first look at


and then mail your expression of interest to

                     senseval-coord at itri.bton.ac.uk

                          Provisional timetable
By end of
Dec 97:         expressions of interest to be registered
Mar 98:         'dry run' data samples distributed to participants
May 98:         test data distributed to participants, and, one week
                later, returned with taggings
Jul 98:         correct answers and scores made available
Sep 98:         workshop

The workshop will comprise:

    (1) reports/papers on the manual tagging and other aspects of
        the evaluation exercise
    (2) reports/papers from participants on the design and
        performance of their system
    (3) other related research papers
    (4) working sessions on the way ahead for WSD evaluation

The deadline for "other related research papers" is

                            16th April 1998

* Preference will be given to papers discussing WSD, with particular
  attention to evaluation issues
* Maximum submission length: 6 pages
* First page to include title, abstract, and author's name(s) and
  contact details
* Electronic submission of postscript documents permitted but must be
  supported by hard copy to arrive not later than 7th April
  (in case of printing problems).
       email:       senseval-submissions at itri.bton.ac.uk
       hard copy:   SENSEVAL Submissions
                    University of Brighton
                    Lewes Road
                    Brighton BN2 4GJ,  UK

A background paper is available on


                    Planning and Programme Committee

Robert Amsler                      Science and Engineering Associates, Inc.
Susan Armstrong                    ISSCO, University of Geneva
Roberto Basili                     University of Rome, Tor Vergata
Henri B\'{e}joint                  University of Lyon II
Bran Boguraev                      Apple Computers
Nicoletta Calzolari                Istituto di Linguistica Computazionale, Pisa
John Carroll                       University of Sussex
Steve Crowdy                       Addison Wesley Longman
Roger Evans                        ITRI, University of Brighton
Helmut Feldweg                     University of T\"{u}bingen
Christiane Fellbaum                WordNet, Princeton University
Patrick Hanks                      Oxford University Press
Marti Hearst                       University of California, Berkeley
Lynette Hirschman                  MITRE Corporation
Nancy Ide                          Vassar College
Adam Kilgarriff (co-ordinator)     ITRI, University of Brighton
Judith Klavans                     Columbia University
Ramesh Krishnamurthy               University of Birmingham
Robert Krovetz                     NEC Research Institute, New Jersey
Marc Light                         University of Stuttgart
Dan Melamed                        University of Pennsylvania
Rosamund Moon                      Collins and University of Birmingham
Boyan Onyshkevych                  US Department of Defense
Martha Palmer (ACL SIGLEX chair)   University of Pennsylvania
Philip Resnik                      University of Maryland
Michael Rundell                    Lexicographer
Hinrich Sch\"{u}tze                Xerox Parc and Stanford University
Fr\'{e}d\'{e}rique Segond          Xerox Research Centre, Grenoble
Jean V\'{e}ronis                   Universit\'{e} de Provence
Evelyne Viegas                     New Mexico State University
Clare Voss                         ARL (US Army Research Lab)
Piek Vossen                        University of Amsterdam
Jan Wiebe                          New Mexico State University
Yorick Wilks                       University of Sheffield
David Yarowsky                     Johns Hopkins University
Antonio Zampolli        University of Pisa

(This document is http://www.itri.brighton.ac.uk/events/senseval/cfp.txt)

LINGUIST List: Vol-8-1706

More information about the Linguist mailing list