Appel: A Special Issue of Knowledge Organization on Evaluation of HLT

Fri May 17 13:37:13 UTC 2002

*****************************************************************

                     Rappel/Reminder

*****************************************************************

Evaluating Human Language Technology:

General Applications to Information Access and Management

A Special Issue of Knowledge Organization on Evaluation of HLT

Guest Editor: Widad Mustafa El Hadi,University of Lille 3

*****************************************************************

Call for papers

The last few years have seen a growing interest in evaluating human
language technology (HLT) in general and its applications to
information access and management in particular. Several initiatives
have been conducted in Europe and in the United States. The first
initiatives on HLT evaluation can be traced back to the Defence
Advanced Research Projects Agency (DARPA) and the Japanese programs on
Machine Translation Evaluation. The DARPA/NIST (National Institute of
Standards and Technology) Projects continued on a more regular basis
and particularly with the Text Retrieval Conferences (TREC) and the
Message Understanding Conferences (MUC). More projects are gaining
momentum, be they independent or part of the famous TREC Tracks
(Language Engineering Track, Question & Answering (Q & A) track), or
within the context of the Translingual Information Detection,
Extraction, and Summarization (TIDES) Document Understanding
Conference (DUC), Cross-Language Information Retrieval (CLIR), Text
Summarization Systems, Filtering, or Spoken Document Retrieval.

Some evaluation projects have been supported within the programs of
the European Commission (EAGLES, ELSE, ISLE, SQALE, etc.) others have
been conducted within national programs (the French evaluation
projects: GRACE Project for evaluating part-of-speech taggers, the AUF
four Projects on evaluating Text Retrieval Systems, Alignment
Technology, Terminology Acquisition System Evaluations, and Message
Understanding). These many endeavours show the relevance of HLT
evaluations. Researchers and practitioners in the field of information
are interested, by an assessment of the contribution of these
technologies in order to measure the progress achieved, to compare
different approaches to a given problem and to assess system usability
and user satisfaction.

This Special Issue of Knowledge Organization (scheduled as volume 29,
number 3/4) will aim at bringing together the community of researchers
in the field of HLT from the perspective of Knowledge Organization,
Information Access and Management. Moreover, it will look forward to
providing an overview of the state-of-the-art of HLT, discuss
problems, exchange information on the use of this technology in
monolingual and multilingual environments, discuss evaluation
methodologies and metrics and promote initiatives for international
cooperation in these areas.

We invite original and unpublished submissions to the special issue
covering any of, but not limited to, the following
topics. Contributions previously published in a language other than
English may be considered.

Possible Topics

General Issues on Evaluation

Historical and theoretical backgrounds of HLT evaluation

Evaluation methodologies, protocols and measures

Benchmarking of systems and products

Linguistic Quality

Information Access

Automated Question & Answering

Monolingual and Cross-Language Information Retrieval

Machine Translation in IR environments

Document Understanding

Multimedia document and image retrieval

Information Management

Automatic tracking and detection of emerging topics from unstructured
data (Text Mining)

Evaluating methods for knowledge extraction and acquisition (e.g.
terms)

Information Filtering

Knowledge mapping and access (lexical information, language modelling)

Text Summarization

Categorization

International Refereeing Committee

Clare Beghtol, Faculty of Information Studies, U. of
Toronto, Canada

Stephane Chaudiron, U. Paris 10 and MENRT, France

Khalid Choukri, ELRA-ELDA, France

Marianne Dabbadie LingPro, Paris, France

Jean-Pierre Descles, LaLIC, U. Paris IV Sorbonne, France

Christian Fluhr, CEA, France

Brigitte Grau, LIMSI-CNRS, France

Tony Hartley, Centre for Translation Studies, U. of Leeds, UK

Michle Hudon, Ecole de bibliotheconomie et des sc.de l'information, U.
de Montreal, Canada

Rebecca Green, College of Information Studies, U. of Maryland, USA

Donna Harman, NIST - TREC Group, USA

Margaret King, ISSCO, U. of Geneva, Switzerland

Bente Maegaard, Center for Sprogteknologi-Copenhagen, Denmark

Joseph Mariani, LIMSI-CNRS, and MENRT France

Jean-Paul Metzger, ERSICO, U. Lyon III, France

Adeline Nazarenko, LIPN - CNRS, U. Paris XIII, France

Peter Ohly, Social Science Information Centre (IZ),  Bonn, Germany

Florence Reeder, MITRE Corporation, USA

Gerhard J.A. Riesthuis, U. of Amsterdam, the Netherlands

Patrick Paroubek, LIMSI-CNRS, France

Alexander Sigel, MT Management & Technologies Consulting GmbH, Cologne,
Germany

Karen Ward, Department of Computer Science, U. of Texas at El Paso, USA

Submission Information

Deadline for submissions is: 31 May 2002

Submissions will be refereed by a special committee of experts in the
field.

Manuscripts should be submitted electronically in Word, WordPerfect or
RTF format in English only and should be accompanied by an indicative
abstract of 100-200 words.

Submission via e-mail are preferred; submissions will be accepted via
post provided that four copies are submitted or that submissions are
accompanied by a 3.5" diskette encoded in Word, WordPerfect or RTF
format.

Please refer to the instructions for authors provided in the attached
file.

Please use the subject line: KO HLT

Submissions should be directed to:

Widad MUSTAFA EL HADI

E-mail: mustafa at univ-lille3.fr

Or at

UFR IDIST
Universite Charles De Gaulle Lille 3
BP 149, 59653 Villeneuve D'Ascq,
FRANCE

----------------------------------

Widad Mustafa el Hadi

UFR IDIST & CERSATES (UMR 8529)

Universite Charles-De-Gaulle-Lille 3

BP 149

59653 Villeneuve d'Ascq Cedex

Tel: 33 (0) 3 20 41 68 15

Fax: 33 (0) 3 20 41 63 79

e-mail: mustafa at univ-lille3.fr

-----------------------------------

-------------------------------------------------------------------------
Message diffusé par la liste Langage Naturel <LN at cines.fr>
Informations, abonnement : http://www.biomath.jussieu.fr/LN/LN-F/
English version          : http://www.biomath.jussieu.fr/LN/LN/
Archives                 : http://listserv.linguistlist.org/archives/ln.html

La liste LN est parrainée par l'ATALA (Association pour le Traitement
Automatique des Langues)
Information et adhésion  : http://www.atala.org/
-------------------------------------------------------------------------