ELRA Press Release

[Magali Duclaux]

**************************************************************************
Press Release - Immediate - Paris, France, January, 21st 2002
**************************************************************************
New Activity Launched at ELRA, European Language
Resources Association and ELDA, Evaluation
and Language resources Distribution Agency

*** ELRA & ELDA to devote more resources to the evaluation
of Human Language Technologies (HLT). ***

If at the very beginning the main activity of ELRA & ELDA in the
framework of the evaluation task was to supply the language
resources appropriate for testing and evaluation, both are now
getting involved in the evaluation process itself, the evaluation
of products, systems, and applications developed for HLT.

The evaluation has become a major activity in the field of HLT.
This activity is highly critical, as its main objectives are to check
the quality of the developed applications and systems, and ensure
that these are ready for the market. To evaluate a specific techno-
logy means measuring the progress achieved, comparing different
approaches to a given problem, and choosing the best solution,
assuming that its advantages and disadvantages have been analy-
sed. The evaluation also involves the assessment of the availability
of technologies for a given application, product benchmarking, and
assessment of system usability and user satisfaction.

ELDA actively participates in evaluation projects, at French (e.g.
Amaryllis), European (e.g. CLEF, Cross-Language Evaluation Forum),
and world (e.g. Aurora) levels.

The main objective of the Amaryllis project is to evaluate information
retrieval systems with French text documents (corpora and questions
and answers), and to provide a common methodology for the evaluation
of such search tools. The corpora designed for these purposes are avai-
lable in the ELDA catalogue, at http://www.elda.fr/catalog.html

The main task of the CLEF project consists of providing an infrastructure
for the evaluation of information retrieval systems that operate on Euro-
pean languages, in multilingual, monolingual, and cross-language contexts.
Test-suites of reusable data will also be developed for benchmarking purposes.

In the framework of CLEF, ELDA is in charge of conducting the user needs
survey, identifying the data and negotiating their distribution rights with
the
owners, and participating in the production of an exit plan that will set some
evaluation procedures and recommendations regarding the evaluation of
multilingual and cross-lingual systems.

As for the Aurora project, it aims at establishing a worldwide standard for
the feature extraction software in a DSR (Distributed Speech Recognition)
system (evaluation of algorithms for front-end feature extraction in back-
ground noise on one hand; on the other hand, the evaluation and comparison
of the performance of speech recognition algorithms' robustness. The databases
which have been created in the framework of Aurora are distributed by ELDA.

Indeed, several resources developed in the framework of these evaluation
projects are already available in our catalogue, which may be consulted at
the following address: http://www.elda.fr/catalog.html

In the near future, as we are getting more and more involved in the evaluation
activity, ELRA & ELDA will add in its catalogue further resources and tools
related to evaluation, and a new team, whose task will include the organi-
sation of evaluation campaigns and every other aspects of the evaluation
activity, will join the agency.

Contact: Khalid Choukri
Email: choukri at elda.fr

ELRA & ELDA
55-57, rue Brillat Savarin
75013 Paris (France)
Tel.: +33 1 43 13 33 33
Fax: +33 1 43 13 33 30

*** About ELRA & ELDA ***

The European Language Resources Association (ELRA) is a non-profit
making organisation founded by the European Commission in 1995,
with the mission of providing a clearing house for language resources
and promoting Human Language Technologies (HLT).
The Evaluation and Language resources Distribution Agency (ELDA) is
ELRA's operational body. ELDA identifies, collects, markets, evaluates
and distributes language resources, along with the dissemination of
general information in the field of HLT. ELDA has considerable knowledge
and skills in HLT applications and has participated in many French,
European and international projects, such as Prisme, Amaryllis,
C-Oral-Rom, Network-DC, ISLE, Speecon, SpeechDat-Car, LRsP&P, etc.
To find out more about ELRA and ELDA, please visit our web site:
http://www.elda.fr

*************************************************************************************