alexis.nasr at LINGUIST.JUSSIEU.FR alexis.nasr at LINGUIST.JUSSIEU.FR
Tue Feb 3 08:47:00 UTC 2004

                            Call for papers

                               Workshop on


                              to be held in
                   Lisbon, Portugal, on May 29, 2004

                         in conjunction with the
       4th Conference on Language Resources and Evaluation (LREC-2004)


A half-day workshop will be held in conjunction with the LREC
Conference in Lisbon on the morning of May 29, 2004. The topic of the
workshop is "Lessons Learned from Evaluation: Towards Transparency and
Integration in Cross-Lingual Information Retrieval (LECLIQ)" with
special focus on the use and benefits of quality gates.


Recent work in Cross-Lingual Information Retrieval (CLIR) has shown
that systems perform differently with respect to queries, topics, data
sets and the corresponding query or result language. Since the
state-of-the-art in CLIR is far from finding an ideal method for this
variety of parameters, a better performance might be possible through
continuous analysis and integration of evaluation results achieved
from several retrieval devices. The process of integrating these
different methodologies might be controlled by quality gates.  Quality
gates - having their origin in car manufacturing and being used in
IT-project management as well - are characterized as preventive and
process accompanying control mechanisms to check critical parameters
and ensure quality standards during the design, development, and
deployment of software. They generally consist of checklists combined
with appropriate rules to guarantee that work procedure fail is
recognized in time to prevent repetition.

The objective of this workshop is to bring together researchers from
the field of Information Retrieval and Language Engineering with three
primary goals:

(1) to discuss, evaluate and judge the different methodologies used in
CLIR and to relate them to the languages being retrieved,

(2) to design the requirements for a dynamic network of quality gates
for the tuning of retrieval results,

(3) to define desired learn effects caused by the interaction of
quality gates.

Topics of interest include, but are not restricted to:

- Lessons learned in evaluation initiatives such as CLEF, TREC, INEX,
NTCIR, etc.

- Criteria, checklists, parameters and metrics for the evaluation of
search and retrieval methods

- Quality criteria and evaluation for resources and tools in CLIR
(thesauri stemmer, translation tools and services, etc.)

- Explanation and verification of success stories and failure analysis

- Data and Text Mining on CLIR evaluation results

- Current evaluation issues: shortcomings, gaps, reliability of
results, etc.

- Transfer of evaluation results to system design and product

- Design for (coupled or networked) quality gates in CLIR (for systems
and building blocks)

- Semaphore logic for the integration of distributed retrieval systems

- Crossing-over techniques for quality assurance

- Hybrid systems for CLIR

The results of this workshop will be a first step toward outlining a
general framework for quality gates in (Cross-Language) Information
Retrieval with further elaboration in future workshops.

We are inviting contributions discussing the themes above from
participants in evaluation initiatives as well as from experienced
researches in the areas of (Cross Language) Information Retrieval,
quality assurance, Human Language Technology (HLT) or related fields.


Norbert Fuhr, University of Duisburg, Germany
Marcello Federico, I.R.S.T, Italy			
Gregory Grefenstette, Clairvoyance Corp., Pittsburgh, USA
Donna Harman, NIST, USA
Paul Heisterkamp, DaimlerChrysler AG, Germany
Thomas Mandl, University of Hildesheim, Germany
Dagobert Soergel, University of Maryland, USA
Christa Womser-Hacker, University of Hildesheim, Germany

[Final list forthcoming]


Presentations and active participation in the workshop will be based
on a 2 - 6 page position statement or research paper. Submissions must
be in English and should consist of an abstract, plus a title page
providing the following information: official title of the paper;
names and affiliations of the authors; full address of the first
author including phone, fax, e-mail, URL; presentation facilities
required.  Only electronic submissions in Word, PDF or RTF will be
accepted.  Demonstrations of cross-lingual retrieval tools will be
considered as well if the demonstration outlines the rationale of the
system and sheds light on the success or failure of the methods
applied.  Contributions should be submitted to:
lecliq at Reception will be acknowledged within three
days. If you do not get an acknowledgement, please check the e-mail
address of your submission.


The workshop organizers will publish proceedings of the
workshop. Ready-to-print versions of the papers have to be submitted
according to LREC Guidelines for Authors, which can be found at the
conference homepage (to be announced in February 2004). They should
not exceed six pages. These final versions should be submitted
electronically to the following e-mail address:
lecliq at

Deadline for submitting papers:             February 15
Notification of acceptance:                 March 15
Final versions of papers for proceedings:   April 4
Workshop:                                   May 29


Information about the workshop such as call, outline, and program can
be found on the LREC web-page:
where you can also find information about the LREC conference itself.
Information about the general purpose of quality gates in
manufacturing and software-engineering and their transduction to the
field of (Cross Lingual) Information Retrieval will be available under

The registration fee for the workshop is:
         - 50 EURO for conference participants
         - 85 EURO for those not attending LREC-2004

Registration and payment procedures are explained on the LREC-2004

The registration fee includes one copy of the proceedings and a coffee


For all questions on the focus of the workshop, please contact the
workshop co-ordinator under lecliq at


Christa Womser-Hacker, University of Hildesheim, Center for
  Information Science (Chair)
Norbert Fuhr, University of Duisburg
Paul Heisterkamp, DaimlerChrysler AG
Thomas Mandl, University of Hildesheim
René Schneider, University of Hildesheim (Co-ordinator)

René Schneider

University of Hildesheim, Marienburger Platz 22, D-31141 Hildesheim,

Phone: +49(0)5121/883-834  Fax: +49(0)5121/883-802

e-mail: rschneid @

Message diffusé par la liste Langage Naturel <LN at>
Informations, abonnement :
English version          :
Archives                 :

La liste LN est parrainée par l'ATALA (Association pour le Traitement
Automatique des Langues)
Information et adhésion  :

More information about the Ln mailing list