34.1871, Calls: CLEF 2024 Conference and Labs of the Evaluation Forum

[The LINGUIST List]

LINGUIST List: Vol-34-1871. Tue Jun 13 2023. ISSN: 1069 - 4875.

Subject: 34.1871, Calls: CLEF 2024 Conference and Labs of the Evaluation Forum

Moderator: Malgorzata E. Cavar, Francis Tyers (linguist at linguistlist.org)
Managing Editor: Lauren Perkins
Team: Helen Aristar-Dry, Steven Franks, Everett Green, Joshua Sims, Daniel Swanson, Matthew Fort, Maria Lucero Guillen Puon, Zackary Leech, Lynzie Coburn
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org

Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           https://funddrive.linguistlist.org/donate/

Editor for this issue: Everett Green <everett at linguistlist.org>
================================================================


Date: 12-Jun-2023
From: Petra Galuscakova [galuscakova at gmail.com]
Subject: CLEF 2024 Conference and Labs of the Evaluation Forum


Full Title: CLEF 2024 Conference and Labs of the Evaluation Forum
Short Title: CLEF 2024

Date: 09-Sep-2024 - 12-Sep-2024
Location: Grenoble, France
Contact Person: Petra Galuscakova
Meeting Email: galuscakova at gmail.com
Web Site: http://clef2024.imag.fr/

Linguistic Field(s): Applied Linguistics

Call Deadline: 07-Jul-2023

Meeting Description:

At its 25th edition, the Conference and Labs of the Evaluation Forum
(CLEF) is a continuation of the very successful series of evaluation
campaigns of the Cross Language Evaluation Forum (CLEF) which ran
between 2000 and 2009, and established a framework of systematic
evaluation of information access systems, primarily through
experimentation on shared tasks. As a leading annual international
conference, CLEF uniquely combines evaluation laboratories and
workshops with research presentations, panels, posters and demo
sessions. In 2024, CLEF takes place in September,  9-12 at the
University of Grenoble Alpes, France.

Researchers and practitioners from all areas of information access and
related communities are invited to submit proposals for running
evaluation labs as part of CLEF 2024. Proposals will be reviewed by a
lab selection committee, composed of researchers with extensive
experience in evaluating information retrieval and extraction systems.
Organisers of selected proposals will be invited to include their lab
in the CLEF 2024 labs programme, possibly subject to suggested
modifications to their proposal to better suit the CLEF lab workflow
or timeline.

The CLEF Initiative (http://www.clef-initiative.eu/) is a
self-organised body whose main mission is to promote research,
innovation, and development of information access systems with an
emphasis on multilingual information in different modalities -
including text and multimedia - with various levels of structure. CLEF
promotes research and development by providing an infrastructure for:
- independent evaluation of information access systems;
- investigation of the use of unstructured, semi-structured,
highly-structured, and semantically enriched data in information
access;
- creation of reusable test collections for benchmarking;
- exploration of new evaluation methodologies and innovative ways of
using experimental data;
- discussion of results, comparison of approaches, exchange of ideas,
and transfer of knowledge.

Call for Papers:

Call for Lab Proposals

We invite submission of proposals for two types of labs:

- “Campaign-style” Evaluation Labs for specific information access
problems (during the twelve months period preceding the conference),
similar in nature to the traditional CLEF campaign “tracks”. Topics
covered by campaign-style labs can be inspired by any information
access-related domain or task.

- Labs that follow a more classical “workshop” pattern, exploring
evaluation methodology, metrics, processes, etc. in information access
and closely related fields, such as natural language processing,
machine translation, and human-computer interaction.

We highly recommend organisers new to the CLEF format of shared task
evaluation campaigns to first consider organising a lab workshop to
discuss the format of their proposed task, the problem space and
practicalities of the shared task. The CLEF 2024 programme will
reserve about half of the conference schedule for lab sessions. During
the conference, the lab organisers will present their overall results
in overview presentations during the plenary scientific paper sessions
to give non-participants insights into where the research frontiers
are moving. During the conference, lab organisers are expected to
organise separate sessions for their lab with ample time for general
discussion and engagement with all participants - not just those
presenting campaign results and papers. Organisers should plan time in
their sessions for activities such as panels, demos, poster sessions,
etc. as appropriate. CLEF is always interested in receiving and
facilitating innovative lab proposals.

Potential task proposers unsure of the suitability of their task
proposal or its format for inclusion at CLEF are encouraged to contact
the CLEF 2024 Lab Organizing Committee Chairs to discuss its
suitability or design at an early stage.

Each submitted proposal will be reviewed by the CLEF 2024 Lab
Organizing Committee. The acceptance decision will be sent by email to
the responsible organiser by 28 July 2023. The final length of the lab
session at the conference will be determined based on the overall
organisation of the conference and the number of participant
submissions received by a lab.



------------------------------------------------------------------------------

Please consider donating to the Linguist List https://give.myiu.org/iu-bloomington/I320011968.html


LINGUIST List is supported by the following publishers:

American Dialect Society/Duke University Press http://dukeupress.edu

Bloomsbury Publishing (formerly The Continuum International Publishing Group) http://www.bloomsbury.com/uk/

Brill http://www.brill.com

Cambridge Scholars Publishing http://www.cambridgescholars.com/

Cambridge University Press http://www.cambridge.org/linguistics

Cascadilla Press http://www.cascadilla.com/

De Gruyter Mouton https://cloud.newsletter.degruyter.com/mouton

Dictionary Society of North America http://dictionarysociety.com/

Edinburgh University Press www.edinburghuniversitypress.com

Equinox Publishing Ltd http://www.equinoxpub.com/

European Language Resources Association (ELRA) http://www.elra.info

Georgetown University Press http://www.press.georgetown.edu

John Benjamins http://www.benjamins.com/

Lincom GmbH https://lincom-shop.eu/

Linguistic Association of Finland http://www.ling.helsinki.fi/sky/

MIT Press http://mitpress.mit.edu/

Multilingual Matters http://www.multilingual-matters.com/

Narr Francke Attempto Verlag GmbH + Co. KG http://www.narr.de/

Netherlands Graduate School of Linguistics / Landelijke (LOT) http://www.lotpublications.nl/

Oxford University Press http://www.oup.com/us

SIL International Publications http://www.sil.org/resources/publications

Springer Nature http://www.springer.com

Wiley http://www.wiley.com


----------------------------------------------------------
LINGUIST List: Vol-34-1871
----------------------------------------------------------