35.409, FYI: DETESTS-Dis at IberLEF 2024

Mon Feb 5 17:05:06 UTC 2024

LINGUIST List: Vol-35-409. Mon Feb 05 2024. ISSN: 1069 - 4875.

Subject: 35.409, FYI: DETESTS-Dis at IberLEF 2024

Moderators: Malgorzata E. Cavar, Francis Tyers (linguist at linguistlist.org)
Managing Editor: Justin Fuller
Team: Helen Aristar-Dry, Steven Franks, Everett Green, Daniel Swanson, Maria Lucero Guillen Puon, Zackary Leech, Lynzie Coburn, Natasha Singh, Erin Steitz
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org

Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           https://funddrive.linguistlist.org/donate/

Editor for this issue: Justin Fuller <justin at linguistlist.org>
================================================================

Date: 05-Feb-2024
From: Simona Frenda [simona.frenda at gmail.com]
Subject: DETESTS-Dis at IberLEF 2024

Task: DETESTS-Dis (DETEction and classification of racial Stereotypes
in Spanish – Learning with Disagreement)

This task will take part of IberLEF 2024, the 6th Workshop on Iberian
Languages Evaluation Forum at the SEPLN 2024 Conference, which will be
held in Valladolid, Spain, on September 24th.

----------------------------------------------------------------------
---------------------------------------------

Here, we introduce the second edition of the DETESTS task
(Ariza-Casabona, 2022), which was first presented at IberLEF 2022. The
aim of the new edition, DETESTS-Dis, is to detect and classify
explicit and implicit stereotypes in texts from social media and
comments on news articles, incorporating learning with disagreement
techniques. Next, a description of both subtasks is provided:

Subtask 1, Stereotype Identification: This is a binary classification
task the aim of which is to determine whether a comment or sentence
contains at least one stereotype or none, considering the full
distribution of labels provided by the annotators. This subtask
follows the SemEval 2021 Task 12 (Uma et al., 2021) proposal about
learning with disagreement, in which the authors state that there does
not necessarily exist a single gold label for every sample in the
dataset. This fact is particularly evident when multiple contradictory
annotations arise at the data labeling stage due to “debatable,
subjective, or linguistic ambiguity”. The actual gold label of this
subtask is left as a proxy to determine the subset of comments that
will be evaluated in the posterior subtask.

Subtask 2 (Optional), Implicitness Identification: This subtask
introduces a novel binary classification problem to determine whether
the stereotype is manifested or latent within the text, that is,
whether the stereotype is implicit or explicit. The added difficulty
in this case is that implicit stereotypes are not directly expressed
in the text, and a process of inference must be applied by the
annotators. Moreover, there are different strategies in which an
implicit stereotype can be coded, such as metaphors, irony and other
figures of speech, evaluations of the in-group, and the
overgeneralization of a social group from features of some of its
members. This subtask will be presented as a hierarchical binary
classification problem.

Although we recommend participating in both subtasks, participants are
allowed to participate just in one of them (e.g., subtask 1).
Teams will be allowed (and encouraged) to submit multiple runs (max.
5).
To avoid any conflict with the sources of the comments regarding their
intellectual property rights (IPR), the data will be sent privately to
each participant who is interested in the task. The corpus will only
be made available for research purposes.

Important dates (All deadlines are 11:59 PM UTC-12:00):

Training dataset release: March 04, 2024
Test dataset release: April 15, 2024
Systems results: April 29, 2024
Results notification: May 13, 2024
Working papers submission: June 3, 2024
Working papers (peer-)reviewed: June 17, 2024
Camera-ready versions: July 4, 2024
Workshop: September 24, 2024

Task organizers:

Mariona Taulé (Universitat de Barcelona, UB)
Wolfgang Schmeisser (Universitat de Barcelona, UB)
Alejandro Ariza (Universitat de Barcelona, UB)
Pol Pastells (Universitat de Barcelona, UB)
Mireia Farrús (Universitat de Barcelona, UB)
Simona Frenda (Università degli Studi di Torino, UniTo)
Paolo Rosso (Universitat Politècnica de València, UPV)

Contact:

Contact the organizers by writing to: detests.iberlef at gmail.com

Web page: https://detests-dis.github.io/

We invite participants to join our Google Groups to be kept up to date
with the latest news related to the task.

Linguistic Field(s): Computational Linguistics

Subject Language(s): Spanish (spa)

------------------------------------------------------------------------------

Please consider donating to the Linguist List https://give.myiu.org/iu-bloomington/I320011968.html

LINGUIST List is supported by the following publishers:

John Benjamins http://www.benjamins.com/

Lincom GmbH https://lincom-shop.eu/

Linguistic Association of Finland http://www.ling.helsinki.fi/sky/

Multilingual Matters http://www.multilingual-matters.com/

Wiley http://www.wiley.com

----------------------------------------------------------
LINGUIST List: Vol-35-409
----------------------------------------------------------