28.1377, Calls: Computational Linguistics/Denmark

Tue Mar 21 01:05:42 UTC 2017

LINGUIST List: Vol-28-1377. Mon Mar 20 2017. ISSN: 1069 - 4875.

Subject: 28.1377, Calls: Computational Linguistics/Denmark

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Helen Aristar-Dry, Robert Coté,
                                   Michael Czerniakowski)
Homepage: http://linguistlist.org

*****************    LINGUIST List Support    *****************
                       Fund Drive 2017
                   25 years of LINGUIST List!
Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Kenneth Steimel <ken at linguistlist.org>
================================================================

Date: Mon, 20 Mar 2017 21:05:26
From: Saif Mohammad [uvgotsaif at gmail.com]
Subject: Shared task on Emotion Intensity

Full Title: Shared task on Emotion Intensity 

Date: 08-Sep-2017 - 08-Sep-2017
Location: Copenhagen, Denmark 
Contact Person: Saif Mohammad
Meeting Email: EmotionIntensity at googlegroups.com
Web Site: http://saifmohammad.com/WebPages/EmotionIntensity-SharedTask.html 

Linguistic Field(s): Computational Linguistics 

Call Deadline: 10-Jun-2017 

Meeting Description:

Part of the 8th Workshop on Computational Approaches to Subjectivity,
Sentiment and Social Media Analysis (WASSA-2017), which is to be held in
conjunction with EMNLP-2017.

Background and Significance: 

Existing emotion datasets are mainly annotated categorically without an
indication of degree of emotion. Further, the tasks are almost always framed
as classification tasks (identify 1 among n emotions for this sentence). In
contrast, it is often useful for applications to know the degree to which an
emotion is expressed in text. In this task, systems have to automatically
determine the intensity of emotions in tweets.

Task: 

Given a tweet and an emotion X, determine the intensity or degree of emotion X
felt by the speaker -- a real-valued score between 0 and 1. The maximum
possible score 1 stands for feeling the maximum amount of emotion X (or having
a mental state maximally inclined towards feeling emotion X). The minimum
possible score 0 stands for feeling the least amount of emotion X (or having a
mental state maximally away from feeling emotion X). The tweet along with the
emotion X will be referred to as an instance. Note that the absolute scores
have no inherent meaning -- they are used only as a means to convey that the
instances with higher scores correspond to a greater degree of emotion X than
instances with lower scores.

Data: 

Training, development, and test datasets are provided for four emotions: joy,
sadness, fear, and anger. For example, the anger training dataset has tweets
along with a real-valued score between 0 and 1 indicating the degree of anger
felt by the speaker. The test data includes only the tweet text. Gold emotion
intensity scores will be released after the evaluation period.

Evaluation: 

For each emotion, systems are evaluated by calculating the Pearson Correlation
Coefficient with Gold ratings. The correlation scores across all four emotions
will be averaged to determine the bottom-line competition metric by which the
submissions will be ranked.

The official evaluation script (which also acts as a format checker) is
available for download. You may want to run it on the training set to
determine your progress, and eventually on the test set to check the format of
your submission.

Web Hosting of the Competition: 

The entire competition will be hosted on CodaLab Competitions
(https://competitions.codalab.org/). A direct link to the Emotion Intensity
CodaLab competition is here:
https://competitions.codalab.org/competitions/16380

(CodaLab has been used in many research evaluation competitions in the past
such as Microsoft COCO Image Captioning Challenge and SemEval-2017.)

Call for Papers:

Participants will be given the opportunity to write a system-description paper
that describes their system, resources used, results, and analysis. This paper
will be part of the official WASSA-2017 proceedings. The paper is to be four
pages long plus two pages at most for references. The papers are to follow the
format and style files provided by EMNLP-2017.

Schedule:

Training data ready: Data for anger, fear, and joy are already available; data
for sadness will be made available in the second half of February 2017
Evaluation period starts: May 02, 2017
Evaluation period ends: May 14, 2017
Results posted: May 21, 2017
Workshop paper submission deadline: June 10, 2017
Author notifications : July 9, 2017
Camera ready submissions due: July 23, 2017

Baseline Weka System for Determining Emotion Intensity:

You are free to build a system from scratch using any available software
packages and resources, as long as they are not against the spirit of fair
competition. In order to assist testing of ideas, we also provide a baseline
emotion intensity system that you can build on. The use of this system is
completely optional. The system is available here:
https://github.com/felipebravom/AffectiveTweets

Organizers of the shared task:

Saif M. Mohammad
saif.mohammad at nrc-cnrc.gc.ca
National Research Council Canada 

Felipe Bravo-Marquez
fjb11 at students.waikato.ac.nz 
The University of Waikato

Alexandra Balahur
alexandra.balahur at jrc.ec.europa.eu
European Commission, Brussels

Contact: 
Saif M. Mohammad, saif.mohammad at nrc-cnrc.gc.ca

------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
                       Fund Drive 2017
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/

This year the LINGUIST List hopes to raise $70,000. This money
will go to help keep the List running by supporting all of our 
Student Editors for the coming year.

Don't forget to check out the Fund Drive 2017 site!

http://funddrive.linguistlist.org/

We collect donations via the eLinguistics Foundation, a
registered 501(c) Non Profit organization with the federal tax
number 45-4211155. The donations can be offset against your
federal and sometimes your state tax return (U.S. tax payers
only). For more information visit the IRS Web-Site, or contact
your financial advisor.

Many companies also offer a gift matching program. Contact
your human resources department and send us the necessary form.

Thank you very much for your support of LINGUIST!

----------------------------------------------------------
LINGUIST List: Vol-28-1377	
----------------------------------------------------------