28.4898, Calls: Applied Ling, Comp Ling, Gen Ling, Lang Documentation, Text/Corpus Ling/Japan

[The LINGUIST List]

LINGUIST List: Vol-28-4898. Tue Nov 21 2017. ISSN: 1069 - 4875.

Subject: 28.4898, Calls: Applied Ling, Comp Ling, Gen Ling, Lang Documentation, Text/Corpus Ling/Japan

Moderators: linguist at linguistlist.org (Damir Cavar, Malgorzata E. Cavar)
Reviews: reviews at linguistlist.org (Helen Aristar-Dry, Robert Coté,
                                   Michael Czerniakowski)
Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           http://funddrive.linguistlist.org/donate/

Editor for this issue: Kenneth Steimel <ken at linguistlist.org>
================================================================


Date: Tue, 21 Nov 2017 15:32:01
From: António Branco [antonio.branco at di.fc.ul.pt]
Subject: Workshop on Replicability and Reproducibility of Research Results in Science and Technology of Language

 
Full Title: Workshop on Replicability and Reproducibility of Research Results in Science and Technology of Language 
Short Title: 4REAL Workshop 2018 

Date: 12-May-2018 - 12-May-2018
Location: Miyazaki, Japan 
Contact Person: António Branco
Meeting Email: antonio.branco at di.fc.ul.pt
Web Site: http://4real2018.di.fc.ul.pt 

Linguistic Field(s): Applied Linguistics; Computational Linguistics; General Linguistics; Language Documentation; Text/Corpus Linguistics 

Call Deadline: 15-Jan-2018 

Meeting Description:

Reproduction and replication of research results are at the heart of the
validation of scientific knowledge and of the scientific endeavor.
Reproduction of results entails arriving at the same overall conclusions that
is, to appropriately validate a set of results, scientists should strive to
reproduce the same answer to a given research question by different means,
e.g. by reimplementing an algorithm or evaluating it on a new dataset, etc.
Replication has a more limited aim, typically involving running the exact same
solution or approach under the same conditions in order to arrive at the same
output result.

Despite their key importance, reproduction and replication have not been
sufficiently encouraged given the prevailing procedures and priorities for the
reviewing, selection and publication of research results.

The immediate motivation for the increased interest on reproducibility and
replicability is to be found in a number of factors, including the realization
that for some published results, their replication is not being obtained (e.g.
Prinz et al., 2011; Belgley and Ellis, 2012); that there may be problems with
the commonly accepted reviewing procedures, where deliberately falsified
submissions, with fabricated errors and fake authors, get accepted even in
respectable journals (e.g. Bohannon, 2013); that the expectation of
researchers vis a vis misconduct, as revealed in inquiries to scientists on
questionable practices, scores higher than one might expect or would be ready
to accept (e.g. Fanelli, 2009); among several others.


Call for Papers:

Workshop on Replicability and Reproducibility of Research Results in Science
and Technology of Language

This workshop seeks to foster the discussion and the advancement on a topic
that has been given insufficient attention in the research area of language
processing tools and resources and that has been an important topic emerging
in other scientific areas, continuing the objectives of the first edition of
the 4REAL workshop, at LREC 2016. We are thus inviting submissions of articles
that present cases, either with positive or negative results, of actual
replication or reproduction exercises of previous published results in our
area.

Specific topics within the scope of the call include, but are not limited to:

- Reproduction and replication of previously published systems, resources,
results, etc
- Analysis of reproducibility challenges in system-oriented and user-oriented
evaluation
- Reproducibility challenges on private or proprietary data
- Reproducibility challenges on ephemeral data, like streaming data, tweets,
etc
- Reproducibility challenges on online experiments
- Reproducibility in evaluation campaigns
- Evaluation infrastructures and Evaluation as a Service (EaaS)
- Experiments on data management, data curation, and data quality
- Reproducible experimental workflows: tools and experiences
- Data citation: citing experimental data, dynamic data sets, samples, etc

We are interested also in articles discussing the challenges, the risk
factors, the appropriate procedures, etc. specific to our area or that should
be adopted, or adapted from other neighboring areas, including methodologies
for monitoring, maintaining or improving citation of language resources and
tools and to assess the importance of data citation for research integrity.
This includes also of course the new risks raised by the replication articles
themselves and their own integrity, in view of the preservation of the
reputation of colleagues and works whose results are reported has having been
replicated, etc.

Submission:

Please submit your papers, duly anonymized, at
https://www.softconf.com/lrec2018/4REAL. Accepted papers will be presented as
oral presentations or as posters. All accepted papers will be published in the
workshop proceedings.

Papers should be formatted according to the stylesheet to be provided on the
LREC 2018 website and should not exceed 8 pages, including references and
appendices. Papers should be submitted in PDF format through the URL mentioned
above.




------------------------------------------------------------------------------

*****************    LINGUIST List Support    *****************
Please support the LL editors and operation with a donation at:
            http://funddrive.linguistlist.org/donate/
 


----------------------------------------------------------
LINGUIST List: Vol-28-4898	
----------------------------------------------------------
Visit LL's Multitree project for over 1000 trees dynamically generated
from scholarly hypotheses about language relationships:
          http://multitree.org/