[Corpora-List] CFP: Linguistic Annotation Workshop (LAW VIII), Dublin, Aug 23-24

Tue Mar 25 08:56:15 UTC 2014

The 8th Linguistic Annotation Workshop (LAW VIII 2014)
Sponsored by the ACL Special Interest Group on Annotation (SIGANN)

Held in Conjunction with the 25th International Conference on Computational Linguistics (Coling 2014)

Dublin, Ireland
August 23-24, 2014

http://www.ling.uni-potsdam.de/acl-lab/law2014

Important dates
---------------

May 2nd, 2014: Submission deadline
June 6th, 2014: Notification of Acceptance
June 27th, 2014: Camera-ready paper due
August 23rd-24th 2014: Workshop in Dublin, Ireland

Workshop overview
-----------------

Linguistic annotation of natural language corpora is the backbone of supervised methods for statistical natural language processing. It also provides valuable data for evaluation of both rule-based and supervised systems and can help formalize and study linguistic phenomena.

The LAW provides a forum for presentation and discussion of innovative research on all aspects of linguistic annotation, including creation/evaluation of annotation schemes, methods for automatic and manual annotation, use and evaluation of annotation software and frameworks, representation of linguistic data and annotations, etc.

Submissions
-----------

We welcome submissions of long (8 pages) and short (4 pages) papers, posters, and demonstrations, relating to any aspect of linguistic annotation, including:

(a) Annotation procedures:
  * Innovative automated and manual strategies for annotation
  * Machine learning and knowledge-based methods for automation of corpus annotation
  * Creation, maintenance, and interactive exploration of annotation structures and annotated data

(b) Annotation evaluation:
  * Inter-annotator agreement and other evaluation metrics and strategies
  * Qualitative evaluation of linguistic representation

(c) Annotation access and use:
  * Representation formats/structures for merged annotations of different phenomena, and means to explore/manipulate them
  * Linguistic considerations for merging annotations of distinct phenomena

(d) Annotation guidelines and standards:
  * Best practices for annotation procedures and/or development and documentation of annotation schemes
  * Interoperability of annotation formats and/or frameworks among different systems as well as different tasks, frameworks, modalities, and languages

(e) Annotation software and frameworks:
  * Development, evaluation and/or innovative use of annotation software frameworks

(f) Annotation schemes:
  * New and innovative annotation schemes
  * Comparison of annotation schemes

Workshop Theme
--------------

This year, we in particular welcome contributions that address the workshop theme:  The good, the bad, and the perfect:  How good does annotation need to be?

It has been said that the perfect is the enemy of the good.   This may be true for some machine learning applications where a small amount of rough annotation gives good results, but it also may be used to justify  low quality annotation or give higher priority or higher amounts of funding to machine learning than to human annotation.

We solicit evidence for and against "The perfect is the enemy of the good".

In favor of high quality annotation, Manning (2011) suggests that the largest opportunity for improvement in part-of-speech tagging lies in improving the tag set and the accuracy of annotation.   (But he also suggests that perfect annotation of words into discrete lexical categories is not possible because some words do not fall cleanly into one category.) Reidsma and Carletta (2008) advocate caution in deciding how good annotation needs to be.   They show that low agreement among annotators may not be harmful to machine learning as long as the disagreements are random, whereas disagreements that follow patterns can lead machine learning astray even when agreement among annotators is high. In a related vein, Min and Grishman (2012) show that it can be more cost-effective for machine learning to have lots of single-pass less-accurate annotation, than a smaller amount of more-accurate adjudicated annotation. Finally, one recent trend focuses on coarse-grained annotation schemes (McDonald et al., 2013; Petrov et al., 2013; Schneider et al., 2013) to speed up annotation and/or benefit cross-lingual training. Coarse-grained annotation schemes are attractive because they are easy to learn, but are they suitable for all applications?

(references: see workshop website)

Submission Information
----------------------
The papers should report original and unpublished research on topics of interest for the workshop. Accepted papers are expected to be presented at the workshop, and will be published in the workshop proceedings. They should emphasize obtained results rather than intended work, and should indicate clearly the state of completion of the reported results.

A paper accepted for presentation at the workshop must not be presented or have been presented at any other meeting with publicly available proceedings.

Submissions must be in PDF format and must be consistent with the Coling 2014 style files, available at
http://www.coling-2014.org/instructions-for-authors.php

The maximum length is eight (8) pages of content for long papers or four (4) pages of content for short papers, posters, and demonstrations, plus up to two (2) pages of references.

Reviewing of papers will be double-blind. Therefore, the paper must not include the authors' names and affiliations, and self-references that reveal the author's identity, e.g., "We previously showed (Smith, 1991) ..." should be replaced with citations such as "Smith (1991) previously showed ...". Papers that do not conform to these requirements will be rejected without review.

Authors of papers that have been or will be submitted to other meetings or publications must provide this information on the START online submission page. Authors of accepted papers must notify the program chairs within 10 days of acceptance if the paper is withdrawn for any reason.

Submission site: https://www.softconf.com/coling2014/WS-6/

Submission deadline: May 2nd, 2014, 23:59 GMT. Papers submitted after the deadline will not be reviewed.

Workshop Chairs
---------------
Lori Levin (Carnegie-Mellon University)
Manfred Stede (University of Potsdam)

Organizing Committee
--------------------
Stefanie Dipper (Ruhr Universität Bochum)
Chu-Ren Huang (The Hong Kong Polytechnic University)
Nancy Ide (Vassar College)
Adam Meyers (New York University)
Antonio Pareja-Lora (SIC & ILSA, UCM / ATLAS, UNED)
Massimo Poesio (University of Trento)
Sameer Pradhan (Harvard University)
Katrin Tomanek (University of Jena)
Fei Xia (University of Washington)
Nianwen Xue (Brandeis University)

Programme Committee
-------------------
(see workshop website)

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora