[Corpora-List] why LREC2012 NOT blind-reviewed?

[Khalid CHOUKRI]

  Dear Colleagues,

First of all, thank you all for this fruitful discussion which has 
certainly provided us with food for thoughts for improving the next 
editions of LREC.

The discussion brought up a number of issues, the most critical (from 
our understanding) being:
-Paper Submission : abstracts vs full papers
-Peer review : non blind vs blind (or double-blind) reviews, reviewers 
anonymity

These issues have been regularly debated within the LREC Program 
Committee. Just few quick comments here:

As you all know LREC is a specific conference on Language Resources and 
Evaluation. We value contributions on all issues including the very 
"concrete" processes to develop/produce resources
and/or conduct evaluation.

  * Paper Submission: abstracts vs full papers

For each submission, LREC requests an extended abstract (1500-2000 
words) which is, in our view, a good compromise between a full paper and 
an abstract. A 2000-word document is already 4 pages.
In the survey conducted after LREC 2008, 46% of respondents were in 
favor of the full paper option whereas 48% preferred the current 
abstract option.
 From this survey and the on-going discussion, we can see that there is 
no consensus within the community.

In addition, the LREC abstracts now come with the description of the 
associated resources and/or the evaluations, information which is 
accessible to the reviewers to better understand and assess the 
contribution.
This is what we refer to as the LRE Map and which constitutes an 
additional element for the reviewers.

  * Peer review : non blind vs blind (or double-blind) reviews,
    reviewers anonymity

This discussion is carried out by LREC authors and LREC reviewers (we 
are inviting these days more than 1,000 reviewers). Many of you are on 
both sides which is common to peer-reviewed conferences.
 From the current discussions, it seems that when blind reviewing, most 
of the reviewers manage to identify the authors or the teams involved in 
a paper.
For us, after 7 editions of LREC, thousands of papers and thousands 
ofresources collected on the LRE Map, it is easier to identify who is 
working on what.

In particular at LREC, it may be very cumbersome to write a paper 
without the possibility of mentioning the resources/tools you are 
working on, and the paper may become even difficult to read/understand. 
Moreover, it would be even contradictory for us to forbid to name in the 
paper the resources you work on, and on the other side in the Map to ask 
explicitly to list them!



Constructive suggestions have emerged from the discussion and we’ll 
carefully consider these: in particular, the fact that some institutions 
only allow their staff to attend the conferences that require full 
papers, which can prevent interesting papers from being submitted to 
LREC due to abstract submission policy.
The Programme Committee takes this issue seriously and will discuss this 
for the next editions.

Finally, we are very happy to see that many of you argued that LREC was 
conducted like this since the beginning in 1998 and that it has worked 
pretty well. It is a great privilege but also our great responsibility 
and we will continue to do everything possible to make it a better 
event, including but not limited to choosing nice locations!

warm thanks to all,

Nicoletta Calzolari, Khalid Choukri, Stelios Piperidis

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20111013/ced74e6d/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora