[Corpora-List] why LREC2012 NOT blind-reviewed?

[Eric Ringger]

Thanks to all for the open discussion.

 

Graeme's reason (1)(a) - the impact on merit review - is for me the
strongest reason to encourage LREC to move away from reviewing extended
abstracts and toward reviewing full papers.

 

Best,

--Eric 

 

From: Graeme Hirst [mailto:gh at cs.toronto.edu] 
Sent: Friday, September 30, 2011 11:02 AM
To: Yorick Wilks
Cc: Eric Ringger; corpora at uib.no
Subject: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Yorick,

 

(1)  Whether a conference is reviewed by abstract or by full paper makes an
enormous difference:

 

   (a) to merit, as perceived by tenure committees, granting agencies, and
others, who count only fully peer-reviewed papers.

   (b) to funding for travel.  Right now, one of my colleagues has the
problem that he cannot be funded to travel to give a paper at LREC because
it isn't a fully-reviewed conference, so he doesn't even bother submitting.

 

You might say that these situations aren't desirable, but they are
nonetheless reality right now.

 

(2)  I wonder how you are so sure that you almost invariably identify the
author of an anonymous paper correctly.  If the paper is not ultimately
accepted at the conference, which is 60 to 80% of them at ACL and COLING
conferences, you will never find out who the authors actually are.  I've
certainly guessed wrongly in the past.  And in my own papers, I often throw
in "hidden signals" to deceive the reviewers.

 

(3)  I think Eric Ringger is 100% right about LREC.  As you say, LREC's
reputation and quality have grown, and for that reason it has to start
acting like a grown-up conference.

 

Regards,

Graeme

 

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science

 

On 2011-09-30, at 11:27, Yorick Wilks wrote:





I disagree strongly. I dont see why all conferences should be exactly like
all others. Extended abstracts are less of a burden on busy academics --both
as writers and reviewers----and there is no evidence they lower the final
quality; COLING used to do this and I am sorry it changed. The whole
blind-review business is a huge nonsense: I rarely meet a paper to review
where i cannot identify the authors from a careful trawl of hidden signals
and the references. Trying to make a paper genuinely anonymous is almost
impossible if one has a body of past work and publication to link it
to---the mental gymnastics required are undignified and best avoided. LRECs
reputation has grown steadily and it will be the quality of its papers that
sustain it--there is no evidence at all anonymity would improve matters in
the least. if it ain't broke........

Yorick Wilks

 

 

On 30 Sep 2011, at 16:02, Eric Ringger wrote:





Greetings.

 

LREC has been operated in this manner since its inception.  Personally and
for the sake of LREC's reputation, I would like to see the reviewing process
for LREC upgraded to double-blind review.

 

I believe that LREC fills a couple of important niches: its focus on
language resources and evaluation/validation is important and not well
served elsewhere, and it does a good job of bringing a large, diverse group
together.  (I should add that it does a good job of selecting attractive
venues as well!)  If implemented well, I believe that double-blind review
would not detract from the primary objectives of the conference but would
refine the quality of the program and improve the reputation of the venue.
I have said as much in private feedback after past LRECs.

 

I also think it is time for LREC to move up from reviewing extended
abstracts to reviewing full papers.

 

Regards,

--Eric

 

 

From: corpora-bounces at uib.no [mailto:corpora-bounces at uib.no] On Behalf Of
Isabella Chiari
Sent: Friday, September 30, 2011 8:45 AM
To: corpora at uib.no
Subject: [Corpora-List] why LREC2012 NOT blind-reviewed?

 

Dear Corpora members,

I just noticed that the LREC2012 call specifies that submissions are NOT
anonymous and there will not be blind-reviewing.

 

Does anyone know why? Which is the policy under this decision?

Best regards,

Isabella Chiari

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20110930/351f7d42/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora