[Corpora-List] why LREC2012 NOT blind-reviewed?

Thu Oct 6 15:30:23 UTC 2011

Ladies and Gents,

This fantastic reviewing-ralated topic has been probably the most active  
one for some time. It looks like most of Corpora members put their 2 cents  
into the bag. The bag seems now quite heavy and shows somewhat big  
potential to turn it into something creative.

My question/proposition is,

How about working out a better reviewing system than the present double  
blind reviewing?

It could actually change something for the better, and since most of us  
took an active part in the discussion we probably have some ideas.

To give a good start let me say my bunch of ideas.

- Field assessment before reviewing. Something that has been implemented  
in easychair and several other systems to some extent. A reviewer selects  
her fields of expertise to give the conference organizers a hint on what  
papers he/she could review with the highest confidence.
- Make blinding a choice. Some researchers want to be anonymous, and some  
want to use their previous research, but don't want to describe their  
previous works too extensively (saving paper space).
- Discussion between authors and reviewers. For conferences there could be  
too short time span for double rebuttal, but for journals there could be  
allowed a longer open discussion. Its sometimes difficult to convince a  
reviewer only in one rebuttal session.
- Using NLP technology to help reviewers and authors. Automatic  
summarization, credibility verification of infromation contained in a  
paper - for the reviewers, and for the other side (or both), opinion  
mining/sentiment analysis of all other reviews of the paper (comparing  
reviews) and of all reviews of one reviewer (is he/she an understanding,  
tolerant reviewer or an innocent bunny killer?)

Let me know about your ideas.

Michal

--------------------------
Od: Anil Singh <anil.phdcl at gmail.com>
Kopia dla: corpora at uib.no
Do: Gemma Boleda <gemma.boleda at upf.edu>
Data: Thu, 6 Oct 2011 02:39:41 +0530
Temat: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

Not to forget the fact that getting a visa from countries like India to  
the US or to Europe is a project in itself. And it costs quite something.  
And if the visa is rejected after all that time, effort and expenditure...  
Rejection of visa is not rare at all: for the US (where a substantial  
proportion of the highest profile conferences are held) the rate is very  
high, especially for gradutate students.

On Thu, Oct 6, 2011 at 2:09 AM, Anil Singh <anil.phdcl at gmail.com> wrote:
  I mostly agree on this point, except that this is quite a radical  
proposal and can be implemented only in the long term. We will need a lot  
of good quality journal to replace conference proceedings. Will be these  
be printed or purely online? Will those purely online considered to have  
the same credibility as those printed (why not?)?

The cost of attending a conference/workshop is, of course, a major hurdle  
for all researchers in developing countries. Nowadays even registration  
fee is so high that it is hard to afford it (say, in India) just for one  
author, even if no one travels abroad to attend the event and present the  
paper. And no solution for this is in sight (never mind the Emerging  
Economy, New Economic Powerhouses  etc.). We are clearly being told  
explicity to not try to publish in conferences, but to try for journals  
(students by supervisors and administration, teachers/researchers by  
administrations and funding agencies). Easy to say, but how many journals  
are out there for the whole of CL/NLP (as compared to the number  
conferences and workshops)?

Still, shifting to journals from conferences (for publication) is  
something that has to happen in CL/NLP.

The academic evaluation forms (in India at least) give a much higher  
weightage to journal publications than to conference publications, which  
is a big disadvantage for those working in CL/NLP under the current  
situation.

Of course, as even Church (2005) had hinted, there is practical problem  
involved. Conference publications mean registrations and registrations  
mean $$. Where will the money for the journals come from? Who will sponsor  
them? If commercial publishers publish them, won't other factors come in  
which might affect their cridibility?

Just some doubts. But generally I support the idea.

On Wed, Oct 5, 2011 at 9:45 PM, Gemma Boleda <gemma.boleda at upf.edu> wrote:
  Dear list members,

  some of the concerns that have been raised in this discussion, such as  
reviewer load and "incrementality" in papers, could be addressed if the  
field moved to journal, rather than conference, publishing, and used  
conferences for dissemination of ideas (where only abstracts would be  
reviewed) and journals for actual publication. This would have the  
positive side-effect of making the citation indices of computational  
linguistics as a subfield go up, thus making it more visible in the  
"scientific market". For a 1.5-page long elaboration of these ideas, see  
http://www.lsi.upc.edu/~gboleda/pubs/gboleda_publishingCL.pdf

  Best,
  Gemma Boleda

  _______________________________________________
  UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
  Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora