[Corpora-List] why LREC2012 NOT blind-reviewed?

Thu Oct 6 15:30:38 UTC 2011

Ladies and Gents,

This fantastic reviewing-ralated topic has been probably the most active
one for some time. It looks like most of Corpora members put their 2 cents
into the bag. The bag seems now quite heavy and shows somewhat big
potential to turn it into something creative.

My question/proposition is,

How about working out a better reviewing system than the present double
blind reviewing?

It could actually change something for the better, and since most of us
took an active part in the discussion we probably have some ideas.

To give a good start let me say my bunch of ideas.

- Field assessment before reviewing. Something that has been implemented
in easychair and several other systems to some extent. A reviewer selects
her fields of expertise to give the conference organizers a hint on what
papers he/she could review with the highest confidence.
- Make blinding a choice. Some researchers want to be anonymous, and some
want to use their previous research, but don't want to describe their
previous works too extensively (saving paper space).
- Discussion between authors and reviewers. For conferences there could be
too short time span for double rebuttal, but for journals there could be
allowed a longer open discussion. Its sometimes difficult to convince a
reviewer only in one rebuttal session.
- Using NLP technology to help reviewers and authors. Automatic
summarization, credibility verification of infromation contained in a
paper - for the reviewers, and for the other side (or both), opinion
mining/sentiment analysis of all other reviews of the paper (comparing
reviews) and of all reviews of one reviewer (is he/she an understanding,
tolerant reviewer or an innocent bunny killer?)

Let me know about your ideas.

Michal

--------------------------
Od: Anil Singh <anil.phdcl at gmail.com>
Kopia dla: corpora at uib.no
Do: Gemma Boleda <gemma.boleda at upf.edu>
Data: Thu, 6 Oct 2011 02:39:41 +0530
Temat: Re: [Corpora-List] why LREC2012 NOT blind-reviewed?

Not to forget the fact that getting a visa from countries like India to
the US or to Europe is a project in itself. And it costs quite something.
And if the visa is rejected after all that time, effort and expenditure...
Rejection of visa is not rare at all: for the US (where a substantial
proportion of the highest profile conferences are held) the rate is very
high, especially for gradutate students.

On Thu, Oct 6, 2011 at 2:09 AM, Anil Singh <anil.phdcl at gmail.com> wrote:
    I mostly agree on this point, except that this is quite a radical
proposal and can be implemented only in the long term. We will need a lot
of good quality journal to replace conference proceedings. Will be these
be printed or purely online? Will those purely online considered to have
the same credibility as those printed (why not?)?

The cost of attending a conference/workshop is, of course, a major hurdle
for all researchers in developing countries. Nowadays even registration
fee is so high that it is hard to afford it (say, in India) just for one
author, even if no one travels abroad to attend the event and present the
paper. And no solution for this is in sight (never mind the Emerging
Economy, New Economic Powerhouses  etc.). We are clearly being told
explicity to not try to publish in conferences, but to try for journals
(students by supervisors and administration, teachers/researchers by
administrations and funding agencies). Easy to say, but how many journals
are out there for the whole of CL/NLP (as compared to the number
conferences and workshops)?

Still, shifting to journals from conferences (for publication) is
something that has to happen in CL/NLP.

The academic evaluation forms (in India at least) give a much higher
weightage to journal publications than to conference publications, which
is a big disadvantage for those working in CL/NLP under the current
situation.

Of course, as even Church (2005) had hinted, there is practical problem
involved. Conference publications mean registrations and registrations
mean $$. Where will the money for the journals come from? Who will sponsor
them? If commercial publishers publish them, won't other factors come in
which might affect their cridibility?

Just some doubts. But generally I support the idea.

On Wed, Oct 5, 2011 at 9:45 PM, Gemma Boleda <gemma.boleda at upf.edu> wrote:
    Dear list members,

    some of the concerns that have been raised in this discussion, such as
reviewer load and "incrementality" in papers, could be addressed if the
field moved to journal, rather than conference, publishing, and used
conferences for dissemination of ideas (where only abstracts would be
reviewed) and journals for actual publication. This would have the
positive side-effect of making the citation indices of computational
linguistics as a subfield go up, thus making it more visible in the
"scientific market". For a 1.5-page long elaboration of these ideas, see
http://www.lsi.upc.edu/~gboleda/pubs/gboleda_publishingCL.pdf

    Best,
    Gemma Boleda

    _______________________________________________
    UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
    Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora