[Corpora-List] Corpora Digest, Vol 52, Issue 3 [was blind reviewing]

[Graeme Hirst]

On 12 Oct 2011, at 10:50, Alon Lischinsky wrote:

> Some papers use multiple reviewers. Others don't. And even those that
> require multiple reviews do not (a) allow the different reviewers to
> discuss the case and seek a common opinion, as happens in jury trials;
> (b) require enough reviews to ensure that personal biases regress to
> the mean.

Well, right here in our own field, the CL journal uses at least three reviewers, sometimes four, and after the individual reviews are in, there is a reviewer discussion period.  Most ACL conferences also have reviewer discussion periods, often with prompts from area chairs (who act like editors here) to help achieve either consensus or at least principled disagreement.  In addition, some ACL conferences have been experimenting with allowing authors to respond to the reviewers before the final decision is made.


> In my experience (and this is, of course, anecdotal evidence, but I
> don't know of any systematic studies), the editor rarely has anything
> to do with the review process (beyond, perhaps, an initial screening
> of obviously rejectable papers). The decision of the reviewer(s) is
> what determines acceptance. And I haven't found a single journal that
> encourages authors to challenge the reviewers' decisions, or even
> two-way communication during the review process.

Well, right here in our own field, it's clear that the editor of the CL journal reads the reviews but comes to his own carefully considered final decision.

Obviously, no journal or conference would encourage an author to "challenge" (as opposed to "respond to") reviewers.  However, FWIW (anecdotal evidence), I did once challenge some journal reviews that I thought were nit-picking and out of line, and was successful in getting the decision reversed.

--
::::  Graeme Hirst
::::  University of Toronto * Department of Computer Science




_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora