[Corpora-List] Blind reviewing

[Anil Singh]

As I have made many comments on this thread, I will clearly state my
position. In most contexts, I would prefer the three kinds of reviewing
systems as below (in decreasing order of preference, i.e., the first one
being the most preferable):

1. Open review system: The reviewer and the author both know each other's
identity
2. Single blind system: Reviewer knows who the author is, but the author
doesn't know who the reviewer is
3. Double blind system: Neither the author nor the reviewer knows the
other's identity

I have tried to give arguments in support of my position. I will skip
repeating them (though I like repetition as a stylistic device).

I think it's a mistake to think that system-1 is not practical at all. The
fact is, most of the peer reviews in the world out there work on this very
basis. Consider book reviews in the press, in journals, wherever. Even a few
scientific journals/conferences are using open peer review. The evidence, so
far as it goes, scientifically speaking, is not conclusive either way. It is
not even very heavily in favour of the one (blind) or the other (open).

System-2 is better than system-3, primarily because of the problems in and
impracticality of hiding one's identity (often needs undigified gymnastic,
as Yorick expressed it: And it's not simply a matter of making it "as Smith
(2008) showed").

The only strong argument in favour of system-3 (based on my knowledge and
experienced) is the possibility of retaliation. I would agree that the
chances of that happening would be higher in system-1 than in system-2 or
system-3, but I am pretty sure that it can happen even in these two systems.
The reason is that often the author can guess who the reviewer was. But with
system-1 the process is transparent, so that if some such thing happens, the
person at the receiving end can at least say that this has happened and
possibly seek some sort of remedy. If those who retaliate for such reasons
know that their actions (which are very seriously unethical and immoral, and
I don't see why they shouldn't be given as much consideration as issues like
plagiarism) can be brought to public attention, they might think twice
before indulging in them. As it happens, with blind reviewing (systems 2 and
3), the person (the reviewer) at the receiving end can't even say that any
such thing has happened, because the immediate dismissive answer would be
that "our system ensures that the identities are not revealed" and you have
run into a wall. A similar thing applies when a reviewer unfairly rejects a
paper (the standard method now seems to be to ensure the paper is killed, to
borrow Church's memorable words, by giving it a score of 1, irrespective of
anything in the paper and sometimes even irrespective of the same reviewer's
comments for the same paper) because he has guessed the author's identity or
has narrowed it down sufficiently. Here I will have to repeat that this can
happen because of various kinds of prejudices (e.g. the
formalism/paradigm/framework that enrages him, to give an example of a
purely technical kind of prejudice, even though the issue of the validity of
that formalism/paradigm/framework is still very much open).


On Wed, Oct 12, 2011 at 10:18 PM, Laurence Anthony <anthony0122 at gmail.com>wrote:

> On Thu, Oct 13, 2011 at 1:23 AM, Yorick Wilks <Y.Wilks at dcs.shef.ac.uk>wrote:
>
>> Thanks, I remember the details. The discussion has gone many ways, some of
>> them arguing the (de)merits of  author-blind --as well as reviewer-blind
>> ---systems. The starting point was LREC and the author-blind system.  Much
>> later, you wrote, after I used the phrase "both systems":
>> ".....what exactly is the alternative system to blind reviewing that is
>> being referred to in the phrase "both systems". Obviously, "against blind
>> reviewing" is not a system in itself. Am I correct in assuming that the
>> 'alternative system' being proposed on this list is simply an open one where
>> both reviewers and authors know each others' names? "
>> My "both systems" referred, as I thought was clear in the context I wrote
>> it, to author-blind and non-blind systems---ACL being like the former and
>> LREC the latter (COLING has oscillated, if memory serves). So no, the
>> opposites are those just listed. Does that clear it up?
>> YW
>>
>>
> Sorry, I'm still confused. I think ACL uses a double-blind system (authors
> and reviewers don't know who the other is). See here:
> http://www.aclweb.org/archive/policies/current/program-committee-guide.html
>
>
> LREC uses an single-blind system (the reviewer knows the author but the
> author doesn't know the reviewer). See here:
> http://www.lrec-conf.org/lrec2012/?Abstract-for-Oral-or-Poster
>
> In view of earlier comments about reviewers needing to reveal their
> identity, neither ACL nor LREC adopt such a policy. In fact, the LREC policy
> in effect gives even more power to the reviewer than a double-blind policy.
> Is this what you were supporting when you wrote,  "The whole blind-review
> business is a huge nonsense...LRECs reputation has grown steadily and it
> will be the quality of its papers that sustain it--there is no evidence at
> all anonymity would improve matters in the least. if it ain't broke........"
>
> Laurence.
>
> (p.s. If it's just me that's confused, feel free to ignore me!)
>
>
>
>
>
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20111012/fdc0c177/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora