As I have made many comments on this thread, I will clearly state my position. In most contexts, I would prefer the three kinds of reviewing systems as below (in decreasing order of preference, i.e., the first one being the most preferable):<div>
<br></div><div>1. Open review system: The reviewer and the author both know each other's identity</div><div>2. Single blind system: Reviewer knows who the author is, but the author doesn't know who the reviewer is</div>
<div>3. Double blind system: Neither the author nor the reviewer knows the other's identity</div><div><br></div><div>I have tried to give arguments in support of my position. I will skip repeating them (though I like repetition as a stylistic device).</div>
<div><br></div><div>I think it's a mistake to think that system-1 is not practical at all. The fact is, most of the peer reviews in the world out there work on this very basis. Consider book reviews in the press, in journals, wherever. Even a few scientific journals/conferences are using open peer review. The evidence, so far as it goes, scientifically speaking, is not conclusive either way. It is not even very heavily in favour of the one (blind) or the other (open).</div>
<div><br></div><div>System-2 is better than system-3, primarily because of the problems in and impracticality of hiding one's identity (often needs undigified gymnastic, as Yorick expressed it: And it's not simply a matter of making it "as Smith (2008) showed").</div>
<div><br></div><div>The only strong argument in favour of system-3 (based on my knowledge and experienced) is the possibility of retaliation. I would agree that the chances of that happening would be higher in system-1 than in system-2 or system-3, but I am pretty sure that it can happen even in these two systems. The reason is that often the author can guess who the reviewer was. But with system-1 the process is transparent, so that if some such thing happens, the person at the receiving end can at least say that this has happened and possibly seek some sort of remedy. If those who retaliate for such reasons know that their actions (which are very seriously unethical and immoral, and I don't see why they shouldn't be given as much consideration as issues like plagiarism) can be brought to public attention, they might think twice before indulging in them. As it happens, with blind reviewing (systems 2 and 3), the person (the reviewer) at the receiving end can't even say that any such thing has happened, because the immediate dismissive answer would be that "our system ensures that the identities are not revealed" and you have run into a wall. A similar thing applies when a reviewer unfairly rejects a paper (the standard method now seems to be to ensure the paper is killed, to borrow Church's memorable words, by giving it a score of 1, irrespective of anything in the paper and sometimes even irrespective of the same reviewer's comments for the same paper) because he has guessed the author's identity or has narrowed it down sufficiently. Here I will have to repeat that this can happen because of various kinds of prejudices (e.g. the formalism/paradigm/framework that enrages him, to give an example of a purely technical kind of prejudice, even though the issue of the validity of that formalism/paradigm/framework is still very much open).</div>
<div><br></div><div><br><div class="gmail_quote">On Wed, Oct 12, 2011 at 10:18 PM, Laurence Anthony <span dir="ltr"><<a href="mailto:anthony0122@gmail.com">anthony0122@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><div class="gmail_quote"><div class="im">On Thu, Oct 13, 2011 at 1:23 AM, Yorick Wilks <span dir="ltr"><<a href="mailto:Y.Wilks@dcs.shef.ac.uk" target="_blank">Y.Wilks@dcs.shef.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Thanks, I remember the details. The discussion has gone many ways, some of them arguing the (de)merits of author-blind --as well as reviewer-blind ---systems. The starting point was LREC and the author-blind system. Much later, you wrote, after I used the phrase "both systems":<div>
".....what exactly is the alternative system to blind reviewing that is being referred to in the phrase "both systems". Obviously, "against blind reviewing" is not a system in itself. Am I correct in assuming that the 'alternative system' being proposed on this list is simply an open one where both reviewers and authors know each others' names? "</div>
<div>My "both systems" referred, as I thought was clear in the context I wrote it, to author-blind and non-blind systems---ACL being like the former and LREC the latter (COLING has oscillated, if memory serves). So no, the opposites are those just listed. Does that clear it up?</div>
<div>YW</div><div><br></div></div></blockquote><div><br></div></div>Sorry, I'm still confused. I think ACL uses a double-blind system (authors and reviewers don't know who the other is). See here:<div><a href="http://www.aclweb.org/archive/policies/current/program-committee-guide.html" target="_blank">http://www.aclweb.org/archive/policies/current/program-committee-guide.html</a> </div>
<div><br></div><div>LREC uses an single-blind system (the reviewer knows the author but the author doesn't know the reviewer). See here:</div><div><a href="http://www.lrec-conf.org/lrec2012/?Abstract-for-Oral-or-Poster" target="_blank">http://www.lrec-conf.org/lrec2012/?Abstract-for-Oral-or-Poster</a></div>
<div><br></div><div>In view of earlier comments about reviewers needing to reveal their identity, neither ACL nor LREC adopt such a policy. In fact, the LREC policy in effect gives even more power to the reviewer than a double-blind policy. Is this what you were supporting when you wrote, <span style="color:rgb(34, 34, 34);font-family:arial, sans-serif;font-size:13px;background-color:rgb(255, 255, 255)"> "The whole blind-review business is a huge nonsense...LRECs reputation has grown steadily and it will be the quality of its papers that sustain it--there is no evidence at all anonymity would improve matters in the least. if it ain't broke........"</span></div>
<div><span style="color:rgb(34, 34, 34);font-family:arial, sans-serif;font-size:13px;background-color:rgb(255, 255, 255)"><br></span></div><div><span style="color:rgb(34, 34, 34);font-family:arial, sans-serif;font-size:13px;background-color:rgb(255, 255, 255)">Laurence.</span></div>
<div><br></div><div>(p.s. If it's just me that's confused, feel free to ignore me!)</div><div><br></div><div><br></div><div><br></div><div><br></div></div></div>
<br>_______________________________________________<br>
UNSUBSCRIBE from this page: <a href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br>
<a href="mailto:Corpora@uib.no">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br></blockquote></div><br></div>