[Corpora-List] Corpora Digest, Vol 52, Issue 3 [was blind reviewing]

[Laurence Anthony]

Mon, Oct 10, 2011 at 10:20 PM, Alon Lischinsky <
alon.lischinsky at kultmed.umu.se> wrote:

>
> > As you say,  "it's not easy to decide whether [a blind practice] is
> > preferable".  However, in these cases, don't we normally look to the
> > lessons we get from history? Throughout history, society has
> > implemented various methods to avoid biases in decision making
> > systems, and 'blind' practices seem to be much more prevalent now.
>
> In some respects, probably. For most of history, blindness was simply
> not practicable because of the small world phenomenon.
>
> However, it seems to me that blind procedures tend to be favoured to
> protect individuals in situations where no single one exerts
> significant power, as in your example of voting systems. In this case,
> individual-level bias is of little consequence. The goal of the blind
> procedure is mainly to protect individuals from ex-post retaliation.
>

I agree that blind procedures protect individuals from ex-post retaliation.
I think this is the most important reason why it is used in voting systems,
regardless of who has power.


> However, where individual decision-makers have significant power and
> their clients are not usually in a position to retaliate (as is the
> case with reviewers and authors), transparency and public oversight
> can offer more substantial advantages than blinding.


Really? Why would you say that the reviewers' 'clients' are not usually in a
position to retaliate? I review many papers and I'm sure many of my
'clients' are more well-known and influential than I am.  (I don't know
because the system is blind!) If they know that I had rejected their paper,
and assuming they also review papers, why *couldn't* they decide to start
rejecting my papers. (This is under the assumption that neither side is
blind).


> You mention peer
> juries as an example; they are a somewhat peculiar one, inasmuch as
> jury decisions are supposed to be unanimous, and hung juries result in
> mistrial, but more important is that they don't much resemble peer
> reviews.


Many juries in many countries can give a verdict without the decision
being unanimous.

The reviewing process is, if anything, more similar to a
> bench trial, where a single person is in charge of all decisions, and
> no checks exist to limit their discretionary judgement.


Why? In the reviewing process, multiple people give reviews on a paper.  If
you are trying to argue that a single person makes all the decisions, that
would be the Editor, who the author will know directly.


> And in all
> forms of bench trial I know, magistrates are required to identify
> themselves, provisions are made for their eventual recusal, and
> tallies of the votes in collegiate bodies (i.e., SCOTUS) identify by
> name who subscribed to each opinion.
>

Not sure about all this. But, the Editor (=magistrate?) is certainly known
to the author.


>
> I don't think blind review is always a bad idea; far from it, evidence
> such as that quoted by Adam suggests it can be quite beneficial in
> many cases. However, it also has disadvantages, and I don't think they
> should be ignored just because it's what everyone else is doing.
>
>
I don't think the supporters of blind-reviewing are ignoring the problems.
As I wrote before, I just think it is strange that people are proposing
dropping the blind-review process altogether just because of its
limitations.

Saying that, it's good to have discussions like this so we can reflect on
our practices.

Laurence.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20111011/66b38463/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora