[Corpora-List] Against the reviewer mediation stage

[Adam Kilgarriff]

Corpora readers,

Do any of you share my feeling about the 'review mediation phase'?

I do reviewing partly out of duty and partly because it's a way of making
sure I read closely at least one arbitrary subset of new work in my area  -
and sometimes I find out about really interesting work in this way.  I do
like the innovation of being able to bid for the papers you actively want to
review.

But an innovation I don't like is the 'review mediation process', as now
widely used by ACL and EMNLP where, if two reviewers disagree, they are
expected to contribute to a discussion where they see if they can reconcile
their differences.  The image is very nice - academics sitting down to sort
out their differences etc., but the reality is (for me) quite different.  I
reviewed the paper maybe three weeks ago and (at this frenetic time of year)
have probably reviewed half a dozen other papers between times.  To make a
considered comment, I need to take my time to re-acquaint myself with the
paper, remind myself of what I said in my review, give careful thought to
the other reviewers' comments, and work out how to respond, which involves
delicate processes (with both interpersonal and intellectual components) of
standing  up for my considered opinion while giving due heed to what others
have said (and being polite even if I think the other person's opinion is
rubbish - no anonymity here).  One good thing about initial reviewing is
that you can do it in your own time.  But that's not true for review
mediation, because there are only two or three days allocated to that
phase.  And here I am expected to devote as much time again to it as I did
to the original version, and there's nothing in it for me, as I've already
read it so I won't find any new ideas.

I think the reviewer mediation phase should be scrapped.  Either use maths
to merge reviewers' scores, or if the chair thinks that would not get a good
result in a particular case, let him/her read and decide.  That's his/her
job.

Adam

-- 
================================================
Adam Kilgarriff
http://www.kilgarriff.co.uk
Lexical Computing Ltd                   http://www.sketchengine.co.uk
Lexicography MasterClass Ltd      http://www.lexmasterclass.com
Universities of Leeds and Sussex       adam at lexmasterclass.com
================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20090529/ef8999c0/attachment.htm>
-------------- next part --------------
_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora