[Corpora-List] Against the reviewer mediation stage

[Martin Reynaert]

Dear German,

As a student of lapsology (Carl James), I would appreciate if you would 
clarify what you intended to write: 'thread', 'threat' or 'treat'. 
Contextually I would think the first, pragmatically I guess the other 
two might be possible ;0)

Cheers,

Martin Reynaert
ILK
TiCC
Tilburg University


German Rigau wrote:
> Hi Adam,
> 
> Thanks for opening this new conversation threat ... ;-)
> 
> I also believe that the "review mediation phase" is improving the 
> overall quality of the reviewing process. Obviously, this process can 
> always be improved (more time, need to reach a consensus, different 
> scoring schemas among area-chairs, etc.)
> 
> However, it would be nice to see the real effect of this phase with 
> respect the original scoring. Only a few changes? Many changes? Changes 
> not in the scoring but on the reviews? ... Where is the effect of this 
> phase reported?
> 
> Best,
> 
> German
> 
> 
> Diana Santos wrote:
> 
>>Sorry Adam, not only I do not share it, as I am an enthusiast about this.
>> 
>>This is the only effective way to prevent people doing dishonest, 
>>careless or uninformed reviews and getting away with it.
>> 
>>Except if one accepts the principle of Signed Reviews 
>>(http://www.linguateca.pt/Diana/SignedReviews.html), but this may have 
>>other consequences.
>> 
>>I suggest you read
>>Chubin, D. R. & E. J. Hackett. /Peerless Science, Peer Review and U.S. 
>>Science Policy/. New York, State University of New York Press. 1990.
>>for a debate and some suggestions.
>>For those of you who read Portuguese, I have a page on these issues as 
>>well, with some further references: 
>>http://www.linguateca.pt/Diana/avalpubl.html
>>Best,
>>Diana
>> 
>> 
>>
>>    ------------------------------------------------------------------------
>>    *From:* corpora-bounces at uib.no [mailto:corpora-bounces at uib.no] *On
>>    Behalf Of *Adam Kilgarriff
>>    *Sent:* 29. mai 2009 10:47
>>    *To:* corpora at uib.no
>>    *Subject:* [Corpora-List] Against the reviewer mediation stage
>>
>>    Corpora readers,
>>     
>>    Do any of you share my feeling about the 'review mediation phase'?
>>     
>>    I do reviewing partly out of duty and partly because it's a way of
>>    making sure I read closely at least one arbitrary subset of new
>>    work in my area  - and sometimes I find out about really
>>    interesting work in this way.  I do like the innovation of being
>>    able to bid for the papers you actively want to review.
>>     
>>    But an innovation I don't like is the 'review mediation process',
>>    as now widely used by ACL and EMNLP where, if two reviewers
>>    disagree, they are expected to contribute to a discussion where
>>    they see if they can reconcile their differences.  The image is
>>    very nice - academics sitting down to sort out their differences
>>    etc., but the reality is (for me) quite different.  I reviewed the
>>    paper maybe three weeks ago and (at this frenetic time of year)
>>    have probably reviewed half a dozen other papers between times. 
>>    To make a considered comment, I need to take my time to
>>    re-acquaint myself with the paper, remind myself of what I said in
>>    my review, give careful thought to the other reviewers' comments,
>>    and work out how to respond, which involves delicate processes
>>    (with both interpersonal and intellectual components) of standing 
>>    up for my considered opinion while giving due heed to what others
>>    have said (and being polite even if I think the other person's
>>    opinion is rubbish - no anonymity here).  One good thing about
>>    initial reviewing is that you can do it in your own time.  But
>>    that's not true for review mediation, because there are only two
>>    or three days allocated to that phase.  And here I am expected to
>>    devote as much time again to it as I did to the original version,
>>    and there's nothing in it for me, as I've already read it so I
>>    won't find any new ideas.
>>     
>>    I think the reviewer mediation phase should be scrapped.  Either
>>    use maths to merge reviewers' scores, or if the chair thinks that
>>    would not get a good result in a particular case, let him/her read
>>    and decide.  That's his/her job.
>>     
>>    Adam
>>
>>    -- 
>>    ================================================
>>    Adam Kilgarriff                                    
>>     http://www.kilgarriff.co.uk              
>>    Lexical Computing Ltd                   http://www.sketchengine.co.uk
>>    Lexicography MasterClass Ltd      http://www.lexmasterclass.com
>>    Universities of Leeds and Sussex       adam at lexmasterclass.com
>>    <mailto:adam at lexmasterclass.com>
>>    ================================================
>>
>>------------------------------------------------------------------------
>>
>>_______________________________________________
>>Corpora mailing list
>>Corpora at uib.no
>>http://mailman.uib.no/listinfo/corpora
>>  
> 
> 
> 
> _______________________________________________
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
> 


_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora