peer review: selecting and helping vs. shaping

[Martin Haspelmath]

I agree that "most editors and referees are competent and reasonable", 
but I am suggesting that the system can be improved. In particular, I 
think that "revise and resubmit" isn't very helpful, and I suspect that 
it is often used when an editor is undecided, not only when an editor is 
convinced that the paper can and should be improved along the lines 
proposed by the reviewers.

Let's assume the following figures for an average linguistics journal 
(if they are way off, forget the rest of the message):

20%: "accepted (with recommended improvements)"
50%: "revise and resubmit (R&R)"
30%: "rejected"

Now of those 50% with an R&R decision, let's assume that about two 
thirds are resubmitted, and that most of these are eventually accepted, i.e.

15%: not resubmitted
30%: accepted after rewriting
5%: rejected even after resubmission

(This would thus lead to a final acceptance rate of 50% (20% 
immediately, 30% after rewriting), and an effective rejection rate of 50%.)

Now compare this with a new system without R&R, where 50% are accepted 
with recommended improvements, and 50% are rejected without invitation 
to resubmit. It seems to me that everyone is better off in the new system:

-- especially the 5% rejected even after rewriting
-- also the 15% who don't resubmit, because many of the authors will 
spend a lot of time considering the option of resubmitting to the same 
journal or trying a different journal
-- and also the 30% accepted after rewriting, because their papers come 
out with a considerable delay (often a year or more), and they come out 
with features that the authors aren't really happy with
-- the editors and reviewers, because they have less work
-- the field as a whole, because journal papers appear more quickly

The only advantage of R&R that I can see is that most of the 30% of 
articles that are accepted after rewriting will be more to the 
reviewers' liking. But is that an advantage for science? Two or three 
individuals can never be representative of the field as a whole. I think 
that peer reviewers have very different roles from professors guiding 
immature students. They can suggest improvements and different 
directions to their colleagues, but it should not be their role to guide 
them.

Martin

Daniel L. Everett schrieb:
> I agree, Lise. I am still not quite understanding the problem with 'revise and resubmit'. Seems like a perfectly sensible recommendation.
>
> I have occasionally decided to submit papers elsewhere because I disagreed with the recommendations of the reviewers. There are plenty of journals, after all (though this doesn't completely rule out the possibility that the same reviewer might get the same ms in succession from more than one journal).
>
> And on at least two occasions that I can remember, an article of mine that received a very negative review was nevertheless still published in the journal - without revision -  (in one case in the most competitive theoretical journal at that time) because the editor thought that the referee had blown it. The editor said explicitly that he was setting aside the referee report (the paper fell within his area of expertise).
>
> My own impression is that most editors and referees are competent and reasonable and that the process still works well.
>
> But I also think that there are times when self-publishing can make more sense. Rarely. But not never.
>
> What am I missing, Martin?
>
> Dan
>
>
>
> On Mar 31, 2010, at 2:42 PM, Lise Menn wrote:
>
>   
>> I'm not sure about that, Martin.  The author always has the option of seeking another journal if s/he gets a 'revise/resubmit'. I've been on both sides of that recommendation, and in all of my cases, the problem was not a matter of the theory, but of how much a single paper could accomplish and/or of needing restructuring in order to make a coherent argument.
>> 	Lise
>>
>> On Mar 31, 2010, at 3:32 AM, Martin Haspelmath wrote:
>>
>>     
>>> Yes, peer review often has the effect of improving a paper, but in my experience, it is equally often the case that a paper changes in the direction desired by the reviewers, without really getting better. The author wants to publish the paper in the journal, so she goes out of her way to please the reviewers.
>>>
>>> I think this latter outcome, which is really unfortunate, could be avoided by giving authors just one of two decisions: "accept with recommended revisions" or "reject".
>>>
>>> If the paper is accepted with recommended revisions, the author can then make use of those suggestions from the reviewers that he finds helpful, while ignoring those that would lead into directions he doesn't want to take.
>>>
>>> So if we eliminate "revise and resubmit", we would retain the positive effects of peer review, while getting rid of the negative effects that arise from reviewers who feel they want to shape a paper. The task of reviewers should be to help authors improve the paper, and to advise the editor on which papers to select for publication. Their task should not be to shape the paper.
>>>
>>> Martin Haspelmath
>>>
>>>
>>>       
>
>