peer reviewing

[Mark P. Line]

A. Katz wrote:
>
> But this problem that you've pointed out extends, it seems to me, beyond
> the issue of peer reviewing and directly into hiring, tenure, and
> everything that goes into deciding whether something has been
> "scientifically proven" or not.
>
> What can we, as a community of thinkers, do about it?

I have a technocratic answer to your question, but first the
non-technocratic part:

If you step back and take the meaning of "peer review" at face value
(instead of taking the conventional sense -- part of a scholarly journal's
quality control process), we find that peers review each other's work in
various ways. Serving as a reviewer for a journal is one of those ways.
Choosing to publish a rebuttal or commentary to a peer's paper is another
way. Evaluating a peer's papers to support hiring, tenure, promotion and
firing decisions is another way (you know you've been "reviewed" by your
peers if you don't get tenure).

Although publishing companies may be interested in quality control for
whatever business reasons they might have, the community of peers is
interested in quality control because it's synonymous with the scientific
method. I think Tom's and Esa's comments boil down to the observation that
the peer review process in the conventional sense is not really supporting
the community's need for quality control -- it doesn't really contribute
to the advancement of our science. Both would probably agree that it has
in fact worked against the advancement of our science in many cases.

We can fix that sad state of affairs by minimizing the impact of the peer
review process on the fine-grained content of what gets published.
Martin's suggestion would accomplish that, but it could only be
implemented if the publishing companies went along with the change in
policy. I might be wrong, but I think that's either unlikely, or else
likely to be a very long time in coming.

That's why I've suggested that the community might consider taking the
matter into its own capable hands -- which brings us to the technocratic
part of my comment:

I have to wonder what would happen if this community of thinkers went
ahead and built its own Web 2.0 platform for the submission, review and
dissemination of papers. "Peer review" could be an ongoing, potentially
never-ending process if submissions were published after relatively
minimal screening (Martin's accept/reject with no revise/resubmit),
perhaps by a rotating selection of peers. If successful, the platform
itself would become a research tool for the community.

I think we're all used to relegating these kinds of ideas to the sort of
futuristic pie-in-the-sky fiction you'd find in a Neal Stephenson novel.
But to a large extent, that future is already here. There are no remaining
technical barriers to this kind of thing, only ideological ones.

So if we built it, would they come (as the phrase goes)? More importantly,
would the community use the contents of such a platform in its hiring and
tenure decisions?


-- Mark

Mark P. Line



>
> --Aya
>
> http://hubpages.com/profile/Aya+Katz
>
>
> On Thu, 15 Apr 2010, Esa Itkonen wrote:
>
>> Just when I was about to participate at the 'peer reviewing' discussion,
>> Tom Givón sent in his contribution which made mine more or less
>> redundant. Still, here is a summary of some musings from those 42 years
>> that have elapsed since the publication of my first article (= 'Zur
>> Charakterisierung der Glossemantik')
>>
>> When (nearly) everybody agrees that A is the case, it seems less
>> interesting to echo this view and bolster it with more data, and more
>> interesting to try to find out if, after all, it is B that is the case,
>> and once having found it out, to prove it. Once you (or, rather, I) have
>> written an article in this spirit and offer it for publication, the
>> referees invariably respond by claiming that this just cannot be,
>> because (as everybody knows) A is the case.
>>
>> The end result has been that if (and when) my article has been
>> published, then (just as in Tom Givón's case) more often than not this
>> has been thanks to the editor of the journal in question, who has
>> quietly overruled the referees. (It has also happened that editors
>> privately solicit an article.) For a good measure, there has also been
>> the occasional editor (= clearly a man of strong convictions and/or
>> antipathies) who, overruling the referees, has rejected the article.
>>
>> In this discussion, there have been those who have confessed not to
>> understand Martin Haspelmath's original point. For me, this can only
>> mean that they are people intrinsically happy with the status quo, i.e.
>> people who claim 'A' when (nearly) everybody does so, and start claiming
>> 'B' only when nudged into doing so by the winds of change.
>>
>> Esa
>> .
>>
>> Homepage: http://users.utu.fi/eitkonen
>>
>>


-- Mark

Mark P. Line
Bartlesville, OK