[Edling] Transparent Peer Review - Your Thoughts?

Wed Feb 3 16:22:23 UTC 2021

Dear Colleagues,

Several folks have responded off-list echoing similar concerns. A few folks
have also mentioned the additional concern about the impact on the peer
review pool.  Discomfort with having one's review published and attached
permanently to someone's article (whether anonymous or not), they note,
might result in more people declining to do reviews for journals that use
transparent peer review.  Two particular points mentioned in this regard
are (1) increasing the difficulty in finding peer reviewers which is
already a challenge for many journal editors and (2) if more qualified
reviewers decline, there is a risk of less qualified reviewers being used
instead.

A few others have also written to share their general thoughts about
transparent peer review reflecting growing trends towards neoliberalism and
the commodification of knowledge work, and somewhat related global
surveillance culture generally and in education in particular.  While
transparency is ostensibly framed as rigor, from a critical perspective it
is also indexed with mistrust and the use of surveillance as an antidote.
With surveillance come implications about who does the watching and the
judging and the resulting power imbalances.  For example, how might
transparent peer reviews be used in tenure and promotion processes?  Grant
reviews?  Job searches.  Etc.

Thanks to those who shared their thoughts.  Additional comments are welcome
on-list or off.

Best,
Francis

--
*Francis M. Hult, PhD, FRGS* | Professor
Department of Education
University of Maryland, Baltimore County (UMBC)

Editor, Educational Linguistics Book Series
<https://www.springer.com/series/5894>
Co-Editor, Contributions to the Sociology of Language Book Series
<https://www.degruyter.com/view/serial/16644>

Web Profile <https://education.umbc.edu/faculty-list/francis-m-hult/> |
Academia.edu <http://umbc.academia.edu/FrancisMHult> | TESOL at UMBC
<http://esol.umbc.edu/>

On Fri, Jan 29, 2021 at 11:59 AM Francis M. Hult <fmhult at umbc.edu> wrote:

> Dear Colleagues,
>
> There is an emerging practice in academic publishing known as "transparent
> peer review."  I encountered it recently for the first time in our field
> when reviewing for a journal by a major publisher (I won't name the journal
> since the editors have not yet responded to my queries about the
> practice).  In talking to colleagues about it, many had not heard about
> transparent peer review so I wanted to raise awareness about it and share
> some thoughts I have after reflecting on it.
>
> If you are not familiar with transparent peer review, you can see an
> overview at these sites:
> https://publons.com/benefits/publishers/transparent-review
>
>
> https://authorservices.wiley.com/Reviewers/journal-reviewers/what-is-peer-review/transparent-peer-review.html
>
> In brief, transparent peer review makes the full reports from peer
> reviewers public in a published format that is permanently connected to the
> published article so that readers can see both the article itself and all
> the peer review and editorial comments on it.
>
> What strikes me about it are the wider sociocultural, sociopolitical, and
> psychological implications and consequences.  At a time when we are only
> getting started addressing the challenges of global inequities in academic
> publishing, it is a worrying development.  A number of journals in
> applied/educational linguistics have taken leading roles in improving
> equitable access to English-medium publishing.  I worry that if so-called
> transparent peer review were to gain footing in our field, it would lead us
> in the wrong direction.  It seems to me that the risks far outweigh any
> possible benefits.
>
> The risks I see are the following, though there would likely be other
> unintended consequences as well:
>
> (1) Many peer reviewers will self-censor their review texts because the
> readership of the review is public.  A review becomes, in effect, an online
> publication.  Accordingly, many reviewers will write for a public audience
> and not for the confidential audience of the editors and authors.  Authors,
> in turn, will be deprived of candid commentary that will help improve their
> work.
>
> (2) Related to the point above, self-censorship among peer reviewers would
> do disproportionate harm to scholars in countries/contexts who have access
> to fewer academic resources, including reference materials as well as
> training in English academic writing.  An important role that peer
> reviewers play in fostering global equity and publication access is
> providing scholars from such contexts with thorough and constructive
> reviews.  Not meant for public consumption, such detailed reviews often
> provide extensive constructive feedback about literature review scope,
> research design, and data analysis because these authors need a deeper
> level of guidance than they might have received in other ways.  Such
> reviews are more so a work product of the writing process than mere
> evaluation of submission quality.  They often lead to extensive further
> development and revision where the final product ends up looking
> substantially different from an earlier submission.  When reviewers are not
> comfortable writing such deep and constructive reviews because they would
> be made public, scholars who need this support will not get it.  This, in
> turn, will result in favoring publications by scholars in academically
> privileged contexts.
>
> (3) Some peer reviewers may write unvarnished peer reviews, whether
> anonymous or not, that can be damaging both psychologically and
> professionally to authors.  This is a particular concern with respect to
> junior scholars for whom the power relations with respect to (often more
> senior) peer reviewers is especially inequitable.  As experienced
> researchers, we know that there is great variation in the tone of peer
> review texts, and it is not uncommon to see blunt or strident statements or
> even mischaracterizations of methods and epistemologies.  Having such a
> peer review forever associated with one's published article can do major
> damage to a researcher's professional reputations and sense of self as a
> scholar.  It can be harmful for mid-career scholars too, but those who have
> already established their reputations and gained respect in the field would
> be able to weather a harmfully framed review while a junior scholar just
> starting their career might not.  It would be similarly more harmful to
> scholars from less academically privileged backgrounds for either
> unvarnished critiques or work-product-style reviews as noted in the point
> above to be made public.
>
> (4) A further implication of concern about publishing peer review reports
> is the chilling effect it will have on researchers' willingness to submit
> papers.  There is already in many contexts around the world high stakes
> associated with publishing in respected international journals.  Having
> coached scholars at various points in their careers in a variety of
> countries, I have seen firsthand the psychological challenges that follow
> from navigating the peer review process.  Even receiving a confidential
> review can be threatening to junior scholars' sense of academic confidence,
> and it stifles their willingness to submit manuscripts to journals.  The
> psychological barrier that would follow from a peer review report being
> forever attached in public to a publication online would be paralyzing for
> many.  This would be especially true of scholars in many global contexts
> who already harbor insecurities about English-medium publication and
> submitting to international journals but are nonetheless under pressure
> from their institutions to do so.
>
> In sum, so-called "transparent" peer review poses many risks and ethical
> dilemmas with which I am greatly troubled.  I wonder if others have also
> encountered it in our field and what your experiences and thoughts are
> about it.
>
> Best,
> Francis
>
> --
> *Francis M. Hult, PhD, FRGS* | Professor
> Department of Education
> University of Maryland, Baltimore County (UMBC)
>
> Editor, Educational Linguistics Book Series
> <https://www.springer.com/series/5894>
> Co-Editor, Contributions to the Sociology of Language Book Series
> <https://www.degruyter.com/view/serial/16644>
>
> Web Profile <https://education.umbc.edu/faculty-list/francis-m-hult/> |
> Academia.edu <http://umbc.academia.edu/FrancisMHult> | TESOL at UMBC
> <http://esol.umbc.edu/>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/edling/attachments/20210203/c5dbbe48/attachment.htm>
-------------- next part --------------

_______________________________________________
Edling mailing list
Edling at lists.mail.umbc.edu
https://lists.mail.umbc.edu/mailman/listinfo/edling