[Edling] Transparent Peer Review - Your Thoughts?

Thu Feb 4 01:48:03 UTC 2021

Thanks, Francis, for raising this important issue. On balance I share these concerns, but am also intrigued by the possibilities.

Reviewers sometimes are perfunctory, or they can't see beyond their personal hobby horse, or they deliver criticism in cutting and unconstructive terms. Might t an awareness that the broader community will read them result in better reviews?

Sometimes authors use review comments to improve the article substantially. Frequently they are expedient, make minimal superficial revisions, and get published simply by wearing down objections. Might authors be more inclined to engage fully with the peer review process if it is highly visible whether they did--or didn't?

We read, or should read, the literature critically, rather than assuming that anything which has made it past peer review is sound. Would an awareness of the issues others have raised in peer review enhance our ability to read an article critically?

Peer review is one of Swales' occluded genres. Would it be beneficial for our postgraduates to be able to see more of the process than just the outcome, i.e., the published article?

I'm not certain that these considerations outweigh the concerns which have been raised earlier. Transparent peer review may not be the solution for the problems which beset the process. However, in criticising it as an alternative, it's good to keep in mind that the currently dominant practice of double-blind peer review does have a multitude of flaws.

Thanks again for prompting an interesting discussion, Francis.

All the best,

Diane

Diane Pecorari
Professor of English
Head, Department of English
Acting Head, Chan Feng Men-ling Chan Shuk-lin Language Centre
City University of Hong Kong

From: Edling <edling-bounces at lists.mail.umbc.edu> On Behalf Of Francis M. Hult via Edling
Sent: Thursday, 4 February 2021 12:22 AM
To: edling at lists.mail.umbc.edu
Cc: Francis M. Hult <fmhult at umbc.edu>
Subject: Re: [Edling] Transparent Peer Review - Your Thoughts?

Dear Colleagues,

Several folks have responded off-list echoing similar concerns. A few folks have also mentioned the additional concern about the impact on the peer review pool.  Discomfort with having one's review published and attached permanently to someone's article (whether anonymous or not), they note, might result in more people declining to do reviews for journals that use transparent peer review.  Two particular points mentioned in this regard are (1) increasing the difficulty in finding peer reviewers which is already a challenge for many journal editors and (2) if more qualified reviewers decline, there is a risk of less qualified reviewers being used instead.

A few others have also written to share their general thoughts about transparent peer review reflecting growing trends towards neoliberalism and the commodification of knowledge work, and somewhat related global surveillance culture generally and in education in particular.  While transparency is ostensibly framed as rigor, from a critical perspective it is also indexed with mistrust and the use of surveillance as an antidote.  With surveillance come implications about who does the watching and the judging and the resulting power imbalances.  For example, how might transparent peer reviews be used in tenure and promotion processes?  Grant reviews?  Job searches.  Etc.

Thanks to those who shared their thoughts.  Additional comments are welcome on-list or off.

Best,
Francis

--
Francis M. Hult, PhD, FRGS | Professor
Department of Education
University of Maryland, Baltimore County (UMBC)

Editor, Educational Linguistics Book Series<https://www.springer.com/series/5894>
Co-Editor, Contributions to the Sociology of Language Book Series<https://www.degruyter.com/view/serial/16644>

Web Profile<https://education.umbc.edu/faculty-list/francis-m-hult/> | Academia.edu<http://umbc.academia.edu/FrancisMHult> | TESOL at UMBC<http://esol.umbc.edu/>

[https://docs.google.com/uc?export=download&id=1fisFF2r5HmuH2StP__rWJ0Um428jss1K&revid=0B3gZQJj4wrwIcGR6Yzc5a1VuQ1NGUExQZkJpL0ZJb2xrUkgwPQ]

On Fri, Jan 29, 2021 at 11:59 AM Francis M. Hult <fmhult at umbc.edu<mailto:fmhult at umbc.edu>> wrote:
Dear Colleagues,

There is an emerging practice in academic publishing known as "transparent peer review."  I encountered it recently for the first time in our field when reviewing for a journal by a major publisher (I won't name the journal since the editors have not yet responded to my queries about the practice).  In talking to colleagues about it, many had not heard about transparent peer review so I wanted to raise awareness about it and share some thoughts I have after reflecting on it.

If you are not familiar with transparent peer review, you can see an overview at these sites:
https://publons.com/benefits/publishers/transparent-review

https://authorservices.wiley.com/Reviewers/journal-reviewers/what-is-peer-review/transparent-peer-review.html

In brief, transparent peer review makes the full reports from peer reviewers public in a published format that is permanently connected to the published article so that readers can see both the article itself and all the peer review and editorial comments on it.

What strikes me about it are the wider sociocultural, sociopolitical, and psychological implications and consequences.  At a time when we are only getting started addressing the challenges of global inequities in academic publishing, it is a worrying development.  A number of journals in applied/educational linguistics have taken leading roles in improving equitable access to English-medium publishing.  I worry that if so-called transparent peer review were to gain footing in our field, it would lead us in the wrong direction.  It seems to me that the risks far outweigh any possible benefits.

The risks I see are the following, though there would likely be other unintended consequences as well:

(1) Many peer reviewers will self-censor their review texts because the readership of the review is public.  A review becomes, in effect, an online publication.  Accordingly, many reviewers will write for a public audience and not for the confidential audience of the editors and authors.  Authors, in turn, will be deprived of candid commentary that will help improve their work.

(2) Related to the point above, self-censorship among peer reviewers would do disproportionate harm to scholars in countries/contexts who have access to fewer academic resources, including reference materials as well as training in English academic writing.  An important role that peer reviewers play in fostering global equity and publication access is providing scholars from such contexts with thorough and constructive reviews.  Not meant for public consumption, such detailed reviews often provide extensive constructive feedback about literature review scope, research design, and data analysis because these authors need a deeper level of guidance than they might have received in other ways.  Such reviews are more so a work product of the writing process than mere evaluation of submission quality.  They often lead to extensive further development and revision where the final product ends up looking substantially different from an earlier submission.  When reviewers are not comfortable writing such deep and constructive reviews because they would be made public, scholars who need this support will not get it.  This, in turn, will result in favoring publications by scholars in academically privileged contexts.

(3) Some peer reviewers may write unvarnished peer reviews, whether anonymous or not, that can be damaging both psychologically and professionally to authors.  This is a particular concern with respect to junior scholars for whom the power relations with respect to (often more senior) peer reviewers is especially inequitable.  As experienced researchers, we know that there is great variation in the tone of peer review texts, and it is not uncommon to see blunt or strident statements or even mischaracterizations of methods and epistemologies.  Having such a peer review forever associated with one's published article can do major damage to a researcher's professional reputations and sense of self as a scholar.  It can be harmful for mid-career scholars too, but those who have already established their reputations and gained respect in the field would be able to weather a harmfully framed review while a junior scholar just starting their career might not.  It would be similarly more harmful to scholars from less academically privileged backgrounds for either unvarnished critiques or work-product-style reviews as noted in the point above to be made public.

(4) A further implication of concern about publishing peer review reports is the chilling effect it will have on researchers' willingness to submit papers.  There is already in many contexts around the world high stakes associated with publishing in respected international journals.  Having coached scholars at various points in their careers in a variety of countries, I have seen firsthand the psychological challenges that follow from navigating the peer review process.  Even receiving a confidential review can be threatening to junior scholars' sense of academic confidence, and it stifles their willingness to submit manuscripts to journals.  The psychological barrier that would follow from a peer review report being forever attached in public to a publication online would be paralyzing for many.  This would be especially true of scholars in many global contexts who already harbor insecurities about English-medium publication and submitting to international journals but are nonetheless under pressure from their institutions to do so.

In sum, so-called "transparent" peer review poses many risks and ethical dilemmas with which I am greatly troubled.  I wonder if others have also encountered it in our field and what your experiences and thoughts are about it.

Best,
Francis

--
Francis M. Hult, PhD, FRGS | Professor
Department of Education
University of Maryland, Baltimore County (UMBC)

Editor, Educational Linguistics Book Series<https://www.springer.com/series/5894>
Co-Editor, Contributions to the Sociology of Language Book Series<https://www.degruyter.com/view/serial/16644>

Web Profile<https://education.umbc.edu/faculty-list/francis-m-hult/> | Academia.edu<http://umbc.academia.edu/FrancisMHult> | TESOL at UMBC<http://esol.umbc.edu/>

[https://docs.google.com/uc?export=download&id=1fisFF2r5HmuH2StP__rWJ0Um428jss1K&revid=0B3gZQJj4wrwIcGR6Yzc5a1VuQ1NGUExQZkJpL0ZJb2xrUkgwPQ]

Disclaimer: This email (including any attachments) is for the use of the intended recipient only and may contain confidential information and/or copyright material. If you are not the intended recipient, please notify the sender immediately and delete this email and all copies from your system. Any unauthorized use, disclosure, reproduction, copying, distribution, or other form of unauthorized dissemination of the contents is expressly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/edling/attachments/20210204/c78c3e2d/attachment.htm>
-------------- next part --------------

_______________________________________________
Edling mailing list
Edling at lists.mail.umbc.edu
https://lists.mail.umbc.edu/mailman/listinfo/edling