35.387, FYI: AI in peer review: Bibliography and reflections on its use in linguistics

Sat Feb 3 19:05:05 UTC 2024

LINGUIST List: Vol-35-387. Sat Feb 03 2024. ISSN: 1069 - 4875.

Subject: 35.387, FYI: AI in peer review: Bibliography and reflections on its use in linguistics

Moderators: Malgorzata E. Cavar, Francis Tyers (linguist at linguistlist.org)
Managing Editor: Justin Fuller
Team: Helen Aristar-Dry, Steven Franks, Everett Green, Daniel Swanson, Maria Lucero Guillen Puon, Zackary Leech, Lynzie Coburn, Natasha Singh, Erin Steitz
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org

Homepage: http://linguistlist.org

Please support the LL editors and operation with a donation at:
           https://funddrive.linguistlist.org/donate/

Editor for this issue: Justin Fuller <justin at linguistlist.org>
================================================================

Date: 01-Feb-2024
From: Nicholas Lo Vecchio [nlovecchio at gmail.com]
Subject: AI in peer review: Bibliography and reflections on its use in linguistics

The rapid emergence and evolution of generative AI has led to urgent
debates particularly within the sciences about the use of AI in
scholarly peer review. For a thoughtful synthesis, see primarily
Hosseini and Horbach (2023). A dedicated debate on AI use in peer
review is worth having for linguistics, overlapping as it does the
humanities, social sciences, and (for some) natural sciences.

As an independent language researcher whose work is firmly humanities
focused (queer lexicon), I have had some recent experiences with the
matter. I have begun to post on my website on this topic:

www.nicospage.eu/blog

>From my recent searching, I have compiled a list of some relevant
publications on the topic. (So far, I only saw pertinent papers in
English.) Most of the below deal with the sciences:
- Checco, Alessandro, et al. 2021. “AI-Assisted Peer Review.”
Humanities and Social Sciences Communications 8, no. 25.
<https://doi.org/10.1057/s41599-020-00703-8>.
- Dergaa, Ismail, et al. 2023. “Enhancing Scholarly Discourse in the
Age of Artificial Intelligence: A Guided Approach to Effective Peer
Review Process.” La Tunisie médicale 101, no. 10, 721-726.
- Donker, Tjibbe. 2023. “The Dangers of Using Large Language Models
for Peer Review.” Letter to The Lancet Infectious Diseases 23, no. 7,
July 2023; and supplementary appendix.
<https://doi.org/10.1016/S1473-3099(23)00290-6>.
- Hosseini, Mohammad / Horbach, Serge P.J.M. 2023. “Fighting Reviewer
Fatigue or Amplifying Bias? Considerations and Recommendations for Use
of ChatGPT and Other Large Language Models in Scholarly Peer Review.”
Research Integrity and Peer Review 8, no. 4.
<https://doi.org/10.1186/s41073-023-00133-5>.
- Kaebnick, Gregory, et al. 2023. “Editors’ Statement on the
Responsible Use of Generative AI Technologies in Scholarly Journal
Publishing.” Hastings Center Report 53, no. 5.
<https://doi.org/10.1002/hast.1507>.
- Kousha, Kayvan / Thelwall, Mike. 2024 [2023]. “Artificial
Intelligence to Support Publishing and Peer Review: A Summary and
Review.” Learned Publishing 37, no. 1.
<https://doi.org/10.1002/leap.1570>.
- Leung, Tiffany, et al. 2023. “Best Practices for Using AI Tools as
an Author, Peer Reviewer, or Editor.” Journal of Medical Internet
Research 25, e51584. <https://doi.org/10.2196/51584>.
- Liang, Weixin, et al. 2023. “Can Large Language Models Provide
Useful Feedback on Research Papers? A Large-Scale Empirical Analysis.”
arXiv, 3 October 2023. <arXiv:2310.01783v1>.
- Nath, Karl, et al. 2024. “AI in Peer Review: Publishing’s Panacea or
a Pandora’s Box of Problems?” Mayo Clinic Proceedings 99, no. 1.
<https://doi.org/10.1016/j.mayocp.2023.11.013>.
- NIH. 2023. “The Use of Generative Artificial Intelligence
Technologies Is Prohibited for the NIH Peer Review Process.” National
Institutes of Health, Notice Number: NOT-OD-23-149, 23 June 2023.
- Schintler, Laurie, et al. 2023. “A Critical Examination of the
Ethics of AI-Mediated Peer Review.” arXiv, 2 September 2023.
<arXiv:2309.12356>.
- Wang Selonick, Lillian, et al. 2022. “AI and Peer Reviewer
Selection: Will Tools Help or Hurt Efforts to Increase Diversity?”
Poster presented at Society for Scholarly Publishing 2022 Annual
Meeting in Chicago.

Meanwhile, press coverage is minimal. Outside of the scientific press,
the media pieces I saw tended to be boosterish stuff about how AI just
might fix the peer review overload in the sciences; a more critical
engagement is therefore desirable.

I see only very limited relevance for linguistics or language study in
such treatments and that is why I appeal to colleagues to begin an
urgent discussion on the use of AI in peer review in our field.

Linguistic Field(s): General Linguistics

------------------------------------------------------------------------------

Please consider donating to the Linguist List https://give.myiu.org/iu-bloomington/I320011968.html

LINGUIST List is supported by the following publishers:

John Benjamins http://www.benjamins.com/

Lincom GmbH https://lincom-shop.eu/

Linguistic Association of Finland http://www.ling.helsinki.fi/sky/

Multilingual Matters http://www.multilingual-matters.com/

Wiley http://www.wiley.com

----------------------------------------------------------
LINGUIST List: Vol-35-387
----------------------------------------------------------