Analysis of single comments left for bioRxiv preprints till September 2019

Recommendations for accelerating open preprint peer review to improve the culture of science

Peer review is an important part of the scientific process, but traditional peer review at journals is coming under increased scrutiny for its inefficiency and lack of transparency. As preprints become more widely used and accepted, they raise the possibility of rethinking the peer-review process. Preprints are enabling new forms of peer review that have the potential to be more thorough, inclusive, and collegial than traditional journal peer review, and to thus fundamentally shift the culture of peer review toward constructive collaboration. In this Consensus View, we make a call to action to stakeholders in the community to accelerate the growing momentum of preprint sharing and provide recommendations to empower researchers to provide open and constructive peer review for preprints.

The experiences of COVID-19 preprint authors: a survey of researchers about publishing and receiving feedback on their work during the pandemic

The COVID-19 pandemic caused a rise in preprinting, triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experiences with preprinting their work and also with publishing their work in a peer-reviewed journal. Our research had the following objectives: 1. to learn about authors' experiences with preprinting, their motivations, and future intentions; 2. to consider preprints in terms of their effectiveness in enabling authors to receive feedback on their work; 3. to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. In our survey, 78% of the new adopters of preprinting reported the intention to also preprint their future work. The boost in preprinting may therefore have a structural effect that will last after the pandemic, although future developments will also depend on other factors, including the broader growth in the adoption of open science practices. A total of 53% of the respondents reported that they had received feedback on their preprints. However, more than half of the feedback was received through "closed" channels-privately to the authors. This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting "open" channels for preprint feedback. Almost a quarter of the feedback received by respondents consisted of detailed comments, showing the potential of preprint feedback to provide valuable comments on research. Respondents also reported that, compared to preprint feedback, journal peer review was more likely to lead to major changes to their work, suggesting that journal peer review provides significant added value compared to feedback received on preprints.

Preprints in Health Professions Education: Raising Awareness and Shifting Culture

A preprint is a version of a research manuscript posted by its authors to a preprint server before peer review. Preprints are associated with a variety of benefits, including the ability to rapidly communicate research, the opportunity for researchers to receive feedback and raise awareness of their research, and broad and unrestricted access. For early-career researchers, preprints also provide a mechanism for demonstrating research progress and productivity without the lengthy timelines of traditional journal publishing. Despite these benefits, few health professions education (HPE) research articles are deposited as preprints, suggesting that preprinting is not currently integrated into HPE culture. In this article, the authors describe preprints, their benefits and related risks, and the potential barriers that hamper their widespread use within HPE. In particular, the authors propose the barriers of discordant messaging and the lack of formal and informal education on how to deposit, critically appraise, and use preprints. To mitigate these barriers, several recommendations are proposed to facilitate preprints in becoming an accepted and encouraged component of HPE culture, allowing the field to take full advantage of this evolving form of research dissemination.

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals

Introduction

Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version.

Methods

Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version.

Results

Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints.

Conclusions

More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

A guide to preprinting for early-career researchers

The use of preprints, research manuscripts shared publicly before completing the traditional peer-review process, is becoming a more common practice among life science researchers. Early-career researchers (ECRs) benefit from posting preprints as they are shareable, citable, and prove productivity. However, preprinting a manuscript involves a discussion among all co-authors, and ECRs are often not the decision-makers. Therefore, ECRs may find themselves in situations where they are interested in depositing a preprint but are unsure how to approach their co-authors or advisor about preprinting. Leveraging our own experiences as ECRs, and feedback from the research community, we have constructed a guide for ECRs who are considering preprinting to enable them to take ownership over the process and to raise awareness about preprinting options. We hope that this guide helps ECRs to initiate conversations about preprinting with co-authors and encourage them to preprint their future research.

Summary: Are you an early-career researcher considering preprinting, but unsure how to approach conversations about the possibility? Here, we discuss preprinting and provide tips to enable you to take ownership over the process.

Is the future of peer review automated?

The rising rate of preprints and publications, combined with persistent inadequate reporting practices and problems with study design and execution, have strained the traditional peer review system. Automated screening tools could potentially enhance peer review by helping authors, journal editors, and reviewers to identify beneficial practices and common problems in preprints or submitted manuscripts. Tools can screen many papers quickly, and may be particularly helpful in assessing compliance with journal policies and with straightforward items in reporting guidelines. However, existing tools cannot understand or interpret the paper in the context of the scientific literature. Tools cannot yet determine whether the methods used are suitable to answer the research question, or whether the data support the authors' conclusions. Editors and peer reviewers are essential for assessing journal fit and the overall quality of a paper, including the experimental design, the soundness of the study's conclusions, potential impact and innovation. Automated screening tools cannot replace peer review, but may aid authors, reviewers, and editors in improving scientific papers. Strategies for responsible use of automated tools in peer review may include setting performance criteria for tools, transparently reporting tool performance and use, and training users to interpret reports.

Promoting constructive feedback on preprints with the FAST principles

Ensuring that public feedback on preprints is focused, appropriate, specific and transparent (or FAST) will help to develop a thriving culture for reviewing and commenting on preprints.