Preprint Servers' Policies, Submission Requirements, and Transparency in Reporting and Research Integrity Recommendations

Challenges and Controversies in Peer Review: JACC Review Topic of the Week

The process of peer review has been the gold standard for evaluating medical science, but significant pressures from the recent COVID-19 pandemic, new methods of communication, larger amounts of research, and an evolving publication landscape have placed significant pressures on this system. A task force convened by the American College of Cardiology identified the 5 most significant controversies associated with the current peer-review process: the effect of preprints, reviewer blinding, reviewer selection, reviewer incentivization, and publication of peer reviewer comments. Although specific solutions to these issues will vary, regardless of how scientific communication evolves, peer review must remain an essential process for ensuring scientific integrity, timely dissemination of information, and better patient care. In medicine, the peer-review process is crucial because harm can occur if poor-quality data or incorrect conclusions are published. With the dramatic increase in scientific publications and new methods of communication, high-quality peer review is more important now than ever.

Rapid, accurate publication and dissemination of clinical trial results: benefits and challenges

Large-scale clinical trials are essential in cardiology and require rapid, accurate publication, and dissemination. Whereas conference presentations, press releases, and social media disseminate information quickly and often receive considerable coverage by mainstream and healthcare media, they lack detail, may emphasize selected data, and can be open to misinterpretation. Preprint servers speed access to research manuscripts while awaiting acceptance for publication by a journal, but these articles are not formally peer-reviewed and sometimes overstate the findings. Publication of trial results in a major journal is very demanding but the use of existing checklists can help accelerate the process. In case of rejection, procedures such as easing formatting requirements and possibly carrying over peer-review to other journals could speed resubmission. Secondary publications can help maximize benefits from clinical trials; publications of secondary endpoints and subgroup analyses further define treatment effects and the patient populations most likely to benefit. These rely on data access, and although data sharing is becoming more common, many challenges remain. Beyond publication in medical journals, there is a need for wider knowledge dissemination to maximize impact on clinical practice. This might be facilitated through plain language summary publications. Social media, websites, mainstream news outlets, and other publications, although not peer-reviewed, are important sources of medical information for both the public and for clinicians. This underscores the importance of ensuring that the information is understandable, accessible, balanced, and trustworthy. This report is based on discussions held on December 2021, at the 18th Global Cardiovascular Clinical Trialists meeting, involving a panel of editors of some of the top medical journals, as well as members of the lay press, industry, and clinical trialists.

Characterization and Reach of Orthopaedic Research Posted to Preprint Servers: Are We "Undercooking" Our Science?

BACKGROUND

Although biomedical preprint servers have grown rapidly over the past several years, the harm to patient health and safety remains a major concern among several scientific communities. Despite previous studies examining the role of preprints during the Coronavirus-19 pandemic, there is limited information characterizing their impact on scientific communication in orthopaedic surgery.

QUESTIONS/PURPOSES

(1) What are the characteristics (subspecialty, study design, geographic origin, and proportion of publications) of orthopaedic articles on three preprint servers? (2) What are the citation counts, abstract views, tweets, and Altmetric score per preprinted article and per corresponding publication?

METHODS

Three of the largest preprint servers (medRxiv, bioRxiv, and Research Square) with a focus on biomedical topics were queried for all preprinted articles published between July 26, 2014, and September 1, 2021, using the following search terms: "orthopaedic," "orthopedic," "bone," "cartilage," "ligament," "tendon," "fracture," "dislocation," "hand," "wrist," "elbow," "shoulder," "spine," "spinal," "hip," "knee," "ankle," and "foot." Full-text articles in English related to orthopaedic surgery were included, while nonclinical studies, animal studies, duplicate studies, editorials, abstracts from conferences, and commentaries were excluded. A total of 1471 unique preprints were included and further characterized in terms of the orthopaedic subspecialty, study design, date posted, and geographic factors. Citation counts, abstract views, tweets, and Altmetric scores were collected for each preprinted article and the corresponding publication of that preprint in an accepting journal. We ascertained whether a preprinted article was published by searching title keywords and the corresponding author in three peer-reviewed article databases (PubMed, Google Scholar, and Dimensions) and confirming that the study design and research question matched.

RESULTS

The number of orthopaedic preprints increased from four in 2017 to 838 in 2020. The most common orthopaedic subspecialties represented were spine, knee, and hip. From 2017 to 2020, the cumulative counts of preprinted article citations, abstract views, and Altmetric scores increased. A corresponding publication was identified in 52% (762 of 1471) of preprints. As would be expected, because preprinting is a form of redundant publication, published articles that are also preprinted saw greater abstract views, citations, and Altmetric scores on a per-article basis.

CONCLUSION

Although preprints remain an extremely small proportion of all orthopaedic research, our findings suggest that nonpeer-reviewed, preprinted orthopaedic articles are being increasingly disseminated. These preprinted articles have a smaller academic and public footprint than their published counterparts, but they still reach a substantial audience through infrequent and superficial online interactions, which are far from equivalent to the engagement facilitated by peer review. Furthermore, the sequence of preprint posting and journal submission, acceptance, and publication is unclear based on the information available on these preprint servers. Thus, it is difficult to determine whether the metrics of preprinted articles are attributable to preprinting, and studies such as the present analysis will tend to overestimate the apparent impact of preprinting. Despite the potential for preprint servers to function as a venue for thoughtful feedback on research ideas, the available metrics data for these preprinted articles do not demonstrate the meaningful engagement that is achieved by peer review in terms of the frequency or depth of audience feedback.

CLINICAL RELEVANCE

Our findings highlight the need for safeguards to regulate research dissemination through preprint media, which has never been shown to benefit patients and should not be considered as evidence by clinicians. Clinician-scientists and researchers have the most important responsibility of protecting patients from the harm of potentially inaccurate biomedical science and therefore must prioritize patient needs first by uncovering scientific truths through the evidence-based processes of peer review, not preprinting. We recommend all journals publishing clinical research adopt the same policy as Clinical Orthopaedics and Related Research® , The Bone & Joint Journal, The Journal of Bone and Joint Surgery, and the Journal of Orthopaedic Research , removing any papers posted to preprint servers from consideration.

Researchers support preprints and open access publishing, but with reservations: A questionnaire survey of MBSJ members

Since the 1990s, journals have become increasingly online and open access. In fact, about 50% of articles published in 2021 were open access. The use of preprints (i.e., non-peer-reviewed articles) has also increased. However, there is limited awareness of these concepts among academics. Therefore, we conducted a questionnaire-based survey among members of the Molecular Biology Society of Japan. The survey was conducted between September 2022 and October 2022, with 633 respondents, 500 of whom (79.0%) were faculty members. In total, 478 (76.6%) respondents had published articles as open access, and 571 (91.5%) wanted to publish their articles in open access. Although 540 (86.5%) respondents knew about preprints, only 183 (33.9%) had posted preprints before. In the open-ended section of the questionnaire survey, several comments were made about the cost burdens associated with open access and the difficulty of how academic preprints are handled. Although open access is widespread, and recognition of preprints is increasing, some issues remain that need to be addressed. Academic and institutional support, and transformative agreement may help reduce the cost burden. Guidelines for handling preprints in academia are also important for responding to changes in the research environment.

Peer Review in a General Medical Research Journal Before and During the COVID-19 Pandemic

Importance

Although peer review is an important component of publication for new research, the viability of this process has been questioned, particularly with the added stressors of the COVID-19 pandemic.

Objective

To characterize rates of peer reviewer acceptance of invitations to review manuscripts, reviewer turnaround times, and editor-assessed quality of reviews before and after the start of the COVID-19 pandemic at a large, open-access general medical journal.

Design, Setting, and Participants

This retrospective, pre-post cohort study examined all research manuscripts submitted to JAMA Network Open between January 1, 2019, and June 29, 2021, either directly or via transfer from other JAMA Network journals, for which at least 1 peer review of manuscript content was solicited. Measures were compared between the period before the World Health Organization declaration of a COVID-19 pandemic on March 11, 2020 (14.3 months), and the period during the pandemic (15.6 months) among all reviewed manuscripts and between pandemic-period manuscripts that did or did not address COVID-19.

Main Outcomes and Measures

For each reviewed manuscript, the number of invitations sent to reviewers, proportions of reviewers accepting invitations, time in days to return reviews, and editor-assessed quality ratings of reviews were determined.

Results

In total, the journal sought review for 5013 manuscripts, including 4295 Original Investigations (85.7%) and 718 Research Letters (14.3%); 1860 manuscripts were submitted during the prepandemic period and 3153 during the pandemic period. Comparing the prepandemic with the pandemic period, the mean (SD) number of reviews rated as high quality (very good or excellent) per manuscript increased slightly from 1.3 (0.7) to 1.5 (0.7) (P < .001), and the mean (SD) time for reviewers to return reviews was modestly shorter (from 15.8 [7.6] days to 14.4 [7.0] days; P < .001), a difference that persisted in linear regression models accounting for manuscript type, study design, and whether the manuscript addressed COVID-19.

Conclusions and Relevance

In this cohort study, the speed and editor-reported quality of peer reviews in an open-access general medical journal improved modestly during the initial year of the pandemic. Additional study will be necessary to understand how the pandemic has affected reviewer burden and fatigue.

TIER2: enhancing Trust, Integrity and Efficiency in Research through next-level Reproducibility

Lack of reproducibility of research results has become a major theme in recent years. As we emerge from the COVID-19 pandemic, economic pressures and exposed consequences of lack of societal trust in science make addressing reproducibility of urgent importance. TIER2 is a new international project funded by the European Commission under their Horizon Europe programme. Covering three broad research areas (social, life and computer sciences) and two cross-disciplinary stakeholder groups (research publishers and funders) to systematically investigate reproducibility across contexts, TIER2 will significantly boost knowledge on reproducibility, create tools, engage communities, implement interventions and policy across different contexts to increase re-use and overall quality of research results in the European Research Area and global R&I, and consequently increase trust, integrity and efficiency in research.

Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS report

BACKGROUND

Retraction is a mechanism for alerting readers to unreliable material and other problems in the published scientific and scholarly record. Retracted publications generally remain visible and searchable, but the intention of retraction is to mark them as "removed" from the citable record of scholarship. However, in practice, some retracted articles continue to be treated by researchers and the public as valid content as they are often unaware of the retraction. Research over the past decade has identified a number of factors contributing to the unintentional spread of retracted research. The goal of the Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda (RISRS) project was to develop an actionable agenda for reducing the inadvertent spread of retracted science. This included identifying how retraction status could be more thoroughly disseminated, and determining what actions are feasible and relevant for particular stakeholders who play a role in the distribution of knowledge.

METHODS

These recommendations were developed as part of a year-long process that included a scoping review of empirical literature and successive rounds of stakeholder consultation, culminating in a three-part online workshop that brought together a diverse body of 65 stakeholders in October-November 2020 to engage in collaborative problem solving and dialogue. Stakeholders held roles such as publishers, editors, researchers, librarians, standards developers, funding program officers, and technologists and worked for institutions such as universities, governmental agencies, funding organizations, publishing houses, libraries, standards organizations, and technology providers. Workshop discussions were seeded by materials derived from stakeholder interviews (N = 47) and short original discussion pieces contributed by stakeholders. The online workshop resulted in a set of recommendations to address the complexities of retracted research throughout the scholarly communications ecosystem.

RESULTS

The RISRS recommendations are: (1) Develop a systematic cross-industry approach to ensure the public availability of consistent, standardized, interoperable, and timely information about retractions; (2) Recommend a taxonomy of retraction categories/classifications and corresponding retraction metadata that can be adopted by all stakeholders; (3) Develop best practices for coordinating the retraction process to enable timely, fair, unbiased outcomes; and (4) Educate stakeholders about pre- and post-publication stewardship, including retraction and correction of the scholarly record.

CONCLUSIONS

Our stakeholder engagement study led to 4 recommendations to address inadvertent citation of retracted research, and formation of a working group to develop the Communication of Retractions, Removals, and Expressions of Concern (CORREC) Recommended Practice. Further work will be needed to determine how well retractions are currently documented, how retraction of code and datasets impacts related publications, and to identify if retraction metadata (fails to) propagate. Outcomes of all this work should lead to ensuring retracted papers are never cited without awareness of the retraction, and that, in public fora outside of science, retracted papers are not treated as valid scientific outputs.

Publishing Identifiable Patient Photographs in Scientific Journals: Scoping Review of Policies and Practices

BACKGROUND

Publishing identifiable patient data in scientific journals may jeopardize patient privacy and confidentiality if best ethical practices are not followed. Current journal practices show considerable diversity in the publication of identifiable patient photographs, and different stakeholders may have different opinions of and practices in publishing patient photographs.

OBJECTIVE

This scoping review aimed to identify existing evidence and map knowledge gaps in medical research on the policies and practices of publishing identifiable photographs in scientific articles.

METHODS

We performed a comprehensive search of the Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, CINAHL with Full Text, Database of Abstracts of Reviews of Effects, Ovid MEDLINE, and Scopus. The Open Science Framework, PROSPERO, BASE, Google Scholar, OpenGrey, ClinicalTrials.gov, the Campbell Collaboration Library, and Science.gov were also searched.

RESULTS

After screening the initial 15,949 titles and abstracts, 98 (0.61%) publications were assessed for eligibility at the full-text level, and 30 (0.19%) publications were included in this review. The studies were published between 1994 and 2020; most had a cross-sectional design and were published in journals covering different medical disciplines. We identified 3 main topics. The first included ethical aspects of the use of facial photographs in publications. In different clinical settings, the consent process was not conducted properly, and health professionals did not recognize the importance of obtaining written patient consent for taking and using patient medical photographs. They often considered verbal consent sufficient or even used the photographs without consent. The second topic included studies that investigated the practices and use of medical photography in publishing. Both patients and doctors asked for confidential storage and maintenance of medical photographs. Patients preferred to be photographed by their physicians using an institutional camera and preferred nonidentifiable medical photographs not only for publication but also in general. Conventional methods of deidentification of facial photographs concealing the eye area were recognized as unsuccessful in protecting patient privacy. The third topic emerged from studies investigating medical photography in journal articles. These studies showed great diversity in publishing practices regarding consent for publication of medical photographs. Journal policies regarding the consent process and consent forms were insufficient, and existing ethical professional guidelines were not fully implemented in actual practices. Patients' photographs from open-access medical journals were found on public web-based platforms.

CONCLUSIONS

This scoping review showed a diversity of practices in publishing identifiable patient photographs and an unsatisfactory level of knowledge of this issue among different stakeholders despite existing standards. Emerging issues include the availability of patients' photographs from open-access journals or preprints in the digital environment. There is a need to improve standards and processes to obtain proper consent to fully protect the privacy of patients in published articles.

Preprint servers in lipidology: current status and future role

PURPOSE OF REVIEW

Preprinting, or the sharing of non-peer reviewed, unpublished scholarly manuscripts, has exploded in all fields of science and medicine over the past 5 years. We searched the literature and evaluated the posting and uptake of preprint publications in the field of lipidology in bioRxiv and medRxiv servers. We also contacted the editorial offices of 20 journals that publish original research in lipidology to gauge their policies on preprints.

RECENT FINDINGS

All 20 journals contacted indicated that they accepted preprints. As of 31 May 2021, 473 and 231 preprints in lipidology had been submitted to bioRxiv and medRxiv, respectively. About half of all lipidology preprints were related to cardiovascular, cardiometabolic, and/or metabolic diseases (CVMD) and their risk factors, but at least 12 other disease categories were also represented. 16.9% and 1.08% of medRxiv and bioRxiv preprints, respectively, were related to coronavirus disease 2019 (COVID-19).

SUMMARY

All identified journals accept lipidology themed preprints for submission, removing any barriers authors may have had regarding preprinting. Based on growing experience with preprinting, this trend should encourage increased community feedback and facilitate higher quality lipidology research in the future.

Biosecurity in an age of open science

The risk of accidental or deliberate misuse of biological research is increasing as biotechnology advances. As open science becomes widespread, we must consider its impact on those risks and develop solutions that ensure security while facilitating scientific progress. Here, we examine the interaction between open science practices and biosecurity and biosafety to identify risks and opportunities for risk mitigation. Increasing the availability of computational tools, datasets, and protocols could increase risks from research with misuse potential. For instance, in the context of viral engineering, open code, data, and materials may increase the risk of release of enhanced pathogens. For this dangerous subset of research, both open science and biosecurity goals may be achieved by using access-controlled repositories or application programming interfaces. While preprints accelerate dissemination of findings, their increased use could challenge strategies for risk mitigation at the publication stage. This highlights the importance of oversight earlier in the research lifecycle. Preregistration of research, a practice promoted by the open science community, provides an opportunity for achieving biosecurity risk assessment at the conception of research. Open science and biosecurity experts have an important role to play in enabling responsible research with maximal societal benefit.

Preprint paper platforms in the academic scholarly communication environment

Academic scholarly communication is the predominant business of researchers, scientists, and scholars. It is the core element of promoting scientific thought, investigation, and building up solid knowledge. The development of preprint platform web interfaces, server repositories of electronic scholarly papers submitted by their authors and openly available to the scientific community proposed a new form of academic communication. The distribution of a preprint of a scientific manuscript allows the authors to claim the priority of discovery, in a manner similar to the conference proceedings output, but also creates an anteriority that prevents protection by a patent application. Herein, we review the scope and the role of preprint papers platforms in academia, we explore individual cases, arXiv, SSRN, OSF Preprints, HAL, bioRxiv, EconStor, RePEc, PhilArchive, Research Square, viXra, Cryptology ePrint Archive, Preprints.org, ChinaXiv, medRxiv, JMIR Preprints, Authorea, ChemRxiv, engrXiv, e-LiS, SciELO, PsyArXiv, F1000 Research, and Zenodo, and discuss their significance in promoting scientific discovery, the potential risks of scientific integrity, as well as the policies of data distribution and intellectual property rights, the plus and minus, for the stakeholders, authors, institutions, states, scientific journals, scientific community, and the public. In this review we explore the scope and policies of the existing preprint papers platforms in different academic research fields.

Attitudes and practices of open data, preprinting, and peer-review-A cross sectional study on Croatian scientists

Attitudes towards open peer review, open data and use of preprints influence scientists’ engagement with those practices. Yet there is a lack of validated questionnaires that measure these attitudes. The goal of our study was to construct and validate such a questionnaire and use it to assess attitudes of Croatian scientists. We first developed a 21-item questionnaire called Attitudes towards Open data sharing, preprinting, and peer-review (ATOPP), which had a reliable four-factor structure, and measured attitudes towards open data, preprint servers, open peer-review and open peer-review in small scientific communities. We then used the ATOPP to explore attitudes of Croatian scientists (n = 541) towards these topics, and to assess the association of their attitudes with their open science practices and demographic information. Overall, Croatian scientists’ attitudes towards these topics were generally neutral, with a median (Md) score of 3.3 out of max 5 on the scale score. We also found no gender (P = 0.995) or field differences (P = 0.523) in their attitudes. However, attitudes of scientist who previously engaged in open peer-review or preprinting were higher than of scientists that did not (Md 3.5 vs. 3.3, P<0.001, and Md 3.6 vs 3.3, P<0.001, respectively). Further research is needed to determine optimal ways of increasing scientists’ attitudes and their open science practices.

Analysis of single comments left for bioRxiv preprints till September 2019

INTRODUCTION

While early commenting on studies is seen as one of the advantages of preprints, the type of such comments, and the people who post them, have not been systematically explored.

MATERIALS AND METHODS

We analysed comments posted between 21 May 2015 and 9 September 2019 for 1983 bioRxiv preprints that received only one comment on the bioRxiv website. The comment types were classified by three coders independently, with all differences resolved by consensus.

RESULTS

Our analysis showed that 69% of comments were posted by non-authors (N = 1366), and 31% by the preprints' authors themselves (N = 617). Twelve percent of non-author comments (N = 168) were full review reports traditionally found during journal review, while the rest most commonly contained praises (N = 577, 42%), suggestions (N = 399, 29%), or criticisms (N = 226, 17%). Authors' comments most commonly contained publication status updates (N = 354, 57%), additional study information (N = 158, 26%), or solicited feedback for the preprints (N = 65, 11%).

CONCLUSIONS

Our results indicate that comments posted for bioRxiv preprints may have potential benefits for both the public and the scholarly community. Further research is needed to measure the direct impact of these comments on comments made by journal peer reviewers, subsequent preprint versions or journal publications.

Delays in reporting and publishing trial results during pandemics: cross sectional analysis of 2009 H1N1, 2014 Ebola, and 2016 Zika clinical trials

BACKGROUND

Pandemic events often trigger a surge of clinical trial activity aimed at rapidly evaluating therapeutic or preventative interventions. Ensuring rapid public access to the complete and unbiased trial record is particularly critical for pandemic research given the urgent associated public health needs. The World Health Organization (WHO) established standards requiring posting of results to a registry within 12 months of trial completion and publication in a peer reviewed journal within 24 months of completion, though compliance with these requirements among pandemic trials is unknown.

METHODS

This cross-sectional analysis characterizes availability of results in trial registries and publications among registered trials performed during the 2009 H1N1 influenza, 2014 Ebola, and 2016 Zika pandemics. We searched trial registries to identify clinical trials testing interventions related to these pandemics, and determined the time elapsed between trial completion and availability of results in the registry. We also performed a comprehensive search of MEDLINE via PubMed, Google Scholar, and EMBASE to identify corresponding peer reviewed publications. The primary outcome was the compliance with either of the WHO's established standards for sharing clinical trial results. Secondary outcomes included compliance with both standards, and assessing the time elapsed between trial completion and public availability of results.

RESULTS

Three hundred thirty-three trials met eligibility criteria, including 261 H1N1 influenza trials, 60 Ebola trials, and 12 Zika trials. Of these, 139 (42%) either had results available in the trial registry within 12 months of study completion or had results available in a peer-reviewed publication within 24 months. Five trials (2%) met both standards. No results were available in either a registry or publication for 59 trials (18%). Among trials with registered results, a median of 42 months (IQR 16-76 months) elapsed between trial completion and results posting. For published trials, the median elapsed time between completion and publication was 21 months (IQR 9-34 months). Results were available within 24 months of study completion in either the trial registry or a peer reviewed publication for 166 trials (50%).

CONCLUSIONS

Very few trials performed during prior pandemic events met established standards for the timely public dissemination of trial results.