Preprints and Scholarly Communication: Adoption, Practices, Drivers and Barriers

Full publication of preprint articles in prevention research: an analysis of publication proportions and results consistency

There is concern that preprint articles will lead to an increase in the amount of scientifically invalid work available. The objectives of this study were to determine the proportion of prevention preprints published within 12 months, the consistency of the effect estimates and conclusions between preprint and published articles, and the reasons for the nonpublication of preprints. Of the 329 prevention preprints that met our eligibility criteria, almost half (48.9%) were published in a peer-reviewed journal within 12 months of being posted. While 16.8% published preprints showed some change in the magnitude of the primary outcome effect estimate, 4.4% were classified as having a major change. The style or wording of the conclusion changed in 42.2%, the content in 3.1%. Preprints on chemoprevention, with a cross-sectional design, and with public and noncommercial funding had the highest probabilities of publication. The main reasons for the nonpublication of preprints were journal rejection or lack of time. The reliability of preprint articles for evidence-based decision-making is questionable. Less than half of the preprint articles on prevention research are published in a peer-reviewed journal within 12 months, and significant changes in effect sizes and/or conclusions are still possible during the peer-review process.

The experiences of COVID-19 preprint authors: a survey of researchers about publishing and receiving feedback on their work during the pandemic

The COVID-19 pandemic caused a rise in preprinting, triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experiences with preprinting their work and also with publishing their work in a peer-reviewed journal. Our research had the following objectives: 1. to learn about authors' experiences with preprinting, their motivations, and future intentions; 2. to consider preprints in terms of their effectiveness in enabling authors to receive feedback on their work; 3. to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. In our survey, 78% of the new adopters of preprinting reported the intention to also preprint their future work. The boost in preprinting may therefore have a structural effect that will last after the pandemic, although future developments will also depend on other factors, including the broader growth in the adoption of open science practices. A total of 53% of the respondents reported that they had received feedback on their preprints. However, more than half of the feedback was received through "closed" channels-privately to the authors. This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting "open" channels for preprint feedback. Almost a quarter of the feedback received by respondents consisted of detailed comments, showing the potential of preprint feedback to provide valuable comments on research. Respondents also reported that, compared to preprint feedback, journal peer review was more likely to lead to major changes to their work, suggesting that journal peer review provides significant added value compared to feedback received on preprints.

Exploring arXiv usage habits among Slovenian scientists

Purpose

This study investigates how important the preprint arXiv is for Slovenian scientists, whether there are differences between scientific disciplines and the reputation of arXiv among Slovenian scientists. We are also interested in what advantages and disadvantages scientists see in using arXiv.

Design/methodology/approach

A voluntary sample of active researchers from the scientific fields covered by arXiv was used. Data were collected over 21 days in September 2021 using a 40-question online survey. In addition to descriptive statistics, nonparametric statistical methods such as Pearson's chi-squared test for independence, Kruskal-Wallis' H-test and Mann-Whitney's U-test were applied to the collected data.

Findings

Among Slovenian scientists there is a wide range of different users of arXiv. The authors note differences among scientific disciplines. Physicists and astronomers are the most engaged, followed by mathematicians. Researchers in computer science, electrical engineering and systems science seem to have recognized the benefits of the archive, but are still hesitant to use it. Researchers from the other scientific fields participated in the survey to a lesser extent, suggesting that arXiv is less popular in these scientific fields. For Slovenian scientists, the main advantages of arXiv are faster access to knowledge, open access, greater impact of scientists' work and the fact that publishing in the archive is free of charge. A negative aspect of using the archive is the frustration caused by the difficulties in assessing the credibility of articles.

Research limitations/implications

A voluntary sample was used, which attracted a larger number of researchers but has a higher risk of sampling bias.

Practical implications

The results are useful for international comparisons, but also provide bases and recommendations for institutional and national policies to evaluate researchers and their performance.

Originality/value

The results provide valuable insights into arXiv usage habits and the reasons for using or not using arXiv by Slovenian scientists. There is no comparable study conducted in Slovenia.

COVID-19 and the scientific publishing system: growth, open access and scientific fields

We model the growth of scientific literature related to COVID-19 and forecast the expected growth from 1 June 2021. Considering the significant scientific and financial efforts made by the research community to find solutions to end the COVID-19 pandemic, an unprecedented volume of scientific outputs is being produced. This questions the capacity of scientists, politicians and citizens to maintain infrastructure, digest content and take scientifically informed decisions. A crucial aspect is to make predictions to prepare for such a large corpus of scientific literature. Here we base our predictions on the Autoregressive Integrated Moving Average (ARIMA) and exponential smoothing models using the Dimensions database. This source has the particularity of including in the metadata information on the date in which papers were indexed. We present global predictions, plus predictions in three specific settings: by type of access (Open Access), by domain-specific repository (SSRN and MedRxiv) and by several research fields. We conclude by discussing our findings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11192-022-04536-x.

Innovating peer review, reconfiguring scholarly communication: an analytical overview of ongoing peer review innovation activities

Purpose

The study aims to provide an analytical overview of current innovations in peer review and their potential impacts on scholarly communication.

Design/methodology/approach

The authors created a survey that was disseminated among publishers, academic journal editors and other organizations in the scholarly communication ecosystem, resulting in a data set of 95 self-defined innovations. The authors ordered the material using a taxonomy that compares innovation projects according to five dimensions. For example, what is the object of review? How are reviewers recruited, and does the innovation entail specific review foci?

Findings

Peer review innovations partly pull in mutually opposed directions. Several initiatives aim to make peer review more efficient and less costly, while other initiatives aim to promote its rigor, which is likely to increase costs; innovations based on a singular notion of “good scientific practice” are at odds with more pluralistic understandings of scientific quality; and the idea of transparency in peer review is the antithesis to the notion that objectivity requires anonymization. These fault lines suggest a need for better coordination.

Originality/value

This paper presents original data that were analyzed using a novel, inductively developed, taxonomy. Contrary to earlier research, the authors do not attempt to gauge the extent to which peer review innovations increase the “reliability” or “quality” of reviews (as defined according to often implicit normative criteria), nor are they trying to measure the uptake of innovations in the routines of academic journals. Instead, they focus on peer review innovation activities as a distinct object of analysis.

A guide to preprinting for early-career researchers

The use of preprints, research manuscripts shared publicly before completing the traditional peer-review process, is becoming a more common practice among life science researchers. Early-career researchers (ECRs) benefit from posting preprints as they are shareable, citable, and prove productivity. However, preprinting a manuscript involves a discussion among all co-authors, and ECRs are often not the decision-makers. Therefore, ECRs may find themselves in situations where they are interested in depositing a preprint but are unsure how to approach their co-authors or advisor about preprinting. Leveraging our own experiences as ECRs, and feedback from the research community, we have constructed a guide for ECRs who are considering preprinting to enable them to take ownership over the process and to raise awareness about preprinting options. We hope that this guide helps ECRs to initiate conversations about preprinting with co-authors and encourage them to preprint their future research.

Summary: Are you an early-career researcher considering preprinting, but unsure how to approach conversations about the possibility? Here, we discuss preprinting and provide tips to enable you to take ownership over the process.

Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets

In this article, we investigate the surge in use of COVID-19-related preprints by media outlets. Journalists are a main source of reliable public health information during crises and, until recently, journalists have been reluctant to cover preprints because of the associated scientific uncertainty. Yet, uploads of COVID-19 preprints and their uptake by online media have outstripped that of preprints about any other topic. Using an innovative approach combining altmetrics methods with content analysis, we identified a diversity of outlets covering COVID-19-related preprints during the early months of the pandemic, including specialist medical news outlets, traditional news media outlets, and aggregators. We found a ubiquity of hyperlinks as citations and a multiplicity of framing devices for highlighting the scientific uncertainty associated with COVID-19 preprints. These devices were rarely used consistently (e.g., mentioning that the study was a preprint, unreviewed, preliminary, and/or in need of verification). About half of the stories we analyzed contained framing devices emphasizing uncertainty. Outlets in our sample were much less likely to identify the research they mentioned as preprint research, compared to identifying it as simply "research." This work has significant implications for public health communication within the changing media landscape. While current best practices in public health risk communication promote identifying and promoting trustworthy sources of information, the uptake of preprint research by online media presents new challenges. At the same time, it provides new opportunities for fostering greater awareness of the scientific uncertainty associated with health research findings.

Examining linguistic shifts between preprints and publications

Preprints allow researchers to make their findings available to the scientific community before they have undergone peer review. Studies on preprints within bioRxiv have been largely focused on article metadata and how often these preprints are downloaded, cited, published, and discussed online. A missing element that has yet to be examined is the language contained within the bioRxiv preprint repository. We sought to compare and contrast linguistic features within bioRxiv preprints to published biomedical text as a whole as this is an excellent opportunity to examine how peer review changes these documents. The most prevalent features that changed appear to be associated with typesetting and mentions of supporting information sections or additional files. In addition to text comparison, we created document embeddings derived from a preprint-trained word2vec model. We found that these embeddings are able to parse out different scientific approaches and concepts, link unannotated preprint-peer-reviewed article pairs, and identify journals that publish linguistically similar papers to a given preprint. We also used these embeddings to examine factors associated with the time elapsed between the posting of a first preprint and the appearance of a peer-reviewed publication. We found that preprints with more versions posted and more textual changes took longer to publish. Lastly, we constructed a web application (https://greenelab.github.io/preprint-similarity-search/) that allows users to identify which journals and articles that are most linguistically similar to a bioRxiv or medRxiv preprint as well as observe where the preprint would be positioned within a published article landscape.

Over-promotion and caution in abstracts of preprints during the COVID-19 crisis

The abstract is known to be a promotional genre where researchers tend to exaggerate the benefit of their research and use a promotional discourse to catch the reader's attention. The COVID-19 pandemic has prompted intensive research and has changed traditional publishing with the massive adoption of preprints by researchers. Our aim is to investigate whether the crisis and the ensuing scientific and economic competition have changed the lexical content of abstracts. We propose a comparative study of abstracts associated with preprints issued in response to the pandemic relative to abstracts produced during the closest pre-pandemic period. We show that with the increase (on average and in percentage) of positive words (especially effective) and the slight decrease of negative words, there is a strong increase in hedge words (the most frequent of which are the modal verbs can and may). Hedge words counterbalance the excessive use of positive words and thus invite the readers, who go probably beyond the 'usual' audience, to be cautious with the obtained results. The abstracts of preprints urgently produced in response to the COVID-19 crisis stand between uncertainty and over-promotion, illustrating the balance that authors have to achieve between promoting their results and appealing for caution.

Publication speed in pharmacy practice journals: A comparative analysis

Background

Scholarly publishing system relies on external peer review. However, the duration of publication process is a major concern for authors and funding bodies.

Objective

To evaluate the duration of the publication process in pharmacy practice journals compared with other biomedical journals indexed in PubMed.

Methods

All the articles published from 2009 to 2018 by the 33 pharmacy practice journals identified in Mendes et al. study and indexed in PubMed were gathered as study group. A comparison group was created through a random selection of 3000 PubMed PMIDs for each year of study period. Articles with publication dates outside the study period were excluded. Metadata of both groups of articles were imported from PubMed. The duration of editorial process was calculated with three periods: acceptance lag (days between ‘submission date’ and ‘acceptance date’), lead lag (days between ‘acceptance date’ and ‘online publication date’), and indexing lag (days between ‘online publication date’ and ‘Entry date’). Null hypothesis significance tests and effect size measures were used to compare these periods between both groups.

Results

The 33 pharmacy practice journals published 26,256 articles between 2009 and 2018. Comparison group random selection process resulted in a pool of 23,803 articles published in 5,622 different journals. Acceptance lag was 105 days (IQR 57–173) for pharmacy practice journals and 97 days (IQR 56–155) for the comparison group with a null effect difference (Cohen’s d 0.081). Lead lag was 13 (IQR 6–35) and 23 days (IQR 9–45) for pharmacy practice and comparison journals, respectively, which resulted in a small effect. Indexing lag was 5 days (IQR 2–46) and 4 days (IQR 2–12) for pharmacy practice and control journals, which also resulted in a small effect. Slight positive time trend was found in pharmacy practice acceptance lag, while slight negative trends were found for lead and indexing lags for both groups.

Conclusions

Publication process duration of pharmacy practice journals is similar to a general random sample of articles from all disciplines.

Analysis of single comments left for bioRxiv preprints till September 2019

INTRODUCTION

While early commenting on studies is seen as one of the advantages of preprints, the type of such comments, and the people who post them, have not been systematically explored.

MATERIALS AND METHODS

We analysed comments posted between 21 May 2015 and 9 September 2019 for 1983 bioRxiv preprints that received only one comment on the bioRxiv website. The comment types were classified by three coders independently, with all differences resolved by consensus.

RESULTS

Our analysis showed that 69% of comments were posted by non-authors (N = 1366), and 31% by the preprints' authors themselves (N = 617). Twelve percent of non-author comments (N = 168) were full review reports traditionally found during journal review, while the rest most commonly contained praises (N = 577, 42%), suggestions (N = 399, 29%), or criticisms (N = 226, 17%). Authors' comments most commonly contained publication status updates (N = 354, 57%), additional study information (N = 158, 26%), or solicited feedback for the preprints (N = 65, 11%).

CONCLUSIONS

Our results indicate that comments posted for bioRxiv preprints may have potential benefits for both the public and the scholarly community. Further research is needed to measure the direct impact of these comments on comments made by journal peer reviewers, subsequent preprint versions or journal publications.

Open Science in Kenya: Where Are We?

According to the United Nations Educational, Scientific, and Cultural Organization (UNESCO), Open Science is the movement to make scientific research and data accessible to all. It has great potential for advancing science. At its core, it includes (but is not limited to) open access, open data, and open research. Some of the associated advantages are promoting collaboration, sharing and reproducibility in research, and preventing the reinvention of the wheel, thus saving resources. As research becomes more globalized and its output grows exponentially, especially in data, the need for open scientific research practices is more evident — the future of modern science. This has resulted in a concerted global interest in open science uptake. Even so, barriers still exist. The formal training curriculum in most, if not all, universities in Kenya does not equip students with the knowledge and tools to subsequently practice open science in their research. Therefore, to work openly and collaboratively, there is a need for awareness and training in the use of open science tools. These have been neglected, especially in most developing countries, and remain barriers to the cause. Moreover, there is scanty research on the state of affairs regarding the practice and/or adoption of open science. Thus, we developed, through the OpenScienceKE framework, a model to narrow the gap. A sensitize-train-hack-collaborate model was applied in Nairobi, the economic and administrative capital of Kenya. Using the model, we sensitized through seminars, trained on the use of tools through workshops, applied the skills learned in training through hackathons to collaboratively answer the question on the state of open science in Kenya. While the former parts of the model had 20–50 participants, the latter part mainly involved participants with a bioinformatics background, leveraging their advanced computational skills. This model resulted in an open resource that researchers can use to publish as open access cost-effectively. Moreover, we observed a growing interest in open science practices in Kenya through literature search and data mining and that lack of awareness and skills may still hinder the adoption and practice of open science. Furthermore, at the time of the analyses, we surprisingly found that out of the 20,069 papers downloaded from BioRXiv, only 18 had Kenyan authors, a majority of which are international (16) collaborations. This may suggest poor uptake of the use of preprints among Kenyan researchers. The findings in this study highlight the state of open science in Kenya and challenges facing its adoption and practice while bringing forth possible areas for primary consideration in the campaign toward open science. It also proposes a model (sensitize-train-hack-collaborate model) that may be adopted by researchers, funders and other proponents of open science to address some of the challenges faced in promoting its adoption in Kenya.

Mycopathologia 2020: Legacy and Change to Remain Relevant for Content, Creation, and Communication

The 2020 COVID-19 pandemic had a profound impact on the publishing landscape. The 'pre-peer-review' publication model is likely to become common as a lag in publishing is not acceptable in a pandemic or other time! Mycopathologia is well placed to adopt such changes with its improved editorial processes, article formats, author engagements, and published articles' access and citation. Mycopathologia had an improved journal impact factor and article downloads in 2018-2019. A limited sampling suggested a slight decrease in the total submissions in 2019 (352 articles) compared to 2018 (371 articles). However, the acceptance rate improved to 30% in 2019 from 19% in 2018. Nearly half of all submissions in 2019 were rejected before peer-review or transferred to other Springer Nature journals. The published articles were contributed from 34 different countries, with authors from China, the USA, and Brazil among the top three contributors. An enhanced editorial oversight allowed peer-reviewers to focus on fewer articles that were well-matched to their expertise, which led to lower rejection rates post-peer-review. The introduction of MycopathologiaGENOME and MycopathologiaIMAGE article types received a good reception with notable downloads and citations.

The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape

The world continues to face a life-threatening viral pandemic. The virus underlying the Coronavirus Disease 2019 (COVID-19), Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), has caused over 98 million confirmed cases and 2.2 million deaths since January 2020. Although the most recent respiratory viral pandemic swept the globe only a decade ago, the way science operates and responds to current events has experienced a cultural shift in the interim. The scientific community has responded rapidly to the COVID-19 pandemic, releasing over 125,000 COVID-19-related scientific articles within 10 months of the first confirmed case, of which more than 30,000 were hosted by preprint servers. We focused our analysis on bioRxiv and medRxiv, 2 growing preprint servers for biomedical research, investigating the attributes of COVID-19 preprints, their access and usage rates, as well as characteristics of their propagation on online platforms. Our data provide evidence for increased scientific and public engagement with preprints related to COVID-19 (COVID-19 preprints are accessed more, cited more, and shared more on various online platforms than non-COVID-19 preprints), as well as changes in the use of preprints by journalists and policymakers. We also find evidence for changes in preprinting and publishing behaviour: COVID-19 preprints are shorter and reviewed faster. Our results highlight the unprecedented role of preprints and preprint servers in the dissemination of COVID-19 science and the impact of the pandemic on the scientific communication landscape.

Preprint Servers in Kidney Disease Research: A Rapid Review

Preprint servers, such as arXiv and bioRxiv, have disrupted the scientific communication landscape by providing rapid access to research before peer review. medRxiv was launched as a free online repository for preprints in the medical, clinical, and related health sciences in 2019. In this review, we present the uptake of preprint server use in nephrology and discuss specific considerations regarding preprint server use in medicine. Distribution of kidney-related research on preprint servers is rising at an exponential rate. Survey of nephrology journals identified that 15 of 17 (88%) are publishing original research accepted submissions that have been uploaded to preprint servers. After reviewing 52 clinically impactful trials in nephrology discussed in the online Nephrology Journal Club (NephJC), an average lag of 300 days was found between study completion and publication, indicating an opportunity for faster research dissemination. Rapid review of papers discussing benefits and risks of preprint server use from the researcher, publisher, or end user perspective identified 53 papers that met criteria. Potential benefits of biomedical preprint servers included rapid dissemination, improved transparency of the peer review process, greater visibility and recognition, and collaboration. However, these benefits come at the risk of rapid spread of results not yet subjected to the rigors of peer review. Preprint servers shift the burden of critical appraisal to the reader. Media may be especially at risk due to their focus on "late-breaking" information. Preprint servers have played an even larger role when late-breaking research results are of special interest, such as during the global coronavirus disease 2019 pandemic. Coronavirus disease 2019 has brought both the benefits and risks of preprint servers to the forefront. Given the prominent online presence of the nephrology community, it is poised to lead the medicine community in appropriate use of preprint servers.

Preprints in Chemistry: An Exploratory Analysis of Differences with Journal Articles

The exploratory analysis of the differences between preprints and the corresponding peer reviewed journal articles for ten studies first published on ChemRxiv and on Preprints, though statistically non-significant, suggests outcomes of relevance for chemistry researchers and educators. The full transition to open science requires new education of doctoral students and young researchers on scholarly communication in the digital age. The preliminary findings of this study will contribute to inform the curriculum of the aforementioned new courses for young chemists, eventually promoting accelerated innovation in a science that, unique amid all basic sciences, originates a huge industry central to the wealth of nations.

Real-Time Bibliometrics: Dimensions as a Resource for Analyzing Aspects of COVID-19

Dimensions was built as a platform to allow stakeholders in the research community, including academic bibliometricians, to more easily create and understand the context of different types of research object through the linkages between these objects. Links between objects are created via persistent identifiers and machine learning techniques, while additional context is introduced via data enhancements such as per-object categorisations and person and institution disambiguation. While these features make analytical use cases accessible for end users, the COVID-19 crisis has highlighted a different set of needs to analyze trends in scholarship as they occur: Real-time bibliometrics. The combination of full-text search, daily data updates, a broad set of scholarly objects including pre-prints and a wider set of data fields for analysis, broadens opportunities for a different style of analysis. A subset of these emerging capabilities is discussed and three basic analyses are presented as illustrations of the potential for real-time bibliometrics.

Peer review and preprint policies are unclear at most major journals

Clear and findable publishing policies are important for authors to choose appropriate journals for publication. We investigated the clarity of policies of 171 major academic journals across disciplines regarding peer review and preprinting. 31.6% of journals surveyed do not provide information on the type of peer review they use. Information on whether preprints can be posted or not is unclear in 39.2% of journals. 58.5% of journals offer no clear information on whether reviewer identities are revealed to authors. Around 75% of journals have no clear policy on co-reviewing, citation of preprints, and publication of reviewer identities. Information regarding practices of open peer review is even more scarce, with <20% of journals providing clear information. Having found a lack of clear information, we conclude by examining the implications this has for researchers (especially early career) and the spread of open research practices.