Over-promotion and caution in abstracts of preprints during the COVID-19 crisis

Reimagining bioRxiv and preprint servers as platforms for academic learning

A popular preprint server, bioRxiv, is important as a tool for increased visibility for life science research. If used properly, however, bioRxiv can also be an important tool for training, as it may expose trainees (degree-seeking students undertaking research or internships directly related to their field of study) to the peer review process. Here, we offer a comprehensive guide to using bioRxiv as a training tool, as well as offer suggestions for improvements in bioRxiv, including confusion that may be caused by bioRxiv articles appearing on PubMed.

Using an expert survey and user feedback to construct PRECHECK: A checklist to evaluate preprints on COVID-19 and beyond

Background

The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public's evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically evaluate preprints, on COVID-19 and beyond.

Methods

To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories.

Results

At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with 'superficial' and 'deep' evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists.

Conclusions

We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.

Linguistic positivity in soft and hard disciplines: temporal dynamics, disciplinary variation, and the relationship with research impact

Previous studies have investigated the use of positive/negative language in academic discourse, and have found a tendency toward using more positive language in academic writing. However, little is known about whether the features and dynamics of linguistic positivity vary across disciplines. In addition, the relationship between linguistic positivity and research impact deserves further evaluation. To address these issues, the present study investigated linguistic positivity in academic writing from a cross-disciplinary perspective. Based on a 111-million-word corpus of research article abstracts collected from the Web of Science, the study examined the diachronic trends of positive/negative language in eight academic disciplines, and explored the relationship between linguistic positivity and citation counts. The results demonstrated that the increase in linguistic positivity is a common phenomenon across the examined academic disciplines. In addition, hard disciplines showed a higher and faster-growing degree of linguistic positivity compared with soft disciplines. Last, a significant positive correlation was identified between citation counts and the degree of linguistic positivity. Reasons for the temporal dynamics and disciplinary variation of linguistic positivity were explored, and implications for the scientific community were discussed.

Does writing style affect gender differences in the research performance of articles?: An empirical study of BERT-based textual sentiment analysis

“Achieve gender equality and empower all women and girls” is essential to reduce gender disparity and improve the status of women. But it remains a challenge to narrow gender differences and improve gender equality in academic research. In this paper, we propose that the impact of articles is lower and writing style of articles is less positive when the article’s first author is female relative to male first authors, and writing style mediates this relationship. Focusing on the positive writing style, we attempt to contribute and explain the research on gender differences in research performance. We use BERT-based textual sentiment analysis to analyse 87 years of 9820 articles published in the top four marketing journals and prove our hypotheses. We also consider a set of control variables and conduct a set of robustness checks to ensure the robustness of our findings. We discuss the theoretical and managerial implications of our findings for researchers.

Is academic writing becoming more positive? A large-scale diachronic case study of Science research articles across 25 years

Academic writing is developing to be more positive. This linguistic positivity bias is confirmed in academic writing across disciplines and genres. The current research adopted sentiment analysis and examined the diachronic change in linguistic positivity in the full texts of 2,556 research articles published in Science in 25 years. The results showed that academic writing in research articles in the journal Science has become significantly more positive in the past 25 years. The findings of this study confirm linguistic positivity bias in academic writing based on empirical data from Science. Reasons for the increasingly positive language use in science articles might include the popularization of science, the growing number of researchers, and the difficulty of publishing in high-impact journals. Finally, this study discussed the implications of our findings for researchers, editors, and peer reviewers.