Is rapid scientific publication also high quality? Bibliometric analysis of highly disseminated COVID-19 research papers

Publication of observational studies making claims of causation over time

To examine methodology characteristics over time and investigate research impact before and after the start of the COVID-19 era, we analyzed original articles published in The New England Journal of Medicine between October 26, 2017 and August 27, 2022. April 1, 2020 was used as the defining date dividing before and after the COVID-19 era. Out of 1051 original articles, 515 (49 %) were before and 536 (51 %) were after the COVID-19 era. Two independent reviewers categorized and reconciled methodology into groups: "randomized trial" (715 articles), "uncontrolled experimental study" (128), "descriptive observational study" (168), and "observational study making a causal claim" (40). We extracted subsequent citations and Altmetric data for each article to assess impact. The median number of social media shares was 2272 (IQR: 743-7821) for observational studies making a causal conclusion, compared to 306 (IQR: 70-606) for randomized trials (p-value=<0.001). The median Altmetric score for randomized COVID-19 trials (2421, IQR: 1063-3920) was not significantly different than that of COVID-19 observational studies making a causal claim (2583, IQR: 1513-6197, p-value = 0.42), but it was significantly lower than descriptive observational COVID-19 studies (4093, IQR: 2545-6823, p-value = 0.04). We conclude that there has been a steady increase in the number and percentage of observational studies that make causal conclusions about the efficacy of an intervention. Research concerning COVID-19, regardless of methodology, has seen a sharp rise in dissemination as measured through Altmetric's social media score and subsequent citations.

Unraveling retraction dynamics in COVID-19 research: Patterns, reasons, and implications

Amid the COVID-19 pandemic, while the world sought solutions, few scholars exploited the situation for personal gains through deceptive studies and manipulated data. This paper presents the extent of 400 retracted COVID-19 papers listed by the RetractionWatch database until the month of February 2024. The primary purpose of the research was to analyze journal quality and retractions trends. Evaluating the journal's quality is vital for stakeholders, as it enables them to effectively address and prevent such incidents and their future repercussions. The present study found that one-fourth of publications were retracted within the first month of their publication, followed by an additional 6% within six months of publication. One third of the retractions originated from Q1 journals, with another significant portion coming from Q2 (29.8%). An analysis of the reasons for retractions indicates that a quarter of retractions were attributed to multiple causes, predominantly associated with publications in Q2 journals, while another quarter were linked to data issues, primarily observed in Q1 publications. Elsevier retracted 31% of papers, with the majority published as Q1, followed by Springer (11.5%), predominantly as Q2. The study also examined author contributions, revealing that 69.3% were male, with females (30.7%) mainly holding middle author positions.

Emerging Adult Resilience to the Early Stages of the COVID-Pandemic: A Systematic Scoping Review

Human resilience to COVID-19 related stressors remains a pressing concern following the aftereffects of the pandemic and in the face of probable future pandemics. In response, we systematically scoped the available literature (n = 2030 records) to determine the nature and extent of research on emerging adults' adaptive responses to COVID-19 stressors in the early stages of the pandemic. Using a multisystem resilience framework, our narrative review of 48 eligible studies unpacks the personal, relational, institutional and/or physical ecological resources that enabled positive emerging adult outcomes to COVID-18 stressors. We found that there is a geographical bias in studies on this topic, with majority world contexts poorly represented. Resources leading to positive outcomes foregrounded psychological and social support, while institutional and ecological supports were seldom mentioned. Multisystemic combinations of resources were rarely considered. This knowledge has valuable implications for understanding resilience in the context of other large-scale adverse conditions.

The impact of COVID-19 on open access publishing in radiology and nuclear medicine: an in-depth analysis

In response to the COVID-19 pandemic, numerous initiatives have been implemented to ensure open access availability of COVID-19-related articles to make published articles accessible for anyone. This study aimed to assess the impact of the COVID-19 pandemic on open-access publishing in radiology and nuclear medicine. We conducted a comprehensive analysis of articles and reviews published in these fields during the COVID-19 publishing era using the Web of Science database. We analyzed several indicators between COVID-19 and non-COVID-19 related articles, including the number and percentage of open-access articles, the top ten cited articles, and the number of reviews. In total, 67,100 articles were published in radiology and nuclear medicine between January 2020 and June 2022. Among those, more than half (51.1%) were open-access articles. Among these publications, 2,336 were COVID-19-related, and 64,764 were non-COVID-19-related. However, articles related to COVID-19 had an open access rate of 91.5%, compared to only 49.6% of the non-COVID-19-related articles. Moreover, COVID-19-related articles had a higher percentage of highly cited and hot papers compared to articles not related to COVID-19. Moreover, most highly cited studies were related to chest computerized tomography (CT) scan findings in COVID-19 patients. The findings emphasize the significant proportion of open access COVID-19-related publications in radiology and nuclear medicine, facilitating widespread and timely access to everyone.

A multimethod synthesis of Covid-19 education research: the tightrope between covidization and meaningfulness

This study offers a comprehensive analysis of COVID-19 research in education. A multi-methods approach was used to capture the full breadth of educational research. As such, a bibliometric analysis, structural topic modeling, and qualitative synthesis of top papers were combined. A total of 4,201 articles were retrieved from Scopus, mostly published from 2019 to 2021. In this work special attention is paid to analyzing and synthesizing findings about: (i) status of research about COVID-19 regarding frequencies, venues, publishing countries, (ii) identification of main topics in the COVID-19 research, and (iii) identification of the major themes in most cited articles and their impact on the educational community. Structural topic modeling identified three main groups of topics that related to education in general, moving to online education, or diverse topics (e.g., perceptions, inclusion, medical education, engagement and motivation, well-being, and equality). A deeper analysis of the papers that received most attention revealed that problem understanding was the dominating theme of papers, followed by challenges, impact, guidance, online migration, and tools and resources. A vast number of papers were produced. However, thoughtful, well-planned, and meaningful research was hard to conceptualize or implement, and a sense of urgency led to a deluge of research with thin contributions in a time of dire need to genuine insights.

Reliability of citations of medRxiv preprints in articles published on COVID-19 in the world leading medical journals

Introduction

Preprints have been widely cited during the COVID-19 pandemics, even in the major medical journals. However, since subsequent publication of preprint is not always mentioned in preprint repositories, some may be inappropriately cited or quoted. Our objectives were to assess the reliability of preprint citations in articles on COVID-19, to the rate of publication of preprints cited in these articles and to compare, if relevant, the content of the preprints to their published version.

Methods

Articles published on COVID in 2020 in the BMJ, The Lancet, the JAMA and the NEJM were manually screened to identify all articles citing at least one preprint from medRxiv. We searched PubMed, Google and Google Scholar to assess if the preprint had been published in a peer-reviewed journal, and when. Published articles were screened to assess if the title, data or conclusions were identical to the preprint version.

Results

Among the 205 research articles on COVID published by the four major medical journals in 2020, 60 (29.3%) cited at least one medRxiv preprint. Among the 182 preprints cited, 124 were published in a peer-reviewed journal, with 51 (41.1%) before the citing article was published online and 73 (58.9%) later. There were differences in the title, the data or the conclusion between the preprint cited and the published version for nearly half of them. MedRxiv did not mentioned the publication for 53 (42.7%) of preprints.

Conclusions

More than a quarter of preprints citations were inappropriate since preprints were in fact already published at the time of publication of the citing article, often with a different content. Authors and editors should check the accuracy of the citations and of the quotations of preprints before publishing manuscripts that cite them.

Comparison of citation rates between Covid-19 and non-Covid-19 articles across 24 major scientific journals

Covid-19 has been front and center in the global landscape since the beginning of 2020. In response, the scientific field has dedicated enormous amounts of resources to researching the virus and its effects. The number of times Covid-19 publications are being cited throughout the literature appears remarkably high but has not been directly compared to non-Covid-19 papers in the same journals over an extended period. In our study, we use Clarivate’s Web of Science—Science Citation Index Expanded™ database to identify Covid-19 papers published in 24 major scientific journals over a period of 24 months from January 1, 2020 to December 31, 2021. We conduct our search using keywords “Covid-19”, “coronavirus”, and “sars-cov-2” to locate publications with these words in the title. We then quantify the number of citations these papers have received and compare rates to non-Covid-19 papers in the same journals over the same timeframe. We find that, across 24 open-access and subscription-based scientific journals, Covid-19 papers published in the past 2 years currently have a median citation rate of 120.79 compared to 21.63 for non-Covid-19 papers. When negative binomial regression is used to minimize the influence of other variables such as article number variation and field of research, Covid-19 papers have still experienced more than 80% increase in citations relative to non-Covid-19 papers. These novel findings demonstrate that Covid-19 papers are being cited at remarkably higher rates than non-Covid-19 articles contained within the same journals. This suggests that journal impact factor, which is a product of the number of citations that recently published articles receive, will likely be drastically influenced by the number of Covid-19 papers that a journal has included within its pages in the previous years.

The Impact of COVID-19 on Primary Educational Publishers in Australia

Based on interviews with 10 professionals from primary educational publishers and educational technology companies based in Australia, this article examines the challenges and impact of COVID-19 on publishing operations and outputs, and the future of the sector. The publishers had to deliver digital materials quickly, effectively and often for free to assist educators with the transition to remote learning, while working remotely themselves. They also had to transfers sales, support and professional learning online. Overall, while operationally challenging, the pandemic has accelerated the demand for digital products and facilitated growth of the sector.

Massive covidization of research citations and the citation elite

The COVID-19 pandemic saw a massive mobilization of the scientific workforce. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021, finding that 20% of citations received to papers published in 2020 to 2021 were to COVID-19–related papers. Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19. A large number of scientists received large numbers of citations to their COVID-19 work, often exceeding the citations they had received to all their work during their entire career. We document a strong covidization of research citations across science. This may have major repercussions for research priorities and the evolution of research on COVID-19 and beyond.

Massive scientific productivity accompanied the COVID-19 pandemic. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021 and assessed the impact on scientist citation profiles. Using Scopus data until August 1, 2021, COVID-19 items accounted for 4% of papers published, 20% of citations received to papers published in 2020 to 2021, and >30% of citations received in 36 of the 174 disciplines of science (up to 79.3% in general and internal medicine). Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19; 110 scientists received ≥10,000 citations for COVID-19 work, but none received ≥10,000 citations for non–COVID-19 work published in 2020 to 2021. For many scientists, citations to their COVID-19 work already accounted for more than half of their total career citation count. Overall, these data show a strong covidization of research citations across science, with major impact on shaping the citation elite.

The reproducibility of COVID-19 data analysis: paradoxes, pitfalls, and future challenges

In the midst of the COVID-19 experience, we learned an important scientific lesson: knowledge acquisition and information quality in medicine depends more on "data quality" rather than "data quantity." The large number of COVID-19 reports, published in a very short time, demonstrated that the most advanced statistical and computational tools cannot properly overcome the poor quality of acquired data. The main evidence for this observation comes from the poor reproducibility of results. Indeed, understanding the data generation process is fundamental when investigating scientific questions such as prevalence, immunity, transmissibility, and susceptibility. Most of COVID-19 studies are case reports based on "non probability" sampling and do not adhere to the general principles of controlled experimental designs. Such collected data suffers from many limitations when used to derive clinical conclusions. These include confounding factors, measurement errors and bias selection effects. Each of these elements represents a source of uncertainty, which is often ignored or assumed to provide an unbiased random contribution. Inference retrieved from large data in medicine is also affected by data protection policies that, while protecting patients' privacy, are likely to reduce consistently usefulness of big data in achieving fundamental goals such as effective and efficient data-integration. This limits the degree of generalizability of scientific studies and leads to paradoxical and conflicting conclusions. We provide such examples from assessing the role of risks factors. In conclusion, new paradigms and new designs schemes are needed in order to reach inferential conclusions that are meaningful and informative when dealing with data collected during emergencies like COVID-19.

Exploring COVID-19 research papers published on journals in the field of LIS

As the COVID-19 pandemic prevails, research related to COVID-19 has spread beyond medicine, health science, and biology to almost all academic fields. Library and information science is one of the most active fields that publish COVID-19-related research papers. This study examined 696 research papers related to COVID-19 whose journal being categorized as “information science & library science” by Web of Science. The result of bibliometric analysis showed that the publications were active and on the rise. Most papers were published in English and produced in the United States. According to the keyword clustering map produced by semantic network analysis, two fields, bibliometrics and health communication, were publishing research papers related to COVID-19 most actively. Moreover, the most productive journal was a library and information science journal focusing on health informatics. Additionally, a tendency was found that researchers preferred to publish on journals with high impact factors. Compared with non-COVID-19-related research papers, there was a significant decrease of “time for acceptance” of COVID-19-related papers, and the proportion of open access was relatively high. Confronting the global crisis of COVID-19, the library and information science field also made efforts and challenges to resolve the slow peer-review, delayed publishing, and high paywalls, which have been recognized as a “chronic diseases” of the academic publishing ecosystem. It is expected that these endeavors can serve as a turning point to reconsider and innovate the traditional research-publishing lifecycle.

Exploring COVID-19 research credibility among Spanish scientists

Amidst a worldwide vaccination campaign, trust in science plays a significant role when addressing the COVID-19 pandemic. Given current concerns regarding research standards, we were interested in how Spanish scholars perceived COVID-19 research and the extent to which questionable research practices and potentially problematic academic incentives are commonplace. We asked researchers to evaluate the expected quality of their COVID-19 projects and other peers' research and compared these assessments with those from scholars not involved in COVID-19 research. We investigated self-admitting and estimated rates of questionable research practices and attitudes towards current research status. Responses from 131 researchers suggested that COVID-19 evaluations followed partisan lines, with scholars being more pessimistic about others' colleagues' research than their own. Additionally,researchers not involved in COVID-19 projects were more negative than their participating peers. These differences were particularly notable for areas such as the expected theoretical foundations or overall quality of the research, among others. Most Spanish scholars expected questionable research practices and inadequate incentives to be widespread. In these two aspects, researchers tended to agree regardless of their involvement in COVID-19 research. We provide specific recommendations for improving future meta-science studies, such as redefining QRPs as inadequate research practices (IRP). This change could help avoid key controversies regarding QRPs' definition while highlighting their detrimental impact. Lastly, we join previous calls to improve transparency and academic career incentives as a cornerstone for generating trust in science.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s12144-022-02797-6.

The rapid, massive growth of COVID-19 authors in the scientific literature

We examined the extent to which the scientific workforce in different fields was engaged in publishing COVID-19-related papers. According to Scopus (data cut, 1 August 2021), 210 183 COVID-19-related publications included 720 801 unique authors, of which 360 005 authors had published at least five full papers in their career and 23 520 authors were at the top 2% of their scientific subfield based on a career-long composite citation indicator. The growth of COVID-19 authors was far more rapid and massive compared with cohorts of authors historically publishing on H1N1, Zika, Ebola, HIV/AIDS and tuberculosis. All 174 scientific subfields had some specialists who had published on COVID-19. In 109 of the 174 subfields of science, at least one in 10 active, influential (top 2% composite citation indicator) authors in the subfield had authored something on COVID-19. Fifty-three hyper-prolific authors had already at least 60 (and up to 227) COVID-19 publications each. Among the 300 authors with the highest composite citation indicator for their COVID-19 publications, most common countries were USA (n = 67), China (n = 52), UK (n = 32) and Italy (n = 18). The rapid and massive involvement of the scientific workforce in COVID-19-related work is unprecedented and creates opportunities and challenges. There is evidence for hyper-prolific productivity.