1
|
do Nascimento YY, de Toledo MA, Pasqui DM, Pacheco RL, Riera R. Primary outcomes and characteristics of clinical trial registries (up to October 2021) on COVID-19 vaccines. J Eval Clin Pract 2024. [PMID: 38962836 DOI: 10.1111/jep.14082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Revised: 05/12/2024] [Accepted: 06/22/2024] [Indexed: 07/05/2024]
Abstract
OBJECTIVES To analyse the general and primary outcome-related characteristics of clinical trials protocols on COVID-19 vaccines. STUDY DESIGN AND SETTING A meta-research study. A search for clinical trial protocols on COVID-19 vaccines was conducted on the ClinicalTrials.gov platform. We considered all protocols of comparative trials registered up to October 26, 2021. RESULTS Two hundred and eighty-two trials were analysed. The median expected trial duration was 445 days (interquartile range [IQR] = 225), and the median target sample size was 420 participants (IQR = 1638). A retrospective registry (after the start date) was observed for 42.55% of the trials. Randomization procedures were planned by 84.75% and full-blinding procedures by 34.75% of the 282 trials. Most trials were labelled as active or still recruiting, and 14 trials (5%) were completed. None of the 14 trials labelled as completed on our search date had results available. Industry funding was reported by 198 trials (70.2%). Most studies declared more than one primary outcome, usually a safety or immunogenicity outcome, and 59 studies (20.9%) had at least one primary efficacy outcome. The description of the primary efficacy outcomes was limited in most cases, referred to as a non-specified 'efficacy' outcome (18.6%) or described as 'COVID-19 cases' (32.2%). CONCLUSION the primary outcomes of clinical trials on COVID-19 vaccines are poorly described, and the registers provide insufficient information about them. The registry was retrospectively fulfilled for many trials, which may lead to bias and research waste. Outcomes were generically described and did not provide transparent information for replication in practice, further trials or meta-analyses.
Collapse
Affiliation(s)
| | | | | | - Rafael L Pacheco
- Hospital Sírio-Libanês (HSL), São Paulo, SP, Brazil
- Centro Universitário São Camilo (CUSC), São Paulo, SP, Brazil
- Oxford-Brazil EBM Alliance, Brazil
| | - Rachel Riera
- Escola Paulista de Medicina, Universidade Federal de São Paulo (Unifesp), São Paulo, SP, Brazil
- Hospital Sírio-Libanês (HSL), São Paulo, SP, Brazil
- Oxford-Brazil EBM Alliance, Brazil
| |
Collapse
|
2
|
Turoman N, Heyard R, Schwab S, Furrer E, Vergauwe E, Held L. Using an expert survey and user feedback to construct PRECHECK: A checklist to evaluate preprints on COVID-19 and beyond. F1000Res 2024; 12:588. [PMID: 38983445 PMCID: PMC11231630 DOI: 10.12688/f1000research.129814.3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 05/30/2024] [Indexed: 07/11/2024] Open
Abstract
Background The quality of COVID-19 preprints should be considered with great care, as their contents can influence public policy. Surprisingly little has been done to calibrate the public's evaluation of preprints and their contents. The PRECHECK project aimed to generate a tool to teach and guide scientifically literate non-experts to critically evaluate preprints, on COVID-19 and beyond. Methods To create a checklist, we applied a four-step procedure consisting of an initial internal review, an external review by a pool of experts (methodologists, meta-researchers/experts on preprints, journal editors, and science journalists), a final internal review, and a Preliminary implementation stage. For the external review step, experts rated the relevance of each element of the checklist on five-point Likert scales, and provided written feedback. After each internal review round, we applied the checklist on a small set of high-quality preprints from an online list of milestone research works on COVID-19 and low-quality preprints, which were eventually retracted, to verify whether the checklist can discriminate between the two categories. Results At the external review step, 26 of the 54 contacted experts responded. The final checklist contained four elements (Research question, study type, transparency and integrity, and limitations), with 'superficial' and 'deep' evaluation levels. When using both levels, the checklist was effective at discriminating a small set of high- and low-quality preprints. Its usability for assessment and discussion of preprints was confirmed in workshops with Bachelors students in Psychology and Medicine, and science journalists. Conclusions We created a simple, easy-to-use tool for helping scientifically literate non-experts navigate preprints with a critical mind and facilitate discussions within, for example, a beginner-level lecture on research methods. We believe that our checklist has potential to help guide decisions about the quality of preprints on COVID-19 in our target audience and that this extends beyond COVID-19.
Collapse
Affiliation(s)
- Nora Turoman
- Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland
| | - Rachel Heyard
- Center for Reproducible Science (CRS), University of Zurich, Zurich, Switzerland
- Department of Biostatistics at the Epidemiology Biostatistics and Prevention Institute (EPBI), University of Zurich, Zurich, Switzerland
| | - Simon Schwab
- Center for Reproducible Science (CRS), University of Zurich, Zurich, Switzerland
- Department of Biostatistics at the Epidemiology Biostatistics and Prevention Institute (EPBI), University of Zurich, Zurich, Switzerland
| | - Eva Furrer
- Center for Reproducible Science (CRS), University of Zurich, Zurich, Switzerland
- Department of Biostatistics at the Epidemiology Biostatistics and Prevention Institute (EPBI), University of Zurich, Zurich, Switzerland
| | - Evie Vergauwe
- Faculty of Psychology and Educational Sciences, University of Geneva, Geneva, Switzerland
- Geneva University Neurocenter, University of Geneva, Geneva, Switzerland
| | - Leonhard Held
- Center for Reproducible Science (CRS), University of Zurich, Zurich, Switzerland
- Department of Biostatistics at the Epidemiology Biostatistics and Prevention Institute (EPBI), University of Zurich, Zurich, Switzerland
| |
Collapse
|
3
|
Malinda RR, Mishra D, Bajaj R, Khaliduzzaman A. Exploring the current dynamics of preprints. Curr Med Res Opin 2024:1-5. [PMID: 38700241 DOI: 10.1080/03007995.2024.2351144] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Accepted: 04/30/2024] [Indexed: 05/05/2024]
Abstract
Preprints are non-peer-reviewed and publicly available articles for open and transparent research communication. Preprint servers host the submission of such manuscripts, and despite the presence of established preprint servers, their numbers have continued to rise in recent times. A steep increasing pattern in posted preprints and their accommodating servers has been observed over the last decade. In this article, we explored the global trends in the preprint adoption and its involvement in promoting open and transparent research findings across various domains. We further emphasized the importance of preprinting, highlighting its significant impact during the pandemic through effective information sharing, and advocating for its broader integration in scholarly communication.
Collapse
Affiliation(s)
- Raj Rajeshwar Malinda
- Indraprastha Institute of Information Technology Delhi, New Delhi, India
- University of Hyogo, Kobe, Hyogo Prefecture, Japan
| | | | - Ruchika Bajaj
- Cardiovascular Research Institute, University of California San Francisco, San Francisco, California, USA
| | - Alin Khaliduzzaman
- Department of Agricultural and Biological Engineering, University of Illinois at Urbana-Champaign, Urbana-Champaign, Illinois, USA
| |
Collapse
|
4
|
Iarkaeva A, Nachev V, Bobrov E. Workflow for detecting biomedical articles with underlying open and restricted-access datasets. PLoS One 2024; 19:e0302787. [PMID: 38718077 PMCID: PMC11078384 DOI: 10.1371/journal.pone.0302787] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 04/11/2024] [Indexed: 05/12/2024] Open
Abstract
To monitor the sharing of research data through repositories is increasingly of interest to institutions and funders, as well as from a meta-research perspective. Automated screening tools exist, but they are based on either narrow or vague definitions of open data. Where manual validation has been performed, it was based on a small article sample. At our biomedical research institution, we developed detailed criteria for such a screening, as well as a workflow which combines an automated and a manual step, and considers both fully open and restricted-access data. We use the results for an internal incentivization scheme, as well as for a monitoring in a dashboard. Here, we describe in detail our screening procedure and its validation, based on automated screening of 11035 biomedical research articles, of which 1381 articles with potential data sharing were subsequently screened manually. The screening results were highly reliable, as witnessed by inter-rater reliability values of ≥0.8 (Krippendorff's alpha) in two different validation samples. We also report the results of the screening, both for our institution and an independent sample from a meta-research study. In the largest of the three samples, the 2021 institutional sample, underlying data had been openly shared for 7.8% of research articles. For an additional 1.0% of articles, restricted-access data had been shared, resulting in 8.3% of articles overall having open and/or restricted-access data. The extraction workflow is then discussed with regard to its applicability in different contexts, limitations, possible variations, and future developments. In summary, we present a comprehensive, validated, semi-automated workflow for the detection of shared research data underlying biomedical article publications.
Collapse
Affiliation(s)
- Anastasiia Iarkaeva
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH) at Charité –Universitätsmedizin Berlin, Berlin, Germany
| | - Vladislav Nachev
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH) at Charité –Universitätsmedizin Berlin, Berlin, Germany
| | - Evgeny Bobrov
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH) at Charité –Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
5
|
Lee JS, Tyler ARB, Veinot TC, Yakel E. Now Is the Time to Strengthen Government-Academic Data Infrastructures to Jump-Start Future Public Health Crisis Response. JMIR Public Health Surveill 2024; 10:e51880. [PMID: 38656780 PMCID: PMC11079773 DOI: 10.2196/51880] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 02/24/2024] [Accepted: 03/05/2024] [Indexed: 04/26/2024] Open
Abstract
During public health crises, the significance of rapid data sharing cannot be overstated. In attempts to accelerate COVID-19 pandemic responses, discussions within society and scholarly research have focused on data sharing among health care providers, across government departments at different levels, and on an international scale. A lesser-addressed yet equally important approach to sharing data during the COVID-19 pandemic and other crises involves cross-sector collaboration between government entities and academic researchers. Specifically, this refers to dedicated projects in which a government entity shares public health data with an academic research team for data analysis to receive data insights to inform policy. In this viewpoint, we identify and outline documented data sharing challenges in the context of COVID-19 and other public health crises, as well as broader crisis scenarios encompassing natural disasters and humanitarian emergencies. We then argue that government-academic data collaborations have the potential to alleviate these challenges, which should place them at the forefront of future research attention. In particular, for researchers, data collaborations with government entities should be considered part of the social infrastructure that bolsters their research efforts toward public health crisis response. Looking ahead, we propose a shift from ad hoc, intermittent collaborations to cultivating robust and enduring partnerships. Thus, we need to move beyond viewing government-academic data interactions as 1-time sharing events. Additionally, given the scarcity of scholarly exploration in this domain, we advocate for further investigation into the real-world practices and experiences related to sharing data from government sources with researchers during public health crises.
Collapse
Affiliation(s)
- Jian-Sin Lee
- School of Information, University of Michigan, Ann Arbor, MI, United States
| | | | - Tiffany Christine Veinot
- School of Information, University of Michigan, Ann Arbor, MI, United States
- Department of Health Behavior and Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, United States
- Department of Learning Health Sciences, School of Medicine, University of Michigan, Ann Arbor, MI, United States
| | - Elizabeth Yakel
- School of Information, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
6
|
Blatch-Jones AJ, Recio Saucedo A, Giddins B. The use and acceptability of preprints in health and social care settings: A scoping review. PLoS One 2023; 18:e0291627. [PMID: 37713422 PMCID: PMC10503772 DOI: 10.1371/journal.pone.0291627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 09/04/2023] [Indexed: 09/17/2023] Open
Abstract
BACKGROUND Preprints are open and accessible scientific manuscript or report that is shared publicly, through a preprint server, before being submitted to a journal. The value and importance of preprints has grown since its contribution during the public health emergency of the COVID-19 pandemic. Funders and publishers are establishing their position on the use of preprints, in grant applications and publishing models. However, the evidence supporting the use and acceptability of preprints varies across funders, publishers, and researchers. The scoping review explored the current evidence on the use and acceptability of preprints in health and social care settings by publishers, funders, and the research community throughout the research lifecycle. METHODS A scoping review was undertaken with no study or language limits. The search strategy was limited to the last five years (2017-2022) to capture changes influenced by COVID-19 (e.g., accelerated use and role of preprints in research). The review included international literature, including grey literature, and two databases were searched: Scopus and Web of Science (24 August 2022). RESULTS 379 titles and abstracts and 193 full text articles were assessed for eligibility. Ninety-eight articles met eligibility criteria and were included for full extraction. For barriers and challenges, 26 statements were grouped under four main themes (e.g., volume/growth of publications, quality assurance/trustworthiness, risks associated to credibility, and validation). For benefits and value, 34 statements were grouped under six themes (e.g., openness/transparency, increased visibility/credibility, open review process, open research, democratic process/systems, increased productivity/opportunities). CONCLUSIONS Preprints provide opportunities for rapid dissemination but there is a need for clear policies and guidance from journals, publishers, and funders. Cautionary measures are needed to maintain the quality and value of preprints, paying particular attention to how findings are translated to the public. More research is needed to address some of the uncertainties addressed in this review.
Collapse
Affiliation(s)
- Amanda Jane Blatch-Jones
- National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom
| | - Alejandra Recio Saucedo
- National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom
| | - Beth Giddins
- National Institute for Health and Care Research (NIHR) Coordinating Centre, School of Healthcare Enterprise and Innovation, University of Southampton, Southampton, Hampshire, United Kingdom
| |
Collapse
|
7
|
Sarkis-Onofre R, Sofi-Mahmudi A, Puljak L, Moraes RR. The importance of meta-research in dentistry. Evid Based Dent 2023; 24:98-99. [PMID: 37737329 DOI: 10.1038/s41432-023-00880-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/23/2023]
Affiliation(s)
| | - Ahmad Sofi-Mahmudi
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Livia Puljak
- Center for Evidence-Based Medicine, Catholic University of Croatia, Zagreb, Croatia
| | - Rafael R Moraes
- School of Dentistry, Universidade Federal de Pelotas, Pelotas, RS, Brazil
| |
Collapse
|
8
|
Sofi-Mahmudi A, Raittio E, Uribe SE. Transparency of COVID-19-related research: A meta-research study. PLoS One 2023; 18:e0288406. [PMID: 37494359 PMCID: PMC10370694 DOI: 10.1371/journal.pone.0288406] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 06/26/2023] [Indexed: 07/28/2023] Open
Abstract
BACKGROUND We aimed to assess the adherence to five transparency practices (data availability, code availability, protocol registration and conflicts of interest (COI), and funding disclosures) from open access Coronavirus disease 2019 (COVID-19) related articles. METHODS We searched and exported all open access COVID-19-related articles from PubMed-indexed journals in the Europe PubMed Central database published from January 2020 to June 9, 2022. With a validated and automated tool, we detected transparent practices of three paper types: research articles, randomized controlled trials (RCTs), and reviews. Basic journal- and article-related information were retrieved from the database. We used R for the descriptive analyses. RESULTS The total number of articles was 258,678, of which we were able to retrieve full texts of 186,157 (72%) articles from the database Over half of the papers (55.7%, n = 103,732) were research articles, 10.9% (n = 20,229) were review articles, and less than one percent (n = 1,202) were RCTs. Approximately nine-tenths of articles (in all three paper types) had a statement to disclose COI. Funding disclosure (83.9%, confidence interval (CI): 81.7-85.8 95%) and protocol registration (53.5%, 95% CI: 50.7-56.3) were more frequent in RCTs than in reviews or research articles. Reviews shared data (2.5%, 95% CI: 2.3-2.8) and code (0.4%, 95% CI: 0.4-0.5) less frequently than RCTs or research articles. Articles published in 2022 had the highest adherence to all five transparency practices. Most of the reviews (62%) and research articles (58%) adhered to two transparency practices, whereas almost half of the RCTs (47%) adhered to three practices. There were journal- and publisher-related differences in all five practices, and articles that did not adhere to transparency practices were more likely published in lowest impact journals and were less likely cited. CONCLUSION While most articles were freely available and had a COI disclosure, adherence to other transparent practices was far from acceptable. A much stronger commitment to open science practices, particularly to protocol registration, data and code sharing, is needed from all stakeholders.
Collapse
Affiliation(s)
- Ahmad Sofi-Mahmudi
- National Pain Centre, Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
- Seqiz Health Network, Kurdistan University of Medical Sciences, Seqiz, Kurdistan
| | - Eero Raittio
- Institute of Dentistry, University of Eastern Finland, Kuopio, Finland
- Department of Dentistry and Oral Health, Aarhus University, Aarhus, Denmark
| | - Sergio E Uribe
- Department of Conservative Dentistry and Oral Health, Riga Stradins University, Riga, Latvia
- School of Dentistry, Universidad Austral de Chile, Valdivia, Chile
- Baltic Biomaterials Centre of Excellence, Headquarters at Riga Technical University, Riga, Latvia
| |
Collapse
|
9
|
Hamilton DG, Hong K, Fraser H, Rowhani-Farid A, Fidler F, Page MJ. Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data. BMJ 2023; 382:e075767. [PMID: 37433624 DOI: 10.1136/bmj-2023-075767] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/13/2023]
Abstract
OBJECTIVES To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability. DESIGN Systematic review with meta-analysis of individual participant data. DATA SOURCES Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022. REVIEW METHODS Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis. RESULTS The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively. CONCLUSIONS The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance. SYSTEMATIC REVIEW REGISTRATION Open Science Framework doi:10.17605/OSF.IO/7SX8U.
Collapse
Affiliation(s)
- Daniel G Hamilton
- MetaMelb Research Group, School of BioSciences, University of Melbourne, Melbourne, VIC, Australia
- Melbourne Medical School, Faculty of Medicine, Dentistry, and Health Sciences, University of Melbourne, Melbourne, VIC, Australia
| | - Kyungwan Hong
- Department of Practice, Sciences, and Health Outcomes Research, University of Maryland School of Pharmacy, Baltimore, MD, USA
| | - Hannah Fraser
- MetaMelb Research Group, School of BioSciences, University of Melbourne, Melbourne, VIC, Australia
| | - Anisa Rowhani-Farid
- Department of Practice, Sciences, and Health Outcomes Research, University of Maryland School of Pharmacy, Baltimore, MD, USA
| | - Fiona Fidler
- MetaMelb Research Group, School of BioSciences, University of Melbourne, Melbourne, VIC, Australia
- School of Historical and Philosophical Studies, University of Melbourne, Melbourne, VIC, Australia
| | - Matthew J Page
- Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
10
|
Zhang L, Ma L. Is open science a double-edged sword?: data sharing and the changing citation pattern of Chinese economics articles. Scientometrics 2023; 128:2803-2818. [PMID: 37101973 PMCID: PMC10028759 DOI: 10.1007/s11192-023-04684-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 03/05/2023] [Indexed: 03/24/2023]
Abstract
Data sharing is an important part of open science (OS), and more and more institutions and journals have been enforcing open data (OD) policies. OD is advocated to help increase academic influences and promote scientific discovery and development, but such a proposition has not been elaborated on well. This study explores the nuanced effects of the OD policies on the citation pattern of articles by using the case of Chinese economics journals. China Industrial Economics (CIE) is the first and only Chinese social science journal so far to adopt a compulsory OD policy, requiring all published articles to share original data and processing codes. We use the article-level data and difference-in-differences (DID) approach to compare the citation performance of articles published in CIE and 36 comparable journals. Firstly, we find that the OD policy quickly increased the number of citations, and each article on average received 0.25, 1.19, 0.86, and 0.44 more citations in the first four years after publication respectively. Furthermore, we also found that the citation benefit of the OD policy rapidly decreased over time, and even became negative in the fifth year after publication. In conclusion, this changing citation pattern suggests that an OD policy can be double edged sword, which can quickly increase citation performance but simultaneously accelerate the aging of articles.
Collapse
Affiliation(s)
- Liwei Zhang
- grid.27255.370000 0004 1761 1174School of Innovation and Entrepreneurship, Shandong University, 72 Binhai Road, Jimo District, Qingdao, Shandong Province 266237 China
| | - Liang Ma
- grid.24539.390000 0004 0368 8103School of Public Administration and Policy, Renmin University of China, 59 Zhongguancun Avenue, Haidian District, Beijing, 100872 China
| |
Collapse
|
11
|
Wang P, Yang H, Hou J, Li Q. A machine learning approach to primacy-peak-recency effect-based satisfaction prediction. Inf Process Manag 2023. [DOI: 10.1016/j.ipm.2022.103196] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
12
|
Zeng L. Changes in health communication in the age of COVID-19: A study on the dissemination of preprints to the public. Front Public Health 2023; 11:1078115. [PMID: 36844813 PMCID: PMC9944950 DOI: 10.3389/fpubh.2023.1078115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 01/05/2023] [Indexed: 02/10/2023] Open
Abstract
Introduction Preprints have become an important tool for meeting the challenges of health communication in the context of COVID-19. They allow scientists to disseminate their results more quickly due to the absence of a peer review process. Preprints have been well-received by scientists, however, there have been concerns about the exposure of wider public audiences to preprints due in part to this lack of peer review. Methods The aim of this study is to examine the dissemination of preprints on medRxiv and bioRxiv during the COVID-19 pandemic using content analysis and statistical analysis. Results Our findings show that preprints have played an unprecedented role in disseminating COVID-19-related science results to the public. Discussion While the overall media coverage of preprints is unsatisfactory, digital native news media performed better than legacy media in reporting preprints, which means that we could make the most of digital native media to improve health communication. This study contributes to understanding how science communication has evolved in response to the COVID-19 pandemic and provides some practical recommendations.
Collapse
|
13
|
Gabelica M, Bojčić R, Puljak L. Many researchers were not compliant with their published data sharing statement: a mixed-methods study. J Clin Epidemiol 2022; 150:33-41. [PMID: 35654271 DOI: 10.1016/j.jclinepi.2022.05.019] [Citation(s) in RCA: 64] [Impact Index Per Article: 32.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Revised: 04/12/2022] [Accepted: 05/24/2022] [Indexed: 11/30/2022]
Abstract
OBJECTIVES The objective of the study was to analyze researchers' compliance with their data availability statement (DAS) from manuscripts published in open-access journals with the mandatory DAS. STUDY DESIGN AND SETTING We analyzed all articles from 333 open-access journals published during January 2019 by BioMed Central. We categorized types of the DAS. We surveyed corresponding authors who wrote in the DAS that they would share the data. Consent to participate in the study was sought for all included manuscripts. After accessing raw data sets, we checked whether data were available in a way that enabled reanalysis. RESULTS Of 3556 analyzed articles, 3416 contained the DAS. The most frequent DAS category (42%) indicated that the data sets are available on reasonable request. Among 1792 manuscripts in which the DAS indicated that authors are willing to share their data, 1669 (93%) authors either did not respond or declined to share their data with us. Among 254 (14%) of 1792 authors who responded to our query for data sharing, only 123 (6.8%) provided the requested data. CONCLUSION Even when authors indicate in their manuscript that they will share data upon request, the compliance rate is the same as for authors who do not provide the DAS, suggesting that the DAS may not be sufficient to ensure data sharing.
Collapse
Affiliation(s)
- Mirko Gabelica
- Department for otorhinolaryngology, with head and neck surgery, University Hospital Centre Split, Spinčićeva 1, 21000, Split, Croatia
| | - Ružica Bojčić
- Institute of Emergency Medicine of Karlovac County, Ul. Dr. Vladka Mačeka 48, 47000, Karlovac, Croatia
| | - Livia Puljak
- Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000, Zagreb, Croatia.
| |
Collapse
|
14
|
Kapp P, Esmail L, Ghosn L, Ravaud P, Boutron I. Transparency and reporting characteristics of COVID-19 randomized controlled trials. BMC Med 2022; 20:363. [PMID: 36154932 PMCID: PMC9510360 DOI: 10.1186/s12916-022-02567-y] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/19/2022] [Accepted: 09/13/2022] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND In the context of the COVID-19 pandemic, randomized controlled trials (RCTs) are essential to support clinical decision-making. We aimed (1) to assess and compare the reporting characteristics of RCTs between preprints and peer-reviewed publications and (2) to assess whether reporting improves after the peer review process for all preprints subsequently published in peer-reviewed journals. METHODS We searched the Cochrane COVID-19 Study Register and L·OVE COVID-19 platform to identify all reports of RCTs assessing pharmacological treatments of COVID-19, up to May 2021. We extracted indicators of transparency (e.g., trial registration, data sharing intentions) and assessed the completeness of reporting (i.e., some important CONSORT items, conflict of interest, ethical approval) using a standardized data extraction form. We also identified paired reports published in preprint and peer-reviewed publications. RESULTS We identified 251 trial reports: 121 (48%) were first published in peer-reviewed journals, and 130 (52%) were first published as preprints. Transparency was poor. About half of trials were prospectively registered (n = 140, 56%); 38% (n = 95) made their full protocols available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was reported in 68% (n = 170) of the reports of which 91% stated their willingness to share. Completeness of reporting was low: only 32% (n = 81) of trials completely defined the pre-specified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported the results for the primary outcomes while only 14% (n = 36) of trials adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% (n = 104) of trials; of them, only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as preprints, 78 were subsequently published in a peer-reviewed journal. There was no major improvement after the journal peer review process for most items. CONCLUSIONS Transparency, completeness, and consistency of reporting of COVID-19 clinical trials were insufficient both in preprints and peer-reviewed publications. A comparison of paired reports published in preprint and peer-reviewed publication did not indicate major improvement.
Collapse
Affiliation(s)
- Philipp Kapp
- Université Paris Cité, Inserm, INRAE, Centre of Research in Epidemiology and Statistics (CRESS), F-75004, Paris, France.,Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France.,Cochrane France, F-75004, Paris, France.,Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, D-79110, Freiburg, Germany
| | - Laura Esmail
- Université Paris Cité, Inserm, INRAE, Centre of Research in Epidemiology and Statistics (CRESS), F-75004, Paris, France.,Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France.,Cochrane France, F-75004, Paris, France
| | - Lina Ghosn
- Université Paris Cité, Inserm, INRAE, Centre of Research in Epidemiology and Statistics (CRESS), F-75004, Paris, France.,Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France.,Cochrane France, F-75004, Paris, France
| | - Philippe Ravaud
- Université Paris Cité, Inserm, INRAE, Centre of Research in Epidemiology and Statistics (CRESS), F-75004, Paris, France.,Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France.,Cochrane France, F-75004, Paris, France
| | - Isabelle Boutron
- Université Paris Cité, Inserm, INRAE, Centre of Research in Epidemiology and Statistics (CRESS), F-75004, Paris, France. .,Centre d'Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France. .,Cochrane France, F-75004, Paris, France.
| |
Collapse
|