Evaluating the reproducibility of research in obstetrics and gynecology

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

OBJECTIVES

To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability.

DESIGN

Systematic review with meta-analysis of individual participant data.

DATA SOURCES

Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022.

REVIEW METHODS

Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis.

RESULTS

The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively.

CONCLUSIONS

The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework doi:10.17605/OSF.IO/7SX8U.

How many US obstetrical trials reach publication? A cross-sectional analysis of ClinicalTrials.gov and PubMed from 2007 to 2019

BACKGROUND

Obstetrical clinical trials are the foundation of evidence-based medicine during pregnancy. As more obstetrical trials are conducted, understanding the publication characteristics of these trials is of utmost importance to advance obstetrical health.

OBJECTIVE

This study aimed to characterize the frequency of publication and trial characteristics associated with publication among obstetrical clinical trials in the United States. We additionally sought to examine time from trial completion to publication.

STUDY DESIGN

This was a cross-sectional analysis of completed obstetrical trials with an intervention design and at least 1 site in the United States registered to ClinicalTrials.gov from 2007 to 2019. Trial characteristics were cross-referenced with PubMed to determine publication status up to 2021 using the National Clinical Trial identification number. Bivariable analyses were conducted to determine trial characteristics associated with publication. Multivariable logistic regression models controlling for prespecified covariates were generated to estimate the relationship between funding, primary purpose, and therapeutic foci with publication. Additional exploratory analyses of other trial characteristics were conducted. Time to publication was analyzed using Kaplan-Meier curves and Cox regression models.

RESULTS

Of the 1879 obstetrical trials with registered completion, a total of 575 (30.6%) had at least 1 site in the United States, were completed before October 1, 2019, and were included in this analysis. Between October 2007 and October 2019, fewer than two-thirds (N=348, 60.5%) of trials reached publication. Annual rates of publication ranged from 46.4% in 2018 to 70.0% in 2007. No difference was observed in publication by funding, primary purpose, or therapeutic foci (all P>.05). Trials with characteristics indicating high trial quality-including randomized allocation scheme, ≥50 participants enrolled, ≥2 sites, and presence of a data safety monitoring committee-had increased odds of publication compared with those without such characteristics (all P<.05). For example, studies with randomized allocation of intervention had 2-fold greater odds of publication than nonrandomized studies (adjusted odds ratio, 2.09; 95% confidence interval, 1.30-3.37). Studies with ≥150 participants had nearly 8-fold odds of publication (adjusted odds ratio, 7.90; 95% confidence interval, 3.78-17.49) relative to studies with <50 participants. Temporal analysis demonstrated variability in time to publication among obstetrical trials, with a median time of 20.1 months after trial completion, and with most trials that reached publication having been published by 40 months. No difference was observed in time to publication by funding, primary purpose, or therapeutic foci (all P>.05).

CONCLUSION

Publication of obstetrical trials remains suboptimal, with significant differences observed between trials with indicators of high quality and those without. Most trials that reach publication are published within 2 years of registered completion on ClinicalTrials.gov.