Data Sharing in Neurosurgery and Neurology Journals

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

OBJECTIVES

To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability.

DESIGN

Systematic review with meta-analysis of individual participant data.

DATA SOURCES

Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022.

REVIEW METHODS

Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis.

RESULTS

The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively.

CONCLUSIONS

The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework doi:10.17605/OSF.IO/7SX8U.

Open Data Policies among Library and Information Science Journals

Journal publishers play an important role in the open research data ecosystem. Through open data policies that include public data archiving mandates and data availability statements, journal publishers help promote transparency in research and wider access to a growing scholarly record. The library and information science (LIS) discipline has a unique relationship with both open data initiatives and academic publishing and may be well-positioned to adopt rigorous open data policies. This study examines the information provided on public-facing websites of LIS journals in order to describe the extent, and nature, of open data guidance provided to prospective authors. Open access journals in the discipline have disproportionately adopted detailed, strict open data policies. Commercial publishers, which account for the largest share of publishing in the discipline, have largely adopted weaker policies. Rigorous policies, adopted by a minority of journals, describe the rationale, application, and expectations for open research data, while most journals that provide guidance on the matter use hesitant and vague language. Recommendations are provided for strengthening journal open data policies.

Profile of Data Sharing in the Clinical Neurosciences

Importance: In today’s climate of high healthcare costs and limited research resources, much attention has been given to inefficiency in research. Open access to research data has been proposed as a way to pool resources and make the most of research funding while also promoting transparency and scientific rigor. 

Objective: The clinical neurosciences stand to benefit greatly from the potential opportunities afforded by open data, and we sought to evaluate the current state of publicly available research findings and data sharing policies within the clinical neurosciences. 

Design: The Clarivate Analytics Web of Science journal citation reports for 2017 were used to sort journals in the category ‘Clinical Neurosciences’ by impact factor. The top 50 journals were selected and reviewed, but data was only collected from journals focused on original research (42/50). For each journal we reviewed the 10 most recent original research articles for 2016, 2017, and 2018 as designated by Scopus. 

Results: A data sharing policy existed for 60% (25/42) of the journals reviewed. Of the articles studied 41% (517/1255) contained source data, and the amount of articles with available source data increased from 2016 to 2018. Of all the articles reviewed, 49.4% (620/1255) were open access. Overall, 6.9% (87/1255) of articles had their source data accessible outside of the manuscript (e.g. registries, databases, etc.) and 8.9% (112/1255) addressed the availability of their source data within the publication itself. The availability of source data outside the manuscript and in-article discussion of source data availability both increased from 2016 to 2018. Only 3.9% (49/1255) of articles reviewed reported negative results for their primary outcome, and 7.6% (95/1255) of the articles could not be defined as primarily reporting positive or negative findings (characterization studies, census reporting, etc.). The distribution of negative versus positive results reported showed no significant trend over the years studied. 

Conclusion and Relevance: Our results demonstrate an opportunity for increased data sharing in neuroscience original research. These findings also suggest a trend towards increased adoption of open data sharing policies among journals and increased availability of unprocessed data in publications. This can increase the quality and speed at which new research is developed in the clinical neurosciences.

Availability of Research Data in High-Impact Addiction Journals with Data Sharing Policies

Although data sharing is one of the primary measures proposed to improve the integrity and quality of published research, studies show it remains the exception not the rule. The current study examines the availability of data in papers reporting the results of analyses of empirical data from original research in high-impact addiction journals. Thirteen high-impact journals with data sharing policies were selected from those included in the substance abuse category of the 2018 Clarivate Analytics' Journal Citation Report. The first 10 full or short original research reports that included empirical data in the most recent complete issue of each journal were electronically searched and reviewed for reference to where their data can be obtained and for a formal data sharing statement. Only eight of the 130 papers contained a data sharing statement in their text or supplementary online materials, and just one contained a direct link to the data analyzed. Data sharing was rare in the 13 high-impact addiction journals reviewed. The nature of the data reported in addiction journals might partly explain this. Currently, data sharing is not a procedure likely to improve the quality and integrity of published addiction research.

Reproducible and transparent research practices in published neurology research

BACKGROUND

The objective of this study was to evaluate the nature and extent of reproducible and transparent research practices in neurology publications.

METHODS

The NLM catalog was used to identify MEDLINE-indexed neurology journals. A PubMed search of these journals was conducted to retrieve publications over a 5-year period from 2014 to 2018. A random sample of publications was extracted. Two authors conducted data extraction in a blinded, duplicate fashion using a pilot-tested Google form. This form prompted data extractors to determine whether publications provided access to items such as study materials, raw data, analysis scripts, and protocols. In addition, we determined if the publication was included in a replication study or systematic review, was preregistered, had a conflict of interest declaration, specified funding sources, and was open access.

RESULTS

Our search identified 223,932 publications meeting the inclusion criteria, from which 400 were randomly sampled. Only 389 articles were accessible, yielding 271 publications with empirical data for analysis. Our results indicate that 9.4% provided access to materials, 9.2% provided access to raw data, 0.7% provided access to the analysis scripts, 0.7% linked the protocol, and 3.7% were preregistered. A third of sampled publications lacked funding or conflict of interest statements. No publications from our sample were included in replication studies, but a fifth were cited in a systematic review or meta-analysis.

CONCLUSIONS

Currently, published neurology research does not consistently provide information needed for reproducibility. The implications of poor research reporting can both affect patient care and increase research waste. Collaborative intervention by authors, peer reviewers, journals, and funding sources is needed to mitigate this problem.