How often do cancer researchers make their data and code available and what factors are associated with sharing?

Daily life in the Open Biologist's second job, as a Data Curator

Background

Data reusability is the driving force of the research data life cycle. However, implementing strategies to generate reusable data from the data creation to the sharing stages is still a significant challenge. Even when datasets supporting a study are publicly shared, the outputs are often incomplete and/or not reusable. The FAIR (Findable, Accessible, Interoperable, Reusable) principles were published as a general guidance to promote data reusability in research, but the practical implementation of FAIR principles in research groups is still falling behind. In biology, the lack of standard practices for a large diversity of data types, data storage and preservation issues, and the lack of familiarity among researchers are some of the main impeding factors to achieve FAIR data. Past literature describes biological curation from the perspective of data resources that aggregate data, often from publications.

Methods

Our team works alongside data-generating, experimental researchers so our perspective aligns with publication authors rather than aggregators. We detail the processes for organizing datasets for publication, showcasing practical examples from data curation to data sharing. We also recommend strategies, tools and web resources to maximize data reusability, while maintaining research productivity.

Conclusion

We propose a simple approach to address research data management challenges for experimentalists, designed to promote FAIR data sharing. This strategy not only simplifies data management, but also enhances data visibility, recognition and impact, ultimately benefiting the entire scientific community.

Sharing Is Caring? International Society for Pharmacoepidemiology Review and Recommendations for Sharing Programming Code

PURPOSE

There is increasing recognition of the importance of transparency and reproducibility in scientific research. This study aimed to quantify the extent to which programming code is publicly shared in pharmacoepidemiology, and to develop a set of recommendations on this topic.

METHODS

We conducted a literature review identifying all studies published in Pharmacoepidemiology and Drug Safety (PDS) between 2017 and 2022. Data were extracted on the frequency and types of programming code shared, and other key open science practices (clinical codelist sharing, data sharing, study preregistration, and stated use of reporting guidelines and preprinting). We developed six recommendations for investigators who choose to share code and gathered feedback from members of the International Society for Pharmacoepidemiology (ISPE).

RESULTS

Programming code sharing by articles published in PDS ranged from 1.8% in 2017 to 9.5% in 2022. It was more prevalent among articles with a methodological focus, simulation studies, and papers which also shared record-level data.

CONCLUSION

Programming code sharing is rare but increasing in pharmacoepidemiology studies published in PDS. We recommend improved reporting of whether code is shared and how available code can be accessed. When sharing programming code, we recommend the use of permanent digital identifiers, appropriate licenses, and, where possible, adherence to good software practices around the provision of metadata and documentation, computational reproducibility, and data privacy.

Overlooked poor-quality patient samples in sequencing data impair reproducibility of published clinically relevant datasets

BACKGROUND

Reproducibility is a major concern in biomedical studies, and existing publication guidelines do not solve the problem. Batch effects and quality imbalances between groups of biological samples are major factors hampering reproducibility. Yet, the latter is rarely considered in the scientific literature.

RESULTS

Our analysis uses 40 clinically relevant RNA-seq datasets to quantify the impact of quality imbalance between groups of samples on the reproducibility of gene expression studies. High-quality imbalance is frequent (14 datasets; 35%), and hundreds of quality markers are present in more than 50% of the datasets. Enrichment analysis suggests common stress-driven effects among the low-quality samples and highlights a complementary role of transcription factors and miRNAs to regulate stress response. Preliminary ChIP-seq results show similar trends. Quality imbalance has an impact on the number of differential genes derived by comparing control to disease samples (the higher the imbalance, the higher the number of genes), on the proportion of quality markers in top differential genes (the higher the imbalance, the higher the proportion; up to 22%) and on the proportion of known disease genes in top differential genes (the higher the imbalance, the lower the proportion). We show that removing outliers based on their quality score improves the resulting downstream analysis.

CONCLUSIONS

Thanks to a stringent selection of well-designed datasets, we demonstrate that quality imbalance between groups of samples can significantly reduce the relevance of differential genes, consequently reducing reproducibility between studies. Appropriate experimental design and analysis methods can substantially reduce the problem.

Journal requirement for data sharing statements in clinical trials: a cross-sectional study

OBJECTIVES

Data sharing statements are considered routine in clinical trial reporting and represent a step toward data transparency. The International Committee of Medical Journal Editors (ICMJE) required clinical trials to publish data sharing statements. We aimed to assess the requirement for data sharing statements of individual participant data by biomedical journals and explore associations between journal characteristics and journal requirements for data sharing statements.

STUDY DESIGN AND SETTING

In this cross-sectional study, we included all biomedical journals that published clinical trials from January 1, 2019, to December 31, 2022, and that were indexed by the Journal Citation Reports. The study outcome was the journal requirement for data sharing statements. Multivariable logistic regression analysis was used to assess the relationship between journal characteristics and requirement for data sharing statements.

RESULTS

Of the 3229 biomedical journals included in the analysis, 2345 (72.6%) required authors to include data sharing statements. Journals published in the UK (OR, 3.19 [95% CI, 2.43-4.22]) and endorsing the Consolidated Standards of Reporting Trials (OR, 3.30 [95% CI, 2.78-3.92]) had greater odds of requiring data sharing statements. Journals that were open access, non-English language, in the Journal Citation Reports group of clinical medicine, and on the ICMJE list had lower odds of requiring data sharing statements, with ORs ranging from 0.18 to 0.81.

CONCLUSION

Despite ICMJE recommendations, more than 27% of the biomedical journals that published clinical trials did not require clinical trials to include data sharing statements, highlighting room for improved transparency.

Cancer researchers' experiences with and perceptions of research data sharing: Results of a cross-sectional survey

BACKGROUND

Despite wide recognition of the benefits of sharing research data, public availability rates have not increased substantially in oncology or medicine more broadly over the last decade.

METHODS

We surveyed 285 cancer researchers to determine their prior experience with sharing data and views on known drivers and inhibitors.

RESULTS

We found that 45% of respondents had shared some data from their most recent empirical publication, with respondents who typically studied non-human research participants, or routinely worked with human genomic data, more likely to share than those who did not. A third of respondents added that they had previously shared data privately, with 74% indicating that doing so had also led to authorship opportunities or future collaborations for them. Journal and funder policies were reported to be the biggest general drivers toward sharing, whereas commercial interests, agreements with industrial sponsors and institutional policies were the biggest prohibitors. We show that researchers' decisions about whether to share data are also likely to be influenced by participants' desires.

CONCLUSIONS

Our survey suggests that increased promotion and support by research institutions, alongside greater championing of data sharing by journals and funders, may motivate more researchers in oncology to share their data.

Considerations for establishing and maintaining international research collaboration: the example of chemotherapy-induced peripheral neurotoxicity (CIPN)-a white paper

PURPOSE

This white paper provides guidance regarding the process for establishing and maintaining international collaborations to conduct oncology/neurology-focused chemotherapy-induced peripheral neurotoxicity (CIPN) research.

METHODS

An international multidisciplinary group of CIPN scientists, clinicians, research administrators, and legal experts have pooled their collective knowledge regarding recommendations for establishing and maintaining international collaboration to foster advancement of CIPN science.

RESULTS

Experts provide recommendations in 10 categories: (1) preclinical and (2) clinical research collaboration; (3) collaborators and consortiums; (4) communication; (5) funding; (6) international regulatory standards; (7) staff training; (8) data management, quality control, and data sharing; (9) dissemination across disciplines and countries; and (10) additional recommendations about feasibility, policy, and mentorship.

CONCLUSION

Recommendations to establish and maintain international CIPN research collaboration will promote the inclusion of more diverse research participants, increasing consideration of cultural and genetic factors that are essential to inform innovative precision medicine interventions and propel scientific discovery to benefit cancer survivors worldwide.

RELEVANCE TO INFORM RESEARCH POLICY

Our suggested guidelines for establishing and maintaining international collaborations to conduct oncology/neurology-focused chemotherapy-induced peripheral neurotoxicity (CIPN) research set forth a challenge to multinational science, clinical, and policy leaders to (1) develop simple, streamlined research designs; (2) address logistical barriers; (3) simplify and standardize regulatory requirements across countries; (4) increase funding to support international collaboration; and (5) foster faculty mentorship.

Open science practices need substantial improvement in prognostic model studies in oncology using machine learning

OBJECTIVE

To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology.

STUDY DESIGN AND SETTING

We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices.

RESULTS

We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs.

CONCLUSION

The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.

Prevalence and predictors of data and code sharing in the medical and health sciences: systematic review with meta-analysis of individual participant data

OBJECTIVES

To synthesise research investigating data and code sharing in medicine and health to establish an accurate representation of the prevalence of sharing, how this frequency has changed over time, and what factors influence availability.

DESIGN

Systematic review with meta-analysis of individual participant data.

DATA SOURCES

Ovid Medline, Ovid Embase, and the preprint servers medRxiv, bioRxiv, and MetaArXiv were searched from inception to 1 July 2021. Forward citation searches were also performed on 30 August 2022.

REVIEW METHODS

Meta-research studies that investigated data or code sharing across a sample of scientific articles presenting original medical and health research were identified. Two authors screened records, assessed the risk of bias, and extracted summary data from study reports when individual participant data could not be retrieved. Key outcomes of interest were the prevalence of statements that declared that data or code were publicly or privately available (declared availability) and the success rates of retrieving these products (actual availability). The associations between data and code availability and several factors (eg, journal policy, type of data, trial design, and human participants) were also examined. A two stage approach to meta-analysis of individual participant data was performed, with proportions and risk ratios pooled with the Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis.

RESULTS

The review included 105 meta-research studies examining 2 121 580 articles across 31 specialties. Eligible studies examined a median of 195 primary articles (interquartile range 113-475), with a median publication year of 2015 (interquartile range 2012-2018). Only eight studies (8%) were classified as having a low risk of bias. Meta-analyses showed a prevalence of declared and actual public data availability of 8% (95% confidence interval 5% to 11%) and 2% (1% to 3%), respectively, between 2016 and 2021. For public code sharing, both the prevalence of declared and actual availability were estimated to be <0.5% since 2016. Meta-regressions indicated that only declared public data sharing prevalence estimates have increased over time. Compliance with mandatory data sharing policies ranged from 0% to 100% across journals and varied by type of data. In contrast, success in privately obtaining data and code from authors historically ranged between 0% and 37% and 0% and 23%, respectively.

CONCLUSIONS

The review found that public code sharing was persistently low across medical research. Declarations of data sharing were also low, increasing over time, but did not always correspond to actual sharing of data. The effectiveness of mandatory data sharing policies varied substantially by journal and type of data, a finding that might be informative for policy makers when designing policies and allocating resources to audit compliance.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework doi:10.17605/OSF.IO/7SX8U.