Implementing clinical trial data sharing requires training a new generation of biomedical researchers

Identifying and Minimizing Incentives for Competing Interests in Sports Medicine Publications

Academics in sports medicine as well as other medical fields are generally expected to publish research and opinions in peer-reviewed journals. The peer-review process is intended to protect against the publication of flawed research and unsubstantiated claims. However, both financial and non-financial competing interests may result in sub-optimal results by affecting investigators, editors, peer reviewers, academic institutions, and publishers. In this article, we focus on the non-financial competing interests created in our current academic system. Because these competing interests are embedded in our current scholastic framework, the potential biases are difficult to quantify. To minimize the effect of these competing interests, we review and highlight some underlying incentives for each stakeholder and some potential solutions to mitigate their effects.

Best practices for data management and sharing in experimental biomedical research

Effective data management is crucial for scientific integrity and reproducibility, a cornerstone of scientific progress. Well-organized and well-documented data enable validation and building on results. Data management encompasses activities including organization, documentation, storage, sharing, and preservation. Robust data management establishes credibility, fostering trust within the scientific community and benefiting researchers' careers. In experimental biomedicine, comprehensive data management is vital due to the typically intricate protocols, extensive metadata, and large datasets. Low-throughput experiments, in particular, require careful management to address variations and errors in protocols and raw data quality. Transparent and accountable research practices rely on accurate documentation of procedures, data collection, and analysis methods. Proper data management ensures long-term preservation and accessibility of valuable datasets. Well-managed data can be revisited, contributing to cumulative knowledge and potential new discoveries. Publicly funded research has an added responsibility for transparency, resource allocation, and avoiding redundancy. Meeting funding agency expectations increasingly requires rigorous methodologies, adherence to standards, comprehensive documentation, and widespread sharing of data, code, and other auxiliary resources. This review provides critical insights into raw and processed data, metadata, high-throughput versus low-throughput datasets, a common language for documentation, experimental and reporting guidelines, efficient data management systems, sharing practices, and relevant repositories. We systematically present available resources and optimal practices for wide use by experimental biomedical researchers.

The REporting of A Disproportionality Analysis for DrUg Safety Signal Detection Using Individual Case Safety Reports in PharmacoVigilance (READUS-PV): Explanation and Elaboration

In pharmacovigilance, disproportionality analyses based on individual case safety reports are widely used to detect safety signals. Unfortunately, publishing disproportionality analyses lacks specific guidelines, often leading to incomplete and ambiguous reporting, and carries the risk of incorrect conclusions when data are not placed in the correct context. The REporting of A Disproportionality analysis for drUg Safety signal detection using individual case safety reports in PharmacoVigilance (READUS-PV) statement was developed to address this issue by promoting transparent and comprehensive reporting of disproportionality studies. While the statement paper explains in greater detail the procedure followed to develop these guidelines, with this explanation paper we present the 14 items retained for READUS-PV guidelines, together with an in-depth explanation of their rationale and bullet points to illustrate their practical implementation. Our primary objective is to foster the adoption of the READUS-PV guidelines among authors, editors, peer reviewers, and readers of disproportionality analyses. Enhancing transparency, completeness, and accuracy of reporting, as well as proper interpretation of their results, READUS-PV guidelines will ultimately facilitate evidence-based decision making in pharmacovigilance.

Expanding the data Ark: an attempt to make the data from highly cited social science papers publicly available

Access to scientific data can enable independent reuse and verification; however, most data are not available and become increasingly irrecoverable over time. This study aimed to retrieve and preserve important datasets from 160 of the most highly-cited social science articles published between 2008-2013 and 2015-2018. We asked authors if they would share data in a public repository-the Data Ark-or provide reasons if data could not be shared. Of the 160 articles, data for 117 (73%, 95% CI [67%-80%]) were not available and data for 7 (4%, 95% CI [0%-12%]) were available with restrictions. Data for 36 (22%, 95% CI [16%-30%]) articles were available in unrestricted form: 29 of these datasets were already available and 7 datasets were made available in the Data Ark. Most authors did not respond to our data requests and a minority shared reasons for not sharing, such as legal or ethical constraints. These findings highlight an unresolved need to preserve important scientific datasets and increase their accessibility to the scientific community.

The Costs of Anonymization: Case Study Using Clinical Data

BACKGROUND

Sharing data from clinical studies can accelerate scientific progress, improve transparency, and increase the potential for innovation and collaboration. However, privacy concerns remain a barrier to data sharing. Certain concerns, such as reidentification risk, can be addressed through the application of anonymization algorithms, whereby data are altered so that it is no longer reasonably related to a person. Yet, such alterations have the potential to influence the data set's statistical properties, such that the privacy-utility trade-off must be considered. This has been studied in theory, but evidence based on real-world individual-level clinical data is rare, and anonymization has not broadly been adopted in clinical practice.

OBJECTIVE

The goal of this study is to contribute to a better understanding of anonymization in the real world by comprehensively evaluating the privacy-utility trade-off of differently anonymized data using data and scientific results from the German Chronic Kidney Disease (GCKD) study.

METHODS

The GCKD data set extracted for this study consists of 5217 records and 70 variables. A 2-step procedure was followed to determine which variables constituted reidentification risks. To capture a large portion of the risk-utility space, we decided on risk thresholds ranging from 0.02 to 1. The data were then transformed via generalization and suppression, and the anonymization process was varied using a generic and a use case-specific configuration. To assess the utility of the anonymized GCKD data, general-purpose metrics (ie, data granularity and entropy), as well as use case-specific metrics (ie, reproducibility), were applied. Reproducibility was assessed by measuring the overlap of the 95% CI lengths between anonymized and original results.

RESULTS

Reproducibility measured by 95% CI overlap was higher than utility obtained from general-purpose metrics. For example, granularity varied between 68.2% and 87.6%, and entropy varied between 25.5% and 46.2%, whereas the average 95% CI overlap was above 90% for all risk thresholds applied. A nonoverlapping 95% CI was detected in 6 estimates across all analyses, but the overwhelming majority of estimates exhibited an overlap over 50%. The use case-specific configuration outperformed the generic one in terms of actual utility (ie, reproducibility) at the same level of privacy.

CONCLUSIONS

Our results illustrate the challenges that anonymization faces when aiming to support multiple likely and possibly competing uses, while use case-specific anonymization can provide greater utility. This aspect should be taken into account when evaluating the associated costs of anonymized data and attempting to maintain sufficiently high levels of privacy for anonymized data.

TRIAL REGISTRATION

German Clinical Trials Register DRKS00003971; https://drks.de/search/en/trial/DRKS00003971.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1093/ndt/gfr456.

Quality and transparency of evidence for implantable cardiovascular medical devices assessed by the CORE-MD consortium

BACKGROUND AND AIMS

The European Union Medical Device Regulation 2017/745 challenges key stakeholders to follow transparent and rigorous approaches to the clinical evaluation of medical devices. The purpose of this study is a systematic evaluation of published clinical evidence underlying selected high-risk cardiovascular medical devices before and after market access in the European Union (CE-marking) between 2000 and 2021.

METHODS

Pre-specified strategies were applied to identify published studies of prospective design evaluating 71 high-risk cardiovascular devices in seven different classes (bioresorbable coronary scaffolds, left atrial appendage occlusion devices, transcatheter aortic valve implantation systems, transcatheter mitral valve repair/replacement systems, surgical aortic and mitral heart valves, leadless pacemakers, subcutaneous implantable cardioverter-defibrillator). The search time span covered 20 years (2000-21). Details of study design, patient population, intervention(s), and primary outcome(s) were summarized and assessed with respect to timing of the corresponding CE-mark approval.

RESULTS

At least one prospective clinical trial was identified for 70% (50/71) of the pre-specified devices. Overall, 473 reports of 308 prospectively designed studies (enrolling 97 886 individuals) were deemed eligible, including 81% (251/308) prospective non-randomized clinical trials (66 186 individuals) and 19% (57/308) randomized clinical trials (31 700 individuals). Pre-registration of the study protocol was available in 49% (150/308) studies, and 16% (48/308) had a peer-reviewed publicly available protocol. Device-related adverse events were evaluated in 82% (253/308) of studies. An outcome adjudication process was reported in 39% (120/308) of the studies. Sample size was larger for randomized in comparison to non-randomized trials (median of 304 vs. 100 individuals, P < .001). No randomized clinical trial published before CE-mark approval for any of the devices was identified. Non-randomized clinical trials were predominantly published after the corresponding CE-mark approval of the device under evaluation (89%, 224/251). Sample sizes were smaller for studies published before (median of 31 individuals) than after (median of 135 individuals) CE-mark approval (P < .001). Clinical trials with larger sample sizes (>50 individuals) and those with longer recruitment periods were more likely to be published after CE-mark approval, and were more frequent during the period 2016-21.

CONCLUSIONS

The quantity and quality of publicly available data from prospective clinical investigations across selected categories of cardiovascular devices, before and after CE approval during the period 2000-21, were deemed insufficient. The majority of studies was non-randomized, with increased risk of bias, and performed in small populations without provision of power calculations, and none of the reviewed devices had randomized trial results published prior to CE-mark certification.

Up Front and Open? Shrouded in Secrecy? Or Somewhere in Between? A Meta-Research Systematic Review of Open Science Practices in Sport Medicine Research

OBJECTIVE: To investigate open science practices in research published in the top 5 sports medicine journals from May 1, 2022, and October 1, 2022. DESIGN: A meta-research systematic review. LITERATURE SEARCH: Open science practices were searched in MEDLINE. STUDY SELECTION CRITERIA: We included original scientific research published in one of the identified top 5 sports medicine journals in 2022 as ranked by Clarivate: (1) British Journal of Sports Medicine, (2) Journal of Sport and Health Science, (3) American Journal of Sports Medicine, (4) Medicine and Science in Sports and Exercise, and (5) Sports Medicine-Open. Studies were excluded if they were systematic reviews, qualitative research, gray literature, or animal or cadaver models. DATA SYNTHESIS: Open science practices were extracted in accordance with the Transparency and Openness Promotion guidelines and patient and public involvement. RESULTS: Two hundred forty-three studies were included. The median number of open science practices in each study was 2, out of a maximum of 12 (range: 0-8; interquartile range: 2). Two hundred thirty-four studies (96%, 95% confidence interval [CI]: 94%-99%) provided an author conflict-of-interest statement and 163 (67%, 95% CI: 62%-73%) reported funding. Twenty-one studies (9%, 95% CI: 5%-12%) provided open-access data. Fifty-four studies (22%, 95% CI: 17%-27%) included a data availability statement and 3 (1%, 95% CI: 0%-3%) made code available. Seventy-six studies (32%, 95% CI: 25%-37%) had transparent materials and 30 (12%, 95% CI: 8%-16%) used a reporting guideline. Twenty-eight studies (12%, 95% CI: 8%-16%) were preregistered. Six studies (3%, 95% CI: 1%-4%) published a protocol. Four studies (2%, 95% CI: 0%-3%) reported an analysis plan a priori. Seven studies (3%, 95% CI: 1%-5%) reported patient and public involvement. CONCLUSION: Open science practices in the sports medicine field are extremely limited. The least followed practices were sharing code, data, and analysis plans. J Orthop Sports Phys Ther 2023;53(12):1-13. Epub 20 October 2023. doi:10.2519/jospt.2023.12016.

Heterogeneity and Utility of Pharmaceutical Company Sharing of Individual-Participant Data Packages

Importance

The pharmaceutical industry has made substantial investments in developing processes for sharing individual-participant data (IPD) from clinical trials. However, the utility and completeness of shared IPD and supporting documents must be evaluated to ensure the potential for scientific advancements from the data sharing ecosystem can be realized.

Objective

To assess the utility and completeness of IPD and supporting documents provided from industry-sponsored clinical trials.

Design, Setting, and Participants

From February 9, 2022, to February 9, 2023, 91 of 203 clinical trials supporting US Food and Drug Administration registrations of anticancer medicines for the treatment of solid tumors from the past decade were confirmed as eligible for IPD request. This quality improvement study performed a retrospective audit of the utility and completeness of the IPD and supporting documents provided from the 91 clinical trials for a planned meta-analysis.

Exposures

Request for IPD from 91 clinical oncology trials indicated as eligible for the request.

Main Outcomes and Measures

The utility and completeness of the IPD and supporting documents provided.

Results

The IPD packages were obtained from 70 of 91 requested clinical trials (77%). The median time to data provision was 123 (range, 117-352) days. Redactions were observed in 18 of the acquired IPD packages (26%) for outcome data, 11 (16%) for assessment variables, and 19 (27%) for adjustment data. Additionally, 20 IPD packages (29%) lacked a clinical study report, 4 (6%) had incomplete or missing data dictionaries, and 20 (29%) were missing anonymization or redaction description files. Access to IPD from 21 eligible trials (23%) was not granted.

Conclusions and Relevance

In this quality improvement study, there was substantial variability within the provided IPD packages regarding the completeness of key data variables and supporting documents. To improve the data sharing ecosystem, key areas for enhancement include (1) ensuring that clinical trials are eligible for IPD sharing, (2) making eligible IPD transparently accessible, and (3) ensuring that IPD packages meet a standard of utility and completeness.

Biomedical supervisors' role modeling of open science practices

Supervision is one important way to socialize Ph.D. candidates into open and responsible research. We hypothesized that one should be more likely to identify open science practices (here publishing open access and sharing data) in empirical publications that were part of a Ph.D. thesis when the Ph.D. candidates' supervisors engaged in these practices compared to those whose supervisors did not or less often did. Departing from thesis repositories at four Dutch University Medical centers, we included 211 pairs of supervisors and Ph.D. candidates, resulting in a sample of 2062 publications. We determined open access status using UnpaywallR and Open Data using Oddpub, where we also manually screened publications with potential open data statements. Eighty-three percent of our sample was published openly, and 9% had open data statements. Having a supervisor who published open access more often than the national average was associated with an odds of 1.99 to publish open access. However, this effect became nonsignificant when correcting for institutions. Having a supervisor who shared data was associated with 2.22 (CI:1.19-4.12) times the odds to share data compared to having a supervisor that did not. This odds ratio increased to 4.6 (CI:1.86-11.35) after removing false positives. The prevalence of open data in our sample was comparable to international studies; open access rates were higher. Whilst Ph.D. candidates spearhead initiatives to promote open science, this study adds value by investigating the role of supervisors in promoting open science.