Overview and experience of the YODA Project with clinical trial data sharing after 5 years

Integrating Evidence to Guide Use of Biologics and Small Molecules for Inflammatory Bowel Diseases

Advances in science have led to the development of multiple biologics and small molecules for the treatment of inflammatory bowel diseases (IBDs). This growth in advanced medical therapies has been accompanied by an increase in methodological innovation to study and compare therapies. Guidelines provide an evidence-based approach to integrating therapies into routine practice, but they are often unable to provide timely recommendations as new therapies come to market, and they have limited incorporation of real-world evidence when making recommendations. This limits the scope and usability of guidelines, and a gap remains in defining how best to position and integrate advanced medical therapies for IBD. In this review, we provide a framework for clinicians and researchers to understand key differences in sources of evidence, how different methodologies are applied to study the comparative effectiveness of advanced medical therapies in IBD, and considerations for how these sources of evidence can be used to better integrate current guideline recommendations. Over time, we anticipate this framework will allow for a transition to living guidelines and/or practice recommendations.

Digital Research Environment(DRE)-enabled Artificial Intelligence (AI) to facilitate early stage drug development

Early-stage drug discovery is highly dependent upon drug target evaluation, understanding of disease progression and identification of patient characteristics linked to disease progression overlaid upon chemical libraries of potential drug candidates. Artificial intelligence (AI) has become a credible approach towards dealing with the diversity and volume of data in the modern drug development phase. There are a growing number of services and solutions available to pharmaceutical sponsors though most prefer to constrain their own data to closed solutions given the intellectual property considerations. Newer platforms offer an alternative, outsourced solution leveraging sponsors data with other, external open-source data to anchor predictions (often proprietary algorithms) which are refined given data indexed upon the sponsor's own chemical libraries. Digital research environments (DREs) provide a mechanism to ingest, curate, integrate and otherwise manage the diverse data types relevant for drug discovery activities and also provide workspace services from which target sharing and collaboration can occur providing yet another alternative with sponsors being in control of the platform, data and predictive algorithms. Regulatory engagement will be essential in the operationalizing of the various solutions and alternatives; current treatment of drug discovery data may not be adequate with respect to both quality and useability in the future. More sophisticated AI/ML algorithms are likely based on current performance metrics and diverse data types (e.g., imaging and genomic data) will certainly be a more consistent part of the myriad of data types that fuel future AI-based algorithms. This favors a dynamic DRE-enabled environment to support drug discovery.

External control arm analysis: an evaluation of propensity score approaches, G-computation, and doubly debiased machine learning

BACKGROUND

An external control arm is a cohort of control patients that are collected from data external to a single-arm trial. To provide an unbiased estimation of efficacy, the clinical profiles of patients from single and external arms should be aligned, typically using propensity score approaches. There are alternative approaches to infer efficacy based on comparisons between outcomes of single-arm patients and machine-learning predictions of control patient outcomes. These methods include G-computation and Doubly Debiased Machine Learning (DDML) and their evaluation for External Control Arms (ECA) analysis is insufficient.

METHODS

We consider both numerical simulations and a trial replication procedure to evaluate the different statistical approaches: propensity score matching, Inverse Probability of Treatment Weighting (IPTW), G-computation, and DDML. The replication study relies on five type 2 diabetes randomized clinical trials granted by the Yale University Open Data Access (YODA) project. From the pool of five trials, observational experiments are artificially built by replacing a control arm from one trial by an arm originating from another trial and containing similarly-treated patients.

RESULTS

Among the different statistical approaches, numerical simulations show that DDML has the smallest bias followed by G-computation. In terms of mean squared error, G-computation usually minimizes mean squared error. Compared to other methods, DDML has varying Mean Squared Error performances that improves with increasing sample sizes. For hypothesis testing, all methods control type I error and DDML is the most conservative. G-computation is the best method in terms of statistical power, and DDML has comparable power at [Formula: see text] but inferior ones for smaller sample sizes. The replication procedure also indicates that G-computation minimizes mean squared error whereas DDML has intermediate performances in between G-computation and propensity score approaches. The confidence intervals of G-computation are the narrowest whereas confidence intervals obtained with DDML are the widest for small sample sizes, which confirms its conservative nature.

CONCLUSIONS

For external control arm analyses, methods based on outcome prediction models can reduce estimation error and increase statistical power compared to propensity score approaches.

Characterization and corroboration of safety signals identified from the US Food and Drug Administration Adverse Event Reporting System, 2008-19: cross sectional study

OBJECTIVE

To characterize potential drug safety signals identified from the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS), from 2008 to 2019, to determine how often these signals resulted in regulatory action by the FDA and whether these actions were corroborated by published research findings or public assessments by the Sentinel Initiative.

DESIGN

Cross sectional study.

SETTING

USA.

POPULATION

Safety signals identified from the FAERS and publicly reported by the FDA between 2008 and 2019; and review of the relevant literature published before and after safety signals were reported in 2014-15. Literature searches were performed in November 2019, Sentinel Initiative assessments were searched in December 2021, and data analysis was finalized in December 2021.

MAIN OUTCOME MEASURES

Safety signals and resulting regulatory actions; number and characteristics of published studies, including corroboration of regulatory action as evidenced by significant associations (or no associations) between the drug related to the signal and the adverse event.

RESULTS

From 2008 to 2019, 603 potential safety signals identified from the FAERS were reported by the FDA (median 48 annually, interquartile range 41-61), of which 413 (68.5%) were resolved as of December 2021 (372 of 399 (93.2%) signals ≥3 years old were resolved). Among the resolved safety signals, 91 (22.0%) led to no regulatory action and 322 (78.0%) resulted in regulatory action, including 319 (77.2%) changes to drug labeling and 59 (14.3%) drug safety communications or other public communications from the FDA. For a subset of 82 potential safety signals reported in 2014-15, a literature search identified 1712 relevant publications; 1201 (70.2%) were case reports or case series. Among these 82 safety signals, 76 (92.7%) were resolved, of which relevant published research was identified for 57 (75.0%) signals and relevant Sentinel Initiative assessments for four (5.3%) signals. Regulatory actions by the FDA were corroborated by at least one relevant published research study for 17 of the 57 (29.8%) resolved safety signals; none of the relevant Sentinel Initiative assessments corroborated FDA regulatory action.

CONCLUSIONS

Most potential safety signals identified from the FAERS led to regulatory action by the FDA. Only a third of regulatory actions were corroborated by published research, however, and none by public assessments from the Sentinel Initiative. These findings suggest that either the FDA is taking regulatory actions based on evidence not made publicly available or more comprehensive safety evaluations might be needed when potential safety signals are identified.

Early termination of pivotal trials in Alzheimer's disease-Preserving optimal value for participants and science

Participants in Alzheimer's disease late-phase clinical trials are frequently confronted with a situation of early termination. We discuss measures to protect the perceived value of study participation and to maximize the scientific value under such circumstances. A communication strategy should ensure that trial participants maintain a positive relationship with the research team and have their informational needs optimally met. Measures to maximize the scientific value may include data/sample sharing, strategies for personalized medicine, as well as scientific follow-up. Critical for the success of such a concept are networks of excellence, extending models of existing initiatives like Global Alzheimer's Platform Foundation Network (GAP-Net). These networks could fundamentally strengthen the role of clinical investigators if they decide on their involvement in trials based upon their estimation of the scientific value and benefit for the participants, actively contribute to scientific analyses, and mediate optimal communication among the relevant trial stakeholders.

Regulatory Frameworks for Clinical Trial Data Sharing: Scoping Review

Background

Although well recognized for its scientific value, data sharing from clinical trials remains limited. Steps toward harmonization and standardization are increasing in various pockets of the global scientific community. This issue has gained salience during the COVID-19 pandemic. Even for agencies willing to share data, data exclusivity practices complicate matters; strict regulations by funders affect this even further. Finally, many low- and middle-income countries (LMICs) have weaker institutional mechanisms. This complex of factors hampers research and rapid response during public health emergencies. This drew our attention to the need for a review of the regulatory landscape governing clinical trial data sharing.

Objective

This review seeks to identify regulatory frameworks and policies that govern clinical trial data sharing and explore key elements of data-sharing mechanisms as outlined in existing regulatory documents. Following from, and based on, this empirical analysis of gaps in existing policy frameworks, we aimed to suggest focal areas for policy interventions on a systematic basis to facilitate clinical trial data sharing.

Methods

We followed the JBI scoping review approach. Our review covered electronic databases and relevant gray literature through a targeted web search. We included records (all publication types, except for conference abstracts) available in English that describe clinical trial data–sharing policies, guidelines, or standard operating procedures. Data extraction was performed independently by 2 authors, and findings were summarized using a narrative synthesis approach.

Results

We identified 4 articles and 13 policy documents; none originated from LMICs. Most (11/17, 65%) of the clinical trial agencies mandated a data-sharing agreement; 47% (8/17) of these policies required informed consent by trial participants; and 71% (12/17) outlined requirements for a data-sharing proposal review committee. Data-sharing policies have, a priori, milestone-based timelines when clinical trial data can be shared. We classify clinical trial agencies as following either controlled- or open-access data-sharing models. Incentives to promote data sharing and distinctions between mandated requirements and supportive requirements for informed consent during the data-sharing process remain gray areas, needing explication. To augment participant privacy and confidentiality, a neutral institutional mechanism to oversee dissemination of information from the appropriate data sets and more policy interventions led by LMICs to facilitate data sharing are strongly recommended.

Conclusions

Our review outlines the immediate need for developing a pragmatic data-sharing mechanism that aims to improve research and innovations as well as facilitate cross-border collaborations. Although a one-policy-fits-all approach would not account for regional and subnational legislation, we suggest that a focus on key elements of data-sharing mechanisms can be used to inform the development of flexible yet comprehensive data-sharing policies so that institutional mechanisms rather than disparate efforts guide data generation, which is the foundation of all scientific endeavor.

Sharing individual participant data: through a systematic reviewer lens

An increasing prevalence of data-sharing models, aimed at making individual participant data (IPD) from clinical trials widely available, should facilitate the conduct of systematic reviews and meta-analyses based on IPD. We have assessed these different data-sharing approaches, from the perspective of experienced IPD reviewers, to examine their utility for conducting systematic reviews based on IPD, and to highlight any challenges. We present an overview of the range of different models, including the traditional, single question approach, topic-based repositories, and the newer generic data platforms, and show that there are benefits and drawbacks to each. In particular, not all of the new models allow researchers to fully realise the well-documented advantages of using IPD for meta-analysis, and we offer potential solutions that can help improve both data quantity and utility. However, to achieve the “nirvana” of an ideal clinical data sharing environment, both for IPD meta-analysis and other secondary research purposes, we propose that data providers, data requestors, funders, and platforms need to adopt a more joined-up and standardised approach.

COVID-19 interventional trials: Analysis of data sharing intentions during a time of pandemic

Background

This survey of COVID-19 interventional studies encompasses, and expands upon, a previous publication [1] examining individual participant level data (IPD) sharing intentions for COVID-related trials and publications prior to June 30, 2020.

Methods

Replicating our inclusion criteria from the original survey, we evaluated a larger dataset of 2759 trials and 281 publications in this follow-up survey for willingness to share IPD and studied if sharing sentiment has evolved since the beginning of the pandemic.

Results

We found that 18 months into the pandemic, data sharing intentions remained static at 15% for trials registered through ClinicalTrials.gov (ClinicalTrials.gov is a digital registry of information about publicly and privately funded clinical studies in which human volunteers participate in interventional or observational scientific research) prior to September 19, 2021 compared to our initial survey. However, a comparison of declared intentions to share IPD at the time of publication revealed a noticeable shift: affirmative intentions grew from 21.4% (6/28) in our original publications survey to 57% (160/281) in this survey. Within the subset of studies published within journals affiliated with the International Committee of Medical Journal Editors (ICMJE), positive sharing intentions are even higher (65%).

Conclusions

Although intent to share data at the time of registration has not changed from our prior study in June 2020, there is growing commitment to sharing data reflected in the increasing number of affirmative declarations at the time of publication. Actual sharing of data will accelerate new insights into COVID-19 through secondary re-use of data.

Cancer patient survival can be parametrized to improve trial precision and reveal time-dependent therapeutic effects

Individual participant data (IPD) from oncology clinical trials is invaluable for identifying factors that influence trial success and failure, improving trial design and interpretation, and comparing pre-clinical studies to clinical outcomes. However, the IPD used to generate published survival curves are not generally publicly available. We impute survival IPD from ~500 arms of Phase 3 oncology trials (representing ~220,000 events) and find that they are well fit by a two-parameter Weibull distribution. Use of Weibull functions with overall survival significantly increases the precision of small arms typical of early phase trials: analysis of a 50-patient trial arm using parametric forms is as precise as traditional, non-parametric analysis of a 90-patient arm. We also show that frequent deviations from the Cox proportional hazards assumption, particularly in trials of immune checkpoint inhibitors, arise from time-dependent therapeutic effects. Trial duration therefore has an underappreciated impact on the likelihood of success.

Analysis of more than 150 Phase 3 oncology clinical trials supports parametric statistical analysis, significantly increasing the precision of small early-phase trials and relating deviations from the Cox proportional hazards model to trial duration.

Comparing current and emerging practice models for the extrapolation of survival data: a simulation study and case-study

BACKGROUND

Estimates of future survival can be a key evidence source when deciding if a medical treatment should be funded. Current practice is to use standard parametric models for generating extrapolations. Several emerging, more flexible, survival models are available which can provide improved within-sample fit. This study aimed to assess if these emerging practice models also provided improved extrapolations.

METHODS

Both a simulation study and a case-study were used to assess the goodness of fit of five classes of survival model. These were: current practice models, Royston Parmar models (RPMs), Fractional polynomials (FPs), Generalised additive models (GAMs), and Dynamic survival models (DSMs). The simulation study used a mixture-Weibull model as the data-generating mechanism with varying lengths of follow-up and sample sizes. The case-study was long-term follow-up of a prostate cancer trial. For both studies, models were fit to an early data-cut of the data, and extrapolations compared to the known long-term follow-up.

RESULTS

The emerging practice models provided better within-sample fit than current practice models. For data-rich simulation scenarios (large sample sizes or long follow-up), the GAMs and DSMs provided improved extrapolations compared with current practice. Extrapolations from FPs were always very poor whilst those from RPMs were similar to current practice. With short follow-up all the models struggled to provide useful extrapolations. In the case-study all the models provided very similar estimates, but extrapolations were all poor as no model was able to capture a turning-point during the extrapolated period.

CONCLUSIONS

Good within-sample fit does not guarantee good extrapolation performance. Both GAMs and DSMs may be considered as candidate extrapolation models in addition to current practice. Further research into when these flexible models are most useful, and the role of external evidence to improve extrapolations is required.

Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine

Over the past decade, artificial intelligence (AI) has contributed substantially to the resolution of various medical problems, including cancer. Deep learning (DL), a subfield of AI, is characterized by its ability to perform automated feature extraction and has great power in the assimilation and evaluation of large amounts of complicated data. On the basis of a large quantity of medical data and novel computational technologies, AI, especially DL, has been applied in various aspects of oncology research and has the potential to enhance cancer diagnosis and treatment. These applications range from early cancer detection, diagnosis, classification and grading, molecular characterization of tumors, prediction of patient outcomes and treatment responses, personalized treatment, automatic radiotherapy workflows, novel anti-cancer drug discovery, and clinical trials. In this review, we introduced the general principle of AI, summarized major areas of its application for cancer diagnosis and treatment, and discussed its future directions and remaining challenges. As the adoption of AI in clinical use is increasing, we anticipate the arrival of AI-powered cancer care.

Benefits and harms of Risperidone and Paliperidone for treatment of patients with schizophrenia or bipolar disorder: a meta-analysis involving individual participant data and clinical study reports

BACKGROUND

Schizophrenia and bipolar disorder are severe mental illnesses which are highly prevalent worldwide. Risperidone and Paliperidone are treatments for either illnesses, but their efficacy compared to other antipsychotics and growing reports of hormonal imbalances continue to raise concerns. As existing evidence on both antipsychotics are solely based on aggregate data, we aimed to assess the benefits and harms of Risperidone and Paliperidone in the treatment of patients with schizophrenia or bipolar disorder, using individual participant data (IPD), clinical study reports (CSRs) and publicly available sources (journal publications and trial registries).

METHODS

We searched MEDLINE, Central, EMBASE and PsycINFO until December 2020 for randomised placebo-controlled trials of Risperidone, Paliperidone or Paliperidone palmitate in patients with schizophrenia or bipolar disorder. We obtained IPD and CSRs from the Yale University Open Data Access project. The primary outcome Positive and Negative Syndrome Scale (PANSS) score was analysed using one-stage IPD meta-analysis. Random-effect meta-analysis of harm outcomes involved methods for coping with rare events. Effect-sizes were compared across all available data sources using the ratio of means or relative risk. We registered our review on PROSPERO, CRD42019140556.

RESULTS

Of the 35 studies, IPD meta-analysis involving 22 (63%) studies showed a significant clinical reduction in the PANSS in patients receiving Risperidone (mean difference - 5.83, 95% CI - 10.79 to - 0.87, I2 = 8.5%, n = 4 studies, 1131 participants), Paliperidone (- 6.01, 95% CI - 8.7 to - 3.32, I2 = 4.3%, n = 13, 3821) and Paliperidone palmitate (- 7.89, 95% CI - 12.1 to - 3.69, I2 = 2.9%, n = 5, 2209). CSRs reported nearly two times more adverse events (4434 vs. 2296 publication, relative difference (RD) = 1.93, 95% CI 1.86 to 2.00) and almost 8 times more serious adverse events (650 vs. 82; RD = 7.93, 95% CI 6.32 to 9.95) than the journal publications. Meta-analyses of individual harms from CSRs revealed a significant increased risk among several outcomes including extrapyramidal disorder, tardive dyskinesia and increased weight. But the ratio of relative risk between the different data sources was not significant. Three treatment-related gynecomastia events occurred, and these were considered mild to moderate in severity.

CONCLUSION

IPD meta-analysis conclude that Risperidone and Paliperidone antipsychotics had a small beneficial effect on reducing PANSS score over 9 weeks, which is more conservative than estimates from reviews based on journal publications. CSRs also contained significantly more data on harms that were unavailable in journal publications or trial registries. Sharing of IPD and CSRs are necessary when performing meta-analysis on the efficacy and safety of antipsychotics.

Characteristics of available studies and dissemination of research using major clinical data sharing platforms

BACKGROUND/AIMS

Over the past decade, numerous data sharing platforms have been launched, providing access to de-identified individual patient-level data and supporting documentation. We evaluated the characteristics of prominent clinical data sharing platforms, including types of studies listed as available for request, data requests received, and rates of dissemination of research findings from data requests.

METHODS

We reviewed publicly available information listed on the websites of six prominent clinical data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center, ClinicalStudyDataRequest.com, Project Data Sphere, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project. We recorded key platform characteristics, including listed studies and available supporting documentation, information on the number and status of data requests, and rates of dissemination of research findings from data requests (i.e. publications in a peer-reviewed journals, preprints, conference abstracts, or results reported on the platform's website).

RESULTS

The number of clinical studies listed as available for request varied among five data sharing platforms: Biological Specimen and Data Repository Information Coordinating Center (n = 219), ClinicalStudyDataRequest.com (n = 2,897), Project Data Sphere (n = 154), Vivli (n = 5426), and the Yale Open Data Access Project (n = 395); Supporting Open Access to Researchers did not provide a list of Bristol Myers Squibb studies available for request. Individual patient-level data were nearly always reported as being available for request, as opposed to only Clinical Study Reports (Biological Specimen and Data Repository Information Coordinating Center = 211/219 (96.3%); ClinicalStudyDataRequest.com = 2884/2897 (99.6%); Project Data Sphere = 154/154 (100.0%); and the Yale Open Data Access Project = 355/395 (89.9%)); Vivli did not provide downloadable study metadata. Of 1201 data requests listed on ClinicalStudyDataRequest.com, Supporting Open Access to Researchers-Bristol Myers Squibb, Vivli, and the Yale Open Data Access Project platforms, 586 requests (48.8%) were approved (i.e. data access granted). The majority were for secondary analyses and/or developing/validating methods (ClinicalStudyDataRequest.com = 262/313 (83.7%); Supporting Open Access to Researchers-Bristol Myers Squibb = 22/30 (73.3%); Vivli = 63/84 (75.0%); the Yale Open Data Access Project = 111/159 (69.8%)); four were for re-analyses or corroborations of previous research findings (ClinicalStudyDataRequest.com = 3/313 (1.0%) and the Yale Open Data Access Project = 1/159 (0.6%)). Ninety-five (16.1%) approved data requests had results disseminated via peer-reviewed publications (ClinicalStudyDataRequest.com = 61/313 (19.5%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 4/84 (4.8%); the Yale Open Data Access Project = 27/159 (17.0%)). Forty-two (6.8%) additional requests reported results through preprints, conference abstracts, or on the platform's website (ClinicalStudyDataRequest.com = 12/313 (3.8%); Supporting Open Access to Researchers-Bristol Myers Squibb = 3/30 (10.0%); Vivli = 2/84 (2.4%); Yale Open Data Access Project = 25/159 (15.7%)).

CONCLUSION

Across six prominent clinical data sharing platforms, information on studies and request metrics varied in availability and format. Most data requests focused on secondary analyses and approximately one-quarter of all approved requests publicly disseminated their results. To further promote the use of shared clinical data, platforms should increase transparency, consistently clarify the availability of the listed studies and supporting documentation, and ensure that research findings from data requests are disseminated.

Status, use and impact of sharing individual participant data from clinical trials: a scoping review

OBJECTIVES

To explore the impact of data-sharing initiatives on the intent to share data, on actual data sharing, on the use of shared data and on research output and impact of shared data.

ELIGIBILITY CRITERIA

All studies investigating data-sharing practices for individual participant data (IPD) from clinical trials.

SOURCES OF EVIDENCE

We searched the Medline database, the Cochrane Library, the Science Citation Index Expanded and the Social Sciences Citation Index via Web of Science, and preprints and proceedings of the International Congress on Peer Review and Scientific Publication. In addition, we inspected major clinical trial data-sharing platforms, contacted major journals/publishers, editorial groups and some funders.

CHARTING METHODS

Two reviewers independently extracted information on methods and results from resources identified using a standardised questionnaire. A map of the extracted data was constructed and accompanied by a narrative summary for each outcome domain.

RESULTS

93 studies identified in the literature search (published between 2001 and 2020, median: 2018) and 5 from additional information sources were included in the scoping review. Most studies were descriptive and focused on early phases of the data-sharing process. While the willingness to share IPD from clinical trials is extremely high, actual data-sharing rates are suboptimal. A survey of journal data suggests poor to moderate enforcement of the policies by publishers. Metrics provided by platforms suggest that a large majority of data remains unrequested. When requested, the purpose of the reuse is more often secondary analyses and meta-analyses, rarely re-analyses. Finally, studies focused on the real impact of data-sharing were rare and used surrogates such as citation metrics.

CONCLUSIONS

There is currently a gap in the evidence base for the impact of IPD sharing, which entails uncertainties in the implementation of current data-sharing policies. High level evidence is needed to assess whether the value of medical research increases with data-sharing practices.

Data Sharing and Privacy in Pharmaceutical Studies

Adverse drug events have been a long-standing concern for the wide-ranging harms to public health, and the substantial disease burden. The key to diminish or eliminate the impacts is to build a comprehensive pharmacovigilance system. Application of the "big data" approach has been proved to assist the detection of adverse drug events by involving previously unavailable data sources and promoting health information exchange. Even though challenges and potential risks still remain. The lack of effective privacy-preserving measures in the flow of medical data is the most important Accepted: one, where urgent actions are required to prevent the threats and facilitate the construction of pharmacovigilance systems. Several privacy protection methods are reviewed in this article, which may be helpful to break the barrier.

Data Sharing Goals for Nonprofit Funders of Clinical Trials

Sharing clinical trial data can provide value to research participants and communities by accelerating the development of new knowledge and therapies as investigators merge data sets to conduct new analyses, reproduce published findings to raise standards for original research, and learn from the work of others to generate new research questions. Nonprofit funders, including disease advocacy and patient-focused organizations, play a pivotal role in the promotion and implementation of data sharing policies. Funders are uniquely positioned to promote and support a culture of data sharing by serving as trusted liaisons between potential research participants and investigators who wish to access these participants’ networks for clinical trial recruitment. In short, nonprofit funders can drive policies and influence research culture. The purpose of this paper is to detail a set of aspirational goals and forward thinking, collaborative data sharing solutions for nonprofit funders to fold into existing funding policies. The goals of this paper convey the complexity of the opportunities and challenges facing nonprofit funders and the appropriate prioritization of data sharing within their organizations and may serve as a starting point for a data sharing toolkit for nonprofit funders of clinical trials to provide the clarity of mission and mechanisms to enforce the data sharing practices their communities already expect are happening.

Evaluation of Data Sharing After Implementation of the International Committee of Medical Journal Editors Data Sharing Statement Requirement

IMPORTANCE

The benefits of responsible sharing of individual-participant data (IPD) from clinical studies are well recognized, but stakeholders often disagree on how to align those benefits with privacy risks, costs, and incentives for clinical trialists and sponsors. The International Committee of Medical Journal Editors (ICMJE) required a data sharing statement (DSS) from submissions reporting clinical trials effective July 1, 2018. The required DSSs provide a window into current data sharing rates, practices, and norms among trialists and sponsors.

OBJECTIVE

To evaluate the implementation of the ICMJE DSS requirement in 3 leading medical journals: JAMA, Lancet, and New England Journal of Medicine (NEJM).

DESIGN, SETTING, AND PARTICIPANTS

This is a cross-sectional study of clinical trial reports published as articles in JAMA, Lancet, and NEJM between July 1, 2018, and April 4, 2020. Articles not eligible for DSS, including observational studies and letters or correspondence, were excluded. A MEDLINE/PubMed search identified 487 eligible clinical trials in JAMA (112 trials), Lancet (147 trials), and NEJM (228 trials). Two reviewers evaluated each of the 487 articles independently.

EXPOSURE

Publication of clinical trial reports in an ICMJE medical journal requiring a DSS.

MAIN OUTCOMES AND MEASURES

The primary outcomes of the study were declared data availability and actual data availability in repositories. Other captured outcomes were data type, access, and conditions and reasons for data availability or unavailability. Associations with funding sources were examined.

RESULTS

A total of 334 of 487 articles (68.6%; 95% CI, 64%-73%) declared data sharing, with nonindustry NIH-funded trials exhibiting the highest rates of declared data sharing (89%; 95% CI, 80%-98%) and industry-funded trials the lowest (61%; 95% CI, 54%-68%). However, only 2 IPD sets (0.6%; 95% CI, 0.0%-1.5%) were actually deidentified and publicly available as of April 10, 2020. The remaining were supposedly accessible via request to authors (143 of 334 articles [42.8%]), repository (89 of 334 articles [26.6%]), and company (78 of 334 articles [23.4%]). Among the 89 articles declaring that IPD would be stored in repositories, only 17 (19.1%) deposited data, mostly because of embargo and regulatory approval. Embargo was set in 47.3% of data-sharing articles (158 of 334), and in half of them the period exceeded 1 year or was unspecified.

CONCLUSIONS AND RELEVANCE

Most trials published in JAMA, Lancet, and NEJM after the implementation of the ICMJE policy declared their intent to make clinical data available. However, a wide gap between declared and actual data sharing exists. To improve transparency and data reuse, journals should promote the use of unique pointers to data set location and standardized choices for embargo periods and access requirements.

Global Regulatory Agencies and Data Transparency

Egilman et al. review the current data sharing practices of three global regulatory agencies - Health Canada, the European Medicines Agency and the Food and Drug Agency. While there has been progress towards increasing transparency over the past decade, progress has been slow.

Timely access to trial data in the context of a pandemic: the time is now

OBJECTIVE

Clinical trial data sharing has the potential to accelerate scientific progress, answer new lines of scientific inquiry, support reproducibility and prevent redundancy. Vivli, a non-profit organisation, operates a global platform for sharing of individual participant-level trial data and associated documents. Sharing of these data collected from each trial participant enables combining of these data to drive new scientific insights or assess reproducibility-not possible with the aggregate or summary data tables historically made available. We report on our initial experience including key metrics, lessons learned and how we see our role in the data sharing ecosystem. We also describe how Vivli is addressing the needs of the COVID-19 challenge through a new dedicated portal that provides a direct search function for COVID-19 studies, availability for fast-tracked request review and data sharing.

DATA SUMMARY

The Vivli platform was established in 2018 and has partnered with 28 diverse members from industry, academic institutions, government platforms and non-profit foundations. Currently, 5400 trials representing 3.6 million participants are shared on the platform. From July 2018 to September 2020, Vivli received 201 requests. To date, 106 of 201 requests received approval, 5 have been declined, 27 withdrew and 27 are in the revision stage.

CONCLUSIONS

The pandemic has only magnified the necessity for data sharing. If most data are shared and in a manner that allows interoperability, then we have hope of moving towards a cohesive scientific understanding more quickly not only for COVID-19 but also for all diseases. Conversely, if only isolated pockets of data are shared then society loses the opportunity to close vital gaps in our understanding of this rapidly evolving epidemic. This current challenge serves to highlight the value of data sharing platforms-critical enablers that help researchers build on prior knowledge.

Predictive approaches to heterogeneous treatment effects: a scoping review

BACKGROUND

Recent evidence suggests that there is often substantial variation in the benefits and harms across a trial population. We aimed to identify regression modeling approaches that assess heterogeneity of treatment effect within a randomized clinical trial.

METHODS

We performed a literature review using a broad search strategy, complemented by suggestions of a technical expert panel.

RESULTS

The approaches are classified into 3 categories: 1) Risk-based methods (11 papers) use only prognostic factors to define patient subgroups, relying on the mathematical dependency of the absolute risk difference on baseline risk; 2) Treatment effect modeling methods (9 papers) use both prognostic factors and treatment effect modifiers to explore characteristics that interact with the effects of therapy on a relative scale. These methods couple data-driven subgroup identification with approaches to prevent overfitting, such as penalization or use of separate data sets for subgroup identification and effect estimation. 3) Optimal treatment regime methods (12 papers) focus primarily on treatment effect modifiers to classify the trial population into those who benefit from treatment and those who do not. Finally, we also identified papers which describe model evaluation methods (4 papers).

CONCLUSIONS

Three classes of approaches were identified to assess heterogeneity of treatment effect. Methodological research, including both simulations and empirical evaluations, is required to compare the available methods in different settings and to derive well-informed guidance for their application in RCT analysis.

Defining the Path Forward for Biomarkers to Address Unmet Needs in Inflammatory Bowel Diseases

Despite major advances in the inflammatory bowel diseases field, biomarkers to enable personalized and effective management are inadequate. Disease course and treatment response are highly variable, with some patients experiencing mild disease progression, whereas other patients experience severe or complicated disease. Periodic endoscopy is performed to assess disease activity; as a result, it takes months to ascertain whether a treatment is having a positive impact on disease progression. Minimally invasive biomarkers for prognosis of disease course, prediction of treatment response, monitoring of disease activity, and accurate diagnosis based on improved disease phenotyping and classification could improve outcomes and accelerate the development of novel therapeutics. Rapidly developing technologies have great potential in this regard; however, the discovery, validation, and qualification of biomarkers will require partnerships including academia, industry, funders, and regulators. The Crohn's & Colitis Foundation launched the IBD Biomarker Summit to bring together key stakeholders to identify and prioritize critical unmet needs; prioritize promising technologies and consortium approaches to address these needs; and propose harmonization approaches to improve comparability of data across studies. Here, we summarize the outcomes of the 2018 and 2019 meetings, including consensus-based unmet needs in the clinical and drug development context. We highlight ongoing consortium efforts and promising technologies with the potential to address these needs in the near term. Finally, we summarize actionable recommendations for harmonization, including data collection tools for improved consistency in disease phenotyping; standardization of informed consenting; and development of guidelines for sample management and assay validation. Taken together, these outcomes demonstrate that there is an exceptional alignment of priorities across stakeholders for a coordinated effort to address unmet needs of patients with inflammatory bowel diseases through biomarker science.

Is it time for computable evidence synthesis?

Efforts aimed at increasing the pace of evidence synthesis have been primarily focused on the use of published articles, but these are a relatively delayed, incomplete, and at times biased source of study results data. Compared to those in bibliographic databases, structured results data available in trial registries may be more timely, complete, and accessible, but these data remain underutilized. Key advantages of using structured results data include the potential to automatically monitor the accumulation of relevant evidence and use it to signal when a systematic review requires updating, as well as to prospectively assign trials to already published reviews. Shifting focus to emerging sources of structured trial data may provide the impetus to build a more proactive and efficient system of continuous evidence surveillance.

Legal regulations, ethical guidelines and recent policies to increase transparency of clinical trials

Timely and accurate dissemination of outcomes is essential to accomplish main benefits of scientific research including clinical trials. Clinical trial results can be disseminated in two main ways: by publication in a peer-reviewed journal and by posting on a publicly available clinical trial register. The credibility of the literature on clinical trials is significantly diminished because a high percentage of trials is not published. While current legal regulations both in the European Union (EU) and the USA impose a duty to submit summary results of clinical trials to a respective register (EU Clinical Trial Register and ClinicalTrials.gov, respectively), the compliance with this requirement has been generally inadequate. Trial outcomes can be also made accessible by data sharing. However, in spite of the wide promotion of this idea, the access of investigators to participant-level datasets remains limited. The main objective of this review is to discuss current legal regulations, international standards, ethical guidelines and recent policies pertaining to dissemination of clinical trial results.

Updating insights into rosiglitazone and cardiovascular risk through shared data: individual patient and summary level meta-analyses

OBJECTIVES

To conduct a systematic review and meta-analysis of the effects of rosiglitazone treatment on cardiovascular risk and mortality using multiple data sources and varying analytical approaches with three aims in mind: to clarify uncertainties about the cardiovascular risk of rosiglitazone; to determine whether different analytical approaches are likely to alter the conclusions of adverse event meta-analyses; and to inform efforts to promote clinical trial transparency and data sharing.

DESIGN

Systematic review and meta-analysis of randomized controlled trials.

DATA SOURCES

GlaxoSmithKline's (GSK's) ClinicalStudyDataRequest.com for individual patient level data (IPD) and GSK's Study Register platforms, MEDLINE, PubMed, Embase, Web of Science, Cochrane Central Registry of Controlled Trials, Scopus, and ClinicalTrials.gov from inception to January 2019 for summary level data.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomized, controlled, phase II-IV clinical trials that compared rosiglitazone with any control for at least 24 weeks in adults.

DATA EXTRACTION AND SYNTHESIS

For analyses of trials for which IPD were available, a composite outcome of acute myocardial infarction, heart failure, cardiovascular related death, and non-cardiovascular related death was examined. These four events were examined independently as secondary analyses. For analyses including trials for which IPD were not available, myocardial infarction and cardiovascular related death were examined, which were determined from summary level data. Multiple meta-analyses were conducted that accounted for trials with zero events in one or both arms with two different continuity corrections (0.5 constant and treatment arm) to calculate odds ratios and risk ratios with 95% confidence intervals.

RESULTS

33 eligible trials were identified from ClinicalStudyDataRequest.com for which IPD were available (21 156 patients). Additionally, 103 trials for which IPD were not available were included in the meta-analyses for myocardial infarction (23 683 patients), and 103 trials for which IPD were not available contributed to the meta-analyses for cardiovascular related death (22 772 patients). Among 29 trials for which IPD were available and that were included in previous meta-analyses using GSK's summary level data, more myocardial infarction events were identified by using IPD instead of summary level data for 26 trials, and fewer cardiovascular related deaths for five trials. When analyses were limited to trials for which IPD were available, and a constant continuity correction of 0.5 and a random effects model were used to account for trials with zero events in only one arm, patients treated with rosiglitazone had a 33% increased risk of a composite event compared with controls (odds ratio 1.33, 95% confidence interval 1.09 to 1.61; rosiglitazone population: 274 events among 11 837 patients; control population: 219 events among 9319 patients). The odds ratios for myocardial infarction, heart failure, cardiovascular related death, and non-cardiovascular related death were 1.17 (0.92 to 1.51), 1.54 (1.14 to 2.09), 1.15 (0.55 to 2.41), and 1.18 (0.60 to 2.30), respectively. For analyses including trials for which IPD were not available, odds ratios for myocardial infarction and cardiovascular related death were attenuated (1.09, 0.88 to 1.35, and 1.12, 0.72 to 1.74, respectively). Results were broadly consistent when analyses were repeated using trials with zero events across both arms and either of the two continuity corrections was used.

CONCLUSIONS

The results suggest that rosiglitazone is associated with an increased cardiovascular risk, especially for heart failure events. Although increased risk of myocardial infarction was observed across analyses, the strength of the evidence varied and effect estimates were attenuated when summary level data were used in addition to IPD. Because more myocardial infarctions and fewer cardiovascular related deaths were reported in the IPD than in the summary level data, sharing IPD might be necessary when performing meta-analyses focused on safety.

SYSTEMATIC REVIEW REGISTRATION

OSF Home https://osf.io/4yvp2/.

The Global Aging & Geriatric Experiments in Bipolar Disorder Database (GAGE-BD) project: Understanding older-age bipolar disorder by combining multiple datasets

OBJECTIVE

There is a dearth of research about the aging process among individuals with bipolar disorder (BD). One potential strategy to overcome the challenge of interpreting findings from existing limited older-age bipolar disorder (OABD) research studies is to pool or integrate data, taking advantage of potential overlap or similarities in assessment methods and harmonizing or cross-walking measurements where different measurement tools are used to evaluate overlapping construct domains. This report describes the methods and initial start-up activities of a first-ever initiative to create an integrated OABD-focused database, the Global Aging & Geriatric Experiments in Bipolar Disorder Database (GAGE-BD) project.

METHODS

Building on preliminary work conducted by members of the International Society for Bipolar Disorders OABD taskforce, the GAGE-BD project will be operationalized in four stages intended to ready the dataset for hypothesis-driven analyses, establish a consortium of investigators to guide exploration, and set the stage for prospective investigation using a common dataset that will facilitate a high degree of generalizability.

RESULTS

Initial efforts in GAGE-BD have brought together 14 international investigators representing a broad geographic distribution and data on over 1,000 OABD. Start-up efforts include communication and guidance on meeting regulatory requirements, establishing a Steering Committee to guide an incremental analysis strategy, and learning from existing multisite data collaborations and other support resources.

DISCUSSION

The GAGE-BD project aims to advance understanding of associations between age, BD symptoms, medical burden, cognition and functioning across the life span and set the stage for future prospective research that can advance the understanding of OABD.

Opportunities for selective reporting of harms in randomized clinical trials: Selection criteria for non-systematic adverse events

Background

Adverse events (AEs) in clinical trials may be reported in multiple sources. Different methods for reporting adverse events across trials or across sources for a single trial may produce inconsistent information about the adverse events associated with interventions.

Methods

We compared the methods authors use to decide which AEs to include in a particular source (i.e., “selection criteria”), including the number of different types of AEs reported (i.e., rather than the number of events). We compared sources (e.g., journal articles, clinical study reports (CSRs)) of trials for two drug-indications—gabapentin for neuropathic pain and quetiapine for bipolar depression. Electronic searches were completed in 2015. We identified selection criteria and assessed how criteria affected AE reporting.

Results

We identified 21 gabapentin and 7 quetiapine trials. We found 6 gabapentin CSRs and 2 quetiapine CSRs, all written by drug manufacturers. All CSRs reported all AEs without applying selection criteria; by comparison, no other source reported all AEs, and 15/68 (22%) gabapentin sources and 19/48 (40%) quetiapine sources reported using selection criteria. Selection criteria greatly affected the number of AEs reported. For example, 67/316 (21%) AEs in one quetiapine trial met the criterion “occurring in ≥2% of participants in any treatment group,” while only 5/316 (2%) AEs met the criterion “occurring in ≥10% of quetiapine-treated patients and twice as frequent in the quetiapine group as the placebo group.”

Conclusions

Selection criteria for reporting AEs vary across trials and across sources for individual trials. If investigators do not pre-specify selection criteria, they might “cherry-pick” AEs based on results. Even if investigators pre-specify selection criteria, selective reporting will produce biased meta-analyses and clinical practice guidelines. Data about all AEs identified in clinical trials should be publicly available; however, sharing data will not solve all the problems identified in this study.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3581-3) contains supplementary material, which is available to authorized users.

How Clinical Trial Data Sharing Platforms Can Advance the Study of Biomarkers

Although data sharing platforms host diverse data types the features of these platforms are well-suited to facilitating biomarker research. Given the current state of biomarker discovery, an innovative paradigm to accelerate biomarker discovery is to utilize platforms such as Vivli to leverage researchers' abilities to integrate certain classes of biomarkers.

Biomarkers as Surrogate Endpoints: Ongoing Opportunities for Validation

Surrogate endpoints are a common application of biomarkers to estimate clinical benefit in clinical trials, despite questions about reliability. This article discusses ongoing opportunities for their validation, in the context of a regulatory environment in which they are increasingly championed.

Sharing of clinical trial data and results reporting practices among large pharmaceutical companies: cross sectional descriptive study and pilot of a tool to improve company practices

OBJECTIVES

To develop and pilot a tool to measure and improve pharmaceutical companies' clinical trial data sharing policies and practices.

DESIGN

Cross sectional descriptive analysis.

SETTING

Large pharmaceutical companies with novel drugs approved by the US Food and Drug Administration in 2015.

DATA SOURCES

Data sharing measures were adapted from 10 prominent data sharing guidelines from expert bodies and refined through a multi-stakeholder deliberative process engaging patients, industry, academics, regulators, and others. Data sharing practices and policies were assessed using data from ClinicalTrials.gov, Drugs@FDA, corporate websites, data sharing platforms and registries (eg, the Yale Open Data Access (YODA) Project and Clinical Study Data Request (CSDR)), and personal communication with drug companies.

MAIN OUTCOME MEASURES

Company level, multicomponent measure of accessibility of participant level clinical trial data (eg, analysis ready dataset and metadata); drug and trial level measures of registration, results reporting, and publication; company level overall transparency rankings; and feasibility of the measures and ranking tool to improve company data sharing policies and practices.

RESULTS

Only 25% of large pharmaceutical companies fully met the data sharing measure. The median company data sharing score was 63% (interquartile range 58-85%). Given feedback and a chance to improve their policies to meet this measure, three companies made amendments, raising the percentage of companies in full compliance to 33% and the median company data sharing score to 80% (73-100%). The most common reasons companies did not initially satisfy the data sharing measure were failure to share data by the specified deadline (75%) and failure to report the number and outcome of their data requests. Across new drug applications, a median of 100% (interquartile range 91-100%) of trials in patients were registered, 65% (36-96%) reported results, 45% (30-84%) were published, and 95% (69-100%) were publicly available in some form by six months after FDA drug approval. When examining results on the drug level, less than half (42%) of reviewed drugs had results for all their new drug applications trials in patients publicly available in some form by six months after FDA approval.

CONCLUSIONS

It was feasible to develop a tool to measure data sharing policies and practices among large companies and have an impact in improving company practices. Among large companies, 25% made participant level trial data accessible to external investigators for new drug approvals in accordance with the current study's measures; this proportion improved to 33% after applying the ranking tool. Other measures of trial transparency were higher. Some companies, however, have substantial room for improvement on transparency and data sharing of clinical trials.