Beyond Randomized Clinical Trials: Use of External Controls

Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations

BACKGROUND AND OBJECTIVE

Missing data and unmeasured confounding are key challenges for external comparator studies. This work evaluates bias and other performance characteristics depending on missingness and unmeasured confounding by means of two case studies and simulations.

METHODS

Two case studies were constructed by taking the treatment arms from two randomised controlled trials and an external real-world data source that exhibited substantial missingness. The indications of the randomised controlled trials were multiple myeloma and metastatic hormone-sensitive prostate cancer. Overall survival was taken as the main endpoint. The effects of missing data and unmeasured confounding were assessed for the case studies by reporting estimated external comparator versus randomised controlled trial treatment effects. Based on the two case studies, simulations were performed broadening the settings by varying the underlying hazard ratio, the sample size, the sample size ratio between the experimental arm and the external comparator, the number of missing covariates and the percentage of missingness. Thereby, bias and other performance metrics could be quantified dependent on these factors.

RESULTS

For the multiple myeloma external comparator study, results were in line with the randomised controlled trial, despite missingness and potential unmeasured confounding, while for the metastatic hormone-sensitive prostate cancer case study missing data led to a low sample size, leading overall to inconclusive results. Furthermore, for the metastatic hormone-sensitive prostate cancer study, missing data in important eligibility criteria led to further limitations. Simulations were successfully applied to gain a quantitative understanding of the effects of missing data and unmeasured confounding.

CONCLUSIONS

This exploratory study confirmed external comparator strengths and limitations by quantifying the impact of missing data and unmeasured confounding using case studies and simulations. In particular, missing data in key eligibility criteria were seen to limit the ability to derive the external comparator target analysis population accurately, while simulations demonstrated the magnitude of bias to expect for various settings.

Externally Controlled Studies Using Real-World Data in Patients With Hematological Cancers: A Systematic Review

Importance

The use of real-world data (RWD) external control arms in prospective studies is increasing. The advantages, including the immediate availability of a control population, must be balanced with the requirements of meeting evidentiary standards.

Objective

To address the question of whether and to what extent the methods of RWD studies compare to standard methods used in randomized clinical trials.

Evidence Review

A systematic search across 4 electronic databases and Google Scholar was conducted from January 1, 2000, to October 23, 2023. Studies were included in the systematic review if they compared an intervention arm in a clinical trial to an RWD control arm in patients with hematological cancers and if they were published between 2000 and 2023.

Findings

Thirty-two prospective intervention studies incorporating external control data from RWD sources of patients with hematological cancers were identified. A total of 4306 patients from intervention arms and 10 594 from RWD control arms were included across all studies. Only 2 studies (6%) included prospectively collected RWD. The complete trial inclusion criteria were applied to the RWD cohort in 7 studies (22%). Four studies (13%) published the statistical analysis plan and prespecified use of RWD. A total of 23 studies (72%) applied matching algorithms for trial and RWD cohorts, including matching for demographic, disease, and/or therapy-related characteristics. The end point criteria were the same as the trial in 8 studies (25%). In contrast, 12 studies (38%) used different end points, and 12 (38%) did not provide an end point definition for the RWD. Twelve studies (38%) had a median follow-up difference of less than a year between arms. Eight studies (25%) reported toxic effect data for the trial arm, of which 5 studies reported toxic effect data for the RWD arm.

Conclusions and Relevance

In this systematic review, limitations were observed in the application of clinical trial eligibility criteria to RWD, statistical rigor and application of matching methods, the definition of end points, follow-up, and reporting of adverse events, which may challenge the conclusions reported in studies using RWD.

Graded exposure therapy for adults with persistent symptoms after mTBI: A historical comparison study

Fear avoidance behaviour is associated with slow recovery from mild traumatic brain injury (mTBI). This study is a preliminary evaluation of graded exposure therapy (GET), which directly targets fear avoidance behaviour, for reducing post-concussion symptoms (PCS) and disability following mTBI. In a historical comparison design, we compared two groups from independent randomized trials. The GET + UC group (N = 34) received GET (delivered over 16 videoconference sessions) in addition to usual care (UC). The historical comparison group (N = 71) received UC only. PCS severity (Rivermead Post Concussion Symptoms Questionnaire; RPQ) and disability (World Health Organization Disability Assessment Schedule; WHODAS 2.0 12-item) were measured at clinic intake (M = 2.7, SD = 1.1 months after injury) and again at M = 4.9 (SD = 1.1) months after injury. Between-group differences were estimated using linear mixed effects regression, with a sensitivity analysis controlling for injury-to-assessment intervals. The estimated average change on the RPQ was -14.3 in the GET + UC group and -5.3 in the UC group. The estimated average change on the WHODAS was -5.3 in the GET + UC group and -3.2 in the UC group. Between-group differences post-treatment were -5.3 on the RPQ and -1.5 on the WHODAS. Treatment effects were larger in sensitivity analyses. Findings suggest that a randomized controlled trial is warranted.

Mixed Reality in Clinical Settings for Pediatric Patients and Their Families: A Literature Review

In the post-pandemic context, there has been an increasing demand for technology-based interventions in education and healthcare systems, such as augmented and mixed reality technologies. Despite the promising outcomes of applying mixed reality (MR), there is limited aggregated evidence focusing on child-patient interventions in hospital-based or clinical settings. This literature review aimed to identify and synthesize existing knowledge on MR technologies applied to pediatric patients in healthcare settings. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, a comprehensive search of the Scopus and Web of Science databases was conducted to identify articles published in the last 10 years that address the application of augmented and/or MR technologies in pediatric hospital settings or clinical environments to improve patient and family outcomes. A total of 45 articles were identified, and following a rigorous screening and eligibility process, 4 review articles were selected for qualitative synthesis. From these reviews, 10 studies with relevant interventions and measured effects were extracted. The extracted studies were analyzed based on eight key attributes: country of origin, study design, characteristics of the study population, primary clinical setting, type of MR device used, nature of the intervention, variables measured, and significant effects observed in the outcome variables. The analysis revealed diverse approaches across different clinical settings, with a common focus on improving both emotional well-being and learning outcomes in pediatric patients and their families. These findings suggest that MR-based pediatric interventions generally provide children and their parents with positive emotional experiences, enhancing both learning and treatment outcomes. However, the studies reviewed were heterogeneous and varied significantly in terms of clinical settings and MR applications. Future research should focus on developing more controlled study designs that specifically target the pediatric population to strengthen the evidence base for MR interventions in healthcare.

Analytical Methods for Comparing Uncontrolled Trials With External Controls From Real-World Data: A Systematic Literature Review and Comparison With European Regulatory and Health Technology Assessment Practice

OBJECTIVES

This study aimed to provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials with external controls from individual patient data real-world data (IPD-RWD) and to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports.

METHODS

A systematic literature review (until March 1, 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively with methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and 4 European HTA organizations (2015-2023).

RESULTS

Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data with IPD-RWD-based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data, and analytical comparative modeling methods. Seven guidelines also focused on research design, RWD quality, and transparency aspects, and 4 of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n = 15) and HTA (n = 35) assessment reports were often based on aggregate data and lacked transparency owing to the few details provided.

CONCLUSIONS

Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.

Real-world evidence to support regulatory submissions: A landscape review and assessment of use cases

Real-world evidence (RWE) has an increasing role in preapproval settings to support the approval of new medicines and indications. The main objectives of this study were to identify and characterize regulatory use cases that utilized RWE and other related observational approaches through targeted review of publications and regulatory review documents. After screening and inclusion/exclusion, the review characterized 85 regulatory applications with RWE. A total of 31 were in oncology and 54 were in non-oncology therapeutic areas. Most were for indications in adults only (N = 42, 49.4%), while 13 were in pediatrics only (15.3%), and 30 were in both (35.3%). In terms of regulatory context, 59 cases (69.4%) were for an original marketing application, 24 (28.2%) were for label expansion, and 2 (2.4%) were for label modification. Most also received special regulatory designations (e.g., orphan indication, breakthrough therapy, fast track, conditional, and accelerated approvals). There were 42 cases that utilized RWE to support single-arm trials. External data to support single-arm trials were utilized in various ways across use cases, including direct matching, benchmarking, natural history studies as well as literature or previous trials. A variety of data sources were utilized, including electronic health records, claims, registries, site-based charts. Endpoints in oncology use cases commonly included overall survival, progression-free survival. In 13 use cases, RWE was not considered supportive/definitive in regulatory decision-making due to design issues (e.g., small sample size, selection bias, missing data). Overall, RWE is utilized in regulatory approval processes for new indications/label expansion across various therapeutic areas with wide range of approaches. Multifaceted cross-sector efforts are needed to further improve the quality and utility of RWE in regulatory decision-making.

Comparative effectiveness of lazertinib in patients with EGFR T790M-positive non-small-cell lung cancer using a real-world external control

Lazertinib is a recently developed third-generation epidermal growth factor receptor (EGFR)-tyrosine kinase inhibitors used for patients with advanced EGFR T790M-positive non-small-cell lung cancer. We evaluated the effectiveness of lazertinib compared with osimertinib using an external control. We obtained individual patient data for the lazertinib arm from the LASER201 trial and the osimertinib arm from registry data at the Samsung Medical Center. In total, 75 and 110 patients were included in the lazertinib and osimertinib groups, respectively. After propensity score matching, each group had 60 patients and all baseline characteristics were balanced. The median follow-up duration was 22.0 and 29.6 months in the lazertinib and osimertinib group, respectively. The objective response rate (ORR) were 76.7% and 86.7% for lazertinib and osimertinib, respectively (p = 0.08). The median progression-free survival (PFS) was 12.3 months (95% confidence interval [CI] 9.5-19.1) and 14.4 months (95% CI 11.8-18.1) for the lazertinib and osimertinib group, respectively (hazard ratio [HR] 0.97; 95% CI 0.64-1.45, p = 0.86). The median overall survival with lazertinib was not reached and that with osimertinib was 29.8 months (HR 0.44; 95% CI 0.25-0.77, p = 0.005). Our study suggests that lazertinib has an ORR and PFS comparable to those of osimertinib and has the potential for superior survival benefits.

Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

Use of External Control Cohorts in Pediatric Brain Tumor Clinical Trials

Why, when, and how to consider external control cohorts in pediatric brain tumor clinical trials.

Utility of real-world evidence in biosimilar development

Biosimilar development refers to the process of creating a biologic drug that is similar to an existing approved biologic drug, also known as a reference drug. Due to the complex nature of biologics drugs and the inherent variability in their manufacturing process biosimilars are not identical but highly similar to the reference drug in terms of quality, safety, and efficacy. Efficacy and safety trials for biosimilars involve large numbers of patients to confirm comparable clinical performance of the biosimilar and the reference product in appropriately sensitive clinical indications and for appropriate sensitive endpoints. The objective of a biosimilar clinical data is to address slight differences observed at previous steps and to confirm comparable clinical performance of the biosimilar and the reference product. In recent years with advances in big data computing, there has been increasing interest to incorporate the totality of information from different data sources (e.g. Real World data and published literature) in design and conduct of clinical trial to support regulatory objectives. The biosimilar development is an ideal framework for utilization of Real-World Evidence in design of trials as potentially large amount of data are available for the reference dug. Hence there may be an opportunity to use RWD in establishing, improving or validating equivalence margins (EQM) for biosimilar designs, specifically in the case there is no historical published data in the intended sensitive population. In this article, we propose a variation of matching method that seems promising to identify the matched set from a real-world data for which the effect size of targeted endpoint would be comparable to historical data. We believe this is a reasonable approach because in design stage, we can view covariates and secondary endpoints as data feature that can be used in a matching method. This approach was illustrated through a case study which indicated the estimate of the primary endpoint is within 1% of published results and thus RWD may be used to justify or estimate the equivalence margin. To ensure consistent results we recommend using this approach in different indications and endpoint scenarios. Thus utilization of RWD/RWE can provide an important opportunity to increase access to biologic therapies, reducing cost by repurposing existing data.

Propensity score stratified MAP prior and posterior inference for incorporating information across multiple potentially heterogeneous data sources

Incorporation of external information is becoming increasingly common when designing clinical trials. Availability of multiple sources of information has inspired the development of methodologies that account for potential heterogeneity not only between the prospective trial and the pooled external data sources but also between the different external data sources themselves. Our approach proposes an intuitive way of handling such a scenario for the continuous outcomes setting by using propensity score-based stratification and then utilizing robust meta-analytic predictive priors for each stratum to incorporate the prior data to distinguish among different external data sources in each stratum. Through extensive simulations, our approach proves to be more efficient and less biased than the currently available methods. A real case study using clinical trials that study schizophrenia from multiple different sources is also included.

Matching by OS Prognostic Score to Construct External Controls in Lung Cancer Clinical Trials

External controls (eControls) leverage historical data to create non-randomized control arms. The lack of randomization can result in confounding between the experimental and eControl cohorts. To balance potentially confounding variables between the cohorts, one of the proposed methods is to match on prognostic scores. Still, the performance of prognostic scores to construct eControls in oncology has not been analyzed yet. Using an electronic health record-derived de-identified database, we constructed eControls using one of three methods: ROPRO, a state-of-the-art prognostic score, or either a propensity score composed of five (5Vars) or 27 covariates (ROPROvars). We compared the performance of these methods in estimating the overall survival (OS) hazard ratio (HR) of 11 recent advanced non-small cell lung cancer. The ROPRO eControls had a lower OS HR error (median absolute deviation (MAD), 0.072, confidence interval (CI): 0.036-0.185), than the 5Vars (MAD 0.081, CI: 0.025-0.283) and ROPROvars eControls (MAD 0.087, CI: 0.054-0.383). Notably, the OS HR errors for all methods were even lower in the phase III studies. Moreover, the ROPRO eControl cohorts included, on average, more patients than the 5Vars (6.54%) and ROPROvars cohorts (11.7%). The eControls matched with the prognostic score reproduced the controls more reliably than propensity scores composed of the underlying variables. Additionally, prognostic scores could allow eControls to be built on many prognostic variables without a significant increase in the variability of the propensity score, which would decrease the number of matched patients.

Perspectives on informative Bayesian methods in pediatrics

Bayesian methods have been proposed as a natural fit for pediatric extrapolation, as they allow the incorporation of relevant external data to reduce the required sample size and hence trial burden for the pediatric patient population. In this paper we will discuss our experience and perspectives with these methods in pediatric trials. We will present some of the background and thinking underlying pediatric extrapolation and discuss the use of Bayesian methods within this context. We will present two recent case examples illustrating the value of a Bayesian approach in this setting and present perspectives on some of the issues that we have encountered in these and other cases.

Addressing statistical issues when leveraging external control data in pediatric clinical trials using Bayesian dynamic borrowing

Conducting a well-powered and adequately controlled clinical trial in children is often challenging. Bayesian approaches are an attractive option for addressing such challenges as they provide a quantitatively rigorous and integrated framework that makes use of current control data to check and borrow information from historical control data. However various practical concerns and related statistical issues emerge when implementing such Bayesian borrowing approaches. In this manuscript we use a motivating case study to discuss a rigorous stepwise approach on how to address those issues within the Bayesian framework. Specifically, a comprehensive quantitative framework is proposed to assess the extent, synergy, and impact of borrowing. Steps on computing the measures to interpret borrowing are illustrated. Those measures can further help to determine whether additional discounting of external information is necessary.

Utility of propensity score-based Bayesian borrowing of external adult data in pediatric trials: A pragmatic evaluation through a case study in acute lymphoblastic leukemia (ALL)

A fully powered randomized controlled cancer trial can be challenging to conduct in children because of difficulties in enrollment of pediatric patients due to low disease incidence. One way to improve the feasibility of trials in pediatric patients, when clinically appropriate, is through borrowing information from comparable external adult trials in the same disease. Bayesian analysis of a pediatric trial provides a way of seamlessly augmenting pediatric trial efficacy data with data from external adult trials. However, not all external adult trial subjects may be equally clinically relevant with respect to the baseline disease severity, prognostic factors, co-morbidities, and prior therapy observed in the pediatric trial of interest. The propensity score matching method provides a way of matching the external adult subjects to the pediatric trial subjects on a set of clinically determined baseline covariates, such as baseline disease severity, prognostic factors and prior therapy. The matching then allows Bayesian information borrowing from only the most clinically relevant external adult subjects. Through a case study in pediatric acute lymphoblastic leukemia (ALL), we examine the utility of propensity score matched mixture and power priors in bringing appropriate external adult efficacy information into pediatric trial efficacy assessment, and present considerations for scaling fixed borrowing from external adult data.

Enriching single-arm clinical trials with external controls: possibilities and pitfalls

For the past decade, it has become commonplace to provide rapid answers and early patient access to innovative treatments in the absence of randomized clinical trials (RCT), with benefits estimated from single-arm trials. This trend is important in oncology, notably when assessing new targeted therapies. Some of those uncontrolled trials further include an external/synthetic control group as an innovative way to provide an indirect comparison with a pertinent control group. We aimed to provide some guidelines as a comprehensive tool for (1) the critical appraisal of those comparisons or (2) for performing a single-arm trial. We used the example of ciltacabtagene autoleucel for the treatment of adult patients with relapsed or refractory multiple myeloma after 3 or more treatment lines as an illustrative example. We propose a 3-step guidance. The first step includes the definition of an estimand, which encompasses the treatment effect and the targeted population (whole population or restricted to single-arm trial or external controls), reflecting a clinical question. The second step relies on the adequate selection of external controls from previous RCTs or real-world data from patient cohorts, registries, or electronic patient files. The third step consists of choosing the statistical approach targeting the treatment effect defined above and depends on the available data (individual-level data or aggregated external data). The validity of the treatment effect derived from indirect comparisons heavily depends on careful methodological considerations included in the proposed 3-step procedure. Because the level of evidence of a well-conducted RCT cannot be guaranteed, the evaluation is more important than in standard settings.

BASIC: A Bayesian adaptive synthetic-control design for phase II clinical trials

BACKGROUND

Randomized controlled trials are considered the gold standard for evaluating experimental treatments but often require large sample sizes. Single-arm trials require smaller sample sizes but are subject to bias when using historical control data for comparative inferences. This article presents a Bayesian adaptive synthetic-control design that exploits historical control data to create a hybrid of a single-arm trial and a randomized controlled trial.

METHODS

The Bayesian adaptive synthetic control design has two stages. In stage 1, a prespecified number of patients are enrolled in a single arm given the experimental treatment. Based on the stage 1 data, applying propensity score matching and Bayesian posterior prediction methods, the usefulness of the historical control data for identifying a pseudo sample of matched synthetic-control patients for making comparative inferences is evaluated. If a sufficient number of synthetic controls can be identified, the single-arm trial is continued. If not, the trial is switched to a randomized controlled trial. The performance of The Bayesian adaptive synthetic control design is evaluated by computer simulation.

RESULTS

The Bayesian adaptive synthetic control design achieves power and unbiasedness similar to a randomized controlled trial but on average requires a much smaller sample size, provided that the historical control data patients are sufficiently comparable to the trial patients so that a good number of matched controls can be identified in the historical control data. Compared to a single-arm trial, The Bayesian adaptive synthetic control design yields much higher power and much smaller bias.

CONCLUSION

The Bayesian adaptive synthetic-control design provides a useful tool for exploiting historical control data to improve the efficiency of single-arm phase II clinical trials, while addressing the problem of bias when comparing trial results to historical control data. The proposed design achieves power similar to a randomized controlled trial but may require a substantially smaller sample size.

Pharmacometrics enhanced Bayesian borrowing approach to improve clinical trial efficiency: Case of empagliflozin in type 2 diabetes

We report use of a pharmacometrics enhanced Bayesian borrowing (PEBB) approach to leverage historical clinical trial data on a drug product to build models, project the outcome of future clinical trials, and borrow information from these projections to improve the efficiency of future target trials. This design takes a two-stage approach. First, a design phase is performed before target trial data are available to determine the operating characteristics and an appropriate tuning parameter that will be used in the subsequent analysis phase of a chosen target trial. Second, once the target trial data are available, the analysis phase is performed with the determined tuning parameter. This step is where borrowing is applied from these projections to inform the results for the target trial. To illustrate how a PEBB could improve the efficiency of clinical trials, we apply our design to trials with empagliflozin for treating patients with type 2 diabetes. We performed a retrospective evaluation applying the method to a phase III target trial and a hypothetical smaller trial. The type I error could be kept below 10% while increasing the trial power and effective sample size. Our findings suggest that a PEBB has the potential to increase the power of clinical trials, while controlling for type I error, by leveraging the information from previous trials through population pharmacokinetic/pharmacodynamic modeling and simulation.

Hybrid Controlled Clinical Trials Using Concurrent Registries in Amyotrophic Lateral Sclerosis: A Feasibility Study

Hybrid designs with both randomized arms and an external control cohort preserve key features of randomization and utilize external information to augment clinical trials. In this study, we propose to leverage high-quality, patient-level concurrent registries to enhance clinical trials and illustrate the impact on trial design for amyotrophic lateral sclerosis. The proposed methodology was evaluated in a randomized, placebo-controlled clinical trial. We used patient-level information from a well-defined, population-based registry, that was running parallel to the randomized clinical trial, to identify concurrently nonparticipating, eligible patients who could be matched with trial participants, and integrate them into the statistical analysis. We assessed the impact of the addition of the external controls on the treatment effect estimate, precision, and time to reach a conclusion. During the runtime of the trial, a total of 1,141 registry patients were alive; 473 (41.5%) of them fulfilled the eligibility criteria and 133 (11.7%) were enrolled in the study. A matched control population could be identified among the nonparticipating patients. Augmenting the randomized controls with matched external controls could have avoided unnecessary randomization of 17 patients (-12.8%) as well as reducing the study duration from 30.1 months to 22.6 months (-25.0%). Matching eligible external controls from a different calendar period led to bias in the treatment effect estimate. Hybrid trial designs utilizing a concurrent registry with rigorous matching can minimize bias due to a mismatch in calendar time and differences in standard of care, and may accelerate the development of new treatments.

Covariate handling approaches in combination with dynamic borrowing for hybrid control studies

Borrowing data from external control has been an appealing strategy for evidence synthesis when conducting randomized controlled trials (RCTs). Often named hybrid control trials, they leverage existing control data from clinical trials or potentially real-world data (RWD), enable trial designs to allocate more patients to the novel intervention arm, and improve the efficiency or lower the cost of the primary RCT. Several methods have been established and developed to borrow external control data, among which the propensity score methods and Bayesian dynamic borrowing framework play essential roles. Noticing the unique strengths of propensity score methods and Bayesian hierarchical models, we utilize both methods in a complementary manner to analyze hybrid control studies. In this article, we review methods including covariate adjustments, propensity score matching and weighting in combination with dynamic borrowing and compare the performance of these methods through comprehensive simulations. Different degrees of covariate imbalance and confounding are examined. Our findings suggested that the conventional covariate adjustment in combination with the Bayesian commensurate prior model provides the highest power with good type I error control under the investigated settings. It has desired performance especially under scenarios of different degrees of confounding. To estimate efficacy signals in the exploratory setting, the covariate adjustment method in combination with the Bayesian commensurate prior is recommended.

On the use of non-concurrent controls in platform trials: a scoping review

BACKGROUND

Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.

METHODS

We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.

RESULTS

Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.

CONCLUSIONS

Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

Dynamic use of historical controls in clinical trials for rare disease research: A re-evaluation of the MILES trial

BACKGROUND

Randomized controlled trials offer the best design for eliminating bias in estimating treatment effects but can be slow and costly in rare disease research. Additionally, an equal randomization approach may not be optimal in studies in which prior evidence of superiority of one or more treatments exist. Supplementing prospectively enrolled, concurrent controls with historical controls can reduce recruitment requirements and provide patients a higher likelihood of enrolling in a new and possibly superior treatment arm. Appropriate methods need to be employed to ensure comparability of concurrent and historical controls to minimize bias and variability in the treatment effect estimates and reduce the chances of drawing incorrect conclusions regarding treatment benefit.

METHODS

MILES was a phase III placebo-controlled trial employing 1:1 randomization that led to US Food and Drug Administration approval of sirolimus for treating patients with lymphangioleiomyomatosis. We re-analyzed the MILES trial data to learn whether substituting concurrent controls with controls from a historical registry could have accelerated subject enrollment while leading to similar study conclusions. We used propensity score matching to identify exchangeable historical controls from a registry balancing the baseline characteristics of the two control groups. This allowed more new patients to be assigned to the sirolimus arm. We used trial data and simulations to estimate key outcomes under an array of alternative designs.

RESULTS

Borrowing information from historical controls would have allowed the trial to enroll fewer concurrent controls while leading to the same conclusion reached in the trial. Simulations showed similar statistical performance for borrowing as for the actual trial design without producing type I error inflation and preserving power for the same study size when concurrent and historical controls are comparable.

CONCLUSION

Substituting concurrent controls with propensity score-matched historical controls can allow more prospectively enrolled patients to be assigned to the active treatment and enable the trial to be conducted with smaller overall sample size, while maintaining covariate balance and study power and minimizing bias in response estimation. This approach does not fully eliminate the concern that introducing non-randomized historical controls in a trial may lead to bias in estimating treatment effects, and should be carefully considered on a case-by-case basis. Borrowing historical controls is best suited when conducting randomized controlled trials with conventional designs is challenging, as in rare disease research. High-quality data on covariates and outcomes must be available for candidate historical controls to ensure the validity of these designs. Additional precautions are needed to maintain blinding of the treatment assignment and to ensure comparability in the assessment of treatment safety.MILES ClinicalTrials.gov Number: NCT00414648.

Development of an External Control Arm Using Electronic Health Record-Based Real-World Data to Evaluate the Efficacy of COVID-19 Treatment

To protect people from severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) infection, tremendous research efforts have been made toward coronavirus disease 19 (COVID-19) treatment development. Externally controlled trials (ECTs) may help reduce their development time. To evaluate whether ECT using real-world data (RWD) of patients with COVID-19 is feasible enough to be used for regulatory decision making, we built an external control arm (ECA) based on RWD as a control arm of a previously conducted randomized controlled trial (RCT), and compared it to the control arm of the RCT. The electronic health record (EHR)-based COVID-19 cohort dataset was used as RWD, and three Adaptive COVID-19 Treatment Trial (ACTT) datasets were used as RCTs. Among the RWD datasets, eligible patients were evaluated as a pool of external control subjects of the ACTT-1, ACTT-2, and ACTT-3 trials, respectively. The ECAs were built using propensity score matching, and the balance of age, sex, and baseline clinical status ordinal scale as covariates between the treatment arms of Asian patients in each ACTT and the pools of external control subjects was assessed before and after 1:1 matching. There was no statistically significant difference in time to recovery between ECAs and the control arms of each ACTT. Among the covariates, the baseline status ordinal score had the greatest influence on the building of ECA. This study demonstrates that ECA based on EHR data of COVID-19 patients could sufficiently replace the control arm of an RCT, and it is expected to help develop new treatments faster in emergency situations, such as the COVID-19 pandemic.

Improving efficiency of inference in clinical trials with external control data

Suppose we are interested in the effect of a treatment in a clinical trial. The efficiency of inference may be limited due to small sample size. However, external control data are often available from historical studies. Motivated by an application to Helicobacter pylori infection, we show how to borrow strength from such data to improve efficiency of inference in the clinical trial. Under an exchangeability assumption about the potential outcome mean, we show that the semiparametric efficiency bound for estimating the average treatment effect can be reduced by incorporating both the clinical trial data and external controls. We then derive a doubly robust and locally efficient estimator. The improvement in efficiency is prominent especially when the external control data set has a large sample size and small variability. Our method allows for a relaxed overlap assumption, and we illustrate with the case where the clinical trial only contains a treated group. We also develop doubly robust and locally efficient approaches that extrapolate the causal effect in the clinical trial to the external population and the overall population. Our results also offer a meaningful implication for trial design and data collection. We evaluate the finite-sample performance of the proposed estimators via simulation. In the Helicobacter pylori infection application, our approach shows that the combination treatment has potential efficacy advantages over the triple therapy.

Current perspectives for external control arms in oncology clinical trials: Analysis of EMA approvals 2016-2021

Leveraging external control data, especially real-world data (RWD), has drawn particular attention in recent years for facilitating oncology clinical development and regulatory decision-making. Medical regulators have published guidance on accelerating the use of RWD and external controls. However, few systematic discussions have been conducted on external controls in cancer drug submissions and regulatory feedback. This study aimed to identify European oncology drug approvals using external control data to demonstrate clinical efficacy. We included 18 eligible submissions employing 24 external controls and then discussed the use of external control, data sources, analysis methods, and regulators' feedback. The external controls have been actively submitted to the European Medical Agency (EMA) recently. We found that 17 % of the EMA-approved cancer drugs in 2016-2021 used external controls, among which 37 % of the cases leveraged RWD. However, nearly one-third of the external controls were not considered supportive evidence by EMA due to limitations regarding heterogeneous patient populations, missing outcome assessment in RWD, and inappropriate statistical analysis. This study highlighted that proper use of external controls requires a careful assessment of clinical settings, data availability, and statistical methodology. For better use of external controls in oncology clinical trials, we recommend: prospective study designs to avoid selection bias, sufficient baseline data to ensure the comparability of study populations, consistent endpoint measurements to enable outcome comparison, robust statistical methodology for comparative analysis, and collaborative efforts of sponsors and regulators to establish regulatory frameworks.

Unit information prior for incorporating real-world evidence into randomized controlled trials

Randomized controlled trials (RCTs) have been widely recognized as the gold standard to infer the treatment effect in clinical research. Recently, there has been growing interest in enhancing and complementing the result in an RCT by integrating real-world evidence from observational studies. The unit information prior (UIP) is a newly proposed technique that can effectively borrow information from multiple historical datasets. We extend this generic approach to synthesize the non-randomized evidence into a current RCT. Not only does the UIP only require summary statistics published from observational studies for ease of implementation, but it also has clear interpretations and can alleviate the potential bias in the real-world evidence via weighting schemes. Extensive numerical experiments show that the UIP can improve the statistical efficiency in estimating the treatment effect for various types of outcome variables. The practical potential of our UIP approach is further illustrated with a real trial of hydroxychloroquine for treating COVID-19 patients.

Real-world-Daten in der Arzneimittelregulation – aktuelle Entwicklungen und Ausblick

Hintergrund

Real-world-Daten (RWD) und die aus ihnen durch statistisch-epidemiologische Analysen abgeleitete Real-world-Evidenz (RWE) spielen eine vielversprechende und zunehmend relevante Rolle bei regulatorischen Entscheidungsfindungen entlang des Produktlebenszyklus von Arzneimitteln.

Ziel der Arbeit

Es wird ein Überblick über den aktuellen Stand, die Entwicklungspotenziale und Initiativen zur verstärkten Nutzung von RWE in der europäischen und internationalen Arzneimittelregulation gegeben.

Material und Methoden

Die Grundlagen für die Übersichtsarbeit sind Originalarbeiten und Reviews aus der aktuellen internationalen Literatur (inklusive eigener Forschungsergebnisse), aktuelle Beispiele aus der regulatorischen Praxis sowie die Einbindung in europäische und internationale Initiativen zur verstärkten Nutzung von RWD/RWE in regulatorischen Entscheidungsprozessen.

Ergebnisse

Aktuell primär zur supportiven Evidenz bei regulatorischen Entscheidungsfindungen wird RWE aus RWD eingesetzt. Neben dem etablierten Einsatz in Phasen nach der Zulassung (z. B. Überwachung der Arzneimittelsicherheit), werden RWD zunehmend auch in der Phase vor der Zulassung und in der Evaluation eingesetzt. Aktuell wird durch verstärkte Vernetzung der Datenquellen auf nationaler und internationaler Ebene eine Gesundheitsdateninfrastruktur aufgebaut, um neue Möglichkeiten zur RWD-Nutzung zu schaffen.

Schlussfolgerung

Neben einer wachsenden Bedeutung von RWD/RWE in der europäischen und internationalen Arzneimittelregulation ergeben sich auch neue Herausforderungen zum Zugang zu und zur Analyse von RWD. Die Variabilität und Heterogenität der RWD-Quellen machen die Entwicklung neuer und optimierter Methoden für RWD-Analysen unerlässlich. Auch neue Leitfaden- und Schulungskonzepte für die beteiligten Stakeholder sind essenziell.

Data Integration in Causal Inference

Integrating data from multiple heterogeneous sources has become increasingly popular to achieve a large sample size and diverse study population. This paper reviews development in causal inference methods that combines multiple datasets collected by potentially different designs from potentially heterogeneous populations. We summarize recent advances on combining randomized clinical trial with external information from observational studies or historical controls, combining samples when no single sample has all relevant variables with application to two-sample Mendelian randomization, distributed data setting under privacy concerns for comparative effectiveness and safety research using real-world data, Bayesian causal inference, and causal discovery methods.

A meta-analytic framework to adjust for bias in external control studies

While randomized controlled trials (RCTs) are the gold standard for estimating treatment effects in medical research, there is increasing use of and interest in using real-world data for drug development. One such use case is the construction of external control arms for evaluation of efficacy in single-arm trials, particularly in cases where randomization is either infeasible or unethical. However, it is well known that treated patients in non-randomized studies may not be comparable to control patients-on either measured or unmeasured variables-and that the underlying population differences between the two groups may result in biased treatment effect estimates as well as increased variability in estimation. To address these challenges for analyses of time-to-event outcomes, we developed a meta-analytic framework that uses historical reference studies to adjust a log hazard ratio estimate in a new external control study for its additional bias and variability. The set of historical studies is formed by constructing external control arms for historical RCTs, and a meta-analysis compares the trial controls to the external control arms. Importantly, a prospective external control study can be performed independently of the meta-analysis using standard causal inference techniques for observational data. We illustrate our approach with a simulation study and an empirical example based on reference studies for advanced non-small cell lung cancer. In our empirical analysis, external control patients had lower survival than trial controls (hazard ratio: 0.907), but our methodology is able to correct for this bias. An implementation of our approach is available in the R package ecmeta.

Bayesian Methods in Human Drug and Biological Products Development in CDER and CBER

The Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) of the U.S. Food and Drug Administration (FDA) have been leaders in protecting and promoting the U.S. public health by helping to ensure that safe and effective drugs and biological products are available in the United States for those who need them. The null hypothesis significance testing approach, along with other considerations, is typically used to demonstrate the effectiveness of a drug or biological product. The Bayesian framework presents an alternative approach to demonstrate the effectiveness of a treatment. This article discusses the Bayesian framework for drug and biological product development, highlights key settings in which Bayesian approaches may be appropriate, and provides recent examples of the use of Bayesian approaches within CDER and CBER.

Natural History and Real-World Data in Rare Diseases: Applications, Limitations, and Future Perspectives

Rare diseases represent a highly heterogeneous group of disorders with high phenotypic and genotypic diversity within individual conditions. Due to the small numbers of people affected, there are unique challenges in understanding rare diseases and drug development for these conditions, including patient identification and recruitment, trial design, and costs. Natural history data and real-world data (RWD) play significant roles in defining and characterizing disease progression, final patient populations, novel biomarkers, genetic relationships, and treatment effects. This review provides an introduction to rare diseases, natural history data, RWD, and real-world evidence, the respective sources and applications of these data in several rare diseases. Considerations for data quality and limitations when using natural history and RWD are also elaborated. Opportunities are highlighted for cross-sector collaboration, standardized and high-quality data collection using new technologies, and more comprehensive evidence generation using quantitative approaches such as disease progression modeling, artificial intelligence, and machine learning. Advanced statistical approaches to integrate natural history data and RWD to further disease understanding and guide more efficient clinical study design and data analysis in drug development in rare diseases are also discussed.

Augmenting control arms with real-world data for cancer trials: Hybrid control arm methods and considerations

Background

Hybrid controlled trials with real-world data (RWD), where the control arm is composed of both trial and real-world patients, could facilitate research when the feasibility of randomized controlled trials (RCTs) is challenging and single-arm trials would provide insufficient information.

Methods

We propose a frequentist two-step borrowing method to construct hybrid control arms. We use parameters informed by a completed randomized trial in metastatic triple-negative breast cancer to simulate the operating characteristics of dynamic and static borrowing methods, highlighting key trade-offs and analytic decisions in the design of hybrid studies.

Results

Simulated data were generated under varying residual-bias assumptions (no bias: HR_RWD = 1) and experimental treatment effects (target trial scenario: HR_Exp = 0.78). Under the target scenario with no residual bias, all borrowing methods achieved the desired 88% power, an improvement over the reference model (74% power) that does not borrow information externally. The effective number of external events tended to decrease with higher bias between RWD and RCT (i.e. HR_RWD away from 1), and with weaker experimental treatment effects (i.e. HR_Exp closer to 1). All dynamic borrowing methods illustrated (but not the static power prior) cap the maximum Type 1 error over the residual-bias range considered. Our two-step model achieved comparable results for power, type 1 error, and effective number of external events borrowed compared to other borrowing methodologies.

Conclusion

By pairing high-quality external data with rigorous simulations, researchers have the potential to design hybrid controlled trials that better meet the needs of patients and drug development.

Estimands and Complex Innovative Designs

Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.

On model-based time trend adjustments in platform trials with non-concurrent controls

BACKGROUND

Platform trials can evaluate the efficacy of several experimental treatments compared to a control. The number of experimental treatments is not fixed, as arms may be added or removed as the trial progresses. Platform trials are more efficient than independent parallel group trials because of using shared control groups. However, for a treatment entering the trial at a later time point, the control group is divided into concurrent controls, consisting of patients randomised to control when that treatment arm is in the platform, and non-concurrent controls, patients randomised before. Using non-concurrent controls in addition to concurrent controls can improve the trial's efficiency by increasing power and reducing the required sample size, but can introduce bias due to time trends.

METHODS

We focus on a platform trial with two treatment arms and a common control arm. Assuming that the second treatment arm is added at a later time, we assess the robustness of recently proposed model-based approaches to adjust for time trends when utilizing non-concurrent controls. In particular, we consider approaches where time trends are modeled either as linear in time or as a step function, with steps at time points where treatments enter or leave the platform trial. For trials with continuous or binary outcomes, we investigate the type 1 error rate and power of testing the efficacy of the newly added arm, as well as the bias and root mean squared error of treatment effect estimates under a range of scenarios. In addition to scenarios where time trends are equal across arms, we investigate settings with different time trends or time trends that are not additive in the scale of the model.

RESULTS

A step function model, fitted on data from all treatment arms, gives increased power while controlling the type 1 error, as long as the time trends are equal for the different arms and additive on the model scale. This holds even if the shape of the time trend deviates from a step function when patients are allocated to arms by block randomisation. However, if time trends differ between arms or are not additive to treatment effects in the scale of the model, the type 1 error rate may be inflated.

CONCLUSIONS

The efficiency gained by using step function models to incorporate non-concurrent controls can outweigh potential risks of biases, especially in settings with small sample sizes. Such biases may arise if the model assumptions of equality and additivity of time trends are not satisfied. However, the specifics of the trial, scientific plausibility of different time trends, and robustness of results should be carefully considered.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Hybrid-control arm construction using historical trial data for an early-phase, randomized controlled trial in metastatic colorectal cancer

Background

Treatment for metastatic colorectal cancer patients beyond the second line remains challenging, highlighting the need for early phase trials of combination therapies for patients who had disease progression during or following two prior lines of therapy. Leveraging hybrid control design in these trials may preserve the benefits of randomization while strengthening evidence by integrating historical trial data. Few examples have been established to assess the applicability of such design in supporting early phase metastatic colorectal cancer trials.

Methods

MORPHEUS-CRC is an umbrella, multicenter, open-label, phase Ib/II, randomized, controlled trial (NCT03555149), with active experimental arms ongoing. Patients enrolled were assigned to a control arm (regorafenib, 15 patients randomized and 13 analysed) or multiple experimental arms for immunotherapy-based treatment combinations. One experimental arm (atezolizumab + isatuximab, 15 patients randomized and analysed) was completed and included in the hybrid-control study, where the hybrid-control arm was constructed by integrating data from the IMblaze370 phase 3 trial (NCT02788279). To estimate treatment efficacy, Cox and logistic regression models were used in a frequentist framework with standardized mortality ratio weighting or in a Bayesian framework with commensurate priors. The primary endpoint is objective response rate, while disease control rate, progression-free survival, and overall survival were the outcomes assessed in the hybrid-control study.

Results

The experimental arm showed no efficacy signal, yet a well-tolerated safety profile in the MORPHEUS-CRC trial. Treatment effects estimated in hybrid control design were comparable to those in the MORPHEUS-CRC trial using either frequentist or Bayesian models.

Conclusions

Hybrid control provides comparable treatment-effect estimates with generally improved precision, and thus can be of value to inform early-phase clinical development in metastatic colorectal cancer.

Bayesian Design of Superiority Trials: Methods and Applications

In this paper, we lay out the basic elements of Bayesian sample size determination (SSD) for the Bayesian design of a two-arm superiority clinical trial. We develop a flowchart of the Bayesian SSD that highlights the critical components of a Bayesian design and provides a practically useful roadmap for designing a Bayesian clinical trial in real world applications. We empirically examine the amount of borrowing, the choice of noninformative priors, and the impact of model misspecification on the Bayesian type I error and power. A formal and statistically rigorous formulation of conditional borrowing within the decision rule framework is developed. Moreover, by extending the partial borrowing power priors, a new borrowing-by-parts power prior for incorporating historical data is proposed. Computational algorithms are also developed to calculate the Bayesian type I error and power. Extensive simulation studies are carried out to explore the operating characteristics of the proposed Bayesian design of a superiority trial.

Using observational study data as an external control group for a clinical trial: an empirical comparison of methods to account for longitudinal missing data

Background

Observational data are increasingly being used to conduct external comparisons to clinical trials. In this study, we empirically examined whether different methodological approaches to longitudinal missing data affected study conclusions in this setting.

Methods

We used data from one clinical trial and one prospective observational study, both Norwegian multicenter studies including patients with recently diagnosed rheumatoid arthritis and implementing similar treatment strategies, but with different stringency. A binary disease remission status was defined at 6, 12, and 24 months in both studies. After identifying patterns of longitudinal missing outcome data, we evaluated the following five approaches to handle missingness: analyses of patients with complete follow-up data, multiple imputation (MI), inverse probability of censoring weighting (IPCW), and two combinations of MI and IPCW.

Results

We found a complex non-monotone missing data pattern in the observational study (N = 328), while missing data in the trial (N = 188) was monotone due to drop-out. In the observational study, only 39.0% of patients had complete outcome data, compared to 89.9% in the trial. All approaches to missing data indicated favorable outcomes of the treatment strategy in the trial and resulted in similar study conclusions. Variations in results across approaches were mainly due to variations in estimated outcomes for the observational data.

Conclusions

Five different approaches to handle longitudinal missing data resulted in similar conclusions in our example. However, the extent and complexity of missing observational data affected estimated comparative outcomes across approaches, highlighting the need for careful consideration of methods to account for missingness in this setting. Based on this empirical examination, we recommend using a prespecified advanced missing data approach to account for longitudinal missing data, and to conduct alternative approaches in sensitivity analyses.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01639-0.

Demonstrating Benefit-Risk Profiles of Novel Therapeutic Strategies in Kidney Transplantation: Opportunities and Challenges of Real-World Evidence

While great progress has been made in transplantation medicine, long-term graft failure and serious side effects still pose a challenge in kidney transplantation. Effective and safe long-term treatments are needed. Therefore, evidence of the lasting benefit-risk of novel therapies is required. Demonstrating superiority of novel therapies is unlikely via conventional randomized controlled trials, as long-term follow-up in large sample sizes pose statistical and operational challenges. Furthermore, endpoints generally accepted in short-term clinical trials need to be translated to real-world (RW) care settings, enabling robust assessments of novel treatments. Hence, there is an evidence gap that calls for innovative clinical trial designs, with RW evidence (RWE) providing an opportunity to facilitate longitudinal transplant research with timely translation to clinical practice. Nonetheless, the current RWE landscape shows considerable heterogeneity, with few registries capturing detailed data to support the establishment of new endpoints. The main recommendations by leading scientists in the field are increased collaboration between registries for data harmonization and leveraging the development of technology innovations for data sharing under high privacy standards. This will aid the development of clinically meaningful endpoints and data models, enabling future long-term research and ultimately establish optimal long-term outcomes for transplant patients.

Improving pediatric multiple sclerosis interventional phase III study design: a meta-analysis

Background:

To support innovative trial designs in a regulatory setting for pediatric-onset multiple sclerosis (MS), the study aimed to perform a systematic literature review and meta-analysis of relapse rates with interferon β (IFN β), fingolimod, and natalizumab and thereby demonstrate potential benefits of Bayesian and non-inferiority designs in this population.

Methods:

We conducted a literature search in MEDLINE and EMBASE from inception until 17 June 2020 of all studies reporting annualized relapse rates (ARR) in IFN β-, fingolimod-, or natalizumab-treated patients with pediatric-onset relapsing–remitting MS. These interventions were chosen because the literature was mainly available for these treatments, and they are currently used for the treatment of pediatric MS. Two researchers independently extracted data and assessed study quality using the Cochrane Effective Practice and Organization of Care – Quality Assessment Tool. The meta-analysis estimates were obtained by Bayesian random effects model. Data were summarized as ARR point estimates and 95% credible intervals.

Results:

We found 19 articles, including 2 randomized controlled trials. The baseline ARR reported was between 1.4 and 3.7. The meta-analysis-based ARR was significantly higher in IFN β-treated patients (0.69, 95% credible interval: 0.51–0.91) versus fingolimod (0.11, 0.04–0.27) and natalizumab (0.17, 0.09–0.31). Based on the meta-analysis results, an appropriate non-inferiority margin versus fingolimod could be in the range of 2.29–2.67 and for natalizumab 1.72–2.29 on the ARR ratio scale. A Bayesian design, which uses historical information for a fingolimod or natalizumab control arm, could reduce the sample size of a new trial by 18 or 14 patients, respectively.

Conclusion:

This meta-analysis provides evidence that relapse rates are considerably higher with IFNs versus fingolimod or natalizumab. The results support the use of innovative Bayesian or non-inferiority designs to avoid exposing patients to less effective comparators in trials and bringing new medications to patients more efficiently.

A Bayesian model with application for adaptive platform trials having temporal changes

Temporal changes exist in clinical trials. Over time, shifts in patients' characteristics, trial conduct, and other features of a clinical trial may occur. In typical randomized clinical trials, temporal effects, i.e., the impact of temporal changes on clinical outcomes and study analysis, are largely mitigated by randomization and usually need not be explicitly addressed. However, temporal effects can be a serious obstacle for conducting clinical trials with complex designs, including the adaptive platform trials that are gaining popularity in recent medical product development. In this paper, we introduce a Bayesian robust prior for mitigating temporal effects based on a hidden Markov model, and propose a particle filtering algorithm for computation. We conduct simulation studies to evaluate the performance of the proposed method and provide illustration examples based on trials of Ebola virus disease therapeutics and hemostat in vascular surgery. This article is protected by copyright. All rights reserved.

Real-world data in drug development strategies for orphan drugs: tafasitamab in B cell lymphoma, a case study for approval based on a single-arm combination trial

Tafasitamab (TAF) plus lenalidomide (LEN) is a novel treatment option for patients with relapsed/refractory diffuse large B cell lymphoma (rrDLBCL) who are not eligible for autologous stem cell transplantation. The initial US/EU approvals for TAF represent precedents because this is the first time that approval of a novel combination therapy was granted based on a pivotal single-arm trial (SAT). Matching real world-data (RWD) helped to disentangle the contribution of individual agents. In this review, we present the TAF development strategy, the prospective incorporation of RWD within the clinical development plan, the corresponding regulatory hurdles of this strategy, and the prior regulatory actions for other cancer drugs that previously incorporated RWD and propensity score matching in EU and US regulatory submissions. We also outline how RWD could further advance and impact orphan drug development.

The use of external control data for predictions and futility interim analyses in clinical trials

BACKGROUND

External control (EC) data from completed clinical trials and electronic health records can be valuable for the design and analysis of future clinical trials. We discuss the use of EC data for early stopping decisions in randomized clinical trials (RCTs).

METHODS

We specify interim analyses (IAs) approaches for RCTs, which allow investigators to integrate external data into early futility stopping decisions. IAs utilize predictions based on early data from the RCT, possibly combined with external data. These predictions at IAs express the probability that the trial will generate significant evidence of positive treatment effects. The trial is discontinued if this predictive probability becomes smaller than a prespecified threshold. We quantify efficiency gains and risks associated with the integration of external data into interim decisions. We then analyze a collection of glioblastoma (GBM) data sets, to investigate if the balance of efficiency gains and risks justify the integration of external data into the IAs of future GBM RCTs.

RESULTS

Our analyses illustrate the importance of accounting for potential differences between the distributions of prognostic variables in the RCT and in the external data to effectively leverage external data for interim decisions. Using GBM data sets, we estimate that the integration of external data increases the probability of early stopping of ineffective experimental treatments by up to 25% compared to IAs that do not leverage external data. Additionally, we observe a reduction of the probability of early discontinuation for effective experimental treatments, which improves the RCT power.

CONCLUSION

Leveraging external data for IAs in RCTs can support early stopping decisions and reduce the number of enrolled patients when the experimental treatment is ineffective.

Similar Clinical Outcomes in Patients with Systemic Juvenile Idiopathic Arthritis and Adult-Onset Still's Disease Treated with Canakinumab: Bayesian and Population Model-Based Analyses

Introduction

Systemic juvenile idiopathic arthritis (sJIA) and adult-onset Still’s disease (AOSD) represent pediatric and adult variants of the Still’s disease continuum. To determine whether clinical outcomes between patients with sJIA and AOSD were similar, Bayesian and population model-based analyses were conducted on endpoints from studies of canakinumab in both patient populations. The objective was to further support the efficacy of canakinumab in patients with AOSD.

Methods

A Bayesian analysis of endpoints from a study of canakinumab in AOSD was conducted borrowing information from five pooled sJIA studies using a robust meta-analytic predictive (MAP) approach. Similarity of clinical outcomes across populations was fulfilled if the AOSD study posterior median fell within the 95% predicted credible interval for the outcome of interest, based on the pooled sJIA data. Population model-based analyses (pharmacokinetic [PK] and PK/pharmacodynamic [PKPD]) were conducted to compare the pharmacokinetics and exposure–response relationships between populations.

Results

The AOSD study posterior medians for adapted American College of Rheumatology (ACR)30 response, continuous adapted ACR response, number of active joints, C-reactive protein, and absence of fever were within the 95% credible interval for the prediction of the MAP analysis from the pooled sJIA data, supporting the similarity in outcomes between patient populations. PK analysis demonstrated comparable exposure across sJIA age groups and patients with AOSD. PKPD relationships were consistent across patient populations. Analyses indicated that no therapeutic benefit can be expected from a dose increase in patients with AOSD.

Conclusion

The analyses presented support the similarity of clinical outcomes following treatment with canakinumab in patients with sJIA and AOSD.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40744-021-00422-9.

Application of Real-World Data to External Control Groups in Oncology Clinical Trial Drug Development

Randomized controlled trials (RCTs) that assess overall survival are considered the "gold standard" when evaluating the efficacy and safety of a new oncology intervention. However, single-arm trials that use surrogate endpoints (e.g., objective response rate or duration of response) to evaluate clinical benefit have become the basis for accelerated or breakthrough regulatory approval of precision oncology drugs for cases where the target and research populations are relatively small. Interpretation of efficacy in single-arm trials can be challenging because such studies lack a standard-of-care comparator arm. Although an external control group can be based on data from other clinical trials, using an external control group based on data collected outside of a trial may not only offer an alternative to both RCTs and uncontrolled single-arm trials, but it may also help improve decision-making by study sponsors or regulatory authorities. Hence, leveraging real-world data (RWD) to construct external control arms in clinical trials that investigate the efficacy and safety of drug interventions in oncology has become a topic of interest. Herein, we review the benefits and challenges associated with the use of RWD to construct external control groups, and the relevance of RWD to early oncology drug development.