Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials

Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations

BACKGROUND AND OBJECTIVE

Missing data and unmeasured confounding are key challenges for external comparator studies. This work evaluates bias and other performance characteristics depending on missingness and unmeasured confounding by means of two case studies and simulations.

METHODS

Two case studies were constructed by taking the treatment arms from two randomised controlled trials and an external real-world data source that exhibited substantial missingness. The indications of the randomised controlled trials were multiple myeloma and metastatic hormone-sensitive prostate cancer. Overall survival was taken as the main endpoint. The effects of missing data and unmeasured confounding were assessed for the case studies by reporting estimated external comparator versus randomised controlled trial treatment effects. Based on the two case studies, simulations were performed broadening the settings by varying the underlying hazard ratio, the sample size, the sample size ratio between the experimental arm and the external comparator, the number of missing covariates and the percentage of missingness. Thereby, bias and other performance metrics could be quantified dependent on these factors.

RESULTS

For the multiple myeloma external comparator study, results were in line with the randomised controlled trial, despite missingness and potential unmeasured confounding, while for the metastatic hormone-sensitive prostate cancer case study missing data led to a low sample size, leading overall to inconclusive results. Furthermore, for the metastatic hormone-sensitive prostate cancer study, missing data in important eligibility criteria led to further limitations. Simulations were successfully applied to gain a quantitative understanding of the effects of missing data and unmeasured confounding.

CONCLUSIONS

This exploratory study confirmed external comparator strengths and limitations by quantifying the impact of missing data and unmeasured confounding using case studies and simulations. In particular, missing data in key eligibility criteria were seen to limit the ability to derive the external comparator target analysis population accurately, while simulations demonstrated the magnitude of bias to expect for various settings.

Real-world evidence to support regulatory submissions: A landscape review and assessment of use cases

Real-world evidence (RWE) has an increasing role in preapproval settings to support the approval of new medicines and indications. The main objectives of this study were to identify and characterize regulatory use cases that utilized RWE and other related observational approaches through targeted review of publications and regulatory review documents. After screening and inclusion/exclusion, the review characterized 85 regulatory applications with RWE. A total of 31 were in oncology and 54 were in non-oncology therapeutic areas. Most were for indications in adults only (N = 42, 49.4%), while 13 were in pediatrics only (15.3%), and 30 were in both (35.3%). In terms of regulatory context, 59 cases (69.4%) were for an original marketing application, 24 (28.2%) were for label expansion, and 2 (2.4%) were for label modification. Most also received special regulatory designations (e.g., orphan indication, breakthrough therapy, fast track, conditional, and accelerated approvals). There were 42 cases that utilized RWE to support single-arm trials. External data to support single-arm trials were utilized in various ways across use cases, including direct matching, benchmarking, natural history studies as well as literature or previous trials. A variety of data sources were utilized, including electronic health records, claims, registries, site-based charts. Endpoints in oncology use cases commonly included overall survival, progression-free survival. In 13 use cases, RWE was not considered supportive/definitive in regulatory decision-making due to design issues (e.g., small sample size, selection bias, missing data). Overall, RWE is utilized in regulatory approval processes for new indications/label expansion across various therapeutic areas with wide range of approaches. Multifaceted cross-sector efforts are needed to further improve the quality and utility of RWE in regulatory decision-making.

Examining external control arms in oncology: A scoping review of applications to date

OBJECTIVES

Randomized controlled trials (RCTs) are the gold standard for evaluating the comparative efficacy and safety of new cancer therapies. However, enrolling patients in control arms of clinical trials can be challenging for rare cancers, particularly in the context of precision oncology and targeted therapies. External Control Arms (ECAs) are a potential solution to address these challenges in clinical research design. We conducted a scoping review to explore the use of ECAs in oncology.

METHODS

We systematically searched four databases, namely MEDLINE, EMBASE, Web of Science, and Scopus. We screened titles, abstracts, and full texts for eligible articles focusing on patients undergoing therapy for cancer, employing ECAs, and reporting clinical outcomes.

RESULTS

Of the 629 articles screened, 23 were included in this review. The earliest included studies were published in 1996, while most studies were published in the past 5 years. 44% (10/23) of ECAs were employed in blood-related cancer studies. Geographically, 30% (7/23) of studies were conducted in the United States, 22% (5/23) in Japan, and 9% (2/23) in South Korea. The primary data sources used to construct the ECAs involved pooled data from previous trials (35%, 8/23), administrative health databases (17%, 4/23) and electronic medical records (17%, 4/23). While 52% (12/23) of the studies employed methods to align treatment and ECAs characteristics, 48% (11/23) lacked explicit strategies.

CONCLUSION

ECAs offer a valuable approach in oncology research, particularly when alternative designs are not feasible. However, careful methodological planning and detailed reporting are essential for meaningful and reliable results.

Comparative effectiveness of lazertinib in patients with EGFR T790M-positive non-small-cell lung cancer using a real-world external control

Lazertinib is a recently developed third-generation epidermal growth factor receptor (EGFR)-tyrosine kinase inhibitors used for patients with advanced EGFR T790M-positive non-small-cell lung cancer. We evaluated the effectiveness of lazertinib compared with osimertinib using an external control. We obtained individual patient data for the lazertinib arm from the LASER201 trial and the osimertinib arm from registry data at the Samsung Medical Center. In total, 75 and 110 patients were included in the lazertinib and osimertinib groups, respectively. After propensity score matching, each group had 60 patients and all baseline characteristics were balanced. The median follow-up duration was 22.0 and 29.6 months in the lazertinib and osimertinib group, respectively. The objective response rate (ORR) were 76.7% and 86.7% for lazertinib and osimertinib, respectively (p = 0.08). The median progression-free survival (PFS) was 12.3 months (95% confidence interval [CI] 9.5-19.1) and 14.4 months (95% CI 11.8-18.1) for the lazertinib and osimertinib group, respectively (hazard ratio [HR] 0.97; 95% CI 0.64-1.45, p = 0.86). The median overall survival with lazertinib was not reached and that with osimertinib was 29.8 months (HR 0.44; 95% CI 0.25-0.77, p = 0.005). Our study suggests that lazertinib has an ORR and PFS comparable to those of osimertinib and has the potential for superior survival benefits.

Lazertinib versus Platinum-Based Chemotherapy with Epidermal Growth Factor Receptor (EGFR)-Positive Non-Small-Cell Lung Cancer after Failing EGFR-Tyrosine Kinase Inhibitor: A Real-World External Comparator Study

BACKGROUND

Lazertinib is a third-generation tyrosine kinase inhibitor of epidermal growth factor receptor (EGFR-TKI) that selectively inhibit common EGFR mutation and T790M mutation in non-small-cell lung cancer (NSCLC) patients. No previous studies have compared lazertinib to platinum-based chemotherapy. We have compared lazertinib with platinum-based chemotherapy in EGFR-mutated NSCLC patients after previous EGFR-TKI therapy.

METHODS

We retrospectively compared 200 patients from LASER201, LASER301, and LASER-PMS studies to 334 patients who were treated with platinum-based chemotherapy after previous EGFR-TKI from the Samsung Medical Center. After propensity score matching (PSM), we selected 156 patients from each group. The primary outcome was progression-free survival (PFS), with overall survival (OS), objective response rate (ORR), and time to treatment discontinuation (TTD) as secondary outcomes.

RESULTS

The median follow-up of PFS was 15.61 months in the lazertinib group and 21.67 months in the external control group. The PFS was significantly longer in patients who were treated with lazertinib than those treated with platinum-based chemotherapy (10.97 months vs. 5.10 months; adjusted hazard ratio (HR) 0.40; 95% confidence interval (CI), 0.29-0.55; p < 0.01) after PSM. Lazertinib showed superior OS (32.23 months vs. 18.73 months; adjusted HR 0.45; 95% CI, 0.29-0.69; p < 0.001), ORR (64.1% vs. 47.4%), and TTD (11.66 months vs. 6.73 months; adjusted HR 0.54; 95% CI, 0.39-0.75; p < 0.001) compared to platinum-based chemotherapy.

CONCLUSION

Based on this retrospective, external control study, lazertinib has demonstrated significantly better efficacy compared with platinum-based chemotherapy. The external controls provide important context to evaluate efficacy in single-arm studies.

Augmenting external control arms using Bayesian borrowing: a case study in first-line non-small cell lung cancer

Aim: This study aimed to improve comparative effectiveness estimates and discuss challenges encountered through the application of Bayesian borrowing (BB) methods to augment an external control arm (ECA) constructed from real-world data (RWD) using historical clinical trial data in first-line non-small-cell lung cancer (NSCLC). Materials & methods: An ECA for a randomized controlled trial (RCT) in first-line NSCLC was constructed using ConcertAI Patient360™ to assess chemotherapy with or without cetuximab, in the bevacizumab-inappropriate subpopulation. Cardinality matching was used to match patient characteristics between the treatment arm (cetuximab + chemotherapy) and ECA. Overall survival (OS) was assessed as the primary outcome using Cox proportional hazards (PH). BB was conducted using a static power prior under a Weibull PH parameterization with borrowing weights from 0.0 to 1.0 and augmentation of the ECA from a historical control trial. Results: The constructed ECA yielded a higher overall survival (OS) hazard ratio (HR) (HR = 1.53; 95% CI: 1.21-1.93) than observed in the matched population of the RCT (HR = 0.91; 95% CI: 0.73-1.13). The OS HR decreased through the incorporation of BB (HR = 1.30; 95% CI: 1.08-1.54, borrowing weight = 1.0). BB was applied to augment the RCT control arm via a historical control which improved the precision of the observed HR estimate (1.03; 95% CI: 0.86-1.22, borrowing weight = 1.0), in comparison to the matched population of the RCT alone. Conclusion: In this study, the RWD ECA was unable to successfully replicate the OS estimates from the matched population of the selected RCT. The inability to replicate could be due to unmeasured confounding and variations in time-periods, follow-up and subsequent therapy. Despite these findings, we demonstrate how BB can improve precision of comparative effectiveness estimates, potentially aid as a bias assessment tool and mitigate challenges of traditional methods when appropriate external data sources are available.

Justifying the source of external comparators in single-arm oncology health technology submissions: a review of NICE and PBAC assessments

Background: The drive to expedite patient access for diseases with high unmet treatment needs has come with an increasing use of single-arm trials (SATs), especially in oncology. However, the lack of control arms in such trials creates challenges to assess and demonstrate comparative efficacy. External control (EC) arms can be used to bridge this gap, with various types of sources available to obtain relevant data. Objective: To examine the source of ECs in single-arm oncology health technology assessment (HTA) submissions to the National Institute for Health and Care Excellence (NICE) and the Pharmaceutical Benefits Advisory Committee (PBAC) and how this selection was justified by manufacturers and assessed by the respective HTA body. Methods: Single-arm oncology HTA submission reports published by NICE (England) and PBAC (Australia) from January 2011 to August 2021 were reviewed, with data qualitatively synthesized to identify themes. Results: Forty-eight oncology submissions using EC arms between 2011 and 2021 were identified, with most submissions encompassing blood and bone marrow cancers (52%). In HTA submissions to NICE and PBAC, the EC arm was typically constructed from a combination of data sources, with the company's justification in data source selection infrequently provided (PBAC [2 out of 19]; NICE [6 out of 29]), although this lack of justification was not heavily criticized by either HTA body. Conclusion: Although HTA bodies such as NICE and PBAC encourage that EC source justification should be provided in submissions, this review found that this is not typically implemented in practice. Guidance is needed to establish best practices as to how EC selection should be documented in HTA submissions.

Framework for the Use of External Controls to Evaluate Treatment Outcomes in Precision Oncology Trials

Advances in genomics have enabled anticancer therapies to be tailored to target specific genomic alterations. Single-arm trials (SATs), including those incorporated within umbrella, basket, and platform trials, are widely adopted when it is not feasible to conduct randomized controlled trials in rare biomarker-defined subpopulations. External controls (ECs), defined as control arm data derived outside the clinical trial, have gained renewed interest as a strategy to supplement evidence generated from SATs to allow comparative analysis. There are increasing examples demonstrating the application of EC in precision oncology trials. The prospective application of EC in conducting comparative studies is associated with distinct methodological challenges, the specific considerations for EC use in biomarker-defined subpopulations have not been adequately discussed, and a formal framework is yet to be established. In this review, we present a framework for conducting a prospective comparative analysis using EC. Key steps are (1) defining the purpose of using EC to address the study question, (2) determining if the external data are fit for purpose, (3) developing a transparent study protocol and a statistical analysis plan, and (iv) interpreting results and drawing conclusions on the basis of a prespecified hypothesis. We specify the considerations required for the biomarker-defined subpopulations, which include (1) specifying the comparator and biomarker status of the comparator group, (2) defining lines of treatment, (3) assessment of the biomarker testing panels used, and (4) assessment of cohort stratification in tumor-agnostic studies. We further discuss novel clinical trial designs and statistical techniques leveraging EC to propose future directions to advance evidence generation and facilitate drug development in precision oncology.

Enriching single-arm clinical trials with external controls: possibilities and pitfalls

For the past decade, it has become commonplace to provide rapid answers and early patient access to innovative treatments in the absence of randomized clinical trials (RCT), with benefits estimated from single-arm trials. This trend is important in oncology, notably when assessing new targeted therapies. Some of those uncontrolled trials further include an external/synthetic control group as an innovative way to provide an indirect comparison with a pertinent control group. We aimed to provide some guidelines as a comprehensive tool for (1) the critical appraisal of those comparisons or (2) for performing a single-arm trial. We used the example of ciltacabtagene autoleucel for the treatment of adult patients with relapsed or refractory multiple myeloma after 3 or more treatment lines as an illustrative example. We propose a 3-step guidance. The first step includes the definition of an estimand, which encompasses the treatment effect and the targeted population (whole population or restricted to single-arm trial or external controls), reflecting a clinical question. The second step relies on the adequate selection of external controls from previous RCTs or real-world data from patient cohorts, registries, or electronic patient files. The third step consists of choosing the statistical approach targeting the treatment effect defined above and depends on the available data (individual-level data or aggregated external data). The validity of the treatment effect derived from indirect comparisons heavily depends on careful methodological considerations included in the proposed 3-step procedure. Because the level of evidence of a well-conducted RCT cannot be guaranteed, the evaluation is more important than in standard settings.

Review article: Externally derived control arms-An opportunity for clinical trials in inflammatory bowel disease?

BACKGROUND

One of the greatest challenges in the current IBD clinical trial landscape is, perhaps, the recruitment and retention of eligible participants. Seamless testing of promising investigational compounds is paramount to address unmet needs, but this is hindered by a number of barriers, particularly patient concerns of placebo assignment.

AIMS

To review the use of novel trial designs leveraging externally derived data to synthetically create control groups or augment existing ones, and to summarise the regulatory position on the use of external controls for market authorisation.

METHODS

We conducted a PubMed literature search without restriction using search terms such as 'external controls' and 'historical controls' to identify relevant articles.

RESULTS

External controls are increasingly being used outside the context of cancer and rare diseases, including IBD, and increasingly recognised by regulatory bodies. Such designs, particularly in earlier phase trials, can inform key nodes in drug development and permit evaluating efficacy of interventions without combating the ethical and numerical enrolment challenges described. However, the lack of randomisation and blinding subjects them to significant bias. Groups require robust statistical and computational approaches to ensure patient-level data across groups are adequately balanced.

CONCLUSIONS

While this approach has several pitfalls, and is not robust enough to replace traditional randomised, placebo-controlled trials, it may offer a compromise to address key research questions at a more rapid pace, with fewer patients, and lower cost.

Can digital health researchers make a difference during the pandemic? Results of the single-arm, chatbot-led Elena+: Care for COVID-19 interventional study

Background

The current paper details findings from Elena+: Care for COVID-19, an app developed to tackle the collateral damage of lockdowns and social distancing, by offering pandemic lifestyle coaching across seven health areas: anxiety, loneliness, mental resources, sleep, diet and nutrition, physical activity, and COVID-19 information.

Methods

The Elena+ app functions as a single-arm interventional study, with participants recruited predominantly via social media. We used paired samples T-tests and within subjects ANOVA to examine changes in health outcome assessments and user experience evaluations over time. To investigate the mediating role of behavioral activation (i.e., users setting behavioral intentions and reporting actual behaviors) we use mixed-effect regression models. Free-text entries were analyzed qualitatively.

Results

Results show strong demand for publicly available lifestyle coaching during the pandemic, with total downloads (N = 7'135) and 55.8% of downloaders opening the app (n = 3,928) with 9.8% completing at least one subtopic (n = 698). Greatest areas of health vulnerability as assessed with screening measures were physical activity with 62% (n = 1,000) and anxiety with 46.5% (n = 760). The app was effective in the treatment of mental health; with a significant decrease in depression between first (14 days), second (28 days), and third (42 days) assessments: F2,38 = 7.01, p = 0.003, with a large effect size (η2G = 0.14), and anxiety between first and second assessments: t54 = 3.7, p = <0.001 with a medium effect size (Cohen d = 0.499). Those that followed the coaching program increased in net promoter score between the first and second assessment: t36 = 2.08, p = 0.045 with a small to medium effect size (Cohen d = 0.342). Mediation analyses showed that while increasing number of subtopics completed increased behavioral activation (i.e., match between behavioral intentions and self-reported actual behaviors), behavioral activation did not mediate the relationship to improvements in health outcome assessments.

Conclusions

Findings show that: (i) there is public demand for chatbot led digital coaching, (ii) such tools can be effective in delivering treatment success, and (iii) they are highly valued by their long-term user base. As the current intervention was developed at rapid speed to meet the emergency pandemic context, the future looks bright for other public health focused chatbot-led digital health interventions.

Considerations for pooling real-world data as a comparator cohort to a single arm trial: a simulation study on assessment of heterogeneity

BACKGROUND

Novel precision medicine therapeutics target increasingly granular, genomically-defined populations. Rare sub-groups make it challenging to study within a clinical trial or single real-world data (RWD) source; therefore, pooling from disparate sources of RWD may be required for feasibility. Heterogeneity assessment for pooled data is particularly complex when contrasting a pooled real-world comparator cohort (rwCC) with a single-arm clinical trial (SAT), because the individual comparisons are not independent as all compare a rwCC to the same SAT. Our objective was to develop a methodological framework for pooling RWD focused on the rwCC use case, and simulate novel approaches of heterogeneity assessment, especially for small datasets.

METHODS

We present a framework with the following steps: pre-specification, assessment of dataset eligibility, and outcome analyses (including assessment of outcome heterogeneity). We then simulated heterogeneity assessments for a binary response outcome in a SAT compared to two rwCCs, using standard methods for meta-analysis, and an Adjusted Cochran's Q test, and directly comparing the individual participant data (IPD) from the rwCCs.

RESULTS

We found identical power to detect a true difference for the adjusted Cochran's Q test and the IPD method, with both approaches superior to a standard Cochran's Q test. When assessing the impact of heterogeneity in the null scenario of no difference between the SAT and rwCCs, a lack of statistical power led to Type 1 error inflation. Similarly, in the alternative scenario of a true difference between SAT and rwCCs, we found substantial Type 2 error, with underpowered heterogeneity testing leading to underestimation of the treatment effect.

CONCLUSIONS

We developed a methodological framework for pooling RWD sources in the context of designing a rwCC for a SAT. When testing for heterogeneity during this process, the adjusted Cochran's Q test matches the statistical power of IPD heterogeneity testing. Limitations of quantitative heterogeneity testing in protecting against Type 1 or Type 2 error indicate these tests are best used descriptively, and after careful selection of datasets based on clinical/data considerations. We hope these findings will facilitate the rigorous pooling of RWD to unlock insights to benefit oncology patients.

Clinical development of new drugs for adults and children with cancer, 2010-2020

BACKGROUND

Many new molecular entities enter clinical development to evaluate potential therapeutic benefits for oncology patients. We characterized adult and pediatric development of the set of new molecular entities that started clinical testing in 2010-2015 worldwide.

METHODS

We extracted data from AdisInsight, an extensive database of global pharmaceutical development, and the FDA.gov website. We followed the cohort of new molecular entities initiating first-in-human phase I clinical trials in 2010-2015 to the end of 2020. For each new molecular entity, we determined whether it was granted US Food and Drug Administration (FDA) approval, studied in a trial open to pediatric enrollment, or stalled during development. We characterized the cumulative incidence of these endpoints using statistical methods for censored data.

RESULTS

The 572 new molecular entities starting first-in-human studies in 2010-2015 were studied in 6142 trials by the end of 2020. Most new molecular entities were small molecules (n = 316, 55.2%), antibodies (n = 148, 25.9%), or antibody-drug conjugates (n = 44, 7.7%). After a mean follow-up of 8.0 years, 173 new molecular entities did not advance beyond first-in-human trials, and 39 were approved by the FDA. New molecular entities had a 10.4% estimated probability (95% confidence interval = 6.6% to 14.1%) of being approved by the FDA within 10 years of first-in-human trials. After a median of 4.6 years since start of first-in-human trials, 67 (11.7%) new molecular entities were tested in trials open to pediatric patients, and 5 (0.9%) were approved for pediatric indications.

CONCLUSIONS

More efficient clinical development strategies are needed to evaluate new cancer therapies, especially for children, and incorporate approaches to ensure knowledge gain from investigational products that stall in development.

Improving the efficiency of clinical trials in multiple sclerosis

BACKGROUND

Phase 3 clinical trials for disease-modifying therapies in relapsing-remitting multiple sclerosis (RRMS) have utilized a limited number of conventional designs with a high degree of success. However, these designs limit the types of questions that can be addressed, and the time and cost required. Moreover, trials involving people with progressive multiple sclerosis (MS) have been less successful.

OBJECTIVE

The objective of this paper is to discuss complex innovative trial designs, intermediate and composite outcomes and to improve the efficiency of trial design in MS and broaden questions that can be addressed, particularly as applied to progressive MS.

METHODS

We held an international workshop with experts in clinical trial design.

RESULTS

Recommendations include increasing the use of complex innovative designs, developing biomarkers to enrich progressive MS trial populations, prioritize intermediate outcomes for further development that target therapeutic mechanisms of action other than peripherally mediated inflammation, investigate acceptability to people with MS of data linkage for studying long-term outcomes of clinical trials, use Bayesian designs to potentially reduce sample sizes required for pediatric trials, and provide sustained funding for platform trials and registries that can support pragmatic trials.

CONCLUSION

Novel trial designs and further development of intermediate outcomes may improve clinical trial efficiency in MS and address novel therapeutic questions.

Larotrectinib versus historical standard of care in patients with infantile fibrosarcoma: protocol of EPI-VITRAKVI

The EPI VITRAKVI study is a retrospective study designed to place the results of the single-arm Phase I/II larotrectinib SCOUT trial into context by comparison with external historical controls. Its primary objective is to compare the time to medical treatment failure between larotrectinib and the historical standard of care (chemotherapy) in patients with infantile fibrosarcoma. External historical cohorts have been selected by using objective criteria. The Inverse Probability of Treatment Weighting method will be used to adjust for potential confounding. The current publication illustrates how an external control arm study can complement data from a single-arm trial and addresses uncertainties encountered in the assessment of therapies targeting rare abnormalities where randomized controlled trials are considered not feasible. Clinical Trial Registration: NCT05236257 (ClinicalTrials.gov).

Development of an External Control Arm Using Electronic Health Record-Based Real-World Data to Evaluate the Efficacy of COVID-19 Treatment

To protect people from severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) infection, tremendous research efforts have been made toward coronavirus disease 19 (COVID-19) treatment development. Externally controlled trials (ECTs) may help reduce their development time. To evaluate whether ECT using real-world data (RWD) of patients with COVID-19 is feasible enough to be used for regulatory decision making, we built an external control arm (ECA) based on RWD as a control arm of a previously conducted randomized controlled trial (RCT), and compared it to the control arm of the RCT. The electronic health record (EHR)-based COVID-19 cohort dataset was used as RWD, and three Adaptive COVID-19 Treatment Trial (ACTT) datasets were used as RCTs. Among the RWD datasets, eligible patients were evaluated as a pool of external control subjects of the ACTT-1, ACTT-2, and ACTT-3 trials, respectively. The ECAs were built using propensity score matching, and the balance of age, sex, and baseline clinical status ordinal scale as covariates between the treatment arms of Asian patients in each ACTT and the pools of external control subjects was assessed before and after 1:1 matching. There was no statistically significant difference in time to recovery between ECAs and the control arms of each ACTT. Among the covariates, the baseline status ordinal score had the greatest influence on the building of ECA. This study demonstrates that ECA based on EHR data of COVID-19 patients could sufficiently replace the control arm of an RCT, and it is expected to help develop new treatments faster in emergency situations, such as the COVID-19 pandemic.

A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies

OBJECTIVES

The development of accelerated approval programs for high morbidity and unmet need conditions has driven the use of single-arm studies in drug development. Regulatory and health technology assessment (HTA) agencies are recognizing that high-quality external control arms (ECAs), built using real-world data, can reduce uncertainties arising from single-arm studies. This review compared 7 case studies of regulatory and HTA agencies' evaluations of oncology ECAs.

METHODS

Food and Drug Administration multidisciplinary reviews for oncology submissions from 2014 to 2021 were screened to identify 7 cases (2 blinatumomab indications, avelumab, and erdafitinib, entrectinib, trastuzumab deruxtecan, and idecabtagene vicleucel) with ECAs to support efficacy claims. Regulatory (Food and Drug Administration, European Medicines Agency, Health Canada) and HTA (pan-Canadian Oncology Drug Review, National Institute for Health and Care Excellence, Federal Joint Committee, Haute Autorité de Santé, and Pharmaceutical Benefits Advisory Committee) submissions for these cases were reviewed. The decision makers' ECA critiques and the level of influence on the decision were analyzed and categorized.

RESULTS

Across case studies, selection bias and confounding were the most common ECA critiques. Nevertheless, agreement in critiques between and among regulators and HTA bodies was low. ECA influence on agencies' decisions also varied.

CONCLUSIONS

Evaluating the same ECA evidence, agencies focused on methodologic issues (ie, selection bias and confounding), but were often not aligned on their critiques. Further research is needed to fully characterize how agencies evaluate ECAs. This study is a first step in critically evaluating agencies' critiques of ECAs and highlights the need for future guidance development around ECA design and generation.

External Comparator Groups Derived from Real-world Data Used in Support of Regulatory Decision Making: Use Cases and Challenges

Real-world data (RWD) from electronic health records (EHRs) and administrative claims databases are used increasingly to generate real-world evidence (RWE). RWE is used to support clinical evidence packages for medicines that inform decision-makers. In this review of current issues in the use of RWD-derived external comparator groups to support regulatory filings, we assess a series of topics that generally apply across many disease indications. However, most of the examples and illustrations focus on the oncology clinical research setting. The topics include an overview of current uses of RWD in drug development, a discussion of regulatory filings using RWD-derived external comparators, a brief overview of guidance documents and white papers pertaining to external comparators, a summary of some limitations and methodological issues in the use of external comparator groups and finally, a look at the future of this area and recommendations.

Real-World Evidence Prediction of a Phase IV Oncology Trial: Comparative Degarelix vs Leuprolide Safety

BACKGROUND

Medical and regulatory communities are increasingly interested in the utility of real-world evidence (RWE) for answering questions pertaining to drug safety and effectiveness but concerns about validity remain. A principled approach to conducting RWE studies may alleviate concerns and increase confidence in findings. This study sought to predict the findings from the PRONOUNCE trial using a principled approach to generating RWE.

METHODS

This propensity-score (PS) matched observational cohort study utilized 3 claims databases to compare the occurrence of major adverse cardiovascular events (MACE) among initiators of degarelix vs. leuprolide. Patients were included if they had history of prostate cancer and atherosclerotic cardiovascular disease. Subjects were excluded if they didn't have continuous database enrollment in the year prior to treatment initiation, were exposed to androgen deprivation therapy or experienced an acute cardiovascular event within 30 days prior to treatment initiation, or had a history or risk factors of QT prolongation.

RESULTS

There were 12,448 leuprolide and 1,969 degarelix study-eligible patients before matching, with 1,887 in each arm after PS-matching. The results for MACE comparing degarelix to leuprolide in the observational analysis (hazard ratio= 1.35; 95% confidence interval = 0.94-1.93) was consistent with the subsequently released PRONOUNCE result (hazard ratio = 1.28; 95% confidence interval = 0.59-2.79).

CONCLUSIONS

This study successfully predicted the result of a comparative cardiovascular safety trial in the oncology setting. Although the findings are encouraging, limitations of measuring cancer stage and tumor progression are representative of challenges in attempting to generalize whether claims-based RWE can be used as actionable evidence.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Treatment effectiveness in a rare oncology indication: Lessons from an external control cohort study

Real-world data (RWD) reflecting patient treatment in routine clinical practice can be used to develop external control groups for single-arm trials. External controls can provide valuable benchmark results on potential comparator drug effectiveness, particularly in rare indications when randomized controlled trials are either infeasible or unethical. This paper describes lessons learned from a descriptive real-world external control cohort study conducted to provide benchmark data for a single-arm clinical trial in a rare oncology biomarker driven disease. Conducting external control cohort studies to evaluate treatment effectiveness in rare indications likely will present with data and analysis challenges as seen in the example study. However, there are mitigating measures that can be applied in the study design, identification of RWD sources, and data analysis. The lessons learned and reported here with a proposal of an external control study framework can provide guidance for future research in this area, and may be applicable as well in other rare indications. Taking these learnings into consideration, the use of real-world external controls to contextualize treatment effectiveness in rare indications is a valuable approach and warrants further application in the future.

Increasing Inclusiveness of Patient-Centric Clinical Evidence Generation in Oncology: Real-World Data and Clinical Trials

Rapid advancements in cancer discovery, diagnosis, and treatment options available to patients with cancer have highlighted the need for enhancements in clinical trial design. The drug development process is costly, with more than 80% of trials failing to reach recruitment targets. Historical approaches to trial design are increasingly burdensome and lack real-world application in the intent-to-treat patient population. Equitable access to clinical trials combined with increased availability of real-world data are creating new opportunities for inclusiveness, improved outcomes, and evidence-based advances in therapies that will generate more generalizable data to better inform clinical decision-making. Clinical trials need to be inclusive if lifesaving data are not to be missed and investigational therapies are to be more accessible to a broader patient base. Real-world data can facilitate the conduct of studies that are identifying and understanding where disparities exist and developing new interventions to improve patient care. The clinical trial design process should be a multistakeholder and consensus- and evidence-driven process in which stakeholders are working together across the health care industry to close the care gap and ensure elimination of barriers that prevent equal access to specialized cancer care and advanced therapies available in clinical trials. The patient voice is essential throughout the trial process; however, it is often excluded from the design process. Integrating real-world data as well as ensuring patient involvement in early trial design during drug development can enhance enrollment and retention, leading to greater diversity.

Real-world data in drug development strategies for orphan drugs: tafasitamab in B cell lymphoma, a case study for approval based on a single-arm combination trial

Tafasitamab (TAF) plus lenalidomide (LEN) is a novel treatment option for patients with relapsed/refractory diffuse large B cell lymphoma (rrDLBCL) who are not eligible for autologous stem cell transplantation. The initial US/EU approvals for TAF represent precedents because this is the first time that approval of a novel combination therapy was granted based on a pivotal single-arm trial (SAT). Matching real world-data (RWD) helped to disentangle the contribution of individual agents. In this review, we present the TAF development strategy, the prospective incorporation of RWD within the clinical development plan, the corresponding regulatory hurdles of this strategy, and the prior regulatory actions for other cancer drugs that previously incorporated RWD and propensity score matching in EU and US regulatory submissions. We also outline how RWD could further advance and impact orphan drug development.

Leveraging external data in the design and analysis of clinical trials in neuro-oncology

Integration of external control data, with patient-level information, in clinical trials has the potential to accelerate the development of new treatments in neuro-oncology by contextualising single-arm studies and improving decision making (eg, early stopping decisions). Based on a series of presentations at the 2020 Clinical Trials Think Tank hosted by the Society of Neuro-Oncology, we provide an overview on the use of external control data representative of the standard of care in the design and analysis of clinical trials. High-quality patient-level records, rigorous methods, and validation analyses are necessary to effectively leverage external data. We review study designs, statistical methods, risks, and potential distortions in using external data from completed trials and real-world data, as well as data sources, data sharing models, ongoing work, and applications in glioblastoma.

A Framework for Extension Studies Using Real-World Data to Examine Long-Term Safety and Effectiveness

Understanding the long-term benefits and risks of treatments, devices, and vaccines is critically important for individual- and population-level healthcare decision-making. Extension studies, or ‘roll-over studies,’ are studies that allow for patients participating in a parent clinical trial to ‘roll-over’ into a subsequent related study to continue to observe and measure long-term safety, tolerability, and/or effectiveness. These designs are not new and are often used as an approach to satisfy regulatory post-approval safety requirements. However, designs using traditional clinical trial infrastructure can be expensive and burdensome to conduct, particularly, when following patients for many years post trial completion. Given the increasing availability and access of real-world data (RWD) sources, direct-to-patient technologies, and novel real-world study designs, there are more cost-efficient approaches to conducting extension studies while assessing important long-term outcomes. Here, we describe various fit-for-purpose design options for extension studies, discuss related methodological considerations, and provide scientific and operational guidance on practices when planning to conduct an extension study using RWD. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

Uncontrolled Extensions of Clinical Trials and the Use of External Controls-Scoping Opportunities and Methods

Increased interest in real-world evidence (RWE) for clinical and regulatory decision making and the need to evaluate long-term benefits and risks of pharmaceutical products raise the importance of understanding the use of external controls (ECs) for uncontrolled extensions of randomized controlled trials (RCTs). We searched clinicaltrials.gov from 2009 to 2019 for uncontrolled extensions and assessed the use of ECs in the trial protocol registry and PubMed. We present characteristics of identified uncontrolled extensions, their adoption of ECs, and a qualitative appraisal of published uncontrolled extensions with ECs according to good pharmacoepidemiologic practice. The number of uncontrolled extensions increased slightly across the study period, resulting in a total of 1,115 studies. Most originated from phase III RCTs (62.2%) and specified safety outcomes (61.9% among those with specified outcomes). Most uncontrolled extensions incorporated no control group with only 7 out of 1,115 (0.6%) employing ECs. For those studies with ECs, all involved treatments for rare conditions and assessment of effectiveness. Attempts to balance comparison groups varied from none mentioned to propensity score matching. We noted consistent deficiencies in outcome ascertainment methods and approaches to address attrition bias. The contrast of the large and growing number of uncontrolled extensions with the small number of studies that utilized ECs showed clear opportunities for enhancement in design, measurement, and analysis of uncontrolled extensions to allow causal inferences on long-term treatment effects. As extensions continue to expand within RWE regulatory frameworks, development of guidelines for use of EC with uncontrolled extensions is needed.

Real-world evidence to support regulatory decision making: New or expanded medical product indications

There is increasing interest in utilizing real-world data (RWD) to produce real-world evidence (RWE) on the benefits and risks of medical products that could support regulatory approval decisions. The field of pharmacoepidemiology has a long history of focusing on data and evidence that would now be termed "real-world," including evidence from healthcare claims, registries, and electronic health records. However, several emerging trends over the past decade are converging to support the use of these and other RWD sources for approval decisions, and there are several recent examples and ongoing research that demonstrate how RWE may be used to support regulatory approval of new or expanded indications. The goal of this article is to review the current landscape and future directions of the use of RWE in this context. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE).

Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials

Purpose

Clinical trials compare outcomes among patients receiving study treatment with comparators drawn from the same source. These internal controls are missing in single arm trials and from long‐term extensions (LTE) of trials including only the treatment arm. An external control group derived from a different setting is then required to assess safety or effectiveness.

Methods

We present examples of external control groups that demonstrate some of the issues that arise and make recommendations to address them through careful assessment of the data source fitness for use, design, and analysis steps.

Results

Inclusion and exclusion criteria and context that produce a trial population may result in trial patients with different clinical characteristics than are present in an external comparison group. If these differences affect the risk of outcomes, then a comparison of outcome occurrence will be confounded. Further, patients who continue into LTE may differ from those initially entering the trial due to treatment effects. Application of appropriate methods is needed to make valid inferences when such treatment or selection effects are present.

Outcome measures in a trial may be ascertained and defined differently from what can be obtained in an external comparison group. Differences in sensitivity and specificity for identification or measurement of study outcomes leads to information bias that can also invalidate inferences.

Conclusion

This review concentrates on threats to the valid use of external control groups both in the scenarios of single arm trials and LTE of randomized controlled trials, along with methodological approaches to mitigate them.