Implementing Historical Controls in Oncology Trials

A bootstrapping method to optimize go/no-go decisions from single-arm, signal-finding studies in oncology

Phase Ib trials are common in oncology development but often are not powered for statistical significance. Go/no-go decisions are largely driven by observed trends in response data. We applied a bootstrapping method to systematically compare tumor dynamic end points to historical control data to identify drugs with clinically meaningful efficacy. A proprietary mathematical model calibrated to phase Ib anti-PD-1 therapy trial data (KEYNOTE-001) was used to simulate thousands of phase Ib trials (n = 30) with a combination of anti-PD-1 therapy and four novel agents with varying efficacy. A redacted bootstrapping method compared these results to a simulated phase III control arm (N = 511) while adjusting for differences in trial duration and cohort size to determine the probability that the novel agent provides clinically meaningful efficacy. Receiver operating characteristic (ROC) analysis showed strong ability to separate drugs with modest (area under ROC [AUROC] = 83%), moderate (AUROC = 96%), and considerable efficacy (AUROC = 99%) from placebo in early-phase trials (n = 30). The method was shown to effectively move drugs with a range of efficacy through an in silico pipeline with an overall success rate of 93% and false-positive rate of 7.5% from phase I to phase III. This model allows for effective comparisons of tumor dynamics from early clinical trials with more mature historical control data and provides a framework to predict drug efficacy in early-phase trials. We suggest this method should be employed to improve decision making in early oncology trials.

Analysis of sodium phenylbutyrate and taurursodiol survival effect in ALS using external controls

OBJECTIVE

Sodium phenylbutyrate and taurursodiol (PB and TURSO) was evaluated in amyotrophic lateral sclerosis (ALS) in the CENTAUR trial encompassing randomized placebo-controlled and open-label extension phases. On intent-to-treat (ITT) survival analysis, median overall survival (OS) was 4.8 months longer and risk of death 36% lower in those originally randomized to an initial 6-month double-blind period of PB and TURSO versus placebo. To estimate PB and TURSO treatment effect without placebo-to-active crossover, we performed a post hoc survival analysis comparing PB and TURSO-randomized participants from CENTAUR and a propensity score-matched, PB and TURSO-naïve external control cohort from the Pooled Resource Open-Access ALS Clinical Trials (PRO-ACT) database.

METHODS

Clinical trial control participants from the PRO-ACT database who met prespecified eligibility criteria were propensity score matched 1:1 with PB and TURSO-randomized CENTAUR participants using prognostically significant covariates in ALS.

RESULTS

Baseline characteristics including propensity score-matched covariates were generally well balanced between CENTAUR PB and TURSO (n = 89) and PRO-ACT external control (n = 85) groups. Estimated median (IQR) OS was 23.54 (14.56-39.32) months in the CENTAUR PB and TURSO group and 13.15 (9.83-19.20) months in the PRO-ACT external control group; hazard of death was 52% lower in the former group (hazard ratio, 0.48; 95% CI, 0.31-0.72; p = 0.00048).

INTERPRETATION

This analysis suggests potentially greater survival benefit with PB and TURSO in ALS without placebo-to-active crossover than seen on ITT analysis in CENTAUR. Analyses using well-matched external controls may provide additional context for evaluating survival effects in future ALS trials.

Enriching single-arm clinical trials with external controls: possibilities and pitfalls

For the past decade, it has become commonplace to provide rapid answers and early patient access to innovative treatments in the absence of randomized clinical trials (RCT), with benefits estimated from single-arm trials. This trend is important in oncology, notably when assessing new targeted therapies. Some of those uncontrolled trials further include an external/synthetic control group as an innovative way to provide an indirect comparison with a pertinent control group. We aimed to provide some guidelines as a comprehensive tool for (1) the critical appraisal of those comparisons or (2) for performing a single-arm trial. We used the example of ciltacabtagene autoleucel for the treatment of adult patients with relapsed or refractory multiple myeloma after 3 or more treatment lines as an illustrative example. We propose a 3-step guidance. The first step includes the definition of an estimand, which encompasses the treatment effect and the targeted population (whole population or restricted to single-arm trial or external controls), reflecting a clinical question. The second step relies on the adequate selection of external controls from previous RCTs or real-world data from patient cohorts, registries, or electronic patient files. The third step consists of choosing the statistical approach targeting the treatment effect defined above and depends on the available data (individual-level data or aggregated external data). The validity of the treatment effect derived from indirect comparisons heavily depends on careful methodological considerations included in the proposed 3-step procedure. Because the level of evidence of a well-conducted RCT cannot be guaranteed, the evaluation is more important than in standard settings.

Increasing FDA-accelerated approval of single-arm trials in oncology (1992 to 2020)

OBJECTIVES

We aimed to map the characteristics of single-arm trials (SAT), report the Food and Drug Administration (FDA) transparency in presenting historical control, and to assess the confirmatory randomized controlled trials (RCTs).

STUDY DESIGN AND SETTING

This metaresearch included a review of all oncology indication approved using SAT by FDA-AA (FDA-Accelerated Approval) from 1992 to 2020. Two independent reviewers identified SAT, extracted data from FDA full medical reviews for historical controls reported and MEDLINE for searching for confirmatory RCT published.

RESULTS

Of 254 FDA-AA approvals, 119 (47%) were approved for oncologic indications using SAT. Fifty-four drugs for 72 oncology indications were for leukemia, lymphoma, lung cancer, urothelial cancer, multiple myeloma, and thyroid cancer. Overall, 37 (52%) treatments were converted into regular approval. Of these, 17 (46%) were based on confirmatory RCTs using overall survival (OS) as an outcome. Five indications were withdrawn from the market. Most trials outcomes were blindly assessed by independent research committees. Median trial sample size was 105 patients (min:8 to max:532). The FDA did not fully specify historical control selection in 75% of cases.

CONCLUSION

The granting of FDA-AAs based on SAT in oncology is increasing with more target drugs approved over time. Transparency in historical control reporting is necessary.

On the use of non-concurrent controls in platform trials: a scoping review

BACKGROUND

Platform trials gained popularity during the last few years as they increase flexibility compared to multi-arm trials by allowing new experimental arms entering when the trial already started. Using a shared control group in platform trials increases the trial efficiency compared to separate trials. Because of the later entry of some of the experimental treatment arms, the shared control group includes concurrent and non-concurrent control data. For a given experimental arm, non-concurrent controls refer to patients allocated to the control arm before the arm enters the trial, while concurrent controls refer to control patients that are randomised concurrently to the experimental arm. Using non-concurrent controls can result in bias in the estimate in case of time trends if the appropriate methodology is not used and the assumptions are not met.

METHODS

We conducted two reviews on the use of non-concurrent controls in platform trials: one on statistical methodology and one on regulatory guidance. We broadened our searches to the use of external and historical control data. We conducted our review on the statistical methodology in 43 articles identified through a systematic search in PubMed and performed a review on regulatory guidance on the use of non-concurrent controls in 37 guidelines published on the EMA and FDA websites.

RESULTS

Only 7/43 of the methodological articles and 4/37 guidelines focused on platform trials. With respect to the statistical methodology, in 28/43 articles, a Bayesian approach was used to incorporate external/non-concurrent controls while 7/43 used a frequentist approach and 8/43 considered both. The majority of the articles considered a method that downweights the non-concurrent control in favour of concurrent control data (34/43), using for instance meta-analytic or propensity score approaches, and 11/43 considered a modelling-based approach, using regression models to incorporate non-concurrent control data. In regulatory guidelines, the use of non-concurrent control data was considered critical but was deemed acceptable for rare diseases in 12/37 guidelines or was accepted in specific indications (12/37). Non-comparability (30/37) and bias (16/37) were raised most often as the general concerns with non-concurrent controls. Indication specific guidelines were found to be most instructive.

CONCLUSIONS

Statistical methods for incorporating non-concurrent controls are available in the literature, either by means of methods originally proposed for the incorporation of external controls or non-concurrent controls in platform trials. Methods mainly differ with respect to how the concurrent and non-concurrent data are combined and temporary changes handled. Regulatory guidance for non-concurrent controls in platform trials are currently still limited.

A Propensity-Matched Retrospective Comparative Study with Historical Control to Determine the Real-World Effectiveness of Durvalumab after Concurrent Chemoradiotherapy in Unresectable Stage III Non-Small Cell Lung Cancer

This study aimed to add real-world evidence to the literature regarding the effectiveness and safety of durvalumab consolidation (DC) after concurrent chemoradiotherapy (CCRT) in the treatment of unresectable stage III non-small cell lung cancer (NSCLC). Using a hospital-based NSCLC patient registry and propensity score matching in a 2:1 ratio, we conducted a retrospective cohort study of patients with unresectable stage III NSCLC who completed CCRT with and without DC. The co-primary endpoints were 2-year progression-free survival and overall survival. For the safety evaluation, we evaluated the risk of any adverse events requiring systemic antibiotics or steroids. Of 386 eligible patients, 222 patients-including 74 in the DC group-were included in the analysis after propensity score matching. Compared with CCRT alone, CCRT with DC was associated with increased progression-free survival (median: 13.3 vs. 7.6 months, hazard ratio[HR]: 0.63, 95% confidence interval[CI]: 0.42-0.96) and overall survival (HR: 0.47, 95% CI: 0.27-0.82) without an increased risk of adverse events requiring systemic antibiotics or steroids. While there were differences in patient characteristics between the present real-world study and the pivotal randomized controlled trial, we demonstrated significant survival benefits and tolerable safety with DC after the completion of CCRT.

Utilization of real-world data in assessing treatment effectiveness for diffuse large B-cell lymphoma

Direct comparisons of the effectiveness of the numerous novel therapies in the diffuse large B-cell lymphoma (DLBCL) treatment landscape in a range of head-to-head randomized phase 3 trials would be time-consuming and costly. Comparative effectiveness studies using real-world data (RWD) represent a complementary approach. Recently, several studies of relapsed/refractory (R/R) DLBCL have used RWD to create observational cohorts to compare patient outcomes with cohorts derived from single-arm phase 2 trials. Using propensity score methods to balance clinically and prognostically relevant baseline covariates, closely matched patient-level cohorts can be generated. By incorporating appropriate measures to assess covariate balance and address potential bias in comparative effectiveness study designs, robust comparative analyses can be performed. Results from such studies have been used to supplement regulatory approval of therapies assessed in single-arm trials. While RWD studies have a greater susceptibility to bias compared to randomized controlled trials, well-designed and appropriately analyzed studies can provide complementary real-world evidence on treatment effectiveness.

Estimands and Complex Innovative Designs

Since the release of the ICH E9(R1) (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use Addendum on Estimands and Sensitivity Analysis in Clinical Trials to the Guideline on Statistical Principles for Clinical Trials) document in 2019, the estimand framework has become a fundamental part of clinical trial protocols. In parallel, complex innovative designs have gained increased popularity in drug development, in particular in early development phases or in difficult experimental situations. While the estimand framework is relevant to any study in which a treatment effect is estimated, experience is lacking as regards its application to these designs. In a basket trial for example, should a different estimand be specified for each subpopulation of interest, defined, for example, by cancer site? Or can a single estimand focusing on the general population (defined, for example, by the positivity to a certain biomarker) be used? In the case of platform trials, should a different estimand be proposed for each drug investigated? In this work we discuss possible ways of implementing the estimand framework for different types of complex innovative designs. We consider trials that allow adding or selecting experimental treatment arms, modifying the control arm or the standard of care, and selecting or pooling populations. We also address the potentially data-driven, adaptive selection of estimands in an ongoing trial and disentangle certain statistical issues that pertain to estimation rather than to estimands, such as the borrowing of nonconcurrent information. We hope this discussion will facilitate the implementation of the estimand framework and its description in the study protocol when the objectives of the trial require complex innovative designs.

Implications of Oncology Trial Design and Uncertainties in Efficacy-Safety Data on Health Technology Assessments

Background: Advances in cancer medicines have resulted in tangible health impacts, but the magnitude of benefits of approved cancer medicines could vary greatly. Health Technology Assessment (HTA) is a multidisciplinary process used to inform resource allocation through a systematic value assessment of health technology. This paper reviews the challenges in conducting HTA for cancer medicines arising from oncology trial designs and uncertainties of safety-efficacy data. Methods: Multiple databases (PubMed, Scopus and Google Scholar) and grey literature (public health agencies and governmental reports) were searched to inform this policy narrative review. Results: A lack of robust efficacy-safety data from clinical trials and other relevant sources of evidence has made HTA for cancer medicines challenging. The approval of cancer medicines through expedited pathways has increased in recent years, in which surrogate endpoints or biomarkers for patient selection have been widely used. Using these surrogate endpoints has created uncertainties in translating surrogate measures into patient-centric clinically (survival and quality of life) and economically (cost-effectiveness and budget impact) meaningful outcomes, with potential effects on diverting scarce health resources to low-value or detrimental interventions. Potential solutions include policy harmonization between regulatory and HTA authorities, commitment to generating robust post-marketing efficacy-safety data, managing uncertainties through risk-sharing agreements, and using value frameworks. Conclusion: A lack of robust efficacy-safety data is a central problem for conducting HTA of cancer medicines, potentially resulting in misinformed resource allocation.

A Review of Causal Inference for External Comparator Arm Studies

Randomized controlled trials (RCTs) are the gold standard design to establish the efficacy of new drugs and to support regulatory decision making. However, a marked increase in the submission of single-arm trials (SATs) has been observed in recent years, especially in the field of oncology due to the trend towards precision medicine contributing to the rise of new therapeutic interventions for rare diseases. SATs lack results for control patients, and information from external sources can be compiled to provide context for better interpretability of study results. External comparator arm (ECA) studies are defined as a clinical trial (most commonly a SAT) and an ECA of a comparable cohort of patients-commonly derived from real-world settings including registries, natural history studies, or medical records of routine care. This publication aims to provide a methodological overview, to sketch emergent best practice recommendations and to identify future methodological research topics. Specifically, existing scientific and regulatory guidance for ECA studies is reviewed and appropriate causal inference methods are discussed. Further topics include sample size considerations, use of estimands, handling of different data sources regarding differential baseline covariate definitions, differential endpoint measurements and timings. In addition, unique features of ECA studies are highlighted, specifically the opportunity to address bias caused by unmeasured ECA covariates, which are available in the SAT.

Safety and efficacy of elsubrutinib or upadacitinib alone or in combination (ABBV-599) in patients with rheumatoid arthritis and inadequate response or intolerance to biological therapies: a multicentre, double-blind, randomised, controlled, phase 2 trial

BACKGROUND

ABBV-599 is a novel fixed-dose combination of the Bruton's tyrosine kinase (BTK) inhibitor elsubrutinib and the Janus kinase (JAK) inhibitor upadacitinib under investigation for the treatment of autoimmune diseases. We aimed to determine whether ABBV-599 could increase the treatment response for patients with active rheumatoid arthritis compared with inhibiting either pathway alone, while maintaining an acceptable safety profile.

METHODS

We conducted a multicentre, double-blind, parallel-group, dose-exploratory, randomised, controlled, phase 2 trial at 75 community sites in eight countries in Europe and North America. We enrolled patients who were 18 years or older with rheumatoid arthritis and inadequate response or intolerance to biological disease-modifying antirheumatic drugs. Eligible patients were randomly assigned (3:2:2:2:2:1) via interactive response technology to receive daily, orally administered ABBV-599 (ie, upadacitinib 15 mg plus elsubrutinib 60 mg), elsubrutinib 60 mg, elsubrutinib 20 mg, elsubrutinib 5 mg, upadacitinib 15 mg, or placebo. Randomisation was stratified by the number of previous biological disease-modifying antirheumatic drugs. The investigator, study site personnel, and patients were masked throughout the study. The primary endpoint was change from baseline in disease activity score of 28 joints with C-reactive protein (DAS28-CRP) at week 12 for all patients who received a study drug. Pharmacokinetics and safety were also assessed. This study is registered with ClinicalTrials.gov, number NCT03682705.

FINDINGS

Between Oct 8, 2018, and March 26, 2020, 242 patients were randomly assigned to receive ABBV-599 (n=62), elsubrutinib 60 mg (n=41), elsubrutinib 20 mg (n=39), elsubrutinib 5 mg (n=41), upadacitinib 15 mg (n=40), or placebo (n=19). Of the 242 patients, 204 (84%) were female, 38 (16%) were male, and 220 (91%) were White; the mean age at baseline was 58·0 years (SD 11·3). Compared with placebo, the least squares mean changes from baseline in DAS28-CRP were -1·44 (90% CI -2·03 to -0·85; p<0·0001) for ABBV-599, -0·40 (-1·03 to 0·23; p=0·29) for elsubrutinib 60 mg, -0·20 (-0·85 to 0·44; p=0·61) for elsubrutinib 20 mg, -0·21 (-0·84 to 0·41; p=0·57) for elsubrutinib 5 mg, and -1·75 (-2·38 to -1·13; p<0·0001) for upadacitinib. No significant improvements in efficacy measures for elsubrutinib alone (any dose) versus placebo were detected, despite adequate plasma exposure and target engagement. Treatment-emergent adverse events were observed in 113 (47%) of 242 patients, with similar proportions for all groups.

INTERPRETATION

Significant improvements in disease activity metrics of rheumatoid arthritis with ABBV-599 were driven by the JAK inhibitor upadacitinib with no discernible effect by the BTK inhibitor elsubrutinib.

FUNDING

AbbVie.

Durvalumab and tremelimumab with definitive chemoradiotherapy for locally advanced esophageal squamous cell carcinoma

BACKGROUND

The current standard treatment for patients with inoperable, locally advanced esophageal squamous cell carcinoma (ESCC) is definitive concurrent chemoradiotherapy (CCRT).

METHODS

Patients with locally advanced ESCC received 2 cycles of 5-fluorouracil, cisplatin, durvalumab, and tremelimumab every 3 weeks with concurrent radiation therapy (60.2 or 64.5 grays). After completing CCRT plus immunotherapy, patients received 2 cycles of consolidative durvalumab and tremelimumab followed by durvalumab monotherapy every 4 weeks for 2 years after enrollment. Their survival outcomes were compared with those from a propensity score-matched historical control group that had received CCRT alone.

RESULTS

In total, 40 patients were enrolled and analyzed. The 24-month progression-free survival (PFS) and overall survival rates were 57.5% and 75%, respectively. Compared with the historical control group (n = 75), the study population had significantly longer PFS (hazard ratio [HR], 0.52; 95% confidence interval [CI], 0.28-0.97; P = .040) and overall survival (HR, 0.49; 95% CI, 0.25-0.98; P = .043). In the study population, patients who had PD-L1-positive tumors (n = 28) had significantly longer PFS (HR, 0.20; 95% CI, 0.07-0.54; P < .001) and overall survival (HR, 0.16; 95% CI, 0.05-0.56; P = .001) compared with those who had PD-L1-negative tumors (n = 11). However, there was no difference in survival outcomes according to PD-L1 status in the historical control group, indicating a strong interaction between PD-L1-positive status and survival outcomes in the treatment groups (PFS, P for interaction = .003; overall survival, P for interaction = .002).

CONCLUSIONS

Durvalumab and tremelimumab with definitive CCRT had promising efficacy in patients with locally advanced ESCC. In addition, PD-L1 expression had strong predictive value in the study population.