The Effects of Model Misspecification in Unanchored Matching-Adjusted Indirect Comparison: Results of a Simulation Study

Matching-Adjusted Indirect Comparison of the 52-Week Efficacy of Bimekizumab Versus Secukinumab and Ixekizumab for the Treatment of Radiographic Axial Spondyloarthritis

INTRODUCTION

A previous network meta-analysis established 16-week relative efficacy with bimekizumab, an inhibitor of interleukin (IL)-17F in addition to IL-17A, versus other treatments for patients with radiographic axial spondyloarthritis (r-axSpA; i.e., ankylosing spondylitis), including the IL-17A inhibitors secukinumab and ixekizumab. This matching-adjusted indirect comparison (MAIC) assessed 52-week relative efficacy of bimekizumab versus secukinumab and ixekizumab.

METHODS

Individual patient data from BE MOBILE 2 (bimekizumab 160 mg; N = 220) were matched to pooled summary data from MEASURE 1/2/3/4 (secukinumab 150 mg), MEASURE 3 (secukinumab 300 mg; escalated dose for inadequate responders), COAST-V (ixekizumab) and COAST-V/-W (ixekizumab). BE MOBILE 2 patients were reweighted using propensity score weights based on age, sex, ethnicity, tumor necrosis factor inhibitor (TNFi) exposure, weight, baseline ASDAS and BASFI (secukinumab) and baseline BASDAI (ixekizumab), and 52-week efficacy outcomes from the trial recalculated. Odds ratios (OR) or mean difference for unanchored comparisons are reported with 95% confidence intervals (CI).

RESULTS

At week 52, MAIC demonstrated that patients may have higher likelihood of improvement in key efficacy outcomes with bimekizumab versus secukinumab 150 mg (e.g., ASAS40: [OR (95% CI): 1.48 (1.05, 2.10); p = 0.026]; effective sample size [ESS] = 177). Differences in 52-week efficacy outcomes between bimekizumab and secukinumab 300 mg dose escalation were non-significant (ESS = 120). Bimekizumab versus ixekizumab 80 mg comparisons (COAST-V only; ESS = 84) also suggested that differences were non-significant for most key efficacy outcomes. Other ixekizumab comparisons (COAST-V/-W; ESS = 45) suggested bimekizumab may have higher comparative efficacy for many of the same efficacy outcomes, however ixekizumab analyses were limited by poor population overlap, likely due to the greater proportion of patients with previous TNFi exposure.

CONCLUSIONS

Patients treated with bimekizumab may have a higher likelihood of achieving improved longer-term efficacy versus secukinumab 150 mg, suggesting bimekizumab may be a favorable therapeutic option for r-axSpA. Differences in efficacy outcomes with bimekizumab versus ixekizumab 80 mg were mostly non-significant, depending on the populations considered.

Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study

Population-adjusted indirect comparisons, developed in the 2010s, enable comparisons between two treatments in different studies by balancing patient characteristics in the case where individual patient-level data (IPD) are available for only one study. Health technology assessment (HTA) bodies increasingly rely on these methods to inform funding decisions, typically using unanchored indirect comparisons (i.e., without a common comparator), due to the need to evaluate comparative efficacy and safety for single-arm trials. Unanchored matching-adjusted indirect comparison (MAIC) and unanchored simulated treatment comparison (STC) are currently the only two approaches available for population-adjusted indirect comparisons based on single-arm trials. However, there is a notable underutilisation of unanchored STC in HTA, largely due to a lack of understanding of its implementation. We therefore develop a novel way to implement unanchored STC by incorporating standardisation/marginalisation and the NORmal To Anything (NORTA) algorithm for sampling covariates. This methodology aims to derive a suitable marginal treatment effect without aggregation bias for HTA evaluations. We use a non-parametric bootstrap and propose separately calculating the standard error for the IPD study and the comparator study to ensure the appropriate quantification of the uncertainty associated with the estimated treatment effect. The performance of our proposed unanchored STC approach is evaluated through a comprehensive simulation study focused on binary outcomes. Our findings demonstrate that the proposed approach is asymptotically unbiased. We argue that unanchored STC should be considered when conducting unanchored indirect comparisons with single-arm studies, presenting a robust approach for HTA decision-making.

Cost-effectiveness analysis of Anaprazole versus Ilaprazole for the treatment of duodenal ulcers in China

Objective

Anaprazole, an innovative drug, has shown promise in initial clinical trials for patients with duodenal ulcers (DU) in China. This study aimed to evaluate the potential effects, safety, and cost-effectiveness of Anaprazole compared to Ilaprazole in the treatment of DU and the budgetary impact on the healthcare system.

Methods

Two multicentre, randomized controlled trials were used as data sources. The efficacy and safety of Anaprazole and Ilaprazole were compared using an anchored matching-adjusted indirect comparison (MAIC). A cost-utility analysis (CUA) was performed using a Markov model. A budget impact analysis (BIA) was conducted to evaluate the impact on the expenditure of the Chinese healthcare system. Deterministic and probabilistic sensitivity analyses were undertaken to test the uncertainty.

Results

The study findings indicated that Anaprazole and Ilaprazole have similar efficacy and safety in treating DU (OR = 1.05; 95% CI, 0.94-1.01; p = 0.35; OR = 0.63; 95% CI, 0.39-1.08; p = 0.12). The ICUR was 2,995.41¥/QALY, which is below the WTP threshold. The CUA results showed that Anaprazole is a cost-effective intervention with a probability of 85% at a given threshold. The results demonstrated strong robustness in the sensitivity analysis. Anaprazole imposed a low burden on the Chinese healthcare budget in the BIA.

Conclusion

Compared with Ilaprazole, Anaprazole has similar efficacy, safety, and high cost-effectiveness, while also impacting the total expenditure of the healthcare system.

Clinical Trial Registration

ClinicalTrials.gov, identifier NCT04215653 and NCT02847455.

Methods for Indirect Treatment Comparison: Results from a Systematic Literature Review

INTRODUCTION

Health technology assessment (HTA) agencies express a clear preference for randomized controlled trials when assessing the comparative efficacy of two or more treatments. However, an indirect treatment comparison (ITC) is often necessary where a direct comparison is unavailable or, in some cases, not possible. Numerous ITC techniques are described in the literature. A systematic literature review (SLR) was conducted to identify all the relevant literature on existing ITC techniques, provide a comprehensive description of each technique and evaluate their strengths and limitations from an HTA perspective in order to develop guidance on the most appropriate method to use in different scenarios.

METHODS

Electronic database searches of Embase and PubMed, as well as grey literature searches, were conducted on 15 November 2021. Eligible articles were peer-reviewed papers that specifically described the methods used for different ITC techniques and were written in English. The review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

A total of 73 articles were included in the SLR, reporting on seven different ITC techniques. All reported techniques were forms of adjusted ITC. Network meta-analysis (NMA) was the most frequently described technique (in 79.5% of the included articles), followed by matching-adjusted indirect comparison (MAIC) (30.1%), network meta-regression (24.7%), the Bucher method (23.3%), simulated treatment comparison (STC) (21.9%), propensity score matching (4.1%) and inverse probability of treatment weighting (4.1%). The appropriate choice of ITC technique is critical and should be based on the feasibility of a connected network, the evidence of heterogeneity between and within studies, the overall number of relevant studies and the availability of individual patient-level data (IPD). MAIC and STC were found to be common techniques in the case of single-arm studies, which are increasingly being conducted in oncology and rare diseases, whilst the Bucher method and NMA provide suitable options where no IPD is available.

CONCLUSION

ITCs can provide alternative evidence where direct comparative evidence may be missing. ITCs are currently considered by HTA agencies on a case-by-case basis; however, their acceptability remains low. Clearer international consensus and guidance on the methods to use for different ITC techniques is needed to improve the quality of ITCs submitted to HTA agencies. ITC techniques continue to evolve quickly, and more efficient techniques may become available in the future.

Unanchored Population-Adjusted Indirect Comparison Methods for Time-to-Event Outcomes Using Inverse Odds Weighting, Regression Adjustment, and Doubly Robust Methods With Either Individual Patient or Aggregate Data

OBJECTIVES

Several methods for unanchored population-adjusted indirect comparisons (PAICs) are available. Exploring alternative adjustment methods, depending on the available individual patient data (IPD) and the aggregate data (AD) in the external study, may help minimize bias in unanchored indirect comparisons. However, methods for time-to-event outcomes are not well understood. This study provides an overview and comparison of methods using a case study to increase familiarity. A recent method is applied to marginalize conditional hazard ratios, which allows for the comparisons of methods, and a doubly robust method is proposed.

METHODS

The following PAIC methods were compared through a case study in third-line small cell lung cancer, comparing nivolumab with standard of care based on a single-arm phase II trial (CheckMate 032) and real-world study (Flatiron) in terms of overall survival: IPD-IPD analyses using inverse odds weighting, regression adjustment, and a doubly robust method; IPD-AD analyses using matching-adjusted indirect comparison, simulated treatment comparison, and a doubly robust method.

RESULTS

Nivolumab extended survival versus standard of care with hazard ratios ranging from 0.63 (95% CI 0.44-0.90) in naive comparisons (identical estimates for IPD-IPD and IPD-AD analyses) to 0.69 (95% CI 0.44-0.98) in the IPD-IPD analyses using regression adjustment. Regression-based and doubly robust estimates yielded slightly wider confidence intervals versus the propensity score-based analyses.

CONCLUSIONS

The proposed doubly robust approach for time-to-event outcomes may help to minimize bias due to model misspecification. However, all methods for unanchored PAIC rely on the strong assumption that all prognostic covariates have been included.

Pricing and reimbursement mechanisms for advanced therapy medicinal products in 20 countries

Introduction: Advanced Therapy Medicinal Products are a type of therapies that, in some cases, hold great potential for patients without an effective current therapeutic approach but they also present multiple challenges to payers. While there are many theoretical papers on pricing and reimbursement (P&R) options, original empirical research is very scarce. This paper aims to provide a comprehensive international review of regulatory and P&R decisions taken for all ATMPs with centralized European marketing authorization in March 2022. Methods: A survey was distributed in July 2022 to representatives of 46 countries. Results: Responses were received from 20 countries out of 46 (43.5%). 14 countries reimbursed at least one ATMP. Six countries in this survey reimbursed no ATMPs. Conclusion: Access to ATMPs is uneven across the countries included in this study. This arises from regulatory differences, commercial decisions by marketing authorization holders, and the divergent assessment processes and criteria applied by payers. Moving towards greater equality of access will require cooperation between countries and stakeholders, for example, through the WHO Regional Office for Europe's Access to Novel Medicines Platform.

Enriching single-arm clinical trials with external controls: possibilities and pitfalls

For the past decade, it has become commonplace to provide rapid answers and early patient access to innovative treatments in the absence of randomized clinical trials (RCT), with benefits estimated from single-arm trials. This trend is important in oncology, notably when assessing new targeted therapies. Some of those uncontrolled trials further include an external/synthetic control group as an innovative way to provide an indirect comparison with a pertinent control group. We aimed to provide some guidelines as a comprehensive tool for (1) the critical appraisal of those comparisons or (2) for performing a single-arm trial. We used the example of ciltacabtagene autoleucel for the treatment of adult patients with relapsed or refractory multiple myeloma after 3 or more treatment lines as an illustrative example. We propose a 3-step guidance. The first step includes the definition of an estimand, which encompasses the treatment effect and the targeted population (whole population or restricted to single-arm trial or external controls), reflecting a clinical question. The second step relies on the adequate selection of external controls from previous RCTs or real-world data from patient cohorts, registries, or electronic patient files. The third step consists of choosing the statistical approach targeting the treatment effect defined above and depends on the available data (individual-level data or aggregated external data). The validity of the treatment effect derived from indirect comparisons heavily depends on careful methodological considerations included in the proposed 3-step procedure. Because the level of evidence of a well-conducted RCT cannot be guaranteed, the evaluation is more important than in standard settings.

Challenges of using external data in clinical trials- an illustration in patients with COVID-19

BACKGROUND

To improve the efficiency of clinical trials, leveraging external data on control and/or treatment effects, which is almost always available, appears to be a promising approach.

METHODS

We used data from the experimental arm of the Covidicus trial evaluating high-dose dexamethasone in severely ill and mechanically ventilated COVID-19 patients, using published data from the Recovery trial as external data, to estimate the 28-day mortality rate. Primary approaches to deal with external data were applied.

RESULTS

Estimates ranged from 0.241 ignoring the external data up to 0.294 using hierarchical Bayesian models. Some evidence of differences in mortality rates between the Covidicus and Recovery trials were observed, with an matched adjusted odds ratio of death in the Covidicus arm of 0.41 compared to the Recovery arm.

CONCLUSIONS

These indirect comparisons appear sensitive to the method used. None of those approaches appear robust enough to overcome randomized clinical trial data.

TRIAL REGISTRATION

Covidicus Trial: NCT04344730, First Posted: 14/04/2020; Recovery trial: NCT04381936.

Parametric G-computation for compatible indirect treatment comparisons with limited individual patient data

Population adjustment methods such as matching-adjusted indirect comparison (MAIC) are increasingly used to compare marginal treatment effects when there are cross-trial differences in effect modifiers and limited patient-level data. MAIC is based on propensity score weighting, which is sensitive to poor covariate overlap and cannot extrapolate beyond the observed covariate space. Current outcome regression-based alternatives can extrapolate but target a conditional treatment effect that is incompatible in the indirect comparison. When adjusting for covariates, one must integrate or average the conditional estimate over the relevant population to recover a compatible marginal treatment effect. We propose a marginalization method based on parametric G-computation that can be easily applied where the outcome regression is a generalized linear model or a Cox model. The approach views the covariate adjustment regression as a nuisance model and separates its estimation from the evaluation of the marginal treatment effect of interest. The method can accommodate a Bayesian statistical framework, which naturally integrates the analysis into a probabilistic framework. A simulation study provides proof-of-principle and benchmarks the method's performance against MAIC and the conventional outcome regression. Parametric G-computation achieves more precise and more accurate estimates than MAIC, particularly when covariate overlap is poor, and yields unbiased marginal treatment effect estimates under no failures of assumptions. Furthermore, the marginalized regression-adjusted estimates provide greater precision and accuracy than the conditional estimates produced by the conventional outcome regression, which are systematically biased because the measure of effect is non-collapsible.

Two-stage matching-adjusted indirect comparison

BACKGROUND

Anchored covariate-adjusted indirect comparisons inform reimbursement decisions where there are no head-to-head trials between the treatments of interest, there is a common comparator arm shared by the studies, and there are patient-level data limitations. Matching-adjusted indirect comparison (MAIC), based on propensity score weighting, is the most widely used covariate-adjusted indirect comparison method in health technology assessment. MAIC has poor precision and is inefficient when the effective sample size after weighting is small.

METHODS

A modular extension to MAIC, termed two-stage matching-adjusted indirect comparison (2SMAIC), is proposed. This uses two parametric models. One estimates the treatment assignment mechanism in the study with individual patient data (IPD), the other estimates the trial assignment mechanism. The first model produces inverse probability weights that are combined with the odds weights produced by the second model. The resulting weights seek to balance covariates between treatment arms and across studies. A simulation study provides proof-of-principle in an indirect comparison performed across two randomized trials. Nevertheless, 2SMAIC can be applied in situations where the IPD trial is observational, by including potential confounders in the treatment assignment model. The simulation study also explores the use of weight truncation in combination with MAIC for the first time.

RESULTS

Despite enforcing randomization and knowing the true treatment assignment mechanism in the IPD trial, 2SMAIC yields improved precision and efficiency with respect to MAIC in all scenarios, while maintaining similarly low levels of bias. The two-stage approach is effective when sample sizes in the IPD trial are low, as it controls for chance imbalances in prognostic baseline covariates between study arms. It is not as effective when overlap between the trials' target populations is poor and the extremity of the weights is high. In these scenarios, truncation leads to substantial precision and efficiency gains but induces considerable bias. The combination of a two-stage approach with truncation produces the highest precision and efficiency improvements.

CONCLUSIONS

Two-stage approaches to MAIC can increase precision and efficiency with respect to the standard approach by adjusting for empirical imbalances in prognostic covariates in the IPD trial. Further modules could be incorporated for additional variance reduction or to account for missingness and non-compliance in the IPD trial.

Matching-adjusted indirect comparison via a polynomial-based non-linear optimisation method

Aim: To demonstrate the potential of fourth-order polynomials within a non-linear optimisation framework for matching-adjusted indirect comparison (MAIC). Materials & methods: Simulated individual patient data were reweighted via fourth-order polynomials (polyMAIC) to match aggregate-level data across multiple baseline characteristics. The polyMAIC approach employed pre-specified matching tolerances and maximum allowable weights. Matching performance against aggregate-level targets was assessed, and also compared against the current industry-standard MAIC approach (Signorovitch). Results: The polyMAIC method matched aggregate-level targets within pre-specified tolerances. Effective sample sizes were either similar to or somewhat higher than those obtained from the Signorovitch method. Performance gains from polyMAIC tended to increase as matching complexity increased. Conclusion: PolyMAIC incorporates greater flexibility than the industry-standard MAIC approach and demonstrates matching potential.

Matching-Adjusted Indirect Comparison of Crisaborole Ointment 2% vs. Topical Calcineurin Inhibitors in the Treatment of Patients with Mild-to-Moderate Atopic Dermatitis

Introduction

Crisaborole topical ointment, 2%, is a nonsteroidal, topical anti-inflammatory phosphodiesterase-4 (PDE4) inhibitor that is approved for the treatment of mild-to-moderate atopic dermatitis (AD). The objective of the current analysis was to compare the efficacy of crisaborole 2% relative to pimecrolimus 1%, tacrolimus 0.03% and tacrolimus 0.1% in patients aged ≥ 2 years with mild-to-moderate AD by comparing improvement in Investigator’s Static Global Assessment scores ( (ISGA scores of 0/1 indicating “clear or almost clear”). ISGA was selected as the primary efficacy outcome given the US Food and Drug Administration’s recommendations on the use of ISGA for assessment of global severity in AD and to align with efficacy measurements in the crisaborole registration trials. Safety endpoints could not be analyzed due to differences in outcome definitions across studies.

Methods

Efficacy of crisaborole was evaluated using individual patient data (IPD) from two pivotal phase III randomized controlled trials (RCTs), and efficacy of comparators was evaluated using published RCTs included in a previous network meta-analysis. Vehicle controls were not comparable due to differences in ingredients and population imbalance and, therefore, an unanchored matching-adjusted indirect comparison (MAIC) was used, which reweighted IPD for crisaborole to estimate absolute response in comparator populations.

Results

The odds of achieving an improvement in ISGA score was higher with crisaborole 2% versus pimecrolimus 1% (odds ratio [OR] 2.03; 95% confidence interval [CI] 1.45–2.85; effective sample size =  627, reduced from 1021; p value < 0.001) and for crisaborole 2% versus tacrolimus 0.03% (OR 1.50; 95% CI 1.09–2.05; effective sample size = 311, reduced from 1021; p = 0.012).

Conclusion

The unanchored MAIC suggests that the odds of achieving an improvement in ISGA score is greater with crisaborole 2% than with pimecrolimus 1% or tacrolimus 0.03% in patients aged ≥ 2 years with mild-to-moderate AD. These results are consistent with findings from the previously published network meta-analysis, which used a different methodology for performing indirect treatment comparisons.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13555-021-00646-1.

Population-adjusted indirect treatment comparison of maintenance PARP inhibitor with or without bevacizumab versus bevacizumab alone in women with newly diagnosed advanced ovarian cancer

BACKGROUND

In patients with newly diagnosed ovarian cancer, bevacizumab and poly(ADP-ribose) polymerase (PARP) inhibitors, alone or in combination, have shown benefit as maintenance treatment following platinum-based chemotherapy. However, no trials have compared a PARP inhibitor plus bevacizumab versus a PARP inhibitor, or a PARP inhibitor versus bevacizumab. We performed an unanchored population-adjusted indirect treatment comparison to estimate the relative efficacy and safety of maintenance treatments for newly diagnosed advanced ovarian cancer.

METHODS

Analyses were performed using aggregate data from the PRIMA trial and patient-level data from a subset of patients from the PAOLA-1 trial that met surgery and staging eligibility criteria of PRIMA. Propensity weights were used to match baseline characteristics of the PAOLA-1 subset to those of the PRIMA population. Analysis was performed in overall (biomarker-unselected) and homologous recombination repair deficiency (HRD)-positive populations.

RESULTS

A total of 595/806 (266/387 HRD-positive) PAOLA-1 patients were included. After matching, the effective sample size for PAOLA-1 was 532 (242 HRD-positive). Maintenance olaparib plus bevacizumab reduced the risk of disease progression or death by 43% [hazard ratio (HR) 0.57; 95% confidence interval (CI): 0.47-0.69] versus niraparib and by 40% (HR 0.60; 95% CI: 0.49-0.74) versus bevacizumab in the biomarker-unselected population and by 43% (HR 0.57; 95% CI: 0.41-0.79) and 60% (HR 0.40; 95% CI: 0.29-0.55), respectively, in the HRD-positive population. Progression-free survival (PFS) benefits of maintenance niraparib and bevacizumab arms were comparable in the biomarker-unselected population (HR 1.07; 95% CI: 0.87-1.32); however, niraparib showed a 30% reduced risk compared with bevacizumab (HR 0.70; 95% CI: 0.51-0.97) in the HRD-positive population.

CONCLUSIONS

In biomarker-unselected and HRD-positive patients, combination treatment with olaparib plus bevacizumab as maintenance treatment improves PFS for women with newly diagnosed advanced ovarian cancer compared with either bevacizumab or niraparib alone. Results are hypothesis generating and could guide randomised trial design.

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.