Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study

Uncertain about uncertainty in matching-adjusted indirect comparisons? A simulation study to compare methods for variance estimation

In health technology assessment, matching-adjusted indirect comparison (MAIC) is the most common method for pairwise comparisons that control for imbalances in baseline characteristics across trials. One of the primary challenges in MAIC is the need to properly account for the additional uncertainty introduced by the matching process. Limited evidence and guidance are available on variance estimation in MAICs. Therefore, we conducted a comprehensive Monte Carlo simulation study to evaluate the performance of different statistical methods across 108 scenarios. Four general approaches for variance estimation were compared in both anchored and unanchored MAICs of binary and time-to-event outcomes: (1) conventional estimators (CE) using raw weights; (2) CE using weights rescaled to the effective sample size (ESS); (3) robust sandwich estimators; and (4) bootstrapping. Several variants of sandwich estimators and bootstrap methods were tested. Performance was quantified on the basis of empirical coverage probabilities for 95% confidence intervals and variability ratios. Variability was underestimated by CE + raw weights when population overlap was poor or moderate. Despite several theoretical limitations, CE + ESS weights accurately estimated uncertainty across most scenarios. Original implementations of sandwich estimators had a downward bias in MAICs with a small ESS, and finite sample adjustments led to marked improvements. Bootstrapping was unstable if population overlap was poor and the sample size was limited. All methods produced valid coverage probabilities and standard errors in cases of strong population overlap. Our findings indicate that the sample size, population overlap, and outcome type are important considerations for variance estimation in MAICs.

Analytical Methods for Comparing Uncontrolled Trials With External Controls From Real-World Data: A Systematic Literature Review and Comparison With European Regulatory and Health Technology Assessment Practice

OBJECTIVES

This study aimed to provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials with external controls from individual patient data real-world data (IPD-RWD) and to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports.

METHODS

A systematic literature review (until March 1, 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively with methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and 4 European HTA organizations (2015-2023).

RESULTS

Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data with IPD-RWD-based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data, and analytical comparative modeling methods. Seven guidelines also focused on research design, RWD quality, and transparency aspects, and 4 of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n = 15) and HTA (n = 35) assessment reports were often based on aggregate data and lacked transparency owing to the few details provided.

CONCLUSIONS

Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.

Implementing Multilevel Network Meta-Regression for Time-To-Event Outcomes: A Case Study in Relapsed Refractory Multiple Myeloma

OBJECTIVES

Multilevel network meta-regression (ML-NMR) leverages individual patient data (IPD) and aggregate data from a network of randomized controlled trials (RCTs) to assess the comparative efficacy of multiple treatments, while adjusting for between-study differences. We provide an overview of ML-NMR for time-to-event outcomes and apply it to an illustrative case study, including example R code.

METHODS

The case study evaluated the comparative efficacy of idecabtagene vicleucel (ide-cel), selinexor+dexamethasone (Sd), belantamab mafodotin (BM), and conventional care (CC) for patients with triple-class exposed relapsed/refractory multiple myeloma in terms of overall survival. Single-arm clinical trials and real-world data were naively combined to create an aggregate data artificial RCT (aRCT) (MAMMOTH-CC versus DREAMM-2-BM versus STORM-2-Sd) and an IPD aRCT (KarMMa-ide-cel versus KarMMa-RW-CC). With some assumptions, we incorporated continuous covariates with skewed distributions, reported as median and range. The ML-NMR models adjusted for number of prior lines, triple-class refractory status, and age and were compared using the leave-one-out information criterion. We summarized predicted hazard ratios and survival (95% credible intervals) in the IPD aRCT population.

RESULTS

The Weibull ML-NMR model had the lowest leave-one-out information criterion. Ide-cel was more efficacious than Sd, BM, and CC in terms of overall survival. Effect modifiers had minimal impact on the model, and only triple-class refractory was a prognostic factor.

CONCLUSIONS

We demonstrate an application of ML-NMR for time-to-event outcomes and introduce code that can be used to aid implementation. Given its benefits, we encourage practitioners to utilize ML-NMR when population adjustment is necessary for comparisons of multiple treatments.

Advancing unanchored simulated treatment comparisons: A novel implementation and simulation study

Population-adjusted indirect comparisons, developed in the 2010s, enable comparisons between two treatments in different studies by balancing patient characteristics in the case where individual patient-level data (IPD) are available for only one study. Health technology assessment (HTA) bodies increasingly rely on these methods to inform funding decisions, typically using unanchored indirect comparisons (i.e., without a common comparator), due to the need to evaluate comparative efficacy and safety for single-arm trials. Unanchored matching-adjusted indirect comparison (MAIC) and unanchored simulated treatment comparison (STC) are currently the only two approaches available for population-adjusted indirect comparisons based on single-arm trials. However, there is a notable underutilisation of unanchored STC in HTA, largely due to a lack of understanding of its implementation. We therefore develop a novel way to implement unanchored STC by incorporating standardisation/marginalisation and the NORmal To Anything (NORTA) algorithm for sampling covariates. This methodology aims to derive a suitable marginal treatment effect without aggregation bias for HTA evaluations. We use a non-parametric bootstrap and propose separately calculating the standard error for the IPD study and the comparator study to ensure the appropriate quantification of the uncertainty associated with the estimated treatment effect. The performance of our proposed unanchored STC approach is evaluated through a comprehensive simulation study focused on binary outcomes. Our findings demonstrate that the proposed approach is asymptotically unbiased. We argue that unanchored STC should be considered when conducting unanchored indirect comparisons with single-arm studies, presenting a robust approach for HTA decision-making.

A comprehensive review and shiny application on the matching-adjusted indirect comparison

Population-adjusted indirect comparison (PAIC) is an increasingly used technique for estimating the comparative effectiveness of different treatments for the health technology assessments when head-to-head trials are unavailable. Three commonly used PAIC methods include matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multilevel network meta-regression (ML-NMR). MAIC enables researchers to achieve balanced covariate distribution across two independent trials when individual participant data are only available in one trial. In this article, we provide a comprehensive review of the MAIC methods, including their theoretical derivation, implicit assumptions, and connection to calibration estimation in survey sampling. We discuss the nuances between anchored and unanchored MAIC, as well as their required assumptions. Furthermore, we implement various MAIC methods in a user-friendly R Shiny application Shiny-MAIC. To our knowledge, it is the first Shiny application that implements various MAIC methods. The Shiny-MAIC application offers choice between anchored or unanchored MAIC, choice among different types of covariates and outcomes, and two variance estimators including bootstrap and robust standard errors. An example with simulated data is provided to demonstrate the utility of the Shiny-MAIC application, enabling a user-friendly approach conducting MAIC for healthcare decision-making. The Shiny-MAIC is freely available through the link: https://ziren.shinyapps.io/Shiny_MAIC/.

Methods for Indirect Treatment Comparison: Results from a Systematic Literature Review

INTRODUCTION

Health technology assessment (HTA) agencies express a clear preference for randomized controlled trials when assessing the comparative efficacy of two or more treatments. However, an indirect treatment comparison (ITC) is often necessary where a direct comparison is unavailable or, in some cases, not possible. Numerous ITC techniques are described in the literature. A systematic literature review (SLR) was conducted to identify all the relevant literature on existing ITC techniques, provide a comprehensive description of each technique and evaluate their strengths and limitations from an HTA perspective in order to develop guidance on the most appropriate method to use in different scenarios.

METHODS

Electronic database searches of Embase and PubMed, as well as grey literature searches, were conducted on 15 November 2021. Eligible articles were peer-reviewed papers that specifically described the methods used for different ITC techniques and were written in English. The review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

A total of 73 articles were included in the SLR, reporting on seven different ITC techniques. All reported techniques were forms of adjusted ITC. Network meta-analysis (NMA) was the most frequently described technique (in 79.5% of the included articles), followed by matching-adjusted indirect comparison (MAIC) (30.1%), network meta-regression (24.7%), the Bucher method (23.3%), simulated treatment comparison (STC) (21.9%), propensity score matching (4.1%) and inverse probability of treatment weighting (4.1%). The appropriate choice of ITC technique is critical and should be based on the feasibility of a connected network, the evidence of heterogeneity between and within studies, the overall number of relevant studies and the availability of individual patient-level data (IPD). MAIC and STC were found to be common techniques in the case of single-arm studies, which are increasingly being conducted in oncology and rare diseases, whilst the Bucher method and NMA provide suitable options where no IPD is available.

CONCLUSION

ITCs can provide alternative evidence where direct comparative evidence may be missing. ITCs are currently considered by HTA agencies on a case-by-case basis; however, their acceptability remains low. Clearer international consensus and guidance on the methods to use for different ITC techniques is needed to improve the quality of ITCs submitted to HTA agencies. ITC techniques continue to evolve quickly, and more efficient techniques may become available in the future.

Sharing information across patient subgroups to draw conclusions from sparse treatment networks

Network meta-analysis (NMA) usually provides estimates of the relative effects with the highest possible precision. However, sparse networks with few available studies and limited direct evidence can arise, threatening the robustness and reliability of NMA estimates. In these cases, the limited amount of available information can hamper the formal evaluation of the underlying NMA assumptions of transitivity and consistency. In addition, NMA estimates from sparse networks are expected to be imprecise and possibly biased as they rely on large-sample approximations that are invalid in the absence of sufficient data. We propose a Bayesian framework that allows sharing of information between two networks that pertain to different population subgroups. Specifically, we use the results from a subgroup with a lot of direct evidence (a dense network) to construct informative priors for the relative effects in the target subgroup (a sparse network). This is a two-stage approach where at the first stage, we extrapolate the results of the dense network to those expected from the sparse network. This takes place by using a modified hierarchical NMA model where we add a location parameter that shifts the distribution of the relative effects to make them applicable to the target population. At the second stage, these extrapolated results are used as prior information for the sparse network. We illustrate our approach through a motivating example of psychiatric patients. Our approach results in more precise and robust estimates of the relative effects and can adequately inform clinical practice in presence of sparse networks.

Four alternative methodologies for simulated treatment comparison: How could the use of simulation be re-invigorated?

Simulated treatment comparison (STC) is an established method for performing population adjustment for the indirect comparison of two treatments, where individual patient data (IPD) are available for one trial but only aggregate level information is available for the other. The most commonly used method is what we call 'standard STC'. Here we fit an outcome model using data from the trial with IPD, and then substitute mean covariate values from the trial where only aggregate level data are available, to predict what the first of these trial's outcomes would have been if its population had been the same as the second. However, this type of STC methodology does not involve simulation and can result in bias when the link function used in the outcome model is non-linear. An alternative approach is to use the fitted outcome model to simulate patient profiles in the trial for which IPD are available, but in the other trial's population. This stochastic alternative presents additional challenges. We examine the history of STC and propose two new simulation-based methods that resolve many of the difficulties associated with the current stochastic approach. A virtue of the simulation-based STC methods is that the marginal estimands are then clearly targeted. We illustrate all methods using a numerical example and explore their use in a simulation study.

Unanchored Population-Adjusted Indirect Comparison Methods for Time-to-Event Outcomes Using Inverse Odds Weighting, Regression Adjustment, and Doubly Robust Methods With Either Individual Patient or Aggregate Data

OBJECTIVES

Several methods for unanchored population-adjusted indirect comparisons (PAICs) are available. Exploring alternative adjustment methods, depending on the available individual patient data (IPD) and the aggregate data (AD) in the external study, may help minimize bias in unanchored indirect comparisons. However, methods for time-to-event outcomes are not well understood. This study provides an overview and comparison of methods using a case study to increase familiarity. A recent method is applied to marginalize conditional hazard ratios, which allows for the comparisons of methods, and a doubly robust method is proposed.

METHODS

The following PAIC methods were compared through a case study in third-line small cell lung cancer, comparing nivolumab with standard of care based on a single-arm phase II trial (CheckMate 032) and real-world study (Flatiron) in terms of overall survival: IPD-IPD analyses using inverse odds weighting, regression adjustment, and a doubly robust method; IPD-AD analyses using matching-adjusted indirect comparison, simulated treatment comparison, and a doubly robust method.

RESULTS

Nivolumab extended survival versus standard of care with hazard ratios ranging from 0.63 (95% CI 0.44-0.90) in naive comparisons (identical estimates for IPD-IPD and IPD-AD analyses) to 0.69 (95% CI 0.44-0.98) in the IPD-IPD analyses using regression adjustment. Regression-based and doubly robust estimates yielded slightly wider confidence intervals versus the propensity score-based analyses.

CONCLUSIONS

The proposed doubly robust approach for time-to-event outcomes may help to minimize bias due to model misspecification. However, all methods for unanchored PAIC rely on the strong assumption that all prognostic covariates have been included.

Model-based standardization using multiple imputation

BACKGROUND

When studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect.

METHODS

The standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence.

RESULTS

We conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization.

CONCLUSION

We demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

A methodological review of population-adjusted indirect comparisons reveals inconsistent reporting and suggests publication bias

OBJECTIVES

Population-adjusted indirect comparisons (PAICs) were developed in the 2010s to allow for comparisons between two treatments evaluated in different trials while accounting for differences in patient characteristics if individual patient data (IPD) are available for only one trial. Such comparisons are increasingly used in market access applications when a pharmaceutical company compares its new treatment (with IPD available) to another treatment developed by a competitor (with only aggregated data available). This study aimed to describe the characteristics of these PAICs, assess their methodology, and describe the reported results.

STUDY DESIGN AND SETTING

Original articles reporting the use of at least one PAIC were searched on PubMed between January 1, 2010 and April 2, 2022. Two reviewers independently selected articles and extracted data.

RESULTS

We included 133 publications reporting the results of 288 PAICs. Half of the articles were published on or after May 7, 2020, and 71 (53%) pertained to onco-hematology. The pharmaceutical industry was involved in 130 (98%) articles. Key methodological aspects were reported inconsistently, with only three articles adequately reporting all aspects. A total of 161 (56%) articles reported a statistically significant benefit for the treatment evaluated on IPD. Conversely, only one PAIC significantly favored the treatment evaluated on aggregated data.

CONCLUSION

Although the number of published PAICs is increasing, the methodology and transparency need to be improved. Moreover, our study strongly suggests a reporting bias. This situation calls for strengthening guidelines to improve trust in PAIC results and thus their reliability in market access applications.

A Comparison of Relative-Efficacy Estimate(S) Derived From Both Matching-Adjusted Indirect Comparisons and Standard Anchored Indirect Treatment Comparisons: A Review of Matching-Adjusted Indirect Comparisons

OBJECTIVES

We present an empirical comparison of relative-efficacy estimate(s) from matching-adjusted indirect comparisons (MAICs) with estimates from corresponding standard anchored indirect treatment comparisons.

METHODS

A total of 80 comparisons were identified from 17 publications through a systematic rapid review. A standardized metric that used reported relative treatment efficacy estimates and their associated uncertainty was used to compare the methods across different treatment indications and outcome measures.

RESULTS

On aggregate, MAICs presented for connected networks tended to report a more favorable relative-efficacy estimate for the treatment for which individual-level patient data were available relative to the reported indirect treatment comparison estimate.

CONCLUSIONS

Although we recognize the importance of MAIC and other population adjustment methods in certain situations, we recommend that results from these analyses are interpreted with caution. Researchers and analysts should carefully consider if MAICs are appropriate where presented and whether MAICs would have added value where omitted.

Transportability of two heart failure trials to a disease registry using individual patient data

OBJECTIVES

Randomized controlled trials are the gold-standard for determining therapeutic efficacy, but are often unrepresentative of real-world settings. Statistical transportation methods (hereafter transportation) can partially account for these differences, improving trial applicability without breaking randomization. We transported treatment effects from two heart failure (HF) trials to a HF registry.

STUDY DESIGN AND SETTING

Individual-patient-level data from two trials (Carvedilol or Metoprolol European Trial (COMET), comparing carvedilol and metoprolol, and digitalis investigation group trial (DIG), comparing digoxin and placebo) and a Scottish HF registry were obtained. The primary end point for both trials was all-cause mortality; composite outcomes were all-cause mortality or hospitalization for COMET and HF-related death or hospitalization for DIG. We performed transportation using regression-based and inverse odds of sampling weights (IOSW) approaches.

RESULTS

Registry patients were older, had poorer renal function and received higher-doses of loop-diuretics than trial participants. For each trial, point estimates were similar for the original and IOSW (e.g., DIG composite outcome: OR 0.75 (0.69, 0.82) vs. 0.73 (0.64, 0.83)). Treatment effect estimates were also similar when examining high-risk (0.64 (0.46, 0.89)) and low-risk registry patients (0.73 (0.61, 0.86)). Similar results were obtained using regression-based transportation.

CONCLUSION

Regression-based or IOSW approaches can be used to transport trial effect estimates to patients administrative/registry data, with only moderate reductions in precision.

Comparison of indirect treatment methods in migraine prevention to address differences in mode of administration

Aim: Indirect treatment comparisons (ITCs) are anchored on a placebo comparator, and the placebo response may vary according to drug administration route. Migraine preventive treatment studies were used to evaluate ITCs and determine whether mode of administration influences placebo response and the overall study findings. Materials & methods: Change from baseline in monthly migraine days produced by monoclonal antibody treatments (subcutaneous, intravenous) was compared using fixed-effects Bayesian network meta-analysis (NMA), network meta-regression (NMR), and unanchored simulated treatment comparison (STC). Results: NMA and NMR provide mixed, rarely differentiated results between treatments, whereas unanchored STC strongly favors eptinezumab over other preventive treatments. Conclusion: Further investigations are needed to determine which ITC best reflects the impact of mode of administration on placebo.

Indirect Treatment Comparisons of Lenacapavir Plus Optimized Background Regimen Versus Other Treatments for Multidrug-Resistant Human Immunodeficiency Virus

BACKGROUND/AIMS

Heavily treatment-experienced (HTE) people with human immunodeficiency virus (HIV) (PWH) may not achieve virologic suppression (VS) with combination antiretroviral therapy due to multidrug resistance (MDR), intolerance, and safety concerns. These PWH often receive highly individualized treatment regimens, but these regimens may not enable PWH to achieve VS, thereby halting disease progression. Novel medications are required for treating individuals with MDR HIV. Lenacapavir (LEN), a first-in-class HIV capsid inhibitor, is under investigation for the treatment of HTE individuals with MDR HIV in the phase 2/3 CAPELLA study. This study aimed to compare LEN plus optimized background regimen (OBR) with fostemsavir (FTR) + OBR, ibalizumab (IBA) + OBR, and OBR alone in terms of VS, CD4 cell count change from baseline, immunologic recovery, and discontinuation due to adverse events, using indirect treatment comparisons.

METHODS

A systematic review identified clinical evidence on HIV-1 treatments in HTE PWH. A feasibility assessment evaluated the identified studies for indirect treatment comparison analyses based on population characteristics, interventions, comparators, and outcomes of interest. Unanchored simulated treatment comparisons of LEN + OBR versus comparators were conducted.

RESULTS

LEN + OBR had 6.57 times higher odds of VS at weeks 24 to 28 than FTR + OBR (95% confidence interval [CI] 1.34-32.28), 8.93 times higher odds of VS than IBA + OBR (95% CI 2.07-38.46), and 12.74 times higher odds of VS than OBR alone (95% CI 1.70-95.37). Change from baseline in CD4 cell count was similar across LEN + OBR, FTR + OBR, and IBA + OBR.

CONCLUSION

LEN + OBR has statistically significantly greater odds of VS at weeks 24 to 28 than its comparators and represents a novel treatment for people with MDR HIV.

Belimumab versus anifrolumab in adults with systemic lupus erythematosus: an indirect comparison of clinical response at 52 weeks

OBJECTIVE

To generate comparative efficacy evidence of belimumab versus anifrolumab in SLE that can inform treatment practices.

METHODS

The SLE Responder Index (SRI)-4 response at 52 weeks of belimumab versus anifrolumab was evaluated with an indirect treatment comparison. The evidence base consisted of randomised trials that were compiled through a systemic literature review.A feasibility assessment was performed to comprehensively compare the eligible trials and to determine the most appropriate indirect treatment comparison analysis method. A multilevel network meta-regression (ML-NMR) was implemented that adjusted for differences across trials in four baseline characteristics: SLE Disease Activity Index-2K, anti-double-stranded DNA antibody positive, low complement (C)3 and low C4. Additional analyses were conducted to explore if the results were robust to different sets of baseline characteristics included for adjustment, alternative adjustment methods and changes to the trials included in the evidence base.

RESULTS

The ML-NMR included eight trials: five belimumab trials (BLISS-52, BLISS-76, NEA, BLISS-SC, EMBRACE) and three anifrolumab trials (MUSE, TULIP-1, TULIP-2). Belimumab and anifrolumab were comparable in terms of SRI-4 response (OR (95% credible interval), 1.04 (0.74-1.45)), with the direction of the point estimate slightly favouring belimumab. Belimumab had a 0.58 probability of being the more effective treatment. The results were highly consistent across all analysis scenarios.

CONCLUSIONS

Our results suggest that the SRI-4 response of belimumab and anifrolumab are similar at 52 weeks in the general SLE population, but the level of uncertainty around the point estimate means we cannot rule out the possibility of a clinically meaningful benefit for either treatment. It remains to be seen if specific groups of patients could derive a greater benefit from anifrolumab or from belimumab, and there is certainly an unmet need to identify robust predictors towards more personalised selection of available biological agents in SLE.

Synthesizing cross-design evidence and cross-format data using network meta-regression

In network meta-analysis (NMA), we synthesize all relevant evidence about health outcomes with competing treatments. The evidence may come from randomized clinical trials (RCT) or non-randomized studies (NRS) as individual participant data (IPD) or as aggregate data (AD). We present a suite of Bayesian NMA and network meta-regression (NMR) models allowing for cross-design and cross-format synthesis. The models integrate a three-level hierarchical model for synthesizing IPD and AD into four approaches. The four approaches account for differences in the design and risk of bias (RoB) in the RCT and NRS evidence. These four approaches variously ignoring differences in RoB, using NRS to construct penalized treatment effect priors and bias-adjustment models that control the contribution of information from high RoB studies in two different ways. We illustrate the methods in a network of three pharmacological interventions and placebo for patients with relapsing-remitting multiple sclerosis. The estimated relative treatment effects do not change much when we accounted for differences in design and RoB. Conducting network meta-regression showed that intervention efficacy decreases with increasing participant age. We also re-analysed a network of 431 RCT comparing 21 antidepressants, and we did not observe material changes in intervention efficacy when adjusting for studies' high RoB. We re-analysed both case studies accounting for different study RoB. In summary, the described suite of NMA/NMR models enables the inclusion of all relevant evidence while incorporating information on the within-study bias in both observational and experimental data and enabling estimation of individualized treatment effects through the inclusion of participant characteristics.

Network meta-interpolation: Effect modification adjustment in network meta-analysis using subgroup analyses

Effect modification (EM) may cause bias in network meta-analysis (NMA). Existing population adjustment NMA methods use individual patient data to adjust for EM but disregard available subgroup information from aggregated data in the evidence network. Additionally, these methods often rely on the shared effect modification (SEM) assumption. In this paper, we propose Network Meta-Interpolation (NMI): a method using subgroup analyses to adjust for EM that does not assume SEM. NMI balances effect modifiers across studies by turning treatment effect (TE) estimates at the subgroup- and study level into TE and standard errors at EM values common to all studies. In an extensive simulation study, we simulate two evidence networks consisting of four treatments, and assess the impact of departure from the SEM assumption, variable EM correlation across trials, trial sample size and network size. NMI was compared to standard NMA, network meta-regression (NMR) and Multilevel NMR (ML-NMR) in terms of estimation accuracy and credible interval (CrI) coverage. In the base case non-SEM dataset, NMI achieved the highest estimation accuracy with root mean squared error (RMSE) of 0.228, followed by standard NMA (0.241), ML-NMR (0.447) and NMR (0.541). In the SEM dataset, NMI was again the most accurate method with RMSE of 0.222, followed by ML-NMR (0.255). CrI coverage followed a similar pattern. NMI's dominance in terms of estimation accuracy and CrI coverage appeared to be consistent across all scenarios. NMI represents an effective option for NMA in the presence of study imbalance and available subgroup data.

Validating the Assumptions of Population Adjustment: Application of Multilevel Network Meta-regression to a Network of Treatments for Plaque Psoriasis

BACKGROUND

Network meta-analysis (NMA) and indirect comparisons combine aggregate data (AgD) from multiple studies on treatments of interest but may give biased estimates if study populations differ. Population adjustment methods such as multilevel network meta-regression (ML-NMR) aim to reduce bias by adjusting for differences in study populations using individual patient data (IPD) from 1 or more studies under the conditional constancy assumption. A shared effect modifier assumption may also be necessary for identifiability. This article aims to demonstrate how the assumptions made by ML-NMR can be assessed in practice to obtain reliable treatment effect estimates in a target population.

METHODS

We apply ML-NMR to a network of evidence on treatments for plaque psoriasis with a mix of IPD and AgD trials reporting ordered categorical outcomes. Relative treatment effects are estimated for each trial population and for 3 external target populations represented by a registry and 2 cohort studies. We examine residual heterogeneity and inconsistency and relax the shared effect modifier assumption for each covariate in turn.

RESULTS

Estimated population-average treatment effects were similar across study populations, as differences in the distributions of effect modifiers were small. Better fit was achieved with ML-NMR than with NMA, and uncertainty was reduced by explaining within- and between-study variation. We found little evidence that the conditional constancy or shared effect modifier assumptions were invalid.

CONCLUSIONS

ML-NMR extends the NMA framework and addresses issues with previous population adjustment approaches. It coherently synthesizes evidence from IPD and AgD studies in networks of any size while avoiding aggregation bias and noncollapsibility bias, allows for key assumptions to be assessed or relaxed, and can produce estimates relevant to a target population for decision-making.

HIGHLIGHTS

Multilevel network meta-regression (ML-NMR) extends the network meta-analysis framework to synthesize evidence from networks of studies providing individual patient data or aggregate data while adjusting for differences in effect modifiers between studies (population adjustment). We apply ML-NMR to a network of treatments for plaque psoriasis with ordered categorical outcomes.We demonstrate for the first time how ML-NMR allows key assumptions to be assessed. We check for violations of conditional constancy of relative effects (such as unobserved effect modifiers) through residual heterogeneity and inconsistency and the shared effect modifier assumption by relaxing this for each covariate in turn.Crucially for decision making, population-adjusted treatment effects can be produced in any relevant target population. We produce population-average estimates for 3 external target populations, represented by the PsoBest registry and the PROSPECT and Chiricozzi 2019 cohort studies.

"Don't Think Twice, It's All Right": Using Additional Data to Reduce Uncertainty Regarding Oncologic Drugs Provided Through Managed Access Agreements in England

OBJECTIVES

The Cancer Drugs Fund (CDF) in England uses managed access agreements to facilitate additional data collection to address uncertainties identified in the appraisals of new drugs. This study reviews the uncertainties highlighted in the original appraisals where recommendations "to use within the CDF" were made and how additional data were used to address these uncertainties in the CDF review appraisals where final decisions on routine commissioning were made.

METHODS

The first 24 drugs exiting the 2016 CDF were included in this review. The information about uncertainty and the use of newly collected data were extracted from the original appraisals and the CDF review appraisals. The additional data used in the CDF review appraisals, distinguishing between clinical trial data and real-world data (RWD), were reviewed to assess the extent to which the additional data were able to reduce the original uncertainties.

RESULTS

The recommendation that the drug be routinely commissioned was made in 87.5% of re-appraisals. Uncertainty stemming from immaturity of the survival data in clinical trials was frequently found in appraisals. Later follow-up of clinical trials was used to address this uncertainty, whereas limited use was made of RWD. The Systemic Anti-Cancer Therapy (SACT) dataset is the most frequently used source of RWD. SACT data were mostly used in review appraisals to support the clinical outcomes based on later follow-up of trial participants and to inform modelling of subsequent treatments or treatment duration.

CONCLUSIONS

While additionally collected RWD attracted attention when the 2016 CDF was introduced, RWD have not been widely used in CDF review appraisals and (to date) have done little to reduce uncertainty. Experience with these appraisals has highlighted the importance of longer follow-up of clinical trials and the relatively limited role of RWD, in general, and of SACT data in particular.

On the double-robustness and semiparametric efficiency of matching-adjusted indirect comparisons

Matching-adjusted indirect comparison (MAIC) enables indirect comparisons of interventions across separate studies when individual patient-level data (IPD) are available for only one study. Due to its similarity with propensity score weighting, it has been speculated that MAIC can be combined with outcome regression models in the spirit of augmented inverse probability weighting estimators to improve robustness and efficiency. We show that MAIC enjoys intrinsic double-robustness and semiparametric efficiency properties for estimating the average treatment effect on the treated in the limited IPD setting without explicit augmentation. A connection between MAIC and the method of simulated treatment comparisons is highlighted. These results clarify conditions under which MAIC is consistent and efficient, informing appropriate application and interpretation of MAIC analyses.

Parametric G-computation for compatible indirect treatment comparisons with limited individual patient data

Population adjustment methods such as matching-adjusted indirect comparison (MAIC) are increasingly used to compare marginal treatment effects when there are cross-trial differences in effect modifiers and limited patient-level data. MAIC is based on propensity score weighting, which is sensitive to poor covariate overlap and cannot extrapolate beyond the observed covariate space. Current outcome regression-based alternatives can extrapolate but target a conditional treatment effect that is incompatible in the indirect comparison. When adjusting for covariates, one must integrate or average the conditional estimate over the relevant population to recover a compatible marginal treatment effect. We propose a marginalization method based on parametric G-computation that can be easily applied where the outcome regression is a generalized linear model or a Cox model. The approach views the covariate adjustment regression as a nuisance model and separates its estimation from the evaluation of the marginal treatment effect of interest. The method can accommodate a Bayesian statistical framework, which naturally integrates the analysis into a probabilistic framework. A simulation study provides proof-of-principle and benchmarks the method's performance against MAIC and the conventional outcome regression. Parametric G-computation achieves more precise and more accurate estimates than MAIC, particularly when covariate overlap is poor, and yields unbiased marginal treatment effect estimates under no failures of assumptions. Furthermore, the marginalized regression-adjusted estimates provide greater precision and accuracy than the conditional estimates produced by the conventional outcome regression, which are systematically biased because the measure of effect is non-collapsible.

Calibrating a network meta-analysis of diabetes trials of sodium glucose cotransporter 2 inhibitors, glucagon-like peptide-1 receptor analogues and dipeptidyl peptidase-4 inhibitors to a representative routine population: a systematic review protocol

INTRODUCTION

Participants in randomised controlled trials (trials) are generally younger and healthier than many individuals encountered in clinical practice. Consequently, the applicability of trial findings is often uncertain. To address this, results from trials can be calibrated to more representative data sources. In a network meta-analysis, using a novel approach which allows the inclusion of trials whether or not individual-level participant data (IPD) is available, we will calibrate trials for three drug classes (sodium glucose cotransporter 2 (SGLT2) inhibitors, glucagon-like peptide-1 (GLP1) receptor analogues and dipeptidyl peptidase-4 (DPP4) inhibitors) to the Scottish diabetes register.

METHODS AND ANALYSIS

Medline and EMBASE databases, the US clinical trials registry (clinicaltrials.gov) and the Chinese Clinical Trial Registry (chictr.org.cn) will be searched from 1 January 2002. Two independent reviewers will apply eligibility criteria to identify trials for inclusion. Included trials will be phase 3 or 4 trials of SGLT2 inhibitors, GLP1 receptor analogues or DPP4 inhibitors, with placebo or active comparators, in participants with type 2 diabetes, with at least one of glycaemic control, change in body weight or major adverse cardiovascular event as outcomes. Unregistered trials will be excluded.We have identified a target population from the population-based Scottish diabetes register. The chosen cohort comprises people in Scotland with type 2 diabetes who either (1) require further treatment due to poor glycaemic control where any of the three drug classes may be suitable, or (2) who have adequate glycaemic control but are already on one of the three drug classes of interest or insulin.

ETHICS AND DISSEMINATION

Ethical approval for IPD use was obtained from the University of Glasgow MVLS College Ethics Committee (Project: 200160070). The Scottish diabetes register has approval from the Scottish A Research Ethics Committee (11/AL/0225) and operates with Public Benefit and Privacy Panel for Health and Social Care approval (1617-0147).

PROSPERO REGISTRATION NUMBER

CRD42020184174.

Two-stage matching-adjusted indirect comparison

BACKGROUND

Anchored covariate-adjusted indirect comparisons inform reimbursement decisions where there are no head-to-head trials between the treatments of interest, there is a common comparator arm shared by the studies, and there are patient-level data limitations. Matching-adjusted indirect comparison (MAIC), based on propensity score weighting, is the most widely used covariate-adjusted indirect comparison method in health technology assessment. MAIC has poor precision and is inefficient when the effective sample size after weighting is small.

METHODS

A modular extension to MAIC, termed two-stage matching-adjusted indirect comparison (2SMAIC), is proposed. This uses two parametric models. One estimates the treatment assignment mechanism in the study with individual patient data (IPD), the other estimates the trial assignment mechanism. The first model produces inverse probability weights that are combined with the odds weights produced by the second model. The resulting weights seek to balance covariates between treatment arms and across studies. A simulation study provides proof-of-principle in an indirect comparison performed across two randomized trials. Nevertheless, 2SMAIC can be applied in situations where the IPD trial is observational, by including potential confounders in the treatment assignment model. The simulation study also explores the use of weight truncation in combination with MAIC for the first time.

RESULTS

Despite enforcing randomization and knowing the true treatment assignment mechanism in the IPD trial, 2SMAIC yields improved precision and efficiency with respect to MAIC in all scenarios, while maintaining similarly low levels of bias. The two-stage approach is effective when sample sizes in the IPD trial are low, as it controls for chance imbalances in prognostic baseline covariates between study arms. It is not as effective when overlap between the trials' target populations is poor and the extremity of the weights is high. In these scenarios, truncation leads to substantial precision and efficiency gains but induces considerable bias. The combination of a two-stage approach with truncation produces the highest precision and efficiency improvements.

CONCLUSIONS

Two-stage approaches to MAIC can increase precision and efficiency with respect to the standard approach by adjusting for empirical imbalances in prognostic covariates in the IPD trial. Further modules could be incorporated for additional variance reduction or to account for missingness and non-compliance in the IPD trial.

Theory and practice of Bayesian and frequentist frameworks for network meta-analysis

Network meta-analysis (NMA) is an increasingly popular statistical method of synthesising evidence to assess the comparative benefits and harms of multiple treatments in a single analysis. Several automated software packages facilitate conducting NMA using either of two alternative approaches, Bayesian or frequentist frameworks. Researchers must choose a framework for conducting NMA (Bayesian or frequentist) and select appropriate model(s), and those conducting NMA need to understand the assumptions and limitations of different approaches. Bayesian models are more frequently used and can be more flexible but require checking additional assumptions and greater statistical expertise that are often ignored. The present paper describes the important theoretical aspects of Bayesian and frequentist models for NMA and the applications and considerations of contrast-synthesis and arm-synthesis NMAs. In addition, we present evidence from a limited number of simulation and empirical studies that compared different frequentist and Bayesian models and provide an overview of available automated software packages to perform NMA. We will conclude that when analysts choose appropriate models, there are seldom important differences in the results of Bayesian and frequentist approaches and that network meta-analysts should therefore focus on model features rather than the statistical framework.

Indirect treatment comparison of anifrolumab efficacy versus belimumab in adults with systemic lupus erythematosus

Aim: Assess the comparative efficacy of anifrolumab 300 mg versus belimumab 10 mg/kg in adults with moderate-to-severe systemic lupus erythematosus (SLE) receiving standard therapy. Patients and methods: Population-adjusted simulated treatment comparisons (primary analyses) and matching-adjusted indirect comparisons (supporting analyses) were conducted using individual patient data from TULIP-1/TULIP-2 and summary-level data from BLISS-52/BLISS-76. Results: Compared with belimumab-treated patients, anifrolumab-treated patients were more than twice as likely to achieve a reduction of four or more points in SLE Disease Activity Index 2000 score (simulated treatment comparison odds ratio: 2.47; 95% CI: 1.16-5.25) and SLE Responder Index-4 response (odds ratio: 2.61; 95% CI: 1.22-5.58) at 52 weeks. Conclusion: Patients with moderate-to-severe SLE are more likely to achieve an improvement in disease activity with anifrolumab than with belimumab.

A comparison of methods for enriching network meta-analyses in the absence of individual patient data

During drug development, a biomarker is sometimes identified as separating a patient population into those with more and those with less benefit from evaluated treatments. Consequently, later studies might be targeted, while earlier ones are performed in mixed patient populations. This poses a challenge in evidence synthesis, especially if only aggregated data are available. Starting from this scenario, we investigate three commonly used network meta-analytic estimation methods, the naive estimation approach, the stand-alone analysis, and the network meta-regression. Additionally, we adapt and modify two methods which are used in evidence synthesis to combine randomized controlled trials with observational studies, the enrichment-through-weighting approach and the informative prior estimation. We evaluate all five methods in a simulation study with 32 scenarios using bias, RMSE, coverage, precision, and power. Additionally, we re-visit a clinical data set to exemplify and discuss the application. In the simulation study, none of the methods was observed to be clearly favorable over all investigated scenarios. However, the stand-alone analysis and the naive estimation performed comparably or worse than the other methods in all evaluated performance measures and simulation scenarios and are therefore not recommended. While substantial between-trial heterogeneity is challenging for all estimation approaches, the performance of the network meta-regression, the enriching-through weighting approach and the informative prior approach was dependent on the simulation scenario and the performance measure of interest. Furthermore, as these estimation methods are drawing slightly different assumptions, some of which require the presence of additional information for estimation, we recommend sensitivity-analyses wherever possible. This article is protected by copyright. All rights reserved.

An anchored simulated treatment comparison of uptitration of amlodipine compared with a low-dose combination treatment with amlodipine 5 mg/bisoprolol 5 mg for patients with hypertension suboptimally controlled by amlodipine 5 mg monotherapy

OBJECTIVE

To compare changes in systolic and diastolic blood pressures (SBP, DBP) from baseline to following 8 weeks of treatment with a low dose combination of amlodipine 5 mg plus bisoprolol 5 mg versus up titration to the maximum daily dose of amlodipine 10 mg, in hypertensive patients uncontrolled by amlodipine 5 mg.

METHODS

Individual patient data (IPD) from a randomized clinical trial (RCT) comparing the combination versus amlodipine 5 mg (EudraCT Number: 2019-000751-13) and aggregated data (AgD) from a published RCT comparing amlodipine 10 mg versus amlodipine 5 mg were utilized in an anchored simulated treatment comparison (STC). The RCT with IPD was used to create models assessing how patients might respond to the combination if they were more comparable to those patients in the RCT with AgD. A population-adjusted indirect comparison of the treatment strategies was then conducted, using amlodipine 5 mg as an anchor.

RESULTS

In the efficacy analyses, a total of 261 patients were included in the amlodipine 10 mg arm of the RCT with AgD; and a total of 178 patients in the low-dose combination arm of the RCT with IPD. Respectively, in the Amlodipine 10 mg arm and in the low-dose combination arm, the mean age was 54.3 years-old (Standard deviation [SD] 10.6), and 57.1 years-old (13.7); 8.7% and 18.8% of patients were diabetics; and the mean baseline SBP/DBP was 149.3 (12.0)/96.5 (4.7) mmHg, and 148.8 (8.2)/90.2 (7.6) mmHg. The final model for SBP and DBP included the following variables: baseline SBP, baseline DBP, duration of hypertension, age, concomitant diabetes, sex, smoking history (final model for SBP only), and body mass index (final model for DBP only). Mean treatment differences (standard error [SE]) at 8 weeks between the combination and uptitration were -1.6 mmHg (1.9) for SBP; and -3.3 mmHg (1.3) for DBP.

CONCLUSION

In this indirect comparison, a more important decrease was observed in DBP with the low-dose combination as compared to the alternative therapeutic approach of up-titration from amlodipine 5 mg to amlodipine 10 mg. No meaningful difference was seen for SBP.

Effectiveness of crizotinib versus entrectinib in ROS1-positive non-small-cell lung cancer using clinical and real-world data

Aims: To compare clinical trial results for crizotinib and entrectinib in ROS1-positive non-small-cell lung cancer and compare clinical trial data and real-world outcomes for crizotinib. Patients & methods: We analyzed four phase I-II studies using a simulated treatment comparison (STC). A STC of clinical trial versus real-world evidence compared crizotinib clinical data to real-world outcomes. Results: Adjusted STC found nonsignificant trends favoring crizotinib over entrectinib: objective response rate, risk ratio = 1.04 (95% CI: 0.85-1.28); median duration of response, mean difference = 16.11 months (95% CI: -1.57- 33.69); median progression-free survival, mean difference = 3.99 months (95% CI: -6.27-14.25); 12-month overall survival, risk ratio = 1.01 (95% CI: 0.90-1.12). Nonsignificant differences were observed between the trial end point values and the real-world evidence for crizotinib. Conclusions: Crizotinib and entrectinib have comparable efficacy in ROS1-positive non-small-cell lung cancer.

Matching-Adjusted Indirect Comparison of Crisaborole Ointment 2% vs. Topical Calcineurin Inhibitors in the Treatment of Patients with Mild-to-Moderate Atopic Dermatitis

Introduction

Crisaborole topical ointment, 2%, is a nonsteroidal, topical anti-inflammatory phosphodiesterase-4 (PDE4) inhibitor that is approved for the treatment of mild-to-moderate atopic dermatitis (AD). The objective of the current analysis was to compare the efficacy of crisaborole 2% relative to pimecrolimus 1%, tacrolimus 0.03% and tacrolimus 0.1% in patients aged ≥ 2 years with mild-to-moderate AD by comparing improvement in Investigator’s Static Global Assessment scores ( (ISGA scores of 0/1 indicating “clear or almost clear”). ISGA was selected as the primary efficacy outcome given the US Food and Drug Administration’s recommendations on the use of ISGA for assessment of global severity in AD and to align with efficacy measurements in the crisaborole registration trials. Safety endpoints could not be analyzed due to differences in outcome definitions across studies.

Methods

Efficacy of crisaborole was evaluated using individual patient data (IPD) from two pivotal phase III randomized controlled trials (RCTs), and efficacy of comparators was evaluated using published RCTs included in a previous network meta-analysis. Vehicle controls were not comparable due to differences in ingredients and population imbalance and, therefore, an unanchored matching-adjusted indirect comparison (MAIC) was used, which reweighted IPD for crisaborole to estimate absolute response in comparator populations.

Results

The odds of achieving an improvement in ISGA score was higher with crisaborole 2% versus pimecrolimus 1% (odds ratio [OR] 2.03; 95% confidence interval [CI] 1.45–2.85; effective sample size =  627, reduced from 1021; p value < 0.001) and for crisaborole 2% versus tacrolimus 0.03% (OR 1.50; 95% CI 1.09–2.05; effective sample size = 311, reduced from 1021; p = 0.012).

Conclusion

The unanchored MAIC suggests that the odds of achieving an improvement in ISGA score is greater with crisaborole 2% than with pimecrolimus 1% or tacrolimus 0.03% in patients aged ≥ 2 years with mild-to-moderate AD. These results are consistent with findings from the previously published network meta-analysis, which used a different methodology for performing indirect treatment comparisons.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13555-021-00646-1.

BRIDGING RANDOMIZED CONTROLLED TRIALS AND SINGLE-ARM TRIALS USING COMMENSURATE PRIORS IN ARM-BASED NETWORK META-ANALYSIS

Network meta-analysis (NMA) is a powerful tool to compare multiple treatments directly and indirectly by combining and contrasting multiple independent clinical trials. Because many NMAs collect only a few eligible randomized controlled trials (RCTs), there is an urgent need to synthesize different sources of information, e.g., from both RCTs and single-arm trials. However, single-arm trials and RCTs may have different populations and quality, so that assuming they are exchangeable may be inappropriate. This article presents a novel method using a commensurate prior on variance (CPV) to borrow variance (rather than mean) information from single-arm trials in an arm-based (AB) Bayesian NMA. We illustrate the advantages of this CPV method by reanalyzing an NMA of immune checkpoint inhibitors in cancer patients. Comprehensive simulations investigate the impact on statistical inference of including single-arm trials. The simulation results show that the CPV method provides efficient and robust estimation even when the two sources of information are moderately inconsistent.

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.

Conflating marginal and conditional treatment effects: Comments on "Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study"

In this commentary, we highlight the importance of: (1) carefully considering and clarifying whether a marginal or conditional treatment effect is of interest in a population-adjusted indirect treatment comparison; and (2) developing distinct methodologies for estimating the different measures of effect. The appropriateness of each methodology depends on the preferred target of inference.