Methods for Indirect Treatment Comparison: Results from a Systematic Literature Review

INTRODUCTION

Health technology assessment (HTA) agencies express a clear preference for randomized controlled trials when assessing the comparative efficacy of two or more treatments. However, an indirect treatment comparison (ITC) is often necessary where a direct comparison is unavailable or, in some cases, not possible. Numerous ITC techniques are described in the literature. A systematic literature review (SLR) was conducted to identify all the relevant literature on existing ITC techniques, provide a comprehensive description of each technique and evaluate their strengths and limitations from an HTA perspective in order to develop guidance on the most appropriate method to use in different scenarios.

METHODS

Electronic database searches of Embase and PubMed, as well as grey literature searches, were conducted on 15 November 2021. Eligible articles were peer-reviewed papers that specifically described the methods used for different ITC techniques and were written in English. The review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

RESULTS

A total of 73 articles were included in the SLR, reporting on seven different ITC techniques. All reported techniques were forms of adjusted ITC. Network meta-analysis (NMA) was the most frequently described technique (in 79.5% of the included articles), followed by matching-adjusted indirect comparison (MAIC) (30.1%), network meta-regression (24.7%), the Bucher method (23.3%), simulated treatment comparison (STC) (21.9%), propensity score matching (4.1%) and inverse probability of treatment weighting (4.1%). The appropriate choice of ITC technique is critical and should be based on the feasibility of a connected network, the evidence of heterogeneity between and within studies, the overall number of relevant studies and the availability of individual patient-level data (IPD). MAIC and STC were found to be common techniques in the case of single-arm studies, which are increasingly being conducted in oncology and rare diseases, whilst the Bucher method and NMA provide suitable options where no IPD is available.

CONCLUSION

ITCs can provide alternative evidence where direct comparative evidence may be missing. ITCs are currently considered by HTA agencies on a case-by-case basis; however, their acceptability remains low. Clearer international consensus and guidance on the methods to use for different ITC techniques is needed to improve the quality of ITCs submitted to HTA agencies. ITC techniques continue to evolve quickly, and more efficient techniques may become available in the future.

Comparison of Methods for Estimating Therapy Effects by Indirect Comparisons: A Simulation Study

Objective. In evidence synthesis, therapeutic options have to be compared despite the lack of head-to-head trials. Indirect comparisons are then widely used, although little is known about their performance in situations where cross-trial differences or effect modification are present. Methods. We contrast the matching adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and the method of Bucher using a simulation study. The different methods are evaluated according to their power and type I error rate as well as with respect to the coverage, bias, and the root mean squared error (RMSE) of the effect estimate for practically relevant scenarios using binary and time-to-event endpoints. In addition, we investigate how the power planned for the head-to-head trials influences the actual power of the indirect comparison. Results. Indirect comparisons are considerably underpowered. None of the methods had substantially superior performance. In situations without cross-trial differences and effect modification, MAIC and Bucher led to similar results, while Bucher has the advantage of preserving the within-study randomization. MAIC and STC could enhance power in some scenarios but at the cost of a potential type I error inflation. Adjusting MAIC and STC for confounders that did not modify the effect led to higher bias and RMSE. Conclusion. The choice of effect modifiers in MAIC and STC influences the precision of the indirect comparison. Therefore, a careful selection of effect modifiers is warranted. In addition, missed differences between trials may lead to low power and partly high bias for all considered methods, and thus, results of indirect comparisons should be interpreted with caution.

The importance of considering differences in study and patient characteristics before undertaking indirect treatment comparisons: a case study of siponimod for secondary progressive multiple sclerosis

Background: Indirect treatment comparisons (ITCs) provide valuable evidence on comparative efficacy where head-to-head clinical trials do not exist; however, differences in patient populations may introduce bias. Therefore, it is essential to assess between-trial heterogeneity to determine the suitability of synthesizing ITC results. We provide an illustrative case study in multiple sclerosis (MS) where we assess the feasibility of conducting ITCs between siponimod and interferon beta-1b (IFN β-1b) and between siponimod and ocrelizumab.Methods: We assessed the feasibility of conducting ITCs using standard unadjusted methods (e.g. Bucher or network meta-analysis [NMA]) as well as matching-adjusted indirect comparisons (MAICs) using individual patient data (IPD) from the siponimod (EXPAND) trial, based on guidance from NICE. Time to confirmed disability progression (CDP) at 3 or 6 months was assessed.Results: Bucher ITCs and NMAs, which rely on summary-level data, were not able to account for important cross-trial differences. Comparisons between siponimod and IFN β-1b were feasible using MAIC; the HRs (95% CI) for CDP-6 and CDP-3 were 0.55 (0.33-0.91) and 0.82 (0.42-1.63), respectively. ITCs were not feasible between siponimod and ocrelizumab because study designs and patient populations were too dissimilar to conduct a reliable ITC.Conclusions: This study highlights the importance of conducting a detailed feasibility assessment before undertaking ITCs to illuminate when excessive between-trial heterogeneity would cause biased results. MAIC was performed for siponimod and IFN β-1b in the absence of a head-to-head trial and was considered a more valid approach than a traditional ITC to examine comparative effectiveness.