1
|
Tanaka S, Igarashi A, De Moor R, Li N, Hirozane M, Hong LW, Wu DBC, Yu DY, Hashim M, Hutton B, Tantakoun K, Olsen C, Mirzayeh Fashami F, Samjoo IA, Cameron C. A Targeted Review of Worldwide Indirect Treatment Comparison Guidelines and Best Practices. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024:S1098-3015(24)02402-1. [PMID: 38843980 DOI: 10.1016/j.jval.2024.05.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 05/10/2024] [Accepted: 05/21/2024] [Indexed: 07/07/2024]
Abstract
OBJECTIVES Controls and governance over the methodology and reporting of indirect treatment comparisons (ITCs) have been introduced to minimize bias and ensure scientific credibility and transparency in healthcare decision making. The objective of this study was to highlight ITC techniques that are key to conducting objective and analytically sound analyses and to ascertain circumstantial suitability of ITCs as a source of comparative evidence for healthcare interventions. METHODS Ovid MEDLINE was searched from January 2010 through August 2023 to identify publicly available ITC-related documents (ie, guidelines and best practices) in the English language. This was supplemented with hand searches of websites of various international organizations, regulatory agencies, and reimbursement agencies of Europe, North America, and Asia-Pacific. The jurisdiction-specific ITC methodology and reporting recommendations were reviewed. RESULTS Sixty-eight guidelines from 10 authorities worldwide were included for synthesis. Many of the included guidelines were updated within the last 5 years and commonly cited the absence of direct comparative studies as primary justification for using ITCs. Most jurisdictions favored population-adjusted or anchored ITC techniques opposed to naive comparisons. Recommendations on the reporting and presentation of these ITCs varied across authorities; however, there was some overlap among the key elements. CONCLUSIONS Given the challenges of conducting head-to-head randomized controlled trials, comparative data from ITCs offer valuable insights into clinical-effectiveness. As such, multiple ITC guidelines have emerged worldwide. According to the most recent versions of the guidelines, the suitability and subsequent acceptability of the ITC technique used depends on the data sources, available evidence, and magnitude of benefit/uncertainty.
Collapse
Affiliation(s)
- Shiro Tanaka
- Faculty of medicine, Kyoto University, Kyoto, Japan
| | - Ataru Igarashi
- Unit of Public Health and Preventive Medicine, Yokohama City University School of Medicine, Yokohama, Japan
| | - Raf De Moor
- Value, Evidence and Access Department, IMAT, Janssen Pharmaceutical K.K., Tokyo, Japan
| | - Nan Li
- Value, Evidence and Access Department, IMAT, Janssen Pharmaceutical K.K., Tokyo, Japan
| | - Mariko Hirozane
- Policy Department, IMAT, Janssen Pharmaceutical K.K., Tokyo, Japan
| | - Li Wen Hong
- Asia Pacific Regional Market Access, Janssen Pharmaceutical Companies of Johnson and Johnson, Singapore
| | - David Bin-Chia Wu
- Asia Pacific Regional Market Access, Janssen Pharmaceutical Companies of Johnson and Johnson, Singapore; Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Dae Young Yu
- Asia Pacific Regional Market Access, Janssen Pharmaceutical Companies of Johnson and Johnson, Singapore
| | - Mahmoud Hashim
- Janssen Vaccines and Prevention B.V., Leiden, The Netherlands
| | - Brian Hutton
- Ottawa Hospital Research Institute, Ottawa, ON, Canada
| | | | | | | | | | - Chris Cameron
- Value and Evidence, EVERSANA, Burlington, ON, Canada.
| |
Collapse
|
2
|
Park JE, Campbell H, Towle K, Yuan Y, Jansen JP, Phillippo D, Cope S. Unanchored Population-Adjusted Indirect Comparison Methods for Time-to-Event Outcomes Using Inverse Odds Weighting, Regression Adjustment, and Doubly Robust Methods With Either Individual Patient or Aggregate Data. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:278-286. [PMID: 38135212 DOI: 10.1016/j.jval.2023.11.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 10/18/2023] [Accepted: 11/15/2023] [Indexed: 12/24/2023]
Abstract
OBJECTIVES Several methods for unanchored population-adjusted indirect comparisons (PAICs) are available. Exploring alternative adjustment methods, depending on the available individual patient data (IPD) and the aggregate data (AD) in the external study, may help minimize bias in unanchored indirect comparisons. However, methods for time-to-event outcomes are not well understood. This study provides an overview and comparison of methods using a case study to increase familiarity. A recent method is applied to marginalize conditional hazard ratios, which allows for the comparisons of methods, and a doubly robust method is proposed. METHODS The following PAIC methods were compared through a case study in third-line small cell lung cancer, comparing nivolumab with standard of care based on a single-arm phase II trial (CheckMate 032) and real-world study (Flatiron) in terms of overall survival: IPD-IPD analyses using inverse odds weighting, regression adjustment, and a doubly robust method; IPD-AD analyses using matching-adjusted indirect comparison, simulated treatment comparison, and a doubly robust method. RESULTS Nivolumab extended survival versus standard of care with hazard ratios ranging from 0.63 (95% CI 0.44-0.90) in naive comparisons (identical estimates for IPD-IPD and IPD-AD analyses) to 0.69 (95% CI 0.44-0.98) in the IPD-IPD analyses using regression adjustment. Regression-based and doubly robust estimates yielded slightly wider confidence intervals versus the propensity score-based analyses. CONCLUSIONS The proposed doubly robust approach for time-to-event outcomes may help to minimize bias due to model misspecification. However, all methods for unanchored PAIC rely on the strong assumption that all prognostic covariates have been included.
Collapse
Affiliation(s)
- Julie E Park
- PRECISIONheor, Evidence Synthesis and Decision Modeling, Vancouver, BC, Canada
| | - Harlan Campbell
- PRECISIONheor, Evidence Synthesis and Decision Modeling, Vancouver, BC, Canada; University of British Columbia, Vancouver, BC, Canada
| | - Kevin Towle
- PRECISIONheor, Evidence Synthesis and Decision Modeling, Vancouver, BC, Canada
| | - Yong Yuan
- Worldwide Health Economics and Outcomes Research, Bristol Myers Squibb, Princeton, NJ, USA
| | - Jeroen P Jansen
- PRECISIONheor, Evidence Synthesis and Decision Modeling, Vancouver, BC, Canada
| | - David Phillippo
- University of Bristol, Bristol Medical School, Bristol, England, UK
| | - Shannon Cope
- PRECISIONheor, Evidence Synthesis and Decision Modeling, Vancouver, BC, Canada.
| |
Collapse
|
3
|
Remiro-Azócar A, Heath A, Baio G. Model-based standardization using multiple imputation. BMC Med Res Methodol 2024; 24:32. [PMID: 38341552 PMCID: PMC10858574 DOI: 10.1186/s12874-024-02157-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Accepted: 01/19/2024] [Indexed: 02/12/2024] Open
Abstract
BACKGROUND When studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect. METHODS The standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence. RESULTS We conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization. CONCLUSION We demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.
Collapse
Affiliation(s)
| | - Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, 686 Bay Street, Toronto, Canada
- Dalla Lana School of Public Health, University of Toronto, 115 College Street, Toronto, Canada
- Department of Statistical Science, University College London, 1-19 Torrington Place, London, UK
| | - Gianluca Baio
- Department of Statistical Science, University College London, 1-19 Torrington Place, London, UK
| |
Collapse
|
4
|
Truong B, Tran LAT, Le TA, Pham TT, Vo TT. Population adjusted-indirect comparisons in health technology assessment: A methodological systematic review. Res Synth Methods 2023; 14:660-670. [PMID: 37400080 DOI: 10.1002/jrsm.1653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2022] [Revised: 06/13/2023] [Accepted: 06/20/2023] [Indexed: 07/05/2023]
Abstract
In health technology assessment (HTA), population-adjusted indirect comparisons (PAICs) are increasingly considered to adjust for the difference in the target population between studies. We aim to assess the conduct and reporting of PAICs in recent HTA practice, by performing, a methodological systematic review of studies implementing PAICs from PubMed, EMBASE Classic, Embase/Ovid Medline All, and Cochrane databases from January 1, 2010 to Feb 13, 2023. Four independent researchers screened the titles, abstracts, and full-texts of the identified records, then extracted data on methodological and reporting characteristics of 106 eligible articles. Most PAIC analyses (96.9%, n = 157) were conducted by (or received funding from) pharmaceutical companies. Prior to adjustment, 44.5% of analyses (n = 72) (partially) aligned the eligibility criteria of different studies to enhance the similarity of their target populations. In 37.0% of analyses (n = 60), the clinical and methodological heterogeneity across studies were extensively assessed. In 9.3% of analyses (n = 15), the quality (or bias) of individual studies was evaluated. Among 18 analyses using methods that required an outcome model specification, results of the model fitting procedure were adequately reported in three analyses (16.7%). These findings suggest that the conduct and reporting of PAICs are remarkably heterogeneous and suboptimal in current practice. More recommendations and guidelines on PAICs are thus warranted to enhance the quality of these analyses in the future.
Collapse
Affiliation(s)
- Bang Truong
- Faculty of Pharmacy, HUTECH University, Ho Chi Minh City, Vietnam
- Department of Health Outcomes Research and Policy, Auburn University Harrison College of Pharmacy, Auburn, Alabama, USA
| | - Lan-Anh T Tran
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium
| | - Tuan Anh Le
- Department of Biology, KU Leuven, Leuven, Belgium
| | - Thi Thu Pham
- Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Tat-Thang Vo
- Department of Statistics and Data Science, The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|