1
|
Si L, Willis MS, Asseburg C, Nilsson A, Tew M, Clarke PM, Lamotte M, Ramos M, Shao H, Shi L, Zhang P, McEwan P, Ye W, Herman WH, Kuo S, Isaman DJ, Schramm W, Sailer F, Brennan A, Pollard D, Smolen HJ, Leal J, Gray A, Patel R, Feenstra T, Palmer AJ. Evaluating the Ability of Economic Models of Diabetes to Simulate New Cardiovascular Outcomes Trials: A Report on the Ninth Mount Hood Diabetes Challenge. Value Health 2020; 23:1163-1170. [PMID: 32940234 DOI: 10.1016/j.jval.2020.04.1832] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/28/2019] [Revised: 03/29/2020] [Accepted: 04/06/2020] [Indexed: 05/27/2023]
Abstract
OBJECTIVES The cardiovascular outcomes challenge examined the predictive accuracy of 10 diabetes models in estimating hard outcomes in 2 recent cardiovascular outcomes trials (CVOTs) and whether recalibration can be used to improve replication. METHODS Participating groups were asked to reproduce the results of the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME) and the Canagliflozin Cardiovascular Assessment Study (CANVAS) Program. Calibration was performed and additional analyses assessed model ability to replicate absolute event rates, hazard ratios (HRs), and the generalizability of calibration across CVOTs within a drug class. RESULTS Ten groups submitted results. Models underestimated treatment effects (ie, HRs) using uncalibrated models for both trials. Calibration to the placebo arm of EMPA-REG OUTCOME greatly improved the prediction of event rates in the placebo, but less so in the active comparator arm. Calibrating to both arms of EMPA-REG OUTCOME individually enabled replication of the observed outcomes. Using EMPA-REG OUTCOME-calibrated models to predict CANVAS Program outcomes was an improvement over uncalibrated models but failed to capture treatment effects adequately. Applying canagliflozin HRs directly provided the best fit. CONCLUSIONS The Ninth Mount Hood Diabetes Challenge demonstrated that commonly used risk equations were generally unable to capture recent CVOT treatment effects but that calibration of the risk equations can improve predictive accuracy. Although calibration serves as a practical approach to improve predictive accuracy for CVOT outcomes, it does not extrapolate generally to other settings, time horizons, and comparators. New methods and/or new risk equations for capturing these CV benefits are needed.
Collapse
Affiliation(s)
- Lei Si
- The George Institute for Global Health, UNSW Sydney, Kensington, Australia; Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia
| | | | | | | | - Michelle Tew
- Centre for Health Policy, School of Population and Global Health, The University of Melbourne, Victoria, Australia
| | - Philip M Clarke
- Centre for Health Policy, School of Population and Global Health, The University of Melbourne, Victoria, Australia; Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, United Kingdom
| | - Mark Lamotte
- Global Health Economics and Outcomes Research, IQVIA, Zaventem, Belgium
| | - Mafalda Ramos
- Global Health Economics and Outcomes Research, IQVIA, Lisbon, Portugal
| | - Hui Shao
- Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, Florida
| | - Lizheng Shi
- Department of Global Health Management and Policy, School of Public Health and Tropical Medicine, Tulane University, New Orleans, Louisiana, USA
| | - Ping Zhang
- Division of Diabetes Translation, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
| | - Phil McEwan
- Health Economics and Outcomes Research Ltd, Cardiff, United Kingdom
| | - Wen Ye
- Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA
| | - William H Herman
- Departments of Internal Medicine and Epidemiology, University of Michigan, Ann Arbor, Michigan, USA
| | - Shihchen Kuo
- Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan, USA
| | - Deanna J Isaman
- Department of Biostatistics, University of Michigan, Ann Arbor, Michigan, USA
| | - Wendelin Schramm
- Centre for Health Economics and Outcomes Research, GECKO Institute, Heilbronn University, Heilbronn, Germany
| | - Fabian Sailer
- Centre for Health Economics and Outcomes Research, GECKO Institute, Heilbronn University, Heilbronn, Germany
| | - Alan Brennan
- School of Health and Related Research, University of Sheffield, Sheffield, United Kingdom
| | - Daniel Pollard
- School of Health and Related Research, University of Sheffield, Sheffield, United Kingdom
| | - Harry J Smolen
- Medical Decision Modeling Inc., Indianapolis, Indiana, USA
| | - José Leal
- National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands
| | - Alastair Gray
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, United Kingdom
| | - Rishi Patel
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, United Kingdom
| | - Talitha Feenstra
- National Institute for Public Health and the Environment (RIVM), Bilthoven, The Netherlands; University of Groningen, Faculty of Science and Engineering, Groningen, The Netherlands
| | - Andrew J Palmer
- Menzies Institute for Medical Research, University of Tasmania, Hobart, Tasmania, Australia; Centre for Health Policy, School of Population and Global Health, The University of Melbourne, Victoria, Australia.
| |
Collapse
|
2
|
Abstract
PURPOSE OF REVIEW This paper provides an overview of type 2 diabetes economic simulation modeling and reviews current topics of discussion and major challenges in the field. RECENT FINDINGS Important challenges in the field include increasing the generalizability of models and improving transparency in model reporting. To identify and address these issues, modeling groups have organized through the Mount Hood Diabetes Challenge meetings and developed tools (i.e., checklist, impact inventory) to standardize modeling methods and reporting of results. Accordingly, many newer diabetes models have begun utilizing these tools, allowing for improved comparability between diabetes models. In the last two decades, type 2 diabetes simulation models have improved considerably, due to the collaborative work performed through the Mount Hood Diabetes Challenge meetings. To continue to improve diabetes models, future work must focus on clarifying diabetes progression in racial/ethnic minorities and incorporating equity considerations into health economic analysis.
Collapse
Affiliation(s)
- Rahul S Dadwani
- Pritzker School of Medicine, University of Chicago, Chicago, IL, USA
| | - Neda Laiteerapong
- Section of General Internal Medicine, University of Chicago, 5841 South Maryland Ave, Chicago, IL, 60637, USA.
| |
Collapse
|