1
|
Pouwels XGLV, Kroeze K, van der Linden N, Kip MMA, Koffijberg H. Validating Health Economic Models With the Probabilistic Analysis Check dashBOARD. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:1073-1084. [PMID: 38641056 DOI: 10.1016/j.jval.2024.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 03/21/2024] [Accepted: 04/09/2024] [Indexed: 04/21/2024]
Abstract
OBJECTIVES Health economic (HE) models are often considered as "black boxes" because they are not publicly available and lack transparency, which prevents independent scrutiny of HE models. Additionally, validation efforts and validation status of HE models are not systematically reported. Methods to validate HE models in absence of their full underlying code are therefore urgently needed to improve health policy making. This study aimed to develop and test a generic dashboard to systematically explore the workings of HE models and validate their model parameters and outcomes. METHODS The Probabilistic Analysis Check dashBOARD (PACBOARD) was developed using insights from literature, health economists, and a data scientist. Functionalities of PACBOARD are (1) exploring and validating model parameters and outcomes using standardized validation tests and interactive plots, (2) visualizing and investigating the relationship between model parameters and outcomes using metamodeling, and (3) predicting HE outcomes using the fitted metamodel. To test PACBOARD, 2 mock HE models were developed, and errors were introduced in these models, eg, negative costs inputs, utility values exceeding 1. PACBOARD metamodeling predictions of incremental net monetary benefit were validated against the original model's outcomes. RESULTS PACBOARD automatically identified all errors introduced in the erroneous HE models. Metamodel predictions were accurate compared with the original model outcomes. CONCLUSIONS PACBOARD is a unique dashboard aiming at improving the feasibility and transparency of validation efforts of HE models. PACBOARD allows users to explore the working of HE models using metamodeling based on HE models' parameters and outcomes.
Collapse
Affiliation(s)
- Xavier G L V Pouwels
- Section of Health Technology and Services Research, Technical Medical Centre, Faculty of Behavioural, Management, and Social Sciences, University of Twente, Enschede, Overijssel, The Netherlands.
| | - Karel Kroeze
- Behavioural Data Science incubator, Faculty of Behavioural, Management, and Social Sciences, University of Twente, Enschede, Overijssel, The Netherlands
| | - Naomi van der Linden
- Section of Health Technology and Services Research, Technical Medical Centre, Faculty of Behavioural, Management, and Social Sciences, University of Twente, Enschede, Overijssel, The Netherlands; Institute for Health Systems Science, Faculty of Technology, Policy and Management, Delft University of Technology, Delft, South Holland, The Netherlands
| | - Michelle M A Kip
- Section of Health Technology and Services Research, Technical Medical Centre, Faculty of Behavioural, Management, and Social Sciences, University of Twente, Enschede, Overijssel, The Netherlands
| | - Hendrik Koffijberg
- Section of Health Technology and Services Research, Technical Medical Centre, Faculty of Behavioural, Management, and Social Sciences, University of Twente, Enschede, Overijssel, The Netherlands
| |
Collapse
|
2
|
Wright DR, Batista M, Wrightson T. #SharingHEOR: Developing Modern Media for Communication and Dissemination of Health Economics and Outcomes Research. APPLIED HEALTH ECONOMICS AND HEALTH POLICY 2024; 22:447-455. [PMID: 38427216 DOI: 10.1007/s40258-023-00863-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/27/2023] [Indexed: 03/02/2024]
Abstract
Questions regarding the effectiveness and safety of health interventions and allocation of health care resources are frequently discussed in mainstream and social media. Additionally, government and foundation funders are increasingly mandating that results be disseminated to the lay public and patients may benefit from being able to digest scientific research regarding their health conditions. Therefore, it is important to widely disseminate and clearly communication health economics and outcomes research (HEOR) findings to a range of interested parties. Digital media features such as graphical or visual abstracts, infographics and videos are informative and add value to research articles by improving reader engagement with articles, potentially increasing their impact, and allowing results to be more widely disseminated. However, use of novel digital media for research dissemination has been relatively limited to date. In this article, we discuss the rationale for developing novel media to communicate and disseminate research findings and offer practical advice for doing so. We conclude by outlining a future agenda for research regarding HEOR communication and dissemination.
Collapse
Affiliation(s)
- Davene R Wright
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401E, Boston, MA, 02215, USA.
| | - Mikaela Batista
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Drive, Suite 401E, Boston, MA, 02215, USA
| | | |
Collapse
|
3
|
Awan J, Faherty LJ, Willis HH. Navigating Uncertainty in Public Health Decisionmaking: The Role of a Value of Information Framework in Threat Agnostic Biosurveillance. Health Secur 2024; 22:39-44. [PMID: 38079227 DOI: 10.1089/hs.2023.0070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/23/2024] Open
Affiliation(s)
- Jalal Awan
- Jalal Awan, MS, MPhil, PhD, is an Energy and Climate Policy Analyst, The Utility Reform Network, Oakland, CA, and an Assistant Policy Researcher, RAND Corporation, Santa Monica, CA
| | - Laura J Faherty
- Laura J. Faherty, MD, MPH, MSHP, is a Physician Policy Researcher, RAND Corporation, and an Attending Physician, Maine Medical Center, Portland, ME
| | - Henry H Willis
- Henry H. Willis, PhD, is a Senior Policy Researcher, RAND Corporation, Pittsburgh, PA
| |
Collapse
|
4
|
Rodriguez Llorian E, Waliji LA, Dragojlovic N, Michaux KD, Nagase F, Lynd LD. Frameworks for Health Technology Assessment at an Early Stage of Product Development: A Review and Roadmap to Guide Applications. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023:S1098-3015(23)00107-9. [PMID: 36990207 DOI: 10.1016/j.jval.2023.03.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 02/18/2023] [Accepted: 03/16/2023] [Indexed: 05/20/2023]
Abstract
OBJECTIVES Early health technology assessment (eHTA) can be used to evaluate and optimize a medical product's value proposition and to inform go/no-go decisions by using health economic modeling, literature scanning, and stakeholder preference studies at an early stage of development. eHTA frameworks offer high-level guidance on conducting this complex, iterative, and multidisciplinary process. The objective of this study was to review and summarize existing eHTA frameworks, understood as systematic approaches to guide early evidence generation and decision making. METHODS Using a rapid review methodology, we identified all relevant studies published in English, French, and Spanish from PubMed/MEDLINE and Embase until February 2022. We only included frameworks relevant to the preclinical and early clinical (phase I) stages of medical product development. RESULTS From 737 reviewed abstracts, 53 publications describing 46 frameworks were selected for inclusion and classified into categories based on their scope: (1) criteria frameworks, which provide an overview of eHTA; (2) process frameworks, which offer stepwise guidance for conducting eHTA, including preferred methods; and (3) methods frameworks, which provide detailed descriptions of specific eHTA methods. Most of the frameworks did not specify their target users or the specific stage of technology development. CONCLUSIONS Despite some variability and gaps found across existing frameworks, the structure provided by this review helps inform eHTA applications. Remaining challenges are the frameworks' limited accessibility to users without a background in health economics, poor distinctions being made among early lifecycle stages and technology types, and the inconsistent terminology used to describe eHTA in different contexts.
Collapse
Affiliation(s)
- Elisabet Rodriguez Llorian
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada
| | - Louloua Ashikhusein Waliji
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada
| | - Nick Dragojlovic
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada
| | - Kristina D Michaux
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada
| | - Fernanda Nagase
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada
| | - Larry D Lynd
- Collaboration for Outcomes Research and Evaluation (CORE), Faculty of Pharmaceutical Sciences, The University of British Columbia, Vancouver, BC, Canada; Centre for Health Evaluation and Outcome Sciences (CHÉOS), St. Paul's Hospital, Vancouver, BC, Canada.
| |
Collapse
|
5
|
Schilling C, Tew M, Bunzli S, Shadbolt C, Lohmander LS, Balogh ZJ, Paolucci F, Choong PF, Dowsey MM, Clarke P. An Economic Model for Estimating Trial Costs with an Application to Placebo Surgery Trials. APPLIED HEALTH ECONOMICS AND HEALTH POLICY 2023; 21:263-273. [PMID: 36575335 PMCID: PMC9931787 DOI: 10.1007/s40258-022-00775-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/20/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVE Waste in clinical trials remains rife. We developed an economic model to predict the cost of trials based on input costs, duration, power, number of sites, recruitment eligibility and consenting rates. METHODS We parameterised the model for three proxy placebo-controlled surgical trials using data from a systematic review, a bespoke cost survey, and from the literature. We used the model to compare target and actual trial performance for (i) a trial that was completed on time but with more sites, (ii) a trial that completed after a time extension, and (iii) an incomplete trial. RESULTS Successful trials more accurately anticipated the true recruitment rate that they achieved and those that overestimated this were most likely to fail. The costs of overestimating recruitment rates were dramatic: all proxy trials had significantly higher costs than planned, with additional funding of at least AUD$600,000 (50% above budget) required for trials that completed after adding more sites or more time, and over AUD$2 million (260% above budget) for incomplete trials. CONCLUSIONS This model shows the trade-offs between time and cost, or both, when recruitment is lower than anticipated. Greater consideration is needed to improve trial planning, reviewing, and funding of these trials to avoid costly overruns and incomplete trials.
Collapse
Affiliation(s)
- Chris Schilling
- Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - Michelle Tew
- Melbourne School of Population and Global Health, Centre for Health Policy, University of Melbourne, Melbourne, Australia
| | - Samantha Bunzli
- Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - Cade Shadbolt
- Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - L. Stefan Lohmander
- Department of Clinical Sciences Lund, Orthopaedics, University of Lund, Lund, Sweden
| | - Zsolt J. Balogh
- School of Medicine and Public Health, Department of Traumatology, John Hunter Hospital, The University of Newcastle, Newcastle, Australia
| | - Francesco Paolucci
- College of Human and Social Futures, The Newcastle Business School, University of Newcastle, Newcastle, Australia
- Schools of Economics and Management, University of Bologna, Bologna, Italy
| | - Peter F. Choong
- Department of Surgery, The University of Melbourne, Melbourne, Australia
- Department of Orthopaedics, St. Vincent’s Hospital, Melbourne, Australia
| | - Michelle M. Dowsey
- Department of Surgery, The University of Melbourne, Melbourne, Australia
- Department of Orthopaedics, St. Vincent’s Hospital, Melbourne, Australia
| | - Philip Clarke
- Melbourne School of Population and Global Health, Centre for Health Policy, University of Melbourne, Melbourne, Australia
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| |
Collapse
|