1
|
Glynn D, Griffin S, Gutacker N, Walker S. Methods to Quantify the Importance of Parameters for Model Updating and Distributional Adaptation. Med Decis Making 2024:272989X241262037. [PMID: 39056289 DOI: 10.1177/0272989x241262037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2024]
Abstract
PURPOSE Decision models are time-consuming to develop; therefore, adapting previously developed models for new purposes may be advantageous. We provide methods to prioritize efforts to 1) update parameter values in existing models and 2) adapt existing models for distributional cost-effectiveness analysis (DCEA). METHODS Methods exist to assess the influence of different input parameters on the results of a decision models, including value of information (VOI) and 1-way sensitivity analysis (OWSA). We apply 1) VOI to prioritize searches for additional information to update parameter values and 2) OWSA to prioritize searches for parameters that may vary by socioeconomic characteristics. We highlight the assumptions required and propose metrics that quantify the extent to which parameters in a model have been updated or adapted. We provide R code to quickly carry out the analysis given inputs from a probabilistic sensitivity analysis (PSA) and demonstrate our methods using an oncology case study. RESULTS In our case study, updating 2 of 21 probabilistic model parameters addressed 71.5% of the total VOI and updating 3 addressed approximately 100% of the uncertainty. Our proposed approach suggests that these are the 3 parameters that should be prioritized. For model adaptation for DCEA, 46.3% of the total OWSA variation came from a single parameter, while the top 10 input parameters were found to account for more than 95% of the total variation, suggesting efforts should be aimed toward these. CONCLUSIONS These methods offer a systematic approach to guide research efforts in updating models with new data or adapting models to undertake DCEA. The case study demonstrated only very small gains from updating more than 3 parameters or adapting more than 10 parameters. HIGHLIGHTS It can require considerable analyst time to search for evidence to update a model or to adapt a model to take account of equity concerns.In this article, we provide a quantitative method to prioritze parameters to 1) update existing models to reflect potential new evidence and 2) adapt existing models to estimate distributional outcomes.We define metrics that quantify the extent to which the parameters in a model have been updated or adapted.We provide R code that can quickly rank parameter importance and calculate quality metrics using only the results of a standard probabilistic sensitivity analysis.
Collapse
Affiliation(s)
- David Glynn
- Centre for Health Economics, University of York, York, UK
| | - Susan Griffin
- Centre for Health Economics, University of York, York, UK
| | - Nils Gutacker
- Centre for Health Economics, University of York, York, UK
| | - Simon Walker
- Centre for Health Economics, University of York, York, UK
| |
Collapse
|
2
|
Dijk SW, Krijkamp E, Kunst N, Labrecque JA, Gross CP, Pandit A, Lu CP, Visser LE, Wong JB, Hunink MGM. Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information. Med Decis Making 2024; 44:512-528. [PMID: 38828516 PMCID: PMC11283736 DOI: 10.1177/0272989x241255047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 04/07/2024] [Indexed: 06/05/2024]
Abstract
BACKGROUND The COVID-19 pandemic underscored the criticality and complexity of decision making for novel treatment approval and further research. Our study aims to assess potential decision-making methodologies, an evaluation vital for refining future public health crisis responses. METHODS We compared 4 decision-making approaches to drug approval and research: the Food and Drug Administration's policy decisions, cumulative meta-analysis, a prospective value-of-information (VOI) approach (using information available at the time of decision), and a reference standard (retrospective VOI analysis using information available in hindsight). Possible decisions were to reject, accept, provide emergency use authorization, or allow access to new therapies only in research settings. We used monoclonal antibodies provided to hospitalized COVID-19 patients as a case study, examining the evidence from September 2020 to December 2021 and focusing on each method's capacity to optimize health outcomes and resource allocation. RESULTS Our findings indicate a notable discrepancy between policy decisions and the reference standard retrospective VOI approach with expected losses up to $269 billion USD, suggesting suboptimal resource use during the wait for emergency use authorization. Relying solely on cumulative meta-analysis for decision making results in the largest expected loss, while the policy approach showed a loss up to $16 billion and the prospective VOI approach presented the least loss (up to $2 billion). CONCLUSION Our research suggests that incorporating VOI analysis may be particularly useful for research prioritization and treatment implementation decisions during pandemics. While the prospective VOI approach was favored in this case study, further studies should validate the ideal decision-making method across various contexts. This study's findings not only enhance our understanding of decision-making strategies during a health crisis but also provide a potential framework for future pandemic responses. HIGHLIGHTS This study reviews discrepancies between a reference standard (retrospective VOI, using hindsight information) and 3 conceivable real-time approaches to research-treatment decisions during a pandemic, suggesting suboptimal use of resources.Of all prospective decision-making approaches considered, VOI closely mirrored the reference standard, yielding the least expected value loss across our study timeline.This study illustrates the possible benefit of VOI results and the need for evidence accumulation accompanied by modeling in health technology assessment for emerging therapies.
Collapse
Affiliation(s)
- Stijntje W. Dijk
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
- Department of Radiology and Nuclear Medicine, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Eline Krijkamp
- Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Natalia Kunst
- Centre for Health Economics, University of York, York, UK
- Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University School of Medicine, New Haven, CT, USA
| | - Jeremy A. Labrecque
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Cary P. Gross
- Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University School of Medicine, New Haven, CT, USA
| | - Aradhana Pandit
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Chia-Ping Lu
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Loes E. Visser
- Department of Hospital Pharmacy, Erasmus University Medical Center, Rotterdam, The Netherlands
- Hospital Pharmacy, Haga Teaching Hospital, The Hague, The Netherlands
| | - John B. Wong
- Division of Clinical Decision Making, Tufts Medical Center, Boston, USA
| | - M. G. Myriam Hunink
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
- Department of Radiology and Nuclear Medicine, Erasmus University Medical Center, Rotterdam, The Netherlands
- Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| |
Collapse
|
3
|
Grimm SE, Pouwels XGLV, Ramaekers BLT, Wijnen B, Grutters J, Joore MA. Response to "UNCERTAINTY MANAGEMENT IN REGULATORY AND HEALTH TECHNOLOGY ASSESSMENT DECISION-MAKING ON DRUGS: GUIDANCE OF THE HTAi-DIA WORKING GROUP". Int J Technol Assess Health Care 2023; 39:e70. [PMID: 37822085 DOI: 10.1017/s026646232300260x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/13/2023]
Affiliation(s)
- Sabine Elisabeth Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| | - Xavier G L V Pouwels
- Department of Health Technology and Services Research, University of Twente, Enschede, The Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| | - Ben Wijnen
- Trimbos-instituut, Utrecht, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| |
Collapse
|
4
|
Kunst N, Siu A, Drummond M, Grimm SE, Grutters J, Husereau D, Koffijberg H, Rothery C, Wilson ECF, Heath A. Consolidated Health Economic Evaluation Reporting Standards - Value of Information (CHEERS-VOI): Explanation and Elaboration. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023; 26:1461-1473. [PMID: 37414276 DOI: 10.1016/j.jval.2023.06.014] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 05/27/2023] [Accepted: 06/20/2023] [Indexed: 07/08/2023]
Abstract
OBJECTIVES Although the ISPOR Value of Information (VOI) Task Force's reports outline VOI concepts and provide good-practice recommendations, there is no guidance for reporting VOI analyses. VOI analyses are usually performed alongside economic evaluations for which the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Statement provides reporting guidelines. Thus, we developed the CHEERS-VOI checklist to provide reporting guidance and checklist to support the transparent, reproducible, and high-quality reporting of VOI analyses. METHODS A comprehensive literature review generated a list of 26 candidate reporting items. These candidate items underwent a Delphi procedure with Delphi participants through 3 survey rounds. Participants rated each item on a 9-point Likert scale to indicate its relevance when reporting the minimal, essential information about VOI methods and provided comments. The Delphi results were reviewed at 2-day consensus meetings and the checklist was finalized using anonymous voting. RESULTS We had 30, 25, and 24 Delphi respondents in rounds 1, 2, and 3, respectively. After incorporating revisions recommended by the Delphi participants, all 26 candidate items proceeded to the 2-day consensus meetings. The final CHEERS-VOI checklist includes all CHEERS items, but 7 items require elaboration when reporting VOI. Further, 6 new items were added to report information relevant only to VOI (eg, VOI methods applied). CONCLUSIONS The CHEERS-VOI checklist should be used when a VOI analysis is performed alongside economic evaluations. The CHEERS-VOI checklist will help decision makers, analysts and peer reviewers in the assessment and interpretation of VOI analyses and thereby increase transparency and rigor in decision making.
Collapse
Affiliation(s)
- Natalia Kunst
- Centre for Health Economics, University of York, York, England, UK; Yale University School of Public Health, New Haven, CT, USA.
| | - Annisa Siu
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Michael Drummond
- Centre for Health Economics, University of York, York, England, UK
| | - Sabine E Grimm
- Department of Epidemiology and Medical Technology Assessment (KEMTA), Maastricht Health Economics and Technology Assessment (Maastricht HETA) Center, Maastricht University Medical Center, Maastricht, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud Institute for Health Sciences, Radboudumc, Nijmegen, The Netherlands
| | - Don Husereau
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada and Institute of Health Economics, Edmonton, Alberta, Canada
| | - Hendrik Koffijberg
- Department of Health Technology & Services Research, TechMed Centre, University of Twente, Enschede, The Netherlands
| | - Claire Rothery
- Centre for Health Economics, University of York, York, England, UK
| | - Edward C F Wilson
- Peninsula Technology Assessment Group, University of Exeter, Exeter, England, UK
| | - Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada; Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada; Department of Statistical Science, University College London, London, England, UK
| |
Collapse
|
5
|
Pei PP, Fitzmaurice KP, Le MH, Panella C, Jones ML, Pandya A, Horsburgh CR, Freedberg KA, Weinstein MC, Paltiel AD, Reddy KP. The Value-of-Information and Value-of-Implementation from Clinical Trials of Diagnostic Tests for HIV-Associated Tuberculosis: A Modeling Analysis. MDM Policy Pract 2023; 8:23814683231198873. [PMID: 37743931 PMCID: PMC10517616 DOI: 10.1177/23814683231198873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Accepted: 07/27/2023] [Indexed: 09/26/2023] Open
Abstract
Objectives. Conventional value-of-information (VOI) analysis assumes complete uptake of an optimal decision. We employed an extended framework that includes value-of-implementation (VOM)-the benefit of encouraging adoption of an optimal strategy-and estimated how future trials of diagnostic tests for HIV-associated tuberculosis could improve public health decision making and clinical and economic outcomes. Methods. We evaluated the clinical outcomes and costs, given current information, of 3 tuberculosis screening strategies among hospitalized people with HIV in South Africa: sputum Xpert (Xpert), sputum Xpert plus urine AlereLAM (Xpert+AlereLAM), and sputum Xpert plus the newer, more sensitive, and costlier urine FujiLAM (Xpert+FujiLAM). We projected the incremental net monetary benefit (INMB) of decision making based on results of a trial comparing mortality with each strategy, rather than decision making based solely on current knowledge of FujiLAM's improved diagnostic performance. We used a validated microsimulation to estimate VOI (the INMB of reducing parameter uncertainty before decision making) and VOM (the INMB of encouraging adoption of an optimal strategy). Results. With current information, adopting Xpert+FujiLAM yields 0.4 additional life-years/person compared with current practices (assumed 50% Xpert and 50% Xpert+AlereLAM). While the decision to adopt this optimal strategy is unaffected by information from the clinical trial (VOI = $ 0 at $3,000/year-of-life saved willingness-to-pay threshold), there is value in scaling up implementation of Xpert+FujiLAM, which results in an INMB (representing VOM) of $650 million over 5 y. Conclusions. Conventional VOI methods account for the value of switching to a new optimal strategy based on trial data but fail to account for the persuasive value of trials in increasing uptake of the optimal strategy. Evaluation of trials should include a focus on their value in reducing barriers to implementation. Highlights In conventional VOI analysis, it is assumed that the optimal decision will always be adopted even without a trial. This can potentially lead to an underestimation of the value of trials when adoption requires new clinical trial evidence. To capture the influence that a trial may have on decision makers' willingness to adopt the optimal decision, we also consider value-of-implementation (VOM), a metric quantifying the benefit of new study information in promoting wider adoption of the optimal strategy. The overall value-of-a-trial (VOT) includes both VOI and VOM.Our model-based analysis suggests that the information obtained from a trial of screening strategies for HIV-associated tuberculosis in South Africa would have no value, when measured using traditional methods of VOI assessment. A novel strategy, which includes the urine FujiLAM test, is optimal from a health economic standpoint but is underutilized. A trial would reduce uncertainties around downstream health outcomes but likely would not change the optimal decision. The high VOT (nearly $700 million over 5 y) lies solely in promoting uptake of FujiLAM, represented as VOM.Our results highlight the importance of employing a more comprehensive approach for evaluating prospective trials, as conventional VOI methods can vastly underestimate their value. Trialists and funders can and should assess the VOT metric instead when considering trial designs and costs. If VOI is low, the VOM and cost of a trial can be compared with the benefits and costs of other outreach programs to determine the most cost-effective way to improve uptake.
Collapse
Affiliation(s)
- Pamela P. Pei
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | | | - Mylinh H. Le
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Christopher Panella
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Michelle L. Jones
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Ankur Pandya
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - C. Robert Horsburgh
- School of Public Health and School of Medicine, Boston University, Boston, MA, USA
| | - Kenneth A. Freedberg
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- Division of Infectious Diseases, Massachusetts General Hospital, Boston, MA, USA
| | - Milton C. Weinstein
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - A. David Paltiel
- Public Health Modeling Unit, Yale School of Public Health, New Haven, CT, USA
| | - Krishna P. Reddy
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- Division of Pulmonary and Critical Care Medicine, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
6
|
Hazen G, Borgonovo E, Lu X. Information Density in Decision Analysis. DECISION ANALYSIS 2023. [DOI: 10.1287/deca.2022.0465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
Information value has been proposed and used as a probabilistic sensitivity measure, the idea being that uncertain parameters having higher information value are precisely those to which an optimal decision is more sensitive. In this paper, we study the notion of information density as a graphical complement to information value analysis, one that augments an information value calculation with associated directions of information gain. We formally examine mathematical details absent from its earlier presentation that guarantee information density exists and is well posed and describe its relationship to alternate measures of information value. We present its application in the context of a realistic case study and discuss the associated insights.
Collapse
Affiliation(s)
- Gordon Hazen
- Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208
| | - Emanuele Borgonovo
- Bocconi Institute for Data Science and Analytics, 20136 Milan, Italy
- Department of Decision Sciences, Bocconi University, 20136 Milan, Italy
| | - Xuefei Lu
- SKEMA Business School, Université Côte d’Azur, Paris, France
| |
Collapse
|
7
|
Parsons J, Bao L. A Unified Approach for Outliers and Influential Data Detection - The Value of Information in Retrospect. Stat (Int Stat Inst) 2022; 11:e442. [PMID: 37908311 PMCID: PMC10617639 DOI: 10.1002/sta4.442] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 11/27/2021] [Indexed: 11/02/2023]
Abstract
Identifying influential and outlying data is important as it would guide the effective collection of future data and the proper use of existing information. We develop a unified approach for outlier detection and influence analysis. Our proposed method is grounded in the intuitive value of information concepts and has a distinct advantage in interpretability and flexibility when compared to existing methods: it decomposes the data influence into the leverage effect (expected to be influential) and the outlying effect (surprisingly more influential than being expected); and it applies to all decision problems such as estimation, prediction, and hypothesis testing. We study the theoretical properties of three value of information quantities, establish the relationship between the proposed measures and classic measures in the linear regression setting, and provide real data analysis examples of how to apply the new value of information approach in the cases of linear regression, generalized linear mixed model, and hypothesis testing.
Collapse
Affiliation(s)
- Jacob Parsons
- Department of Statistics, Penn State University, University Park, PA, U.S
| | - Le Bao
- Department of Statistics, Penn State University, University Park, PA, U.S
| |
Collapse
|
8
|
On the Reality of Signaling in Auctions. INFORMATION 2022. [DOI: 10.3390/info13110549] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Over the last two decades, auctions have become an integral part of e-commerce and a promising field for applying artificial intelligence technologies. The use of signals has been studied extensively in the existing auction literature. Specifically, it has been shown that when an external strategic entity (such as an information broker) is present, it can be beneficial to use signaling as a preliminary step before offering to sell information. However, these results apply only in cases where all auction participants are completely rational agents. However, in many real-life scenarios some of the participants are humans, and hence are easily affected by external factors, i.e., their rationality is bounded. In this paper, we offer a thorough investigation of a case in which the prospective information buyer is a human auctioneer. Using a set of MTurk-based experiments with people, we tested 10,000 independent auctions with diverse characteristics, and were able to identify a varied set of practical insights regarding human behavior. Real-life strategic information brokers could potentially use these insights to achieve a better understanding of how humans operate, paving the way for optimizing the benefit obtainable from the information they own.
Collapse
|
9
|
Drummond M, Federici C, Reckers‐Droog V, Torbica A, Blankart CR, Ciani O, Kaló Z, Kovács S, Brouwer W. Coverage with evidence development for medical devices in Europe: Can practice meet theory? HEALTH ECONOMICS 2022; 31 Suppl 1:179-194. [PMID: 35220644 PMCID: PMC9545598 DOI: 10.1002/hec.4478] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 12/26/2021] [Accepted: 01/12/2022] [Indexed: 06/14/2023]
Abstract
Health economists have written extensively on the design and implementation of coverage with evidence development (CED) schemes and have proposed theoretical frameworks based on cost-effectiveness modeling and value of information analysis. CED may aid decision-makers when there is uncertainty about the (cost-)effectiveness of a new health technology at the time of reimbursement. Medical devices are potential candidates for CED schemes, as regulatory regimes do not usually require the same level of efficacy and safety data normally needed for pharmaceuticals. The purpose of this research is to assess whether the actual practice of CED for medical devices in Europe meets the theoretical principles proposed by health economists and whether theory and practice can be more closely aligned. Based on decision-makers' perceptions of the challenges associated with CED schemes, plus examples from the schemes themselves, we discuss a series of proposals for assessing the desirability of schemes, their design, implementation, and evaluation. These proposals, while reflecting the practical challenges with developing CED programs, embody many of the principles suggested by economists and should support decision-makers in dealing with uncertainty about the real-world performance of devices.
Collapse
Affiliation(s)
| | - Carlo Federici
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
- School of EngineeringUniversity of WarwickCoventryUK
| | - Vivian Reckers‐Droog
- Erasmus School of Health Policy & ManagementErasmus UniversityRotterdamThe Netherlands
| | - Aleksandra Torbica
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
| | - Carl Rudolf Blankart
- Kompetenzzentrum für Public ManagementUniversität BernBernSwitzerland
- Swiss Institute for Translational and Entrepreneurial MedicineBernSwitzerland
| | - Oriana Ciani
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
| | - Zoltán Kaló
- Syreon Research InstituteBudapestHungary
- Centre for Health Technology AssessmentSemmelweis UniversityBudapestHungary
| | | | - Werner Brouwer
- Erasmus School of Health Policy & ManagementErasmus UniversityRotterdamThe Netherlands
- Erasmus School of EconomicsErasmus University RotterdamRotterdamThe Netherlands
| |
Collapse
|
10
|
Dijk SW, Krijkamp EM, Kunst N, Gross CP, Wong JB, Hunink MGM. Emerging Therapies for COVID-19: The Value of Information From More Clinical Trials. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:1268-1280. [PMID: 35490085 PMCID: PMC9045876 DOI: 10.1016/j.jval.2022.03.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/04/2021] [Revised: 02/14/2022] [Accepted: 03/13/2022] [Indexed: 05/05/2023]
Abstract
OBJECTIVES The COVID-19 pandemic necessitates time-sensitive policy and implementation decisions regarding new therapies in the face of uncertainty. This study aimed to quantify consequences of approving therapies or pursuing further research: immediate approval, use only in research, approval with research (eg, emergency use authorization), or reject. METHODS Using a cohort state-transition model for hospitalized patients with COVID-19, we estimated quality-adjusted life-years (QALYs) and costs associated with the following interventions: hydroxychloroquine, remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, tocilizumab, lopinavir-ritonavir, interferon beta-1a, and usual care. We used the model outcomes to conduct cost-effectiveness and value of information analyses from a US healthcare perspective and a lifetime horizon. RESULTS Assuming a $100 000-per-QALY willingness-to-pay threshold, only remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, and tocilizumab were (cost-) effective (incremental net health benefit 0.252, 0.164, 0.545, 0.668, and 0.524 QALYs and incremental net monetary benefit $25 249, $16 375, $54 526, $66 826, and $52 378). Our value of information analyses suggest that most value can be obtained if these 5 therapies are approved for immediate use rather than requiring additional randomized controlled trials (RCTs) (net value $20.6 billion, $13.4 billion, $7.4 billion, $54.6 billion, and $7.1 billion), hydroxychloroquine (net value $198 million) is only used in further RCTs if seeking to demonstrate decremental cost-effectiveness and otherwise rejected, and interferon beta-1a and lopinavir-ritonavir are rejected (ie, neither approved nor additional RCTs). CONCLUSIONS Estimating the real-time value of collecting additional evidence during the pandemic can inform policy makers and clinicians about the optimal moment to implement therapies and whether to perform further research.
Collapse
Affiliation(s)
- Stijntje W Dijk
- Departments of Epidemiology and Radiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Eline M Krijkamp
- Departments of Epidemiology and Radiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Natalia Kunst
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA; Cancer Outcomes, Public Policy, and Effectiveness Research Center, Yale University School of Medicine, New Haven, CT, USA
| | - Cary P Gross
- Cancer Outcomes, Public Policy, and Effectiveness Research Center, Yale University School of Medicine, New Haven, CT, USA
| | - John B Wong
- Division of Clinical Decision Making, Tufts Medical Center, Boston, MA, USA
| | - M G Myriam Hunink
- Departments of Epidemiology and Radiology, Erasmus University Medical Center, Rotterdam, The Netherlands; Netherlands Institute for Health Sciences, Erasmus University Medical Center, Rotterdam, The Netherlands; Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
| |
Collapse
|
11
|
Jackson CH, Baio G, Heath A, Strong M, Welton NJ, Wilson EC. Value of Information Analysis in Models to Inform Health Policy. ANNUAL REVIEW OF STATISTICS AND ITS APPLICATION 2022; 9:95-118. [PMID: 35415193 PMCID: PMC7612603 DOI: 10.1146/annurev-statistics-040120-010730] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Value of information (VoI) is a decision-theoretic approach to estimating the expected benefits from collecting further information of different kinds, in scientific problems based on combining one or more sources of data. VoI methods can assess the sensitivity of models to different sources of uncertainty and help to set priorities for further data collection. They have been widely applied in healthcare policy making, but the ideas are general to a range of evidence synthesis and decision problems. This article gives a broad overview of VoI methods, explaining the principles behind them, the range of problems that can be tackled with them, and how they can be implemented, and discusses the ongoing challenges in the area.
Collapse
Affiliation(s)
| | - Gianluca Baio
- Department of Statistical Science, University College London, London WC1E 6BT, United Kingdom
| | - Anna Heath
- The Hospital for Sick Children, Toronto, Ontario M5G 1X8, Canada
| | - Mark Strong
- School of Health and Related Research, University of Sheffield, Sheffield S1 4DA, United Kingdom
| | - Nicky J. Welton
- Bristol Medical School (PHS), University of Bristol, Bristol BS8 1QU, United Kingdom
| | | |
Collapse
|
12
|
Jackson C, Johnson R, de Nazelle A, Goel R, de Sá TH, Tainio M, Woodcock J. A guide to value of information methods for prioritising research in health impact modelling. EPIDEMIOLOGIC METHODS 2021; 10:20210012. [PMID: 35127249 PMCID: PMC7612319 DOI: 10.1515/em-2021-0012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Health impact simulation models are used to predict how a proposed policy or scenario will affect population health outcomes. These models represent the typically-complex systems that describe how the scenarios affect exposures to risk factors for disease or injury (e.g. air pollution or physical inactivity), and how these risk factors are related to measures of population health (e.g. expected survival). These models are informed by multiple sources of data, and are subject to multiple sources of uncertainty. We want to describe which sources of uncertainty contribute most to uncertainty about the estimate or decision arising from the model. Furthermore, we want to decide where further research should be focused to obtain further data to reduce this uncertainty, and what form that research might take. This article presents a tutorial in the use of Value of Information methods for uncertainty analysis and research prioritisation in health impact simulation models. These methods are based on Bayesian decision-theoretic principles, and quantify the expected benefits from further information of different kinds. The expected value of partial perfect information about a parameter measures sensitivity of a decision or estimate to uncertainty about that parameter. The expected value of sample information represents the expected benefit from a specific proposed study to get better information about the parameter. The methods are applicable both to situationswhere the model is used to make a decision between alternative policies, and situations where the model is simply used to estimate a quantity (such as expected gains in survival under a scenario). This paper explains how to calculate and interpret the expected value of information in the context of a simple model describing the health impacts of air pollution from motorised transport. We provide a general-purpose R package and full code to reproduce the example analyses.
Collapse
Affiliation(s)
| | - Robert Johnson
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK; and Imperial College London, London, UK
| | | | - Rahul Goel
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK
| | - Thiago Hérick de Sá
- World Health Organization, Geneva, Switzerland; and Center for Epidemiological Research in Nutrition and Health, University of Sao Paulo
| | - Marko Tainio
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK; and Finnish Environment Institute, Helsinki, Finland
| | - James Woodcock
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK
| |
Collapse
|
13
|
Redelmeier DA, Thiruchelvam D, Tibshirani RJ. Testing for a Sweet Spot in Randomized Trials. Med Decis Making 2021; 42:208-216. [PMID: 34378458 PMCID: PMC8777310 DOI: 10.1177/0272989x211025525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Introduction Randomized trials recruit diverse patients, including some individuals who
may be unresponsive to the treatment. Here we follow up on prior conceptual
advances and introduce a specific method that does not rely on
stratification analysis and that tests whether patients in the intermediate
range of disease severity experience more relative benefit than patients at
the extremes of disease severity (sweet spot). Methods We contrast linear models to sigmoidal models when describing associations
between disease severity and accumulating treatment benefit. The Gompertz
curve is highlighted as a specific sigmoidal curve along with the Akaike
information criterion (AIC) as a measure of goodness of fit. This approach
is then applied to a matched analysis of a published landmark randomized
trial evaluating whether implantable defibrillators reduce overall mortality
in cardiac patients (n = 2,521). Results The linear model suggested a significant survival advantage across the
spectrum of increasing disease severity (β = 0.0847, P <
0.001, AIC = 2,491). Similarly, the sigmoidal model suggested a significant
survival advantage across the spectrum of disease severity (α = 93, β =
4.939, γ = 0.00316, P < 0.001 for all, AIC = 1,660). The
discrepancy between the 2 models indicated worse goodness of fit with a
linear model compared to a sigmoidal model (AIC: 2,491 v. 1,660,
P < 0.001), thereby suggesting a sweet spot in the
midrange of disease severity. Model cross-validation using computational
statistics also confirmed the superior goodness of fit of the sigmoidal
curve with a concentration of survival benefits for patients in the midrange
of disease severity. Conclusion Systematic methods are available beyond simple stratification for identifying
a sweet spot according to disease severity. The approach can assess whether
some patients experience more relative benefit than other patients in a
randomized trial. Highlights Randomized trials may recruit patients at extremes of disease
severity who experience less relative benefit than patients
at the middle range of disease severity. We introduce a method to check for possible differential
effects in a randomized trial based on the assumption that a
sweet spot is related to disease severity. The method avoids a proliferation of secondary stratified
analyses and can apply to a randomized trial with a
continuous, binary, or censored survival primary
outcome. The method can work automatically in a randomized trial and
requires no additional information, data collection, special
software, or investigator judgment. Such an analysis for identifying a potential sweet spot can
also help check whether a negative trial correctly excludes
a meaningful effect.
Collapse
Affiliation(s)
- Donald A Redelmeier
- Department of Medicine, University of Toronto, Toronto, ON, Canada.,Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada.,Institute for Clinical Evaluative Sciences.,Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, ON, Canada.,Center for Leading Injury Prevention Practice Education & Research
| | - Deva Thiruchelvam
- Evaluative Clinical Sciences Program, Sunnybrook Research Institute, Toronto, ON, Canada.,Institute for Clinical Evaluative Sciences
| | - Robert J Tibshirani
- Department of Biomedical Data Sciences, Stanford University, Stanford, CA, USA.,Department of Statistics, Stanford University, Stanford, CA, USA
| |
Collapse
|
14
|
Federici C, Pecchia L. Early health technology assessment using the MAFEIP tool. A case study on a wearable device for fall prediction in elderly patients. HEALTH AND TECHNOLOGY 2021. [DOI: 10.1007/s12553-021-00580-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
AbstractBy using a case-study on a fall-prediction device for elderly patients with orthostatic hypotension we aim to demonstrate how the MAFEIP tool, developed as part of the European Innovation Programme on Active and Healthy Ageing (EIP on AHA), can be used to inform manufacturers on their product development based on a cost-effectiveness criterion. Secondly, we critically appraise the tool and suggest further improvements that may be needed for a larger-scale adoption of MAFEIP within and beside the EIP on AHA initiative. The model was implemented using the MAFEIP tool. Within the tool one way sensitivity analyses were performed to assess the robustness of the model against the relative effectiveness of the fall-prevention device at different price levels. The MAFEIP tool was applied to a novel fall-prediction device and used to estimate the expected cost-effectiveness and perform threshold analysis. In our case study, the device produced estimated gains of 0.035 QALYs per patient and incremental costs of £ 518 (incremental cost-effectiveness ratio £14,719). Based on the one-way sensitivity analysis, the maximum achievable price at a willingness to pay threshold of £20,000 per QALY is estimated close to £900. The MAFEIP allows to quickly create early economic models, and to explore model uncertainty by performing deterministic sensitivity analysis for single parameters. However, the integration within the MAFEIP of common analytical tools such as probabilistic sensitivity analysis and Value of information would greatly contribute to its relevance for evaluating innovative technologies within and beside the EIP on AHA initiative.
Collapse
|
15
|
Heath A, Myriam Hunink MG, Krijkamp E, Pechlivanoglou P. Prioritisation and design of clinical trials. Eur J Epidemiol 2021; 36:1111-1121. [PMID: 34091766 PMCID: PMC8629779 DOI: 10.1007/s10654-021-00761-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2020] [Accepted: 05/10/2021] [Indexed: 11/30/2022]
Abstract
Clinical trials require participation of numerous patients, enormous research resources and substantial public funding. Time-consuming trials lead to delayed implementation of beneficial interventions and to reduced benefit to patients. This manuscript discusses two methods for the allocation of research resources and reviews a framework for prioritisation and design of clinical trials. The traditional error-driven approach of clinical trial design controls for type I and II errors. However, controlling for those statistical errors has limited relevance to policy makers. Therefore, this error-driven approach can be inefficient, waste research resources and lead to research with limited impact on daily practice. The novel value-driven approach assesses the currently available evidence and focuses on designing clinical trials that directly inform policy and treatment decisions. Estimating the net value of collecting further information, prior to undertaking a trial, informs a decision maker whether a clinical or health policy decision can be made with current information or if collection of extra evidence is justified. Additionally, estimating the net value of new information guides study design, data collection choices, and sample size estimation. The value-driven approach ensures the efficient use of research resources, reduces unnecessary burden to trial participants, and accelerates implementation of beneficial healthcare interventions.
Collapse
Affiliation(s)
- Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada.,Division of Biostatistics, University of Toronto, Toronto, ON, Canada.,Department of Statistical Science, University College London, London, UK
| | - M G Myriam Hunink
- Department of Epidemiology, Erasmus MC, University Medical Center, Rotterdam, Netherlands. .,Department of Radiology, Erasmus MC, University Medical Center, Rotterdam, Netherlands. .,Netherlands Institute for Health Sciences, Erasmus MC, University Medical Center, Rotterdam, Netherlands. .,Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, MA, USA.
| | - Eline Krijkamp
- Department of Epidemiology, Erasmus MC, University Medical Center, Rotterdam, Netherlands.,Netherlands Institute for Health Sciences, Erasmus MC, University Medical Center, Rotterdam, Netherlands
| | - Petros Pechlivanoglou
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada.,Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
16
|
Grimm SE, Pouwels X, Ramaekers BLT, Wijnen B, Knies S, Grutters J, Joore MA. Building a trusted framework for uncertainty assessment in rare diseases: suggestions for improvement (Response to "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases"). Orphanet J Rare Dis 2021; 16:62. [PMID: 33522936 PMCID: PMC7849113 DOI: 10.1186/s13023-020-01666-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Accepted: 12/22/2020] [Indexed: 01/21/2023] Open
Abstract
The aim of this letter to the editor is to provide a comprehensive summary of uncertainty assessment in Health Technology Assessment, with a focus on transferability to the setting of rare diseases. The authors of "TRUST4RD: tool for reducing uncertainties in the evidence generation for specialised treatments for rare diseases" presented recommendations for reducing uncertainty in rare diseases. Their article is of great importance but unfortunately suffers from a lack of references to the wider uncertainty in Health Technology Assessment and research prioritisation literature and consequently fails to provide a trusted framework for decision-making in rare diseases. In this letter to the editor we critique the authors' tool and provide pointers as to how their proposal can be strengthened. We present references to the literature, including our own tool for uncertainty assessment (TRUST; unrelated to the authors' research), apply TRUST to two assessments of orphan drugs in rare diseases and provide a broader perspective on uncertainty and risk management in rare diseases, including a detailed research agenda.
Collapse
Affiliation(s)
- Sabine E Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, Netherlands.
| | - Xavier Pouwels
- University of Twente, Hallenweg 5, 7522 NH, Enschede, Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, Netherlands
| | - Ben Wijnen
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, Netherlands
| | - Saskia Knies
- Zorginstituut Nederland, Eekholt 4, 1112 XH, Diemen, Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud University Medical Centre, Post 133, PO Box 9101, 6500 HB, Nijmegen, Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, Netherlands
| |
Collapse
|
17
|
Optimal cholesterol treatment plans and genetic testing strategies for cardiovascular diseases. Health Care Manag Sci 2021; 24:1-25. [PMID: 33483911 DOI: 10.1007/s10729-020-09537-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2020] [Accepted: 11/30/2020] [Indexed: 12/25/2022]
Abstract
Atherosclerotic cardiovascular disease (ASCVD) is among the leading causes of death in the US. Although research has shown that ASCVD has genetic elements, the understanding of how genetic testing influences its prevention and treatment has been limited. To this end, we model the health trajectory of patients stochastically and determine treatment and testing decisions simultaneously. Since the cholesterol level of patients is one controllable risk factor for ASCVD events, we model cholesterol treatment plans as Markov decision processes. We determine whether and when patients should receive a genetic test using value of information analysis. By simulating the health trajectory of over 64 million adult patients, we find that 6.73 million patients undergo genetic testing. The optimal treatment plans informed with clinical and genetic information save 5,487 more quality-adjusted life-years while costing $1.18 billion less than the optimal treatment plans informed with clinical information only. As precision medicine becomes increasingly important, understanding the impact of genetic information becomes essential.
Collapse
|
18
|
Walker AM. Complementary hypotheses in safety surveillance. Seq Anal 2021. [DOI: 10.1080/07474946.2020.1823195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
19
|
Zang X, Jalal H, Krebs E, Pandya A, Zhou H, Enns B, Nosyk B. Prioritizing Additional Data Collection to Reduce Decision Uncertainty in the HIV/AIDS Response in 6 US Cities: A Value of Information Analysis. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2020; 23:1534-1542. [PMID: 33248508 PMCID: PMC7705607 DOI: 10.1016/j.jval.2020.06.013] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 06/08/2020] [Accepted: 06/30/2020] [Indexed: 06/12/2023]
Abstract
OBJECTIVES The ambitious goals of the US Ending the HIV Epidemic initiative will require a targeted, context-specific public health response. Model-based economic evaluation provides useful guidance for decision making while characterizing decision uncertainty. We aim to quantify the value of eliminating uncertainty about different parameters in selecting combination implementation strategies to reduce the public health burden of HIV/AIDS in 6 US cities and identify future data collection priorities. METHODS We used a dynamic compartmental HIV transmission model developed for 6 US cities to evaluate the cost-effectiveness of a range of combination implementation strategies. Using a metamodeling approach with nonparametric and deep learning methods, we calculated the expected value of perfect information, representing the maximum value of further research to eliminate decision uncertainty, and the expected value of partial perfect information for key groups of parameters that would be collected together in practice. RESULTS The population expected value of perfect information ranged from $59 683 (Miami) to $54 108 679 (Los Angeles). The rank ordering of expected value of partial perfect information on key groups of parameters were largely consistent across cities and highest for parameters pertaining to HIV risk behaviors, probability of HIV transmission, health service engagement, HIV-related mortality, health utility weights, and healthcare costs. Los Angeles was an exception, where parameters on retention in pre-exposure prophylaxis ranked highest in contributing to decision uncertainty. CONCLUSIONS Funding additional data collection on HIV/AIDS may be warranted in Baltimore, Los Angeles, and New York City. Value of information analysis should be embedded into decision-making processes on funding future research and public health intervention.
Collapse
Affiliation(s)
- Xiao Zang
- BC Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada; Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada
| | - Hawre Jalal
- Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, USA
| | - Emanuel Krebs
- BC Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada
| | - Ankur Pandya
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Cambridge, MA, USA
| | - Haoxuan Zhou
- BC Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada
| | - Benjamin Enns
- BC Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada
| | - Bohdan Nosyk
- BC Centre for Excellence in HIV/AIDS, Vancouver, British Columbia, Canada; Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada.
| |
Collapse
|
20
|
Houy N, Flaig J. Informed and uninformed empirical therapy policies. MATHEMATICAL MEDICINE AND BIOLOGY-A JOURNAL OF THE IMA 2020; 37:334-350. [PMID: 31875921 DOI: 10.1093/imammb/dqz015] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2019] [Revised: 07/16/2019] [Accepted: 10/02/2019] [Indexed: 12/21/2022]
Abstract
We argue that a proper distinction must be made between informed and uninformed decision making when setting empirical therapy policies, as this allows one to estimate the value of gathering more information about the pathogens and their transmission and thus to set research priorities. We rely on the stochastic version of a compartmental model to describe the spread of an infecting organism in a health care facility and the emergence and spread of resistance to two drugs. We focus on information and uncertainty regarding the parameters of this model. We consider a family of adaptive empirical therapy policies. In the uninformed setting, the best adaptive policy allowsone to reduce the average cumulative infected patient days over 2 years by 39.3% (95% confidence interval (CI), 30.3-48.1%) compared to the combination therapy. Choosing empirical therapy policies while knowing the exact parameter values allows one to further decrease the cumulative infected patient days by 3.9% (95% CI, 2.1-5.8%) on average. In our setting, the benefit of perfect information might be offset by increased drug consumption.
Collapse
Affiliation(s)
- Nicolas Houy
- University of Lyon, Lyon, F-69007, France.,CNRS, GATE Lyon Saint-Etienne, F-69130, France
| | | |
Collapse
|
21
|
Kunst N, Wilson ECF, Glynn D, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Goldhaber-Fiebert JD, Jackson C, Jalal H, Menzies NA, Strong M, Thom H, Heath A. Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2020; 23:734-742. [PMID: 32540231 PMCID: PMC8183576 DOI: 10.1016/j.jval.2020.02.010] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2019] [Revised: 12/19/2019] [Accepted: 02/11/2020] [Indexed: 05/09/2023]
Abstract
Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.
Collapse
Affiliation(s)
- Natalia Kunst
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway; Yale University School of Medicine, New Haven, CT, USA; Department of Epidemiology and Biostatistics, Amsterdam UMC, Amsterdam, The Netherlands; LINK Medical Research, Oslo, Norway.
| | - Edward C F Wilson
- Health Economics Group, Norwich Medical School, University of East Anglia, Norwich, England, UK
| | | | | | | | - Alan Brennan
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, England, UK
| | - Michael Fairley
- Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA
| | - Jeremy D Goldhaber-Fiebert
- Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA
| | - Chris Jackson
- MRC Biostatistics Unit, University of Cambridge, Cambridge, England, UK
| | - Hawre Jalal
- University of Pittsburgh, Pittsburgh, PA, USA
| | - Nicolas A Menzies
- Harvard TH Chan School of Public Health, Harvard University, Boston, MA, USA
| | - Mark Strong
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, England, UK
| | | | - Anna Heath
- University College London, London, England, UK; The Hospital for Sick Children, Toronto, ON, Canada; University of Toronto, Toronto, ON, Canada
| |
Collapse
|
22
|
Heath A, Kunst N, Jackson C, Strong M, Alarid-Escudero F, Goldhaber-Fiebert JD, Baio G, Menzies NA, Jalal H. Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies. Med Decis Making 2020; 40:314-326. [PMID: 32297840 PMCID: PMC7968749 DOI: 10.1177/0272989x20912402] [Citation(s) in RCA: 21] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.
Collapse
Affiliation(s)
- Anna Heath
- The Hospital for Sick Children, Toronto, ON, Canada
- University of Toronto, Toronto, ON, Canada
- University College London, London, UK
| | - Natalia Kunst
- Department of Health Management and Health Economics, Institute of Health and Society, Faculty of Medicine, University of Oslo, Oslo, Norway
- Cancer Outcomes, Public Policy and Effectiveness Research (COPPER) Center, Yale University School of Medicine and Yale Cancer Center, New Haven, CT, USA
- Department of Epidemiology and Biostatistics, Amsterdam UMC, Amsterdam, the Netherlands
- LINK Medical Research, Oslo, Norway
| | | | - Mark Strong
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | | | - Jeremy D Goldhaber-Fiebert
- Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA
| | | | | | - Hawre Jalal
- University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
23
|
Degeling K, IJzerman MJ, Lavieri MS, Strong M, Koffijberg H. Introduction to Metamodeling for Reducing Computational Burden of Advanced Analyses with Health Economic Models: A Structured Overview of Metamodeling Methods in a 6-Step Application Process. Med Decis Making 2020; 40:348-363. [PMID: 32428428 PMCID: PMC7754830 DOI: 10.1177/0272989x20912233] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Accepted: 02/14/2020] [Indexed: 01/24/2023]
Abstract
Metamodels can be used to reduce the computational burden associated with computationally demanding analyses of simulation models, although applications within health economics are still scarce. Besides a lack of awareness of their potential within health economics, the absence of guidance on the conceivably complex and time-consuming process of developing and validating metamodels may contribute to their limited uptake. To address these issues, this article introduces metamodeling to the wider health economic audience and presents a process for applying metamodeling in this context, including suitable methods and directions for their selection and use. General (i.e., non-health economic specific) metamodeling literature, clinical prediction modeling literature, and a previously published literature review were exploited to consolidate a process and to identify candidate metamodeling methods. Methods were considered applicable to health economics if they are able to account for mixed (i.e., continuous and discrete) input parameters and continuous outcomes. Six steps were identified as relevant for applying metamodeling methods within health economics: 1) the identification of a suitable metamodeling technique, 2) simulation of data sets according to a design of experiments, 3) fitting of the metamodel, 4) assessment of metamodel performance, 5) conducting the required analysis using the metamodel, and 6) verification of the results. Different methods are discussed to support each step, including their characteristics, directions for use, key references, and relevant R and Python packages. To address challenges regarding metamodeling methods selection, a first guide was developed toward using metamodels to reduce the computational burden of analyses of health economic models. This guidance may increase applications of metamodeling in health economics, enabling increased use of state-of-the-art analyses (e.g., value of information analysis) with computationally burdensome simulation models.
Collapse
Affiliation(s)
- Koen Degeling
- />Health Technology and Services Research Department, Faculty of Behavioural, Management and Social Sciences, Technical Medical Centre, University of Twente, Enschede, Overijssel, the Netherlands
- />Cancer Health Services Research Unit, School of Population and Global Health, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia
| | - Maarten J. IJzerman
- />Victorian Comprehensive Cancer Centre, Melbourne, Australia
- />Health Technology and Services Research Department, Faculty of Behavioural, Management and Social Sciences, Technical Medical Centre, University of Twente, Enschede, Overijssel, the Netherlands
- />Cancer Health Services Research Unit, School of Population and Global Health, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia
| | - Mariel S. Lavieri
- Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI, USA
| | - Mark Strong
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, England, UK
| | - Hendrik Koffijberg
- Health Technology and Services Research Department, Faculty of Behavioural, Management and Social Sciences, Technical Medical Centre, University of Twente, Enschede, Overijssel, the Netherlands
| |
Collapse
|
24
|
Rothery C, Strong M, Koffijberg HE, Basu A, Ghabri S, Knies S, Murray JF, Sanders Schmidler GD, Steuten L, Fenwick E. Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2020; 23:277-286. [PMID: 32197720 PMCID: PMC7373630 DOI: 10.1016/j.jval.2020.01.004] [Citation(s) in RCA: 59] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Accepted: 01/16/2020] [Indexed: 05/19/2023]
Abstract
The allocation of healthcare resources among competing priorities requires an assessment of the expected costs and health effects of investing resources in the activities and of the opportunity cost of the expenditure. To date, much effort has been devoted to assessing the expected costs and health effects, but there remains an important need to also reflect the consequences of uncertainty in resource allocation decisions and the value of further research to reduce uncertainty. Decision making with uncertainty may turn out to be suboptimal, resulting in health loss. Consequently, there may be value in reducing uncertainty, through the collection of new evidence, to better inform resource decisions. This value can be quantified using value of information (VOI) analysis. This report from the ISPOR VOI Task Force describes methods for computing 4 VOI measures: the expected value of perfect information, expected value of partial perfect information (EVPPI), expected value of sample information (EVSI), and expected net benefit of sampling (ENBS). Several methods exist for computing EVPPI and EVSI, and this report provides guidance on selecting the most appropriate method based on the features of the decision problem. The report provides a number of recommendations for good practice when planning, undertaking, or reviewing VOI analyses. The software needed to compute VOI is discussed, and areas for future research are highlighted.
Collapse
Affiliation(s)
- Claire Rothery
- Centre for Health Economics, University of York, York, England, UK.
| | - Mark Strong
- School of Health and Related Research, University of Sheffield, Sheffield, England, UK
| | - Hendrik Erik Koffijberg
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands
| | - Anirban Basu
- The Comparative Health Outcomes, Policy, and Economics Institute, School of Pharmacy, University of Washington, Seattle, Washington, DC, USA
| | - Salah Ghabri
- French National Authority for Health, Paris, France
| | - Saskia Knies
- National Health Care Institute (Zorginstituut Nederland), Diemen, The Netherlands
| | | | - Gillian D Sanders Schmidler
- Duke-Margolis Center for Health Policy, Duke Clinical Research Institute and Department of Population Health Sciences, Duke University, Durham, NC, USA
| | | | | |
Collapse
|
25
|
Franklin M, Lomas J, Richardson G. Conducting Value for Money Analyses for Non-randomised Interventional Studies Including Service Evaluations: An Educational Review with Recommendations. PHARMACOECONOMICS 2020; 38:665-681. [PMID: 32291596 PMCID: PMC7319287 DOI: 10.1007/s40273-020-00907-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
This article provides an educational review covering the consideration of conducting ‘value for money’ analyses as part of non-randomised study designs including service evaluations. These evaluations represent a vehicle for producing evidence such as value for money of a care intervention or service delivery model. Decision makers including charities and local and national governing bodies often rely on evidence from non-randomised data and service evaluations to inform their resource allocation decision-making. However, as randomised data obtained from randomised controlled trials are considered the ‘gold standard’ for assessing causation, the use of this alternative vehicle for producing an evidence base requires careful consideration. We refer to value for money analyses, but reflect on methods associated with economic evaluations as a form of analysis used to inform resource allocation decision-making alongside a finite budget. Not all forms of value for money analysis are considered a full economic evaluation with implications for the information provided to decision makers. The type of value for money analysis to be conducted requires considerations such as the outcome(s) of interest, study design, statistical methods to control for confounding and bias, and how to quantify and describe uncertainty and opportunity costs to decision makers in any resulting value for money estimates. Service evaluations as vehicles for producing evidence present different challenges to analysts than what is commonly associated with research, randomised controlled trials and health technology appraisals, requiring specific study design and analytic considerations. This educational review describes and discusses these considerations, as overlooking them could affect the information provided to decision makers who may make an ‘ill-informed’ decision based on ‘poor’ or ‘inaccurate’ information with long-term implications. We make direct comparisons between randomised controlled trials relative to non-randomised data as vehicles for assessing causation; given ‘gold standard’ randomised controlled trials have limitations. Although we use UK-based decision makers as examples, we reflect on the needs of decision makers internationally for evidence-based decision-making specific to resource allocation. We make recommendations based on the experiences of the authors in the UK, reflecting on the wide variety of methods available, used as documented in the empirical literature. These methods may not have been fully considered relevant to non-randomised study designs and/or service evaluations, but could improve and aid the analysis conducted to inform the relevant value for money decision problem.
Collapse
Affiliation(s)
- Matthew Franklin
- Health Economics and Decision Science (HEDS), School of Health and Related Research (ScHARR), University of Sheffield, West Court, 1 Mappin Street, Sheffield, S1 4DT UK
| | - James Lomas
- Centre for Health Economics, University of York, Heslington, York UK
| | - Gerry Richardson
- Centre for Health Economics, University of York, Heslington, York UK
| |
Collapse
|
26
|
Jones DA, Smith J, Mei XW, Hawkins MA, Maughan T, van den Heuvel F, Mee T, Kirkby K, Kirkby N, Gray A. A systematic review of health economic evaluations of proton beam therapy for adult cancer: Appraising methodology and quality. Clin Transl Radiat Oncol 2020; 20:19-26. [PMID: 31754652 PMCID: PMC6854069 DOI: 10.1016/j.ctro.2019.10.007] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2019] [Accepted: 10/28/2019] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND AND PURPOSE With high treatment costs and limited capacity, decisions on which adult patients to treat with proton beam therapy (PBT) must be based on the relative value compared to the current standard of care. Cost-utility analyses (CUAs) are the gold-standard method for doing this. We aimed to appraise the methodology and quality of CUAs in this area. MATERIALS AND METHODS We performed a systematic review of the literature to identify CUA studies of PBT in adult disease using MEDLINE, EMBASE, EconLIT, NHS Economic Evaluation Database (NHS EED), Web of Science, and the Tufts Medical Center Cost-Effectiveness Analysis Registry from 1st January 2010 up to 6th June 2018. General characteristics, information relating to modelling approaches, and methodological quality were extracted and synthesized narratively. RESULTS Seven PBT CUA studies in adult disease were identified. Without randomised controlled trials to inform the comparative effectiveness of PBT, studies used either results from one-armed studies, or dose-response models derived from radiobiological and epidemiological studies of PBT. Costing methods varied widely. The assessment of model quality highlighted a lack of transparency in the identification of model parameters, and absence of external validation of model outcomes. Furthermore, appropriate assessment of uncertainty was often deficient. CONCLUSION In order to foster credibility, future CUA studies must be more systematic in their approach to evidence synthesis and expansive in their consideration of uncertainties in light of the lack of clinical evidence.
Collapse
Affiliation(s)
- David A. Jones
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
| | - Joel Smith
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Oxford, UK
| | - Xue W. Mei
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
| | | | - Tim Maughan
- CRUK/MRC Oxford Institute for Radiation Oncology, Oxford, UK
| | - Frank van den Heuvel
- CRUK/MRC Oxford Institute for Radiation Oncology, Oxford, UK
- Department of Haematology/Oncology, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Thomas Mee
- Division of Cancer Sciences, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Karen Kirkby
- Division of Cancer Sciences, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Norman Kirkby
- Division of Cancer Sciences, School of Medical Sciences, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| | - Alastair Gray
- Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
27
|
Kunst NR, Alarid-Escudero F, Paltiel AD, Wang SY. A Value of Information Analysis of Research on the 21-Gene Assay for Breast Cancer Management. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2019; 22:1102-1110. [PMID: 31563252 PMCID: PMC7343670 DOI: 10.1016/j.jval.2019.05.004] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2018] [Revised: 04/14/2019] [Accepted: 05/15/2019] [Indexed: 05/02/2023]
Abstract
OBJECTIVES The 21-gene assay Oncotype DX (21-GA) shows promise as a guide in deciding when to initiate adjuvant chemotherapy in women with hormone receptor-positive early-stage breast cancer. Nevertheless, its routine use remains controversial, owing to insufficient evidence of its clinical utility and cost-effectiveness. Accordingly, we aim to quantify the value of conducting further research to reduce decision uncertainty in the use of the 21-GA. METHODS Using value of information methods, we first generated probability distributions of survival and costs for decision making with and without the 21-GA alongside traditional risk prediction. These served as the input to a comparison of 3 alternative study designs: a retrospective observational study to update risk classification from the 21-GA, a prospective observational study to estimate prevalence of chemotherapy use, and a randomized controlled trial (RCT) of the 21-GA predictive value. RESULTS We found that current evidence strongly supports the use of the 21-GA in intermediate- and high-risk women. Further research should focus on low-risk women, among whom the cost-effectiveness findings remained equivocal. For this population, we identified a high value of reducing uncertainty in the 21-GA use for all proposed research studies. The RCT had the greatest potential to efficiently reduce the likelihood of choosing a suboptimal strategy, providing a value between $162 million and $1.1 billion at willingness-to-pay thresholds of $150 000 to $200 000/quality-adjusted life years. CONCLUSION Future research to inform 21-GA decision making is of high value. The RCT of the 21-GA predictive value has the greatest potential to efficiently reduce decision uncertainty around 21-GA use in women with low-risk early-stage breast cancer.
Collapse
Affiliation(s)
- Natalia R Kunst
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway; Department of Internal Medicine, Yale University School of Medicine, New Haven, CT, USA; Department of Epidemiology and Biostatistics, Amsterdam UMC, Amsterdam, the Netherlands; LINK Medical Research, Oslo, Norway.
| | - Fernando Alarid-Escudero
- Drug Policy Program, Center for Research and Teaching in Economics (CIDE), Aguascalientes, Mexico; National Council on Science and Technology (CONACyT), Mexico City, Mexico
| | - A David Paltiel
- Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
| | - Shi-Yi Wang
- Department of Chronic Disease Epidemiology, Yale University School of Public Health, New Haven, CT, USA; Cancer Outcomes, Public Policy, and Effectiveness Research Center, Yale Cancer Center and Yale University School of Medicine, New Haven, CT, USA
| |
Collapse
|
28
|
Makhoul N. Seismic risk mitigation in buildings using a new method to encode a joint weighting function in multi-attribute utility theory. SN APPLIED SCIENCES 2019. [DOI: 10.1007/s42452-019-1136-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022] Open
|
29
|
Jutkowitz E, Alarid-Escudero F, Kuntz KM, Jalal H. The Curve of Optimal Sample Size (COSS): A Graphical Representation of the Optimal Sample Size from a Value of Information Analysis. PHARMACOECONOMICS 2019; 37:871-877. [PMID: 30761461 PMCID: PMC6556417 DOI: 10.1007/s40273-019-00770-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/23/2023]
Abstract
Value of information (VOI) analysis quantifies the opportunity cost associated with decision uncertainty, and thus informs the value of collecting further information to avoid this cost. VOI can inform study design, optimal sample size selection, and research prioritization. Recent methodological advances have reduced the computational burden of conducting VOI analysis and have made it easier to evaluate the expected value of sample information, the expected net benefit of sampling, and the optimal sample size of a study design ([Formula: see text]). The volume of VOI analyses being published is increasing, and there is now a need for VOI studies to conduct sensitivity analyses on VOI-specific parameters. In this practical application, we introduce the curve of optimal sample size (COSS), which is a graphical representation of [Formula: see text] over a range of willingness-to-pay thresholds and VOI parameters (example data and R code are provided). In a single figure, the COSS presents summary data for decision makers to determine the sample size that optimizes research funding given their operating characteristics. The COSS also presents variation in the optimal sample size given variability or uncertainty in VOI parameters. The COSS represents an efficient and additional approach for summarizing results from a VOI analysis.
Collapse
Affiliation(s)
- Eric Jutkowitz
- Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, RI, USA
| | - Fernando Alarid-Escudero
- Drug Policy Program, Center for Research and Teaching in Economics (CIDE)-CONACyT, 20313, Aguascalientes, AGS, Mexico.
| | - Karen M Kuntz
- Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN, USA
| | - Hawre Jalal
- Division of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
30
|
Heath A, Manolopoulou I, Baio G. Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression. Med Decis Making 2019; 39:346-358. [PMID: 31161867 DOI: 10.1177/0272989x19837983] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background. The expected value of sample information (EVSI) determines the economic value of any future study with a specific design aimed at reducing uncertainty about the parameters underlying a health economic model. This has potential as a tool for trial design; the cost and value of different designs could be compared to find the trial with the greatest net benefit. However, despite recent developments, EVSI analysis can be slow, especially when optimizing over a large number of different designs. Methods. This article develops a method to reduce the computation time required to calculate the EVSI across different sample sizes. Our method extends the moment-matching approach to EVSI estimation to optimize over different sample sizes for the underlying trial while retaining a similar computational cost to a single EVSI estimate. This extension calculates the posterior variance of the net monetary benefit across alternative sample sizes and then uses Bayesian nonlinear regression to estimate the EVSI across these sample sizes. Results. A health economic model developed to assess the cost-effectiveness of interventions for chronic pain demonstrates that this EVSI calculation method is fast and accurate for realistic models. This example also highlights how different trial designs can be compared using the EVSI. Conclusion. The proposed estimation method is fast and accurate when calculating the EVSI across different sample sizes. This will allow researchers to realize the potential of using the EVSI to determine an economically optimal trial design for reducing uncertainty in health economic models. Limitations. Our method involves rerunning the health economic model, which can be more computationally expensive than some recent alternatives, especially in complex models.
Collapse
Affiliation(s)
- Anna Heath
- The Hospital for Sick Children, Toronto, Canada and University of Toronto, Canada
| | | | - Gianluca Baio
- Department of Statistical Science, University College London, London, UK
| |
Collapse
|
31
|
Hatswell AJ, Bullement A, Briggs A, Paulden M, Stevenson MD. Probabilistic Sensitivity Analysis in Cost-Effectiveness Models: Determining Model Convergence in Cohort Models. PHARMACOECONOMICS 2018; 36:1421-1426. [PMID: 30051268 DOI: 10.1007/s40273-018-0697-3] [Citation(s) in RCA: 64] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Probabilistic sensitivity analysis (PSA) demonstrates the parameter uncertainty in a decision problem. The technique involves sampling parameters from their respective distributions (rather than simply using mean/median parameter values). Guidance in the literature, and from health technology assessment bodies, on the number of simulations that should be performed suggests a 'sufficient number', or until 'convergence', which is seldom defined. The objective of this tutorial is to describe possible outcomes from PSA, discuss appropriate levels of accuracy, and present guidance by which an analyst can determine if a sufficient number of simulations have been conducted, such that results are considered to have converged. The proposed approach considers the variance of the outcomes of interest in cost-effectiveness analysis as a function of the number of simulations. A worked example of the technique is presented using results from a published model, with recommendations made on best practice. While the technique presented remains essentially arbitrary, it does give a mechanism for assessing the level of simulation error, and thus represents an advance over current practice of a round number of simulations with no assessment of model convergence.
Collapse
Affiliation(s)
- Anthony J Hatswell
- University College London, London, UK.
- Delta Hat Limited, Nottingham, UK.
| | - Ash Bullement
- Delta Hat Limited, Nottingham, UK
- BresMed Health Solutions, Sheffield, UK
| | - Andrew Briggs
- University of Glasgow, Glasgow, UK
- Memorial Sloan-Kettering Cancer Center, New York, NY, USA
| | | | | |
Collapse
|
32
|
Degeling K, IJzerman M, Koffijberg H. A scoping review of metamodeling applications and opportunities for advanced health economic analyses. Expert Rev Pharmacoecon Outcomes Res 2018; 19:181-187. [DOI: 10.1080/14737167.2019.1548279] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Affiliation(s)
- K. Degeling
- Health Technology and Services Research Department, Technical Medical Centre, University of Twente, Enschede, the Netherlands
| | - M.J. IJzerman
- Health Technology and Services Research Department, Technical Medical Centre, University of Twente, Enschede, the Netherlands
- Cancer Health Services Research Unit, School of Population and Global Health, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, Australia
- Victorian Comprehensive Cancer Centre, Melbourne, Australia
| | - H. Koffijberg
- Health Technology and Services Research Department, Technical Medical Centre, University of Twente, Enschede, the Netherlands
| |
Collapse
|
33
|
Heath A, Baio G. Calculating the Expected Value of Sample Information Using Efficient Nested Monte Carlo: A Tutorial. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2018; 21:1299-1304. [PMID: 30442277 DOI: 10.1016/j.jval.2018.05.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/07/2017] [Revised: 03/27/2018] [Accepted: 05/07/2018] [Indexed: 06/09/2023]
Abstract
OBJECTIVE The expected value of sample information (EVSI) quantifies the economic benefit of reducing uncertainty in a health economic model by collecting additional information. This has the potential to improve the allocation of research budgets. Despite this, practical EVSI evaluations are limited partly due to the computational cost of estimating this value using the gold-standard nested simulation methods. Recently, however, Heath et al. developed an estimation procedure that reduces the number of simulations required for this gold-standard calculation. Up to this point, this new method has been presented in purely technical terms. STUDY DESIGN This study presents the practical application of this new method to aid its implementation. We use a worked example to illustrate the key steps of the EVSI estimation procedure before discussing its optimal implementation using a practical health economic model. METHODS The worked example is based on a three-parameter linear health economic model. The more realistic model evaluates the cost-effectiveness of a new chemotherapy treatment, which aims to reduce the number of side effects experienced by patients. We use a Markov model structure to evaluate the health economic profile of experiencing side effects. RESULTS This EVSI estimation method offers accurate estimation within a feasible computation time, seconds compared to days, even for more complex model structures. The EVSI estimation is more accurate if a greater number of nested samples are used, even for a fixed computational cost. CONCLUSIONS This new method reduces the computational cost of estimating the EVSI by nested simulation.
Collapse
Affiliation(s)
- Anna Heath
- Department of Statistical Science, University College London, London, UK.
| | - Gianluca Baio
- Department of Statistical Science, University College London, London, UK
| |
Collapse
|
34
|
Koffijberg H, Rothery C, Chalkidou K, Grutters J. Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations. Med Decis Making 2018; 38:888-900. [DOI: 10.1177/0272989x18797948] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.
Collapse
Affiliation(s)
- Hendrik Koffijberg
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands (HK)
- Centre for Health Economics, University of York, York, Heslington, UK (CR)
- Global Health and Development Group, Institute for Global Health Innovation, Imperial College London, London, UK (KC)
- Department for Health Evidence, Radboud University Medical Center, Nijmegen, Gelderland, The Netherlands (JG)
| | - Claire Rothery
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands (HK)
- Centre for Health Economics, University of York, York, Heslington, UK (CR)
- Global Health and Development Group, Institute for Global Health Innovation, Imperial College London, London, UK (KC)
- Department for Health Evidence, Radboud University Medical Center, Nijmegen, Gelderland, The Netherlands (JG)
| | - Kalipso Chalkidou
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands (HK)
- Centre for Health Economics, University of York, York, Heslington, UK (CR)
- Global Health and Development Group, Institute for Global Health Innovation, Imperial College London, London, UK (KC)
- Department for Health Evidence, Radboud University Medical Center, Nijmegen, Gelderland, The Netherlands (JG)
| | - Janneke Grutters
- Department of Health Technology & Services Research, Technical Medical Centre, University of Twente, Enschede, The Netherlands (HK)
- Centre for Health Economics, University of York, York, Heslington, UK (CR)
- Global Health and Development Group, Institute for Global Health Innovation, Imperial College London, London, UK (KC)
- Department for Health Evidence, Radboud University Medical Center, Nijmegen, Gelderland, The Netherlands (JG)
| |
Collapse
|
35
|
Walker AM. Conditional power as an aid in making interim decisions in observational studies. Eur J Epidemiol 2018; 33:777-784. [PMID: 29808341 DOI: 10.1007/s10654-018-0413-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2017] [Accepted: 05/22/2018] [Indexed: 01/19/2023]
Abstract
Conditional power combines the findings of a partially completed study with assumptions about the future. The goal is to estimate the probability that the eventual study result will be incompatible with a criterion value, such as acceptable risk or the null hypothesis. Some history and motivation for conditional power calculations are provided, with examples illustrating the application to drug safety studies. This is an expository article suggesting that conditional power, which is well-established in clinical trials research, also has application to observational studies. The utility may be highest in regulatory settings where resources are limited and interim decisions have to be made accurately in the shortest possible time.
Collapse
Affiliation(s)
- Alexander Muir Walker
- World Health Information Science Consultants, 275 Grove Street, Suite 2-400, Newton, MA, 02466, USA.
| |
Collapse
|
36
|
McCullagh L, Schmitz S, Barry M, Walsh C. Examining the Feasibility and Utility of Estimating Partial Expected Value of Perfect Information (via a Nonparametric Approach) as Part of the Reimbursement Decision-Making Process in Ireland: Application to Drugs for Cancer. PHARMACOECONOMICS 2017; 35:1177-1185. [PMID: 28770453 DOI: 10.1007/s40273-017-0552-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
BACKGROUND In Ireland, all new drugs for which reimbursement by the healthcare payer is sought undergo a health technology assessment by the National Centre for Pharmacoeconomics. The National Centre for Pharmacoeconomics estimate expected value of perfect information but not partial expected value of perfect information (owing to computational expense associated with typical methodologies). OBJECTIVE The objective of this study was to examine the feasibility and utility of estimating partial expected value of perfect information via a computationally efficient, non-parametric regression approach. METHODS This was a retrospective analysis of evaluations on drugs for cancer that had been submitted to the National Centre for Pharmacoeconomics (January 2010 to December 2014 inclusive). Drugs were excluded if cost effective at the submitted price. Drugs were excluded if concerns existed regarding the validity of the applicants' submission or if cost-effectiveness model functionality did not allow required modifications to be made. For each included drug (n = 14), value of information was estimated at the final reimbursement price, at a threshold equivalent to the incremental cost-effectiveness ratio at that price. The expected value of perfect information was estimated from probabilistic analysis. Partial expected value of perfect information was estimated via a non-parametric approach. Input parameters with a population value at least €1 million were identified as potential targets for research. RESULTS All partial estimates were determined within minutes. Thirty parameters (across nine models) each had a value of at least €1 million. These were categorised. Collectively, survival analysis parameters were valued at €19.32 million, health state utility parameters at €15.81 million and parameters associated with the cost of treating adverse effects at €6.64 million. Those associated with drug acquisition costs and with the cost of care were valued at €6.51 million and €5.71 million, respectively. CONCLUSION This research demonstrates that the estimation of partial expected value of perfect information via this computationally inexpensive approach could be considered feasible as part of the health technology assessment process for reimbursement purposes within the Irish healthcare system. It might be a useful tool in prioritising future research to decrease decision uncertainty.
Collapse
Affiliation(s)
- Laura McCullagh
- Department of Pharmacology and Therapeutics, Trinity College Dublin, Dublin, Ireland.
- National Centre for Pharmacoeconomics, St James's Hospital, Dublin, Ireland.
| | - Susanne Schmitz
- National Centre for Pharmacoeconomics, St James's Hospital, Dublin, Ireland
- Health Economics and Evidence Synthesis Research Unit, Department of Population Health, Luxembourg Institute of Health, Strassen, Luxembourg
| | - Michael Barry
- Department of Pharmacology and Therapeutics, Trinity College Dublin, Dublin, Ireland
- National Centre for Pharmacoeconomics, St James's Hospital, Dublin, Ireland
| | - Cathal Walsh
- Health Research Institute, University of Limerick, Limerick, Ireland
| |
Collapse
|