1
|
Glynn D, Gc VS, Claxton K, Littlewood C, Rothery C. Rapid Assessment of the Need for Evidence: Applying the Principles of Value of Information to Research Prioritisation. PHARMACOECONOMICS 2024; 42:919-928. [PMID: 38900241 DOI: 10.1007/s40273-024-01403-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 05/26/2024] [Indexed: 06/21/2024]
Abstract
We propose a short-cut heuristic approach to rapidly estimate value of information (VOI) using information commonly reported in a research funding application to make a case for the need for further evaluative research. We develop a "Rapid VOI" approach, which focuses on uncertainty in the primary outcome of clinical effectiveness and uses this to explore the health consequences of decision uncertainty. We develop a freely accessible online tool, Rapid Assessment of the Need for Evidence (RANE), to allow for the efficient computation of the value of research. As a case study, the method was applied to a proposal for research on shoulder pain rehabilitation. The analysis was included as part of a successful application for research funding to the UK National Institute for Health and Care Research. Our approach enables research funders and applicants to rapidly estimate the value of proposed research. Rapid VOI relies on information that is readily available and reported in research funding applications. Rapid VOI supports research prioritisation and commissioning decisions where there is insufficient time and resources available to develop and validate complex decision-analytic models. The method provides a practical means for implementing VOI in practice, thus providing a starting point for deliberation and contributing to the transparency and accountability of research prioritisation decisions.
Collapse
Affiliation(s)
- David Glynn
- Centre for Health Economics, University of York, York, UK.
| | - Vijay S Gc
- School of Human and Health Sciences, University of Huddersfield, Huddersfield, UK
| | - Karl Claxton
- Centre for Health Economics, University of York, York, UK
| | - Chris Littlewood
- Allied Health, Social Work & Wellbeing, Edgehill University, Ormskirk, UK
| | - Claire Rothery
- Centre for Health Economics, University of York, York, UK
| |
Collapse
|
2
|
Towse A, Fenwick E. It Takes 2 to Tango. Setting Out the Conditions in Which Performance-Based Risk-Sharing Arrangements Work for Both Parties. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:1058-1065. [PMID: 38615938 DOI: 10.1016/j.jval.2024.03.2196] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 03/20/2024] [Accepted: 03/25/2024] [Indexed: 04/16/2024]
Abstract
OBJECTIVES Faster regulatory approval processes often fail to achieve faster patient access. We seek an approach, using performance-based risk-sharing arrangements, to address uncertainty for payers regarding the relative effectiveness and value for money of products launched through accelerated approval schemes. One important reason for risk sharing is to resolve differences of opinion between innovators and payers about a technology's underlying value. To date, there has been no formal attempt to set out the circumstances in which risk sharing can address these differences. METHODS We use a value of information framework to understand what a performance-based risk-sharing arrangements can, in principle, add to a reimbursement scheme, separating payer perspectives on cost-effectiveness and the value of research from those of the innovator. We find 16 scenarios, developing 5 rules to analyze these 16 scenarios, identifying cases in which risk sharing adds value for both parties. RESULTS We find that risk sharing provides an improved solution in 9 out of 16 combinations of payer and innovator expectations about treatment outcome and the value of further research. Among our assumptions, who pays for research and scheme administration costs are key. CONCLUSIONS Steps should be undertaken to make risk sharing more practical, ensuring that payers consider it an option. This requires additional costs to the health system falling on the innovator in an efficient way that aligns incentives for product development for global markets. Health systems benefits are earlier patient access to cost-effective treatments and payers with higher confidence of not wasting money. Innovators get greater returns while conducting research.
Collapse
Affiliation(s)
- Adrian Towse
- Senior Visiting Fellow, Office of Health Economics, London, UK.
| | | |
Collapse
|
3
|
Yaesoubi R, Kunst N. Net Monetary Benefit Lines Augmented with Value-of-Information Measures to Present the Results of Economic Evaluations under Uncertainty. Med Decis Making 2024:272989X241262343. [PMID: 39056310 DOI: 10.1177/0272989x241262343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/28/2024]
Abstract
BACKGROUND Methods to present the result of cost-effectiveness analyses under parameter uncertainty include cost-effectiveness planes (CEPs), cost-effectiveness acceptability curves/frontier (CEACs/CEAF), expected loss curves (ELCs), and net monetary benefit (NMB) lines. We describe how NMB lines can be augmented to present NMB values that could be achieved by reducing or resolving parameter uncertainty. We evaluated the ability of these methods to correctly 1) identify the alternative with the highest expected NMB and 2) communicate the magnitude of parameter and decision uncertainty. METHODS We considered 4 hypothetical decision problems representing scenarios with high variance or correlated cost and effect estimates and alternatives with similar cost-effectiveness ratios. We used these decision problems to demonstrate the limitations of existing methods and the potential of augmented NMB lines to resolve these issues. RESULTS CEPs and CEACs/CEAF could falsely imply the lack of sufficient evidence to identify the optimal option if cost and effect estimates have high variance, are correlated across alternatives, or when alternatives have similar cost-effectiveness ratios. The augmented NMB lines and ELCs can correctly identify the option with the highest expected NMB and communicate the potential benefit of resolving uncertainties. Like ELCs, the augmented NMB lines provide information about the value of resolving parameter uncertainties, but augmented NMB lines may be easier to interpret for decision makers. CONCLUSIONS Our analysis supports recommending the augment NMB lines as an important method to present the results of economic evaluation studies under parameter uncertainty. HIGHLIGHTS The results of cost-effectiveness analyses (CEAs) when the cost and effect estimates of alternatives are uncertain are commonly presented using cost-effectiveness planes (CEPs), cost-effectiveness acceptability curves/frontier (CEACs/CEAF), and expected loss curves (ELCs).Although currently not often used, net monetary benefit (NMB) lines could present the results of cost-effectiveness to identify the alternative with the highest expected NMB values given the current level of uncertainty. Furthermore, NMB lines can be augmented to 1) show metrics of value of information, which measure the value of additional research to reduce or eliminate the decision uncertainty, and 2) display the confidence intervals along the NMB lines to ensure that NMB values are estimated accurately using a sufficiently large number of parameter samples.Using several decision problems, we demonstrate the limitation of existing methods to present the results of CEAs under parameter uncertainty and how augmented NMB lines could resolve these issues.Our analysis supports recommending augmented NMB lines as an important method to present the results of CEA under uncertainty since they 1) correctly identify the alternative with the highest expected NMB value given the current evidence, 2) provide information about the potential value of additional research to improve the decision by reducing or resolving uncertainty in model parameters, 3) assist the analysis to visually ensure that enough parameter samples are used to estimate the expected NMB of alternatives, and 4) are easier to interpret for decision makers compared with other methods.
Collapse
Affiliation(s)
- Reza Yaesoubi
- Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
- Public Health Modeling Unit, Yale School of Public Health, New Haven, CT, USA
| | - Natalia Kunst
- Department of Health Policy and Management, Yale School of Public Health, New Haven, CT, USA
- Public Health Modeling Unit, Yale School of Public Health, New Haven, CT, USA
- Centre for Health Economics, University of York, York, UK
| |
Collapse
|
4
|
Flight L, Brennan A, Wilson I, Chick SE. A Tutorial on Value-Based Adaptive Designs: Could a Value-Based Sequential 2-Arm Design Have Created More Health Economic Value for the Big CACTUS Trial? VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024:S1098-3015(24)02742-6. [PMID: 38977182 DOI: 10.1016/j.jval.2024.06.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 04/29/2024] [Accepted: 06/03/2024] [Indexed: 07/10/2024]
Abstract
OBJECTIVES Value-based trials aim to maximize the expected net benefit by balancing technology adoption decisions and clinical trial costs. Adaptive trials offer additional efficiency. This article provides guidance on determining whether a value-based sequential design is the best option for an adaptive 2-arm trial, illustrated through a case study. METHODS We outlined 4 steps for the value-based sequential approach. The case study re-evaluates the Big CACTUS trial design using pilot trial data and a model-based health economic analysis. Expected net benefit is computed for (1) original fixed design, (2) value-based design with fixed sample size, and (3) optimal value-based sequential design with adaptive stopping. We compare pretrial modeling with the actual Big CACTUS trial results. RESULTS Over 10 years, the adoption decision would affect approximately 215 378 patients. Pretrial modeling shows that the expected net benefit minus costs are (1) £102 million for the original fixed design, (2) £107 million (+5.3% higher) for the value-based design with optimal fixed sample size, and (3) £109 million (+6.7% higher) for the optimal value-based sequential design with maximum sample size of 435 per arm. Post hoc analysis using actual Big CACTUS trial data indicates that the value-adaptive trial with a maximum sample size of 95 participant pairs would not have stopped early. Bootstrap simulations reveal a 9.76% probability of early completion with n = 95 pairs compared with 31.50% with n = 435 pairs. CONCLUSIONS The 4-step approach to value-based sequential 2-arm design with adaptive stopping was successfully implemented. Further application of value-based adaptive approaches could be useful to assess the efficiency of alternative study designs.
Collapse
Affiliation(s)
- Laura Flight
- Sheffield Centre for Health and Related Research, The University of Sheffield, Sheffield, England, UK
| | - Alan Brennan
- Sheffield Centre for Health and Related Research, The University of Sheffield, Sheffield, England, UK
| | - Isabelle Wilson
- Sheffield Centre for Health and Related Research, The University of Sheffield, Sheffield, England, UK.
| | - Stephen E Chick
- INSEAD, Technology and Operations Management Area, Fontainebleau, France
| |
Collapse
|
5
|
O'Malley DM, Crabtree BF, Kaloth S, Ohman-Strickland P, Ferrante J, Hudson SV, Kinney AY. Strategic use of resources to enhance colorectal cancer screening for patients with diabetes (SURE: CRC4D) in federally qualified health centers: a protocol for hybrid type ii effectiveness-implementation trial. BMC PRIMARY CARE 2024; 25:242. [PMID: 38969987 PMCID: PMC11225128 DOI: 10.1186/s12875-024-02496-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2024] [Accepted: 06/26/2024] [Indexed: 07/07/2024]
Abstract
BACKGROUND Persons with diabetes have 27% elevated risk of developing colorectal cancer (CRC) and are disproportionately from priority health disparities populations. Federally qualified health centers (FQHCs) struggle to implement CRC screening programs for average risk patients. Strategies to effectively prioritize and optimize CRC screening for patients with diabetes in the primary care safety-net are needed. METHODS Guided by the Exploration, Preparation, Implementation and Sustainment Framework, we conducted a stakeholder-engaged process to identify multi-level change objectives for implementing optimized CRC screening for patients with diabetes in FQHCs. To identify change objectives, an implementation planning group of stakeholders from FQHCs, safety-net screening programs, and policy implementers were assembled and met over a 7-month period. Depth interviews (n = 18-20) with key implementation actors were conducted to identify and refine the materials, methods and strategies needed to support an implementation plan across different FQHC contexts. The planning group endorsed the following multi-component implementation strategies: identifying clinic champions, development/distribution of patient educational materials, developing and implementing quality monitoring systems, and convening clinical meetings. To support clinic champions during the initial implementation phase, two learning collaboratives and bi-weekly virtual facilitation will be provided. In single group, hybrid type 2 effectiveness-implementation trial, we will implement and evaluate these strategies in a in six safety net clinics (n = 30 patients with diabetes per site). The primary clinical outcomes are: (1) clinic-level colonoscopy uptake and (2) overall CRC screening rates for patients with diabetes assessed at baseline and 12-months post-implementation. Implementation outcomes include provider and staff fidelity to the implementation plan, patient acceptability, and feasibility will be assessed at baseline and 12-months post-implementation. DISCUSSION Study findings are poised to inform development of evidence-based implementation strategies to be tested for scalability and sustainability in a future hybrid 2 effectiveness-implementation clinical trial. The research protocol can be adapted as a model to investigate the development of targeted cancer prevention strategies in additional chronically ill priority populations. TRIAL REGISTRATION This study was registered in ClinicalTrials.gov (NCT05785780) on March 27, 2023 (last updated October 21, 2023).
Collapse
Affiliation(s)
- Denalee M O'Malley
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA.
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA.
| | - Benjamin F Crabtree
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Srivarsha Kaloth
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA
| | - Pamela Ohman-Strickland
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Biostatistics and Epidemiology, School of Public Health, Rutgers University, New Brunswick, NJ, USA
| | - Jeanne Ferrante
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Shawna V Hudson
- Department of Family Medicine and Community Health, Research Division, Rutgers Robert Wood Johnson Medical School, 303 George Street, Rm 309, New Brunswick, NJ, USA
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
| | - Anita Y Kinney
- Rutgers Cancer Institute of New Jersey, New Brunswick, NJ, USA
- Department of Biostatistics and Epidemiology, School of Public Health, Rutgers University, New Brunswick, NJ, USA
| |
Collapse
|
6
|
Dijk SW, Krijkamp E, Kunst N, Labrecque JA, Gross CP, Pandit A, Lu CP, Visser LE, Wong JB, Hunink MGM. Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information. Med Decis Making 2024; 44:512-528. [PMID: 38828516 PMCID: PMC11283736 DOI: 10.1177/0272989x241255047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 04/07/2024] [Indexed: 06/05/2024]
Abstract
BACKGROUND The COVID-19 pandemic underscored the criticality and complexity of decision making for novel treatment approval and further research. Our study aims to assess potential decision-making methodologies, an evaluation vital for refining future public health crisis responses. METHODS We compared 4 decision-making approaches to drug approval and research: the Food and Drug Administration's policy decisions, cumulative meta-analysis, a prospective value-of-information (VOI) approach (using information available at the time of decision), and a reference standard (retrospective VOI analysis using information available in hindsight). Possible decisions were to reject, accept, provide emergency use authorization, or allow access to new therapies only in research settings. We used monoclonal antibodies provided to hospitalized COVID-19 patients as a case study, examining the evidence from September 2020 to December 2021 and focusing on each method's capacity to optimize health outcomes and resource allocation. RESULTS Our findings indicate a notable discrepancy between policy decisions and the reference standard retrospective VOI approach with expected losses up to $269 billion USD, suggesting suboptimal resource use during the wait for emergency use authorization. Relying solely on cumulative meta-analysis for decision making results in the largest expected loss, while the policy approach showed a loss up to $16 billion and the prospective VOI approach presented the least loss (up to $2 billion). CONCLUSION Our research suggests that incorporating VOI analysis may be particularly useful for research prioritization and treatment implementation decisions during pandemics. While the prospective VOI approach was favored in this case study, further studies should validate the ideal decision-making method across various contexts. This study's findings not only enhance our understanding of decision-making strategies during a health crisis but also provide a potential framework for future pandemic responses. HIGHLIGHTS This study reviews discrepancies between a reference standard (retrospective VOI, using hindsight information) and 3 conceivable real-time approaches to research-treatment decisions during a pandemic, suggesting suboptimal use of resources.Of all prospective decision-making approaches considered, VOI closely mirrored the reference standard, yielding the least expected value loss across our study timeline.This study illustrates the possible benefit of VOI results and the need for evidence accumulation accompanied by modeling in health technology assessment for emerging therapies.
Collapse
Affiliation(s)
- Stijntje W. Dijk
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
- Department of Radiology and Nuclear Medicine, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Eline Krijkamp
- Erasmus School of Health Policy & Management, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Natalia Kunst
- Centre for Health Economics, University of York, York, UK
- Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University School of Medicine, New Haven, CT, USA
| | - Jeremy A. Labrecque
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Cary P. Gross
- Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University School of Medicine, New Haven, CT, USA
| | - Aradhana Pandit
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Chia-Ping Lu
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Loes E. Visser
- Department of Hospital Pharmacy, Erasmus University Medical Center, Rotterdam, The Netherlands
- Hospital Pharmacy, Haga Teaching Hospital, The Hague, The Netherlands
| | - John B. Wong
- Division of Clinical Decision Making, Tufts Medical Center, Boston, USA
| | - M. G. Myriam Hunink
- Department of Epidemiology, Erasmus University Medical Center, Rotterdam, The Netherlands
- Department of Radiology and Nuclear Medicine, Erasmus University Medical Center, Rotterdam, The Netherlands
- Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| |
Collapse
|
7
|
Edney LC, Pellizzer ML. Adaptive design trials in eating disorder research: A scoping review. Int J Eat Disord 2024; 57:1278-1290. [PMID: 38619362 DOI: 10.1002/eat.24198] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Revised: 03/01/2024] [Accepted: 03/01/2024] [Indexed: 04/16/2024]
Abstract
OBJECTIVE This scoping review sought to map the breadth of literature on the use of adaptive design trials in eating disorder research. METHOD A systematic literature search was conducted in Medline, Scopus, PsycInfo, Emcare, Econlit, CINAHL and ProQuest Dissertations and Theses. Articles were included if they reported on an intervention targeting any type of eating disorder (including anorexia nervosa, bulimia nervosa, binge-eating disorder, and other specified feeding or eating disorders), and employed the use of an adaptive design trial to evaluate the intervention. Two independent reviewers screened citations for inclusion, and data abstraction was performed by one reviewer and verified by a second. RESULTS We identified five adaptive design trials targeting anorexia nervosa, bulimia nervosa and binge-eating disorder conducted in the USA and Australia. All employed adaptive treatment arm switching based on early response to treatment and identified a priori stopping rules. None of the studies included value of information analysis to guide adaptive design decisions and none included lived experience perspectives. DISCUSSION The limited use of adaptive designs in eating disorder trials represents a missed opportunity to improve enrolment targets, attrition rates, treatment outcomes and trial efficiency. We outline the range of adaptive methodologies, how they could be applied to eating disorder research, and the specific operational and statistical considerations relevant to adaptive design trials. PUBLIC SIGNIFICANCE Adaptive design trials are increasingly employed as flexible, efficient alternatives to fixed trial designs, but they are not often used in eating disorder research. This first scoping review identified five adaptive design trials targeting anorexia nervosa, bulimia nervosa and binge-eating disorder that employed treatment arm switching adaptive methodology. We make recommendations on the use of adaptive design trials for future eating disorder trials.
Collapse
Affiliation(s)
- Laura C Edney
- Flinders University Institute for Mental Health and Wellbeing, Flinders University, Adelaide, South Australia, Australia
| | - Mia L Pellizzer
- Flinders University Institute for Mental Health and Wellbeing, Flinders University, Adelaide, South Australia, Australia
- Blackbird Initiative, Flinders University, Adelaide, Australia
| |
Collapse
|
8
|
Heath A, Baio G, Manolopoulou I, Welton NJ. Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators. PHARMACOECONOMICS 2024; 42:479-486. [PMID: 38583100 DOI: 10.1007/s40273-024-01372-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/05/2024] [Indexed: 04/08/2024]
Abstract
Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.
Collapse
Affiliation(s)
- Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada.
- Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.
- Department of Statistical Science, University College London, London, UK.
| | - Gianluca Baio
- Department of Statistical Science, University College London, London, UK
| | | | - Nicky J Welton
- Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
9
|
Kunst N, Burger EA, Coupé VMH, Kuntz KM, Aas E. A Guide to an Iterative Approach to Model-Based Decision Making in Health and Medicine: An Iterative Decision-Making Framework. PHARMACOECONOMICS 2024; 42:363-371. [PMID: 38157129 DOI: 10.1007/s40273-023-01341-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/16/2023] [Indexed: 01/03/2024]
Abstract
Decision makers frequently face decisions about optimal resource allocation. A model-based economic evaluation can be used to guide decision makers in their choices by systematically evaluating the magnitude of expected health effects and costs of decision options and by making trade-offs explicit. We provide a guide to an iterative approach to the medical decision-making process by following a coherent framework, and outline the overarching iterative steps of model-based decision making. We systematized the framework by performing three steps. First, we compiled the existing guidelines provided by the ISPOR-SMDM Modeling Good Research Practices Task Force, and the ISPOR Value of Information Task Force. Second, we identified other previous work related to frameworks and guidelines for model-based decision analyses through a literature search in PubMed. Third, we assessed the role of the evidence and iterative process in decision making and formalized key steps in a model-based decision-making framework. We provide guidance on an iterative approach to medical decision making by applying the compiled iterative model-based decision-making framework. The framework formally combines the decision problem conceptualization (Part I), the model conceptualization and development (Part II), and the process of model-based decision analysis (Part III). Following the overarching steps of the framework ensures compliance to the principles of evidence-based medicine and regular updates of the evidence, given that value of information analysis represents an essential component of model-based decision analysis in the framework. Following the provided guide and the steps outlined in the framework can help inform various health care decisions, and therefore it has the potential to improve decision making.
Collapse
Affiliation(s)
- Natalia Kunst
- Centre for Health Economics, University of York, Heslington, York, YO10 5DD, UK.
- Public Health Modeling Unit, Yale School of Public Health, New Haven, CT, USA.
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway.
| | - Emily A Burger
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway
- Center for Health Decision Science, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Veerle M H Coupé
- Department of Epidemiology and Data Science, Amsterdam University Medical Centers, Amsterdam, The Netherlands
| | - Karen M Kuntz
- Division of Health Policy and Management, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Eline Aas
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway
- Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway
| |
Collapse
|
10
|
Keeney E, Elwenspoek MMC, Jackson J, Roadevin C, Jones HE, O'Donnell R, Sheppard AL, Dawson S, Lane D, Stubbs J, Everitt H, Watson JC, Hay AD, Gillett P, Robins G, Mallett S, Whiting PF, Thom H. Identifying the Optimum Strategy for Identifying Adults and Children With Celiac Disease: A Cost-Effectiveness and Value of Information Analysis. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2024; 27:301-312. [PMID: 38154593 DOI: 10.1016/j.jval.2023.12.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 11/08/2023] [Accepted: 12/11/2023] [Indexed: 12/30/2023]
Abstract
OBJECTIVES Celiac disease (CD) is thought to affect around 1% of people in the United Kingdom, but only approximately 30% are diagnosed. The aim of this work was to assess the cost-effectiveness of strategies for identifying adults and children with CD in terms of who to test and which tests to use. METHODS A decision tree and Markov model were used to describe testing strategies and model long-term consequences of CD. The analysis compared a selection of pre-test probabilities of CD above which patients should be screened, as well as the use of different serological tests, with or without genetic testing. Value of information analysis was used to prioritize parameters for future research. RESULTS Using serological testing alone in adults, immunoglobulin A (IgA) tissue transglutaminase (tTG) at a 1% pre-test probability (equivalent to population screening) was most cost-effective. If combining serological testing with genetic testing, human leukocyte antigen combined with IgA tTG at a 5% pre-test probability was most cost-effective. In children, the most cost-effective strategy was a 10% pre-test probability with human leukocyte antigen plus IgA tTG. Value of information analysis highlighted the probability of late diagnosis of CD and the accuracy of serological tests as important parameters. The analysis also suggested prioritizing research in adult women over adult men or children. CONCLUSIONS For adults, these cost-effectiveness results suggest UK National Screening Committee Criteria for population-based screening for CD should be explored. Substantial uncertainty in the results indicate a high value in conducting further research.
Collapse
Affiliation(s)
- Edna Keeney
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK.
| | - Martha M C Elwenspoek
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK; The National Institute for Health Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol NHS Foundation Trust, Bristol, England, UK
| | - Joni Jackson
- The National Institute for Health Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol NHS Foundation Trust, Bristol, England, UK
| | - Cristina Roadevin
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | - Hayley E Jones
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | - Rachel O'Donnell
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK; The National Institute for Health Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol NHS Foundation Trust, Bristol, England, UK
| | - Athena L Sheppard
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK; The National Institute for Health Research Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol NHS Foundation Trust, Bristol, England, UK; Swansea University Medical School, Swansea University, Swansea, England, UK
| | - Sarah Dawson
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | | | | | - Hazel Everitt
- Primary Care Research Centre, Population Sciences and Medical Education, University of Southampton, Southampton, England, UK
| | - Jessica C Watson
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | - Alastair D Hay
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | - Peter Gillett
- Paediatric Gastroenterology, Hepatology and Nutrition Department, Royal Hospital for Sick Children, Edinburgh EH9 1LF Scotland, England, UK
| | - Gerry Robins
- Department of Gastroenterology, York Teaching Hospital NHS Foundation Trust, York, England, UK
| | - Sue Mallett
- Centre for Medical Imaging, University College London, London, England, UK
| | - Penny F Whiting
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| | - Howard Thom
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, England, UK
| |
Collapse
|
11
|
Robinson SA, Moy ML, Ney JP. Value of Information Analysis of a Web-Based Self-Management Intervention for Chronic Obstructive Pulmonary Disease. Telemed J E Health 2024; 30:518-526. [PMID: 37615601 PMCID: PMC10877383 DOI: 10.1089/tmj.2023.0010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Revised: 06/14/2023] [Accepted: 06/19/2023] [Indexed: 08/25/2023] Open
Abstract
Objective: Technology-based programs can be cost-effective in the management of chronic obstructive pulmonary disease (COPD). However, cost-effectiveness estimates always contain some uncertainty, and decisions based upon them carry some risk. We conducted a value of information (VOI) analysis to estimate the value of additional research of a web-based self-management intervention for COPD to reduce the costs associated with uncertainty. Methods: We used a 10,000-iteration cost-effectiveness model from the health care payer perspective to calculate the expected value of perfect information (EVPI) at the patient- and population-level. An opportunity loss was incurred when the web-based intervention did not produce a greater net monetary benefit than usual care in an iteration. We calculated the probability of opportunity loss and magnitude of opportunity costs as a function of baseline health utility. We aggregated opportunity costs over the projected incident population of inpatient COPD patients over 10 years and estimated it as a function of the willingness-to-pay (WTP) threshold. Costs are in 2022 U.S. Dollars. Results: Opportunity losses were found in 22.7% of the iterations. The EVPIpatient was $78 per patient (95% confidence interval: $75-$82). The probability that the intervention was the optimal strategy varied across baseline health utilities. The EVPIpopulation was $506,666,882 over 10 years for a WTP of $50,000. Conclusions: Research estimated to cost up to $500 million would be warranted to reduce uncertainty. Future research could focus on identifying the impact of baseline health utilities to maximize the cost savings of the intervention. Other considerations for future research priorities include implementation efforts for technology-based interventions.
Collapse
Affiliation(s)
- Stephanie A. Robinson
- Center for Healthcare Organization and Implementation Research, VA Bedford Healthcare System, Bedford, Massachusetts, USA
- The Pulmonary Center, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Marilyn L. Moy
- Pulmonary and Critical Care Medicine Section, VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Medicine, Harvard Medical School, Boston, Massachusetts, USA
| | - John P. Ney
- VA Connecticut Healthcare System, West Haven, Connecticut, USA
- Department of Neurology, Boston University School of Medicine, Boston, Massachusetts, USA
| |
Collapse
|
12
|
Kunst N, Siu A, Drummond M, Grimm S, Grutters J, Husereau D, Koffijberg H, Rothery C, Wilson ECF, Heath A. Reporting Economic Evaluations with Value of Information Analyses Using the CHEERS Value of Information (CHEERS-VOI) Reporting Guideline. Med Decis Making 2024; 44:127-128. [PMID: 38097383 DOI: 10.1177/0272989x231214791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/15/2024]
Affiliation(s)
- Natalia Kunst
- Centre for Health Economics, University of York, York, UK
- Yale University School of Public Health, New Haven, CT, USA
| | - Annisa Siu
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
| | | | - Sabine Grimm
- Department of Epidemiology and Medical Technology Assessment (KEMTA), Maastricht Health Economics and Technology Assessment (Maastricht HETA) Center, Maastricht University Medical Center, Maastricht, Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud Institute for Health Sciences, Radboudumc, Nijmegen, The Netherlands
| | - Don Husereau
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
- Institute of Health Economics, Edmonton, AB, Canada
| | - Hendrik Koffijberg
- Department of Health Technology & Services Research, TechMed Centre, University of Twente, Enschede, the Netherlands
| | - Claire Rothery
- Centre for Health Economics, University of York, York, UK
| | - Edward C F Wilson
- Peninsula Technology Assessment Group, University of Exeter, Exeter, UK
| | - Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
- Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Department of Statistical Science, University College London, London, UK
| |
Collapse
|
13
|
Jiao B. Estimating the Potential Benefits of Confirmatory Trials for Drugs with Accelerated Approval: A Comprehensive Value of Information Framework. PHARMACOECONOMICS 2023; 41:1617-1627. [PMID: 37490206 DOI: 10.1007/s40273-023-01303-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 07/04/2023] [Indexed: 07/26/2023]
Abstract
BACKGROUND The US Food and Drug Administration's Accelerated Approval (AA) policy provides a pathway for patients to access potentially life-saving drugs rapidly. However, the use of surrogate endpoints, single-arm designs, and small sample sizes in preliminary trials that support AAs can lead to uncertainty regarding the clinical benefits of such drugs. This study aims to develop a comprehensive value of information (VOI) framework for assessing the potential benefits of future confirmatory trials, accounting for the various uncertainties inherent in preliminary trials. METHODS I formulated an expected value of information from confirmatory trial (EVICT) metric, which evaluates the potential benefits of a confirmatory trial that would reduce those uncertainties by using a clinically meaningful endpoint, a randomized control, and increased sample size. The EVICT metric can quantify the expected benefits of a well-designed confirmatory trial or an inadequately designed one that continues to use surrogate endpoints or single-arm design. The framework was illustrated using a hypothetical AA drug for metastatic breast cancer. RESULTS The case study demonstrates that a highly uncertain preliminary trial of an AA drug was associated with a substantial EVICT. A confirmatory trial with an increased sample size for this AA drug, utilizing a clinically meaningful endpoint and randomized control, yielded a population-level EVICT of $12.6 million. Persistently using a surrogate endpoint and single-arm trial design would reduce the EVICT by 60%. CONCLUSIONS This framework can provide accurate VOI estimates to guide coverage policies, value-based pricing, and the design of confirmatory trials for AA drugs.
Collapse
Affiliation(s)
- Boshen Jiao
- Harvard T.H. Chan School of Public Health, 90 Smith St, Boston, MA, 02120, USA.
| |
Collapse
|
14
|
Grimm SE, Pouwels XGLV, Ramaekers BLT, Wijnen B, Grutters J, Joore MA. Response to "UNCERTAINTY MANAGEMENT IN REGULATORY AND HEALTH TECHNOLOGY ASSESSMENT DECISION-MAKING ON DRUGS: GUIDANCE OF THE HTAi-DIA WORKING GROUP". Int J Technol Assess Health Care 2023; 39:e70. [PMID: 37822085 DOI: 10.1017/s026646232300260x] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/13/2023]
Affiliation(s)
- Sabine Elisabeth Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| | - Xavier G L V Pouwels
- Department of Health Technology and Services Research, University of Twente, Enschede, The Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| | - Ben Wijnen
- Trimbos-instituut, Utrecht, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud University Medical Centre, Nijmegen, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre and Maastricht Health Economics and Technology Assessment Centre, School for Public Health and Primary Care (CAPHRI), Maastricht, The Netherlands
| |
Collapse
|
15
|
Kunst N, Siu A, Drummond M, Grimm SE, Grutters J, Husereau D, Koffijberg H, Rothery C, Wilson ECF, Heath A. Consolidated Health Economic Evaluation Reporting Standards - Value of Information (CHEERS-VOI): Explanation and Elaboration. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023; 26:1461-1473. [PMID: 37414276 DOI: 10.1016/j.jval.2023.06.014] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Revised: 05/27/2023] [Accepted: 06/20/2023] [Indexed: 07/08/2023]
Abstract
OBJECTIVES Although the ISPOR Value of Information (VOI) Task Force's reports outline VOI concepts and provide good-practice recommendations, there is no guidance for reporting VOI analyses. VOI analyses are usually performed alongside economic evaluations for which the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) 2022 Statement provides reporting guidelines. Thus, we developed the CHEERS-VOI checklist to provide reporting guidance and checklist to support the transparent, reproducible, and high-quality reporting of VOI analyses. METHODS A comprehensive literature review generated a list of 26 candidate reporting items. These candidate items underwent a Delphi procedure with Delphi participants through 3 survey rounds. Participants rated each item on a 9-point Likert scale to indicate its relevance when reporting the minimal, essential information about VOI methods and provided comments. The Delphi results were reviewed at 2-day consensus meetings and the checklist was finalized using anonymous voting. RESULTS We had 30, 25, and 24 Delphi respondents in rounds 1, 2, and 3, respectively. After incorporating revisions recommended by the Delphi participants, all 26 candidate items proceeded to the 2-day consensus meetings. The final CHEERS-VOI checklist includes all CHEERS items, but 7 items require elaboration when reporting VOI. Further, 6 new items were added to report information relevant only to VOI (eg, VOI methods applied). CONCLUSIONS The CHEERS-VOI checklist should be used when a VOI analysis is performed alongside economic evaluations. The CHEERS-VOI checklist will help decision makers, analysts and peer reviewers in the assessment and interpretation of VOI analyses and thereby increase transparency and rigor in decision making.
Collapse
Affiliation(s)
- Natalia Kunst
- Centre for Health Economics, University of York, York, England, UK; Yale University School of Public Health, New Haven, CT, USA.
| | - Annisa Siu
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Michael Drummond
- Centre for Health Economics, University of York, York, England, UK
| | - Sabine E Grimm
- Department of Epidemiology and Medical Technology Assessment (KEMTA), Maastricht Health Economics and Technology Assessment (Maastricht HETA) Center, Maastricht University Medical Center, Maastricht, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud Institute for Health Sciences, Radboudumc, Nijmegen, The Netherlands
| | - Don Husereau
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada and Institute of Health Economics, Edmonton, Alberta, Canada
| | - Hendrik Koffijberg
- Department of Health Technology & Services Research, TechMed Centre, University of Twente, Enschede, The Netherlands
| | - Claire Rothery
- Centre for Health Economics, University of York, York, England, UK
| | - Edward C F Wilson
- Peninsula Technology Assessment Group, University of Exeter, Exeter, England, UK
| | - Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada; Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada; Department of Statistical Science, University College London, London, England, UK
| |
Collapse
|
16
|
Qureshi N, Woods B, Neves de Faria R, Saramago Goncalves P, Cox E, Leonardi Bee J, Condon L, Weng S, Akyea RK, Iyen B, Roderick P, Humphries SE, Rowlands W, Watson M, Haralambos K, Kenny R, Datta D, Miedzybrodzka Z, Byrne C, Kai J. Alternative cascade-testing protocols for identifying and managing patients with familial hypercholesterolaemia: systematic reviews, qualitative study and cost-effectiveness analysis. Health Technol Assess 2023; 27:1-140. [PMID: 37924278 PMCID: PMC10658348 DOI: 10.3310/ctmd0148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2023] Open
Abstract
Background Cascade testing the relatives of people with familial hypercholesterolaemia is an efficient approach to identifying familial hypercholesterolaemia. The cascade-testing protocol starts with identifying an index patient with familial hypercholesterolaemia, followed by one of three approaches to contact other relatives: indirect approach, whereby index patients contact their relatives; direct approach, whereby the specialist contacts the relatives; or a combination of both direct and indirect approaches. However, it is unclear which protocol may be most effective. Objectives The objectives were to determine the yield of cases from different cascade-testing protocols, treatment patterns, and short- and long-term outcomes for people with familial hypercholesterolaemia; to evaluate the cost-effectiveness of alternative protocols for familial hypercholesterolaemia cascade testing; and to qualitatively assess the acceptability of different cascade-testing protocols to individuals and families with familial hypercholesterolaemia, and to health-care providers. Design and methods This study comprised systematic reviews and analysis of three data sets: PASS (PASS Software, Rijswijk, the Netherlands) hospital familial hypercholesterolaemia databases, the Clinical Practice Research Datalink (CPRD)-Hospital Episode Statistics (HES) linked primary-secondary care data set, and a specialist familial hypercholesterolaemia register. Cost-effectiveness modelling, incorporating preceding analyses, was undertaken. Acceptability was examined in interviews with patients, relatives and health-care professionals. Result Systematic review of protocols: based on data from 4 of the 24 studies, the combined approach led to a slightly higher yield of relatives tested [40%, 95% confidence interval (CI) 37% to 42%] than the direct (33%, 95% CI 28% to 39%) or indirect approaches alone (34%, 95% CI 30% to 37%). The PASS databases identified that those contacted directly were more likely to complete cascade testing (p < 0.01); the CPRD-HES data set indicated that 70% did not achieve target treatment levels, and demonstrated increased cardiovascular disease risk among these individuals, compared with controls (hazard ratio 9.14, 95% CI 8.55 to 9.76). The specialist familial hypercholesterolaemia register confirmed excessive cardiovascular morbidity (standardised morbidity ratio 7.17, 95% CI 6.79 to 7.56). Cost-effectiveness modelling found a net health gain from diagnosis of -0.27 to 2.51 quality-adjusted life-years at the willingness-to-pay threshold of £15,000 per quality-adjusted life-year gained. The cost-effective protocols cascaded from genetically confirmed index cases by contacting first- and second-degree relatives simultaneously and directly. Interviews found a service-led direct-contact approach was more reliable, but combining direct and indirect approaches, guided by index patients and family relationships, may be more acceptable. Limitations Systematic reviews were not used in the economic analysis, as relevant studies were lacking or of poor quality. As only a proportion of those with primary care-coded familial hypercholesterolaemia are likely to actually have familial hypercholesterolaemia, CPRD analyses are likely to underestimate the true effect. The cost-effectiveness analysis required assumptions related to the long-term cardiovascular disease risk, the effect of treatment on cholesterol and the generalisability of estimates from the data sets. Interview recruitment was limited to white English-speaking participants. Conclusions Based on limited evidence, most cost-effective cascade-testing protocols, diagnosing most relatives, select index cases by genetic testing, with services directly contacting relatives, and contacting second-degree relatives even if first-degree relatives have not been tested. Combined approaches to contact relatives may be more suitable for some families. Future work Establish a long-term familial hypercholesterolaemia cohort, measuring cholesterol levels, treatment and cardiovascular outcomes. Conduct a randomised study comparing different approaches to contact relatives. Study registration This study is registered as PROSPERO CRD42018117445 and CRD42019125775. Funding This project was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 27, No. 16. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Nadeem Qureshi
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Bethan Woods
- Centre for Health Economics, University of York, York, UK
| | | | | | - Edward Cox
- Centre for Health Economics, University of York, York, UK
| | - Jo Leonardi Bee
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Laura Condon
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Stephen Weng
- Cardiovascular and Metabolism, Janssen Research and Development, High Wycombe, UK
| | - Ralph K Akyea
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Barbara Iyen
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Paul Roderick
- Primary Care, Population Sciences and Medical Education, University of Southampton, Southampton, UK
| | - Steve E Humphries
- Centre for Cardiovascular Genetics, Institute for Cardiovascular Science, University College London, London, UK
| | | | - Melanie Watson
- Wessex Clinical Genetics Service, University Hospital Southampton NHS Foundation Trust, Southampton, UK
| | - Kate Haralambos
- Familial Hypercholesterolaemia Service, University Hospital of Wales, Cardiff, UK
| | - Ryan Kenny
- Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Dev Datta
- Lipid Unit, University Hospital Llandough, Penarth, UK
| | | | - Christopher Byrne
- Southampton National Institute for Health and Care Research Biomedical Research Centre, University Hospital Southampton NHS Foundation Trust, Southampton, UK
| | - Joe Kai
- PRISM Research Group, Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| |
Collapse
|
17
|
Pei PP, Fitzmaurice KP, Le MH, Panella C, Jones ML, Pandya A, Horsburgh CR, Freedberg KA, Weinstein MC, Paltiel AD, Reddy KP. The Value-of-Information and Value-of-Implementation from Clinical Trials of Diagnostic Tests for HIV-Associated Tuberculosis: A Modeling Analysis. MDM Policy Pract 2023; 8:23814683231198873. [PMID: 37743931 PMCID: PMC10517616 DOI: 10.1177/23814683231198873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Accepted: 07/27/2023] [Indexed: 09/26/2023] Open
Abstract
Objectives. Conventional value-of-information (VOI) analysis assumes complete uptake of an optimal decision. We employed an extended framework that includes value-of-implementation (VOM)-the benefit of encouraging adoption of an optimal strategy-and estimated how future trials of diagnostic tests for HIV-associated tuberculosis could improve public health decision making and clinical and economic outcomes. Methods. We evaluated the clinical outcomes and costs, given current information, of 3 tuberculosis screening strategies among hospitalized people with HIV in South Africa: sputum Xpert (Xpert), sputum Xpert plus urine AlereLAM (Xpert+AlereLAM), and sputum Xpert plus the newer, more sensitive, and costlier urine FujiLAM (Xpert+FujiLAM). We projected the incremental net monetary benefit (INMB) of decision making based on results of a trial comparing mortality with each strategy, rather than decision making based solely on current knowledge of FujiLAM's improved diagnostic performance. We used a validated microsimulation to estimate VOI (the INMB of reducing parameter uncertainty before decision making) and VOM (the INMB of encouraging adoption of an optimal strategy). Results. With current information, adopting Xpert+FujiLAM yields 0.4 additional life-years/person compared with current practices (assumed 50% Xpert and 50% Xpert+AlereLAM). While the decision to adopt this optimal strategy is unaffected by information from the clinical trial (VOI = $ 0 at $3,000/year-of-life saved willingness-to-pay threshold), there is value in scaling up implementation of Xpert+FujiLAM, which results in an INMB (representing VOM) of $650 million over 5 y. Conclusions. Conventional VOI methods account for the value of switching to a new optimal strategy based on trial data but fail to account for the persuasive value of trials in increasing uptake of the optimal strategy. Evaluation of trials should include a focus on their value in reducing barriers to implementation. Highlights In conventional VOI analysis, it is assumed that the optimal decision will always be adopted even without a trial. This can potentially lead to an underestimation of the value of trials when adoption requires new clinical trial evidence. To capture the influence that a trial may have on decision makers' willingness to adopt the optimal decision, we also consider value-of-implementation (VOM), a metric quantifying the benefit of new study information in promoting wider adoption of the optimal strategy. The overall value-of-a-trial (VOT) includes both VOI and VOM.Our model-based analysis suggests that the information obtained from a trial of screening strategies for HIV-associated tuberculosis in South Africa would have no value, when measured using traditional methods of VOI assessment. A novel strategy, which includes the urine FujiLAM test, is optimal from a health economic standpoint but is underutilized. A trial would reduce uncertainties around downstream health outcomes but likely would not change the optimal decision. The high VOT (nearly $700 million over 5 y) lies solely in promoting uptake of FujiLAM, represented as VOM.Our results highlight the importance of employing a more comprehensive approach for evaluating prospective trials, as conventional VOI methods can vastly underestimate their value. Trialists and funders can and should assess the VOT metric instead when considering trial designs and costs. If VOI is low, the VOM and cost of a trial can be compared with the benefits and costs of other outreach programs to determine the most cost-effective way to improve uptake.
Collapse
Affiliation(s)
- Pamela P. Pei
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | | | - Mylinh H. Le
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Christopher Panella
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Michelle L. Jones
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
| | - Ankur Pandya
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - C. Robert Horsburgh
- School of Public Health and School of Medicine, Boston University, Boston, MA, USA
| | - Kenneth A. Freedberg
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- Division of Infectious Diseases, Massachusetts General Hospital, Boston, MA, USA
| | - Milton C. Weinstein
- Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - A. David Paltiel
- Public Health Modeling Unit, Yale School of Public Health, New Haven, CT, USA
| | - Krishna P. Reddy
- Medical Practice Evaluation Center, Massachusetts General Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
- Division of Pulmonary and Critical Care Medicine, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
18
|
Sagoo GS, Robinson T, Coughlan D, Meader N, Rice S, Vale L. Evaluating high-cost technologies - no need to throw the baby out with the bathwater. Expert Rev Pharmacoecon Outcomes Res 2023; 23:1177-1183. [PMID: 37755333 DOI: 10.1080/14737167.2023.2263647] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Accepted: 09/22/2023] [Indexed: 09/28/2023]
Abstract
INTRODUCTION Evidence generation for the health technology assessment (HTA) of a new technology is a long and expensive process with no guarantees that the health technology will be adopted and implemented into a health-care system. This would suggest that there is a greater risk of failure for a company developing a high-cost technology and therefore incentives (such as increasing the funding available for research or additional market exclusivity) may be needed to encourage development of such technologies as has been seen with many high-cost orphan drugs. AREAS COVERED This paper discusses some of the key issues relating to the evaluation of high-cost technologies through the use of existing HTA processes and what the challenges will be going forward. EXPERT OPINION We propose that while the current HTA process is robust, its evolution into accommodating the incorporation of real-world data and evidence alongside a life-cycle HTA approach should better enable developers to produce the evidence required on effectiveness and cost-effectiveness. This should lead to reduced decision uncertainty for HTA agencies to make adoption decisions in a more timely and efficient manner. Furthermore, budget impact analysis remains important in understanding the actual financial impact on health-care systems and budgets outside of the cost-effectiveness framework used to aid decision-making.
Collapse
Affiliation(s)
- Gurdeep S Sagoo
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| | - Tomos Robinson
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| | - Diarmuid Coughlan
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| | - Nick Meader
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| | - Stephen Rice
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| | - Luke Vale
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, UK
| |
Collapse
|
19
|
Breeze PR, Squires H, Ennis K, Meier P, Hayes K, Lomax N, Shiell A, Kee F, de Vocht F, O’Flaherty M, Gilbert N, Purshouse R, Robinson S, Dodd PJ, Strong M, Paisley S, Smith R, Briggs A, Shahab L, Occhipinti J, Lawson K, Bayley T, Smith R, Boyd J, Kadirkamanathan V, Cookson R, Hernandez‐Alava M, Jackson CH, Karapici A, Sassi F, Scarborough P, Siebert U, Silverman E, Vale L, Walsh C, Brennan A. Guidance on the use of complex systems models for economic evaluations of public health interventions. HEALTH ECONOMICS 2023; 32:1603-1625. [PMID: 37081811 PMCID: PMC10947434 DOI: 10.1002/hec.4681] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Revised: 03/13/2023] [Accepted: 03/14/2023] [Indexed: 05/03/2023]
Abstract
To help health economic modelers respond to demands for greater use of complex systems models in public health. To propose identifiable features of such models and support researchers to plan public health modeling projects using these models. A working group of experts in complex systems modeling and economic evaluation was brought together to develop and jointly write guidance for the use of complex systems models for health economic analysis. The content of workshops was informed by a scoping review. A public health complex systems model for economic evaluation is defined as a quantitative, dynamic, non-linear model that incorporates feedback and interactions among model elements, in order to capture emergent outcomes and estimate health, economic and potentially other consequences to inform public policies. The guidance covers: when complex systems modeling is needed; principles for designing a complex systems model; and how to choose an appropriate modeling technique. This paper provides a definition to identify and characterize complex systems models for economic evaluations and proposes guidance on key aspects of the process for health economics analysis. This document will support the development of complex systems models, with impact on public health systems policy and decision making.
Collapse
Affiliation(s)
- Penny R. Breeze
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | - Hazel Squires
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | - Kate Ennis
- British Medical Journal Technology Appraisal GroupLondonUK
| | - Petra Meier
- MRC/CSO Social and Public Health Sciences UnitUniversity of GlasgowScotlandUK
| | - Kate Hayes
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | - Nik Lomax
- School of GeographyUniversity of LeedsLeedsUK
| | - Alan Shiell
- Department of Public HealthLaTrobe UniversityMelbourneAustralia
| | - Frank Kee
- Centre for Public HealthQueen's University BelfastBelfastUK
| | - Frank de Vocht
- Population Health SciencesBristol Medical SchoolUniversity of BristolBristolUK
- NIHR Applied Research Collaboration West (ARC West)BristolUK
| | - Martin O’Flaherty
- Department of Public Health, Policy and SystemsUniversity of LiverpoolLiverpoolUK
| | | | - Robin Purshouse
- Department of Automatic Control and Systems EngineeringUniversity of SheffieldSheffieldUK
| | | | - Peter J Dodd
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | - Mark Strong
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | | | - Richard Smith
- College of Medicine and HealthUniversity of ExeterExeterUK
| | - Andrew Briggs
- London School of Hygiene & Tropical MedicineLondonUK
| | - Lion Shahab
- Department of Behavioural Science and HealthUCLLondonUK
| | - Jo‐An Occhipinti
- Brain and Mind CentreUniversity of SydneyNew South WalesCamperdownAustralia
| | - Kenny Lawson
- Brain and Mind CentreUniversity of SydneyNew South WalesCamperdownAustralia
| | | | - Robert Smith
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| | - Jennifer Boyd
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
- MRC/CSO Social and Public Health Sciences UnitUniversity of GlasgowGlasgowUK
| | | | | | | | | | - Amanda Karapici
- NIHR SPHRLondon School of Hygiene and Tropical MedicineLondonUK
| | - Franco Sassi
- Centre for Health Economics & Policy InnovationImperial College Business SchoolLondonUK
| | - Peter Scarborough
- Nuffield Department of Population HealthUniversity of OxfordOxfordshireOxfordUK
| | - Uwe Siebert
- Department of Public Health, Health Services Research and Health Technology AssessmentUMIT TIROL ‐ University for Health Sciences and TechnologyHall in TirolTyrolAustria
- Division of Health Technology Assessment and BioinformaticsONCOTYROL ‐ Center for Personalized Cancer MedicineInnsbruckAustria
- Center for Health Decision ScienceDepartments of Epidemiology and Health Policy & ManagementHarvard T.H. Chan School of Public HealthMassachusettsBostonUSA
- Program on Cardiovascular Research, Institute for Technology Assessment and Department of RadiologyMassachusetts General HospitalHarvard Medical SchoolMassachusettsBostonUSA
| | - Eric Silverman
- MRC/CSO Social and Public Health Sciences UnitUniversity of GlasgowGlasgowUK
| | - Luke Vale
- Health Economics GroupPopulation Health Sciences InstituteNewcastle UniversityNewcastleUK
| | - Cathal Walsh
- Health Research Institute and MACSIUniversity of LimerickLimerickIreland
| | - Alan Brennan
- School of Health and Related ResearchUniversity of SheffieldSheffieldUK
| |
Collapse
|
20
|
Otten TM, Grimm SE, Ramaekers B, Joore MA. Comprehensive Review of Methods to Assess Uncertainty in Health Economic Evaluations. PHARMACOECONOMICS 2023; 41:619-632. [PMID: 36943674 PMCID: PMC10163110 DOI: 10.1007/s40273-023-01242-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/11/2023] [Indexed: 05/06/2023]
Abstract
Uncertainty assessment is a cornerstone in model-based health economic evaluations (HEEs) that inform reimbursement decisions. No comprehensive overview of available uncertainty assessment methods currently exists. We aimed to review methods for uncertainty assessment for use in model-based HEEs, by conducting a snowballing review. We categorised all methods according to their stage of use relating to uncertainty assessment (identification, analysis, communication). Additionally, we classified identification methods according to sources of uncertainty, and subdivided analysis and communication methods according to their purpose. The review identified a total of 80 uncertainty methods: 30 identification, 28 analysis, and 22 communication methods. Uncertainty identification methods exist to address uncertainty from different sources. Most identification methods were developed with the objective to assess related concepts such as validity, model quality, and relevance. Almost all uncertainty analysis and communication methods required uncertainty to be quantified and inclusion of uncertainties in probabilistic analysis. Our review can help analysts and decision makers in selecting uncertainty assessment methods according to their aim and purpose of the assessment. We noted a need for further clarification of terminology and guidance on the use of (combinations of) methods to identify uncertainty and related concepts such as validity and quality. A key finding is that uncertainty assessment relies heavily on quantification, which may necessitate increased use of expert elicitation and/or the development of methods to assess unquantified uncertainty.
Collapse
Affiliation(s)
- Thomas Michael Otten
- Department of Clinical Epidemiology and Medical Technology Assessment (KEMTA), P. Debyelaan 25, Oxford Building, PO Box 5800a, Maastricht, Limburg, The Netherlands.
| | - Sabine E Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment (KEMTA), P. Debyelaan 25, Oxford Building, PO Box 5800a, Maastricht, Limburg, The Netherlands
| | - Bram Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment (KEMTA), P. Debyelaan 25, Oxford Building, PO Box 5800a, Maastricht, Limburg, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment (KEMTA), P. Debyelaan 25, Oxford Building, PO Box 5800a, Maastricht, Limburg, The Netherlands
| |
Collapse
|
21
|
Huckvale K, Hoon L, Stech E, Newby JM, Zheng WY, Han J, Vasa R, Gupta S, Barnett S, Senadeera M, Cameron S, Kurniawan S, Agarwal A, Kupper JF, Asbury J, Willie D, Grant A, Cutler H, Parkinson B, Ahumada-Canale A, Beames JR, Logothetis R, Bautista M, Rosenberg J, Shvetcov A, Quinn T, Mackinnon A, Rana S, Tran T, Rosenbaum S, Mouzakis K, Werner-Seidler A, Whitton A, Venkatesh S, Christensen H. Protocol for a bandit-based response adaptive trial to evaluate the effectiveness of brief self-guided digital interventions for reducing psychological distress in university students: the Vibe Up study. BMJ Open 2023; 13:e066249. [PMID: 37116996 PMCID: PMC10151864 DOI: 10.1136/bmjopen-2022-066249] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 04/30/2023] Open
Abstract
INTRODUCTION Meta-analytical evidence confirms a range of interventions, including mindfulness, physical activity and sleep hygiene, can reduce psychological distress in university students. However, it is unclear which intervention is most effective. Artificial intelligence (AI)-driven adaptive trials may be an efficient method to determine what works best and for whom. The primary purpose of the study is to rank the effectiveness of mindfulness, physical activity, sleep hygiene and an active control on reducing distress, using a multiarm contextual bandit-based AI-adaptive trial method. Furthermore, the study will explore which interventions have the largest effect for students with different levels of baseline distress severity. METHODS AND ANALYSIS The Vibe Up study is a pragmatically oriented, decentralised AI-adaptive group sequential randomised controlled trial comparing the effectiveness of one of three brief, 2-week digital self-guided interventions (mindfulness, physical activity or sleep hygiene) or active control (ecological momentary assessment) in reducing self-reported psychological distress in Australian university students. The adaptive trial methodology involves up to 12 sequential mini-trials that allow for the optimisation of allocation ratios. The primary outcome is change in psychological distress (Depression, Anxiety and Stress Scale, 21-item version, DASS-21 total score) from preintervention to postintervention. Secondary outcomes include change in physical activity, sleep quality and mindfulness from preintervention to postintervention. Planned contrasts will compare the four groups (ie, the three intervention and control) using self-reported psychological distress at prespecified time points for interim analyses. The study aims to determine the best performing intervention, as well as ranking of other interventions. ETHICS AND DISSEMINATION Ethical approval was sought and obtained from the UNSW Sydney Human Research Ethics Committee (HREC A, HC200466). A trial protocol adhering to the requirements of the Guideline for Good Clinical Practice was prepared for and approved by the Sponsor, UNSW Sydney (Protocol number: HC200466_CTP). TRIAL REGISTRATION NUMBER ACTRN12621001223820.
Collapse
Affiliation(s)
- Kit Huckvale
- Centre for Digital Transformation of Health, University of Melbourne, Melbourne, Victoria, Australia
| | - Leonard Hoon
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Eileen Stech
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Jill M Newby
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
- School of Psychology, UNSW Sydney, Sydney, New South Wales, Australia
| | - Wu Yi Zheng
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Jin Han
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Rajesh Vasa
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Sunil Gupta
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Scott Barnett
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Manisha Senadeera
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Stuart Cameron
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Stefanus Kurniawan
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Akash Agarwal
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Joost Funke Kupper
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Joshua Asbury
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - David Willie
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Alasdair Grant
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Henry Cutler
- Centre for the Health Economy, Macquarie University, Sydney, New South Wales, Australia
| | - Bonny Parkinson
- Centre for the Health Economy, Macquarie University, Sydney, New South Wales, Australia
| | | | - Joanne R Beames
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Rena Logothetis
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Marya Bautista
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Jodie Rosenberg
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Artur Shvetcov
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Thomas Quinn
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Andrew Mackinnon
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Santu Rana
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Truyen Tran
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Simon Rosenbaum
- School of Psychiatry, UNSW Sydney, Sydney, New South Wales, Australia
| | - Kon Mouzakis
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | | | - Alexis Whitton
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| | - Svetha Venkatesh
- Applied Artificial Intelligence Institute, Deakin University, Melbourne, Victoria, Australia
| | - Helen Christensen
- Black Dog Institute, UNSW Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
22
|
Hazen G, Borgonovo E, Lu X. Information Density in Decision Analysis. DECISION ANALYSIS 2023. [DOI: 10.1287/deca.2022.0465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
Information value has been proposed and used as a probabilistic sensitivity measure, the idea being that uncertain parameters having higher information value are precisely those to which an optimal decision is more sensitive. In this paper, we study the notion of information density as a graphical complement to information value analysis, one that augments an information value calculation with associated directions of information gain. We formally examine mathematical details absent from its earlier presentation that guarantee information density exists and is well posed and describe its relationship to alternate measures of information value. We present its application in the context of a realistic case study and discuss the associated insights.
Collapse
Affiliation(s)
- Gordon Hazen
- Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208
| | - Emanuele Borgonovo
- Bocconi Institute for Data Science and Analytics, 20136 Milan, Italy
- Department of Decision Sciences, Bocconi University, 20136 Milan, Italy
| | - Xuefei Lu
- SKEMA Business School, Université Côte d’Azur, Paris, France
| |
Collapse
|
23
|
Yuan F, Bangdiwala SI, Tong W, Lamy A. The impact of statistical properties of incremental monetary net benefit and incremental cost-effectiveness ratio on health economic modeling choices. Expert Rev Pharmacoecon Outcomes Res 2023; 23:69-78. [PMID: 36334614 DOI: 10.1080/14737167.2023.2144838] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
INTRODUCTION There is controversy on whether to use incremental monetary net benefit (INMB) or incremental cost-effectiveness ratio (ICER) in health economic evaluations alongside randomized controlled trials. We studied the impact of restricted mean survival time (RMST) on the long-term projection of INMB and ICER. METHODS We analyzed the unbiasedness and efficiency of ICER and INMB by (1) deriving the metrics' expected values and variances based on theoretical probability distributions, (2) simulating their 15-year post-trial projections based on between-arm-RMST-gained through a 2 × 4 × 2 factorial experiment of Markov 2-state microsimulations. Simulations and comparison were run on the data from the Cardiovascular Outcomes for People Using Anticoagulation Strategies Study (COMPASS). RESULTS Our simulation findings using RMST showed that ICER was more efficient than INMB, regardless of disease populations, time horizon, modeling choices, and underlying probability distributions of incremental mean cost and effect. ICER had a small variance and thus showed its robustness to the choices of models. CONCLUSION INMB's variance varies with a willingness-to-pay (WTP) threshold quadratically while ICER's variance with a WTP threshold value quadratically while ICER's variance with incremental-mean-cost quadratically. A simple and naïve model can sufficiently estimate ICER. Future metrics are expected to be health-economic-meaningful, unambiguous, unbiased, efficient, and statistical-inference-friendly.
Collapse
Affiliation(s)
- Fei Yuan
- Department of Statistics, Population Health Research Institute, Hamilton, ON, Canada.,Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Shrikant I Bangdiwala
- Department of Statistics, Population Health Research Institute, Hamilton, ON, Canada.,Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Wesley Tong
- Department of Perioperative and Surgery, Population Health Research Institute, Hamilton, ON, Canada
| | - Andre Lamy
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.,Department of Perioperative and Surgery, Population Health Research Institute, Hamilton, ON, Canada.,Hamilton Health Sciences, Hamilton, ON, Canada
| |
Collapse
|
24
|
Simons MJHG, Uyl-de Groot CA, Retèl VP, Mankor JM, Ramaekers BLT, Joore MA, van Harten WH. Cost-Effectiveness and Budget Impact of Future Developments With Whole-Genome Sequencing for Patients With Lung Cancer. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2023; 26:71-80. [PMID: 35973926 DOI: 10.1016/j.jval.2022.07.006] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/27/2021] [Revised: 06/19/2022] [Accepted: 07/08/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES This study aimed to investigate the cost-effectiveness, budget impact (BI), and impact of uncertainty of future developments concerning whole-genome sequencing (WGS) as a clinical diagnostic test compared with standard of care (SoC) in patients with locally advanced and metastatic non-small cell lung cancer. METHODS A total of 3 likely scenarios to take place within 5 years (according to experts) were simulated using a previously developed, peer reviewed, and published decision model. The scenarios concerned "WGS results used for treatment selection" (scenario 1), "WGS-based biomarker for immunotherapy" (scenario 2), and "off-label drug approval for WGS results" (scenario 3). Two diagnostic strategies of the original model, "SoC" and "WGS as a diagnostic test" (base model), were used to compare our scenarios with. Outcomes were reported for the base model, all scenarios separately, combined (combined unweighted), and weighted by likelihood (combined weighted). Cost-effectiveness, BI, and value of information analyses were performed for WGS compared with SoC. RESULTS Total costs and quality-adjusted life-years for SoC in metastatic non-small cell lung cancer were €149 698 and 1.235. Incremental outcomes of WGS were €1529/0.002(base model), -€222/0.020(scenario 1), -€2576/0.023(scenario 2), €388/0.024(scenario 3), -€5041/0.060(combined unweighted), and -€1715/0.029(combined weighted). The annual BI for adopting WGS for this population in The Netherlands ranged between €682 million (combined unweighted) and €714 million (base model). The consequences of uncertainty amounted to €3.4 million for all scenarios (combined weighted) and to €699 000 for the diagnostic yield of WGS alone (combined weighted). CONCLUSIONS Our findings suggest that it is likely for WGS to become cost-effective within the near future if it identifies more patients with actionable targets and show the impact of uncertainty regarding its diagnostic yield. Modeling future scenarios can be useful to consider early adoption of WGS while timely anticipating on unforeseen developments before final conclusions are reached.
Collapse
Affiliation(s)
- Martijn J H G Simons
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands; Care And Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Carin A Uyl-de Groot
- Erasmus School of Health Policy and Management/Institute for Medical Technology Assessment, Erasmus University Rotterdam, Rotterdam, The Netherlands
| | - Valesca P Retèl
- Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute-Antoni van Leeuwenhoek hospital, Amsterdam, The Netherlands; Department of Health Technology and Services Research, University of Twente, Enschede, The Netherlands
| | - Joanne M Mankor
- Department of Pulmonary Medicine, Erasmus Medical Centre, Rotterdam, The Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands; Care And Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands; Care And Public Health Research Institute, Maastricht University, Maastricht, The Netherlands.
| | - Wim H van Harten
- Department of Psychosocial Research and Epidemiology, Netherlands Cancer Institute-Antoni van Leeuwenhoek hospital, Amsterdam, The Netherlands; Department of Health Technology and Services Research, University of Twente, Enschede, The Netherlands
| |
Collapse
|
25
|
Naylor NR, Williams J, Green N, Lamrock F, Briggs A. Extensions of Health Economic Evaluations in R for Microsoft Excel Users: A Tutorial for Incorporating Heterogeneity and Conducting Value of Information Analyses. PHARMACOECONOMICS 2023; 41:21-32. [PMID: 36437359 PMCID: PMC9702728 DOI: 10.1007/s40273-022-01203-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Accepted: 09/29/2022] [Indexed: 06/16/2023]
Abstract
Advanced health economic analysis techniques currently performed in Microsoft Excel, such as incorporating heterogeneity, time-dependent transitions and a value of information analysis, can be easily transferred to R. Often the outputs of survival analyses (such as Weibull regression models) will estimate the impacts of correlated patient characteristics on patient outcomes, and are utilised directly as inputs for health economic decision models. This tutorial provides a step-by-step guide of how to conduct such analyses with a Markov model developed in R, and offers a comparison with established analyses performed in Microsoft Excel. This is done through the conversion of a previously published Microsoft Excel case study of a hip replacement surgery cost-effectiveness model. We hope that this paper can act as a facilitator in switching decision models from Microsoft Excel to R for complex health economic analyses, providing open-access code and data, suitable for future adaptation.
Collapse
Affiliation(s)
- Nichola R Naylor
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK.
- HCAI, Fungal, AMR, AMU and Sepsis Division, UK Health Security Agency, London, UK.
| | - Jack Williams
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK
| | - Nathan Green
- Department of Statistical Science, University College London, London, UK
| | - Felicity Lamrock
- Mathematical Sciences Research Centre, Queen's University Belfast, Belfast, UK
| | - Andrew Briggs
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, London, UK
| |
Collapse
|
26
|
Morton RL, Tuffaha H, Blaya-Novakova V, Spencer J, Hawley CM, Peyton P, Higgins A, Marsh J, Taylor WJ, Huckson S, Sillett A, Schneemann K, Balagurunanthan A, Cumpston M, Scuffham PA, Glasziou P, Simes RJ. Approaches to prioritising research for clinical trial networks: a scoping review. Trials 2022; 23:1000. [PMID: 36510214 PMCID: PMC9743749 DOI: 10.1186/s13063-022-06928-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 11/15/2022] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Prioritisation of clinical trials ensures that the research conducted meets the needs of stakeholders, makes the best use of resources and avoids duplication. The aim of this review was to identify and critically appraise approaches to research prioritisation applicable to clinical trials, to inform best practice guidelines for clinical trial networks and funders. METHODS A scoping review of English-language published literature and research organisation websites (January 2000 to January 2020) was undertaken to identify primary studies, approaches and criteria for research prioritisation. Data were extracted and tabulated, and a narrative synthesis was employed. RESULTS Seventy-eight primary studies and 18 websites were included. The majority of research prioritisation occurred in oncology and neurology disciplines. The main reasons for prioritisation were to address a knowledge gap (51 of 78 studies [65%]) and to define patient-important topics (28 studies, [35%]). In addition, research organisations prioritised in order to support their institution's mission, invest strategically, and identify best return on investment. Fifty-seven of 78 (73%) studies used interpretative prioritisation approaches (including Delphi surveys, James Lind Alliance and consensus workshops); six studies used quantitative approaches (8%) such as prospective payback or value of information (VOI) analyses; and 14 studies used blended approaches (18%) such as nominal group technique and Child Health Nutritional Research Initiative. Main criteria for prioritisation included relevance, appropriateness, significance, feasibility and cost-effectiveness. CONCLUSION Current research prioritisation approaches for groups conducting and funding clinical trials are largely interpretative. There is an opportunity to improve the transparency of prioritisation through the inclusion of quantitative approaches.
Collapse
Affiliation(s)
- Rachael L. Morton
- grid.1013.30000 0004 1936 834XNational Health and Medical Research Council Clinical Trials Centre (NHMRC CTC), University of Sydney, Sydney, Australia
| | - Haitham Tuffaha
- grid.1003.20000 0000 9320 7537Centre for the Business and Economics of Health, University of Queensland, Brisbane, Australia
| | - Vendula Blaya-Novakova
- grid.1013.30000 0004 1936 834XNational Health and Medical Research Council Clinical Trials Centre (NHMRC CTC), University of Sydney, Sydney, Australia
| | - Jenean Spencer
- Australian Clinical Trials Alliance (ACTA), Melbourne, Victoria Australia
| | - Carmel M. Hawley
- grid.1003.20000 0000 9320 7537Australasian Kidney Trials Network (AKTN), Faculty of Medicine, University of Queensland, Brisbane, Australia
| | - Phil Peyton
- grid.418175.e0000 0001 2225 7841Australian and New Zealand College of Anaesthetists (ANZCA), Melbourne, Australia
| | - Alisa Higgins
- grid.1002.30000 0004 1936 7857Australian and New Zealand Intensive Care Research Centre (ANZIC-RC), Monash University, Melbourne, Victoria Australia
| | - Julie Marsh
- grid.414659.b0000 0000 8828 1230Telethon Kids Institute, West Perth, Australia
| | - William J. Taylor
- grid.29980.3a0000 0004 1936 7830University of Otago, Rehabilitation Teaching and Research Unit, Dunedin, New Zealand
| | - Sue Huckson
- grid.489411.10000 0004 5905 1670Australian and New Zealand Intensive Care Society (ANZICS), Camberwell, Victoria Australia
| | - Amy Sillett
- grid.467202.50000 0004 0445 3920AstraZeneca Australia, Macquarie Park, New South Wales Australia
| | - Kieran Schneemann
- Australian Clinical Trials Alliance (ACTA), Melbourne, Victoria Australia ,grid.467202.50000 0004 0445 3920AstraZeneca Australia, Macquarie Park, New South Wales Australia
| | | | - Miranda Cumpston
- Australian Clinical Trials Alliance (ACTA), Melbourne, Victoria Australia ,grid.266842.c0000 0000 8831 109XSchool of Medicine and Public Health, The University of Newcastle, Newcastle, Australia
| | - Paul A. Scuffham
- grid.1003.20000 0000 9320 7537Centre for the Business and Economics of Health, University of Queensland, Brisbane, Australia
| | - Paul Glasziou
- grid.1033.10000 0004 0405 3820Faculty of Health Sciences & Medicine, Bond University, Gold Coast, Australia
| | - Robert J. Simes
- grid.1013.30000 0004 1936 834XNational Health and Medical Research Council Clinical Trials Centre (NHMRC CTC), University of Sydney, Sydney, Australia
| |
Collapse
|
27
|
Borre ED, Myers ER, Dubno JR, Emmett SD, Pavon JM, Francis HW, Ogbuoji O, Sanders Schmidler GD. Estimated Monetary Value of Future Research Clarifying Uncertainties Around the Optimal Adult Hearing Screening Schedule. JAMA HEALTH FORUM 2022; 3:e224065. [PMID: 36367737 PMCID: PMC9652748 DOI: 10.1001/jamahealthforum.2022.4065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Accepted: 09/14/2022] [Indexed: 11/13/2022] Open
Abstract
Importance Adult hearing screening is not routinely performed, and most individuals with hearing loss (HL) have never had their hearing tested as adults. Objective To project the monetary value of future research clarifying uncertainties around the optimal adult hearing screening schedule. Design, Setting, and Participants In this economic evaluation, a validated decision model of HL (DeciBHAL-US: Decision model of the Burden of Hearing loss Across the Lifespan) was used to simulate current detection and treatment of HL vs hearing screening schedules. Key model inputs included HL incidence (0.06%-10.42%/y), hearing aid uptake (0.54%-8.14%/y), screening effectiveness (1.62 × hearing aid uptake), utility benefits of hearing aids (+0.11), and hearing aid device costs ($3690). Distributions to model parameters for probabilistic uncertainty analysis were assigned. The expected value of perfect information (EVPI) and expected value of partial perfect information (EVPPI) using a willingness to pay of $100 000 per quality-adjusted life-year (QALY) was estimated. The EVPI and EVPPI estimate the upper bound of the dollar value of future research. This study was based on 40-year-old persons over their remaining lifetimes in a US primary care setting. Exposures Screening schedules beginning at ages 45, 55, 65, and 75 years, and frequencies of every 1 or 5 years. Main Outcomes and Measures The main outcomes were QALYs and costs (2020 US dollars) from a health system perspective. Results The average incremental cost-effectiveness ratio for yearly screening beginning at ages 55 to 75 years ranged from $39 200 to $80 200/QALY. Yearly screening beginning at age 55 years was the optimal screening schedule in 38% of probabilistic uncertainty analysis simulations. The population EVPI, or value of reducing all uncertainty, was $8.2 to $12.6 billion varying with willingness to pay and the EVPPI, or value of reducing all screening effectiveness uncertainty, was $2.4 billion. Conclusions and Relevance In this economic evaluation of US adult hearing screening, large uncertainty around the optimal adult hearing screening schedule was identified. Future research on hearing screening has a high potential value so is likely justified.
Collapse
Affiliation(s)
- Ethan D. Borre
- Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina
- Duke-Margolis Center for Health Policy, Duke University, Durham, North Carolina
| | - Evan R. Myers
- Division of Women’s Community and Population Health, Department of Obstetrics & Gynecology, Duke University School of Medicine, Durham, North Carolina
| | - Judy R. Dubno
- Department of Otolaryngology–Head and Neck Surgery, Medical University of South Carolina, Charleston, South Carolina
| | - Susan D. Emmett
- Department of Head and Neck Surgery and Communication Sciences, Duke University School of Medicine, Durham, North Carolina
- Duke Global Health Institute, Duke University, Durham, North Carolina
| | - Juliessa M. Pavon
- Division of Geriatrics, Department of Medicine, Duke University School of Medicine, Durham, North Carolina
| | - Howard W. Francis
- Department of Head and Neck Surgery and Communication Sciences, Duke University School of Medicine, Durham, North Carolina
| | - Osondu Ogbuoji
- Duke Global Health Institute, Duke University, Durham, North Carolina
- Center for Policy Impact in Global Health, Duke Global Health Institute, Durham, North Carolina
| | - Gillian D. Sanders Schmidler
- Department of Population Health Sciences, Duke University School of Medicine, Durham, North Carolina
- Duke-Margolis Center for Health Policy, Duke University, Durham, North Carolina
- Duke Clinical Research Institute, Duke University School of Medicine, Durham, North Carolina
| |
Collapse
|
28
|
Elwenspoek MM, Thom H, Sheppard AL, Keeney E, O'Donnell R, Jackson J, Roadevin C, Dawson S, Lane D, Stubbs J, Everitt H, Watson JC, Hay AD, Gillett P, Robins G, Jones HE, Mallett S, Whiting PF. Defining the optimum strategy for identifying adults and children with coeliac disease: systematic review and economic modelling. Health Technol Assess 2022; 26:1-310. [PMID: 36321689 PMCID: PMC9638887 DOI: 10.3310/zuce8371] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022] Open
Abstract
BACKGROUND Coeliac disease is an autoimmune disorder triggered by ingesting gluten. It affects approximately 1% of the UK population, but only one in three people is thought to have a diagnosis. Untreated coeliac disease may lead to malnutrition, anaemia, osteoporosis and lymphoma. OBJECTIVES The objectives were to define at-risk groups and determine the cost-effectiveness of active case-finding strategies in primary care. DESIGN (1) Systematic review of the accuracy of potential diagnostic indicators for coeliac disease. (2) Routine data analysis to develop prediction models for identification of people who may benefit from testing for coeliac disease. (3) Systematic review of the accuracy of diagnostic tests for coeliac disease. (4) Systematic review of the accuracy of genetic tests for coeliac disease (literature search conducted in April 2021). (5) Online survey to identify diagnostic thresholds for testing, starting treatment and referral for biopsy. (6) Economic modelling to identify the cost-effectiveness of different active case-finding strategies, informed by the findings from previous objectives. DATA SOURCES For the first systematic review, the following databases were searched from 1997 to April 2021: MEDLINE® (National Library of Medicine, Bethesda, MD, USA), Embase® (Elsevier, Amsterdam, the Netherlands), Cochrane Library, Web of Science™ (Clarivate™, Philadelphia, PA, USA), the World Health Organization International Clinical Trials Registry Platform ( WHO ICTRP ) and the National Institutes of Health Clinical Trials database. For the second systematic review, the following databases were searched from January 1990 to August 2020: MEDLINE, Embase, Cochrane Library, Web of Science, Kleijnen Systematic Reviews ( KSR ) Evidence, WHO ICTRP and the National Institutes of Health Clinical Trials database. For prediction model development, Clinical Practice Research Datalink GOLD, Clinical Practice Research Datalink Aurum and a subcohort of the Avon Longitudinal Study of Parents and Children were used; for estimates for the economic models, Clinical Practice Research Datalink Aurum was used. REVIEW METHODS For review 1, cohort and case-control studies reporting on a diagnostic indicator in a population with and a population without coeliac disease were eligible. For review 2, diagnostic cohort studies including patients presenting with coeliac disease symptoms who were tested with serological tests for coeliac disease and underwent a duodenal biopsy as reference standard were eligible. In both reviews, risk of bias was assessed using the quality assessment of diagnostic accuracy studies 2 tool. Bivariate random-effects meta-analyses were fitted, in which binomial likelihoods for the numbers of true positives and true negatives were assumed. RESULTS People with dermatitis herpetiformis, a family history of coeliac disease, migraine, anaemia, type 1 diabetes, osteoporosis or chronic liver disease are 1.5-2 times more likely than the general population to have coeliac disease; individual gastrointestinal symptoms were not useful for identifying coeliac disease. For children, women and men, prediction models included 24, 24 and 21 indicators of coeliac disease, respectively. The models showed good discrimination between patients with and patients without coeliac disease, but performed less well when externally validated. Serological tests were found to have good diagnostic accuracy for coeliac disease. Immunoglobulin A tissue transglutaminase had the highest sensitivity and endomysial antibody the highest specificity. There was little improvement when tests were used in combination. Survey respondents (n = 472) wanted to be 66% certain of the diagnosis from a blood test before starting a gluten-free diet if symptomatic, and 90% certain if asymptomatic. Cost-effectiveness analyses found that, among adults, and using serological testing alone, immunoglobulin A tissue transglutaminase was most cost-effective at a 1% pre-test probability (equivalent to population screening). Strategies using immunoglobulin A endomysial antibody plus human leucocyte antigen or human leucocyte antigen plus immunoglobulin A tissue transglutaminase with any pre-test probability had similar cost-effectiveness results, which were also similar to the cost-effectiveness results of immunoglobulin A tissue transglutaminase at a 1% pre-test probability. The most practical alternative for implementation within the NHS is likely to be a combination of human leucocyte antigen and immunoglobulin A tissue transglutaminase testing among those with a pre-test probability above 1.5%. Among children, the most cost-effective strategy was a 10% pre-test probability with human leucocyte antigen plus immunoglobulin A tissue transglutaminase, but there was uncertainty around the most cost-effective pre-test probability. There was substantial uncertainty in economic model results, which means that there would be great value in conducting further research. LIMITATIONS The interpretation of meta-analyses was limited by the substantial heterogeneity between the included studies, and most included studies were judged to be at high risk of bias. The main limitations of the prediction models were that we were restricted to diagnostic indicators that were recorded by general practitioners and that, because coeliac disease is underdiagnosed, it is also under-reported in health-care data. The cost-effectiveness model is a simplification of coeliac disease and modelled an average cohort rather than individuals. Evidence was weak on the probability of routine coeliac disease diagnosis, the accuracy of serological and genetic tests and the utility of a gluten-free diet. CONCLUSIONS Population screening with immunoglobulin A tissue transglutaminase (1% pre-test probability) and of immunoglobulin A endomysial antibody followed by human leucocyte antigen testing or human leucocyte antigen testing followed by immunoglobulin A tissue transglutaminase with any pre-test probability appear to have similar cost-effectiveness results. As decisions to implement population screening cannot be made based on our economic analysis alone, and given the practical challenges of identifying patients with higher pre-test probabilities, we recommend that human leucocyte antigen combined with immunoglobulin A tissue transglutaminase testing should be considered for adults with at least a 1.5% pre-test probability of coeliac disease, equivalent to having at least one predictor. A more targeted strategy of 10% pre-test probability is recommended for children (e.g. children with anaemia). FUTURE WORK Future work should consider whether or not population-based screening for coeliac disease could meet the UK National Screening Committee criteria and whether or not it necessitates a long-term randomised controlled trial of screening strategies. Large prospective cohort studies in which all participants receive accurate tests for coeliac disease are needed. STUDY REGISTRATION This study is registered as PROSPERO CRD42019115506 and CRD42020170766. FUNDING This project was funded by the National Institute for Health and Care Research ( NIHR ) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 44. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Martha Mc Elwenspoek
- National Institute for Health and Care Research Applied Research Collaboration West, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Howard Thom
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Athena L Sheppard
- National Institute for Health and Care Research Applied Research Collaboration West, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
- Department of Health Sciences, University of Leicester, Leicester, UK
| | - Edna Keeney
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Rachel O'Donnell
- National Institute for Health and Care Research Applied Research Collaboration West, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Joni Jackson
- National Institute for Health and Care Research Applied Research Collaboration West, University Hospitals Bristol NHS Foundation Trust, Bristol, UK
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Cristina Roadevin
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Sarah Dawson
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | | | | | - Hazel Everitt
- Primary Care Research Centre, Population Sciences and Medical Education, University of Southampton, Southampton, UK
| | - Jessica C Watson
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Alastair D Hay
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Peter Gillett
- Paediatric Gastroenterology, Hepatology and Nutrition Department, Royal Hospital for Sick Children, Edinburgh, UK
| | - Gerry Robins
- Department of Gastroenterology, York Teaching Hospital NHS Foundation Trust, York, UK
| | - Hayley E Jones
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Sue Mallett
- Centre for Medical Imaging, University College London, London, UK
| | - Penny F Whiting
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| |
Collapse
|
29
|
Veličković V, Janković D. Challenges around quantifying uncertainty in a holistic approach to hard-to-heal wound management: Health economic perspective. Int Wound J 2022; 20:792-798. [PMID: 36073595 PMCID: PMC9927906 DOI: 10.1111/iwj.13924] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2022] [Accepted: 07/27/2022] [Indexed: 11/29/2022] Open
Abstract
Treatment of hard-to-heal wounds involves a holistic approach for choosing between available treatment options. However, evidence for informing these choices is sparse, introducing uncertainty into decisions about the optimum treatment pathways that reflect the vast heterogeneity in this patient population. This paper discusses the existing clinical and health economic literature in order to provide insight into sources of uncertainty in the evaluation of the holistic approach to management of the hard-to-heal wounds, and how this uncertainty can be appropriately reflected in research. We identified three key sources of uncertainty in the evaluation of chronic wound treatments, namely heterogeneity in aetiology and patient populations, heterogeneity in treatment pathways, and challenges around capturing all relevant outcomes. Reflecting these complexities requires sophisticated modelling of treatment sequencing and long-term outcomes. The paper discusses how the scope specification, scenario analyses, and sensitivity analyses can be used to fully characterise analytical uncertainty.
Collapse
Affiliation(s)
- Vladica Veličković
- Health Economics and Outcome ResearchHartmann GroupHeidenheimGermany,Institute of Public HealthMedical Decision Making and HTA, UMITHall in TirolAustria
| | | |
Collapse
|
30
|
Drummond M, Federici C, Reckers‐Droog V, Torbica A, Blankart CR, Ciani O, Kaló Z, Kovács S, Brouwer W. Coverage with evidence development for medical devices in Europe: Can practice meet theory? HEALTH ECONOMICS 2022; 31 Suppl 1:179-194. [PMID: 35220644 PMCID: PMC9545598 DOI: 10.1002/hec.4478] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 12/26/2021] [Accepted: 01/12/2022] [Indexed: 06/14/2023]
Abstract
Health economists have written extensively on the design and implementation of coverage with evidence development (CED) schemes and have proposed theoretical frameworks based on cost-effectiveness modeling and value of information analysis. CED may aid decision-makers when there is uncertainty about the (cost-)effectiveness of a new health technology at the time of reimbursement. Medical devices are potential candidates for CED schemes, as regulatory regimes do not usually require the same level of efficacy and safety data normally needed for pharmaceuticals. The purpose of this research is to assess whether the actual practice of CED for medical devices in Europe meets the theoretical principles proposed by health economists and whether theory and practice can be more closely aligned. Based on decision-makers' perceptions of the challenges associated with CED schemes, plus examples from the schemes themselves, we discuss a series of proposals for assessing the desirability of schemes, their design, implementation, and evaluation. These proposals, while reflecting the practical challenges with developing CED programs, embody many of the principles suggested by economists and should support decision-makers in dealing with uncertainty about the real-world performance of devices.
Collapse
Affiliation(s)
| | - Carlo Federici
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
- School of EngineeringUniversity of WarwickCoventryUK
| | - Vivian Reckers‐Droog
- Erasmus School of Health Policy & ManagementErasmus UniversityRotterdamThe Netherlands
| | - Aleksandra Torbica
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
| | - Carl Rudolf Blankart
- Kompetenzzentrum für Public ManagementUniversität BernBernSwitzerland
- Swiss Institute for Translational and Entrepreneurial MedicineBernSwitzerland
| | - Oriana Ciani
- Centre for Research on Health and Social Care Management (CERGAS)Universitá BocconiMilanItaly
| | - Zoltán Kaló
- Syreon Research InstituteBudapestHungary
- Centre for Health Technology AssessmentSemmelweis UniversityBudapestHungary
| | | | - Werner Brouwer
- Erasmus School of Health Policy & ManagementErasmus UniversityRotterdamThe Netherlands
- Erasmus School of EconomicsErasmus University RotterdamRotterdamThe Netherlands
| |
Collapse
|
31
|
Federici C, Pecchia L. Exploring the misalignment on the value of further research between payers and manufacturers. A case study on a novel total artificial heart. HEALTH ECONOMICS 2022; 31 Suppl 1:98-115. [PMID: 35460307 PMCID: PMC9546170 DOI: 10.1002/hec.4520] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 03/09/2022] [Accepted: 03/29/2022] [Indexed: 06/14/2023]
Abstract
Payers and manufacturers can disagree on the appropriate level of evidence that is required for new medical devices, resulting in high societal costs due to decisions taken with sub-optimal information. A cost-effectiveness model of a hypothetical total artificial heart was built using data from the literature and the (simulated) results of a pivotal study. The expected value of perfect information (EVPI) was calculated from both the payer and manufacturer perspectives, using net monetary benefit and the company's return on investment respectively. A function was also defined, linking effectiveness to market shares. Additional constraints such as a minimum clinical difference or maximum budget impact were introduced into the company's decisions to simulate additional barriers to adoption. The difference in the EVPI between manufacturers and payers varied greatly depending on the underlying decision rules and constraints. The manufacturer's EVPI depends on the probability of being reimbursed, the uncertainty on the (cost-)effectiveness of the technology, as well as other parameters relating to initial investments, operating costs and market dynamics. The use of Value of information for both perspectives can outline potential misalignments and can be particularly useful to inform early dialogs between manufacturers and payers, or negotiations on conditional reimbursement schemes.
Collapse
Affiliation(s)
- Carlo Federici
- SDA Bocconi School of ManagementCentre for Research on Health and Social Care Management (CERGAS)MilanItaly
- School of EngineeringUniversity of WarwickCoventryUK
| | | |
Collapse
|
32
|
Wason JMS, Dimairo M, Biggs K, Bowden S, Brown J, Flight L, Hall J, Jaki T, Lowe R, Pallmann P, Pilling MA, Snowdon C, Sydes MR, Villar SS, Weir CJ, Wilson N, Yap C, Hancock H, Maier R. Practical guidance for planning resources required to support publicly-funded adaptive clinical trials. BMC Med 2022; 20:254. [PMID: 35945610 PMCID: PMC9364623 DOI: 10.1186/s12916-022-02445-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Accepted: 06/20/2022] [Indexed: 11/15/2022] Open
Abstract
Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.
Collapse
Affiliation(s)
- James M S Wason
- Biostatistics Research Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK.
| | - Munyaradzi Dimairo
- School of Health and Related Research, Clinical Trials Research Unit, University of Sheffield, Sheffield, UK
| | - Katie Biggs
- School of Health and Related Research, Clinical Trials Research Unit, University of Sheffield, Sheffield, UK
| | - Sarah Bowden
- Cancer Research UK Clinical Trials Unit (CRCTU), University of Birmingham, Birmingham, UK
| | - Julia Brown
- Cancer Research UK CTU, University of Leeds, Leeds, UK
| | - Laura Flight
- School of Health and Related Research, Health Economics and Decision Science, University of Sheffield, Sheffield, UK
| | - Jamie Hall
- School of Health and Related Research, Clinical Trials Research Unit, University of Sheffield, Sheffield, UK
| | - Thomas Jaki
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
- Department of Mathematics and Statistics, Lancaster University, Lancaster, UK
| | - Rachel Lowe
- Centre for Trials Research, Cardiff University, Cardiff, UK
| | | | - Mark A Pilling
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
| | - Claire Snowdon
- The Institute of Cancer Research Clinical Trials & Statistics Unit, London, UK
| | | | - Sofía S Villar
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK
| | - Christopher J Weir
- Edinburgh Clinical Trials Unit, Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Nina Wilson
- Biostatistics Research Group, Population Health Sciences Institute, Newcastle University, Newcastle upon Tyne, UK
| | - Christina Yap
- The Institute of Cancer Research Clinical Trials & Statistics Unit, London, UK
| | - Helen Hancock
- Newcastle Clinical Trials Unit, Newcastle University, Newcastle upon Tyne, UK
| | - Rebecca Maier
- Newcastle Clinical Trials Unit, Newcastle University, Newcastle upon Tyne, UK
| |
Collapse
|
33
|
Kirwin E, Round J, Bond K, McCabe C. A Conceptual Framework for Life-Cycle Health Technology Assessment. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2022; 25:1116-1123. [PMID: 35779939 DOI: 10.1016/j.jval.2021.11.1373] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 11/11/2021] [Accepted: 11/23/2021] [Indexed: 05/06/2023]
Abstract
OBJECTIVES Health technology assessment (HTA) uses evidence appraisal and synthesis with economic evaluation to inform adoption decisions. Standard HTA processes sometimes struggle to (1) support decisions that involve significant uncertainty and (2) encourage continued generation of and adaptation to new evidence. We propose the life-cycle (LC)-HTA framework, addressing these challenges by providing additional tools to decision makers and improving outcomes for all stakeholders. METHODS Under the LC-HTA framework, HTA processes align to LC management. LC-HTA introduces changes in HTA methods to minimize analytic time while optimizing decision certainty. Where decision uncertainty exists, we recommend risk-based pricing and research-oriented managed access (ROMA). Contractual procurement agreements define the terms of reassessment and provide additional decision options to HTA agencies. LC-HTA extends value-of-information methods to inform ROMA agreements, leveraging routine, administrative data, and registries to reduce uncertainty. RESULTS LC-HTA enables the adoption of high-value high-risk innovations while improving health system sustainability through risk-sharing and reducing uncertainty. Responsiveness to evolving evidence is improved through contractually embedded decision rules to simplify reassessment. ROMA allows conditional adoption to obtain additional information, with confidence that the net value of that adoption decision is positive. CONCLUSIONS The LC-HTA framework improves outcomes for patients, sponsors, and payers. Patients benefit through earlier access to new technologies. Payers increase the value of the technologies they invest in and gain mechanisms to review investments. Sponsors benefit through greater certainty in outcomes related to their investment, swifter access to markets, and greater opportunities to demonstrate value.
Collapse
Affiliation(s)
- Erin Kirwin
- Institute of Health Economics, Edmonton, AB, Canada; Health Organisation, Policy, and Economics, School of Health Sciences, University of Manchester, Manchester, England, UK.
| | - Jeff Round
- Institute of Health Economics, Edmonton, AB, Canada; Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Ken Bond
- Institute of Health Economics, Edmonton, AB, Canada
| | - Christopher McCabe
- Institute of Health Economics, Edmonton, AB, Canada; Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
34
|
Vervaart M, Strong M, Claxton KP, Welton NJ, Wisløff T, Aas E. An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial. Med Decis Making 2022; 42:612-625. [PMID: 34967237 PMCID: PMC9189722 DOI: 10.1177/0272989x211068019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2021] [Accepted: 11/30/2021] [Indexed: 11/15/2022]
Abstract
BACKGROUND Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model. METHODS We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies. RESULTS There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty. CONCLUSIONS We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed. HIGHLIGHTS Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.
Collapse
Affiliation(s)
- Mathyn Vervaart
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway
- Norwegian Medicines Agency, Oslo, Norway
| | - Mark Strong
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | - Karl P. Claxton
- Centre for Health Economics, University of York, York, UK
- Department of Economics and Related Studies, University of York, York, UK
| | - Nicky J. Welton
- Population Health Sciences, University of Bristol, Bristol, UK
| | - Torbjørn Wisløff
- Department of Community Medicine, UiT The Arctic University of Norway, Oslo, Norway
- Norwegian Institute of Public Health, Oslo, Norway
| | - Eline Aas
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway
| |
Collapse
|
35
|
Wolff HB, Qendri V, Kunst N, Alarid-Escudero F, Coupé VMH. Methods for Communicating the Impact of Parameter Uncertainty in a Multiple-Strategies Cost-Effectiveness Comparison. Med Decis Making 2022; 42:956-968. [PMID: 35587181 PMCID: PMC9452448 DOI: 10.1177/0272989x221100112] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Purpose Analyzing and communicating uncertainty is essential in medical decision
making. To judge whether risks are acceptable, policy makers require
information on the expected outcomes but also on the uncertainty and
potential losses related to the chosen strategy. We aimed to compare methods
used to represent the impact of uncertainty in decision problems involving
many strategies, enhance existing methods, and provide an open-source and
easy-to-use tool. Methods We conducted a systematic literature search to identify methods used to
represent the impact of uncertainty in cost-effectiveness analyses comparing
multiple strategies. We applied the identified methods to probabilistic
sensitivity analysis outputs of 3 published decision-analytic models
comparing multiple strategies. Subsequently, we compared the following
characteristics: type of information conveyed, use of a fixed or flexible
willingness-to-pay threshold, output interpretability, and the graphical
discriminatory ability. We further proposed adjustments and integration of
methods to overcome identified limitations of existing methods. Results The literature search resulted in the selection of 9 methods. The 3 methods
with the most favorable characteristics to compare many strategies were 1)
the cost-effectiveness acceptability curve (CEAC) and cost-effectiveness
acceptability frontier (CEAF), 2) the expected loss curve (ELC), and 3) the
incremental benefit curve (IBC). The information required to assess
confidence in a decision often includes the average loss and the probability
of cost-effectiveness associated with each strategy. Therefore, we proposed
the integration of information presented in an ELC and CEAC into a single
heat map. Conclusions This article presents an overview of methods presenting uncertainty in
multiple-strategy cost-effectiveness analyses, with their strengths and
shortcomings. We proposed a heat map as an alternative method that
integrates all relevant information required for health policy and medical
decision making. Highlights
Collapse
Affiliation(s)
- Henri B Wolff
- Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health, Amsterdam, the Netherlands
| | - Venetia Qendri
- Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health, Amsterdam, the Netherlands
| | - Natalia Kunst
- Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health, Amsterdam, the Netherlands.,Department of Health Management and Health Economics, Faculty of Medicine, University of Oslo, Oslo, Norway.,Cancer Outcomes, Public Policy and Effectiveness Research (COPPER) Center, Yale University School of Medicine and Yale Cancer Center, New Haven, CT, USA.,Public Health Modeling Unit, Yale University School of Public Health, New Haven, CT, USA
| | - Fernando Alarid-Escudero
- Division of Public Administration, Center for Research and Teaching in Economics (CIDE), Aguascalientes, AGS, Mexico, MX-AGU, Mexico
| | - Veerle M H Coupé
- Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health, Amsterdam, the Netherlands
| |
Collapse
|
36
|
Williams J, Ker K, Roberts I, Shakur-Still H, Miners A. A cost-effectiveness and value of information analysis to inform future research of tranexamic acid for older adults experiencing mild traumatic brain injury. Trials 2022; 23:370. [PMID: 35505387 PMCID: PMC9066715 DOI: 10.1186/s13063-022-06244-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Accepted: 03/28/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Tranexamic acid reduces head injury deaths in patients with CT scan evidence of intracranial bleeding after mild traumatic brain injury (TBI). However, the cost-effectiveness of tranexamic acid for people with mild TBI in the pre-hospital setting, prior to CT scanning, is uncertain. A large randomised controlled trial (CRASH-4) is planned to address this issue, but the economic justification for it has not been established. The aim of the analysis was to estimate the likelihood of tranexamic acid being cost-effective given current evidence, the treatment effects required for cost-effectiveness, and the expected value of performing further research. METHODS An early economic decision model compared usual care for mild TBI with and without tranexamic acid, for adults aged 70 and above. The evaluation was performed from a UK healthcare perspective over a lifetime time horizon, with costs reported in 2020 pounds (GBP) and outcomes reported as quality-adjusted life years (QALYs). All analyses used a £20,000 per QALY cost-effectiveness threshold. RESULTS In the base case analysis, tranexamic acid was associated with an incremental cost-effectiveness ratio of £4885 per QALY gained, but the likelihood of it being cost-effective was highly dependent on the all-cause mortality treatment effect. The value of perfect information was £22.4 million, and the value of perfect information for parameters that could be collected in a trial was £21.9 million. The all-cause mortality risk ratio for tranexamic acid and the functional outcomes following TBI had the most impact on cost-effectiveness. CONCLUSIONS There is a high degree of uncertainty in the cost-effectiveness of tranexamic acid for older adults experiencing mild TBI, meaning there is a high value of performing future research in the UK. The value in a global context is likely to be far higher.
Collapse
Affiliation(s)
- Jack Williams
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK.
| | - Katharine Ker
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK
| | - Ian Roberts
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK
| | - Haleema Shakur-Still
- Clinical Trials Unit, London School of Hygiene & Tropical Medicine, Keppel Street, London, WC1E 7HT, UK
| | - Alec Miners
- Department of Health Services Research and Policy, London School of Hygiene & Tropical Medicine, 15-17 Tavistock Place, London, WC1H 9SH, UK
| |
Collapse
|
37
|
Simões Corrêa Galendi J, Kautz-Freimuth S, Stock S, Müller D. Uptake Rates of Risk-Reducing Surgeries for Women at Increased Risk of Hereditary Breast and Ovarian Cancer Applied to Cost-Effectiveness Analyses: A Scoping Systematic Review. Cancers (Basel) 2022; 14:cancers14071786. [PMID: 35406563 PMCID: PMC8997187 DOI: 10.3390/cancers14071786] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2021] [Revised: 03/23/2022] [Accepted: 03/26/2022] [Indexed: 01/09/2023] Open
Abstract
Simple Summary For women who have tested positive for BRCA mutations, the decision to make use of preventive surgical options, such as risk-reducing mastectomy (RRM) or risk-reducing bilateral salpingo-oophorectomy (RRSO), depends on the women’s personal preferences and the cultural/social context. Among others, the cost-effectiveness of RRM and RRSO can be affected by the uptake rate of these preventive surgical options. Uptake rates of surgery should be given more attention in the conceptualization of health economic modeling studies for RRM and RRSO. Prospective multicenter studies are recommended to reflect regional and national variations in women’s preferences for preventive surgery. Abstract The cost-effectiveness of genetic screen-and-treat strategies for women at increased risk for breast and ovarian cancer often depends on the women’s willingness to make use of risk-reducing mastectomy (RRM) or salpingo-oophorectomy (RRSO). To explore the uptake rates of RRM and RRSO applied in health economic modeling studies and the impact of uptake rates on the incremental cost-effectiveness ratios (ICER), we conducted a scoping literature review. In addition, using our own model, we conducted a value of information (VOI) analysis. Among the 19 models included in the review, the uptake rates of RRM ranged from 6% to 47% (RRSO: 10% to 88%). Fifty-seven percent of the models applied retrospective data obtained from registries, hospital records, or questionnaires. According to the models’ deterministic sensitivity analyses, there is a clear trend that a lower uptake rate increased the ICER and vice versa. Our VOI analysis showed high decision uncertainty associated with the uptake rates. In the future, uptake rates should be given more attention in the conceptualization of health economic modeling studies. Prospective studies are recommended to reflect regional and national variations in women’s preferences for preventive surgery.
Collapse
|
38
|
Heath A, Strong M, Glynn D, Kunst N, Welton NJ, Goldhaber-Fiebert JD. Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial. Med Decis Making 2022; 42:143-155. [PMID: 34388954 PMCID: PMC8793320 DOI: 10.1177/0272989x211026292] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 05/20/2021] [Indexed: 12/13/2022]
Abstract
The expected value of sample information (EVSI) can be used to prioritize avenues for future research and design studies that support medical decision making and offer value for money spent. EVSI is calculated based on 3 key elements. Two of these, a probabilistic model-based economic evaluation and updating model uncertainty based on simulated data, have been frequently discussed in the literature. By contrast, the third element, simulating data from the proposed studies, has received little attention. This tutorial contributes to bridging this gap by providing a step-by-step guide to simulating study data for EVSI calculations. We discuss a general-purpose algorithm for simulating data and demonstrate its use to simulate 3 different outcome types. We then discuss how to induce correlations in the generated data, how to adjust for common issues in study implementation such as missingness and censoring, and how individual patient data from previous studies can be leveraged to undertake EVSI calculations. For all examples, we provide comprehensive code written in the R language and, where possible, Excel spreadsheets in the supplementary materials. This tutorial facilitates practical EVSI calculations and allows EVSI to be used to prioritize research and design studies.
Collapse
Affiliation(s)
- Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
- Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Department of Statistical Science, University College London, London, UK
| | - Mark Strong
- School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK
| | - David Glynn
- Centre for Health Economics, University of York, York, UK
| | - Natalia Kunst
- Harvard Medical School & Harvard Pilgrim Health Care Institute, Harvard University, Boston, MA
| | - Nicky J. Welton
- School of Social and Community Medicine, University of Bristol, Bristol, UK
| | - Jeremy D. Goldhaber-Fiebert
- Stanford Health Policy, Centers for Health Policy and Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA
| |
Collapse
|
39
|
Kunst N, Stout NK, O’Brien G, Christensen KD, McMahon PM, Wu AC, Diller LR, Yeh JM. Population-Based Newborn Screening for Germline TP53 Variants: Clinical Benefits, Cost-Effectiveness, and Value of Further Research. J Natl Cancer Inst 2022; 114:722-731. [PMID: 35043946 PMCID: PMC9086756 DOI: 10.1093/jnci/djac013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Revised: 11/01/2021] [Accepted: 01/13/2022] [Indexed: 01/21/2023] Open
Abstract
BACKGROUND Identification of children and infants with Li-Fraumeni syndrome prompts tumor surveillance and allows potential early cancer detection. We assessed the clinical benefits and cost-effectiveness of population-wide newborn screening for TP53 variants (TP53-NBS). METHODS We simulated the impact of TP53-NBS using data regarding TP53-associated pediatric cancers and pathogenic or likely pathogenic (P/LP) TP53 variants from Surveillance, Epidemiology, and End Results; ClinVar and gnomAD; and clinical studies. We simulated an annual US birth cohort under usual care and TP53-NBS and estimated clinical benefits, life-years, and costs associated with usual care and TP53-NBS. RESULTS Under usual care, of 4 million newborns, 608 (uncertainty interval [UI] = 581-636) individuals would develop TP53-associated cancers before age 20 years. Under TP53-NBS, 894 individuals would have P/LP TP53 variants detected. These individuals would undergo routine surveillance after detection of P/LP TP53 variants decreasing the number of cancer-related deaths by 7.2% (UI = 4.0%-12.1%) overall via early malignancy detection. Compared with usual care, TP53-NBS had an incremental cost-effectiveness ratio of $106 009 per life-year gained. Probabilistic analysis estimated a 40% probability that TP53-NBS would be cost-effective given a $100 000 per life-year gained willingness-to-pay threshold. Using this threshold, a value of information analysis found that additional research on the prevalence of TP53 variants among rhabdomyosarcoma cases would resolve most of the decision uncertainty, resulting in an expected benefit of 349 life-years gained (or $36.6 million). CONCLUSIONS We found that TP53-NBS could be cost-effective; however, our findings suggest that further research is needed to reduce the uncertainty in the potential health outcomes and costs associated with TP53-NBS.
Collapse
Affiliation(s)
- Natalia Kunst
- Correspondence to: Natalia Kunst, PhD, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, 401 Park Dr, Suite 401, Boston, MA 02215, USA (e-mail: )
| | - Natasha K Stout
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - Grace O’Brien
- Department of Pediatrics, Boston Children’s Hospital, Boston, MA, USA
| | - Kurt D Christensen
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA,Broad Institute of MIT and Harvard, Boston, MA, USA
| | - Pamela M McMahon
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA
| | - Ann Chen Wu
- Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, MA, USA,Department of Pediatrics, Boston Children’s Hospital, Boston, MA, USA
| | - Lisa R Diller
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA,Department of Pediatric Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Jennifer M Yeh
- Department of Pediatrics, Boston Children’s Hospital, Boston, MA, USA,Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
40
|
Abstract
BACKGROUND The expected value of sample information (EVSI) calculates the value of collecting additional information through a research study with a given design. However, standard EVSI analyses do not account for the slow and often incomplete implementation of the treatment recommendations that follow research. Thus, standard EVSI analyses do not correctly capture the value of the study. Previous research has developed measures to calculate the research value while adjusting for implementation challenges, but estimating these measures is a challenge. METHODS Based on a method that assumes the implementation level is related to the strength of evidence in favor of the treatment, 2 implementation-adjusted EVSI calculation methods are developed. These novel methods circumvent the need for analytical calculations, which were restricted to settings in which normality could be assumed. The first method developed in this article uses computationally demanding nested simulations, based on the definition of the implementation-adjusted EVSI. The second method is based on adapting the moment matching method, a recently developed efficient EVSI computation method, to adjust for imperfect implementation. The implementation-adjusted EVSI is then calculated with the 2 methods across 3 examples. RESULTS The maximum difference between the 2 methods is at most 6% in all examples. The efficient computation method is between 6 and 60 times faster than the nested simulation method in this case study and could be used in practice. CONCLUSIONS This article permits the calculation of an implementation-adjusted EVSI using realistic assumptions. The efficient estimation method is accurate and can estimate the implementation-adjusted EVSI in practice. By adapting standard EVSI estimation methods, adjustments for imperfect implementation can be made with the same computational cost as a standard EVSI analysis. HIGHLIGHTS Standard expected value of sample information (EVSI) analyses do not account for the fact that treatment implementation following research is often slow and incomplete, meaning they incorrectly capture the value of the study.Two methods, based on nested Monte Carlo sampling and the moment matching EVSI calculation method, are developed to adjust EVSI calculations for imperfect implementation when the speed and level of the implementation of a new treatment depends on the strength of evidence in favor of the treatment.The 2 methods we develop provide similar estimates for the implementation-adjusted EVSI.Our methods extend current EVSI calculation algorithms and thus require limited additional computational complexity.
Collapse
Affiliation(s)
- Anna Heath
- Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada.,Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada.,Department of Statistical Science, University College London, London, UK
| |
Collapse
|
41
|
Flight L, Julious S, Brennan A, Todd S. Expected Value of Sample Information to Guide the Design of Group Sequential Clinical Trials. Med Decis Making 2021; 42:461-473. [PMID: 34859693 PMCID: PMC9005835 DOI: 10.1177/0272989x211045036] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Introduction Adaptive designs allow changes to an ongoing trial based on prespecified early examinations of accrued data. Opportunities are potentially being missed to incorporate health economic considerations into the design of these studies. Methods We describe how to estimate the expected value of sample information for group sequential design adaptive trials. We operationalize this approach in a hypothetical case study using data from a pilot trial. We report the expected value of sample information and expected net benefit of sampling results for 5 design options for the future full-scale trial including the fixed-sample-size design and the group sequential design using either the Pocock stopping rule or the O’Brien-Fleming stopping rule with 2 or 5 analyses. We considered 2 scenarios relating to 1) using the cost-effectiveness model with a traditional approach to the health economic analysis and 2) adjusting the cost-effectiveness analysis to incorporate the bias-adjusted maximum likelihood estimates of trial outcomes to account for the bias that can be generated in adaptive trials. Results The case study demonstrated that the methods developed could be successfully applied in practice. The results showed that the O’Brien-Fleming stopping rule with 2 analyses was the most efficient design with the highest expected net benefit of sampling in the case study. Conclusions Cost-effectiveness considerations are unavoidable in budget-constrained, publicly funded health care systems, and adaptive designs can provide an alternative to costly fixed-sample-size designs. We recommend that when planning a clinical trial, expected value of sample information methods be used to compare possible adaptive and nonadaptive trial designs, with appropriate adjustment, to help justify the choice of design characteristics and ensure the cost-effective use of research funding. Highlights
Collapse
Affiliation(s)
- Laura Flight
- Laura Flight, School of Health and Related Research (ScHARR), University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA, UK; ()
| | - Steven Julious
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | - Alan Brennan
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | - Susan Todd
- Department of Mathematics and Statistics, University of Reading, Reading, UK
| |
Collapse
|
42
|
Jackson C, Johnson R, de Nazelle A, Goel R, de Sá TH, Tainio M, Woodcock J. A guide to value of information methods for prioritising research in health impact modelling. EPIDEMIOLOGIC METHODS 2021; 10:20210012. [PMID: 35127249 PMCID: PMC7612319 DOI: 10.1515/em-2021-0012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Health impact simulation models are used to predict how a proposed policy or scenario will affect population health outcomes. These models represent the typically-complex systems that describe how the scenarios affect exposures to risk factors for disease or injury (e.g. air pollution or physical inactivity), and how these risk factors are related to measures of population health (e.g. expected survival). These models are informed by multiple sources of data, and are subject to multiple sources of uncertainty. We want to describe which sources of uncertainty contribute most to uncertainty about the estimate or decision arising from the model. Furthermore, we want to decide where further research should be focused to obtain further data to reduce this uncertainty, and what form that research might take. This article presents a tutorial in the use of Value of Information methods for uncertainty analysis and research prioritisation in health impact simulation models. These methods are based on Bayesian decision-theoretic principles, and quantify the expected benefits from further information of different kinds. The expected value of partial perfect information about a parameter measures sensitivity of a decision or estimate to uncertainty about that parameter. The expected value of sample information represents the expected benefit from a specific proposed study to get better information about the parameter. The methods are applicable both to situationswhere the model is used to make a decision between alternative policies, and situations where the model is simply used to estimate a quantity (such as expected gains in survival under a scenario). This paper explains how to calculate and interpret the expected value of information in the context of a simple model describing the health impacts of air pollution from motorised transport. We provide a general-purpose R package and full code to reproduce the example analyses.
Collapse
Affiliation(s)
| | - Robert Johnson
- MRC Biostatistics Unit, University of Cambridge, Cambridge, UK; and Imperial College London, London, UK
| | | | - Rahul Goel
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK
| | - Thiago Hérick de Sá
- World Health Organization, Geneva, Switzerland; and Center for Epidemiological Research in Nutrition and Health, University of Sao Paulo
| | - Marko Tainio
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK; and Finnish Environment Institute, Helsinki, Finland
| | - James Woodcock
- MRC Epidemiology Unit, University of Cambridge, Cambridge, UK
| |
Collapse
|
43
|
Wang Y, Rattanavipapong W, Teerawattananon Y. Using health technology assessment to set priority, inform target product profiles, and design clinical study for health innovation. TECHNOLOGICAL FORECASTING AND SOCIAL CHANGE 2021; 172:121000. [PMID: 34732945 PMCID: PMC8524319 DOI: 10.1016/j.techfore.2021.121000] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Revised: 05/04/2021] [Accepted: 06/25/2021] [Indexed: 05/29/2023]
Abstract
Early health technology assessment (early HTA) is a useful tool in guiding the innovation development process in medical technology development. However, the application of early HTA is sub-optimal amongst research and development (R&D) communities due to several challenges. In this paper, we presented a case study of application of early HTA by drawing on the experience from a workshop conducted for the Singapore government's medical technology innovation agency. The framework developed can help maximise the chance of the newly developed technology being accepted and widely used. By providing step-by-step guidance, this work aims to translate early HTA into a practical tool and promote the application of early HTA amongst R&D communities.
Collapse
Affiliation(s)
- Yi Wang
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
| | - Waranya Rattanavipapong
- Health Intervention and Technology Assessment Program, Ministry of Public Health, Nonthaburi, Thailand
| | - Yot Teerawattananon
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
- Health Intervention and Technology Assessment Program, Ministry of Public Health, Nonthaburi, Thailand
| |
Collapse
|
44
|
Cadham CJ, Knoll M, Sánchez-Romero LM, Cummings KM, Douglas CE, Liber A, Mendez D, Meza R, Mistry R, Sertkaya A, Travis N, Levy DT. The Use of Expert Elicitation among Computational Modeling Studies in Health Research: A Systematic Review. Med Decis Making 2021; 42:684-703. [PMID: 34694168 DOI: 10.1177/0272989x211053794] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND Expert elicitation (EE) has been used across disciplines to estimate input parameters for computational modeling research when information is sparse or conflictual. OBJECTIVES We conducted a systematic review to compare EE methods used to generate model input parameters in health research. DATA SOURCES PubMed and Web of Science. STUDY ELIGIBILITY Modeling studies that reported the use of EE as the source for model input probabilities were included if they were published in English before June 2021 and reported health outcomes. DATA ABSTRACTION AND SYNTHESIS Studies were classified as "formal" EE methods if they explicitly reported details of their elicitation process. Those that stated use of expert opinion but provided limited information were classified as "indeterminate" methods. In both groups, we abstracted citation details, study design, modeling methodology, a description of elicited parameters, and elicitation methods. Comparisons were made between elicitation methods. STUDY APPRAISAL Studies that conducted a formal EE were appraised on the reporting quality of the EE. Quality appraisal was not conducted for studies of indeterminate methods. RESULTS The search identified 1520 articles, of which 152 were included. Of the included studies, 40 were classified as formal EE and 112 as indeterminate methods. Most studies were cost-effectiveness analyses (77.6%). Forty-seven indeterminate method studies provided no information on methods for generating estimates. Among formal EEs, the average reporting quality score was 9 out of 16. LIMITATIONS Elicitations on nonhealth topics and those reported in the gray literature were not included. CONCLUSIONS We found poor reporting of EE methods used in modeling studies, making it difficult to discern meaningful differences in approaches. Improved quality standards for EEs would improve the validity and replicability of computational models. HIGHLIGHTS We find extensive use of expert elicitation for the development of model input parameters, but most studies do not provide adequate details of their elicitation methods.Lack of reporting hinders greater discussion of the merits and challenges of using expert elicitation for model input parameter development.There is a need to establish expert elicitation best practices and reporting guidelines.
Collapse
Affiliation(s)
- Christopher J Cadham
- Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA
| | - Marie Knoll
- Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA
| | | | - K Michael Cummings
- Department of Psychiatry & Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Clifford E Douglas
- Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA.,University of Michigan, Tobacco Research Network, Ann Arbor, MI, USA
| | - Alex Liber
- Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA
| | - David Mendez
- Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA
| | - Rafael Meza
- Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, MI, USA
| | - Ritesh Mistry
- Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor, MI, USA
| | | | - Nargiz Travis
- Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA.,Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA
| | - David T Levy
- Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA
| |
Collapse
|
45
|
Grimm SE, Pouwels X, Ramaekers BLT, Wijnen B, Otten T, Grutters J, Joore MA. State of the ART? Two New Tools for Risk Communication in Health Technology Assessments. PHARMACOECONOMICS 2021; 39:1185-1196. [PMID: 34278550 PMCID: PMC8476369 DOI: 10.1007/s40273-021-01060-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 06/10/2021] [Indexed: 05/22/2023]
Abstract
PURPOSE Outcomes of health technology assessments (HTA) are uncertain, and decision-making is associated with a risk. This risk, consisting of the probability of making a wrong decision and its impact, is rarely considered in HTA. This hampers transparent and consistent risk assessment and management. The aim of this study was to develop risk communication tools in the context of health technology decision-making under uncertainty. METHODS We performed a scoping review of tools for uncertainty and risk communication within HTA using citation pearl-growing. We developed two tools, drawing on existing publications on risk and uncertainty communication for inspiration. Individual semi-structured interviews with HTA stakeholders were performed to identify potential improvements in usefulness, user-friendliness, and information adequacy. Tools were amended and further evaluated in a real-world HTA and workshop with HTA stakeholders. RESULTS The identified risk communication tools did not include non-quantified uncertainties, and did not link to risk management strategies. We developed two tools: the Assessment of Risk Table (ART), for a summary of quantified and non-quantified uncertainties and the resulting risk assessment, and the Appraisal of Risk Chart (ARCH), for linking net benefit and risk outcomes to appropriate risk management strategies. Stakeholders appreciated the usefulness of the tools. They also highlighted that more information on local policy options was required for optimal risk management use, and HTA processes may need adapting. CONCLUSION The risk communication tools presented here can help assess risk, facilitate communication between analysts and decision-makers, and guide the appropriate use of available risk management strategies.
Collapse
Affiliation(s)
- Sabine E Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, The Netherlands.
| | - Xavier Pouwels
- Department of Health Technology and Services Research, Faculty of Behavioural, Management and Social Sciences, University of Twente, P.O. box 217, 7500 AE, Enschede, The Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, The Netherlands
| | - Ben Wijnen
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, The Netherlands
| | - Thomas Otten
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud University Medical Centre, Post 133, PO Box 9101, 6500 HB, Nijmegen, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care (CAPHRI), Maastricht University Medical Centre, P. Debyelaan 25, PO Box 5800, 6202 AZ, Maastricht, The Netherlands
| |
Collapse
|
46
|
Tuffaha H, Rothery C, Kunst N, Jackson C, Strong M, Birch S. A Review of Web-Based Tools for Value-of-Information Analysis. APPLIED HEALTH ECONOMICS AND HEALTH POLICY 2021; 19:645-651. [PMID: 34046866 PMCID: PMC7613968 DOI: 10.1007/s40258-021-00662-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 05/11/2021] [Indexed: 05/29/2023]
Abstract
Value-of-information analysis (VOI) is a decision-theoretic approach that is used to inform reimbursement decisions, optimise trial design and set research priorities. The application of VOI analysis for informing policy decisions in practice has been limited due, in part, to the perceived complexity associated with the calculation of VOI measures. Recent efforts have resulted in the development of efficient methods to estimate VOI measures and the development of user-friendly web-based tools to facilitate VOI calculations. We review the existing web-based tools including Sheffield Accelerated Value of Information (SAVI), the web interface to the BCEA (Bayesian Cost-Effectiveness Analysis) R package (BCEAweb), Rapid Assessment of Need for Evidence (RANE), and Value of Information for Cardiovascular Trials and Other Comparative Research (VICTOR). We describe what each tool is designed to do, the inputs they require, and the outputs they produce. Finally, we discuss how tools for VOI calculations might be improved in the future to facilitate the use of VOI analysis in practice.
Collapse
Affiliation(s)
- Haitham Tuffaha
- Centre for the Business and Economics of Health, The University of Queensland, Brisbane, QLD, 4067, Australia.
| | - Claire Rothery
- Centre for Health Economics, University of York, York, UK
| | - Natalia Kunst
- Department of Health Management and Health Economics, University of Oslo, Oslo, Norway
- Public Health Modeling Unit, Yale University School of Public Health, New Haven, CT, USA
| | - Chris Jackson
- Medical Research Council Biostatistics Unit, University of Cambridge, Cambridge, UK
| | - Mark Strong
- School for Health and Related Research, University of Sheffield, Sheffield, UK
| | - Stephen Birch
- Centre for the Business and Economics of Health, The University of Queensland, Brisbane, QLD, 4067, Australia
| |
Collapse
|
47
|
Grimm SE, Pouwels X, Ramaekers BLT, van Ravesteyn NT, Sankatsing VDV, Grutters J, Joore MA. Implementation Barriers to Value of Information Analysis in Health Technology Decision Making: Results From a Process Evaluation. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2021; 24:1126-1136. [PMID: 34372978 DOI: 10.1016/j.jval.2021.03.013] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Revised: 02/10/2021] [Accepted: 03/29/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVES Value of information (VOI) analysis can support health technology assessment decision making, but it is a long way from being standard use. The objective of this study was to understand barriers to the implementation of VOI analysis and propose actions to overcome these. METHODS We performed a process evaluation of VOI analysis use within decision making on tomosynthesis versus digital mammography for use in the Dutch breast cancer population screening. Based on steering committee meeting attendance and regular meetings with analysts, we developed a list of barriers to VOI use, which were analyzed using an established diffusion model. We proposed actions to address these barriers. Barriers and actions were discussed and validated in a workshop with stakeholders representing patients, clinicians, regulators, policy advisors, researchers, and the industry. RESULTS Consensus was reached on groups of barriers, which included characteristics of VOI analysis itself, stakeholder's attitudes, analysts' and policy makers' skills and knowledge, system readiness, and implementation in the organization. Observed barriers did not only pertain to VOI analysis itself but also to formulating the objective of the assessment, economic modeling, and broader aspects of uncertainty assessment. Actions to overcome these barriers related to organizational changes, knowledge transfer, cultural change, and tools. CONCLUSIONS This in-depth analysis of barriers to implementation of VOI analysis and resulting actions and tools may be useful to health technology assessment organizations that wish to implement VOI analysis in technology assessment and research prioritization. Further research should focus on application and evaluation of the proposed actions in real-world assessment processes.
Collapse
Affiliation(s)
- Sabine E Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care, Maastricht University Medical Centre, Maastricht, The Netherlands.
| | - Xavier Pouwels
- Department of Health Technology and Services Research, Faculty of Behavioural, Management and Social Sciences, University of Twente, Enschede, The Netherlands
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care, Maastricht University Medical Centre, Maastricht, The Netherlands
| | | | - Valérie D V Sankatsing
- Department of Public Health, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Janneke Grutters
- Department for Health Evidence, Radboud University Medical Center, Nijmegen, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, School for Public Health and Primary Care, Maastricht University Medical Centre, Maastricht, The Netherlands
| |
Collapse
|
48
|
Petersohn S, Grimm SE, Ramaekers BLT, Ten Cate-Hoek AJ, Joore MA. Exploring the Feasibility of Comprehensive Uncertainty Assessment in Health Economic Modeling: A Case Study. VALUE IN HEALTH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR PHARMACOECONOMICS AND OUTCOMES RESEARCH 2021; 24:983-994. [PMID: 34243842 DOI: 10.1016/j.jval.2021.01.004] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Revised: 11/04/2020] [Accepted: 01/06/2021] [Indexed: 05/22/2023]
Abstract
OBJECTIVES Decision makers adopt health technologies based on health economic models that are subject to uncertainty. In an ideal world, these models parameterize all uncertainties and reflect them in the cost-effectiveness probability and risk associated with the adoption. In practice, uncertainty assessment is often incomplete, potentially leading to suboptimal reimbursement recommendations and risk management. This study examines the feasibility of comprehensive uncertainty assessment in health economic models. METHODS A state transition model on peripheral arterial disease treatment was used as a case study. Uncertainties were identified and added to the probabilistic sensitivity analysis if possible. Parameter distributions were obtained by expert elicitation, and structural uncertainties were either parameterized or explored in scenario analyses, which were model averaged. RESULTS A truly comprehensive uncertainty assessment, parameterizing all uncertainty, could not be achieved. Expert elicitation informed 8 effectiveness, utility, and cost parameters. Uncertainties were parameterized or explored in scenario analyses and with model averaging. Barriers included time and resource constraints, also of clinical experts, and lacking guidance regarding some aspects of expert elicitation, evidence aggregation, and handling of structural uncertainty. The team's multidisciplinary expertise and existing literature and tools were facilitators. CONCLUSIONS While comprehensive uncertainty assessment may not be attainable, improvements in uncertainty assessment in general are no doubt desirable. This requires the development of detailed guidance and hands-on tutorials for methods of uncertainty assessment, in particular aspects of expert elicitation, evidence aggregation, and handling of structural uncertainty. The issue of benefits of uncertainty assessment versus time and resources needed remains unclear.
Collapse
Affiliation(s)
- Svenja Petersohn
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Sabine E Grimm
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands.
| | - Bram L T Ramaekers
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Arina J Ten Cate-Hoek
- Department of Internal Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Manuela A Joore
- Department of Clinical Epidemiology and Medical Technology Assessment, Maastricht University Medical Centre, Maastricht, The Netherlands
| |
Collapse
|
49
|
Jongeneel G, Greuter MJE, Kunst N, van Erning FN, Koopman M, Medema JP, Vermeulen L, Ijzermans JNM, Vink GR, Punt CJA, Coupé VMH. Early Cost-effectiveness Analysis of Risk-Based Selection Strategies for Adjuvant Treatment in Stage II Colon Cancer: The Potential Value of Prognostic Molecular Markers. Cancer Epidemiol Biomarkers Prev 2021; 30:1726-1734. [PMID: 34162659 DOI: 10.1158/1055-9965.epi-21-0078] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Revised: 03/28/2021] [Accepted: 06/09/2021] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND To explore the potential value of consensus molecular subtypes (CMS) in stage II colon cancer treatment selection, we carried out an early cost-effectiveness assessment of a CMS-based strategy for adjuvant chemotherapy. METHODS We used a Markov cohort model to evaluate three selection strategies: (i) the Dutch guideline strategy (MSS+pT4), (ii) the mutation-based strategy (MSS plus a BRAF and/or KRAS mutation or MSS plus pT4), and (iii) the CMS-based strategy (CMS4 or pT4). Outcomes were number of colon cancer deaths per 1,000 patients, total discounted costs per patient (pp), and quality-adjusted life-years (QALY) pp. The analyses were conducted from a Dutch societal perspective. The robustness of model predictions was assessed in sensitivity analyses. To evaluate the value of future research, we performed a value of information (VOI) analysis. RESULTS The Dutch guideline strategy resulted in 8.10 QALYs pp and total costs of €23,660 pp. The CMS-based and mutation-based strategies were more effective and more costly, with 8.12 and 8.13 QALYs pp and €24,643 and €24,542 pp, respectively. Assuming a threshold of €50,000/QALY, the mutation-based strategy was considered as the optimal strategy in an incremental analysis. However, the VOI analysis showed substantial decision uncertainty driven by the molecular markers (expected value of partial perfect information: €18M). CONCLUSIONS On the basis of current evidence, our analyses suggest that the mutation-based selection strategy would be the best use of resources. However, the extensive decision uncertainty for the molecular markers does not allow selection of an optimal strategy at present. IMPACT Future research is needed to eliminate decision uncertainty driven by molecular markers.
Collapse
Affiliation(s)
- Gabrielle Jongeneel
- Department of Epidemiology and Data Science, Amsterdam UMC, VU University, Amsterdam, the Netherlands.
| | - Marjolein J E Greuter
- Department of Epidemiology and Data Science, Amsterdam UMC, VU University, Amsterdam, the Netherlands
| | - Natalia Kunst
- Department of Epidemiology and Data Science, Amsterdam UMC, VU University, Amsterdam, the Netherlands.,Harvard Medical School & Harvard Pilgrim Health Care Institute, Boston, Massachusetts.,Cancer Outcomes, Public Policy, and Effectiveness Research (COPPER) Center, Yale University School of Medicine, New Haven, Connecticut.,Public Health Modeling Unit, Yale University School of Public Health, New Haven, Connecticut
| | - Felice N van Erning
- Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, the Netherlands
| | - Miriam Koopman
- University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Jan P Medema
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Center for Experimental Molecular Medicine (CEMM), Amsterdam, the Netherlands.,Oncode Institute, Amsterdam, the Netherlands
| | - Louis Vermeulen
- Amsterdam UMC, University of Amsterdam, Cancer Center Amsterdam, Center for Experimental Molecular Medicine (CEMM), Amsterdam, the Netherlands.,Oncode Institute, Amsterdam, the Netherlands
| | - Jan N M Ijzermans
- Department of General Surgery, Erasmus MC University Medical Center, Rotterdam, the Netherlands
| | - Geraldine R Vink
- Department of Research and Development, Netherlands Comprehensive Cancer Organisation (IKNL), Utrecht, the Netherlands.,University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Cornelis J A Punt
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, the Netherlands
| | - Veerle M H Coupé
- Department of Epidemiology and Data Science, Amsterdam UMC, VU University, Amsterdam, the Netherlands
| |
Collapse
|
50
|
de Carvalho TM, van Rosmalen J, Wolff HB, Koffijberg H, Coupé VMH. Choosing a Metamodel of a Simulation Model for Uncertainty Quantification. Med Decis Making 2021; 42:28-42. [PMID: 34098793 DOI: 10.1177/0272989x211016307] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification. METHODS We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM. RESULTS In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%. CONCLUSION To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.
Collapse
Affiliation(s)
- Tiago M de Carvalho
- Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands
| | | | - Harold B Wolff
- Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands
| | - Hendrik Koffijberg
- Health Technology and Services Research Department, Faculty of Behavioral Management and Social Sciences, Technical Medical Centre, University of Twente, Enschede, the Netherlands
| | - Veerle M H Coupé
- Department of Epidemiology and Biostatistics, Amsterdam UMC, Location VUMC, Amsterdam, the Netherlands
| |
Collapse
|