Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling

Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information

BACKGROUND

The COVID-19 pandemic underscored the criticality and complexity of decision making for novel treatment approval and further research. Our study aims to assess potential decision-making methodologies, an evaluation vital for refining future public health crisis responses.

METHODS

We compared 4 decision-making approaches to drug approval and research: the Food and Drug Administration's policy decisions, cumulative meta-analysis, a prospective value-of-information (VOI) approach (using information available at the time of decision), and a reference standard (retrospective VOI analysis using information available in hindsight). Possible decisions were to reject, accept, provide emergency use authorization, or allow access to new therapies only in research settings. We used monoclonal antibodies provided to hospitalized COVID-19 patients as a case study, examining the evidence from September 2020 to December 2021 and focusing on each method's capacity to optimize health outcomes and resource allocation.

RESULTS

Our findings indicate a notable discrepancy between policy decisions and the reference standard retrospective VOI approach with expected losses up to $269 billion USD, suggesting suboptimal resource use during the wait for emergency use authorization. Relying solely on cumulative meta-analysis for decision making results in the largest expected loss, while the policy approach showed a loss up to $16 billion and the prospective VOI approach presented the least loss (up to $2 billion).

CONCLUSION

Our research suggests that incorporating VOI analysis may be particularly useful for research prioritization and treatment implementation decisions during pandemics. While the prospective VOI approach was favored in this case study, further studies should validate the ideal decision-making method across various contexts. This study's findings not only enhance our understanding of decision-making strategies during a health crisis but also provide a potential framework for future pandemic responses.

HIGHLIGHTS

This study reviews discrepancies between a reference standard (retrospective VOI, using hindsight information) and 3 conceivable real-time approaches to research-treatment decisions during a pandemic, suggesting suboptimal use of resources.Of all prospective decision-making approaches considered, VOI closely mirrored the reference standard, yielding the least expected value loss across our study timeline.This study illustrates the possible benefit of VOI results and the need for evidence accumulation accompanied by modeling in health technology assessment for emerging therapies.

Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators

Value of Information (VOI) analyses calculate the economic value that could be generated by obtaining further information to reduce uncertainty in a health economic decision model. VOI has been suggested as a tool for research prioritisation and trial design as it can highlight economically valuable avenues for future research. Recent methodological advances have made it increasingly feasible to use VOI in practice for research; however, there are critical differences between the VOI approach and the standard methods used to design research studies such as clinical trials. We aimed to highlight key differences between the research design approach based on VOI and standard clinical trial design methods, in particular the importance of considering the full decision context. We present two hypothetical examples to demonstrate that VOI methods are only accurate when (1) all feasible comparators are included in the decision model when designing research, and (2) all comparators are retained in the decision model once the data have been collected and a final treatment recommendation is made. Omitting comparators from either the design or analysis phase of research when using VOI methods can lead to incorrect trial designs and/or treatment recommendations. Overall, we conclude that incorrectly specifying the health economic model by ignoring potential comparators can lead to misleading VOI results and potentially waste scarce research resources.

Metamodeling for Policy Simulations with Multivariate Outcomes

PURPOSE

Metamodels are simplified approximations of more complex models that can be used as surrogates for the original models. Challenges in using metamodels for policy analysis arise when there are multiple correlated outputs of interest. We develop a framework for metamodeling with policy simulations to accommodate multivariate outcomes.

METHODS

We combine 2 algorithm adaptation methods-multitarget stacking and regression chain with maximum correlation-with different base learners including linear regression (LR), elastic net (EE) with second-order terms, Gaussian process regression (GPR), random forests (RFs), and neural networks. We optimize integrated models using variable selection and hyperparameter tuning. We compare the accuracy, efficiency, and interpretability of different approaches. As an example application, we develop metamodels to emulate a microsimulation model of testing and treatment strategies for hepatitis C in correctional settings.

RESULTS

Output variables from the simulation model were correlated (average ρ = 0.58). Without multioutput algorithm adaptation methods, in-sample fit (measured by R2) ranged from 0.881 for LR to 0.987 for GPR. The multioutput algorithm adaptation method increased R2 by an average 0.002 across base learners. Variable selection and hyperparameter tuning increased R2 by 0.009. Simpler models such as LR, EE, and RF required minimal training and prediction time. LR and EE had advantages in model interpretability, and we considered methods for improving the interpretability of other models.

CONCLUSIONS

In our example application, the choice of base learner had the largest impact on R2; multioutput algorithm adaptation and variable selection and hyperparameter tuning had a modest impact. Although advantages and disadvantages of specific learning algorithms may vary across different modeling applications, our framework for metamodeling in policy analyses with multivariate outcomes has broad applicability to decision analysis in health and medicine.

Emerging Therapies for COVID-19: The Value of Information From More Clinical Trials

OBJECTIVES

The COVID-19 pandemic necessitates time-sensitive policy and implementation decisions regarding new therapies in the face of uncertainty. This study aimed to quantify consequences of approving therapies or pursuing further research: immediate approval, use only in research, approval with research (eg, emergency use authorization), or reject.

METHODS

Using a cohort state-transition model for hospitalized patients with COVID-19, we estimated quality-adjusted life-years (QALYs) and costs associated with the following interventions: hydroxychloroquine, remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, tocilizumab, lopinavir-ritonavir, interferon beta-1a, and usual care. We used the model outcomes to conduct cost-effectiveness and value of information analyses from a US healthcare perspective and a lifetime horizon.

RESULTS

Assuming a $100 000-per-QALY willingness-to-pay threshold, only remdesivir, casirivimab-imdevimab, dexamethasone, baricitinib-remdesivir, and tocilizumab were (cost-) effective (incremental net health benefit 0.252, 0.164, 0.545, 0.668, and 0.524 QALYs and incremental net monetary benefit $25 249, $16 375, $54 526, $66 826, and $52 378). Our value of information analyses suggest that most value can be obtained if these 5 therapies are approved for immediate use rather than requiring additional randomized controlled trials (RCTs) (net value $20.6 billion, $13.4 billion, $7.4 billion, $54.6 billion, and $7.1 billion), hydroxychloroquine (net value $198 million) is only used in further RCTs if seeking to demonstrate decremental cost-effectiveness and otherwise rejected, and interferon beta-1a and lopinavir-ritonavir are rejected (ie, neither approved nor additional RCTs).

CONCLUSIONS

Estimating the real-time value of collecting additional evidence during the pandemic can inform policy makers and clinicians about the optimal moment to implement therapies and whether to perform further research.

Personalization of Medical Treatment Decisions: Simplifying Complex Models while Maintaining Patient Health Outcomes

BACKGROUND

Personalizing medical treatments based on patient-specific risks and preferences can improve patient health. However, models to support personalized treatment decisions are often complex and difficult to interpret, limiting their clinical application.

METHODS

We present a new method, using machine learning to create meta-models, for simplifying complex models for personalizing medical treatment decisions. We consider simple interpretable models, interpretable ensemble models, and noninterpretable ensemble models. We use variable selection with a penalty for patient-specific risks and/or preferences that are difficult, risky, or costly to obtain. We interpret the meta-models to the extent permitted by their model architectures. We illustrate our method by applying it to simplify a previously developed model for personalized selection of antipsychotic drugs for patients with schizophrenia.

RESULTS

The best simplified interpretable, interpretable ensemble, and noninterpretable ensemble models contained at most half the number of patient-specific risks and preferences compared with the original model. The simplified models achieved 60.5% (95% credible interval [crI]: 55.2-65.4), 60.8% (95% crI: 55.5-65.7), and 83.8% (95% crI: 80.8-86.6), respectively, of the net health benefit of the original model (quality-adjusted life-years gained). Important variables in all models were similar and made intuitive sense. Computation time for the meta-models was orders of magnitude less than for the original model.

LIMITATIONS

The simplified models share the limitations of the original model (e.g., potential biases).

CONCLUSIONS

Our meta-modeling method is disease- and model- agnostic and can be used to simplify complex models for personalization, allowing for variable selection in addition to improved model interpretability and computational performance. Simplified models may be more likely to be adopted in clinical settings and can help improve equity in patient outcomes.

Identifying a Single Optimal Integrated Cervical Cancer Prevention Policy in Norway: A Cost-Effectiveness Analysis

BACKGROUND

Interventions targeting the same disease but at different points along the disease continuum (e.g., screening and vaccination to prevent cervical cancer [CC]) are often evaluated in isolation, which can affect cost-effectiveness profiles and policy conclusions. We evaluated nonavalent human papillomavirus (HPV) vaccine (9vHPV) compared with bivalent HPV vaccine (2vHPV) alongside deintensified screening intervals for a vaccinated birth cohort to inform a single optimal integrated CC prevention policy.

METHODS

Using a multimodeling approach, we evaluated the health and economic impacts of alternative CC screening strategies for a Norwegian birth cohort eligible for HPV vaccination in 2021 assuming they received 1) 2vHPV or 2) 9vHPV. We conducted 1) a restricted analysis that evaluated the optimal HPV vaccine under current screening guidelines; and 2) a comprehensive analysis including alternative screening and vaccination strategy combinations. We calculated incremental cost-effectiveness ratios (ICERs) and evaluated them according to different cost-effectiveness thresholds.

RESULTS

Assuming a cost-effectiveness threshold of $40,000 per quality-adjusted life year (QALY) gained, we found that, while holding screening intensity fixed, switching the routine vaccination program in Norway from 2vHPV to 9vHPV would not be considered cost-effective (ICER of $132,700 per QALY gained). However, when allowing for varying intensities of CC screening, we found that switching to 9vHPV would be cost-effective compared with 2vHPV under an alternative threshold of $55,000 per QALY gained, if coupled with reductions in the number of lifetime screens.

CONCLUSIONS

Our analysis highlights the importance of evaluating the full potential policy landscape for country-level decision makers considering policy adoption, including nonindependent primary and secondary prevention efforts, to draw appropriate conclusions and avoid sub-optimal outcomes.

HIGHLIGHTS

Without evaluating the full potential policy landscape, including primary and secondary prevention efforts, country-level decision makers may not be able to draw appropriate policy conclusions, resulting in suboptimal outcomes.An applied example from cervical cancer prevention in Norway compared a restricted analysis of current screening guidelines to a comprehensive analysis including alternative screening and vaccination strategy combinations.We found that a switch from bivalent to nonavalent human papillomavirus vaccine would be considered cost-effective in Norway if coupled with reductions in the number of lifetime screens compared with the current screening strategy.A comprehensive analysis that considers how different types of interventions along the disease continuum affect each other will be critical for decision makers interpreting cost-effectiveness analysis results.

Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation

BACKGROUND

The expected value of sample information (EVSI) calculates the value of collecting additional information through a research study with a given design. However, standard EVSI analyses do not account for the slow and often incomplete implementation of the treatment recommendations that follow research. Thus, standard EVSI analyses do not correctly capture the value of the study. Previous research has developed measures to calculate the research value while adjusting for implementation challenges, but estimating these measures is a challenge.

METHODS

Based on a method that assumes the implementation level is related to the strength of evidence in favor of the treatment, 2 implementation-adjusted EVSI calculation methods are developed. These novel methods circumvent the need for analytical calculations, which were restricted to settings in which normality could be assumed. The first method developed in this article uses computationally demanding nested simulations, based on the definition of the implementation-adjusted EVSI. The second method is based on adapting the moment matching method, a recently developed efficient EVSI computation method, to adjust for imperfect implementation. The implementation-adjusted EVSI is then calculated with the 2 methods across 3 examples.

RESULTS

The maximum difference between the 2 methods is at most 6% in all examples. The efficient computation method is between 6 and 60 times faster than the nested simulation method in this case study and could be used in practice.

CONCLUSIONS

This article permits the calculation of an implementation-adjusted EVSI using realistic assumptions. The efficient estimation method is accurate and can estimate the implementation-adjusted EVSI in practice. By adapting standard EVSI estimation methods, adjustments for imperfect implementation can be made with the same computational cost as a standard EVSI analysis.

HIGHLIGHTS

Standard expected value of sample information (EVSI) analyses do not account for the fact that treatment implementation following research is often slow and incomplete, meaning they incorrectly capture the value of the study.Two methods, based on nested Monte Carlo sampling and the moment matching EVSI calculation method, are developed to adjust EVSI calculations for imperfect implementation when the speed and level of the implementation of a new treatment depends on the strength of evidence in favor of the treatment.The 2 methods we develop provide similar estimates for the implementation-adjusted EVSI.Our methods extend current EVSI calculation algorithms and thus require limited additional computational complexity.

Expected Value of Sample Information to Guide the Design of Group Sequential Clinical Trials

Introduction

Adaptive designs allow changes to an ongoing trial based on prespecified early examinations of accrued data. Opportunities are potentially being missed to incorporate health economic considerations into the design of these studies.

Methods

We describe how to estimate the expected value of sample information for group sequential design adaptive trials. We operationalize this approach in a hypothetical case study using data from a pilot trial. We report the expected value of sample information and expected net benefit of sampling results for 5 design options for the future full-scale trial including the fixed-sample-size design and the group sequential design using either the Pocock stopping rule or the O’Brien-Fleming stopping rule with 2 or 5 analyses. We considered 2 scenarios relating to 1) using the cost-effectiveness model with a traditional approach to the health economic analysis and 2) adjusting the cost-effectiveness analysis to incorporate the bias-adjusted maximum likelihood estimates of trial outcomes to account for the bias that can be generated in adaptive trials.

Results

The case study demonstrated that the methods developed could be successfully applied in practice. The results showed that the O’Brien-Fleming stopping rule with 2 analyses was the most efficient design with the highest expected net benefit of sampling in the case study.

Conclusions

Cost-effectiveness considerations are unavoidable in budget-constrained, publicly funded health care systems, and adaptive designs can provide an alternative to costly fixed-sample-size designs. We recommend that when planning a clinical trial, expected value of sample information methods be used to compare possible adaptive and nonadaptive trial designs, with appropriate adjustment, to help justify the choice of design characteristics and ensure the cost-effective use of research funding.

Highlights

Multilevel and Quasi Monte Carlo Methods for the Calculation of the Expected Value of Partial Perfect Information

The expected value of partial perfect information (EVPPI) provides an upper bound on the value of collecting further evidence on a set of inputs to a cost-effectiveness decision model. Standard Monte Carlo estimation of EVPPI is computationally expensive as it requires nested simulation. Alternatives based on regression approximations to the model have been developed but are not practicable when the number of uncertain parameters of interest is large and when parameter estimates are highly correlated. The error associated with the regression approximation is difficult to determine, while MC allows the bias and precision to be controlled. In this article, we explore the potential of quasi Monte Carlo (QMC) and multilevel Monte Carlo (MLMC) estimation to reduce the computational cost of estimating EVPPI by reducing the variance compared with MC while preserving accuracy. We also develop methods to apply QMC and MLMC to EVPPI, addressing particular challenges that arise where Markov chain Monte Carlo (MCMC) has been used to estimate input parameter distributions. We illustrate the methods using 2 examples: a simplified decision tree model for treatments for depression and a complex Markov model for treatments to prevent stroke in atrial fibrillation, both of which use MCMC inputs. We compare the performance of QMC and MLMC with MC and the approximation techniques of generalized additive model (GAM) regression, Gaussian process (GP) regression, and integrated nested Laplace approximations (INLA-GP). We found QMC and MLMC to offer substantial computational savings when parameter sets are large and correlated and when the EVPPI is large. We also found that GP and INLA-GP were biased in those situations, whereas GAM cannot estimate EVPPI for large parameter sets.

Prioritizing Research Informing Antibiotic Prophylaxis Guidelines for Knee Arthroplasty Patients

INTRODUCTION

Guidelines for routine antibiotic prophylaxis (AP) before dental procedures to prevent periprosthetic joint infection (PJI) have been hampered by the lack of prospective clinical trials.

OBJECTIVES

To apply value-of-information (VOI) analysis to quantify the value of conducting further clinical research to reduce decision uncertainty regarding the cost-effectiveness of AP strategies for dental patients undergoing total knee arthroplasty (TKA).

METHODS

An updated decision model and probabilistic sensitivity analysis (PSA) evaluated the cost-effectiveness of AP and decision uncertainty for 3 AP strategies: no AP, 2-y AP, and lifetime AP. VOI analyses estimated the value and cost of conducting a randomized controlled trial (RCT) or observational study. We used a linear regression meta-modeling approach to calculate the population expected value of partial perfect information and a Gaussian approximation to calculate population expected value of sample information, and we subtracted the cost for research to obtain the expected net benefit of sampling (ENBS). We determined the optimal trial sample sizes that maximized ENBS.

RESULTS

Using a willingness-to-pay threshold of $100,000 per quality-adjusted life-year, the PSA found that a no-AP strategy had the highest expected net benefit, with a 60% probability of being cost-effective, and 2-y AP had a 37% probability. The optimal sample size for an RCT to determine AP efficacy and dental-related PJI risk would require approximately 421 patients per arm with an estimated cost of $14.7 million. The optimal sample size for an observational study to inform quality-of-life parameters would require 2,211 patients with an estimated cost of $1.2 million. The 2 trial designs had an ENBS of approximately $25 to $26 million.

CONCLUSION

Given the uncertainties associated with AP guidelines for dental patients after TKA, we conclude there is value in conducting further research to inform the risk of PJI, effectiveness of AP, and quality-of-life values.

KNOWLEDGE TRANSFER STATEMENT

The results of this value-of-information analysis demonstrate that there is substantial uncertainty around clinical, health status, and economic parameters that may influence the antibiotic prophylaxis guidance for dental patients with total knee arthroplasty. The analysis supports the contention that conducting additional clinical research to reduce decision uncertainty is worth pursuing and will inform the antibiotic prophylaxis debate for clinicians and dental patients with prosthetic joints.

Is further research on adult pneumococcal vaccine uptake improvement programs worthwhile? Α value of information analysis

BACKGROUND

Pneumococcal vaccination policy for US adults is evolving, but previous research has shown that programs to increase vaccine uptake are economically favorable, despite parameter uncertainty. Using value of information (VOI) analysis and prior analyses, we examine the value of further research on vaccine uptake program parameters.

METHODS

In US 50-64-year-olds, current vaccine recommendations with and without an uptake program were analyzed. In older adults, current recommendations and an alternative strategy (polysaccharide vaccine for all, adding conjugate vaccine only for the immunocompromised) with and without uptake programs were examined. Uptake program parameters were derived from a clinical trial (absolute uptake increase 12.3% [range 0-25%], per-person cost $1.78 [range $0.70-$2.26]), with other parameters obtained from US databases. VOI analyses incorporated probabilistic sensitivity analysis outputs into R-based regression techniques.

RESULTS

In 50-64-year-olds, an uptake program cost $54,900/QALY gained compared to no uptake program. For ages ≥65, the program cost $287,000/QALY gained with the alternative strategy and $765,000/QALY with current recommendations. In younger adults, population-level expected value of perfect information (EVPI) was $59.7 million at $50,000/QALY gained and $2.8 million at $100,000/QALY gained. In older adults, EVPI values ranged from ~$1 million to $34.5 million at $100,000 and $200,000/QALY thresholds. The population expected value of partial perfect information (EVPPI) for combined uptake program cost and uptake improvement parameters in the younger population was $368,700 at $50,000/QALY and $43,900 at $100,000/QALY gained thresholds. In older adults, population EVPPI for vaccine uptake program parameters was $0 at both thresholds, reaching a maximum value of $445,000 at a $225,000/QALY threshold. Other model parameters comprised larger components of the global EVPI.

CONCLUSION

VOI results do not support further research on pneumococcal vaccine uptake programs in adults at commonly cited US cost-effectiveness benchmarks. Further research to reduce uncertainty in other aspects of adult pneumococcal vaccination is justifiable.

Cost-Effectiveness of Primary Liver Transplantation Versus Hepatoportoenterostomy in the Management of Biliary Atresia in the United States

Biliary atresia (BA) is the leading indication to perform a pediatric liver transplantation (LT). Timely hepatoportoenterostomy (HPE) attempts to interrupt the natural history and allow for enteric bile flow; however, most patients who are treated with HPE require LT by the age of 10 years. We determined the cost-effectiveness of foregoing HPE to perform primary LT (pLT) in children with BA compared with standard-of-care HPE management. A Markov model was developed to simulate BA treatment over 10 years. Costs were measured in 2018 US dollars and effectiveness in life-years (LYs). The primary outcome was incremental cost-effectiveness ratio (ICER) between treatments. Model parameters were derived from the literature. In the base model, we assumed similar LT outcomes after HPE and pLT. Sensitivity analyses on all model parameters were performed, including a scenario in which pLT led to 100% patient and graft survival after LT. Children undergoing HPE accumulated $316,692 in costs and 8.17 LYs per patient. Children undergoing pLT accumulated $458,059 in costs and 8.24 LYs per patient, costing $1,869,164 per LY gained compared with HPE. With parameter variation over plausible ranges, only post-HPE and post-LT costs reduced the ICER below a typical threshold of $100,000 per LY gained. On probabilistic sensitivity analysis, 93% of iterations favored HPE at that threshold. With 100% patient and graft survival after pLT, pLT cost $283,478 per LY gained. HPE is more economically favorable than pLT for BA. pLT is unfavorable even with no graft or patient loss. The ability to predict those patients who may experience high costs after HPE or low costs after LT may help identify those patients for whom pLT could be considered.

Cost-effectiveness of ethanol lock prophylaxis to prevent central line-associated bloodstream infections in children with intestinal failure in the United States

INTRODUCTION

Central line-associated bloodstream infections (CLABSIs) lead to significant morbidity and mortality in children with intestinal failure (IF). Ethanol lock prophylaxis (ELP) greatly reduces CLABSI frequency with minimal side effects. However, in the United States, a recently approved orphan drug designation for dehydrated alcohol has greatly increased 70% ethanol cost from about $10/day to $1000/day. We examined the cost-effectiveness of ELP in relation to these changes.

METHODS

We simulated a previously developed IF Markov model over 1 year. Costs were measured in 2020 US dollars and effectiveness in quality-adjusted life-years (QALYs). CLABSI rate with and without ELP was estimated from the largest available comparative observational study. The primary outcome was incremental cost-effectiveness ratio (ICER) between treatments. Secondary outcomes included CLABSI frequency. Sensitivity analyses on all model parameters were performed.

RESULTS

In the base model, children with IF not using ELP accumulated $131,815 in costs and 0.32 QALYs per patient compared with $437,884 and 0.33 QALYs per patient in those using ELP. The ICER was nearly $17 million/QALY gained. ELP resulted in a 40% reduction in CLABSI frequency. ELP became cost-effective at $68/day and cost-saving at $63/day. Sensitivity analysis identified no other plausible parameter variation to reach the benchmark of $100,000/QALY gained.

CONCLUSIONS

At the current price, ELP is not cost-effective for CLABSI prevention in children with IF in the United States. This study highlights the critical need for the approval of an affordable lock therapy option to prevent CLABSIs in these children.

Comparison of Decision Modeling Approaches for Health Technology and Policy Evaluation

We discuss tradeoffs and errors associated with approaches to modeling health economic decisions. Through an application in pharmacogenomic (PGx) testing to guide drug selection for individuals with a genetic variant, we assessed model accuracy, optimal decisions, and computation time for an identical decision scenario modeled 4 ways: using 1) coupled-time differential equations (DEQ), 2) a cohort-based discrete-time state transition model (MARKOV), 3) an individual discrete-time state transition microsimulation model (MICROSIM), and 4) discrete event simulation (DES). Relative to DEQ, the net monetary benefit for PGx testing (v. a reference strategy of no testing) based on MARKOV with rate-to-probability conversions using commonly used formulas resulted in different optimal decisions. MARKOV was nearly identical to DEQ when transition probabilities were embedded using a transition intensity matrix. Among stochastic models, DES model outputs converged to DEQ with substantially fewer simulated patients (1 million) v. MICROSIM (1 billion). Overall, properly embedded Markov models provided the most favorable mix of accuracy and runtime but introduced additional complexity for calculating cost and quality-adjusted life year outcomes due to the inclusion of "jumpover" states after proper embedding of transition probabilities. Among stochastic models, DES offered the most favorable mix of accuracy, reliability, and speed.

Prioritizing Additional Data Collection to Reduce Decision Uncertainty in the HIV/AIDS Response in 6 US Cities: A Value of Information Analysis

OBJECTIVES

The ambitious goals of the US Ending the HIV Epidemic initiative will require a targeted, context-specific public health response. Model-based economic evaluation provides useful guidance for decision making while characterizing decision uncertainty. We aim to quantify the value of eliminating uncertainty about different parameters in selecting combination implementation strategies to reduce the public health burden of HIV/AIDS in 6 US cities and identify future data collection priorities.

METHODS

We used a dynamic compartmental HIV transmission model developed for 6 US cities to evaluate the cost-effectiveness of a range of combination implementation strategies. Using a metamodeling approach with nonparametric and deep learning methods, we calculated the expected value of perfect information, representing the maximum value of further research to eliminate decision uncertainty, and the expected value of partial perfect information for key groups of parameters that would be collected together in practice.

RESULTS

The population expected value of perfect information ranged from $59 683 (Miami) to $54 108 679 (Los Angeles). The rank ordering of expected value of partial perfect information on key groups of parameters were largely consistent across cities and highest for parameters pertaining to HIV risk behaviors, probability of HIV transmission, health service engagement, HIV-related mortality, health utility weights, and healthcare costs. Los Angeles was an exception, where parameters on retention in pre-exposure prophylaxis ranked highest in contributing to decision uncertainty.

CONCLUSIONS

Funding additional data collection on HIV/AIDS may be warranted in Baltimore, Los Angeles, and New York City. Value of information analysis should be embedded into decision-making processes on funding future research and public health intervention.

Active learning for efficiently training emulators of computationally expensive mathematical models

An emulator is a fast-to-evaluate statistical approximation of a detailed mathematical model (simulator). When used in lieu of simulators, emulators can expedite tasks that require many repeated evaluations, such as sensitivity analyses, policy optimization, model calibration, and value-of-information analyses. Emulators are developed using the output of simulators at specific input values (design points). Developing an emulator that closely approximates the simulator can require many design points, which becomes computationally expensive. We describe a self-terminating active learning algorithm to efficiently develop emulators tailored to a specific emulation task, and compare it with algorithms that optimize geometric criteria (random latin hypercube sampling and maximum projection designs) and other active learning algorithms (treed Gaussian Processes that optimize typical active learning criteria). We compared the algorithms' root mean square error (RMSE) and maximum absolute deviation from the simulator (MAX) for seven benchmark functions and in a prostate cancer screening model. In the empirical analyses, in simulators with greatly varying smoothness over the input domain, active learning algorithms resulted in emulators with smaller RMSE and MAX for the same number of design points. In all other cases, all algorithms performed comparably. The proposed algorithm attained satisfactory performance in all analyses, had smaller variability than the treed Gaussian Processes, and, on average, had similar or better performance as the treed Gaussian Processes in six out of seven benchmark functions and in the prostate cancer model.

Cost-effectiveness of teduglutide in pediatric patients with short bowel syndrome: Markov modeling using traditional cost-effectiveness criteria

BACKGROUND

Teduglutide use in pediatric patients with short bowel syndrome can aid in the achievement of enteral autonomy, but with a price of >$400,000 per y.

OBJECTIVE

The current study evaluated the cost-effectiveness of using teduglutide in conjunction with offering intestinal transplantation in US pediatric patients with short bowel syndrome.

DESIGN

A Markov model was used to evaluate the costs (in US dollars) and effectiveness [in quality-adjusted life years (QALYs)] of using teduglutide compared with offering intestinal transplantation. Parameters were estimated from published data where available. The primary effect modeled was the probability of weaning from parenteral nutrition while on teduglutide. Sensitivity analyses were performed on all model parameters.

RESULTS

Compared with offering only intestinal transplantation, adding teduglutide cost ${\$}$124,353/QALY gained. Reducing the cost of the medication by 16% allowed the cost to reach the typical benchmark of ${\$}$100,000/QALY gained. Probabilistic sensitivity analysis favored transplantation without offering teduglutide in 68% of iterations at a ${\$}$100,000/QALY threshold. Never using teduglutide created an opportunity cost of over ${\$}$100,000 per patient.

CONCLUSIONS

At its current price, teduglutide does not provide a cost-effective addition to transplantation in the treatment of pediatric short bowel syndrome. Further work should look to identify cost-reducing strategies, including alternative dosing regimens.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.

Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies

Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

Cost-effectiveness of teduglutide in adult patients with short bowel syndrome: Markov modeling using traditional cost-effectiveness criteria

BACKGROUND

Adults with short bowel syndrome have a high mortality and significant morbidity due to unsuccessful attempts at rehabilitation that necessitate chronic use of parenteral nutrition (PN). Teduglutide is a novel therapy that promotes intestinal adaptation to improve rehabilitation but with a price >$400,000/y.

OBJECTIVE

The current study evaluated the cost-effectiveness of using teduglutide in US adult patients with short bowel syndrome.

METHODS

A Markov model evaluated the costs (in US dollars) and effectiveness (in quality-adjusted life years, or QALYs) of treatment compared with no teduglutide use, with a presumed starting age of 40 y. Parameters were obtained from published data or estimation. The primary effect modeled was the increased likelihood of reduced PN days per week when using teduglutide, leading to greater quality of life and lower PN costs. Sensitivity analyses were performed on all model parameters.

RESULTS

In the base scenario, teduglutide cost $949,910/QALY gained. In 1-way sensitivity analyses, only reducing teduglutide cost decreased the cost/QALY gained to below the typical threshold of $100,000/QALY gained. Specifically, teduglutide cost would need to be reduced by >65% for it to reach the threshold value. Probabilistic sensitivity analysis favored no teduglutide use in 80% of iterations at a $100,000/QALY threshold. However, teduglutide therapy was cost-saving in 13% of model iterations.

CONCLUSIONS

Teduglutide does not meet a traditional cost-effectiveness threshold as treatment for PN reduction in adult patients with short bowel syndrome compared with standard intestinal rehabilitation. Subpopulations that demonstrate maximum benefit could be cost-saving, and complete nonuse could lead to financial loss. Teduglutide becomes economically reasonable only if its cost is substantially reduced.

Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers

Background. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies. Methods. We developed linear regression metamodels of the Cost-Effectiveness of Preventing AIDS Complications Pediatric (CEPAC-P) microsimulation model used to predict life expectancy and lifetime HIV-related costs/person of two infant HIV testing/screening programs in South Africa. Metamodel performance was assessed with cross-validation and Bland-Altman plots, showing between-method differences in predicted outcomes against their means. Predictive validity was determined by the percentage of simulations in which the metamodels accurately predicted the strategy with the greatest net health benefit (NHB) as projected by the CEPAC-P model. We introduced a zone of indifference and investigated the width needed to produce between-method agreement in 95% of the simulations. We also calculated NHB losses from "wrong" decisions by the metamodel. Results. In cross-validation, linear regression metamodels accurately approximated CEPAC-P-projected outcomes. For life expectancy, Bland-Altman plots showed good agreement between CEPAC-P and the metamodel (within 1.1 life-months difference). For costs, 95% of between-method differences were within $65/person. The metamodels predicted the same optimal strategy as the CEPAC-P model in 87.7% of simulations, increasing to 95% with a zone of indifference of 0.24 life-months ( ∼ 7 days). The losses in health benefits due to "wrong" choices by the metamodel were modest (range: 0.0002-1.1 life-months). Conclusions. For this policy question, linear regression metamodels offered sufficient predictive validity for the optimal testing strategy as compared with the CEPAC-P model. Metamodels can simulate different scenarios in real time, based on sets of input parameters that can be depicted in a widely accessible decision-support tool.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Artificial neural network metamodel for sensitivity analysis in a total hip replacement health economic model

Objectives: Metamodels have been used to approximate complex simulations and have many applications with sensitivity analysis, optimization, etc. However, their use in health economics is very limited. Application of artificial neural network (ANN) with a health economic model has never been investigated. The study intends to introduce ANN as a metamodeling method to conduct sensitivity analysis in a total hip replacement decision analytical model and compare its performance with two other counterparts. Methods: First, a nonlinear factor screening method was adopted to screen out unimportant factors from the simulation. Second, an ANN was developed using the important variables to approximate the simulation. Performance of the ANN metamodel was then compared with its Gaussian Process (GP) and multiple linear regression (MLR) counterparts. Results: Out of 31, the factor screening method identified 12 important variables from the simulation. ANN metamodels showed best predictive capabilities in terms of performance measures (mean squared error of prediction, MSEP and mean absolute percentage deviation, MAPD) used for predicting both costs and quality-adjusted life years (QALYs) for two prostheses. Conclusion: The study provides a methodological development in sensitivity analysis and demonstrates that an ANN metamodel is a potential approximation method for computationally expensive health economic simulations.

The Curve of Optimal Sample Size (COSS): A Graphical Representation of the Optimal Sample Size from a Value of Information Analysis

Value of information (VOI) analysis quantifies the opportunity cost associated with decision uncertainty, and thus informs the value of collecting further information to avoid this cost. VOI can inform study design, optimal sample size selection, and research prioritization. Recent methodological advances have reduced the computational burden of conducting VOI analysis and have made it easier to evaluate the expected value of sample information, the expected net benefit of sampling, and the optimal sample size of a study design ([Formula: see text]). The volume of VOI analyses being published is increasing, and there is now a need for VOI studies to conduct sensitivity analyses on VOI-specific parameters. In this practical application, we introduce the curve of optimal sample size (COSS), which is a graphical representation of [Formula: see text] over a range of willingness-to-pay thresholds and VOI parameters (example data and R code are provided). In a single figure, the COSS presents summary data for decision makers to determine the sample size that optimizes research funding given their operating characteristics. The COSS also presents variation in the optimal sample size given variability or uncertainty in VOI parameters. The COSS represents an efficient and additional approach for summarizing results from a VOI analysis.

Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression

Background. The expected value of sample information (EVSI) determines the economic value of any future study with a specific design aimed at reducing uncertainty about the parameters underlying a health economic model. This has potential as a tool for trial design; the cost and value of different designs could be compared to find the trial with the greatest net benefit. However, despite recent developments, EVSI analysis can be slow, especially when optimizing over a large number of different designs. Methods. This article develops a method to reduce the computation time required to calculate the EVSI across different sample sizes. Our method extends the moment-matching approach to EVSI estimation to optimize over different sample sizes for the underlying trial while retaining a similar computational cost to a single EVSI estimate. This extension calculates the posterior variance of the net monetary benefit across alternative sample sizes and then uses Bayesian nonlinear regression to estimate the EVSI across these sample sizes. Results. A health economic model developed to assess the cost-effectiveness of interventions for chronic pain demonstrates that this EVSI calculation method is fast and accurate for realistic models. This example also highlights how different trial designs can be compared using the EVSI. Conclusion. The proposed estimation method is fast and accurate when calculating the EVSI across different sample sizes. This will allow researchers to realize the potential of using the EVSI to determine an economically optimal trial design for reducing uncertainty in health economic models. Limitations. Our method involves rerunning the health economic model, which can be more computationally expensive than some recent alternatives, especially in complex models.

Calculating the Expected Value of Sample Information Using Efficient Nested Monte Carlo: A Tutorial

OBJECTIVE

The expected value of sample information (EVSI) quantifies the economic benefit of reducing uncertainty in a health economic model by collecting additional information. This has the potential to improve the allocation of research budgets. Despite this, practical EVSI evaluations are limited partly due to the computational cost of estimating this value using the gold-standard nested simulation methods. Recently, however, Heath et al. developed an estimation procedure that reduces the number of simulations required for this gold-standard calculation. Up to this point, this new method has been presented in purely technical terms.

STUDY DESIGN

This study presents the practical application of this new method to aid its implementation. We use a worked example to illustrate the key steps of the EVSI estimation procedure before discussing its optimal implementation using a practical health economic model.

METHODS

The worked example is based on a three-parameter linear health economic model. The more realistic model evaluates the cost-effectiveness of a new chemotherapy treatment, which aims to reduce the number of side effects experienced by patients. We use a Markov model structure to evaluate the health economic profile of experiencing side effects.

RESULTS

This EVSI estimation method offers accurate estimation within a feasible computation time, seconds compared to days, even for more complex model structures. The EVSI estimation is more accurate if a greater number of nested samples are used, even for a fixed computational cost.

CONCLUSIONS

This new method reduces the computational cost of estimating the EVSI by nested simulation.

Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations

Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.

A Gaussian Approximation Approach for Value of Information Analysis

Most decisions are associated with uncertainty. Value of information (VOI) analysis quantifies the opportunity loss associated with choosing a suboptimal intervention based on current imperfect information. VOI can inform the value of collecting additional information, resource allocation, research prioritization, and future research designs. However, in practice, VOI remains underused due to many conceptual and computational challenges associated with its application. Expected value of sample information (EVSI) is rooted in Bayesian statistical decision theory and measures the value of information from a finite sample. The past few years have witnessed a dramatic growth in computationally efficient methods to calculate EVSI, including metamodeling. However, little research has been done to simplify the experimental data collection step inherent to all EVSI computations, especially for correlated model parameters. This article proposes a general Gaussian approximation (GA) of the traditional Bayesian updating approach based on the original work by Raiffa and Schlaifer to compute EVSI. The proposed approach uses a single probabilistic sensitivity analysis (PSA) data set and involves 2 steps: 1) a linear metamodel step to compute the EVSI on the preposterior distributions and 2) a GA step to compute the preposterior distribution of the parameters of interest. The proposed approach is efficient and can be applied for a wide range of data collection designs involving multiple non-Gaussian parameters and unbalanced study designs. Our approach is particularly useful when the parameters of an economic evaluation are correlated or interact.

Implementing Generalized Additive Models to Estimate the Expected Value of Sample Information in a Microsimulation Model: Results of Three Case Studies

BACKGROUND

The expected value of sample information (EVSI) can help prioritize research but its application is hampered by computational infeasibility, especially for complex models. We investigated an approach by Strong and colleagues to estimate EVSI by applying generalized additive models (GAM) to results generated from a probabilistic sensitivity analysis (PSA).

METHODS

For 3 potential HIV prevention and treatment strategies, we estimated life expectancy and lifetime costs using the Cost-effectiveness of Preventing AIDS Complications (CEPAC) model, a complex patient-level microsimulation model of HIV progression. We fitted a GAM-a flexible regression model that estimates the functional form as part of the model fitting process-to the incremental net monetary benefits obtained from the CEPAC PSA. For each case study, we calculated the expected value of partial perfect information (EVPPI) using both the conventional nested Monte Carlo approach and the GAM approach. EVSI was calculated using the GAM approach.

RESULTS

For all 3 case studies, the GAM approach consistently gave similar estimates of EVPPI compared with the conventional approach. The EVSI behaved as expected: it increased and converged to EVPPI for larger sample sizes. For each case study, generating the PSA results for the GAM approach required 3 to 4 days on a shared cluster, after which EVPPI and EVSI across a range of sample sizes were evaluated in minutes. The conventional approach required approximately 5 weeks for the EVPPI calculation alone.

CONCLUSION

Estimating EVSI using the GAM approach with results from a PSA dramatically reduced the time required to conduct a computationally intense project, which would otherwise have been impractical. Using the GAM approach, we can efficiently provide policy makers with EVSI estimates, even for complex patient-level microsimulation models.

Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching

BACKGROUND

The Expected Value of Sample Information (EVSI) is used to calculate the economic value of a new research strategy. Although this value would be important to both researchers and funders, there are very few practical applications of the EVSI. This is due to computational difficulties associated with calculating the EVSI in practical health economic models using nested simulations.

METHODS

We present an approximation method for the EVSI that is framed in a Bayesian setting and is based on estimating the distribution of the posterior mean of the incremental net benefit across all possible future samples, known as the distribution of the preposterior mean. Specifically, this distribution is estimated using moment matching coupled with simulations that are available for probabilistic sensitivity analysis, which is typically mandatory in health economic evaluations.

RESULTS

This novel approximation method is applied to a health economic model that has previously been used to assess the performance of other EVSI estimators and accurately estimates the EVSI. The computational time for this method is competitive with other methods.

CONCLUSION

We have developed a new calculation method for the EVSI which is computationally efficient and accurate.

LIMITATIONS

This novel method relies on some additional simulation so can be expensive in models with a large computational cost.

An Overview of R in Health Decision Sciences

As the complexity of health decision science applications increases, high-level programming languages are increasingly adopted for statistical analyses and numerical computations. These programming languages facilitate sophisticated modeling, model documentation, and analysis reproducibility. Among the high-level programming languages, the statistical programming framework R is gaining increased recognition. R is freely available, cross-platform compatible, and open source. A large community of users who have generated an extensive collection of well-documented packages and functions supports it. These functions facilitate applications of health decision science methodology as well as the visualization and communication of results. Although R’s popularity is increasing among health decision scientists, methodological extensions of R in the field of decision analysis remain isolated. The purpose of this article is to provide an overview of existing R functionality that is applicable to the various stages of decision analysis, including model design, input parameter estimation, and analysis of model outputs.

Value of Information Analysis Informing Adoption and Research Decisions in a Portfolio of Health Care Interventions

Background: Value of information (VOI) analysis quantifies the value of additional research in reducing decision uncertainty. It addresses adoption and research decisions simultaneously by comparing the expected benefits and costs of research studies. Nevertheless, the application of this approach in practice remains limited. Objectives: To apply VOI analysis in health care interventions to guide adoption decisions, optimize trial design, and prioritize research. Methods: The analysis was from the perspective of Queensland Health, Australia. It included four interventions: clinically indicated catheter replacement, tissue adhesive for securing catheters, negative pressure wound therapy (NPWT) in caesarean sections, and nutritional support for preventing pressure ulcers. For each intervention, cost-effectiveness analysis was performed, decision uncertainty characterized, and VOI calculated using Monte Carlo simulations. The benefits and costs of additional research were considered together with the costs and consequences of acting now versus waiting for more information. All values are reported in 2014 Australian dollars (AU$). Results: All interventions were cost-effective, but with various levels of decision uncertainty. The current evidence is sufficient to support the adoption of clinically indicated catheter replacement. For the tissue adhesive, an additional study before adoption is worthwhile with a four-arm trial of 220 patients per arm. Additional research on NPWT before adoption is worthwhile with a two-arm trial of 200 patients per arm. Nutritional support should be adopted with a two-arm trial of 1200 patients per arm. Based on the expected net monetary benefits, the studies were ranked as follows: 1) NPWT (AU$1.2 million), 2) tissue adhesive (AU$0.3 milliion), and 3) nutritional support (AU$0.1 million). Conclusions: VOI analysis is a useful and practical approach to inform adoption and research decisions. Efforts should be focused on facilitating its integration into decision making frameworks.

Efficient Value of Information Calculation Using a Nonparametric Regression Approach: An Applied Perspective

BACKGROUND

Value-of-information (VOI) analysis provides an analytical framework to assess whether obtaining additional evidence is worthwhile to reduce decision uncertainty. The reporting of VOI measures, particularly the expected value of perfect parameter information (EVPPI) and the expected value of sample information (EVSI), is limited because of the computational burden associated with typical two-level Monte-Carlo-based solution. Recently, a nonparametric regression approach was proposed that allows the estimation of multiparameter EVPPI and EVSI directly from a probabilistic sensitivity analysis sample.

OBJECTIVES

To demonstrate the value of the nonparametric regression approach in calculating VOI measures in real-world cases and to compare its performance with the standard approach of the Monte-Carlo simulation.

METHODS

We used the regression approach to calculate EVPPI and EVSI in two models, and compared the results with the estimates obtained via the standard Monte-Carlo simulation.

RESULTS

The VOI values from the two approaches were very close; computation using the regression method, however, was faster.

CONCLUSION

The nonparametric regression approach provides an efficient and easy-to-implement alternative for EVPPI and EVSI calculation in economic models.