Expected Value of Sample Information Calculations in Medical Decision Modeling

Methods to assess cost-effectiveness and value of further research when data are sparse: negative-pressure wound therapy for severe pressure ulcers

Health care resources are scarce, and decisions have to be made about how to allocate funds. Often, these decisions are based on sparse or imperfect evidence. One such example is negative-pressure wound therapy (NPWT), which is a widely used treatment for severe pressure ulcers; however, there is currently no robust evidence that it is effective or cost-effective. This work considers the decision to adopt NPWT given a range of alternative treatments, using a decision analytic modeling approach. Literature searches were conducted to identify existing evidence on model parameters. Given the limited evidence base, a second source of evidence, beliefs elicited from experts, was used. Judgments from experts on relevant (uncertain) quantities were obtained through a formal elicitation exercise. Additionally, data derived from a pilot trial were also used to inform the model. The 3 sources of evidence were collated, and the impact of each on cost-effectiveness was evaluated. An analysis of the value of further information indicated that a randomized controlled trial may be worthwhile in reducing decision uncertainty, where from a set of alternative designs, a 3-arm trial with longer follow-up was estimated to be the most efficient. The analyses presented demonstrate how allocation decisions about medical technologies can be explicitly informed when data are sparse and how this kind of analyses can be used to guide future research prioritization, not only indicating whether further research is worthwhile but what type of research is needed and how it should be designed.

Cost effectiveness of abdominal aortic aneurysm screening and rescreening in men in a modern context: evaluation of a hypothetical cohort using a decision analytical model

OBJECTIVE

To assess the cost effectiveness of different screening strategies for abdominal aortic aneurysm in men, from the perspective of a national health service.

SETTING

Screening units at regional hospitals.

PARTICIPANTS

Hypothetical cohort of 65 year old men from the general population.

MAIN OUTCOME MEASURES

Costs (£ in 2010) and effect on health outcomes (quality adjusted life years (QALYs)).

RESULTS

Screening seems to be highly cost effective compared with not screening. The model estimated a 92% probability that some form of screening would be cost effective at a threshold of £20,000 (€24,790; $31,460). If men with an aortic diameter of 25-29 mm at the initial screening were rescreened once after five years, 452 men per 100,000 initially screened would benefit from early detection, whereas lifetime rescreening every five years would detect 794 men per 100,000. We estimated the associated incremental cost effectiveness ratios for rescreening once and lifetime rescreening to be £10,013 and £29,680 per QALY, respectively. The individual probability of being the most cost effective strategy was higher for each rescreening strategy than for the screening once strategy (in view of the £20,000 threshold).

CONCLUSIONS

This study confirms the cost effectiveness of screening versus no screening and lends further support to considerations of rescreening men at least once for abdominal aortic aneurysm.

Value of information and pricing new healthcare interventions

Previous application of value-of-information methods to optimal clinical trial design have predominantly taken a societal decision-making perspective, implicitly assuming that healthcare costs are covered through public expenditure and trial research is funded by government or donation-based philanthropic agencies. In this paper, we consider the interaction between interrelated perspectives of a societal decision maker (e.g. the National Institute for Health and Clinical Excellence [NICE] in the UK) charged with the responsibility for approving new health interventions for reimbursement and the company that holds the patent for a new intervention. We establish optimal decision making from societal and company perspectives, allowing for trade-offs between the value and cost of research and the price of the new intervention. Given the current level of evidence, there exists a maximum (threshold) price acceptable to the decision maker. Submission for approval with prices above this threshold will be refused. Given the current level of evidence and the decision maker's threshold price, there exists a minimum (threshold) price acceptable to the company. If the decision maker's threshold price exceeds the company's, then current evidence is sufficient since any price between the thresholds is acceptable to both. On the other hand, if the decision maker's threshold price is lower than the company's, then no price is acceptable to both and the company's optimal strategy is to commission additional research. The methods are illustrated using a recent example from the literature.

Expected net present value of sample information: from burden to investment

The Expected Value of Information Framework has been proposed as a method for identifying when health care technologies should be immediately reimbursed and when any reimbursement should be withheld while awaiting more evidence. This framework assesses the value of obtaining additional evidence to inform a current reimbursement decision. This represents the burden of not having the additional evidence at the time of the decision. However, when deciding whether to reimburse now or await more evidence, decision makers need to know the value of investing in more research to inform a future decision. Assessing this value requires consideration of research costs, research time, and what happens to patients while the research is undertaken and after completion. The investigators describe a development of the calculation of the expected value of sample information that assesses the value of investing in further research, including an only-in-research strategy and an only-with-research strategy.

The role of the expected value of individualized care in cost-effectiveness analyses and decision making

OBJECTIVE

To explore the feasibility and potential role of the expected value of individualized care (EVIC) framework.

METHODS

The EVIC quantifies how much benefits are forgone when a treatment decision is based on the best-expected outcomes in the population rather than in the individual patient. We have reviewed which types of patient-level attributes contribute to the EVIC and how they affect the interpretation of the outcomes. In addition, we have applied the EVIC framework to the outcomes of a microsimulation-based cost-effectiveness analysis for glaucoma treatment.

RESULTS

For EVIC outcomes to inform decisions about clinical practice, we need to calculate the parameter-specific EVIC of known or knowable patient-level attributes and compare it with the real costs of implementing individualized care. In the case study, the total EVIC was €580 per patient, but patient-level attributes known at treatment decision had minimal impact. A subgroup policy based on individual disease progression could be worthwhile if a predictive test for glaucoma progression could be developed and implemented for less than €130 per patient.

CONCLUSIONS

The EVIC framework is feasible in cost-effectiveness analyses and can be informative for decision making. The EVIC outcomes are particularly informative when they are (close to) zero. When the EVIC has a high value, implications depend on the type of patient-level attribute. EVIC can be a useful tool to identify opportunities to improve efficiency in health care by individualization of care and to quantify the maximal investment opportunities for implementing subgroup policy.

Potential uncertainty reduction in model-averaged benchmark dose estimates informed by an additional dose study

A methodology is presented for assessing the information value of an additional dosage experiment in existing bioassay studies. The analysis demonstrates the potential reduction in the uncertainty of toxicity metrics derived from expanded studies, providing insights for future studies. Bayesian methods are used to fit alternative dose-response models using Markov chain Monte Carlo (MCMC) simulation for parameter estimation and Bayesian model averaging (BMA) is used to compare and combine the alternative models. BMA predictions for benchmark dose (BMD) are developed, with uncertainty in these predictions used to derive the lower bound BMDL. The MCMC and BMA results provide a basis for a subsequent Monte Carlo analysis that backcasts the dosage where an additional test group would have been most beneficial in reducing the uncertainty in the BMD prediction, along with the magnitude of the expected uncertainty reduction. Uncertainty reductions are measured in terms of reduced interval widths of predicted BMD values and increases in BMDL values that occur as a result of this reduced uncertainty. The methodology is illustrated using two existing data sets for TCDD carcinogenicity, fitted with two alternative dose-response models (logistic and quantal-linear). The example shows that an additional dose at a relatively high value would have been most effective for reducing the uncertainty in BMA BMD estimates, with predicted reductions in the widths of uncertainty intervals of approximately 30%, and expected increases in BMDL values of 5-10%. The results demonstrate that dose selection for studies that subsequently inform dose-response models can benefit from consideration of how these models will be fit, combined, and interpreted.

Probabilistic sensitivity analysis in health economics

Health economic evaluations have recently become an important part of the clinical and medical research process and have built upon more advanced statistical decision-theoretic foundations. In some contexts, it is officially required that uncertainty about both parameters and observable variables be properly taken into account, increasingly often by means of Bayesian methods. Among these, probabilistic sensitivity analysis has assumed a predominant role. The objective of this article is to review the problem of health economic assessment from the standpoint of Bayesian statistical decision theory with particular attention to the philosophy underlying the procedures for sensitivity analysis.

Is a HIV vaccine a viable option and at what price? An economic evaluation of adding HIV vaccination into existing prevention programs in Thailand

Background

This study aims to determine the maximum price at which HIV vaccination is cost-effective in the Thai healthcare setting. It also aims to identify the relative importance of vaccine characteristics and risk behavior changes among vaccine recipients to determine how they affect this cost-effectiveness.

Methods

A semi-Markov model was developed to estimate the costs and health outcomes of HIV prevention programs combined with HIV vaccination in comparison to the existing HIV prevention programs without vaccination. The estimation was based on a lifetime horizon period (99 years) and used the government perspective. The analysis focused on both the general population and specific high-risk population groups. The maximum price of cost-effective vaccination was defined by using threshold analysis; one-way and probabilistic sensitivity analyses were performed. The study employed an expected value of perfect information (EVPI) analysis to determine the relative importance of parameters and to prioritize future studies.

Results

The most expensive HIV vaccination which is cost-effective when given to the general population was 12,000 Thai baht (US$1 = 34 Thai baht in 2009). This vaccination came with 70% vaccine efficacy and lifetime protection as long as risk behavior was unchanged post-vaccination. The vaccine would be considered cost-ineffective at any price if it demonstrated low efficacy (30%) and if post-vaccination risk behavior increased by 10% or more, especially among the high-risk population groups. The incremental cost-effectiveness ratios were the most sensitive to change in post-vaccination risk behavior, followed by vaccine efficacy and duration of protection. The EVPI indicated the need to quantify vaccine efficacy, changed post-vaccination risk behavior, and the costs of vaccination programs.

Conclusions

The approach used in this study differentiated it from other economic evaluations and can be applied for the economic evaluation of other health interventions not available in healthcare systems. This study is important not only for researchers conducting future HIV vaccine research but also for policy decision makers who, in the future, will consider vaccine adoption.

Development of a transparent interactive decision interrogator to facilitate the decision-making process in health care

BACKGROUND

Decisions about the use of new technologies in health care are often based on complex economic models. Decision makers frequently make informal judgments about evidence, uncertainty, and the assumptions that underpin these models.

OBJECTIVES

Transparent interactive decision interrogator (TIDI) facilitates more formal critique of decision models by decision makers such as members of appraisal committees of the National Institute for Health and Clinical Excellence in the UK. By allowing them to run advanced statistical models under different scenarios in real time, TIDI can make the decision process more efficient and transparent, while avoiding limitations on pre-prepared analysis.

METHODS

TIDI, programmed in Visual Basic for applications within Excel, provides an interface for controlling all components of a decision model developed in the appropriate software (e.g., meta-analysis in WinBUGS and the decision model in R) by linking software packages using RExcel and R2WinBUGS. TIDI's graphical controls allow the user to modify assumptions and to run the decision model, and results are returned to an Excel spreadsheet. A tool displaying tornado plots helps to evaluate the influence of individual parameters on the model outcomes, and an interactive meta-analysis module allows the user to select any combination of available studies, explore the impact of bias adjustment, and view results using forest plots. We demonstrate TIDI using an example of a decision model in antenatal care.

CONCLUSION

Use of TIDI during the NICE appraisal of tumor necrosis factor-alpha inhibitors (in psoriatic arthritis) successfully demonstrated its ability to facilitate critiques of the decision models by decision makers.

Estimating expected value of sample information for incomplete data models using Bayesian approximation

Expected value of sample information (EVSI) involves simulating data collection, Bayesian updating, and reexamining decisions. Bayesian updating in incomplete data models typically requires Markov chain Monte Carlo (MCMC). This article describes a revision to a form of Bayesian Laplace approximation for EVSI computation to support decisions in incomplete data models. The authors develop the approximation, setting out the mathematics for the likelihood and log posterior density function, which are necessary for the method. They compare the accuracy of EVSI estimates in a case study cost-effectiveness model using first- and second-order versions of their approximation formula and traditional Monte Carlo. Computational efficiency gains depend on the complexity of the net benefit functions, the number of inner-level Monte Carlo samples used, and the requirement or otherwise for MCMC methods to produce the posterior distributions. This methodology provides a new and valuable approach for EVSI computation in health economic decision models and potential wider benefits in many fields requiring Bayesian approximation.

Addressing Adoption and Research Design Decisions Simultaneously

Methods to estimate the cost-effectiveness of technologies are well developed with increasing experience of their application to inform adoption decisions in a timely way. However, the experience of using similarly explicit methods to inform the associated research decisions is less well developed despite appropriate methods being available with an increasing number of applications in health. The authors demonstrate that evaluation of both adoption and research decisions is feasible within typical time and resource constraints relevant to policy decisions, even in situations in which data are sparse and formal elicitation is required. In addition to demonstrating the application of expected value of sample information (EVSI) in these circumstances, the authors examine and carefully distinguish the impact that the research decision is expected to have on patients while enrolled in the trial, those not enrolled, and once the trial reports. In doing so, the authors are able to account for the range of opportunity cost associated with research and evaluate a number of research designs including length of follow-up and sample size. The authors also explore the implications for research design of conducting research while the technology is approved for widespread use and whether approval should be withheld until research reports. In doing so, the authors highlight the impact of irrecoverable opportunity costs when the initial costs of a technology are compensated only by later gains in health outcome.

Value of information in the osteoarthritis setting: cost effectiveness of COX-2 selective inhibitors, traditional NSAIDs and proton pump inhibitors

BACKGROUND

Recent National Institute for Health and Clinical Excellence (NICE) guidance recommended that when traditional NSAIDs or cyclo-oxygenase (COX)-2 selective inhibitors are used by people with osteoarthritis (OA), they should be prescribed along with a proton pump inhibitor (PPI). However, specific recommendations about the type of NSAID or COX-2 could not be made due to high levels of uncertainty in the economic evaluation.

OBJECTIVE

To investigate the value of obtaining further evidence to inform the economic evaluation of NSAIDs, COX-2s and PPIs for people with OA.

METHODS

An economic evaluation with an expected value of perfect information (EVPI) analysis was conducted, using a Markov model with data identified from a systematic review. The base-case model used adverse event data from the three largest randomized trials of COX-2 inhibitors, and we repeated the analysis using observational adverse event data. The model was run for a hypothetical population of people with OA, and subgroup analyses were conducted for people with raised gastrointestinal (GI) and cardiovascular (CV) risk. The EVPI was based upon the OA population in England - approximately 2.8 million people. Of these, 50% were assumed to use NSAIDs or COX-2 selective inhibitors for 3 months per year and 56% of these were assumed to be patients with raised GI and CV risk.

RESULTS

The value of further information for this decision problem was very high. Population-level EVPI was £85.1 million in the low-risk group and £179.5 million in the high-risk group (2007-8 values). Expected value of partial perfect information (EVPPI) analysis showed that the groups of parameters for which further evidence was likely to be of most value were CV adverse event risks and all adverse event rates associated with the specific drugs celecoxib and ibuprofen. The value of perfect information remained high even when observational adverse event data were used.

CONCLUSIONS

There is a very high value associated with obtaining further information on uncertain parameters for the economic evaluation of NSAIDs, COX-2 selective inhibitors and PPIs for people with OA. Obtaining further randomized or observational information on CV risks is likely to be particularly cost effective.

Dangerous omissions: the consequences of ignoring decision uncertainty

Institutions with the responsibility for making adoption (reimbursement) decisions in health care often lack the remit to demand or commission further research: adoption decisions are their only policy instrument. The decision to adopt a technology also influences the prospects of acquiring further evidence because the incentives to conduct research are reduced and the ethical basis of further clinical trials maybe undermined. In these circumstances the decision maker must consider whether the benefits of immediate access to a technology exceeds the value of the evidence which maybe forgone for future patients. We outline how these expected opportunity losses can be established from the perspective of a societal decision maker with and without the remit to commission research, and demonstrate how these considerations change the appropriate decision rules in cost-effectiveness analysis. Importantly, we identify those circumstances in which the approval of a technology that is expected to be cost-effective should be withheld, i.e. when an 'only in research' recommendation should be made. We demonstrate that a sufficient condition for immediate adoption of a technology can provide incentives for manufacturers to reduce the price or provide additional supporting evidence. However, decisions based solely on expected net benefit provide no such incentives, may undermine the evidence base for future clinical practice and reduce expected net health benefits for the patient population.

Historical lifetimes of drugs in England: application to value of information and cost-effectiveness analyses

OBJECTIVES

The expected lifetime of a health technology is a critical parameter in value of information analysis and in two methodologies for cost-effectiveness analysis which have recently been suggested. The first method allows for the possibility that a superior technology will become available in the future. The second advocates modeling both the prevalent and all future incident patient cohorts. Unfortunately, for value of information analysis, the period of time over which information about the decision problem would be useful is very uncertain, and existing estimates are seemingly arbitrary. Furthermore, there is very little literature on the historical lifetimes of technologies. Here, I quantify and analyze the historical lifetimes of drugs in England. I then apply this information to inform the value of further research and the cost-effectiveness of health technologies.

METHODS

A Weibull regression model was fitted to the historical drug lifetimes of 455 drugs. These represented all British National Formulary drugs in England which were launched from 1981 to 2007, and which did not have very low sales volumes.

RESULTS

The mean drug lifetime was 57 years (95% confidence interval 39-79 years), and the median was 46 years (35-60 years). Drugs with low sales volumes tended to have shorter lifetimes. Under certain assumptions, the ratio of population level to per-year expected value of information is 21. Drug lifetimes are used to parameterize the two models of cost-effectiveness.

CONCLUSIONS

The distribution function of the historical lifetimes of drugs can inform suitable time horizons for: 1) value of information; and 2) cost-effectiveness analyses related to drugs.

How is evidence on test performance synthesized for economic decision models of diagnostic tests? A systematic appraisal of Health Technology Assessments in the UK since 1997

OBJECTIVES

The study aims to assess how evidence on diagnostic test accuracy is synthesized and used to inform economic decision modeling for HTA.

METHODS

All reports evaluating diagnostic test via an economic decision model published by the NHS Research and Development Health Technology Assessment (HTA) program since 1997 were identified. The methods for evidence synthesis of diagnostic test accuracy data and its use in economic decision modeling in this sample were reviewed.

RESULTS

Forty-four HTA reports out of 474 concerned diagnostic accuracy, of which 11 did not do any economic evaluation. Of the remaining 33 HTAs, 14 conducted meta-analyses of diagnostic accuracy in the clinical review but only 8 used such pooled estimates to inform the decision model. A number of meta-analysis methods ranging in complexity were applied to estimate diagnostic accuracy. Nevertheless, when it came to informing the economic decision model, the majority of reviews used independent meta-analytic estimates of sensitivity and specificity.

CONCLUSIONS

Often, very simplistic methods to estimate diagnostic test accuracy were used for purposes of informing an economic decision model. The assumptions made by the simplistic methods are usually invalid which may lead to suboptimal decisions being made. It is desirable that decision modelers become aware of the rapid evolution of meta-analysis methods in this area; however, further research is still required to identify how the pooled results obtained from the different meta-analysis models should best be used to inform economic decision models.

The Combined Analysis of Uncertainty and Patient Heterogeneity in Medical Decision Models

The analysis of both patient heterogeneity and parameter uncertainty in decision models is increasingly recommended. In addition, the complexity of current medical decision models commonly requires simulating individual subjects, which introduces stochastic uncertainty. The combined analysis of uncertainty and heterogeneity often involves complex nested Monte Carlo simulations to obtain the model outcomes of interest. In this article, the authors distinguish eight model types, each dealing with a different combination of patient heterogeneity, parameter uncertainty, and stochastic uncertainty. The analyses that are required to obtain the model outcomes are expressed in equations, explained in stepwise algorithms, and demonstrated in examples. Patient heterogeneity is represented by frequency distributions and analyzed with Monte Carlo simulation. Parameter uncertainty is represented by probability distributions and analyzed with 2nd-order Monte Carlo simulation (aka probabilistic sensitivity analysis). Stochastic uncertainty is analyzed with 1st-order Monte Carlo simulation (i.e., trials or random walks). This article can be used as a reference for analyzing complex models with more than one type of uncertainty and patient heterogeneity.

Long term costs and effects of reducing the number of twin pregnancies in IVF by single embryo transfer: the TwinSing study

Background

Pregnancies induced by in vitro fertilisation (IVF) often result in twin gestations, which are associated with both maternal and perinatal complications. An effective way to reduce the number of IVF twin pregnancies is to decrease the number of embryos transferred from two to one. The interpretation of current studies is limited because they used live birth as outcome measure and because they applied limited time horizons. So far, research on long-term outcomes of IVF twins and singletons is scarce and inconclusive. The objective of this study is to investigate the short (1-year) and long-term (5 and 18-year) costs and health outcomes of IVF singleton and twin children and to consider these in estimating the cost-effectiveness of single embryo transfer compared with double embryo transfer, from a societal and a healthcare perspective.

Methods/Design

A multi-centre cohort study will be performed, in which IVF singletons and IVF twin children born between 2003 and 2005 of whom parents received IVF treatment in one of the five participating Dutch IVF centres, will be compared. Data collection will focus on children at risk of health problems and children in whom health problems actually occurred. First year of life data will be collected in approximately 1,278 children (619 singletons and 659 twin children). Data up to the fifth year of life will be collected in approximately 488 children (200 singletons and 288 twin children). Outcome measures are health status, health-related quality of life and costs. Data will be obtained from hospital information systems, a parent questionnaire and existing registries. Furthermore, a prognostic model will be developed that reflects the short and long-term costs and health outcomes of IVF singleton and twin children. This model will be linked to a Markov model of the short-term cost-effectiveness of single embryo transfer strategies versus double embryo transfer strategies to enable the calculation of the long-term cost-effectiveness.

Discussion

This is, to our knowledge, the first study that investigates the long-term costs and health outcomes of IVF singleton and twin children and the long-term cost-effectiveness of single embryo transfer strategies versus double embryo transfer strategies.

Early-stage valuation of medical devices: the role of developmental uncertainty

BACKGROUND

At the concept stage, many uncertainties surround the commercial viability of a new medical device. These include the ultimate functionality of the device, the cost of producing it and whether, and at what price, it can be sold to a health-care provider (HCP). Simple assessments of value can be made by estimating such unknowns, but the levels of uncertainty may mean that their operational value for investment decisions is unclear. However, many decisions taken at the concept stage are reversible and will be reconsidered later before the product is brought to market. This flexibility can be exploited to enhance early-stage valuations.

OBJECTIVES

To develop a framework for valuing a new medical device at the concept stage that balances benefit to the HCP against commercial costs. This is done within a simplified stage-gated model of the development cycle for new products. The approach is intended to complement existing proposals for the evaluation of the commercial headroom available to new medical products.

CONCLUSIONS

A model based on two decision gates can lead to lower bounds (underestimates) for product value that can serve to support a decision to develop the product. Quantifiable uncertainty that can be resolved before the device is brought to market will generally enhance early-stage valuations of the device, and this remains true even when some components of uncertainty cannot be fully described. Clinical trials and other evidence-gathering activities undertaken as part of the development process can contribute to early-stage estimates of value.

Health economics in drug development: efficient research to inform healthcare funding decisions

In order to decide whether a new treatment should be used in patients, a robust estimate of efficacy and toxicity is no longer sufficient. As a result of increasing healthcare costs across the globe healthcare payers and providers now seek estimates of cost-effectiveness as well. Most trials currently being designed still only consider the need for prospective efficacy and toxicity data during the development life-cycle of a new intervention. Hence the cost-effectiveness estimates are inevitably less precise than the clinical data on which they are based. Methods based on decision theory are being developed by health economists that can contribute to the design of clinical trials in such a way that they can more effectively lead to better informed drug funding decisions on the basis of cost-effectiveness in addition to clinical outcomes. There is an opportunity to apply these techniques prospectively in the design of future clinical trials. This article describes the problems encountered by those responsible for drug reimbursement decisions as a consequence of the current drug development pathway. The potential for decision theoretic methods to help overcome these problems is introduced and potential obstacles in implementation are highlighted.

The cost effectiveness of a randomized controlled trial to establish the relative efficacy of vitamin K1 compared with alendronate

PURPOSE

The authors aimed to evaluate whether vitamin K(1) or alendronate (the recommended treatment in England and Wales for postmenopausal women with a previous fracture) appeared to be the more cost-effective treatment for fracture prevention. Furthermore, expected value of sample information (EVSI) analyses were undertaken to estimate whether a head-to-head trial of alendronate and vitamin K(1) would be considered cost effective.

METHOD

A published osteoporosis model structure, populated with data from literature reviews, was used to evaluate the costs and quality-adjusted life-years associated with each intervention being provided to women at high risk of fracture, given current information. A lifetime horizon and a national health service and personal social services cost perspective were used. Observed outcomes from head-to-head randomized controlled trials (RCTs) of predetermined sizes were simulated and synthesized with existing data to formulate posterior distributions, which were used to estimate the more cost-effective treatment given these additional data. The EVSI was estimated and the expected net benefit of sampling (ENBS) calculated by subtracting the proposed trial costs.

RESULTS

Given current information, vitamin K(1) is expected to dominate alendronate. However, this was subject to a considerable degree of uncertainty; dominance was reversed when it was assumed that vitamin K(1) had no effect on hip fractures. EVSI analysis indicated that an RCT of 2000 or 5000 women per arm produced high, and comparable, ENBS. These results were maintained in sensitivity analyses.

CONCLUSIONS

It is concluded that an RCT recruiting between 2000 and 5000 women per arm to evaluate the relative efficacy of alendronate and vitamin K(1) appears to be cost effective for informing decision making in England and Wales.

Value of information analyses of economic randomized controlled trials: the treatment of intermittent claudication

OBJECTIVE

The aim of this study is to design the optimal study comparing endovascular revascularization and supervised exercise training for patients with intermittent claudication and to demonstrate value of information (VOI) analysis of patient-level data from an economic randomized controlled trial to guide future research.

METHODS

We applied a net benefit framework to patient-level data on costs and quality-of-life of a previous randomized controlled trial. VOI analyses were performed using Monte Carlo simulation. We estimated the total expected value of perfect information (total EVPI), the total expected value of sample information (total EVSI), the partial expected value of perfect information (partial EVPI), and the partial expected value of sample information (partial EVSI). These VOI analyses identified the key parameters and the optimal sample size of future study designs. Sensitivity analyses were performed to explore the robustness of our assumptions about the population to benefit, the willingness-to-pay threshold, and the study costs. The VOI analyses are demonstrated in statistical software (R) and a spreadsheet (Excel) allowing other investigators to apply VOI analysis to their patient-level data.

RESULTS

The optimal study design for the treatment of intermittent claudication involves a randomized controlled trial collecting data on the quality-adjusted life expectancy and additional admission costs for 525 patients per treatment arm. The optimal sample size remained between 400 and 600 patients for a willingness-to-pay threshold between euro30,000 and euro100,000/quality-adjusted life-years, for even extreme assumptions about the study costs, and for a range of 3 to 7 years that future patients will benefit from the results of the proposed study.

CONCLUSIONS

1) The optimal study for patients with intermittent claudication collects data on two key parameters for 525 patients per trial arm; and 2) we have shown that value of information analysis provides an explicit framework to determine the optimal sample size and identify key parameters for the design of future clinical trials.

Uncertainty and Patient Heterogeneity in Medical Decision Models

Parameter uncertainty, patient heterogeneity, and stochastic uncertainty of outcomes are increasingly important concepts in medical decision models. The purpose of this study is to demonstrate the various methods to analyze uncertainty and patient heterogeneity in a decision model. The authors distinguish various purposes of medical decision modeling, serving various stakeholders. Differences and analogies between the analyses are pointed out, as well as practical issues. The analyses are demonstrated with an example comparing imaging tests for patients with chest pain. For complicated analyses step-by-step algorithms are provided. The focus is on Monte Carlo simulation and value of information analysis. Increasing model complexity is a major challenge for probabilistic sensitivity analysis and value of information analysis. The authors discuss nested analyses that are required in patient-level models, and in nonlinear models for analyses of partial value of information analysis.

Cost-effectiveness of breech version by acupuncture-type interventions on BL 67, including moxibustion, for women with a breech foetus at 33 weeks gestation: a modelling approach

OBJECTIVES

To assess, using a modelling approach, the effectiveness and costs of breech version with acupuncture-type interventions on BL67 (BVA-T), including moxibustion, compared to expectant management for women with a foetal breech presentation at 33 weeks gestation.

DESIGN

A decision tree was developed to predict the number of caesarean sections prevented by BVA-T compared to expectant management to rectify breech presentation. The model accounted for external cephalic versions (ECV), treatment compliance, and costs for 10,000 simulated breech presentations at 33 weeks gestational age. Event rates were taken from Dutch population data and the international literature, and the relative effectiveness of BVA-T was based on a specific meta-analysis. Sensitivity analyses were conducted to evaluate the robustness of the results.

MAIN OUTCOME MEASURES

We calculated percentages of breech presentations at term, caesarean sections, and costs from the third-party payer perspective. Odds ratios (OR) and cost differences of BVA-T versus expectant management were calculated. (Probabilistic) sensitivity analysis and expected value of perfect information analysis were performed.

RESULTS

The simulated outcomes demonstrated 32% breech presentations after BVA-T versus 53% with expectant management (OR 0.61, 95% CI 0.43, 0.83). The percentage caesarean section was 37% after BVA-T versus 50% with expectant management (OR 0.73, 95% CI 0.59, 0.88). The mean cost-savings per woman was euro 451 (95% CI euro 109, euro 775; p=0.005) using moxibustion. Sensitivity analysis showed that if 16% or more of women offered moxibustion complied, it was more effective and less costly than expectant management. To prevent one caesarean section, 7 women had to use BVA-T. The expected value of perfect information from further research was euro0.32 per woman.

CONCLUSIONS

The results suggest that offering BVA-T to women with a breech foetus at 33 weeks gestation reduces the number of breech presentations at term, thus reducing the number of caesarean sections, and is cost-effective compared to expectant management, including external cephalic version.

Budgetary policies and available actions: a generalisation of decision rules for allocation and research decisions

The allocation problem in health care can be characterised as a mathematical programming problem but attempts to incorporate uncertainty in costs and effect have suffered from important limitations. A two-stage stochastic mathematical programming formulation is developed and applied to a numerical example to explore and demonstrate the implications of this more general and comprehensive approach. The solution to the allocation problem for different budgets, budgetary policies, and available actions are then demonstrated. This analysis is used to evaluate different budgetary policies and examine the adequacy of standard decision rules in cost-effectiveness analysis. The research decision is then considered alongside the allocation problem. This more general formulation demonstrates that the value of further research depends on: (i) the budgetary policy in place; (ii) the realisations revealed during the budget period; (iii) remedial actions that may be available; and (iv) variability in parameters values.

The value of value of information: best informing research design and prioritization using current methods

Value of information (VOI) methods have been proposed as a systematic approach to inform optimal research design and prioritization. Four related questions arise that VOI methods could address. (i) Is further research for a health technology assessment (HTA) potentially worthwhile? (ii) Is the cost of a given research design less than its expected value? (iii) What is the optimal research design for an HTA? (iv) How can research funding be best prioritized across alternative HTAs? Following Occam's razor, we consider the usefulness of VOI methods in informing questions 1-4 relative to their simplicity of use. Expected value of perfect information (EVPI) with current information, while simple to calculate, is shown to provide neither a necessary nor a sufficient condition to address question 1, given that what EVPI needs to exceed varies with the cost of research design, which can vary from very large down to negligible. Hence, for any given HTA, EVPI does not discriminate, as it can be large and further research not worthwhile or small and further research worthwhile. In contrast, each of questions 1-4 are shown to be fully addressed (necessary and sufficient) where VOI methods are applied to maximize expected value of sample information (EVSI) minus expected costs across designs. In comparing complexity in use of VOI methods, applying the central limit theorem (CLT) simplifies analysis to enable easy estimation of EVSI and optimal overall research design, and has been shown to outperform bootstrapping, particularly with small samples. Consequently, VOI methods applying the CLT to inform optimal overall research design satisfy Occam's razor in both improving decision making and reducing complexity. Furthermore, they enable consideration of relevant decision contexts, including option value and opportunity cost of delay, time, imperfect implementation and optimal design across jurisdictions. More complex VOI methods such as bootstrapping of the expected value of partial EVPI may have potential value in refining overall research design. However, Occam's razor must be seriously considered in application of these VOI methods, given their increased complexity and current limitations in informing decision making, with restriction to EVPI rather than EVSI and not allowing for important decision-making contexts. Initial use of CLT methods to focus these more complex partial VOI methods towards where they may be useful in refining optimal overall trial design is suggested. Integrating CLT methods with such partial VOI methods to allow estimation of partial EVSI is suggested in future research to add value to the current VOI toolkit.

Exploring the research decision space: the expected value of information for sequential research designs

PURPOSE

To investigate the expected value of partial perfect information (EVPPI) and the research decisions it can address.

METHODS

Expected value of information (EVI) analysis assesses the expected gain in net benefit from further research. Where the expected value of perfect information (EVPI) exceeds the costs of additional research, EVPPI can be used to identify parameters that contribute most to the EVPI and parameters with no EVPPI that may be disregarded as targets for further research. Recently, it was noted that parameters with low EVPPI for a one-off

RESEARCH DESIGN

may be associated with high EVPPI when considered as part of a sequential design. This article examines the characteristics and role of conditional and sequential EVPPI in EVI analysis.

RESULTS

The calculation of EVPPI is demonstrated for single parameters, groups of parameters, and conditional and sequential EVPPI. Conditional EVPPI is the value of perfect information about one parameter, conditional on having obtained perfect information about another. Sequential EVPPI is the value of perfect information for a sequential research design to investigate first one parameter, then another. Conditional EVPPI differs from the individual EVPPI for a single parameter. Sequential EVPPI includes elements from the joint EVPPI for the parameters and the EVPPI for the first parameter in sequence. Sequential designs allow abandonment of research on the second parameter on the basis of additional information obtained on the first.

CONCLUSIONS

The research decision space addressed by EVI analyses can be widened by incorporating sequential EVPPI to assess sequential research designs.

The ISPOR Good Practices for Quality Improvement of Cost-Effectiveness Research Task Force Report

OBJECTIVES

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Health Science Policy Council recommended and the ISPOR Board of Directors approved the formation of a Task Force to critically examine the major issues related to Quality Improvement in Cost-effectiveness Research (QICER). The Council's primary recommendation for this Task Force was that it should report on the quality of cost-effectiveness research and make recommendations to facilitate the improvement of pharmacoeconomics and health outcomes research and its use in stimulating better health care and policy. Task force members were knowledgeable and experienced in medicine, pharmacy, biostatistics, health policy and health-care decision-making, biomedical knowledge transfer, health economics, and pharmacoeconomics. They were drawn from industry, academia, consulting organizations, and advisors to governments and came from Japan, the Netherlands, Canada and the United States.

METHODS

Face-to-face meetings of the Task Force were held at ISPOR North American and European meetings and teleconferences occurred every few months. Literature reviews and surveys were conducted and the first preliminary findings presented at an open forum at the May 2008 ISPOR meeting in Toronto. The final draft report was circulated to the expert reviewer group and then to the entire membership for comment. The draft report was posted on the ISPOR Web site in April 2009. All formal comments received were posted to the association Web site and presented for discussion at the Task Force forum during the ISPOR 14th Annual International Meeting in May 2009. Comments and feedback from the forums, reviewers and membership were considered in the final report. Once Task Force consensus was reached, the article was submitted to Value in Health.

CONCLUSIONS

The QICER Task Force recommends that ISPOR implement the following: * With respect to CER guidelines, that ISPOR promote harmonization of guidelines, allowing for differences in application, regional needs and politics; evaluate available instruments or promote development of a new one that will allow standardized quantification of the impact of CER guidelines on the quality of CER studies; report periodically on those countries or regions that have developed guidelines; periodically evaluate the quality of published studies (those journals with CER guidances) or those submitted to decision-making bodies (as public transparency increases). * With respect to methodologies, that ISPOR promote publication of methodological guidelines in more applied journals in more easily understandable format to transfer knowledge to researchers who need to apply more rigorous methods; promote full availability of models in electronic format to combat space restrictions in hardcopy publications; promote consistency of methodological review for all CER studies; promote adoption of explicit best practices guidelines among regulatory and reimbursement authorities; periodically update all ISPOR Task Force reports; periodically review use of ISPOR Task Force guidelines; periodically report on statistical and methodological challenges in HE; evaluate periodically whether ISPOR's methodological guidelines lead to improved quality; and support training and knowledge transfer of rigorous CER methodologies to researchers and health care decision-makers. * With respect to publications, that ISPOR develop standard CER guidances to which journals will be able to refer their authors and their reviewers; lobby to establish these guidances within the International Committee for Medical Journal Editors (ICMJE) Requirements to which most journals refer in their Author Instructions; provide support in terms of additional reviewer expertise to those journals lacking appropriate reviewers; periodically report on journals publishing CER research; periodically report on the quality of CER publications; and support training and knowledge transfer of the use of these guidelines to researchers and reviewers. * With respect to evidence-based health-care decision-making, that ISPOR recognize at its annual meetings those countries/agencies/private companies/researchers using CER well, and those practitioners and researchers supporting good patient use of CER in decision-making; and promote public presentation of case studies of applied use of CER concepts or guidelines.

Enhancing value of information analyses

OBJECTIVE

The aim of this study was to demonstrate that it is feasible and recommendable to present value of information (VOI) outcomes in terms of underlying costs and effects in addition to costs alone.

METHODS

The benefits of collecting additional information on health economic outcomes before deciding on a preferred policy when evaluating alternative strategies with uncertain outcomes are quantified in a VOI analysis. In general, costs and effects are combined into one single dimension to determine the expected monetary VOI. Separate information on costs and effects is lost. This information, however, remains relevant to the decision-maker. The concept of the attributable VOI (AVOI) is introduced which enables separate presentation of expected changes in health outcomes and costs.

RESULTS

The use of the attributable expected value of perfect information is illustrated with a few examples. These examples demonstrate the benefits of the new approach, as well as its calculation. The benefits are: 1) insight into the expected costs and expected effects gained as a result of carrying out further research to reduce or eliminate decision uncertainty; and 2) the likelihood that the outcome of additional research will result in a change in preferred policy.

CONCLUSIONS

Decision-making may be enhanced and clarified by adding results from AVOI analyses. Obtaining these results is straightforward and requires only a minimal computational effort. Therefore, use of the AVOI extension is recommended for all future VOI analyses.

Dimensions of design space: a decision-theoretic approach to optimal research design

Bayesian decision theory can be used not only to establish the optimal sample size and its allocation in a single clinical study but also to identify an optimal portfolio of research combining different types of study design. Within a single study, the highest societal payoff to proposed research is achieved when its sample sizes and allocation between available treatment options are chosen to maximize the expected net benefit of sampling (ENBS). Where a number of different types of study informing different parameters in the decision problem could be conducted, the simultaneous estimation of ENBS across all dimensions of the design space is required to identify the optimal sample sizes and allocations within such a research portfolio. This is illustrated through a simple example of a decision model of zanamivir for the treatment of influenza. The possible study designs include: 1) a single trial of all the parameters, 2) a clinical trial providing evidence only on clinical endpoints, 3) an epidemiological study of natural history of disease, and 4) a survey of quality of life. The possible combinations, samples sizes, and allocation between trial arms are evaluated over a range of cost-effectiveness thresholds. The computational challenges are addressed by implementing optimization algorithms to search the ENBS surface more efficiently over such large dimensions.

The cost-effectiveness of an RCT to establish whether 5 or 10 years of bisphosphonate treatment is the better duration for women with a prior fracture

PURPOSE

Five years of bisphosphonate treatment have proven efficacy in reducing fractures. Concerns exist that long-term bisphosphonate treatment may actually result in an increased number of fractures. This study evaluates, in the context of England and Wales, whether it is cost-effective to conduct a randomized controlled trial (RCT) and what sample size may be optimal to estimate the efficacy of bisphosphonates in fracture prevention beyond 5 years.

METHOD

An osteoporosis model was constructed to evaluate the cost-effectiveness of extending bisphosphonate treatment from 5 years to 10 years. Two scenarios were run. The 1st uses long-term efficacy data from published literature, and the 2nd uses distributions elicited from clinical experts.

RESULTS

of a proposed RCT were simulated. The expected value of sample information technique was applied to calculate the expected net benefit of sampling from conducting such an RCT at varying levels of participants per arm and to compare this with proposed trial costs. Results. Without further information, the better duration of bisphosphonate treatment was estimated to be 5 years using the published data but 10 years using the elicited expert opinions, although in both cases uncertainty was substantial. The net benefit of sampling was consistently high when between 2000 and 5000 participants per arm were recruited.

CONCLUSIONS

An RCT to evaluate the long-term efficacy of bisphosphonates in fracture prevention appears to be cost-effective for informing decision making in England and Wales.

Evidence synthesis as the key to more coherent and efficient research

Background

Systematic review and meta-analysis currently underpin much of evidence-based medicine. Such methodologies bring order to previous research, but future research planning remains relatively incoherent and inefficient.

Methods

To outline a framework for evaluation of health interventions, aimed at increasing coherence and efficiency through i) making better use of information contained within the existing evidence-base when designing future studies; and ii) maximising the information available and thus potentially reducing the need for future studies.

Results

The framework presented insists that an up-to-date meta-analysis of existing randomised controlled trials (RCTs) should always be considered before future trials are conducted. Such a meta-analysis should inform critical design issues such as sample size determination. The contexts in which the use of individual patient data meta-analysis and mixed treatment comparisons modelling may be beneficial before further RCTs are conducted are considered. Consideration should also be given to how any newly planned RCTs would contribute to the totality of evidence through its incorporation into an updated meta-analysis. We illustrate how new RCTs can have very low power to change inferences of an existing meta-analysis, particularly when between study heterogeneity is taken into consideration.

Conclusion

While the collation of existing evidence as the basis for clinical practice is now routine, a more coherent and efficient approach to planning future RCTs to strengthen the evidence base needs to be developed. The framework presented is a proposal for how this situation can be improved.

Surgical treatments for men with benign prostatic enlargement: cost effectiveness study

OBJECTIVE

To determine which surgical treatment for lower urinary tract symptoms suggestive of benign prostate enlargement is cost effective.

DESIGN

Care pathways describing credible treatment strategies were decided by consensus. Cost-utility analysis used Markov modelling and Monte Carlo simulation.

DATA SOURCES

Clinical effectiveness data came from a systematic review and an individual level dataset. Utility values came from previous economic evaluations. Costs were calculated from National Health Service (NHS) and commercial sources.

METHODS

The Markov model included parameters with associated measures of uncertainty describing health states between which individuals might move at three monthly intervals over 10 years. Successive annual cohorts of 25,000 men were entered into the model and the probability that treatment strategies were cost effective was assessed with Monte Carlo simulation with 10,000 iterations.

RESULTS

A treatment strategy of initial diathermy vaporisation of the prostate followed by endoscopic holmium laser enucleation of the prostate in case of failure to benefit or subsequent relapse had an 85% probability of being cost effective at a willingness to pay value of pound20,000 (euro21,595, $28,686)/quality adjusted life year (QALY) gained. Other strategies with diathermy vaporisation as the initial treatment were generally cheaper and more effective than the current standard of transurethral resection repeated once if necessary. The use of potassium titanyl phosphate laser vaporisation incurred higher costs and was less effective than transurethral resection, and strategies involving initial minimally invasive treatment with microwave thermotherapy were not cost effective. Findings were unchanged by wide ranging sensitivity analyses.

CONCLUSION

The outcome of this economic model should be interpreted cautiously because of the limitations of the data used. The finding that initial vaporisation followed by holmium laser enucleation for failure or relapse might be advantageous both to men with lower urinary tract symptoms and to healthcare providers requires confirmation in a good quality prospective clinical trial before any change in current practice. Potassium titanyl phosphate laser vaporisation was unlikely to be cost effective in our model, which argues against its unrestricted use until further evidence of effectiveness and cost reduction is obtained.

Value of information and value of implementation: application of an analytic framework to inform resource allocation decisions in metastatic hormone-refractory prostate cancer

OBJECTIVE

In a budget-constrained health-care system, decisions about investing in strategies to promote implementation have to be made alongside decisions about health-care provision and research funding. Using a Bayesian decision-theoretic approach, an analytic framework has been developed to inform these separate but related decisions, establishing the expected value of both perfect information (EVPI) and perfect implementation (EVPIM). We applied this framework to inform decision-making about resource allocation to metastatic hormone-refractory prostate cancer (mHRPC) in the UK.

METHODS

Based on available evidence on the cost-effectiveness of all plausible treatments for mHRPC, we determined which treatment option(s) were cost-effective and explored the uncertainty surrounding this decision. Given the decision uncertainty and the variation in care provided by health-care professionals, we then determined the EVPI and EVPIM. Finally, we performed sensitivity analyses to explore the influence of alternative assumptions regarding various decision parameters on the efficiency of resource allocation.

RESULTS

Depending on the cost-effectiveness threshold (lambda), we identified mitoxantrone plus prednisone/prednisolone and docetaxel plus prednisone/prednisolone (3 weekly) as the optimal treatments for mHRPC. Given current clinical practice, there appears to be considerable scope for improving the efficiency of health-care provision: the EVPI (estimated to be over pound13 million) indicates that acquiring further information could be cost-effective; and the EVPIM (estimated to be over pound4 million) suggests that investing in strategies to implement the treatments regimens being identified as optimal is potentially worthwhile. Through sensitivity analyses, we found that the EVPI and EVPIM are mainly driven by lambda, the number of treatment options being considered, the current level of implementation, and the size of the eligible patient population.

CONCLUSION

The application demonstrates that the framework provides a simple and useful analytic tool for decision-makers to address resource allocation problems between health-care provision, further research, and implementation efforts.

Multiparameter evidence synthesis in epidemiology and medical decision-making

Meta-analysis has been well-established for many years, but has been largely confined to pooling evidence on pair-wise contrasts. Broader forms of synthesis have also been described, apparently re-invented in disparate fields, each time taking different computational approaches. The potential value of Bayesian estimation of a joint posterior parameter distribution and simultaneously sampling from it for decision analysis has also been appreciated. However, applications have been relatively few in number, sometimes stylized, and presented mainly to a statistical methods audience. As a result, the potential for multiparameter evidence synthesis in both epidemiology and health technology assessment has remained largely unrecognized. The advent of flexible software for Bayesian Markov chain Monte Carlo in the shape of WinBUGS has the made these earlier strands of work more widely available. Researchers can now carry out synthesis at a realistic level of complexity. The Bristol programme has not only contributed to a growing body of literature on how to synthesize different evidence structures, but also on how to check the consistency of multiple information sources and how to use the resulting models to prioritize future research.

TPMT testing in azathioprine: a ‘cost-effective use of healthcare resources’?

This study aimed to critically appraise the current level of economic evidence available for thiopurine S-methyltransferase (TPMT) testing of thiopurine drugs, such as azathioprine. Six economic evaluations of testing were identified, which all recommended that TPMT testing is a cost-effective use of healthcare resources. Critical appraisal, using published guidelines, showed potential limitations in model structures, approaches to data analysis and input parameters, which were mainly based on expert opinion. Where data did exist these were from retrospective studies. To conduct economic evaluations with more robust findings, decision analysts need good quality data for the following key parameters: current prevalence of profound neutropenia among patients prescribed thiopurine drugs; mean length of related hospitalization and clinical outcome; impact of introducing the test on clinical pathways in terms of resource use; and clinical effectiveness data in terms of number of cases of neutropenia averted and subsequent impact on mortality and health-related quality of life. An iterative approach may be used to stimulate the production of a sufficient evidence base for innovative technologies, such as pharmacogenetic testing. Such an iterative approach involves starting with simple models using available existing clinical and resource use data, as in the case of TPMT testing. The use of formal value of information methods may guide the decision whether prospective studies are required to address uncertainties in the key parameters driving the model results. The results from well-designed prospective studies can then be used to populate more complex economic models.

A decision-theoretic framework for the application of cost-effectiveness analysis in regulatory processes

Cost-effectiveness analysis (CEA) represents the most important tool in the health economics literature to quantify and qualify the reasoning behind the optimal decision process in terms of the allocation of resources to a given health intervention. However, the practical application of CEA in the regulatory process is often limited by some critical barriers, and decisions in clinical practice are frequently influenced by factors that do not contribute to efficient resource allocation, leading to inappropriate drug prescription and utilization. Moreover, most of the time there is uncertainty about the real cost-effectiveness profile of an innovative intervention, with the consequence that it is usually impossible to obtain an immediate and perfect substitution of a product with another having a better cost-effectiveness ratio. The objective of this article is to propose a rational approach to CEA within regulatory processes, basing our analysis in a Bayesian decision-theoretic framework and proposing an extension of the application of well known tools (such as the expected value of information) to such cases. The regulator can use these tools to identify the economic value of reducing the uncertainty surrounding the cost-effectiveness profile of the several alternatives. This value can be compared with the one that is generated by the actual market share of the different treatment options: one that is the most cost effective and others in the same therapeutic category that, despite producing clinical benefits, are less cost effective.

Expected value of perfect information: an empirical example of reducing decision uncertainty by conducting additional research

OBJECTIVE

Value of information (VOI) analysis informs decision-makers about the expected value of conducting more research to support a decision. This expected value of (partial) perfect information (EV(P)PI) can be estimated by simultaneously eliminating uncertainty on all (or some) parameters involved in model-based decision-making. This study aimed to calculate the EVPPI, before and after collecting additional information on the parameter of a probabilistic Markov model with the highest EVPPI.

METHODS

The model assessed the 5-year costs per quality-adjusted life year (QALY) of three bronchodilators in chronic obstructive pulmonary disease (COPD). It had identified tiotropium as the bronchodilator with the highest expected net benefit. Total EVPI was estimated plus the EVPPIs for four groups of parameters: 1) transition probabilities between COPD severity stages; 2) exacerbation probabilities; 3) utility weights; and 4) costs. Partial EVPI analyses were performed using one-level and two-level sampling algorithms.

RESULTS

Before additional research, the total EVPI was Euro 1985 per patient at a threshold value of Euro 20,000 per QALY. EVPPIs were Euro 1081 for utilities, Euro 724 for transition probabilities, and relatively small for exacerbation probabilities and costs. A large study was performed to obtain more precise EQ-5D utilities by COPD severity stages. After using posterior utilities, the EVPPI for utilities decreased to almost zero. The total EVPI for the updated model was reduced to Euro 1037. With an EVPPI of Euro 856, transition probabilities were now the single most important parameter contributing to the EVPI.

CONCLUSIONS

This VOI analysis clearly identified parameters for which additional research is most worthwhile. After conducting additional research on the most important parameter, i.e., the utilities, total EVPI was substantially reduced.

Models based on value and probability in health improve shared decision making

RATIONALE, AIMS AND OBJECTIVES

Diagnostic reasoning and treatment decisions are a key competence of doctors. A model based on values and probability provides a conceptual framework for clinical judgments and decisions, and also facilitates the integration of clinical and biomedical knowledge into a diagnostic decision.

METHOD

Both value and probability are usually estimated values in clinical decision making. Therefore, model assumptions and parameter estimates should be continually assessed against data, and models should be revised accordingly. Introducing parameter estimates for both value and probability, which usually pertain in clinical work, gives the model labelled subjective expected utility. Estimated values and probabilities are involved sequentially for every step in the decision-making process.

RESULTS

Introducing decision-analytic modelling gives a more complete picture of variables that influence the decisions carried out by the doctor and the patient.

CONCLUSION

A model revised for perceived values and probabilities by both the doctor and the patient could be used as a tool for engaging in a mutual and shared decision-making process in clinical work.

Estimating the expected value of partial perfect information: a review of methods

BACKGROUND

Value of information analysis provides a framework for the analysis of uncertainty within economic analysis by focussing on the value of obtaining further information to reduce uncertainty. The mathematical definition of the expected value of perfect information (EVPI) is fixed, though there are different methods in the literature for its estimation. In this paper these methods are explored and compared.

METHODS

Analysis was conducted using a disease model for Parkinson's disease. Five methods for estimating partial EVPIs (EVPPIs) were used: a single Monte Carlo simulation (MCS) method, the unit normal loss integral (UNLI) method, a two-stage method using MCS, a two-stage method using MCS and quadrature and a difference method requiring two MCS. EVPPI was estimated for each individual parameter in the model as well as for three groups of parameters (transition probabilities, costs and utilities).

RESULTS

Using 5,000 replications, four methods returned similar results for EVPPIs. With 5 million replications, results were near identical. However, the difference method repeatedly gave estimates substantially different to the other methods.

CONCLUSIONS

The difference method is not rooted in the mathematical definition of EVPI and is clearly an inappropriate method for estimating EVPPI. The single MCS and UNLI methods were the least complex methods to use, but are restricted in their appropriateness. The two-stage MCS and quadrature-based methods are complex and time consuming. Thus, where appropriate, EVPPI should be estimated using either the single MCS or UNLI method. However, where neither of these methods is appropriate, either of the two-stage MCS and quadrature methods should be used.

Integration of Meta-analysis and Economic Decision Modeling for Evaluating Diagnostic Tests

Meta-analysis of diagnostic test accuracy data is more difficult than of effectiveness data because of 1) statistical challenges of dealing with multiple measures of accuracy (e.g., sensitivity and specificity) simultaneously and 2) incorporating threshold effects. A number of meta-analysis models are in use, ranging from naïve synthesis of independent sensitivity and specificity to optimization of a hierarchical summary receiver operating characteristic (SROC) curve. Little work has been done on how such analyses should inform decision models. This article aims to present a unified framework for the synthesis of primary data and economic evaluation of alternative diagnostic testing strategies using Bayesian Markov Chain Monte Carlo simulation methods. The authors extend this previous work by using systematic review to derive model parameters, fully allowing for uncertainty in their estimation, and formally incorporating variability between study results into the decision analysis. Using a simple decision model comparing alternative testing strategies for suspected deep vein thrombosis as an example, the authors consider how to use outputs of different alternative meta-analysis models in decision models. They also explore the limitations of diagnostic test studies, particularly when there is no obvious threshold value. To correct some of the limitations of diagnostic test studies, they propose that tests with implicit and explicit thresholds should be studied using distinctly different frameworks. Specifically, when a threshold exists, quantitative threshold information should be included in meta-analysis models to aid interpretation of SROCs. Setting policy to relate to a specific point may be much more difficult for studies with implicit thresholds.

Time and expected value of sample information wait for no patient

OBJECTIVE

The expected value of sample information (EVSI) from prospective trials has previously been modeled as the product of EVSI per patient, and the number of patients across the relevant time horizon less those "used up" in trials. However, this implicitly assumes the eligible patient population to which information from a trial can be applied across a time horizon are independent of time for trial accrual, follow-up and analysis.

METHODS

This article demonstrates that in calculating the EVSI of a trial, the number of patients who benefit from trial information should be reduced by those treated outside as well as within the trial over the time until trial evidence is updated, including time for accrual, follow-up and analysis.

RESULTS

Accounting for time is shown to reduce the eligible patient population: 1) independent of the size of trial in allowing for time of follow-up and analysis, and 2) dependent on the size of trial for time of accrual, where the patient accrual rate is less than incidence. Consequently, the EVSI and expected net gain (ENG) at any given trial size are shown to be lower when accounting for time, with lower ENG reinforced in the case of trials undertaken while delaying decisions by additional opportunity costs of time.

CONCLUSIONS

Appropriately accounting for time reduces the EVSI of trial design and increase opportunity costs of trials undertaken with delay, leading to lower likelihood of trialing being optimal and smaller trial designs where optimal.

Assessing the impact of censoring of costs and effects on health-care decision-making: an example using the Atrial Fibrillation Follow-up Investigation of Rhythm Management (AFFIRM) study

OBJECTIVES

Losses to follow-up and administrative censoring can cloud the interpretation of trial-based economic evaluations. A number of investigators have examined the impact of different levels of adjustment for censoring, including nonadjustment, adjustment of effects only, and adjustment for both costs and effects. Nevertheless, there is a lack of research on the impact of censoring on decision-making. The objective of this study was to estimate the impact of adjustment for censoring on the interpretation of cost-effectiveness results and expected value of perfect information (EVPI), using a trial-based analysis that compared rate- and rhythm-control treatments for persons with atrial fibrillation.

METHODS

Three different levels of adjustment for censoring were examined: no censoring of cost and effects, censoring of effects only, and censoring of both costs and effects. In each case, bootstrapping was used to estimate the uncertainty incosts and effects, and the EVPI was calculated to determine the potential worth of further research.

RESULTS

Censoring did not impact the adoption decision. Nevertheless, this was not the case for the decision uncertainty or the EVPI. For a threshold of $50,000 per life-year, the EVPI varied between $626,000 (partial censoring) to $117 million (full censoring) for the eligible US population.

CONCLUSIONS

The level of adjustment for censoring in trial-based cost-effectiveness analyses can impact on the decisions to fund a new technology and to devote resources for further research. Only when censoring is taken into account for both costs and effects are these decisions appropriately addressed.

The half-life of truth: what are appropriate time horizons for research decisions?

PURPOSE

To evaluate alternative approaches taken to estimate the population that could benefit from research and to demonstrate that explicitly modeling future change leads to more appropriate estimates of the expected value of information (EVI).

METHODS

Existing approaches to estimating the population typically focus on the time horizon for decisions, employing seemingly arbitrary estimates of the appropriate horizon. These approaches implicitly use the time horizon as a proxy for future changes in technologies, prices, and information. Different approaches to quantifying the time horizon are explored, in the context of a stylized model, to demonstrate the impact of uncertainty in this estimate on EVI. An alternative approach is developed that explicitly models future changes in technologies, prices, and information and that demonstrates the impact on EVI estimates.

RESULTS

Explicitly modeling future changes means that the EVI for the decision problem may increase or decrease over time, but the EVI for the group of parameters that can be evaluated by current research tends to decline. The finite and infinite time horizons for the decision problem represent special cases (e.g., price shock or no changes, respectively). This type of analysis can be used to inform policy decisions relating to the timing of research.

CONCLUSIONS

The value of information depends on future changes in technologies, prices, and evidence. Finite time horizons for decision problems can be seen as a proxy for the complex and uncertain process of future change. A more explicit approach to modeling these changes could provide a more appropriate basis for calculating EVI, but this raises a number of significant methodological and technical challenges.