Value of information analysis in healthcare: a review of principles and applications

Community based complex interventions to sustain independence in older people: systematic review and network meta-analysis

OBJECTIVE

To synthesise evidence of the effectiveness of community based complex interventions, grouped according to their intervention components, to sustain independence for older people.

DESIGN

Systematic review and network meta-analysis.

DATA SOURCES

Medline, Embase, CINAHL, PsycINFO, CENTRAL, clinicaltrials.gov, and International Clinical Trials Registry Platform from inception to 9 August 2021 and reference lists of included studies.

ELIGIBILITY CRITERIA

Randomised controlled trials or cluster randomised controlled trials with ≥24 weeks' follow-up studying community based complex interventions for sustaining independence in older people (mean age ≥65 years) living at home, with usual care, placebo, or another complex intervention as comparators.

MAIN OUTCOMES

Living at home, activities of daily living (personal/instrumental), care home placement, and service/economic outcomes at 12 months.

DATA SYNTHESIS

Interventions were grouped according to a specifically developed typology. Random effects network meta-analysis estimated comparative effects; Cochrane's revised tool (RoB 2) structured risk of bias assessment. Grading of recommendations assessment, development and evaluation (GRADE) network meta-analysis structured certainty assessment.

RESULTS

The review included 129 studies (74 946 participants). Nineteen intervention components, including "multifactorial action from individualised care planning" (a process of multidomain assessment and management leading to tailored actions), were identified in 63 combinations. For living at home, compared with no intervention/placebo, evidence favoured multifactorial action from individualised care planning including medication review and regular follow-ups (routine review) (odds ratio 1.22, 95% confidence interval 0.93 to 1.59; moderate certainty); multifactorial action from individualised care planning including medication review without regular follow-ups (2.55, 0.61 to 10.60; low certainty); combined cognitive training, medication review, nutritional support, and exercise (1.93, 0.79 to 4.77; low certainty); and combined activities of daily living training, nutritional support, and exercise (1.79, 0.67 to 4.76; low certainty). Risk screening or the addition of education and self-management strategies to multifactorial action from individualised care planning and routine review with medication review may reduce odds of living at home. For instrumental activities of daily living, evidence favoured multifactorial action from individualised care planning and routine review with medication review (standardised mean difference 0.11, 95% confidence interval 0.00 to 0.21; moderate certainty). Two interventions may reduce instrumental activities of daily living: combined activities of daily living training, aids, and exercise; and combined activities of daily living training, aids, education, exercise, and multifactorial action from individualised care planning and routine review with medication review and self-management strategies. For personal activities of daily living, evidence favoured combined exercise, multifactorial action from individualised care planning, and routine review with medication review and self-management strategies (0.16, -0.51 to 0.82; low certainty). For homecare recipients, evidence favoured addition of multifactorial action from individualised care planning and routine review with medication review (0.60, 0.32 to 0.88; low certainty). High risk of bias and imprecise estimates meant that most evidence was low or very low certainty. Few studies contributed to each comparison, impeding evaluation of inconsistency and frailty.

CONCLUSIONS

The intervention most likely to sustain independence is individualised care planning including medicines optimisation and regular follow-up reviews resulting in multifactorial action. Homecare recipients may particularly benefit from this intervention. Unexpectedly, some combinations may reduce independence. Further research is needed to investigate which combinations of interventions work best for different participants and contexts.

REGISTRATION

PROSPERO CRD42019162195.

Economic evaluations in medical technological innovations a mapping review of methodologies

RATIONALE

Economic evaluations play an important role in the development and implementation of healthcare innovations. For pharmaceutical products, the methodologies used are laid down in guidelines, whereas for medical technologies the guidelines are not as strenuous. The aim of this review was therefore to analyze what types of methodologies are used in economic evaluations of medical technologies.

METHODS

We performed a mapping review to identify economic evaluations for medical technologies. We decided to limit our search to one year (2022) and included cost utility and cost effectiveness analyses in which health technologies were evaluated. For each included study we identified the main methodological characteristics.

RESULTS

A total of 364 papers were included in the analysis, 268 (74%) contained cost-utility analyses and 91 (25%) cost-effectiveness analyses. A model was used in 236 (64%) analyses, 117 analyses were trial based evaluations. Probabilistic sensitivity analyses and/or bootstrapping was performed in 266 (73%) analyses. Deterministic sensitivity analyses were used in 306 (84%). Time horizon and perspective were underreported in 15-25% of the included studies.

CONCLUSIONS

This review shows the wide range of methodologies used in economic evaluations as well as the extent and rigor in which these methodologies are used. Many of the included papers did no use or did not sufficiently report the use of appropriate standard methods. This may lead to research waste, a delay in successful implementation of valuable innovations and in the end may delay improvement patient outcomes.

Value of Information Analysis of a Web-Based Self-Management Intervention for Chronic Obstructive Pulmonary Disease

Objective: Technology-based programs can be cost-effective in the management of chronic obstructive pulmonary disease (COPD). However, cost-effectiveness estimates always contain some uncertainty, and decisions based upon them carry some risk. We conducted a value of information (VOI) analysis to estimate the value of additional research of a web-based self-management intervention for COPD to reduce the costs associated with uncertainty. Methods: We used a 10,000-iteration cost-effectiveness model from the health care payer perspective to calculate the expected value of perfect information (EVPI) at the patient- and population-level. An opportunity loss was incurred when the web-based intervention did not produce a greater net monetary benefit than usual care in an iteration. We calculated the probability of opportunity loss and magnitude of opportunity costs as a function of baseline health utility. We aggregated opportunity costs over the projected incident population of inpatient COPD patients over 10 years and estimated it as a function of the willingness-to-pay (WTP) threshold. Costs are in 2022 U.S. Dollars. Results: Opportunity losses were found in 22.7% of the iterations. The EVPIpatient was $78 per patient (95% confidence interval: $75-$82). The probability that the intervention was the optimal strategy varied across baseline health utilities. The EVPIpopulation was $506,666,882 over 10 years for a WTP of $50,000. Conclusions: Research estimated to cost up to $500 million would be warranted to reduce uncertainty. Future research could focus on identifying the impact of baseline health utilities to maximize the cost savings of the intervention. Other considerations for future research priorities include implementation efforts for technology-based interventions.

Methodological challenges and potential solutions for economic evaluations of palliative and end-of-life care: A systematic review

BACKGROUND

Given the increasing demand for palliative and end-of-life care, along with the introduction of costly new treatments, there is a pressing need for robust evidence on value. However, comprehensive guidance is missing on methods for conducting economic evaluations in this field.

AIM

To identify and summarise existing information on methodological challenges and potential solutions/recommendations for economic evaluations of palliative and end-of-life care.

DESIGN

We conducted a systematic review of publications on methodological considerations for economic evaluations of adult palliative and end-of-life care as per our PROSPERO protocol CRD42020148160. Following initial searches, we conducted a two-stage screening process and quality appraisal. Information was thematically synthesised, coded, categorised into common themes and aligned with the items specified in the Consolidated Health Economic Evaluation Reporting Standards statement.

DATA SOURCES

The databases Medline, Embase, HTADatabase, NHSEED and grey literature were searched between 1 January 1999 and 5 June 2023.

RESULTS

Out of the initial 6502 studies, 81 were deemed eligible. Identified challenges could be grouped into nine themes: ambiguous and inaccurate patient identification, restricted generalisability due to poor geographic transferability of evidence, narrow costing perspective applied, difficulties defining comparators, consequences of applied time horizon, ambiguity in the selection of outcomes, challenged outcome measurement, non-standardised measurement and valuation of costs as well as challenges regarding a reliable preference-based outcome valuation.

CONCLUSION

Our review offers a comprehensive context-specific overview of methodological considerations for economic evaluations of palliative and end-of-life care. It also identifies the main knowledge gaps to help prioritise future methodological research specifically for this field.

IMproving facial PRosthesis construction with contactlESs Scanning and Digital workflow (IMPRESSeD): study protocol for a feasibility crossover randomised controlled trial of digital versus conventional manufacture of facial prostheses in patients with orbital or nasal facial defects

BACKGROUND

Facial prostheses can have a profound impact on patients' appearance, function and quality of life. There has been increasing interest in the digital manufacturing of facial prostheses which may offer many benefits to patients and healthcare services compared with conventional manufacturing processes. Most facial prosthesis research has adopted observational study designs with very few randomised controlled trials (RCTs) documented. There is a clear need for a well-designed RCT to compare the clinical and cost-effectiveness of digitally manufactured facial prostheses versus conventionally manufactured facial prostheses. This study protocol describes the planned conduct of a feasibility RCT which aims to address this knowledge gap and determine whether it is feasible to conduct a future definitive RCT.

METHODS

The IMPRESSeD study is a multi-centre, 2-arm, crossover, feasibility RCT with early health technology assessment and qualitative research. Up to 30 participants with acquired orbital or nasal defects will be recruited from the Maxillofacial Prosthetic Departments of participating NHS hospitals. All trial participants will receive 2 new facial prostheses manufactured using digital and conventional manufacturing methods. The order of receiving the facial prostheses will be allocated centrally using minimisation. The 2 prostheses will be made in tandem and marked with a colour label to mask the manufacturing method to the participants. Participants will be reviewed 4 weeks following the delivery of the first prosthesis and 4 weeks following the delivery of the second prosthesis. Primary feasibility outcomes include eligibility, recruitment, conversion, and attrition rates. Data will also be collected on patient preference, quality of life and resource use from the healthcare perspective. A qualitative sub-study will evaluate patients' perception, lived experience and preference of the different manufacturing methods.

DISCUSSION

There is uncertainty regarding the best method of manufacturing facial prostheses in terms of clinical effectiveness, cost-effectiveness and patient acceptability. There is a need for a well-designed RCT to compare digital and conventional manufacturing of facial prostheses to better inform clinical practice. The feasibility study will evaluate key parameters needed to design a definitive trial and will incorporate early health technology assessment and a qualitative sub-study to identify the potential benefits of further research.

TRIAL REGISTRATION

ISRCTN ISRCTN10516986). Prospectively registered on 08 June 2021,  https://www.isrctn.com/ISRCTN10516986 .

Cost-effectiveness of surgical treatment of thumb carpometacarpal joint arthritis: a value of information study

BACKGROUND

Thumb carpometacarpal (CMC) joint arthritis is one of the most prevalent arthritic conditions commonly treated with trapeziectomy alone or trapeziectomy with ligament reconstruction and tendon interposition (LRTI). We evaluate the cost-effectiveness and value of perfect and sample information of trapeziectomy alone, LRTI, and non-operative treatment.

METHODS

A societal perspective decision tree was modeled. To understand the value of future research in comparing quality-of-life after trapeziectomy, LRTI, and non-operative management we characterized uncertainty by fitting distributions to EQ-5D utility data published from the United Kingdom hand surgery registry. We used Monte Carlo simulation for the probabilistic sensitivity analysis and to evaluate the value of perfect and sample information.

RESULTS

Both trapeziectomy alone and LRTI were cost-effective compared to non-operative management ($2,540 and $3,511/QALY respectively). Trapeziectomy alone (base case total cost $8,251, QALY 14.08) was dominant compared to LRTI (base case total cost $8,798, QALY 13.34). However, probabilistic sensitivity analysis suggested there is a 12.5% chance LRTI may be preferred at a willingness-to-pay of $50,000/QALY. Sensitivity analysis revealed postoperative utilities are the most influential factors in determining cost-effectiveness. The value of perfect information was approximately $1,503/person. A study evaluating the quality-of-life of 1,000 patients in each arm undergoing trapeziectomy alone or LRTI could provide an expected $1,117 of information value. With approximately 40,000 CMC arthroplasties performed each year in the U.S., the annual value is close to $45 million.

CONCLUSIONS

Trapeziectomy without LRTI appears to be the most cost-effective procedure in treating late-stage CMC arthritis and should be considered as first-line surgical treatment. There is substantial societal value in conducting additional research to better understand the relative quality-of-life improvements gained from these two common hand surgeries.

Cost-Effectiveness and Value of Information Analysis of an Ambient Intelligent Geriatric Management (AmbIGeM) System Compared to Usual Care to Prevent Falls in Older People in Hospitals

BACKGROUND

The Ambient Intelligent Geriatric Management (AmbIGeM) system combines wearable sensors with artificial intelligence to trigger alerts to hospital staff before a fall. A clinical trial found no effect across a heterogenous population, but reported a reduction in the injurious falls rate in a post hoc analysis of patients on Geriatric Evaluation Management Unit (GEMU) wards. Cost-effectiveness and Value of Information (VoI) analyses of the AmbIGeM system in GEMU wards was undertaken.

METHODS

An Australian health-care system perspective and 5-year time horizon were used for the cost-effectiveness analysis. Implementation costs, inpatient costs and falls data were collected. Injurious falls were defined as causing bruising, laceration, fracture, loss of consciousness, or if the patient reported persistent pain. To compare costs and outcomes, generalised linear regression models were used to adjust for baseline differences between the intervention and usual care groups. Bootstrapping was used to represent uncertainty. For the VoI analysis, 10,000 different sample sizes with randomly sampled values ranging from 1 to 50,000 were tested to estimate the optimal sample size of a new trial that maximised the Expected Net Benefits of Sampling.

RESULTS

An adjusted 0.036 fewer injurious falls (adjusted rate ratio of 0.56) and AUD$4554 lower costs were seen in the intervention group. However, uncertainty that the intervention is cost effective for the prevention of an injurious fall was present at all monetary values of this effectiveness outcome. A new trial with a sample of 4376 patients was estimated to maximise the Expected Net Benefit of Sampling, generating a net benefit of AUD$186,632 at a benefit-to-cost ratio of 1.1.

CONCLUSIONS

The benefits to cost ratio suggests that a new trial of the AmbIGeM system in GEMU wards may not be high-value compared to other potential trials, and that the system should be implemented. However, a broader analysis of options for preventing falls in GEMU is required to fully inform decision making.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trial Registry (ACTRN 12617000981325).

Gene Therapy Evidence Generation and Economic Analysis: Pragmatic Considerations to Facilitate Fit-for-Purpose Health Technology Assessment

Gene therapies (GTs) are considered to be a paradigm-shifting class of treatments with the potential to treat previously incurable diseases or those with significant unmet treatment needs. However, considerable challenges remain in their health technology assessment (HTA), mainly stemming from the inability to perform robust clinical trials to convince decision-makers to pay the high prices for the potential long-term treatment benefits provided. This article aims to review the recommendations that have been published for evidence generation and economic analysis for GTs against the feasibility of their implementation within current HTA decision analysis frameworks. After reviewing the systematically identified literature, we found that questions remain on the appropriateness of GT evidence generation, considering that additional, broader values brought by GTs seem insufficiently incorporated within proposed analytic methods. In cases where innovative methods are proposed, HTA organizations remain highly conservative and resistant to change their reference case and decision analysis framework. Such resistances are largely attributed to the substantial evidence uncertainty, resource-consuming administration process, and the absence of consensus on the optimized methodology to balance all the advantages and potential pitfalls of GTs.

A Review of Web-Based Tools for Value-of-Information Analysis

Value-of-information analysis (VOI) is a decision-theoretic approach that is used to inform reimbursement decisions, optimise trial design and set research priorities. The application of VOI analysis for informing policy decisions in practice has been limited due, in part, to the perceived complexity associated with the calculation of VOI measures. Recent efforts have resulted in the development of efficient methods to estimate VOI measures and the development of user-friendly web-based tools to facilitate VOI calculations. We review the existing web-based tools including Sheffield Accelerated Value of Information (SAVI), the web interface to the BCEA (Bayesian Cost-Effectiveness Analysis) R package (BCEAweb), Rapid Assessment of Need for Evidence (RANE), and Value of Information for Cardiovascular Trials and Other Comparative Research (VICTOR). We describe what each tool is designed to do, the inputs they require, and the outputs they produce. Finally, we discuss how tools for VOI calculations might be improved in the future to facilitate the use of VOI analysis in practice.

Healthcare Funding Decisions and Real-World Benefits: Reducing Bias by Matching Untreated Patients

Governments and health insurers often make funding decisions based on health gains from randomised controlled trials. These decisions are inherently uncertain because health gains in trials may not translate to practice owing to differences in the population, treatment use and setting. Post-market analysis of real-world data can provide additional evidence but estimates from standard matching methods may be biased when unobserved characteristics explain whether a patient is treated and their outcomes. We propose a new untreated matching approach that can reduce this bias. Our approach utilises the outcomes of contemporaneous untreated patients to improve the matching of treated and historical control patients. We assess the performance of this new approach compared to standard matching using a simulation study and demonstrate the steps required using a funding decision for prostate cancer treatments in Australia. Our simulation study shows that our new matching approach eliminates nearly all bias when unobserved treatment selection is related to outcomes, and outperforms standard matching in most scenarios. In our empirical example, standard matching overestimated survival by 15% (95% confidence interval 2-34) compared to our untreated matching approach. The health gains estimated using our approach were slightly lower than expected based on the trial evidence, but we also found evidence that in practice prescribers ceased prior therapies earlier, treated a more vulnerable population and continued treatment for longer. Our untreated matching approach offers researchers a new tool for reducing uncertainty in healthcare funding decisions using real-world data.

Combining Model-Based Clinical Trial Simulation, Pharmacoeconomics, and Value of Information to Optimize Trial Design

The Bayesian decision‐analytic approach to trial design uses prior distributions for treatment effects, updated with likelihoods for proposed trial data. Prior distributions for treatment effects based on previous trial results risks sample selection bias and difficulties when a proposed trial differs in terms of patient characteristics, medication adherence, or treatment doses and regimens. The aim of this study was to demonstrate the utility of using pharmacometric‐based clinical trial simulation (CTS) to generate prior distributions for use in Bayesian decision‐theoretic trial design. The methods consisted of four principal stages: a CTS to predict the distribution of treatment response for a range of trial designs; Bayesian updating for a proposed sample size; a pharmacoeconomic model to represent the perspective of a reimbursement authority in which price is contingent on trial outcome; and a model of the pharmaceutical company return on investment linking drug prices to sales revenue. We used a case study of febuxostat versus allopurinol for the treatment of hyperuricemia in patients with gout. Trial design scenarios studied included alternative treatment doses, inclusion criteria, input uncertainty, and sample size. Optimal trial sample sizes varied depending on the uncertainty of model inputs, trial inclusion criteria, and treatment doses. This interdisciplinary framework for trial design and sample size calculation may have value in supporting decisions during later phases of drug development and in identifying costly sources of uncertainty, and thus inform future research and development strategies.

Economic Evaluations of Pharmacological Treatments in Heart Failure Patients: A Methodological Review with a Focus on Key Model Drivers

Various decision analytic models exist for evaluating the cost-effectiveness of pharmacological interventions for heart failure (HF). Despite this, studies that explore drivers influencing these modeling approaches remain scarce. Through a systematic review of the literature, the present study sought to identify model drivers that emerge from economic evaluations of HF pharmacological interventions. Among the 72 cost effectiveness papers evaluating HF drug interventions, the most frequently identified, top 5 ranked model drivers impacting the incremental cost-effectiveness ratio (ICER) were cost of treatment and utility, identified in 10% of studies, respectively. Other drivers that emerged as top 5 ranked drivers in > 5% of studies included treatment effect on mortality (or cardiovascular mortality), duration of treatment, and baseline cardiovascular mortality. Model drivers reported at the top of tornado diagrams were treatment effect on mortality or on cardiovascular mortality. Collectively, these observations highlight the key importance of treatment effect in driving cost-effectiveness models for HF.

Methodological aspects of economic evaluations conducted in the palliative or end of life care settings: a systematic review protocol

INTRODUCTION

In light of this growing palliative care and end of life care patient population, as well as new (expensive) drugs and treatments, quality research providing evidence for decision-making is required. However, common research guidance is lacking in this field, especially in respect to the methods applied in economic evaluations. Therefore, the aim of the planned systematic review is to identify and summarise relevant information on methodological challenges, potential solutions and recommendations for conducting economic evaluations of interventions in adult patients, irrespective of their underlying disease and gender in the palliative or end of life care settings, with no restrictions in regards to countries/geographical regions. The results of this systematic review may help to clarify the current methodological questions and form the basis of new, setting specific methods guidelines and support ongoing applied economic evaluations in the field.

METHODS AND ANALYSIS

A systematic review will be conducted using Medline, Embase, Health Technology Assessment Database and NHS Economic Evaluation Database to identify the studies published from 1999 onwards with relevant information on methodological challenges, potential solutions and recommendations for conducting economic evaluations in the palliative or end of life care settings. Articles in English, German, Spanish, French or Dutch language will be considered. Two independent reviewers will conduct the screening of articles; any discrepancies will be resolved by discussion and involvement of a third reviewer. Predesigned data extraction forms will be applied, consequently narratively synthesised and categorised. Studies' methodological quality will be critically appraised. Besides existing economic guidelines and checklists for specific information on the palliative and end of life care sector will be searched.

ETHICS AND DISSEMINATION

Ethical approval is not required, as this is a planned systematic review of published literature. An article will be disseminated in a related peer-reviewed journal, as well as presented at leading palliative care and health economic conferences.

PROSPERO REGISTRATION NUMBER

CRD42020148160.

A Framework to Prioritise Health Research Proposals for Funding: Integrating Value for Money

When making funding decisions, research organisations largely consider the merits (e.g. scientific rigour and feasibility) of submitted research proposals; yet, there is often little or no reference to their value for money. This may be attributed to the challenges of assessing and integrating value of research into existing research prioritisation processes. We propose a framework that considers both the merits of research and its value for money to guide health research funding decisions. A practical framework is developed based on current processes followed by funding organizations for assessing investigator-initiated research proposals, and analytical methods for evaluating the expected value of research. We apply the analytical methods to estimate the expected return on investment of two real-world grant applications. The framework comprises four sequential steps: (1) initial screening of applications for eligibility and completeness; (2) merit assessment of eligible proposals; (3) estimating the expected value of research for the shortlisted proposals that pass the first two steps and ranking of proposals based on return on investment; and (4) selecting research proposals for funding. We demonstrate how the expected value for money can be efficiently estimated using certain information provided in funding applications. The proposed framework integrates value-for-money assessment into the existing research prioritisation processes. Considering value for money to inform research funding decisions is vital to achieve efficient utilisation of research budgets and maximise returns on research investments.

Decision analytic modeling was useful to assess the impact of a prediction model on health outcomes before a randomized trial

OBJECTIVE

To demonstrate how decision analytic models (DAMs) can be used to quantify impact of using a (diagnostic or prognostic) prediction model in clinical practice and provide general guidance on how to perform such assessments.

STUDY DESIGN AND SETTING

A DAM was developed to assess the impact of using the HEART score for predicting major adverse cardiac events (MACE). Impact on patient health outcomes and health care costs was assessed in scenarios by varying compliance with and informed deviation (ID) (using additional clinical knowledge) from HEART score management recommendations. Probabilistic sensitivity analysis was used to assess estimated impact robustness.

RESULTS

Impact of using the HEART score on health outcomes and health care costs was influenced by an interplay of compliance with and ID from HEART score management recommendations. Compliance of 50% (with 0% ID) resulted in increased missed MACE and costs compared with usual care. Any compliance combined with at least 50% ID reduced both costs and missed MACE. Other scenarios yielded a reduction in missed MACE at higher costs.

CONCLUSION

Decision analytic modeling is a useful approach to assess impact of using a prediction model in practice on health outcomes and health care costs. This approach is recommended before conducting an impact trial to improve its design and conduct.

The Curve of Optimal Sample Size (COSS): A Graphical Representation of the Optimal Sample Size from a Value of Information Analysis

Value of information (VOI) analysis quantifies the opportunity cost associated with decision uncertainty, and thus informs the value of collecting further information to avoid this cost. VOI can inform study design, optimal sample size selection, and research prioritization. Recent methodological advances have reduced the computational burden of conducting VOI analysis and have made it easier to evaluate the expected value of sample information, the expected net benefit of sampling, and the optimal sample size of a study design ([Formula: see text]). The volume of VOI analyses being published is increasing, and there is now a need for VOI studies to conduct sensitivity analyses on VOI-specific parameters. In this practical application, we introduce the curve of optimal sample size (COSS), which is a graphical representation of [Formula: see text] over a range of willingness-to-pay thresholds and VOI parameters (example data and R code are provided). In a single figure, the COSS presents summary data for decision makers to determine the sample size that optimizes research funding given their operating characteristics. The COSS also presents variation in the optimal sample size given variability or uncertainty in VOI parameters. The COSS represents an efficient and additional approach for summarizing results from a VOI analysis.

Value of Information Choices that Influence Estimates: A Systematic Review of Prevailing Considerations

Background. Although value of information (VOI) analyses are increasingly advocated and used for research prioritization and reimbursement decisions, the interpretation and usefulness of VOI outcomes depend critically on the underlying choices and assumptions used in the analysis. In this article, we present a structured overview of all items reported in literature to potentially influence VOI outcomes. Use of this overview increases awareness and transparency of choices and assumptions underpinning VOI outcomes. Methods. A systematic literature review was performed to identify aspects of VOI analyses that were found to potentially influence VOI outcomes. Identified aspects were grouped to develop a structured overview. Explanations were defined for all items included in the overview. Results. We retrieved 687 unique papers, of which 71 original papers and 8 reviews were included. In the full text of these 79 papers, 16 aspects were found that may influence VOI outcomes. These aspects related to the underlying evidence (bias, synthesis, heterogeneity, correlation), uncertainty (structural, future pricing), model (relevance, approach, population), choices in VOI calculation (estimation technique, implementation level, population size, perspective), and aspects specifically for assessing the value of future study designs (reversal costs, efficient estimator). These aspects were aggregated into 7 items to provide a structured overview. Conclusion. The developed overview should increase awareness of key choices underlying VOI analysis and facilitate structured reporting of such choices and interpretation of the ensuing VOI outcomes by researchers and policy makers. Use of this overview should improve prioritization and reimbursement decisions.

Optimal Information Collection Policies in a Markov Decision Process Framework

BACKGROUND

The cost-effectiveness and value of additional information about a health technology or program may change over time because of trends affecting patient cohorts and/or the intervention. Delaying information collection even for parameters that do not change over time may be optimal.

METHODS

We present a stochastic dynamic programming approach to simultaneously identify the optimal intervention and information collection policies. We use our framework to evaluate birth cohort hepatitis C virus (HCV) screening. We focus on how the presence of a time-varying parameter (HCV prevalence) affects the optimal information collection policy for a parameter assumed constant across birth cohorts: liver fibrosis stage distribution for screen-detected diagnosis at age 50.

RESULTS

We prove that it may be optimal to delay information collection until a time when the information more immediately affects decision making. For the example of HCV screening, given initial beliefs, the optimal policy (at 2010) was to continue screening and collect information about the distribution of liver fibrosis at screen-detected diagnosis in 12 years, increasing the expected incremental net monetary benefit (INMB) by $169.5 million compared to current guidelines.

CONCLUSIONS

The option to delay information collection until the information is sufficiently likely to influence decisions can increase efficiency. A dynamic programming framework enables an assessment of the marginal value of information and determines the optimal policy, including when and how much information to collect.

The Australian Managed Entry Scheme: Are We Getting it Right?

In 2010, the Australian Government introduced the managed entry scheme (MES) to improve patient access to subsidised drugs on the Pharmaceutical Benefits Scheme and enhance the quality of evidence provided to decision makers. The aim of this paper was to critically review the Australian MES experience. We performed a comprehensive review of publicly available Pharmaceutical Benefits Advisory Committee online documents from January 2010 to July 2017. Relevant information on each MES agreement was systematically extracted, including its rationale, the conditions that guided its implementation and its policy outcomes. We identified 11 drugs where an MES was considered. Most of the identified drugs (75%) were antineoplastic agents and the main uncertainty was the overall survival benefit. More than half of the MES proposals were made by sponsors and most of the schemes were considered after previous rejected/deferred submissions for reimbursement. An MES was not established in 8 of 11 drugs (73%) despite the high evidence uncertainty. Nevertheless, six of these eight drugs were listed after the sponsors reduced their prices. Three MESs were established and implemented by Deeds of Agreement. The three cases were concluded and the required data were submitted within the agreed time frames. The need for feasibility and value of an MES should be carefully considered by stakeholders before embarking on such an agreement. It is essential to engage major stakeholders, including patient representatives, in this process. The conditions governing MESs should be clear, transparent and balanced to address the expectations of various stakeholders.

Combining value of information analysis and ethical argumentation in decisions on participation of vulnerable patients in clinical research

BACKGROUND

The participation of vulnerable patients in clinical research poses apparent ethical dilemmas. Depending on the nature of the vulnerability, their participation may challenge the ethical principles of autonomy, non-maleficence, or justice. On the other hand, non-participation may preclude the building of a knowledge base that is a prerequisite for defining the optimal clinical management of vulnerable patients. Such clinical uncertainty may also incur substantial economic costs.

MAIN TEXT

We present the participation of pre-menopausal women with atrial fibrillation in trials of novel oral anticoagulant drugs as a case study. Due to their non-participation in pivotal trials, it is uncertain whether for them, the risks that are associated with these drugs are outweighed by the advantages compared with conventional treatment. We addressed the question whether research of this new class of drugs in this subgroup would be appropriate from both, an ethical as well an economic perspective. We used the method of specifying norms as a wider framework to resolve the apparent ethical dilemma, while incorporating the question whether research of oral anticoagulants in premenopausal women with atrial fibrillation can be justified on economic grounds. For the latter, the results of a value-of-information analysis were used.

CONCLUSIONS

Further clinical research on NOACs in premenopausal women with atrial fibrillation can be justified on both, ethical and economic grounds. Addressing apparent ethical dilemmas by invoking a method such as specifying norms can improve the quality of public practical reasoning. As such, the method should also prove valuable to committees that have formally been granted the authority to review trial protocols and proposals for scientific research.

The Impact of Decision Makers' Constraints on the Outcome of Value of Information Analysis

BACKGROUND

When proven effective, decision making regarding reimbursement of new health technology typically involves ethical, social, legal, and health economic aspects and constraints. Nevertheless, when applying standard value of information (VOI) analysis, the value of collecting additional evidence is typically estimated assuming that only cost-effectiveness outcomes guide such decisions.

OBJECTIVES

To illustrate how decision makers' constraints can be incorporated into VOI analyses and how these may influence VOI outcomes.

METHODS

A simulation study was performed to estimate the cost-effectiveness of a new hypothetical technology compared with usual care. Constraints were defined for the new technology on 1) the maximum acceptable rate of complications and 2) the maximum acceptable additional budget. The expected value of perfect information (EVPI) for the new technology was estimated in various scenarios, both with and without incorporating these constraints.

RESULTS

For a willingness-to-pay threshold of €20,000 per quality-adjusted life-year, the probability that the new technology was cost-effective equaled 57%, with an EVPI of €1868 per patient. Applying the complication rate constraint reduced the EVPI to €1137. Similarly, the EVPI reduced to €770 when applying the budget constraint. Applying both constraints simultaneously further reduced the EVPI to €318.

CONCLUSIONS

When decision makers explicitly apply additional constraints, beyond a willingness-to-pay threshold, to reimbursement decisions, these constraints can and should be incorporated into VOI analysis as well, because they may influence VOI outcomes. This requires continuous interaction between VOI analysts and decision makers and is expected to improve both the relevance and the acceptance of VOI outcomes.

Economic evaluation of medical tests at the early phases of development: a systematic review of empirical studies

INTRODUCTION

There is little specific guidance on the implementation of cost-effectiveness modelling at the early stage of test development. The aim of this study was to review the literature in this field to examine the methodologies and tools that have been employed to date.

AREAS COVERED

A systematic review to identify relevant studies in established literature databases. Five studies were identified and included for narrative synthesis. These studies revealed that there is no consistent approach in this growing field. The perspective of patients and the potential for value of information (VOI) to provide information on the value of future research is often overlooked. Test accuracy is an essential consideration, with most studies having described and included all possible test results in their analysis, and conducted extensive sensitivity analyses on important parameters. Headroom analysis was considered in some instances but at the early development stage (not the concept stage).

EXPERT COMMENTARY

The techniques available to modellers that can demonstrate the value of conducting further research and product development (i.e. VOI analysis, headroom analysis) should be better utilized. There is the need for concerted efforts to develop rigorous methodology in this growing field to maximize the value and quality of such analysis.

Cost-Effectiveness and Value of Information Analysis of Brief Interventions to Promote Physical Activity in Primary Care

BACKGROUND

Brief interventions (BIs) delivered in primary care have shown potential to increase physical activity levels and may be cost-effective, at least in the short-term, when compared with usual care. Nevertheless, there is limited evidence on their longer term costs and health benefits.

OBJECTIVES

To estimate the cost-effectiveness of BIs to promote physical activity in primary care and to guide future research priorities using value of information analysis.

METHODS

A decision model was used to compare the cost-effectiveness of three classes of BIs that have been used, or could be used, to promote physical activity in primary care: 1) pedometer interventions, 2) advice/counseling on physical activity, and (3) action planning interventions. Published risk equations and data from the available literature or routine data sources were used to inform model parameters. Uncertainty was investigated with probabilistic sensitivity analysis, and value of information analysis was conducted to estimate the value of undertaking further research.

RESULTS

In the base-case, pedometer interventions yielded the highest expected net benefit at a willingness to pay of £20,000 per quality-adjusted life-year. There was, however, a great deal of decision uncertainty: the expected value of perfect information surrounding the decision problem for the National Health Service Health Check population was estimated at £1.85 billion.

CONCLUSIONS

Our analysis suggests that the use of pedometer BIs is the most cost-effective strategy to promote physical activity in primary care, and that there is potential value in further research into the cost-effectiveness of brief (i.e., <30 minutes) and very brief (i.e., <5 minutes) pedometer interventions in this setting.

Value of Information Analysis Informing Adoption and Research Decisions in a Portfolio of Health Care Interventions

Background: Value of information (VOI) analysis quantifies the value of additional research in reducing decision uncertainty. It addresses adoption and research decisions simultaneously by comparing the expected benefits and costs of research studies. Nevertheless, the application of this approach in practice remains limited. Objectives: To apply VOI analysis in health care interventions to guide adoption decisions, optimize trial design, and prioritize research. Methods: The analysis was from the perspective of Queensland Health, Australia. It included four interventions: clinically indicated catheter replacement, tissue adhesive for securing catheters, negative pressure wound therapy (NPWT) in caesarean sections, and nutritional support for preventing pressure ulcers. For each intervention, cost-effectiveness analysis was performed, decision uncertainty characterized, and VOI calculated using Monte Carlo simulations. The benefits and costs of additional research were considered together with the costs and consequences of acting now versus waiting for more information. All values are reported in 2014 Australian dollars (AU$). Results: All interventions were cost-effective, but with various levels of decision uncertainty. The current evidence is sufficient to support the adoption of clinically indicated catheter replacement. For the tissue adhesive, an additional study before adoption is worthwhile with a four-arm trial of 220 patients per arm. Additional research on NPWT before adoption is worthwhile with a two-arm trial of 200 patients per arm. Nutritional support should be adopted with a two-arm trial of 1200 patients per arm. Based on the expected net monetary benefits, the studies were ranked as follows: 1) NPWT (AU$1.2 million), 2) tissue adhesive (AU$0.3 milliion), and 3) nutritional support (AU$0.1 million). Conclusions: VOI analysis is a useful and practical approach to inform adoption and research decisions. Efforts should be focused on facilitating its integration into decision making frameworks.

Efficient Value of Information Calculation Using a Nonparametric Regression Approach: An Applied Perspective

BACKGROUND

Value-of-information (VOI) analysis provides an analytical framework to assess whether obtaining additional evidence is worthwhile to reduce decision uncertainty. The reporting of VOI measures, particularly the expected value of perfect parameter information (EVPPI) and the expected value of sample information (EVSI), is limited because of the computational burden associated with typical two-level Monte-Carlo-based solution. Recently, a nonparametric regression approach was proposed that allows the estimation of multiparameter EVPPI and EVSI directly from a probabilistic sensitivity analysis sample.

OBJECTIVES

To demonstrate the value of the nonparametric regression approach in calculating VOI measures in real-world cases and to compare its performance with the standard approach of the Monte-Carlo simulation.

METHODS

We used the regression approach to calculate EVPPI and EVSI in two models, and compared the results with the estimates obtained via the standard Monte-Carlo simulation.

RESULTS

The VOI values from the two approaches were very close; computation using the regression method, however, was faster.

CONCLUSION

The nonparametric regression approach provides an efficient and easy-to-implement alternative for EVPPI and EVSI calculation in economic models.

Estimating the expected value of partial perfect information in health economic evaluations using integrated nested Laplace approximation

The Expected Value of Perfect Partial Information (EVPPI) is a decision‐theoretic measure of the ‘cost’ of parametric uncertainty in decision making used principally in health economic decision making. Despite this decision‐theoretic grounding, the uptake of EVPPI calculations in practice has been slow. This is in part due to the prohibitive computational time required to estimate the EVPPI via Monte Carlo simulations. However, recent developments have demonstrated that the EVPPI can be estimated by non‐parametric regression methods, which have significantly decreased the computation time required to approximate the EVPPI. Under certain circumstances, high‐dimensional Gaussian Process (GP) regression is suggested, but this can still be prohibitively expensive. Applying fast computation methods developed in spatial statistics using Integrated Nested Laplace Approximations (INLA) and projecting from a high‐dimensional into a low‐dimensional input space allows us to decrease the computation time for fitting these high‐dimensional GP, often substantially. We demonstrate that the EVPPI calculated using our method for GP regression is in line with the standard GP regression method and that despite the apparent methodological complexity of this new method, R functions are available in the package BCEA to implement it simply and efficiently. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

A review and classification of approaches for dealing with uncertainty in multi-criteria decision analysis for healthcare decisions

Multi-criteria decision analysis (MCDA) is increasingly used to support decisions in healthcare involving multiple and conflicting criteria. Although uncertainty is usually carefully addressed in health economic evaluations, whether and how the different sources of uncertainty are dealt with and with what methods in MCDA is less known. The objective of this study is to review how uncertainty can be explicitly taken into account in MCDA and to discuss which approach may be appropriate for healthcare decision makers. A literature review was conducted in the Scopus and PubMed databases. Two reviewers independently categorized studies according to research areas, the type of MCDA used, and the approach used to quantify uncertainty. Selected full text articles were read for methodological details. The search strategy identified 569 studies. The five approaches most identified were fuzzy set theory (45% of studies), probabilistic sensitivity analysis (15%), deterministic sensitivity analysis (31%), Bayesian framework (6%), and grey theory (3%). A large number of papers considered the analytic hierarchy process in combination with fuzzy set theory (31%). Only 3% of studies were published in healthcare-related journals. In conclusion, our review identified five different approaches to take uncertainty into account in MCDA. The deterministic approach is most likely sufficient for most healthcare policy decisions because of its low complexity and straightforward implementation. However, more complex approaches may be needed when multiple sources of uncertainty must be considered simultaneously.

Cost-effectiveness and value of information analysis of nutritional support for preventing pressure ulcers in high-risk patients: implement now, research later

BACKGROUND

Pressure ulcers are a major cause of mortality, morbidity, and increased healthcare cost. Nutritional support may reduce the incidence of pressure ulcers in hospitalised patients who are at risk of pressure ulcer and malnutrition.

OBJECTIVES

To evaluate the cost-effectiveness of nutritional support in preventing pressure ulcers in high-risk hospitalised patients, and to assess the value of further research to inform the decision to implement this intervention using value of information analysis (VOI).

METHODS

The analysis was from the perspective of Queensland Health, Australia using a decision model with evidence derived from a systematic review and meta-analysis. Resources were valued using 2014 prices and the time horizon of the analysis was one year. Monte Carlo simulation was used to estimate net monetary benefits (NB) and to calculate VOI measures.

RESULTS

Compared with standard hospital diet, nutritional support was cost saving at AU$425 per patient, and more effective with an average 0.005 quality-adjusted life years (QALY) gained. At a willingness-to-pay of AU$50,000 per QALY, the incremental NB was AU$675 per patient, with a probability of 87 % that nutritional support is cost-effective. The expected value of perfect information was AU$5 million and the expected value of perfect parameter information was highest for the relative risk of developing a pressure ulcer at AU$2.5 million. For a future trial investigating the relative effectiveness of the interventions, the expected net benefit of research would be maximised at AU$100,000 with 1,200 patients in each arm if nutritional support was perfectly implemented. The opportunity cost of withholding the decision to implement the intervention until the results of the future study are available would be AU$14 million.

CONCLUSIONS

Nutritional support is cost-effective in preventing pressure ulcers in high-risk hospitalised patients compared with standard diet. Future research to reduce decision uncertainty is worthwhile; however, given the opportunity losses associated with delaying the implementation, "implement and research" is the approach recommended for this intervention.

HTA agencies facing model biases: the case of type 2 diabetes

When evaluating new drugs or treatments eligible for reimbursement, health technology assessment (HTA) agencies are repeatedly faced with cost-effectiveness analyses that evidence lack of adequate data and modeling biases. The case of type 2 diabetes illustrates this difficulty. In spite of its high disease burden, type 2 diabetes is poorly documented through existing cost-effectiveness analyses. We support this statement by an exhaustive literature review that enables us to precisely pinpoint the limitations of models used for the assessment of newly marketed (and expensive) drugs. We find that models are mostly restricted to surrogate endpoints and based on non-inferiority clinical trial data; they also show biases in the choice of comparators and inclusion criteria. Such limitations undermine the scope and applicability of HTA practice guidelines based on cost-effectiveness evidence. Nevertheless, cost-effectiveness models remain an opportunity to better inform decision makers and to reduce the uncertainty surrounding their decisions. HTA agencies are best placed to provide incentives for companies to improve the quality of the cost-effectiveness studies submitted for pricing and reimbursement decisions. One such incentive is to include stages of discussion between the company and the health authority during the evaluation process.