Cost effectiveness scenario analysis for a proposed trial of hormone replacement therapy

Organising Research and Development for evidence-informed health care: some universal characteristics and a case study from the UK

Research and Development (R&D) in health and health care has several intriguing characteristics which, separately and in combination, have significant implications for the ways in which it is organised, funded and managed. We review the characteristics, some of which apply under most circumstances and others of which may be context-specific, explore their implications for the organisation and management of health-related R&D, and illustrate the main features from the UK experience in the 1990s.

Adjusting Estimates of the Expected Value of Information for Implementation

Background: Value of information (VoI) calculations give the expected benefits of decision making under perfect information (EVPI) or sample information (EVSI), typically on the premise that any treatment recommendations made in light of this information will be implemented instantly and fully. This assumption is unlikely to hold in health care; evidence shows that obtaining further information typically leads to “improved” rather than “perfect” implementation. Objectives: To present a method of calculating the expected value of further research that accounts for the reality of improved implementation. Methods: This work extends an existing conceptual framework by introducing additional states of the world regarding information (sample information, in addition to current and perfect information) and implementation (improved implementation, in addition to current and optimal implementation). The extension allows calculating the “implementation-adjusted” EVSI (IA-EVSI), a measure that accounts for different degrees of implementation. Calculations of implementation-adjusted estimates are illustrated under different scenarios through a stylized case study in non–small cell lung cancer. Results: In the particular case study, the population values for EVSI and IA-EVSI were £25 million and £8 million, respectively; thus, a decision assuming perfect implementation would have overestimated the expected value of research by about £17 million. IA-EVSI was driven by the assumed time horizon and, importantly, the specified rate of change in implementation: the higher the rate, the greater the IA-EVSI and the lower the difference between IA-EVSI and EVSI. Conclusions: Traditionally calculated measures of population VoI rely on unrealistic assumptions about implementation. This article provides a simple framework that accounts for improved, rather than perfect, implementation and offers more realistic estimates of the expected value of research.

A Practical Application of Value of Information and Prospective Payback of Research to Prioritize Evaluative Research

BACKGROUND AND OBJECTIVES

Efforts to ensure that funded research represents "value for money" have led to increasing calls for the use of analytic methods in research prioritization. A number of analytic approaches have been proposed to assist research funding decisions, the most prominent of which are value of information (VOI) and prospective payback of research (PPoR). Despite the increasing interest in the topic, there are insufficient VOI and PPoR applications on the same case study to contrast their methods and compare their outcomes. We undertook VOI and PPoR analyses to determine the value of conducting 2 proposed research programs. The application served as a vehicle for identifying differences and similarities between the methods, provided insight into the assumptions and practical requirements of undertaking prospective analyses for research prioritization, and highlighted areas for future research.

METHODS

VOI and PPoR were applied to case studies representing proposals for clinical trials in advanced non-small-cell lung cancer and prostate cancer. Decision models were built to synthesize the evidence available prior to the funding decision. VOI (expected value of perfect and sample information) and PPoR (PATHS model) analyses were undertaken using the developed models.

RESULTS AND CONCLUSIONS

VOI and PPoR results agreed in direction, suggesting that the proposed trials would be cost-effective investments. However, results differed in magnitude, largely due to the way each method conceptualizes the possible outcomes of further research and the implementation of research results in practice. Compared with VOI, PPoR is less complex but requires more assumptions. Although the approaches are not free from limitations, they can provide useful input for research funding decisions.

The value of implementation and the value of information: combined and uneven development

AIM

In a budget-constrained health care system, the decision to invest in strategies to improve the implementation of cost-effective technologies must be made alongside decisions regarding investment in the technologies themselves and investment in further research. This article presents a single, unified framework that simultaneously addresses the problem of allocating funds between these separate but linked activities.

METHODS

The framework presents a simple 4-state world where both information and implementation can be either at the current level or "perfect.'' Through this framework, it is possible to determine the maximum return to further research and an upper bound on the value of adopting implementation strategies. The framework is illustrated through case studies of health care technologies selected from those previously considered by the UK National Institute for Health and Clinical Excellence (NICE).

RESULTS

Through the case studies, several key factors that influence the expected values of perfect information and perfect implementation are identified. These factors include the maximum acceptable cost-effectiveness ratio, the level of uncertainty surrounding the adoption decision, the expected net benefits associated with the technologies, the current level of implementation, and the size of the eligible population.

CONCLUSIONS

Previous methods for valuing implementation strategies have not distinguished the value of efficacy research and the value of strategies to change the level of implementation. This framework demonstrates that the value of information and the value of implementation can be examined separately but simultaneously in a single framework. This can usefully inform policy decisions about investment in health care services, further research, and adopting implementation strategies that are likely to differ between technologies.

Setting priorities for research: a practical application of 'payback' and expected value of information

BACKGROUND

Setting priorities for research using economic in addition to scientific criteria can ensure that resources are spent efficiently and equitably.

OBJECTIVE

This study applies two priority setting methods 'payback' and expected value of information (EVI) to two research areas (osteoporosis and pressure ulcers) and where appropriate to four clinical trials: the Record Trial, the Vitamin D and Calcium Trial and the Hip Protector Trial (osteoporosis), and the Pressure Trial (wound care).

METHODS

Two decision-analytic models were developed. For 'payback', the PATHS model was used to estimate the expected net benefits of conducting the four clinical trials. An EVI framework was applied to estimate the cost-effectiveness of conducting further research in the two disease areas investigated.

RESULTS

The application of 'payback' suggests that the Record Trial and the Vitamin D and Calcium Trial would be cost-effective. The Hip Protector and the Pressure Ulcer Trial are cost-effective under certain assumptions concerning the likelihood of obtaining positive, negative or inconclusive results. The EVI method suggests that research would be potentially cost-effective in these areas in the populations considered.

CONCLUSION

EVI provides strategic information for setting priorities for research between disease areas and study populations. 'Payback' provides information on the cost-effectiveness of specific research designs. However, further work in this area, particularly concerning the issue of implementation of research, is required.

Identifying Research Priorities: The Value of Information Associated with Repeat Screening for Age-Related Macular Degeneration

The authors report an analysis that was developed as part of a pilot study examining the use of decision analysis and value-of-information methods to inform research prioritization decisions for the UK health care system. This analysis was conducted to inform decision makers whether additional research on screening for age-related macular degeneration (AMD) would be worthwhile and to demonstrate the benefits and feasibility of using such analytic methods to inform policy decision within the time-lines demanded by existing procedures. A probabilistic decision model was developed to establish the cost-effectiveness of a policy of repeat screening for AMD using the Amsler grid followed by treatment with photodynamic therapy (PDT) compared with 2 alternatives: PDT without screening (self-referral) and no screening or treatment. Screening for AMD appears to be cost-effective on the basis of existing evidence; however, the decision to implement a policy of screening is somewhat uncertain, with a probability that screening is cost-effective of 0.87 and 0.72 for the 20/40 and 20/80 models, respectively, at a threshold of £30,000 per quality-adjusted life-year. The expected value of perfect information (EVPI) associated with this decision is substantial (£6.9 million for the 20/40 model and £14.5 million for the 20/80 model), with a sizeable EVPI associated with the effect of PDT on quality of life. The analysis demonstrates that EVPI analysis can be implemented in a timely fashion to inform the type of research prioritization decisions faced by any health care system. This case study also illustrates the need to account for any structural uncertainties appropriately.

Priority setting for research in health care: an application of value of information analysis to glycoprotein IIb/Illa antagonists in non-ST elevation acute coronary syndrome

The purpose of this study is to explain the rationale for the value of information approach to priority setting for research and to describe the methods intuitively for those familiar with basic decision analytical modeling. A policy-relevant case study is used to show the feasibility of the method and to illustrate the type of output that is generated and how these might be used to frame research recommendations. The case study relates to the use of glycoprotein Ilb/Illa antagonists for the treatment of patients with non-ST elevation acute coronary syndrome. This is an area that recently has been appraised by the National Institute for Health and Clinical Excellence.

Postmenopausal hormone therapy increases use of health services: experience from the Estonian Postmenopausal Hormone Therapy Trial [ISRCTN35338757]

OBJECTIVE

This study was undertaken to compare utilization of health services and health care costs in a randomized hormone therapy trial.

STUDY DESIGN

A total of 1823 healthy postmenopausal women aged 50 to 64 years at the time of sampling were allocated to combined continuous hormone therapy or placebo or no treatment. The analysis was based on routinely collected electronic data in the Estonian Health Insurance Fund database during a follow-up period from 2 to 5 years.

RESULTS

In the nonblind subtrial, the number of all health care visits was 10% higher and the number of visits to family practitioners 16% higher per person-year in the hormone therapy arm. Per person-year, the number of vaginal sonograms was 14% and the number of electrocardiograms 19% higher in the nonblind hormone therapy arm. Outpatient health care costs and drug expenses were higher in the nonblind hormone therapy arm. In the blind subtrial, the number of gynecologic operations, vaginal sonograms and total health care costs was higher in the hormone therapy arm.

CONCLUSION

Hormone therapy caused additional expenses on health care.

Using value of information analysis to prioritise health research: some lessons from recent UK experience

Decisions to adopt, reimburse or issue guidance on the use of health technologies are increasingly being informed by explicit cost-effectiveness analyses of the alternative interventions. Healthcare systems also invest heavily in research and development to support these decisions. However, the increasing transparency of adoption and reimbursement decisions, based on formal analysis, contrasts sharply with research prioritisation and commissioning. This is despite the fact that formal measures of the value of evidence generated by research are readily available. The results of two recent opportunities to apply value of information analysis to directly inform policy decisions about research priorities in the UK are presented. These include a pilot study for the UK National Co-ordinating Centre for Health Technology Assessment (NCCHTA) and a pilot study for the National Institute for Health and Clinical Excellence (NICE). We demonstrate how these results can be used to address a series of policy questions, including: is further research required to support the use of a technology and, if so, what type of research would be most valuable? We also show how the results can be used to address other questions such as, which patient subgroups should be included in subsequent research, which comparators and endpoints should be included, and what length of follow up would be most valuable.

Setting priorities for research

Setting priorities for research should be conducted in order to make the most efficient use of scarce resources. Yet the uptake in practice of such methods by researchers and commissioners of research alike has been slow, in part because the methodologies available to do so have not been widely disseminated. This paper argues that an appropriate priority-setting methodology should meet the objectives of the health system, that is to provide the most health benefits to the population that it serves within the budget constraint and while respecting equity considerations. A condition for these criteria to be met is to construct and operationalise an appropriate definition of the value of research. Five different ways that have been used in practice to value research and set priorities were reviewed. Shortcomings in the ways research is valued make it unlikely that the application of subjective methods, burden of disease methods, and clinical variations and payback methods meet the objectives of the health system. Using the fifth method, value of information, priority-setting can meet the objectives of the health system because it expresses the value of research using the same overall cost-effectiveness framework that is employed for decisions on service provision. However, this method still requires further work to evaluate how research outcomes can then be communicated to clinical practitioners and how practitioners can be encouraged to implement them.

Do HEDIS measures reflect cost-effective practices?

PURPOSE

Whether the Health Plan Employer Data and Information Set (HEDIS) performance measures for managed care plans encourage a cost-effective use of society's resources has not been quantified. Our study objectives were to examine the cost-effectiveness evidence for the clinical practices underlying HEDIS 2000 measures and to develop a list of practices not reflected in HEDIS that have evidence of cost effectiveness.

DATA SOURCES

Two databases of economic evaluations (Harvard School of Public Health Cost-Utility Registry and the Health Economics Evaluation Database) and two published lists of cost-effectiveness ratios in health and medicine.

STUDY SELECTION

For each of the 15 "effectiveness of care" measures in HEDIS 2000, we searched the data through 1998 for cost-effectiveness ratios of similar interventions and target populations. We also searched for important interventions with evidence of cost-effectiveness (<$20,000 per life-year [LY] or quality-adjusted life year [QALY] gained), which are not included in HEDIS. All ratios were standardized to 1998 dollars. The data were collected and analyzed during fall 2000 to summer 2001.

DATA EXTRACTION

Cost-effectiveness ratios reporting outcomes in terms of cost/LY or cost/QALY gained were included if they matched the intervention and population covered by the HEDIS measure.

DATA SYNTHESIS

Evidence was available for 11 of the 15 HEDIS measures. Cost-effectiveness ranges from cost saving to $660,000/LY gained. There are numerous non-HEDIS interventions with some evidence of cost effectiveness, particularly interventions to promote healthy behaviors.

CONCLUSIONS

HEDIS measures generally reflect cost-effective practices; however, in a number of cases, practices may not be cost effective for certain subgroups. Data quality and availability as well as study perspective remain key challenges in judging cost effectiveness. Opportunities exist to refine existing measures and to develop additional measures, which may promote a more efficient use of societal resources, although more research is needed on whether these measures would also satisfy other desirable attributes of HEDIS.

Planning the efficient allocation of research funds: an adapted application of a non-parametric Bayesian value of information analysis

The issue of the efficient allocation of research funds has been addressed using various quantitative methods. Bayesian value of information (VoI) analysis provides an explicit and comprehensive analytic process for the comparison of alternative sources of research. This paper presents an adapted non-parametric application of a VoI analysis of prospective trials comparing alternative adjuvant therapies for postmenopausal women with node positive early breast cancer. The results show that such trials would produce substantial net benefits, though the extent of the net benefits is clearly influenced by the assumed length of usefulness of the research. The application of the VoI methodology shows that such analyses are practical and the recent increase in the use of stochastic decision models in the economic evaluation of health care technologies facilitates further applications of VoI analyses to inform the allocation of research funds.

The preliminary economic evaluation of health technologies for the prioritization of health technology assessments. A discussion

This paper critically evaluates methods for the preliminary economic evaluation of health technologies and the prioritization of health technology assessment projects. It reports on the literature, and considers methods currently employed and the purposes of preliminary appraisal. It concludes that a preliminary economic appraisal needs to be applied to the two main stages of the prioritization process; to have transparent criteria; to allow for an appropriate range of potential outcomes; to be practicable, flexible, and efficient; and to be relevant to the assessment of different research projects.