Conducting Value for Money Analyses for Non-randomised Interventional Studies Including Service Evaluations: An Educational Review with Recommendations

Is Economic Evaluation and Care Commissioning Focused on Achieving the Same Outcomes? Resource-Allocation Considerations and Challenges Using England as a Case Study

Commissioning describes the process of contracting appropriate care services to address pre-identified needs through pre-agreed payment structures. Outcomes-based commissioning (i.e., paying services for pre-agreed outcomes) shares a common goal with economic evaluation: achieving value for money for relevant outcomes (e.g., health) achieved from a finite budget. We describe considerations and challenges as to the practical role of relevant outcomes for evaluation and commissioning, seeking to bridge a gap between economic evaluation evidence and care commissioning. We describe conceptual (e.g., what are 'relevant' outcomes) alongside practical considerations (e.g., quantifying and using relevant endpoint or surrogate outcomes) and pertinent issues when linking outcomes to commissioning-based payment mechanisms, using England as a case study. Economic evaluation often focuses on a single endpoint health-focused maximand, e.g., quality-adjusted life-years (QALYs), whereas commissioning often focuses on activity-based surrogate outcomes (e.g., health monitoring), as easier-to-measure key performance indicators that are more acceptable (e.g., by clinicians) and amenable to being linked with payment structures. However, payments linked to endpoint and/or surrogate outcomes can lead to market inefficiencies; for example, when surrogates do not have the intended causal effect on endpoint outcomes or when service activity focuses on only people who can achieve prespecified payment-linked outcomes. Accounting for and explaining direct links from commissioners' payment structures to surrogate and then endpoint economic outcomes is a vital step to bridging a gap between economic evaluation approaches and commissioning. Decision-analytic models could aid this but they must be designed to account for relevant surrogate and endpoint outcomes, the payments assigned to such outcomes, and their interaction with the system commissioners purport to influence.

Cost-Utility Analysis of Major System Change in Specialist Cancer Surgery in London, England, Using Linked Patient-Level Electronic Health Records and Difference-in-Differences Analysis

BACKGROUND

Studies have shown that centralising surgical treatment for some cancers can improve patient outcomes, but there is limited evidence of the impact on costs or health-related quality of life.

OBJECTIVES

We report the results of a cost-utility analysis of the RESPECT-21 study using difference-in-differences, which investigated the reconfiguration of specialist surgery services for four cancers in an area of London, compared to the Rest of England (ROE).

METHODS

Electronic health records data were obtained from the National Cancer Registration and Analysis Service for patients diagnosed with one of the four cancers of interest between 2012 and 2017. The analysis for each tumour type used a short-term decision tree followed by a 10-year Markov model with 6-monthly cycles. Costs were calculated by applying National Health Service (NHS) Reference Costs to patient-level hospital resource use and supplemented with published data. Cancer-specific preference-based health-related quality-of-life values were obtained from the literature to calculate quality-adjusted life-years (QALYs). Total costs and QALYs were calculated before and after the reconfiguration, in the London Cancer (LC) area and in ROE, and probabilistic sensitivity analysis was performed to illustrate the uncertainty in the results.

RESULTS

At a threshold of £30,000/QALY gained, LC reconfiguration of prostate cancer surgery services had a 79% probability of having been cost-effective compared to non-reconfigured services using difference-in-differences. The oesophago-gastric, bladder and renal reconfigurations had probabilities of 62%, 49% and 12%, respectively, of being cost-effective at the same threshold. Costs and QALYs per surgical patient increased over time for all cancers across both regions to varying degrees. Bladder cancer surgery had the smallest patient numbers and changes in costs, and QALYs were not significant. The largest improvement in outcomes was in renal cancer surgery in ROE, making the relative renal improvements in LC appear modest, and the probability of the LC reconfiguration having been cost-effective low.

CONCLUSIONS

Prostate cancer reconfigurations had the highest probability of being cost-effective. It is not clear, however, whether the prostate results can be considered in isolation, given the reconfigurations occurred simultaneously with other system changes, and healthcare delivery in the NHS is highly networked and collaborative. Routine collection of quality-of-life measures such as the EQ-5D-5L would have improved the analysis.

Economic Evaluation Evidence for Resource-Allocation Decision Making: Bridging the Gap for Local Decision Makers Using English Case Studies

Best-practice economic evaluation methods for health-related decision making at a national level in England are well established, and as a first principle generally involve attempting to maximise the amount of health generated from the health system's budget. Such methods are applied in ways that are broadly transparent and accountable, often at arm's length from explicit political pressures. At local levels of decision making, however, decision making is arguably less likely to be applied according to established overarching principles, is less transparent and is more subject to political pressures. This may be owing to a multiplicity of reasons, for example, undesirability/inappropriateness of such methods, or a failure to make the methods clear to local decision makers. We outline principles for economic evaluations and break down these methods into their component parts, considering their relevance in the English local context. These include taxonomies of decision-making frameworks, budgets, costs, outcome, and characterisations of cost effectiveness. We also explore the role of broader factors, including the relevance of assuming a single fixed budget, pressures resulting from political and budgetary cycles and affordability. We consider the data requirements to inform such deliberations. By setting out principles for economic evaluation methods in a clear language aimed at local decision making, a potential role for such methods can be established, which to date has failed to emerge. While the extent to which these methods can and should be applied are a matter for continued debate, the establishment of such a mutual understanding may assist in the improvement of methods for such decision making and the outcomes resulting from their application.

Harmonization issues in unit costing of service use for multi-country, multi-sectoral health economic evaluations: a scoping review

BACKGROUND

Valuation is a critical part of the costing process in health economic evaluations. However, an overview of specific issues relevant to the European context on harmonizing methodological requirements for the valuation of costs to be used in health economic evaluation is lacking. We aimed to inform the development of an international, harmonized and multi-sectoral costing framework, as sought in the European PECUNIA (ProgrammE in Costing, resource use measurement and outcome valuation for Use in multi-sectoral National and International health economic evaluAtions) project.

METHODS

We conducted a scoping review (information extraction 2008-2021) to a) to demonstrate the degree of heterogeneity that currently exists in the literature regarding central terminology, b) to generate an overview of the most relevant areas for harmonization in multi-sectoral and multi-national costing processes for health economic evaluations, and c) to provide insights into country level variation regarding economic evaluation guidance. A complex search strategy was applied covering key publications on costing methods, glossaries, and international costing recommendations augmented by a targeted author and reference search as well as snowballing. Six European countries served as case studies to describe country-specific harmonization issues. Identified information was qualitatively synthesized and cross-checked using a newly developed, pilot-tested data extraction form.

RESULTS

Costing methods for services were found to be heterogeneous between sectors and country guidelines and may, in practice, be often driven by data availability and reimbursement systems in place. The lack of detailed guidance regarding specific costing methods, recommended data sources, double-counting of costs between sectors, adjustment of unit costs for inflation, transparent handling of overhead costs as well as the unavailability of standardized unit costing estimates in most countries were identified as main drivers of country specific differences in costing methods with a major impact on valuation and cost-effectiveness evidence.

CONCLUSION

This review provides a basic summary of existing costing practices for evaluative purposes across sectors and countries and highlights several common methodological factors influencing divergence in cost valuation methods that would need to be systematically incorporated and addressed in future costing practices to achieve more comparable, harmonized health economic evaluation evidence.

Economic models of community-based falls prevention: a systematic review with subsequent commissioning and methodological recommendations

Background

Falls impose significant health and economic burdens among older populations, making their prevention a priority. Health economic models can inform whether the falls prevention intervention represents a cost-effective use of resources and/or meet additional objectives such as reducing social inequities of health. This study aims to conduct a systematic review (SR) of community-based falls prevention economic models to: (i) systematically identify such models; (ii) synthesise and critically appraise modelling methods/results; and (iii) formulate methodological and commissioning recommendations.

Methods

The SR followed PRISMA 2021 guideline, covering the period 2003–2020, 12 academic databases and grey literature. A study was included if it: targeted community-dwelling persons aged 60 and over and/or aged 50–59 at high falls risk; evaluated intervention(s) designed to reduce falls or fall-related injuries; against any comparator(s); reported outcomes of economic evaluation; used decision modelling; and had English full text. Extracted data fields were grouped by: (A) model and evaluation overview; (B) falls epidemiology features; (C) falls prevention intervention features; and (D) evaluation methods and outcomes. A checklist for falls prevention economic evaluations was used to assess reporting/methodological quality. Extracted fields were narratively synthesised and critically appraised to inform methodological and commissioning recommendations. The SR protocol is registered in the Prospective Register of Systematic Reviews (CRD42021232147).

Results

Forty-six models were identified. The most prevalent issue according to the checklist was non-incorporation of all-cause care costs. Based on general population, lifetime models conducting cost-utility analyses, seven interventions produced favourable ICERs relative to no intervention under the cost-effectiveness threshold of US$41,900 (£30,000) per QALY gained; of these, results for (1) combined multifactorial and environmental intervention, (2) physical activity promotion for women, and (3) targeted vitamin D supplementation were from validated models. Decision-makers should explore the transferability and reaches of interventions in their local settings. There was some evidence that exercise and home modification exacerbate existing social inequities of health. Sixteen methodological recommendations were formulated.

Conclusion

There is significant methodological heterogeneity across falls prevention models. This SR’s appraisals of modelling methods should facilitate the conceptualisation of future falls prevention models. Its synthesis of evaluation outcomes, though limited to published evidence, could inform commissioning.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07647-6.