Scoping review of Australian evaluations of healthcare delivery models: are we making the most of the evidence?

Using Expert Elicitation to Adjust Published Intervention Effects to Reflect the Local Context

Background. Local health services make limited use of economic evaluation to inform decisions to fund new health service interventions. One barrier is the relevance of published intervention effects to the local setting, given these effects can strongly reflect the original evaluation context. Expert elicitation methods provide a structured approach to explicitly and transparently adjust published effect estimates, which can then be used in local-level economic evaluations to increase their local relevance. Expert elicitation was used to adjust published effect estimates for 2 interventions targeting the prevention of inpatient hypoglycemia. Methods. Elicitation was undertaken with 6 clinical experts. They were systematically presented with information regarding potential differences in patient characteristics and quality of care between the published study and local contexts, and regarding the design and application of the published study. The experts then assessed the intervention effects and provided estimates of the most realistic, most pessimistic, and most optimistic intervention effect sizes in the local context. Results. The experts estimated both interventions would be less effective in the local setting compared with the published effect estimates. For one intervention, the experts expected the lower complexity of admitted patients in the local setting would reduce the intervention's effectiveness. For the other intervention, the reduced effect was largely driven by differences in the scope of implementation (hospital-wide in the local setting compared with targeted implementation in the evaluation). Conclusions. The pragmatic elicitation methods reported in this article provide a feasible and acceptable approach to assess and adjust published intervention effects to better reflect expected effects in the local context. Further development and application of these methods is proposed to facilitate the use of local-level economic evaluation.

Highlights

Local health services make limited use of economic evaluation to inform their decisions on the funding of new health service interventions. One barrier to use is the relevance of published intervention evaluations to the local setting.Expert elicitation methods provide a structured way to consider differences between the evaluation and local settings and to explicitly and transparently adjust published effect estimates for use in local economic evaluations.The pragmatic elicitation methods reported in this article offer a feasible and acceptable approach to adjusting published intervention effects to better reflect the effects expected in the local context. This increases the relevance of economic evaluations for local decision makers.

Embedding an economist in regional and rural health services to add value and reduce waste by improving local-level decision-making: protocol for the 'embedded Economist' program and evaluation

Background

Systematic approaches to the inclusion of economic evaluation in national healthcare decision-making are usual. It is less common for economic evaluation to be routinely undertaken at the ‘local-level’ (e.g. in a health service or hospital) despite the largest proportion of health care expenditure being determined at this service level and recognition by local health service decision makers of the need for capacity building in economic evaluation skills. This paper describes a novel program – the embedded Economist (eE) Program. The eE Program aims to increase local health service staff awareness of, and develop their capacity to access and apply, economic evaluation principles in decision making. The eE program evaluation is also described. The aim of the evaluation is to capture the contextual, procedural and relational aspects that assist and detract from the eE program aims; as well as the outcomes and impact from the specific eE projects.

Methods

The eE Program consists of a embedding a health economist in six health services and the provision of supported education in applied economic evaluation, provided via a community of practice and a university course. The embedded approach is grounded in co-production, embedded researchers and ‘slow science’. The sites, participants, and program design are described. The program evaluation includes qualitative data collection via surveys, semi-structured interviews, observations and field diaries. In order to share interim findings, data are collected and analysed prior, during and after implementation of the eE program, at each of the six health service sites. The surveys will be analysed by calculating frequencies and descriptive statistics. A thematic analysis will be conducted on interview, observation and filed diary data. The Framework to Assess the Impact from Translational health research (FAIT) is utilised to assess the overall impact of the eE Program.

Discussion

This program and evaluation will contribute to knowledge about how best to build capacity and skills in economic evaluation amongst decision-makers working in local-level health services. It will examine the extent to which participants are able to improve their ability to utilise evidence to inform decisions, avoid waste and improve the value of care delivery.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06181-1.