Applying risk adjusted cost-effectiveness (RAC-E) analysis to hospitals: estimating the costs and consequences of variation in clinical practice

Measuring the overall performance of mental healthcare providers

To date there have been no attempts to construct composite measures of healthcare provider performance which reflect preferences for health and non-health benefits, as well as costs. Health and non-health benefits matter to patients, healthcare providers and the general public. We develop a novel provider performance measurement framework that combines health gain, non-health benefit, and cost and illustrate it with an application to 54 English mental health providers. We apply estimates from a discrete choice experiment eliciting the UK general population's valuation of non-health benefits relative to health gains, to administrative and patient survey data for years 2013-2015 to calculate equivalent health benefit (eHB) for providers. We measure costs as forgone health and quantify the relative performance of providers in terms of equivalent net health benefit (eNHB): the value of the health and non-health benefits minus the forgone benefit equivalent of cost. We compare rankings of providers by eHB, eNHB, and by the rankings produced by the hospital sector regulator. We find that taking account of the non-health benefits in the eNHB measure makes a substantial difference to the evaluation of provider performance. Our study demonstrates that the provider performance evaluation space can be extended beyond measures of health gain and cost, and that this matters for comparison of providers.

Nurses' protocol-based care decision-making: a multiple case study

AIM

To describe and explain nurses' protocol-based care decision-making.

BACKGROUND

Protocol-based care is a strategy to reduce variability in clinical practice. There are no studies looking at protocol-based care decision-making. Understand this process is key to successful implementation.

METHOD

A multiple embedded case study was carried out. Nurses' protocol-based care decision-making was studied in three inpatient wards (medical, surgical and medical-surgical) of a university hospital in northern Spain. Data collection was performed between 2015 and 2016 including documentary analysis, non-participant observations, participant observations and interviews. Analysis of quantitative data involved descriptive statistics and qualitative data was submitted to Burnard's method of content analysis (1996). The data integration comprised the integration of the data set of each case separately and the integration of the findings resulting from the comparison of the cases. The following the thread method of data integration was used for this purpose. The SRQR guideline was used for reporting.

RESULTS

The multiple embedded case study revealed protocol-based care decision-making as a linear and variable process that depends on the context and consists of multiple interrelated elements, among which the risk perception is foremost.

CONCLUSION

This study has allowed progress in protocol-based care decision-making characterisation. This knowledge is crucial to support the design of educational and management strategies aimed at implementing protocol-based care.

RELEVANCE TO CLINICAL PRACTICE

Strategies to promote protocol-based care should address the contexts of practice and the ability of professionals' to accurately assess the degree of risk of clinical activity. Hence, it will promote quality of care, patient safety and efficiency in healthcare cost.

Addressing unwarranted clinical variation: A rapid review of current evidence

INTRODUCTION

Unwarranted clinical variation (UCV) can be described as variation that can only be explained by differences in health system performance. There is a lack of clarity regarding how to define and identify UCV and, once identified, to determine whether it is sufficiently problematic to warrant action. As such, the implementation of systemic approaches to reducing UCV is challenging. A review of approaches to understand, identify, and address UCV was undertaken to determine how conceptual and theoretical frameworks currently attempt to define UCV, the approaches used to identify UCV, and the evidence of their effectiveness.

DESIGN

Rapid evidence assessment (REA) methodology was used.

DATA SOURCES

A range of text words, synonyms, and subject headings were developed for the major concepts of unwarranted clinical variation, standards (and deviation from these standards), and health care environment. Two electronic databases (Medline and Pubmed) were searched from January 2006 to April 2017, in addition to hand searching of relevant journals, reference lists, and grey literature.

DATA SYNTHESIS

Results were merged using reference-management software (Endnote) and duplicates removed. Inclusion criteria were independently applied to potentially relevant articles by 3 reviewers. Findings were presented in a narrative synthesis to highlight key concepts addressed in the published literature.

RESULTS

A total of 48 relevant publications were included in the review; 21 articles were identified as eligible from the database search, 4 from hand searching published work and 23 from the grey literature. The search process highlighted the voluminous literature reporting clinical variation internationally; yet, there is a dearth of evidence regarding systematic approaches to identifying or addressing UCV.

CONCLUSION

Wennberg's classification framework is commonly cited in relation to classifying variation, but no single approach is agreed upon to systematically explore and address UCV. The instances of UCV that warrant investigation and action are largely determined at a systems level currently, and stakeholder engagement in this process is limited. Lack of consensus on an evidence-based definition for UCV remains a substantial barrier to progress in this field.

Multidimensional performance assessment of public sector organisations using dominance criteria

Public sector organisations pursue multiple objectives and serve a number of stakeholders. But stakeholders are rarely explicit about the valuations they attach to different objectives, nor are these valuations likely to be identical. This complicates the assessment of their performance because no single set of weights can be chosen legitimately to aggregate outputs into unidimensional composite scores. We propose the use of dominance criteria in a multidimensional performance assessment framework to identify best practice and poor performance under relatively weak assumptions about stakeholders' preferences. We use as an example providers of hip replacement surgery in the English National Health Service and estimate multivariate multilevel models to study their performance in terms of length of stay, readmission rates, post-operative patient-reported health status and waiting time. We find substantial correlation between objectives and demonstrate that ignoring the correlation can lead to incorrect assessments of performance.

Clinical Practice Variation Needs to be Considered in Cost-Effectiveness Analyses: A Case Study of Patients with a Recent Transient Ischemic Attack or Minor Ischemic Stroke

BACKGROUND AND OBJECTIVE

The cost-effectiveness of clinical interventions is often assessed using current care as the comparator, with national guidelines as a proxy. However, this comparison is inadequate when clinical practice differs from guidelines, or when clinical practice differs between hospitals. We examined the degree of variation in the way patients with a recent transient ischemic attack (TIA) or minor ischemic stroke are assessed and used the results to illustrate the importance of investigating possible clinical practice variation, and the need to perform hospital-level cost-effectiveness analyses (CEAs) when variation exists.

METHODS

Semi-structured interviews were conducted with 16 vascular neurologists in hospitals throughout the Netherlands. Questions were asked about the use of initial and confirmatory diagnostic imaging tests to assess carotid stenosis in patients with a recent TIA or minor ischemic stroke, criteria to perform confirmatory tests, and criteria for treatment. We also performed hospital-level CEAs to illustrate the consequences of the observed diagnostic strategies in which the diagnostic test costs, sensitivity and specified were varied according to the local hospital conditions.

RESULTS

56 % (9/16) of the emergency units and 63 % (10/16) of the outpatient clinics use the initial and confirmatory diagnostic tests to assess carotid stenosis in accordance with the national guidelines. Of the hospitals studied, only one uses the recommended criteria for use of a confirmatory test, 38 % (6/16) follow the guidelines for treatment. The most cost-effective diagnostic test strategy differs between hospitals.

CONCLUSIONS

If important practice variation exists, hospital-level CEAs should be performed. These CEAs should include an assessment of the feasibility and costs of switching to a different strategy.

Process Mining for Clinical Processes

Business process analysis and process mining, particularly within the health care domain, remain under-utilized. Applied research that employs such techniques to routinely collected health care data enables stakeholders to empirically investigate care as it is delivered by different health providers. However, cross-organizational mining and the comparative analysis of processes present a set of unique challenges in terms of ensuring population and activity comparability, visualizing the mined models, and interpreting the results. Without addressing these issues, health providers will find it difficult to use process mining insights, and the potential benefits of evidence-based process improvement within health will remain unrealized. In this article, we present a brief introduction on the nature of health care processes, a review of process mining in health literature, and a case study conducted to explore and learn how health care data and cross-organizational comparisons with process-mining techniques may be approached. The case study applies process-mining techniques to administrative and clinical data for patients who present with chest pain symptoms at one of four public hospitals in South Australia. We demonstrate an approach that provides detailed insights into clinical (quality of patient health) and fiscal (hospital budget) pressures in the delivery of health care. We conclude by discussing the key lessons learned from our experience in conducting business process analysis and process mining based on the data from four different hospitals.

Can the real opportunity cost stand up: displaced services, the straw man outside the room

In current literature, displaced services have been suggested to provide a basis for determining a threshold value for the effects of a new technology as part of a reimbursement process when budgets are fixed. We critically examine the conditions under which displaced services would represent an economically meaningful threshold value. We first show that if we assume that the least cost-effective services are displaced to finance a new technology, then the incremental cost-effectiveness ratio (ICER) of the displaced services (d) only coincides with that related to the opportunity cost of adopting that new technology, the ICER of the most cost-effective service in expansion (n), under highly restrictive conditions-namely, complete allocative efficiency in existing provision of health care interventions. More generally, reimbursement of new technology with a fixed budget comprises two actions; adoption and financing through displacement and the effect of reimbursement is the net effect of these two actions. In order for the reimbursement process to be a pathway to allocative efficiency within a fixed budget, the net effect of the strategy of reimbursement is compared with the most cost-effective alternative strategy for reimbursement: optimal reallocation, the health gain maximizing expansion of existing services financed by the health loss minimizing contraction. The shadow price of the health effects of a new technology, βc = (1/n + 1/d - 1/m)(-1), accounts for both imperfect displacement (the ICER of the displaced service, d < m, the ICER of the least cost-effective of the existing services in contraction) and the allocative inefficiency (n < m) characteristic of health systems.

Practice nurse involvement in primary care depression management: an observational cost-effectiveness analysis

Background

Most evidence on the effect of collaborative care for depression is derived in the selective environment of randomised controlled trials. In collaborative care, practice nurses may act as case managers. The Primary Care Services Improvement Project (PCSIP) aimed to assess the cost-effectiveness of alternative models of practice nurse involvement in a real world Australian setting. Previous analyses have demonstrated the value of high level practice nurse involvement in the management of diabetes and obesity. This paper reports on their value in the management of depression.

Methods

General practices were assigned to a low or high model of care based on observed levels of practice nurse involvement in clinical-based activities for the management of depression (i.e. percentage of depression patients seen, percentage of consultation time spent on clinical-based activities). Linked, routinely collected data was used to determine patient level depression outcomes (proportion of depression-free days) and health service usage costs. Standardised depression assessment tools were not routinely used, therefore a classification framework to determine the patient’s depressive state was developed using proxy measures (e.g. symptoms, medications, referrals, hospitalisations and suicide attempts). Regression analyses of costs and depression outcomes were conducted, using propensity weighting to control for potential confounders.

Results

Capacity to determine depressive state using the classification framework was dependent upon the level of detail provided in medical records. While antidepressant medication prescriptions were a strong indicator of depressive state, they could not be relied upon as the sole measure. Propensity score weighted analyses of total depression-related costs and depression outcomes, found that the high level model of care cost more (95% CI: -$314.76 to $584) and resulted in 5% less depression-free days (95% CI: -0.15 to 0.05), compared to the low level model. However, this result was highly uncertain, as shown by the confidence intervals.

Conclusions

Classification of patients’ depressive state was feasible, but time consuming, using the classification framework proposed. Further validation of the framework is required. Unlike the analyses of diabetes and obesity management, no significant differences in the proportion of depression-free days or health service costs were found between the alternative levels of practice nurse involvement.

A risk-adjusted economic evaluation of alternative models of involvement of practice nurses in management of type 2 diabetes

AIMS

To determine the cost-effectiveness of alternative models of practice nurse involvement in the management of type 2 diabetes within the primary care setting.

METHODS

Linked routinely collected clinical data and resource use (general practitioner visits, hospital services and pharmaceuticals) were used to undertake a risk-adjusted cost-effectiveness analysis of alternative models of care for the management of diabetes patients. These models were based on the reported level of involvement of practice nurses in the provision of clinical-based activities. Potential confounders were controlled for by using propensity score-weighted regression analyses. The impact of alternative models of care on outcomes and costs was measured and incremental cost-effectiveness estimated. The uncertainty around the estimates of cost-effectiveness was illustrated through bootstrapping.

RESULTS

Although the difference in total cost between two models of care was not statistically significant, the high-level model was associated with better outcomes (larger mean reductions in HbA(1c)). The upper 95% confidence intervals showed that the incremental cost per 1% decrease in HbA(1c) is only $454, and per one additional patient to achieve an HbA(1c) value of less than 53 mmol/mol (7.0%) is $323. Further analyses showed little uncertainty surrounding the decision to adopt the high-level model.

CONCLUSIONS

The results provide a strong indication that the high-level model is a cost-effective way of managing diabetes patients. Our findings highlight the need for effective incentives to encourage general practices to better integrate practice nurses in the provision of clinical services.

Accounting for costs, QALYs, and capacity constraints: using discrete-event simulation to evaluate alternative service delivery and organizational scenarios for hospital-based glaucoma services

BACKGROUND

Decision-analytic models are routinely used as a framework for cost-effectiveness analyses of health care services and technologies; however, these models mostly ignore resource constraints. In this study, we use a discrete-event simulation model to inform a cost-effectiveness analysis of alternative options for the organization and delivery of clinical services in the ophthalmology department of a public hospital. The model is novel, given that it represents both disease outcomes and resource constraints in a routine clinical setting.

METHODS

A 5-year discrete-event simulation model representing glaucoma patient services at the Royal Adelaide Hospital (RAH) was implemented and calibrated to patient-level data. The data were sourced from routinely collected waiting and appointment lists, patient record data, and the published literature. Patient-level costs and quality-adjusted life years were estimated for a range of alternative scenarios, including combinations of alternate follow-up times, booking cycles, and treatment pathways.

RESULTS

The model shows that a) extending booking cycle length from 4 to 6 months, b) extending follow-up visit times by 2 to 3 months, and c) using laser in preference to medication are more cost-effective than current practice at the RAH eye clinic.

CONCLUSIONS

The current simulation model provides a useful tool for informing improvements in the organization and delivery of glaucoma services at a local level (e.g., within a hospital), on the basis of expected effects on costs and health outcomes while accounting for current capacity constraints. Our model may be adapted to represent glaucoma services at other hospitals, whereas the general modeling approach could be applied to many other clinical service areas.

Evaluating the effects of variation in clinical practice: a risk adjusted cost-effectiveness (RAC-E) analysis of acute stroke services

BACKGROUND

Methods for the cost-effectiveness analysis of health technologies are now well established, but such methods may also have a useful role in the context of evaluating the effects of variation in applied clinical practice. This study illustrates a general methodology for the comparative analysis of applied clinical practice at alternative institutions--risk adjusted cost-effectiveness (RAC-E) analysis--with an application that compares acute hospital services for stroke patients admitted to the main public hospitals in South Australia.

METHODS

Using linked, routinely collected data on all South Australian hospital separations from July 2001 to June 2008, an analysis of the RAC-E of services provided at four metropolitan hospitals was undertaken using a decision analytic framework. Observed (plus extrapolated) and expected lifetime costs and survival were compared across patient populations, from which the relative cost-effectiveness of services provided at the different hospitals was estimated.

RESULTS

Unadjusted results showed that at one hospital patients incurred fewer costs and gained more life years than at the other hospitals (i.e. it was the dominant hospital). After risk adjustment, the cost minimizing hospital incurred the lowest costs, but with fewer life-years gained than one other hospital. The mean incremental cost per life-year gained of services provided at the most effective hospital was under $20,000, with an associated 65% probability of being cost-effective at a $50,000 per life year monetary threshold.

CONCLUSIONS

RAC-E analyses can be used to identify important variation in the costs and outcomes associated with clinical practice at alternative institutions. Such data provides an impetus for further investigation to identify specific areas of variation, which may then inform the dissemination of best practice service delivery and organisation.