Value for money of changing healthcare services? Economic evaluation of quality improvement

Engaging stakeholders to retrospectively discern implementation strategies to support program evaluation: Proposed method and case study

BACKGROUND

Availability of evidence-based practices (EBPs) is critical for improving health care outcomes, but diffusion can be challenging. Implementation activities increase the adoption of EBPs and support sustainability. However, when implementation activities are a part of quality improvement processes, evaluation of the time and cost associated with these activities is challenged by the need for a correct classification of these activities to a known taxonomy of implementation strategies by implementation actors.

DESIGN

Observational study of a four-stage, stakeholder-engaged process for identifying implementation activities and estimating the associated costs.

RESULTS

A national initiative in the Veterans Health Administration (VHA) to improve Advance Care Planning (ACP) via Group Visits (ACP-GV) for rural veterans identified 49 potential implementation activities. Evaluators translated and reduced these to 14 strategies used across three groups with the aid of implementation actors. Data were collected to determine the total implementation effort and applied cost estimates to estimate the budget impact of implementation for VHA.

LIMITATIONS

Recall bias may influence the identification of potential implementation activities.

CONCLUSIONS

This process improved understanding of the implementation effort and allowed estimation of ACP-GV 's budget impact.

IMPLICATIONS

A four-stage, stakeholder-engaged methodology can be applied to other initiatives when a pragmatic evaluation of implementation efforts is needed.

Reporting standards, outcomes and costs of quality improvement studies in Ireland: a scoping review

OBJECTIVES

To profile the aims and characteristics of quality improvement (QI) initiatives conducted in Ireland, to review the quality of their reporting and to assess outcomes and costs.

DESIGN

Scoping review.

DATA SOURCES

Systematic searches were conducted in PubMed, Web of Science, Embase, Google Scholar, Lenus and rian.ie. Two researchers independently screened abstracts (n=379) and separately reviewed 43 studies identified for inclusion using a 70-item critique tool. The tool was based on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS), an appraisal instrument for QI intervention publications, and health economics reporting criteria. After reaching consensus, the final dataset was analysed using descriptive statistics. To support interpretations, findings were presented at a national stakeholder workshop.

ELIGIBILITY CRITERIA

QI studies implemented and evaluated in Ireland and published between January 2015 and April 2020.

RESULTS

The 43 studies represented various QI interventions. Most studies were peer-reviewed publications (n=37), conducted in hospitals (n=38). Studies mainly aimed to improve the 'effectiveness' (65%), 'efficiency' (53%), 'timeliness' (47%) and 'safety' (44%) of care. Fewer aimed to improve 'patient-centredness' (30%), 'value for money' (23%) or 'staff well-being' (9%). No study aimed to increase 'equity'. Seventy per cent of studies described 14 of 16 QI-MQCS dimensions. Least often studies reported the 'penetration/reach' of an initiative and only 35% reported health outcomes. While 53% of studies expressed awareness of costs, only eight provided at least one quantifiable figure for costs or savings. No studies assessed the cost-effectiveness of the QI.

CONCLUSION

Irish QI studies included in our review demonstrate varied aims and high reporting standards. Strategies are needed to support greater stimulation and dissemination of QI beyond the hospital sector and awareness of equity issues as QI work. Systematic measurement and reporting of costs and outcomes can be facilitated by integrating principles of health economics in QI education and guidelines.

Practical Guide to Implementation Science for Surgical Oncologists: Case Study of Breast Cancer Short Stay Program

BACKGROUND

Long lags exist in adoption and uptake of evidence-based interventions into real-world clinical practice based on oncology clinical trial results. Implementation science (IS) is a distinct field of health services research that aims to understand the barriers related to adoption of evidence-based guidelines and research in clinical practice.

METHODS

Use of IS study design, methods, and outcomes can be elusive to surgical oncologists despite the tremendous need for the application of IS to bridge the evidence-to-practice gap. This report describes key components of high-quality IS.

RESULTS

Herein, we illustrate how IS can be used in surgical oncology practice. Examples from implementation of the breast cancer Short Stay Program (SSP) in Netherlands is used to illustrate IS methods. Specific funding and training opportunities in implementation science are described in detail.

CONCLUSION

Use of IS in surgical oncology can help improve the uptake of evidence based medicine.

Clinical effectiveness and cost effectiveness of individual mental health workers colocated within primary care practices: a systematic literature review

OBJECTIVES

Mental health disorders contribute significantly to the global burden of disease and lead to extensive strain on health systems. The integration of mental health workers into primary care has been proposed as one possible solution, but evidence of clinical and cost effectiveness of this approach is unclear. We reviewed the clinical and cost effectiveness of mental health workers colocated within primary care practices.

DESIGN

Systematic literature review.

DATA SOURCES

We searched the Medline, Embase, PsycINFO, Healthcare Management Information Consortium (HMIC) and Global Health databases.

ELIGIBILITY CRITERIA

All quantitative studies published before July 2019 were eligible for the review; participants of any age and gender were included. Studies did not need to report a certain outcome measure or comparator in order to be eligible.

DATA EXTRACTION AND SYNTHESIS

Data were extracted using a standardised table; however, pooled analysis proved unfeasible. Studies were assessed for risk of bias using the Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I) tool and the Cochrane collaboration's tool for assessing risk of bias in randomised trials.

RESULTS

Fifteen studies from four countries were included. Mental health worker integration was associated with mental health benefits to varied populations, including minority groups and those with comorbid chronic diseases. Furthermore, the interventions were correlated with high patient satisfaction and increases in specialist mental health referrals among minority populations. However, there was insufficient evidence to suggest clinical outcomes were significantly different from usual general practitioner care.

CONCLUSIONS

While there appear to be some benefits associated with mental health worker integration in primary care practices, we found insufficient evidence to conclude that an onsite primary care mental health worker is significantly more clinically or cost effective when compared with usual general practitioner care. There should therefore be an increased emphasis on generating new evidence from clinical trials to better understand the benefits and effectiveness of mental health workers colocated within primary care practices.

The business case for quality improvement

Value in healthcare can be defined as providing the optimal outcome per health dollar spent. Improving the value of healthcare for patients and healthcare organizations requires an understanding and evaluation of the costs and benefits. Investing in quality improvement (QI) work can bring about financial results for healthcare organizations over time, have beneficial organizational effects, and improve outcomes for patients. This article continues a series of QI educational papers in the Journal of Perinatology, and reviews financial and economic measures used to create the business case for QI. Ultimately, the business case for QI is better defined as a business strategy for success.

Introducing a reserve waiting list initiative for elective general surgery at a District General Hospital

Cancelled operations represent a significant burden on the National Health Service in terms of theatre efficiency, financial implications and lost training opportunities. Moreover, they carry considerable physical and psychological effects to patients and their relatives. Evidence has shown that up to 93% of cancelled operations are due to patient-related factors. An analysis at our District General Hospital revealed that approximately 18 operations are cancelled on the day of surgery each month. This equates to 27 hours of allocated operating time valued by the trust as £67 500, not being used effectively. This retrospective quality improvement report aims to reduce unused theatre time due to cancelled elective operations in general surgery theatres-thereby improving theatre efficiency and patient care. To ascertain the baseline number of cancelled operations, an initial review of theatre cases was undertaken. Further review was then completed after implementation of two improvements-a short notice surgical waiting list and fast track pre-assessment clinics. The results showed that implementation of the reserve surgical waiting list reduced unused operating time by an average of 2.25 hours per month. By further adding in the fast track preassessment clinic, these figures increased to an average of 11.5 hours over the next 3 months. This precipitated a reutilisation of otherwise wasted theatre time. Economic impact of this time amounts around £28 750 a month, after implementation of both improvements. Simple protocol changes can lead to large improvements in the efficient running of theatres. The resultant change has improved patient satisfaction, led to greater training opportunities and improved theatre efficiency. Extrapolation of our results show better usage of previously underused theatre time, to the equivalent worth of £345 000. Further implementation of these improvements in other surgical specialities and hospitals would be beneficial.

Accounting for Capacity Constraints in Economic Evaluations of Precision Medicine: A Systematic Review

BACKGROUND AND OBJECTIVE

Precision (stratified or personalised) medicine is underpinned by the premise that it is feasible to identify known heterogeneity using a specific test or algorithm in patient populations and to use this information to guide patient care to improve health and well-being. This study aimed to understand if, and how, previous economic evaluations of precision medicine had taken account of the impact of capacity constraints.

METHODS

A meta-review was conducted of published systematic reviews of economic evaluations of precision medicine (test-treat interventions) and individual studies included in these reviews. Due to the volume of studies identified, a sample of papers published from 2007 to 2015 was collated. A narrative analysis identified whether potential capacity constraints were discussed qualitatively in the studies and, if relevant, which quantitative methods were used to account for capacity constraints.

RESULTS

A total of 45 systematic reviews of economic evaluations of precision medicine were identified, from which 222 studies focusing on test-treat interventions, published between 2007 and 2015, were extracted. Of these studies, 33 (15%) qualitatively discussed the potential impact of capacity constraints, including budget constraints; quality of tests and the testing process; ease of use of tests in clinical practice; and decision uncertainty. Quantitative methods (nine studies) to account for capacity constraints included static methods such as capturing inefficiencies in trials or models and sensitivity analysis around model parameters; and dynamic methods, which allow the impact of capacity constraints on cost effectiveness to change over time.

CONCLUSIONS

Understanding the cost effectiveness of precision medicine is necessary, but not sufficient, evidence for its successful implementation. There are currently few examples of evaluations that have quantified the impact of capacity constraints, which suggests an area of focus for future research.

Emergent Challenges in Determining Costs for Economic Evaluations

This paper describes methods of determining costs for economic evaluations of healthcare and considers how cost determination is being affected by recent developments in healthcare. The literature was reviewed to identify the strengths and weaknesses of the four principal methods of cost determination: micro-costing, activity-based costing, charge-based costing, and gross costing. A scoping review was conducted to identify key trends in healthcare delivery and to identify costing issues associated with these changes. Existing guidelines provide information on how to implement various costing methods. Bottom-up costing is needed when accuracy is paramount, but top-down approaches are often the only feasible approach. We describe six healthcare trends that have important implications for costing methodology: (1) reform in payment mechanisms; (2) care delivery in less restrictive settings; (3) the growth of telehealth interventions; (4) the proliferation of new technology; (5) patient privacy concerns; and (6) growing efforts to implement guidelines. Some costs are difficult to measure and have been overlooked. These include physician services for inpatients, facility costs for outpatient services, the cost of developing treatment innovations, patient and caregiver costs, and the indirect costs of organizational interventions. Standardized methods are needed to determine social welfare and productivity costs. In the future, cost determination will be facilitated by technological advances but hindered by the shift to capitated payment, to the provision of care in less restrictive settings, and by heightened concern for medical record privacy.

A detailed report of the resource use and costs associated with implementation of a short stay programme for breast cancer surgery

BACKGROUND

Despite the increased attention for assessing the effectiveness of implementation strategies, most implementation studies provide little or no information on its associated costs. The focus of the current study was to provide a detailed report of the resource use and costs associated with implementation of a short stay programme for breast cancer surgery in four Dutch hospitals.

METHODS

The analysis was performed alongside a multi-centre implementation study. The process of identification, measurement and valuation of the implementation activities was based on recommendations for the design, analysis and reporting of health technology assessments. A scoring form was developed to prospectively determine the implementation activities at professional and implementation expert level. A time horizon of 5 years was used to calculate the implementation costs per patient.

RESULTS

Identified activities were consisted of development and execution of the implementation strategy during the implementation project. Total implementation costs over the four hospitals were €83.293. Mean implementation costs, calculated for 660 patients treated over a period of 5 years, were €25 per patient. Subgroup analyses showed that the implementation costs ranged from €3.942 to €32.000 on hospital level. From a local hospital perspective, overall implementation costs were €21 per patient, after exclusion of the costs made by the expert centre.

CONCLUSIONS

We provided a detailed case description of how implementation costs can be determined. Notable differences in implementation costs between hospitals were observed.

TRIAL REGISTRATION

ISRCTN

ISRCTN77253391.

De-implementation of expensive blood saving measures in hip and knee arthroplasties: study protocol for the LISBOA-II cluster randomized trial

Background

Despite evidence that erythropoietin and intra- and postoperative blood salvage are expensive techniques considered to be non-cost-effective in primary elective total hip and knee arthroplasties in the Netherlands, Dutch medical professionals use them frequently to prevent the need for allogeneic transfusion. To actually change physicians’ practice, a tailored strategy aimed at barriers that hinder physicians in abandoning the use of erythropoietin and perioperative blood salvage was systematically developed. The study aims to examine the effectiveness, feasibility and costs of this tailored de-implementation strategy compared to a control strategy.

Methods/Design

A cluster randomized controlled trial including an effect, process and economic evaluation will be conducted in a minimum of 20 Dutch hospitals. Randomisation takes place at hospital level. The hospitals in the intervention group will receive a tailored de-implementation strategy that consists of four components: interactive education, feedback in educational outreach visits, electronically sent reports on hospital performance (all aimed at orthopedic surgeons and anesthesiologists), and information letters or emails aimed at other involved professionals within the intervention hospital (transfusion committee, OR-personnel, pharmacists). The hospitals in the control group will receive a control strategy (i.e., passive dissemination of available evidence). Outcomes will be measured at patient level, using retrospective medical record review. This will be done in all hospitals at baseline and after completion of the intervention period. The primary outcome of the effect evaluation is the percentage of patients undergoing primary elective total hip or knee arthroplasty in which erythropoietin or perioperative blood salvage is applied. The actual exposure to the tailored strategy and users’ experiences will be assessed in the process evaluation. In the economic evaluation, the costs of the tailored strategy and the control strategy in relation to the difference in their effectiveness will be compared.

Discussion

This study will show whether a systematically developed tailored strategy is more effective for de-implementation of non-cost-effective blood saving measures than the control strategy. This knowledge can be used in national and international initiatives to make healthcare more efficient. It also provides more generalized knowledge regarding de-implementation strategies.

Trial registration

This trial is registered at the Dutch Trial Register NTR4044.

Sustainability of healthcare innovations (SUSHI): long term effects of two implemented surgical care programmes (protocol)

Background

Two healthcare innovations were successfully implemented using different implementation strategies. First, a Short Stay Programme for breast cancer surgery (MaDO) was implemented in four early adopter hospitals, using a hospital-tailored implementation strategy. Second, the Enhanced Recovery After Surgery (ERAS) programme for colonic surgery was implemented in 33 Dutch hospitals, using a generic breakthrough implementation strategy. Both strategies resulted in a shorter hospital length of stay without a decrease in quality of care. Currently, it is unclear to what extent these innovative programmes and their results have been sustained three to five years following implementation. The aim of the sustainability of healthcare innovations (SUSHI) study is to analyse sustainability and its determinants using two implementation cases.

Methods

This observational study uses a mixed methods approach. The study will be performed in 14 hospitals in the Netherlands, from November 2010. For both implementation cases, the programme aspects and the effects will be evaluated by means of a follow-up measurement in 160 patients who underwent breast cancer surgery and 300 patients who underwent colonic surgery. A policy cost-effectiveness analysis from a societal perspective will be performed prospectively for the Short Stay Programme for breast cancer surgery in 160 patients. To study determinants of sustainability key professionals in the multidisciplinary care processes and implementation change agents will be interviewed using semi-structured interviews.

Discussion

The concept of sustainability is not commonly studied in implementation science. The SUSHI study will provide insight in to what extent the short-term implementation benefits have been maintained and in the determinants of long-term continuation of programme activities.

Implementing guidelines into clinical practice: what is the value?

RATIONALE AND OBJECTIVE

In budget-constrained health systems, decision makers need to consider both the costs and effects of introducing and actively implementing clinical guidance. We aim to demonstrate how, as an alternative to conventional methods, a total net benefit approach to economic evaluation can be used to inform decision making about guidelines and specific implementation strategies, like education or financial incentives.

METHODS

Aside from providing more detail on the decision framework, we describe how to collect and analyse the relevant data for calculating the total net benefit of guideline use and the value of implementation. We illustrate the process of decision analysis for a stylized example on improving diabetes care in the UK. For the analysis, economic evidence on intensified glycemic control and that on audit and feedback to promote control is combined with information on diabetes practice.

RESULTS

Our illustration demonstrates that the total net benefit of guideline use and the value of implementation can vary substantially, depending on the clinical intervention chosen, the health system being studied and the specific implementation strategies. This also holds for the threshold value for cost-effectiveness, the duration of guideline usage or validity, the size of the patient population served, and the trends and ceiling rates in the implementation of clinical guidance.

CONCLUSIONS

In comparison with conventional methods for health economic evaluation, a total net benefit approach allows for the explicit consideration of the current (or future) use of guidelines or guideline recommendations, the cost of implementation and the scope of clinical practice. Decisions made on the basis of the total net benefit of all plausible combinations of clinical guidance and implementation strategies provide optimal patient care and an efficient use of resources.

Viewpoint: Economic evaluation of package of care interventions employing clinical guidelines

Increasingly attention is shifting towards delivering essential packages of care, often based on clinical practice guidelines, as a means to improve maternal, child and newborn survival in low-income settings. Cost effectiveness analysis (CEA), allied to the evaluation of less complex intervention, has become an increasingly important tool for priority setting. Arguably such analyses should be extended to inform decisions around the deployment of more complex interventions. In the discussion, we illustrate some of the challenges facing the extension of CEA to this area. We suggest that there are both practical and methodological challenges to overcome when conducting economic evaluation for packages of care interventions that incorporate clinical guidelines. Some might be overcome by developing specific guidance on approaches, for example clarity in identifying relevant costs. Some require consensus on methods. The greatest challenge, however, lies in how to incorporate, as measures of effectiveness, process measures of service quality. Questions on which measures to use, how multiple measures might be combined, how improvements in one area might be compared with those in another and what value is associated with improvement in health worker practices are yet to be answered.

Cost and impact of a quality improvement programme in mental health services

OBJECTIVE

To estimate the cost and impact of a centrally-driven quality improvement initiative in four UK mental health communities.

METHODS

Total costs in year 1 were identified using documentation, a staff survey, semi-structured interviews and discussion groups. Few outcome data were collected within the programme so thematic analysis was used to identify the programme's impact within its five broad underlying principles.

RESULTS

The survey had a 40% response. Total costs ranged between pound164,000 and pound458,000 per site, plus staff time spent on workstreams. There was a very hazy view of the resources absorbed and poor recording of expenditure and activity. The initiative generated little demonstrable improvements in service quality but some participants reported changes in attitudes.

CONCLUSIONS

Given the difficult contexts, short time-scales and capacity constraints, the programme's lack of impact is not surprising. It may, however, represent a worthwhile investment in cultural change which might facilitate improvements in how services are delivered.

Modeling the value for money of changing clinical practice change: a stochastic application in diabetes care

BACKGROUND

Decision making about resource allocation for guideline implementation to change clinical practice is inevitably undertaken in a context of uncertainty surrounding the cost-effectiveness of both clinical guidelines and implementation strategies. Adopting a total net benefit approach, a model was recently developed to overcome problems with the use of combined ratio statistics when analyzing decision uncertainty.

OBJECTIVE

To demonstrate the stochastic application of the model for informing decision making about the adoption of an audit and feedback strategy for implementing a guideline recommending intensive blood glucose control in type 2 diabetes in primary care in the Netherlands.

METHODS

An integrated Bayesian approach to decision modeling and evidence synthesis is adopted, using Markov Chain Monte Carlo simulation in WinBUGs. Data on model parameters is gathered from various sources, with effectiveness of implementation being estimated using pooled, random-effects meta-analysis. Decision uncertainty is illustrated using cost-effectiveness acceptability curves and frontier.

RESULTS

Decisions about whether to adopt intensified glycemic control and whether to adopt audit and feedback alter for the maximum values that decision makers are willing to pay for health gain. Through simultaneously incorporating uncertain economic evidence on both guidance and implementation strategy, the cost-effectiveness acceptability curves and cost-effectiveness acceptability frontier show an increase in decision uncertainty concerning guideline implementation.

CONCLUSIONS

The stochastic application in diabetes care demonstrates that the model provides a simple and useful tool for quantifying and exploring the (combined) uncertainty associated with decision making about adopting guidelines and implementation strategies and, therefore, for informing decisions about efficient resource allocation to change clinical practice.

Value of information and value of implementation: application of an analytic framework to inform resource allocation decisions in metastatic hormone-refractory prostate cancer

OBJECTIVE

In a budget-constrained health-care system, decisions about investing in strategies to promote implementation have to be made alongside decisions about health-care provision and research funding. Using a Bayesian decision-theoretic approach, an analytic framework has been developed to inform these separate but related decisions, establishing the expected value of both perfect information (EVPI) and perfect implementation (EVPIM). We applied this framework to inform decision-making about resource allocation to metastatic hormone-refractory prostate cancer (mHRPC) in the UK.

METHODS

Based on available evidence on the cost-effectiveness of all plausible treatments for mHRPC, we determined which treatment option(s) were cost-effective and explored the uncertainty surrounding this decision. Given the decision uncertainty and the variation in care provided by health-care professionals, we then determined the EVPI and EVPIM. Finally, we performed sensitivity analyses to explore the influence of alternative assumptions regarding various decision parameters on the efficiency of resource allocation.

RESULTS

Depending on the cost-effectiveness threshold (lambda), we identified mitoxantrone plus prednisone/prednisolone and docetaxel plus prednisone/prednisolone (3 weekly) as the optimal treatments for mHRPC. Given current clinical practice, there appears to be considerable scope for improving the efficiency of health-care provision: the EVPI (estimated to be over pound13 million) indicates that acquiring further information could be cost-effective; and the EVPIM (estimated to be over pound4 million) suggests that investing in strategies to implement the treatments regimens being identified as optimal is potentially worthwhile. Through sensitivity analyses, we found that the EVPI and EVPIM are mainly driven by lambda, the number of treatment options being considered, the current level of implementation, and the size of the eligible patient population.

CONCLUSION

The application demonstrates that the framework provides a simple and useful analytic tool for decision-makers to address resource allocation problems between health-care provision, further research, and implementation efforts.

Value for money in changing clinical practice: should decisions about guidelines and implementation strategies be made sequentially or simultaneously?

Decisions about clinical practice change, that is, which guidelines to adopt and how to implement them, can be made sequentially or simultaneously. Decision makers adopting a sequential approach first compare the costs and effects of alternative guidelines to select the best set of guideline recommendations for patient management and subsequently examine the implementation costs and effects to choose the best strategy to implement the selected guideline. In an integral approach, decision makers simultaneously decide about the guideline and the implementation strategy on the basis of the overall value for money in changing clinical practice. This article demonstrates that the decision to use a sequential v. an integral approach affects the need for detailed information and the complexity of the decision analytic process. More importantly, it may lead to different choices of guidelines and implementation strategies for clinical practice change. The differences in decision making and decision analysis between the alternative approaches are comprehensively illustrated using 2 hypothetical examples. We argue that, in most cases, an integral approach to deciding about change in clinical practice is preferred, as this provides more efficient use of scarce health-care resources.

Nurse-led motivational interviewing to change the lifestyle of patients with type 2 diabetes (MILD-project): protocol for a cluster, randomized, controlled trial on implementing lifestyle recommendations

Background

The diabetes of many patients is managed in general practice; healthcare providers aim to promote healthful behaviors, such as healthful diet, adequate physical activity, and smoking cessation. These measures may decrease insulin resistance, improve glycemic control, lipid abnormalities, and hypertension. They may also prevent cardiovascular disease and complications of diabetes. However, professionals do not adhere optimally to guidelines for lifestyle counseling. Motivational interviewing to change the lifestyle of patients with type 2 diabetes is intended to improve diabetes care in accordance with the national guidelines for lifestyle counseling. Primary care nurses will be trained in motivational interviewing embedded in structured care in general practice. The aim of this paper is to describe the design and methods of a study evaluating the effects of the nurses' training on patient outcomes.

Methods/Design

A cluster, randomized, controlled trial involving 70 general practices (35 practices in the intervention arm and 35 in the control arm) starting in March 2007. A total of 700 patients with type 2 diabetes will be recruited. The patients in the intervention arm will receive care from the primary care nurse, who will receive training in an implementation strategy with motivational interviewing as the core component. Other components of this strategy will be adaptation of the diabetes protocol to local circumstances, introduction of a social map for lifestyle support, and educational and supportive tools for sustaining motivational interviewing. The control arm will be encouraged to maintain usual care. The effect measures will be the care process, metabolic parameters (glycosylated hemoglobin, blood pressure and lipids), lifestyle (diet, physical activity, smoking, and alcohol), health-related quality of life, and patients' willingness to change behaviors. The measurements will take place at baseline and after 14 months.

Discussion

Applying motivational interviewing for patients with diabetes in primary care has been studied, but to our knowledge, no other study has yet evaluated the implementation and sustainability of motivating and involving patients in day-to-day diabetes care in general practice. If this intervention proves to be effective and cost-effective, large-scale implementation of this nurse-oriented intervention will be considered and anticipated.

Trial registration

Current Controlled Trials ISRCTN68707773.

Implementing cognitive behavior therapy for chronic fatigue syndrome in mental health care: a costs and outcomes analysis

Background

This study investigated the costs and outcomes of implementing cognitive behavior therapy (CBT) for chronic fatigue syndrome (CFS) in a mental health center (MHC). CBT is an evidence-based treatment for CFS that was scarcely available until now. To investigate the possibilities for wider implementation, a pilot implementation project was set up.

Method

Costs and effects were evaluated in a non-controlled before- and after study with an eight months time-horizon. Both the costs of performing the treatments and the costs of implementing the treatment program were included in the analysis. The implementation interventions included: informing general practitioners (GPs) and CFS patients, training therapists, and instructing the MHC employees. Given the non-controlled design, cost outcome ratios (CORs) and their acceptability curves were analyzed. Analyses were done from a health care perspective and from a societal perspective. Bootstrap analyses were performed to estimate the uncertainty around the cost and outcome results.

Results

125 CFS patients were included in the study. After treatment 37% had recovered from CFS and the mean gained QALY was 0.03. Costs of patients' health care and productivity losses had decreased significantly. From the societal perspective the implementation led to cost savings and to higher health states for patients, indicating dominancy. From the health care perspective the implementation revealed overall costs of €5.320 per recovered patient, with an acceptability curve showing a 100% probability for a positive COR at a willingness to pay threshold of €6.500 per recovered patient.

Conclusion

Implementing CBT for CFS in a MHC appeared to have a favorable cost outcome ratio (COR) from a societal perspective. From a health care perspective the COR depended on how much a recovered CFS patient is being valued. The strength of the evidence was limited by the non-controlled design. The outcomes of this study might facilitate health care providers when confronted with the decision whether or not to adopt CBT for CFS in their institution.

The perceived financial impact of quality improvement efforts in community health centers

We administered surveys to 100 chief executive officers (CEOs) of community health centers to determine their perceptions of the financial impact of the Health Disparities Collaboratives, a national quality improvement initiative. One third of the CEOs believed that the HDC had a negative financial impact on their health center, and this perception was significantly correlated with centers having a higher proportion of uninsured patients. Performance-based payment incentives may improve care but may also add new financial burdens to facilities that treat the uninsured population. As such, a provider's payer mix may need to be considered in the design of QI programs if they are to be sustainable.

The role of economics in the QUERI program: QUERI Series

BACKGROUND

The United States (U.S.) Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) has implemented economic analyses in single-site and multi-site clinical trials. To date, no one has reviewed whether the QUERI Centers are taking an optimal approach to doing so. Consistent with the continuous learning culture of the QUERI Program, this paper provides such a reflection.

METHODS

We present a case study of QUERI as an example of how economic considerations can and should be integrated into implementation research within both single and multi-site studies. We review theoretical and applied cost research in implementation studies outside and within VA. We also present a critique of the use of economic research within the QUERI program.

RESULTS

Economic evaluation is a key element of implementation research. QUERI has contributed many developments in the field of implementation but has only recently begun multi-site implementation trials across multiple regions within the national VA healthcare system. These trials are unusual in their emphasis on developing detailed costs of implementation, as well as in the use of business case analyses (budget impact analyses).

CONCLUSION

Economics appears to play an important role in QUERI implementation studies, only after implementation has reached the stage of multi-site trials. Economic analysis could better inform the choice of which clinical best practices to implement and the choice of implementation interventions to employ. QUERI economics also would benefit from research on costing methods and development of widely accepted international standards for implementation economics.

The value of implementation and the value of information: combined and uneven development

AIM

In a budget-constrained health care system, the decision to invest in strategies to improve the implementation of cost-effective technologies must be made alongside decisions regarding investment in the technologies themselves and investment in further research. This article presents a single, unified framework that simultaneously addresses the problem of allocating funds between these separate but linked activities.

METHODS

The framework presents a simple 4-state world where both information and implementation can be either at the current level or "perfect.'' Through this framework, it is possible to determine the maximum return to further research and an upper bound on the value of adopting implementation strategies. The framework is illustrated through case studies of health care technologies selected from those previously considered by the UK National Institute for Health and Clinical Excellence (NICE).

RESULTS

Through the case studies, several key factors that influence the expected values of perfect information and perfect implementation are identified. These factors include the maximum acceptable cost-effectiveness ratio, the level of uncertainty surrounding the adoption decision, the expected net benefits associated with the technologies, the current level of implementation, and the size of the eligible population.

CONCLUSIONS

Previous methods for valuing implementation strategies have not distinguished the value of efficacy research and the value of strategies to change the level of implementation. This framework demonstrates that the value of information and the value of implementation can be examined separately but simultaneously in a single framework. This can usefully inform policy decisions about investment in health care services, further research, and adopting implementation strategies that are likely to differ between technologies.

Improving patient adherence to lifestyle advice (IMPALA): a cluster-randomised controlled trial on the implementation of a nurse-led intervention for cardiovascular risk management in primary care (protocol)

BACKGROUND

Many patients at high risk of cardiovascular diseases are managed and monitored in general practice. Recommendations for cardiovascular risk management, including lifestyle change, are clearly described in the Dutch national guideline. Although lifestyle interventions, such as advice on diet, physical exercise, smoking and alcohol, have moderate, but potentially relevant effects in these patients, adherence to lifestyle advice in general practice is not optimal. The IMPALA study intends to improve adherence to lifestyle advice by involving patients in decision making on cardiovascular prevention by nurse-led clinics. The aim of this paper is to describe the design and methods of a study to evaluate an intervention aimed at involving patients in cardiovascular risk management.

METHODS

A cluster-randomised controlled trial in 20 general practices, 10 practices in the intervention arm and 10 in the control arm, starting on October 2005. A total of 720 patients without existing cardiovascular diseases but eligible for cardiovascular risk assessment will be recruited. In both arms, the general practitioners and nurses will be trained to apply the national guideline for cardiovascular risk management. Nurses in the intervention arm will receive an extended training in risk assessment, risk communication, the use of a decision aid and adapted motivational interviewing. This communication technique will be used to support the shared decision-making process about risk reduction. The intervention comprises 2 consultations and 1 follow-up telephone call. The nurses in the control arm will give usual care after the risk estimation, according to the national guideline. Primary outcome measures are self-reported adherence to lifestyle advice and drug treatment. Secondary outcome measures are the patients' perception of risk and their motivation to change their behaviour. The measurements will take place at baseline and after 12 and 52 weeks. Clinical endpoints will not be measured, but the absolute 10-year risk of cardiovascular events will be estimated for each patient from medical records at baseline and after 1 year.

DISCUSSION

The combined use of risk communication, a decision aid and motivational interviewing to enhance patient involvement in decision making is an innovative aspect of the intervention.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN51556722.

Introduction of a breast cancer care programme including ultra short hospital stay in 4 early adopter centres: framework for an implementation study

Background

Whereas ultra-short stay (day care or 24 hour hospitalisation) following breast cancer surgery was introduced in the US and Canada in the 1990s, it is not yet common practice in Europe. This paper describes the design of the MaDO study, which involves the implementation of ultra short stay admission for patients after breast cancer surgery, and evaluates whether the targets of the implementation strategy are reached. The ultra short stay programme and the applied implementation strategy will be evaluated from the economic perspective.

Methods/design

The MaDO study is a pre-post-controlled multi-centre study, that is performed in four hospitals in the Netherlands. It includes a pre and post measuring period of six months each with six months of implementation in between in at least 40 patients per hospital per measurement period.

Primary outcome measure is the percentage of patients treated in ultra short stay. Secondary endpoints are the percentage of patients treated according to protocol, degree of involvement of home care nursing, quality of care from the patient's perspective, cost-effectiveness of the ultra short stay programme and cost-effectiveness of the implementation strategy. Quality of care will be measured by the QUOTE-breast cancer instrument, cost-effectiveness of the ultra short stay programme will be measured by means of the EuroQol (administered at four time-points) and a cost book for patients. Cost-effectiveness analysis will be performed from a societal perspective. Cost-effectiveness of the implementation strategy will be measured by determination of the costs of implementation activities.

Discussion

This study will reveal barriers and facilitators for implementation of the ultra short stay programme. Moreover, the results of the study will provide information about the cost-effectiveness of the ultra short stay programme and the implementation strategy.

Trial registration

Current Controlled Trials ISRCTN77253391.

The methodological quality of economic evaluations of guideline implementation into clinical practice: a systematic review of empiric studies

OBJECTIVES

Despite the emphasis on efficiency of health-care services delivery, there is an imperfect evidence base to inform decisions about whether and how to develop and implement guidelines into clinical practice. In general, studies evaluating the economics of guideline implementation lack methodological rigor. We conducted a systematic review of empiric studies to assess advances in the economic evaluations of guideline implementation.

METHODS

The Cochrane Effective Professional and Organisational Change Group specialized register and the MEDLINE database were searched for English publications between January 1998 and July 2004 that reported objective effect measures and implementation costs. We extracted data on study characteristics, quality of study design, and economic methodology. It was assessed whether the economic evaluations followed methodological guidance.

RESULTS

We included 24 economic evaluations, involving 21 controlled trials and three interrupted time series designs. The studies involved varying settings, targeted professionals, targeted behaviors, clinical guidelines, and implementation strategies. Overall, it was difficult to determine the quality of study designs owing to poor reporting. In addition, most economic evaluations were methodologically flawed: studies did not follow guidelines for evaluation design, data collection, and data analysis.

CONCLUSIONS

The increasing importance of the value for money of providing health care seems to be reflected by an increase in empiric economic evaluations of guideline implementation. Because of the heterogeneity and poor methodological quality of these studies, however, the resulting evidence is still of limited use in decision-making. There seems to be a need for more methodological guidance, especially in terms of data collection and data synthesis, to appropriately evaluate the economics of developing and implementing guidelines into clinical practice.

Effects of a behaviour independent financial incentive on prescribing behaviour of general practitioners

RATIONALE AND OBJECTIVES

It is difficult to keep control over prescribing behaviour in general practice. The purpose of this study was to assess the initial effects of a behaviour independent financial incentive on the volume of drug prescribing of general practitioners (GPs).

DESIGN

2-Year Controlled Before After study with an intervention region and a concurrent control region.

SETTING AND PARTICIPANTS

GPs in two regions in the Netherlands (n = 119 and n = 118).

INTERVENTION

A financial incentive for prescribing according to local guidelines on specific drugs or drug categories. The financial incentive consisted of a non-recurrent, behaviour-independent allowance.

MAIN OUTCOME MEASURE

Change in the number of prescriptions for 10 targeted drugs or drug groups.

RESULTS

Significant changes were seen only in three types of antibiotics and in recommended gastric medicines. In almost all cases, effects were temporary.

CONCLUSION

Behaviour independent financial incentives can be a help in changing prescription behaviour of GPs, but effects are small-scale and temporary.

Improving management of patients with acute cough by C-reactive protein point of care testing and communication training (IMPAC3T): study protocol of a cluster randomised controlled trial

Background

Most antibiotic prescriptions for acute cough due to lower respiratory tract infections (LRTI) in primary care are not warranted. Diagnostic uncertainty and patient expectations and worries are major drivers of unnecessary antibiotic prescribing. A C-reactive protein (CRP) point of care test may help GPs to better guide antibiotic treatment by ruling out pneumonia in cases of low test results. Alternatively, enhanced communication skills training to help clinicians address patients' expectations and worries could lead to a decrease in antibiotic prescribing, without compromising clinical recovery, while enhancing patient enablement. The aim of this paper is to describe the design and methods of a study to assess two interventions for improving LRTI management in general practice.

Methods/Design

This cluster randomised controlled, factorial trial will introduce two interventions in general practice; point of care CRP testing and enhanced communication skills training for LRTI. Twenty general practices with two participating GPs per practice will recruit 400 patients with LRTI during two winter periods. Patients will be followed up for at least 28 days. The primary outcome measure is the antibiotic prescribing rate. Secondary outcomes are clinical recovery, cost-effectiveness, use of other diagnostic tests and medical services (including reconsultation), and patient enablement.

Discussion

This trial is the first cluster randomised trial to evaluate the influence of point of care CRP testing in the hands of the general practitioner and enhanced communication skills, on the management of LRTI in primary care. The pragmatic nature of the study, which leaves treatment decisions up to the responsible clinicians, will enhance the applicability and generalisability of findings. The factorial design will allow conclusion to be made about the value of CRP testing on its own, communication skills training on its own, and the two combined. Evaluating a biomedical and communication based intervention ('hard' and 'soft' technologies) together in this way makes this trial unique in its field.

The VHA New England Medication Error Prevention Initiative as a Model for Long-Term Improvement Collaboratives

BACKGROUND

Quality improvement collaboratives (QICs) are a widely applied strategy for implementing change in health care organizations. Alternative collaborative methodologies were compared to gain insight into the elements important for QIC success.

METHODS

A modified version of a previously described QIC evaluation tool was used to assess the methods and characteristics of the Medication Error Prevention Initiative (MEPI) and to compare MEPI with two other long-term ongoing QICs--the Vermont-Oxford Network's Neonatal Intensive Care QIC and the Northern New England Cardiovascular Disease Study Group, and the shorter-term Breakthrough Series QICs of the Institute for Healthcare Improvement (IHI).

RESULTS

The modified QIC assessment tool was a useful framework for QIC assessment and comparison. The MEPI differed in scope of topic, team members, and the method for learning about and making improvements.

CONCLUSIONS

Long-term QIC methods such as those used by MEPI may be particularly applicable when QICs address broad, complex, comprehensive, or organizationwide improvement needs.

Does a joint development and dissemination of multidisciplinary guidelines improve prescribing behaviour: a pre/post study with concurrent control group and a randomised trial

BACKGROUND

It is difficult to keep control over prescribing behaviour in general practices. The purpose of this study was to assess the effects of a dissemination strategy of multidisciplinary guidelines on the volume of drug prescribing.

METHODS

The study included two designs, a quasi-experimental pre/post study with concurrent control group and a random sample of GPs within the intervention group. The intervention area with 53 GPs was compared with a control group of 54 randomly selected GPs in the south and centre of the Netherlands. Additionally, a randomisation was executed in the intervention group to create two arms with 27 GPs who were more intensively involved in the development of the guideline and 26 GPs in the control group. A multidisciplinary committee developed prescription guidelines. Subsequently these guidelines were disseminated to all GPs in the intervention region. Additional effects were studied in the subgroup trial in which GPs were invited to be more intensively involved in the guideline development procedure. The guidelines contained 14 recommendations on antibiotics, asthma/COPD drugs and cholesterol drugs. The main outcome measures were prescription data of a three-year period (one year before and 2 years after guideline dissemination) and proportion of change according to recommendations.

RESULTS

Significant short-term improvements were seen for one recommendation: mupirocin. Long-term changes were found for cholesterol drug prescriptions. No additional changes were seen for the randomised controlled study in the subgroup. GPs did not take up the invitation for involvement.

CONCLUSION

Disseminating multidisciplinary guidelines that were developed within a region, has no clear effect on prescribing behaviour even though GPs and specialists were involved more intensively in their development. Apparently, more effort is needed to bring about change.

Patient‐centred quality improvement audit

PURPOSE

This article aims to describe the research process, and the development of the instrument now employed in auditing patients' perceptions of quality improvement in a community health care trust in a coastal town in Essex, England.

DESIGN/METHODOLOGY/APPROACH

The new instrument is currently being implemented and the findings thus far are described.

FINDINGS

The instrument has measured health outcomes in terms of quality improvement from the users' perspective, and has also highlighted gaps between what the service offers in terms of quality and users' perceptions of what is delivered. The study demonstrates the importance of the professional role in quality improvement.

ORIGINALITY/VALUE

Patient-centred quality improvement audit should be undertaken regularly so that both non-clinical managers and health care professionals can establish whether or not they are providing services that are patient-friendly and effective from the user's viewpoint. In the course of their work, professionals and managers discuss patients and speak on their behalf in various forums, and knowing what patients actually expect and perceive before speaking on their behalf may be of great benefit in such instances.

Achieving better in-hospital and after-hospital care of patients with acute cardiac disease

In patients hospitalised with acute coronary syndromes (ACS) and congestive heart failure (CHF), evidence suggests opportunities for improving in-hospital and after-hospital care, patient self-care, and hospital-community integration. A multidisciplinary quality improvement program was designed and instigated in Brisbane in October 2000 involving 250 clinicians at three teaching hospitals, 1080 general practitioners (GPs) from five Divisions of General Practice, 1594 patients with ACS and 904 patients with CHF. Quality improvement interventions were implemented over 17 months after a 6-month baseline period and included: clinical decision support (clinical practice guidelines, reminders, checklists, clinical pathways); educational interventions (seminars, academic detailing); regular performance feedback; patient self-management strategies; and hospital-community integration (discharge referral summaries; community pharmacist liaison; patient prompts to attend GPs). Using a before-after study design to assess program impact, significantly more program patients compared with historical controls received: ACS: Angiotensin-converting enzyme (ACE) inhibitors and lipid-lowering agents at discharge, aspirin and beta-blockers at 3 months after discharge, inpatient cardiac counselling, and referral to outpatient cardiac rehabilitation. CHF: Assessment for reversible precipitants, use of prophylaxis for deep-venous thrombosis, beta-blockers at discharge, ACE inhibitors at 6 months after discharge, imaging of left ventricular function, and optimal management of blood pressure levels. Risk-adjusted mortality rates at 6 and 12 months decreased, respectively, from 9.8% to 7.4% (P = 0.06) and from 13.4% to 10.1% (P = 0.06) for patients with ACS and from 22.8% to 15.2% (P < 0.001) and from 32.8% to 22.4% (P = 0.005) for patients with CHF. Quality improvement programs that feature multifaceted interventions across the continuum of care can change clinical culture, optimise care and improve clinical outcomes.