Should pay-for-performance schemes be locally designed? Evidence from the Commissioning for Quality and Innovation (CQUIN) Framework

Establishing the content validity of a new emergency department patient-reported experience measure (ED PREM): a Delphi study

Background

Patient-reported experience measures aim to capture the patient’s perspective of what happened during a care encounter and how it happened. However, due to a lack of guidance to support patient-reported experience measure development and reporting, the content validity of many instruments is unclear and ambiguous. Thus, the aim of this study was to establish the content validity of a newly developed Emergency Department Patient-Reported Experience Measure (ED PREM).

Methods

ED PREM items were developed based on the findings of a systematic mixed studies review, and qualitative interviews with Emergency Department patients that occurred during September and October, 2020. Individuals who participated in the qualitative interviews were approached again during August 2021 to participate in the ED PREM content validation study. The preliminary ED PREM comprised 37 items. A two-round modified, online Delphi study was undertaken where patient participants were asked to rate the clarity, relevance, and importance of ED PREM items on a 4-point content validity index scale. Each round lasted for two-weeks, with 1 week in between for analysis. Consensus was a priori defined as item-level content validity index scores of ≥0.80. A scale-level content validity index score was also calculated.

Results

Fifteen patients participated in both rounds of the online Delphi study. At the completion of the study, two items were dropped and 13 were revised, resulting in a 35-item ED PREM. The scale-level content validity index score for the final 35-item instrument was 0.95.

Conclusions

The newly developed ED PREM demonstrates good content validity and aligns strongly with the concept of Emergency Department patient experience as described in the literature. The ED PREM will next be administered in a larger study to establish its’ construct validity and reliability. There is an imperative for clear guidance on PREM content validation methodologies. Thus, this study may inform the efforts of other researchers undertaking PREM content validation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12873-022-00617-5.

Pay for performance system in Turkey and the world; a global overview

Objective: This study aimed to compare the pay for performance system applied nationally in Turkey and in other countries around the world and to reveal the effects of the system applied in our country on the general surgery.

Material and Methods: Current literature and countries’ programs on the implementation of the pay for performance system were recorded. The results of the Turkish Surgical Association’s performance and Healthcare Implementation Communique (HIC) commission studies were evaluated in light of the literature.

Results: Many countries have implemented performance systems on a limited scale to improve quality, speed up the diagnosis, treatment, and control of certain diseases, and they have generally applied it as a financial promotion by receiving the support of health insurance companies and nongovernmental organizations. It turns out that surgeons in our country feel that they are being wronged because of the injustice in the current system because the property of their works is not appreciated and they cannot get the reward for the work they do. This is also the reason for the reluctance of medical school graduates to choose general surgery.

Conclusion: Authorities should pay attention to the opinions of associations and experts in the related field when creating lists of interventional procedures related to surgery. Equal pay should be given to equal work nationally, and surgeons should be encouraged by incentives to perform detailed, qualified surgeries. There is a possibility that the staff positions opened for general surgery, as well as, all surgical branches will remain empty in the near future.

The effects of structure, process and outcome incentives on primary care referrals to a national prevention programme

Despite widespread use, evidence is sparse on whether financial incentives in healthcare should be linked to structure, process or outcome. We examine the impact of different incentive types on the quantity and effectiveness of referrals made by general practices to a new national prevention programme in England. We measured effectiveness by the number of referrals resulting in programme attendance. We surveyed local commissioners about their use of financial incentives and linked this information to numbers of programme referrals and attendances from 5170 general practices between April 2016 and March 2018. We used multivariate probit regressions to identify commissioner characteristics associated with the use of different incentive types and negative binomial regressions to estimate their effect on practice rates of referral and attendance. Financial incentives were offered by commissioners in the majority of areas (89%), with 38% using structure incentives, 69% using process incentives and 22% using outcome incentives. Compared to practices without financial incentives, neither structure nor process incentives were associated with statistically significant increases in referrals or attendances, but outcome incentives were associated with 84% more referrals and 93% more attendances. Outcome incentives were the only form of pay-for-performance to stimulate more participation in this national disease prevention programme.

Optimal hospital payment rules under rationing by waiting

We derive optimal rules for paying hospitals for non-emergency care when providers choose quality and capacity, and patient demand is rationed by waiting time. Waiting for treatment is costly for patients, so that hospital payment rules should take account of their effect on waiting time as well as on quality. Since deterministic waiting time models imply that profit maximising hospitals will never choose to have both positive quality and positive waiting time, we develop a stochastic model of rationing by waiting in which both quality and expected waiting are positive in equilibrium. We use it to show that, although a prospective output price gives hospitals an incentive to attract patients by raising quality and reducing waiting times, it must be supplemented by a price attached to hospital decisions on quality or capacity or to a performance indicator which depends on those decisions (such as average waiting time, or average length of stay). A prospective output price by itself can support the optimal quality and waiting time distribution only if the welfare function respects patient preferences over quality and waiting time, if patients' marginal rates of substitution between quality and waiting time are independent of income, and if waiting for treatment does not reduce the productivity of patients. If these conditions do not hold, supplementing the output price with a reward linked to the hospital's cost can increase welfare, though it is possible that costs should be taxed rather than subsidised.

A comparison of methods for health policy evaluation with controlled pre-post designs

Objective

To compare interactive fixed effects (IFE) and generalized synthetic control (GSC) methods to methods prevalent in health policy evaluation and re‐evaluate the impact of the hip fracture best practice tariffs introduced for hospitals in England in 2010.

Data Sources

Simulations and Hospital Episode Statistics.

Study Design

Best practice tariffs aimed to incentivize providers to deliver care in line with guidelines. Under the scheme, 62 providers received an additional payment for each hip fracture admission, while 49 providers did not. We estimate the impact using difference‐in‐differences (DiD), synthetic control (SC), IFE, and GSC methods. We contrast the estimation methods' performance in a Monte Carlo simulation study.

Principal Findings

Unlike DiD, SC, and IFE methods, the GSC method provided reliable estimates across a range of simulation scenarios and was preferred for this case study. The introduction of best practice tariffs led to a 5.9 (confidence interval: 2.0 to 9.9) percentage point increase in the proportion of patients having surgery within 48 hours and a statistically insignificant 0.6 (confidence interval: −1.4 to 0.4) percentage point reduction in 30‐day mortality.

Conclusions

The GSC approach is an attractive method for health policy evaluation. We cannot be confident that best practice tariffs were effective.

Pay for performance for specialised care in England: Strengths and weaknesses

Pay-for-Performance (P4P) schemes have become increasingly common internationally, yet evidence of their effectiveness remains ambiguous. P4P has been widely used in England for over a decade both in primary and secondary care. A prominent P4P programme in secondary care is the Commissioning for Quality and Innovation (CQUIN) framework. The most recent addition to this framework is Prescribed Specialised Services (PSS) CQUIN, introduced into the NHS in England in 2013. This study offers a review and critique of the PSS CQUIN scheme for specialised care. A key feature of PSS CQUIN is that whilst it is centrally developed, performance targets are agreed locally. This means that there is variation across providers in the schemes selected from the national menu, the achievement level needed to earn payment, and the proportion of the overall payment attached to each scheme. Specific schemes vary in terms of what is incentivised - structure, process and/or outcome - and how they are incentivised. Centralised versus decentralised decision making, the nature of the performance measures, the tiered payment structure and the dynamic nature of the schemes have created a sophisticated but complex P4P programme which requires evaluation to understand the effect of such incentives on specialised care.

Design and effects of outcome-based payment models in healthcare: a systematic review

INTRODUCTION

Outcome-based payment models (OBPMs) might solve the shortcomings of fee-for-service or diagnostic-related group (DRG) models using financial incentives based on outcome indicators of the provided care. This review provides an analysis of the characteristics and effectiveness of OBPMs, to determine which models lead to favourable effects.

METHODS

We first developed a definition for OBPMs. Next, we searched four data sources to identify the models: (1) scientific literature databases; (2) websites of relevant governmental and scientific agencies; (3) the reference lists of included articles; (4) experts in the field. We only selected studies that examined the impact of the payment model on quality and/or costs. A narrative evidence synthesis was used to link specific design features to effects on quality of care or healthcare costs.

RESULTS

We included 88 articles, describing 12 OBPMs. We identified two groups of models based on differences in design features: narrow OBPMs (financial incentives based on quality indicators) and broad OBPMs (combination of global budgets, risk sharing, and financial incentives based on quality indicators). Most (5 out of 9) of the narrow OBPMs showed positive effects on quality; the others had mixed (2) or negative (2) effects. The effects of narrow OBPMs on healthcare utilization or costs, however, were unfavourable (3) or unknown (6). All broad OBPMs (3) showed positive effects on quality of care, while reducing healthcare cost growth.

DISCUSSION

Although strong empirical evidence on the effects of OBPMs on healthcare quality, utilization, and costs is limited, our findings suggest that broad OBPMs may be preferred over narrow OBPMs.

Brazilian Payment for Performance (PMAQ) Seen From a Global Health and Public Policy Perspective: What Does It Mean for Research and Policy?

This supplement of the Journal of Ambulatory Care Management on the Brazilian National Program for Improving Access and Quality of Primary Care (PMAQ) reveals a relevant gap in the Brazilian literature on pay for performance/PMAQ, and is therefore an opportunity to bring contributions from global health and public policy to the debate. We discuss the relevant gap in the light of developments in evaluation and policy analysis. We afterward present the state of knowledge regarding global health and public policy in pay for performance, giving attention to diverse themes, methods, types of analyses, theoretical contributions, and limitations. Finally, we suggest some possible implications for research and policy in Brazil.

DIAMOND (DIgital Alcohol Management ON Demand): a feasibility RCT and embedded process evaluation of a digital health intervention to reduce hazardous and harmful alcohol use recruiting in hospital emergency departments and online

BACKGROUND

The harmful use of alcohol is a causal factor in more than 200 disease and injury conditions and leads to over 3 million deaths every year worldwide. Relatively few problem alcohol users access treatment due to stigma and lack of services. Alcohol-specific digital health interventions (DHI) may help them, but trial data comparing DHI with face-to-face treatment are lacking.

METHODS

We conducted a feasibility RCT of an alcohol DHI, testing recruitment, online data-collection and randomisation processes, with an embedded process evaluation. Recruitment ran from October 2015 for 12 months. Participants were adults, drinking at hazardous and harmful levels, recruited from hospital emergency departments (ED) in London or recruited online. Participants were randomised to HeLP-Alcohol, a six module DHI with weekly reminder prompts (phone, email or text message), or to face-to-face treatment as usual (TAU). Participants were invited to take part in qualitative interviews after the trial.

RESULTS

The trial website was accessed 1074 times: 420 people completed online eligibility questionnaires; 350 did not meet eligibility criteria, 51 declined to participate, and 19 were recruited and randomised. Follow-up data were collected from three participants (retention 3/19), and four agreed to be interviewed for the process evaluation. The main themes of the interviews were:Participants were not at equipoise. They wanted to try the website and were disappointed to be randomised to face-to-face, so they were less engaged and dropped out.Other reasons for drop out included not accepting that they had a drink problem; problem drinking interfering with their ability to take part in a trial or forgetting appointments; having a busy life and being randomised to TAU made it difficult to attend appointments.

CONCLUSIONS

This feasibility RCT aimed to test recruitment, randomisation, retention, and data collection methods, but recruited only 19 participants. This illustrates the importance of undertaking feasibility studies prior to fully powered RCTs. From the qualitative interviews we found that potential recruits were not at equipoise for recruitment. An alternative methodology, for example a preference RCT recruiting from multiple locations, needs to be explored in future trials.

TRIAL REGISTRATION

International Standard Randomized Controlled Trial Number: ISRCTN31789096.

Autonomy to health care professionals as a vehicle for value-based health care? Results of a quasi-experiment in hospital governance

Health care systems increasingly aim to create value for money by simultaneous incentivizing of quality along with classical goals such as activity increase and cost containment. It has recently been suggested that letting health care professionals choose the performance metrics on which they are evaluated may improve value of care by facilitating greater employee initiative, especially in the quality domain. There is a risk that this strategy leads to loss of performance as measured by the classical goals, if these goals are not prioritized by health care professionals. In this study we investigate the performance of eight hospital departments in the second largest region of Denmark that were delegated the authority to choose their own performance focus during a three-year test period from 2013 to 2016. The usual activity-based remuneration was suspended and departments were instructed to keep their global budgets and maintain activity levels, while managing according to their newly chosen performance focuses. Our analysis is based on monthly observations from two years before to three years after delegation. We collected data for 32 new performance indicators chosen by hospital department managements; 11 new performance indicators chosen by a centre management under which 5 of the departments were organised; and 3 classical indicators of priority to the central administration (activity, productivity, and cost containment). Interrupted time series analysis is used to estimate the effect of delegation on these indicators. We find no evidence that this particular proposal for giving health care professionals greater autonomy leads to consistent quality improvements but, on the other hand, also no consistent evidence of harm to the classical goals. Future studies could consider alternative possibilities to create greater autonomy for hospital departments.

Quality target negotiation in health care: evidence from the English NHS

We examine how public sector third-party purchasers and hospitals negotiate quality targets when a fixed proportion of hospital revenue is required to be linked to quality. We develop a bargaining model linking the number of quality targets to purchaser and hospital characteristics. Using data extracted from 153 contracts for acute hospital services in England in 2010/2011, we find that the number of quality targets is associated with the purchaser's population health and its budget, the hospital type, whether the purchaser delegated negotiation to an agency, and the quality targets imposed by the supervising regional health authority.

Estimating causal effects: considering three alternatives to difference-in-differences estimation

Difference-in-differences (DiD) estimators provide unbiased treatment effect estimates when, in the absence of treatment, the average outcomes for the treated and control groups would have followed parallel trends over time. This assumption is implausible in many settings. An alternative assumption is that the potential outcomes are independent of treatment status, conditional on past outcomes. This paper considers three methods that share this assumption: the synthetic control method, a lagged dependent variable (LDV) regression approach, and matching on past outcomes. Our motivating empirical study is an evaluation of a hospital pay-for-performance scheme in England, the best practice tariffs programme. The conclusions of the original DiD analysis are sensitive to the choice of approach. We conduct a Monte Carlo simulation study that investigates these methods’ performance. While DiD produces unbiased estimates when the parallel trends assumption holds, the alternative approaches provide less biased estimates of treatment effects when it is violated. In these cases, the LDV approach produces the most efficient and least biased estimates.

Lessons learnt from a primary care asthma improvement project

Asthma is a very common disease that can occur at any age. In the UK and in many other countries it is mainly managed in primary care. The published evidence suggests that the key to improving diagnosis and management lies in better training and education rather than in the discovery of new medications. An asthma improvement project managed through the British Lung Foundation is attempting to do this. The project has three pilot sites: two in England supported by the Department of Health and one in Scotland supported by the Scottish Government. If the project is successful it will be rolled out to other health areas within the UK. The results of this project are not yet available. This article highlights the challenges encountered in setting up the project and may well be applicable to other areas in the UK and to other countries where similar healthcare systems exist. The encountered challenges reflect the complex nature of healthcare systems and electronic data capture in primary care. We discuss the differences between general practices in their ability and willingness to support the project, the training and education of their staff on asthma management, governance issues in relation to information technology systems, and the quality of data capture. Virtually all the challenges have now been overcome, but discussing them should ensure that others become aware of them at an early stage should they wish to undertake similar projects in the future.

Using Patient-Reported Information to Improve Clinical Practice

OBJECTIVE

To assess what is known about the relationship between patient experience measures and incentives designed to improve care, and to identify how public policy and medical practices can promote patient-valued outcomes in health systems with strong financial incentives.

DATA SOURCES/STUDY SETTING

Existing literature (gray and peer-reviewed) on measuring patient experience and patient-reported outcomes, identified from Medline and Cochrane databases; evaluations of pay-for-performance programs in the United States, Europe, and the Commonwealth countries.

STUDY DESIGN/DATA COLLECTION

We analyzed (1) studies of pay-for-performance, to identify those including metrics for patient experience, and (2) studies of patient experience and of patient-reported outcomes to identify evidence of influence on clinical practice, whether through public reporting or private reporting to clinicians.

PRINCIPAL FINDINGS

First, we identify four forms of "patient-reported information" (PRI), each with distinctive roles shaping clinical practice: (1) patient-reported outcomes measuring self-assessed physical and mental well-being, (2) surveys of patient experience with clinicians and staff, (3) narrative accounts describing encounters with clinicians in patients' own words, and (4) complaints/grievances signaling patients' distress when treatment or outcomes fall short of expectations. Because these forms vary in crucial ways, each must be distinctively measured, deployed, and linked with financial incentives. Second, although the literature linking incentives to patients experience is limited, implementing pay-for-performance systems appears to threaten certain patient-valued aspects of health care. But incentives can be made compatible with the outcomes patients value if: (a) a sufficient portion of incentives is tied to patient-reported outcomes and experiences, (b) incentivized forms of PRI are complemented by other forms of patient feedback, and (c) health care organizations assist clinicians to interpret and respond to PRI. Finally, we identify roles for the public and private sectors in financing PRI and orchestrating an appropriate balance among its four forms.

CONCLUSIONS

Unless public policies are attentive to patients' perspectives, stronger financial incentives for clinicians can threaten aspects of care that patients most value. Certain policy parameters are already clear, but additional research is required to clarify how best to collect patient narratives in varied settings, how to report narratives to consumers in conjunction with quantified metrics, and how to promote a "culture of learning" at the practice level that incorporates patient feedback.

Screening for the metabolic side effects of antipsychotic medication: findings of a 6-year quality improvement programme in the UK

OBJECTIVES

To increase the frequency and quality of screening for the metabolic syndrome in people prescribed continuing antipsychotic medication.

DESIGN

An audit-based, quality improvement programme (QIP) with customised feedback to participating mental health services after each audit, including benchmarked data on their relative and absolute performance against an evidence-based practice standard and the provision of bespoke change interventions.

SETTING

Adult, assertive outreach, community psychiatric services in the UK.

PARTICIPANTS

6 audits were conducted between 2006 and 2012. 21 mental health Trusts participated in the baseline audit in 2006, submitting data on screening for 1966 patients, while 32 Trusts participated in the 2012 audit, submitting data on 1591 patients.

RESULTS

Over the 6 years of the programme, there was a statistically significant increase in the proportion of patients for whom measures for all 4 aspects of the metabolic syndrome had been documented in the clinical records in the previous year, from just over 1 in 10 patients in 2006 to just over 1 in 3 by 2012. The proportion of patients with no evidence of any screening fell from almost ½ to 1 in 7 patients over the same period.

CONCLUSIONS

The findings suggest that audit-based QIPs can help improve clinical practice in relation to physical healthcare screening. Nevertheless, they also reveal that only a minority of community psychiatric patients prescribed antipsychotic medication is screened for the metabolic syndrome in accordance with best practice recommendations, and therefore potentially remediable causes of poor physical health remain undetected and untreated.

Mandatory implementation of NICE Guidelines for the care of bipolar disorder and other conditions in England and Wales

BACKGROUND

Bipolar disorder is a common long-term mental health condition characterised by episodes of mania or hypomania and depression resulting in disability, early death, and high health and society costs. Public money funds the National Institute of Healthcare and Clinical Excellence (NICE) to produce clinical guidelines by systematically identifying the most up to date research evidence and costing its main recommendations for healthcare organisations and professionals to follow in England and Wales. Most governments, including those of England and Wales, need to improve healthcare but at reduced cost. There is evidence, particularly in bipolar disorder, that systematically following clinical guidelines achieves these outcomes.

DISCUSSION

NICE clinical guidelines, including those regarding bipolar disorder, remain variably implemented. They give clinicians and patients a non-prescriptive basis for deciding their care. Despite the passing of the Health and Social Care Act in 2012 in England requiring all healthcare organisations to consider NICE clinical guidelines in commissioning, delivering, and inspecting healthcare services, healthcare organisations in the National Health Service may ignore them with little accountability and few consequences. There is no mechanism to ensure that healthcare professionals know or consider them. Barriers to their implementation include the lack of political and professional leadership, the complexity of the organisation of care and policy, mistrust of some processes and recommendations of clinical guidelines, and a lack of a clear implementation model, strategy, responsibility, or accountability. Mitigation to these barriers is presented herein.

SUMMARY

The variability, safety, and quality of healthcare might be improved and its cost reduced if the implementation of NICE clinical guidelines, such as those for bipolar disorder, were made the minimum starting point for clinical decision-making and mandatory responsibilities of all healthcare organisations and professionals.

What are the Costs and Benefits of Providing Comprehensive Seven-day Services for Emergency Hospital Admissions?

The English National Health Service is moving towards providing comprehensive 7-day hospital services in response to higher death rates for emergency weekend admissions. Using Hospital Episode Statistics between 1st April 2010 and 31st March 2011 linked to all-cause mortality within 30 days of admission, we estimate the number of excess deaths and the loss in quality-adjusted life years associated with emergency weekend admissions. The crude 30-day mortality rate was 3.70% for weekday admissions and 4.05% for weekend admissions. The excess weekend death rate equates to 4355 (risk adjusted 5353) additional deaths each year. The health gain of avoiding these deaths would be 29 727-36 539 quality-adjusted life years per year. The estimated cost of implementing 7-day services is £1.07-£1.43 bn, which exceeds by £339-£831 m the maximum spend based on the National Institute for Health and Care Excellence threshold of £595 m-£731 m. There is as yet no clear evidence that 7-day services will reduce weekend deaths or can be achieved without increasing weekday deaths. The planned cost of implementing 7-day services greatly exceeds the maximum amount that the National Health Service should spend on eradicating the weekend effect based on current evidence. Policy makers and service providers should focus on identifying specific service extensions for which cost-effectiveness can be demonstrated.

A qualitative and quantitative evaluation of the Advancing Quality pay-for-performance programme in the NHS North West

Background

Advancing Quality (AQ) is a voluntary programme providing financial incentives for improvement in the quality of care provided to NHS patients in the north-west of England.

Objectives

(1) To identify the impact of AQ on key stakeholders and clinical practice; (2) to assess its cost-effectiveness; (3) to identify key factors that assist or impede its successful implementation; and (4) to provide lessons for the wider implementation of pay-for-performance schemes across the NHS.

Design

We tested whether or not the financial incentives of AQ had an impact on mortality using two methods: a between-region difference-in-differences analysis comparing the North West region and the rest of England for the incentivised and non-incentivised conditions and a triple-difference analysis comparing performance on the incentivised conditions, as well as the non-incentivised conditions, in the North West region and the rest of England. A cost-effectiveness analysis of AQ based on the first 18 months of the programme was also undertaken. We used interviews and observation to explore how and why changes occurred.

Results

Risk-adjusted mortality rates for all three of the conditions we studied (pneumonia, heart failure and myocardial infarction) decreased in both the North West region and the rest of England during the first 18 months of the scheme. The reduction in mortality for incentivised conditions was greater in the North West region than in the rest of England. Compared with non-incentivised conditions within the North West region, there was a significant reduction in overall mortality for incentivised conditions, comprising a statistically significant reduction in pneumonia and non-significant reductions in the other two conditions. Comparing mortality for the incentivised conditions with mortality for these conditions in other regions, there was a significant reduction in overall mortality in the North West region, again made up of individually significant reductions in pneumonia and non-significant reductions in the other two conditions. The reduction in mortality over the 18-month period studied for non-incentivised conditions was not significantly different between the North West region and the rest of England. The between-region difference-in-differences analysis after 42 months showed that risk-adjusted mortality for the incentivised conditions fell in the rest of England and the North West region. This reduction in the rest of England was significantly larger than in the North West region and was concentrated in pneumonia. However, the reductions in mortality were larger for the non-incentivised conditions in the North West region than in the rest of England between these periods. For incentivised conditions, the triple-difference analysis shows a larger reduction in mortality for the rest of England than in the North West region between the short- and long-term periods.

Conclusions

Based on the first 18 months, the AQ programme was a relatively effective and cost-effective intervention. However, findings at 42 months are open to interpretation. One interpretation is that the short-term improvements were not sustained and that the observed improvements in mortality in the non-incentivised conditions within hospitals participating in AQ were unrelated to the programme. An alternative interpretation is that these improvements are related to the positive spillover effect of AQ. Further research should be undertaken to determine the explanation for the findings.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

'Just another incentive scheme': a qualitative interview study of a local pay-for-performance scheme for primary care

BACKGROUND

A range of policy initiatives have addressed inequalities in healthcare and health outcomes. Local pay-for-performance schemes for primary care have been advocated as means of enhancing clinical ownership of the quality agenda and better targeting local need compared with national schemes such as the UK Quality and Outcomes Framework (QOF). We investigated whether professionals' experience of a local scheme in one English National Health Service (NHS) former primary care trust (PCT) differed from that of the national QOF in relation to the goal of reducing inequalities.

METHODS

We conducted retrospective semi-structured interviews with primary care professionals implementing the scheme and those involved in its development. We purposively sampled practices with varying levels of population socio-economic deprivation and achievement. Interviews explored perceptions of the scheme and indicators, likely mechanisms of influence on practice, perceived benefits and harms, and how future schemes could be improved. We used a framework approach to analysis.

RESULTS

Thirty-eight professionals from 16 general practices and six professionals involved in developing local indicators participated. Our findings cover four themes: ownership, credibility of the indicators, influences on behaviour, and exacerbated tensions. We found little evidence that the scheme engendered any distinctive sense of ownership or experiences different from the national scheme. Although the indicators and their evidence base were seldom actively questioned, doubts were expressed about their focus on health promotion given that eventual benefits relied upon patient action and availability of local resources. Whilst practices serving more affluent populations reported status and patient benefit as motivators for participating in the scheme, those serving more deprived populations highlighted financial reward. The scheme exacerbated tensions between patient and professional consultation agendas, general practitioners benefitting directly from incentives and nurses who did much of the work, and practices serving more and less affluent populations which faced different challenges in achieving targets.

CONCLUSIONS

The contentious nature of pay-for-performance was not necessarily reduced by local adaptation. Those developing future schemes should consider differential rewards and supportive resources for practices serving more deprived populations, and employing a wider range of levers to promote professional understanding and ownership of indicators.

Evaluating the Labour Government's English NHS health system reforms: the 2008 Darzi reforms

Starting in 2002, the UK Labour Government of 1997-2010 introduced a series of changes to the National Health Service (NHS) in England designed to increase patients' choices of the place of elective hospital care and encourage competition among public and private providers of elective hospital services for NHS-funded patients. In 2006, the Department of Health initiated the Health Reform Evaluation Programme (HREP) to assess the impact of the changes. In June 2008, the White Paper, High quality care for all, was published. It represented the government's desire to focus the next phase of health care system reform in England as much on the quality of care as on improving its responsiveness and efficiency. The 2008 White Paper led to the commissioning of a further wave of evaluative research under the auspices of HREP, as follows: an evaluation of the implementation and outcomes of care planning for people with long-term conditions; an evaluation of the personal health budget pilots; an evaluation of the implementation and outcomes of the Commissioning for Quality and Innovation (CQUIN) framework; and an evaluation of cultural and behavioural change in the NHS focused on ensuring high quality care for all. This Supplement includes papers from each project. The evaluations present a mixed picture of the impact and success of the 2008 reforms. All the studies identify some limitations of the policies in the White Paper. The introduction of personal health budgets appears to have been the least problematic and, depending on assumptions, likely to be cost-effective for the sorts of patients involved in the pilot. For the rest of the changes, impacts ranged from little or none (CQUIN and care planning for people with chronic conditions) to patchy and highly variable (instilling a culture of quality in acute hospitals) in the three years following the publication of the White Paper. On the other hand, each of the studies identifies important insights relevant to modifying and improving the policies. These findings have continuing relevance since both the 2008 White Paper's policies, and the issues they were focused on remedying, remain central to the current Coalition Government's reform agenda.

Effect of stratified care for low back pain in family practice (IMPaCT Back): a prospective population-based sequential comparison

PURPOSE

We aimed to determine the effects of implementing risk-stratified care for low back pain in family practice on physician's clinical behavior, patient outcomes, and costs.

METHODS

The IMPaCT Back Study (IMplementation to improve Patient Care through Targeted treatment) prospectively compared separate patient cohorts in a preintervention phase (6 months of usual care) and a postintervention phase (12 months of stratified care) in family practice, involving 64 family physicians and linked physical therapy services. A total of 1,647 adults with low back pain were invited to participate. Stratified care entailed use of a risk stratification tool to classify patients into groups at low, medium, or high risk for persistent disability and provision of risk-matched treatment. The primary outcome was 6-month change in disability as assessed with the Roland-Morris Disability Questionnaire. Process outcomes captured physician behavior change in risk-appropriate referral to physical therapy, diagnostic tests, medication prescriptions, and sickness certifications. A cost-utility analysis estimated incremental quality-adjusted life-years and back-related health care costs. Analysis was by intention to treat.

RESULTS

The 922 patients studied (368 in the preintervention phase and 554 in the postintervention phase) had comparable baseline characteristics. At 6 months follow-up, stratified care had a small but significant benefit relative to usual care as seen from a mean difference in Roland-Morris Disability Questionnaire scores of 0.7 (95% CI, 0.1-1.4), with a large, clinically important difference in the high risk group of 2.3 (95% CI, 0.8-3.9). Mean time off work was 50% shorter (4 vs 8 days, P = .03) and the proportion of patients given sickness certifications was 30% lower (9% vs 15%, P = .03) in the postintervention cohort. Health care cost savings were also observed.

CONCLUSIONS

Stratified care for back pain implemented in family practice leads to significant improvements in patient disability outcomes and a halving in time off work, without increasing health care costs. Wider implementation is recommended.

Paying for performance in healthcare organisations

Aligning Financial Incentives (FIs) to health policy goals is becoming increasingly popular. In many cases, such initiatives have failed to deliver anticipated benefits. Attributing this to the actions of self-interested and resistant professionals is not an entirely helpful approach. It is important to avoid simplistic assumptions to build knowledge of how and why schemes are implemented in practice to inform future policy in this area.