Design, rationale, and baseline characteristics of a cluster randomized controlled trial of pay for performance for hypertension treatment: study protocol

Improving the quality of quality metrics in neonatology

This article examines approaches for improving the efficiency and effectiveness of quality metrics currently in use in neonatal care. Desirable characteristics of quality metrics are discussed, the criteria and process for their development are presented, and the uses and limitations of current neonatal outcome and process metrics are explored together with approaches for improving metric performance. Discussion includes enhancing quality metrics through optimizing improvement readiness, sustaining improvements once achieved, and use of improvement science methods to improve metric validity.

Reports of unintended consequences of financial incentives to improve management of hypertension

Background

Given the increase in financial-incentive programs nationwide, many physicians and physician groups are concerned about potential unintended consequences of providing financial incentives to improve quality of care. However, few studies examine whether actual unintended consequences result from providing financial incentives to physicians. We sought to document the extent to which the unintended consequences discussed in the literature were observable in a randomized clinical trial (RCT) of financial incentives.

Methods

We conducted a qualitative observational study nested within a larger RCT of financial incentives to improve hypertension care. We conducted 30-minute telephone interviews with primary care personnel at facilities participating in the RCT housed at12 geographically dispersed Veterans Affairs Medical Centers nationwide. Participants answered questions about unintended effects, clinic team dynamics, organizational impact on care delivery, study participation. We employed a blend of inductive and deductive qualitative techniques for analysis.

Participants

Sixty-five participants were recruited from RCT enrollees and personnel not enrolled in the larger RCT, plus one primary care leader per site.

Results

Emergent themes included possible patient harm, emphasis on documentation over improving care, reduced professional morale, and positive spillover. All discussions of unintended consequences involving patient harm were only concerns, not actual events. Several unintended consequences concerned ancillary initiatives for quality improvement (e.g., practice guidelines and performance measurement systems) rather than financial incentives.

Conclusions

Many unintended consequences of financial incentives noted were either only concerns or attributable to ancillary quality-improvement initiatives. Actual unintended consequences included improved documentation of care without necessarily improving actual care, and positive unintended consequences.

Trial registration

Clinicaltrials.gov Identifier: NCT00302718

Impact of a Pay-for-Performance Program on Care for Black Patients with Hypertension: Important Answers in the Era of the Affordable Care Act

OBJECTIVE

Evaluate the effect of a pay-for-performance intervention on the quality of hypertension care provided to black patients and determine whether it produced risk selection.

DATA SOURCE/STUDY SETTING

Primary data collected between 2007 and 2009 from Veterans Affairs physicians and their primary care panels.

STUDY DESIGN

Nested study within a cluster randomized controlled trial of three types of financial incentives and no incentives (control). We compared the proportion of physicians' black patients meeting hypertension performance measures for baseline and final performance periods. We measured risk selection by comparing the proportion of patients who switched providers, patient visit frequency, and panel turnover. Due to limited power, we prespecified in the analysis plan combining the three incentive groups and oversampling black patients.

DATA COLLECTION/EXTRACTION METHOD

Data collected electronically and by chart review.

PRINCIPAL FINDINGS

The proportion of black patients who achieved blood pressure control or received an appropriate response to uncontrolled blood pressure in the final period was 6.3 percent (95 percent confidence interval, 0.8-11.7 percent) greater for physicians who received an incentive than for controls. There was no difference between intervention and controls in the proportion of patients who switched providers, visit frequency, or panel turnover.

CONCLUSIONS AND RELEVANCE

A pay-for-performance intervention improved blood pressure control or appropriate response to uncontrolled blood pressure in black patients and did not produce risk selection.

Theory-based and evidence-based design of audit and feedback programmes: examples from two clinical intervention studies

BACKGROUND

Audit and feedback (A&F) is a common intervention used to change healthcare provider behaviour and, thus, improve healthcare quality. Although A&F can be effective its effectiveness varies, often due to the details of how A&F interventions are implemented. Some have suggested that a suitable conceptual framework is needed to organise the elements of A&F and also explain any observed differences in effectiveness. Through two examples from applied research studies, this article demonstrates how a suitable explanatory theory (in this case Kluger & DeNisi's Feedback Intervention Theory (FIT)) can be systematically applied to design better feedback interventions in healthcare settings.

METHODS

Case 1: this study's objective was to reduce inappropriate diagnosis of catheter-associated urinary tract infections (CAUTI) in inpatient wards. Learning to identify the correct clinical course of action from the case details was central to this study; consequently, the feedback intervention featured feedback elements that FIT predicts would best activate learning processes (framing feedback in terms of group performance and providing of correct solution information). We designed a highly personalised, interactive, one-on-one intervention with healthcare providers to improve their capacity to distinguish between CAUTI and asymptomatic bacteruria (ASB) and treat ASB appropriately. Case 2: Simplicity and scalability drove this study's intervention design, employing elements that FIT predicted positively impacted effectiveness yet still facilitated deployment and scalability (eg, delivered via computer, delivered in writing). We designed a web-based, report-style feedback intervention to help primary care physicians improve their care of patients with hypertension.

RESULTS

Both studies exhibited significant improvements in their desired outcome and in both cases interventions were received positively by feedback recipients.

SUMMARY

A&F has been a popular, yet inconsistently implemented and variably effective tool for changing healthcare provider behaviour and, improving healthcare quality. Through the systematic use of theory such as FIT, robust feedback interventions can be designed that yield greater effectiveness. Future work should look to comparative effectiveness of specific design elements and contextual factors that identify A&F as the optimal intervention to effectuate healthcare provider behaviour change.

TRIAL REGISTRATION NUMBER

NCT01052545, NCT00302718; post-results.

Interventions to improve adherence to cardiovascular disease guidelines: a systematic review

BACKGROUND

Successful management of cardiovascular disease (CVD) is impaired by poor adherence to clinical practice guidelines. The objective of our review was to synthesize evidence about the effectiveness of interventions that target healthcare providers to improve adherence to CVD guidelines and patient outcomes.

METHODS

We searched PubMed, EMBASE, Cochrane Library, PsycINFO, Web of Science and CINAHL databases from inception to June 2014, using search terms related to adherence and clinical practice guidelines. Studies were limited to randomized controlled trials testing an intervention to improve adherence to guidelines that measured both a patient and adherence outcome. Descriptive summary tables were created from data extractions. Meta-analyses were conducted on clinically homogeneous comparisons, and sensitivity analyses and subgroup analyses were carried out where possible. GRADE summary of findings tables were created for each comparison and outcome.

RESULTS AND DISCUSSION

We included 38 RCTs in our review. Interventions included guideline dissemination, education, audit and feedback, and academic detailing. Meta-analyses were conducted for several outcomes by intervention type. Many comparisons favoured the intervention, though only the adherence outcome for the education intervention showed statistically significant improvement compared to usual care (standardized mean difference = 0.58 [95 % confidence interval 0.35 to 0.8]).

CONCLUSIONS

Many interventions show promise to improve practitioner adherence to CVD guidelines. The quality of evidence and number of trials limited our ability to draw conclusions.

Calculations of Financial Incentives for Providers in a Pay-for-Performance Program: Manual Review Versus Data From Structured Fields in Electronic Health Records

BACKGROUND

Hospital report cards and financial incentives linked to performance require clinical data that are reliable, appropriate, timely, and cost-effective to process. Pay-for-performance plans are transitioning to automated electronic health record (EHR) data as an efficient method to generate data needed for these programs.

OBJECTIVE

To determine how well data from automated processing of structured fields in the electronic health record (AP-EHR) reflect data from manual chart review and the impact of these data on performance rewards.

RESEARCH DESIGN

Cross-sectional analysis of performance measures used in a cluster randomized trial assessing the impact of financial incentives on guideline-recommended care for hypertension.

SUBJECTS

A total of 2840 patients with hypertension assigned to participating physicians at 12 Veterans Affairs hospital-based outpatient clinics. Fifty-two physicians and 33 primary care personnel received incentive payments.

MEASURES

Overall, positive and negative agreement indices and Cohen's kappa were calculated for assessments of guideline-recommended antihypertensive medication use, blood pressure (BP) control, and appropriate response to uncontrolled BP. Pearson's correlation coefficient was used to assess how similar participants' calculated earnings were between the data sources.

RESULTS

By manual chart review data, 72.3% of patients were considered to have received guideline-recommended antihypertensive medications compared with 65.0% by AP-EHR review (κ=0.51). Manual review indicated 69.5% of patients had controlled BP compared with 66.8% by AP-EHR review (κ=0.87). Compared with 52.2% of patients per the manual review, 39.8% received an appropriate response by AP-EHR review (κ=0.28). Participants' incentive payments calculated using the 2 methods were highly correlated (r≥0.98). Using the AP-EHR data to calculate earnings, participants' payment changes ranged from a decrease of $91.00 (-30.3%) to an increase of $18.20 (+7.4%) for medication use (interquartile range, -14.4% to 0%) and a decrease of $100.10 (-31.4%) to an increase of $36.40 (+15.4%) for BP control or appropriate response to uncontrolled BP (interquartile range, -11.9% to -6.1%).

CONCLUSIONS

Pay-for-performance plans that use only EHR data should carefully consider the measures and the structure of the EHR before data collection and financial incentive disbursement. For this study, we feel that a 10% difference in the total amount of incentive earnings disbursed based on AP-EHR data compared with manual review is acceptable given the time and resources required to abstract data from medical records.

Effects of individual physician-level and practice-level financial incentives on hypertension care: a randomized trial

IMPORTANCE

Pay for performance is intended to align incentives to promote high-quality care, but results have been contradictory.

OBJECTIVE

To test the effect of explicit financial incentives to reward guideline-recommended hypertension care.

DESIGN, SETTING, AND PARTICIPANTS

Cluster randomized trial of 12 Veterans Affairs outpatient clinics with 5 performance periods and a 12-month washout that enrolled 83 primary care physicians and 42 nonphysician personnel (eg, nurses, pharmacists).

INTERVENTIONS

Physician-level (individual) incentives, practice-level incentives, both, or none. Intervention participants received up to 5 payments every 4 months; all participants could access feedback reports.

MAIN OUTCOMES AND MEASURES

Among a random sample, number of patients achieving guideline-recommended blood pressure thresholds or receiving an appropriate response to uncontrolled blood pressure, number of patients prescribed guideline-recommended medications, and number who developed hypotension.

RESULTS

Mean (SD) total payments over the study were $4270 ($459), $2672 ($153), and $1648 ($248) for the combined, individual, and practice-level interventions, respectively. The unadjusted baseline and final percentages and the adjusted absolute change over the study in patients meeting the combined blood pressure/appropriate response measure were 75% to 84% and 8.84% (95% CI, 4.20% to 11.80%) for the individual group, 80% to 85% and 3.70% (95% CI, 0.24% to 7.68%) for the practice-level group, 79% to 88% and 5.54% (95% CI, 1.92% to 9.52%) for the combined group, and 86% to 86% and 0.47% (95% CI, -3.12% to 4.04%) for the control group. The adjusted absolute estimated difference in the change between the proportion of patients with blood pressure control/appropriate response for individual incentive and control groups was 8.36% (95% CI, 2.40% to 13.00%; P=.005). The other incentive groups did not show a significant change compared with controls for this outcome. For medications, the unadjusted baseline and final percentages and the adjusted absolute change were 61% to 73% and 9.07% (95% CI, 4.52% to 13.44%), 56% to 65% and 4.98% (95% CI, 0.64% to 10.08%), 65% to 80% and 7.26% (95% CI, 2.92% to 12.48%), and 63% to 72% and 4.35% (95% CI, -0.28% to 9.28%), respectively. These changes in the use of guideline-recommended medications were not significant in any of the incentive groups compared with controls, nor was the incidence of hypotension. The effect of the incentive was not sustained after a washout.

CONCLUSIONS AND RELEVANCE

Individual financial incentives, but not practice-level or combined incentives, resulted in greater blood pressure control or appropriate response to uncontrolled blood pressure; none of the incentives resulted in greater use of guideline-recommended medications or increased incidence of hypotension compared with controls. Further research is needed on the factors that contributed to these findings.

TRIAL REGISTRATION

clinicaltrials.gov Identifier: NCT00302718.

Financial incentives and physician commitment to guideline-recommended hypertension management

OBJECTIVES

To examine the impact of financial incentives on physician goal commitment to guideline-recommended hypertension care.

STUDY DESIGN

Clinic-level cluster-randomized trial with 4 arms: individual, group, or combined incentives, and control.

METHODS

A total of 83 full-time primary care physicians at 12 Veterans Affairs medical centers completed web-based surveys measuring their goal commitment to guideline-recommended hypertension care every 4 months and telephone interviews at months 8 and 16. Intervention arm participants received performance-based incentives every 4 months for 5 periods. All participants received guideline education at baseline and audit and feedback every 4 months.

RESULTS

Physician goal commitment did not vary over time or across arms. Participants reported patient nonadherence was a perceived barrier and consistent follow-up was a perceived facilitator to successful hypertension care, suggesting that providers may perceive hypertension management as more of a patient responsibility (external locus of control).

CONCLUSIONS

Financial incentives may constitute an insufficiently strong intervention to influence goal commitment when providers attribute performance to external forces beyond their control.

How variability in the institutional review board review process affects minimal-risk multisite health services research

BACKGROUND

The Department of Health and Human Services recently called for public comment on human subjects research protections.

OBJECTIVE

To assess variability in reviews across institutional review boards (IRBs) for a multisite, minimal-risk trial of financial incentives for evidence-based hypertension care and to quantify the effect of review determinations on site participation, budget, and timeline.

DESIGN

A natural experiment occurring from multiple IRBs reviewing the same protocol for a multicenter trial (May 2005 to October 2007).

PARTICIPANTS

25 Veterans Affairs (VA) medical centers.

MEASUREMENTS

Number of submissions, time to approval, and costs were evaluated; patient complexity, academic affiliation, size, and location (urban or rural) between participating and nonparticipating VA medical centers were compared.

RESULTS

Of 25 eligible VA medical centers, 6 did not meet requirements for IRB review and 2 declined to participate. Of 17 applications, 14 were approved. The process required 115 submissions, lasted 27 months, and cost close to $170 000 in staff salaries. One IRB's concern about incentivizing a particular medication recommended by national guidelines prompted a change in our design to broaden our inclusion criteria beyond uncomplicated hypertension. The change required amending the protocol at 14 sites to preserve internal validity. The IRBs that approved the protocol classified it as minimal risk. The 12 sites that ultimately participated in the trial were more likely to be urban and academically affiliated and to care for more complex patients, which limits the external validity of the trial's findings.

LIMITATION

Because data came from a single multisite trial in the VA system that uses a 2-stage review process, generalizability is limited.

CONCLUSION

Complying with IRB requirements for a minimal-risk study required substantial resources and threatened the study's internal and external validity. The current review of regulatory requirements may address some of these problems.