Effect of Social Comparison Feedback on Laboratory Test Ordering for Hospitalized Patients: A Randomized Controlled Trial

Effect of a Performance Feedback Dashboard on Hospitalist Laboratory Test Utilization

BACKGROUND

Healthcare spending continues to be an area of improvement across all forms of medicine. Overtreatment or low-value care, including overutilization of laboratory testing, has an estimated annual cost of waste of $75.7-$101.2 billion annually. Providing performance feedback to hospitalists has been shown to be an effective way to encourage the practice of quality-improvement-focused medicine. There remains limited data regarding the implementation of performance feedback and direct results on hospital laboratory testing spending in the short term.

OBJECTIVE

The objective of this project was to identify whether performance-based feedback on laboratory utilization between both hospitalists and resident teams results in more conservative utilization of laboratory testing.

DESIGN, SETTING, PARTICIPANTS

This quality improvement project was conducted at a tertiary academic medical center, including both direct-care and house-staff teams.

INTERVENTION OR EXPOSURE

A weekly performance feedback report was generated and distributed to providers detailing laboratory test utilization by all hospitalists in a ranked system, normalized by the census of patients, for 3 months.

MAIN OUTCOMES AND MEASURES

The outcome measure was cumulative laboratory utilization during the intervention period compared to baseline utilization during the corresponding 3 months in the year prior and the weekly trend in laboratory utilization over 52 weeks. The aggregate laboratory utilization rate during intervention and control time periods was defined as the total number of laboratory tests ordered divided by the total number of patient encounters. Additionally, the cost difference was averaged per quarter and reported. The week-by-week trend in laboratory utilization was evaluated using a statistical process control (SPC) chart.

RESULTS

We found that following intervention during January-March 2020, the cumulative complete blood count utilization rate decreased from 5.54 to 4.83 per patient encounter and the basic metabolic panels/CMP utilization rate decreased from 6.65 to 6.11 per patient encounter compared with January-March 2019. This equated to cost savings of ~$42,700 in total for the quarter. Nonrandom variation was seen on SPC charts in weekly laboratory utilization rates for common laboratory tests during the intervention period.

CONCLUSIONS

We found that our intervention did result in a decrease in laboratory test utilization rates across direct-care and house-staff teams. This study lays promising groundwork for one tool that can be used to eliminate a source of hospital waste and improve the quality and efficiency of patient care.

Characteristics promoting behaviour change: physician experience with a coalition-led quality improvement initiative to reduce excessive laboratory test ordering

BACKGROUND

Attempts have been made to reduce excessive laboratory test ordering; however, the problem persists and barriers to physician involvement in quality improvement (QI) remain. We sought to understand physician participation experience following a laboratory test overuse initiative supported by a QI coalition.

METHODS

As part of a larger mixed-methods study, structured virtual interviews were conducted with 12 physicians. The Theoretical Domains Framework (TDF) and the Behavioural Change Wheel (BCW) were used to identify characteristics that influence physician behaviour change for QI leadership and participation and appropriate blood urea nitrogen (BUN) test ordering. A content analysis of physicians' statements to the TDF was performed, resulting in overarching themes; relevant TDF domains were mapped to the intervention functions of the BCW.

RESULTS

Nine overarching themes emerged from the data. Eight of 14 TDF domains influence QI leadership and participation, and 10 influence appropriate BUN-test ordering behaviours. The characteristics participants described that promoted a change in their QI participation, leadership and appropriate BUN-test ordering were: QI education with hands-on training; physician peer mentorship/support; personnel assistance (QI and analytics) and communication from a trusted/credible physician leader who shares data and insights about the physician role in the initiative, clinical best practice and past project success. Other elements included: a simply designed initiative requiring minimal effort and no clinical workflow disruptions; revised order forms/panels and limiting test-order frequency when laboratory tests are normal. Additionally, various future intervention strategies were identified. For their initial initiative participation, physicians acknowledged coalition leader or member credibility was more important than awareness of the coalition.

CONCLUSIONS

Based on physicians' described perceptions and experiences, coalition characteristics that influenced their QI leadership and participation, and appropriate BUN-test ordering behaviours were revealed; these characteristics aligned to several TDF domains. The findings suggest that these behaviours are multidimensional, requiring a multistrategy approach to change behaviour.

The impact of performance feedback reports on physician ordering behavior in the use of computed tomography pulmonary angiography (CTPA)

PURPOSE

The increased utilization, and potential overutilization, of computed tomography pulmonary angiography (CTPA) is a well-recognized issue within emergency departments (EDs). The objective of this study is to determine the impact of performance feedback reports on CTPA ordering behavior among ED physicians.

METHODS

We conducted a prospective study of the impact of individualized performance feedback reports on the ordering behavior of physicians working at two high-volume community EDs in Ontario, Canada. We generated individualized reports (or "Dashboards") for each ED physician containing detailed feedback and peer comparison for each physician's CTPA ordering. Our baseline pre-intervention period was January 1 to December 31, 2018, and our intervention period was January 1, 2019, to December 31, 2021. We tracked individual and group ordering behavior through the study period. Our primary outcomes are impact of feedback on (1) overall group ordering rate and (2) overall diagnostic yield. Secondary analysis was done to determine the impact of the intervention on those physicians with the highest CTPA utilization rate.

RESULTS

There was no statistically significant difference in the diagnostic yield of the included physicians in either of the years of the intervention period. There was a statically significant increase in the utilization rate for CTPA from 2018 to 2020 and 2021 from 5.9 to 7.9 and 11.4 CTPAs per 1000 ED visits respectively (p < 0.5).

CONCLUSION

Our study found no consistent significant impact of individualized feedback and peer comparison on physician ordering of CTPAs. This points to a potentially greater impact of environmental and institutional factors, as opposed to physician-targeted quality improvement measures, on physician ordering behavior.

Reduction of urea test ordering in the emergency department: multicomponent intervention including education, electronic ordering, and data feedback

Introduction

In the emergency department (ED), laboratory testing accounts for a significant portion of the medical assessment. Although excess laboratory test ordering has been proven to be prevalent, different types of interventions have been used to encourage a behavioural change in how physicians order tests. In one western Canadian hospital medicine program, a quality improvement project aimed to reduce the total monthly blood urea nitrogen (BUN) test ordered by physicians was found to be successful. The objective of this project was to evaluate a similar multicomponent intervention aimed at ED physician ordering, with the primary goal of reducing the number of monthly BUN tests ordered per ED visit.

Methods

A pre post intervention design was conducted over 12-months. The first intervention component was an educational presentation conducted by physician leaders. Second, a regularly used order panel within the ED electronic order system was modified, removing the BUN test. The third component involved audit and feedback; the total monthly BUN test ordered for the ED department post intervention start was shared with all ED physicians twice (at 5 and 12 months).An interrupted time series analysis was completed to evaluate the multicomponent intervention effect.

Results

The total monthly ordered BUN test declined from an average of 1905 pre-intervention to 448 post-intervention, and the total monthly BUN test to total ED visit ratio declined from 0.46 to 0.1. These results were a statistically significant reduction in physician BUN test ordering.

Conclusions

Targeted education, order panel design and data feedback interventions can impact physician ordering behaviour in the emergent healthcare context, where diagnostic tests are often over used.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43678-022-00333-w.

Effectiveness of clinical dashboards as audit and feedback or clinical decision support tools on medication use and test ordering: a systematic review of randomized controlled trials

BACKGROUND

Clinical dashboards used as audit and feedback (A&F) or clinical decision support systems (CDSS) are increasingly adopted in healthcare. However, their effectiveness in changing the behavior of clinicians or patients is still unclear. This systematic review aims to investigate the effectiveness of clinical dashboards used as CDSS or A&F tools (as a standalone intervention or part of a multifaceted intervention) in primary care or hospital settings on medication prescription/adherence and test ordering.

METHODS

Seven major databases were searched for relevant studies, from inception to August 2021. Two authors independently extracted data, assessed the risk of bias using the Cochrane RoB II scale, and evaluated the certainty of evidence using GRADE. Data on trial characteristics and intervention effect sizes were extracted. A narrative synthesis was performed to summarize the findings of the included trials.

RESULTS

Eleven randomized trials were included. Eight trials evaluated clinical dashboards as standalone interventions and provided conflicting evidence on changes in antibiotic prescribing and no effects on statin prescribing compared to usual care. Dashboards increased medication adherence in patients with inflammatory arthritis but not in kidney transplant recipients. Three trials investigated dashboards as part of multicomponent interventions revealing decreased use of opioids for low back pain, increased proportion of patients receiving cardiovascular risk screening, and reduced antibiotic prescribing for upper respiratory tract infections.

CONCLUSION

There is limited evidence that dashboards integrated into electronic medical record systems and used as feedback or decision support tools may be associated with improvements in medication use and test ordering.

OUP accepted manuscript

Objectives

(1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes.

Methods

We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement.

Results

Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6–10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes.

Conclusions

e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans.

Protocol Registration

PROSPERO CRD42016048695.

Systematic review of clinician-directed nudges in healthcare contexts

OBJECTIVE

Nudges are interventions that alter the way options are presented, enabling individuals to more easily select the best option. Health systems and researchers have tested nudges to shape clinician decision-making with the aim of improving healthcare service delivery. We aimed to systematically study the use and effectiveness of nudges designed to improve clinicians' decisions in healthcare settings.

DESIGN

A systematic review was conducted to collect and consolidate results from studies testing nudges and to determine whether nudges directed at improving clinical decisions in healthcare settings across clinician types were effective. We systematically searched seven databases (EBSCO MegaFILE, EconLit, Embase, PsycINFO, PubMed, Scopus and Web of Science) and used a snowball sampling technique to identify peer-reviewed published studies available between 1 January 1984 and 22 April 2020. Eligible studies were critically appraised and narratively synthesised. We categorised nudges according to a taxonomy derived from the Nuffield Council on Bioethics. Included studies were appraised using the Cochrane Risk of Bias Assessment Tool.

RESULTS

We screened 3608 studies and 39 studies met our criteria. The majority of the studies (90%) were conducted in the USA and 36% were randomised controlled trials. The most commonly studied nudge intervention (46%) framed information for clinicians, often through peer comparison feedback. Nudges that guided clinical decisions through default options or by enabling choice were also frequently studied (31%). Information framing, default and enabling choice nudges showed promise, whereas the effectiveness of other nudge types was mixed. Given the inclusion of non-experimental designs, only a small portion of studies were at minimal risk of bias (33%) across all Cochrane criteria.

CONCLUSIONS

Nudges that frame information, change default options or enable choice are frequently studied and show promise in improving clinical decision-making. Future work should examine how nudges compare to non-nudge interventions (eg, policy interventions) in improving healthcare.

Theory and practical guidance for effective de-implementation of practices across health and care services: a realist synthesis

Background

Health-care systems across the globe are facing increased pressures to balance the efficient use of resources and at the same time provide high-quality care. There is greater requirement for services to be evidence based, but practices that are of limited clinical effectiveness or cost-effectiveness still occur.

Objectives

Our objectives included completing a concept analysis of de-implementation, surfacing decision-making processes associated with de-implementing through stakeholder engagement, and generating an evidence-based realist programme theory of ‘what works’ in de-implementation.

Design

A realist synthesis was conducted using an iterative stakeholder-driven four-stage approach. Phase 1 involved scoping the literature and conducting stakeholder interviews to develop the concept analysis and an initial programme theory. In Phase 2, systematic searches of the evidence were conducted to test and develop this theory, expressed in the form of contingent relationships. These are expressed as context–mechanism–outcomes to show how particular contexts or conditions trigger mechanisms to generate outcomes. Phase 3 consisted of validation and refinement of programme theories through stakeholder interviews. The final phase (i.e. Phase 4) formulated actionable recommendations for service leaders.

Participants

In total, 31 stakeholders (i.e. user/patient representatives, clinical managers, commissioners) took part in focus groups and telephone interviews.

Data sources

Using keywords identified during the scoping work and concept analysis, searches of bibliographic databases were conducted in May 2018. The databases searched were the Cochrane Library, Campbell Collaboration, MEDLINE (via EBSCOhost), the Cumulative Index to Nursing and Allied Health Literature (via EBSCOhost), the National Institute for Health Research Journals Library and the following databases via the ProQuest platform: Applied Social Sciences Index and Abstracts, Social Services Abstracts, International Bibliography of the Social Sciences, Social Sciences Database and Sociological Abstracts. Alerts were set up for the MEDLINE database from May 2018 to December 2018. Online sources were searched for grey literature and snowballing techniques were used to identify clusters of evidence.

Results

The concept analysis showed that de-implementation is associated with five main components in context and over time: (1) what is being de-implemented, (2) the issues driving de-implementation, (3) the action characterising de-implementation, (4) the extent that de-implementation is planned or opportunistic and (5) the consequences of de-implementation. Forty-two papers were synthesised to identify six context–mechanism–outcome configurations, which focused on issues ranging from individual behaviours to organisational procedures. Current systems can perpetuate habitual decision-making practices that include low-value treatments. Electronic health records can be designed to hide or remove low-value treatments from choice options, foregrounding best evidence. Professionals can be made aware of their decision-making strategies through increasing their attention to low-value practice behaviours. Uncertainty about diagnosis or patients’ expectations for certain treatments provide opportunities for ‘watchful waiting’ as an active strategy to reduce inappropriate investigations and prescribing. The emotional component of clinician–patient relationships can limit opportunities for de-implementation, requiring professional support through multimodal educational interventions. Sufficient alignment between policy, public and professional perspectives is required for de-implementation success.

Limitations

Some specific clinical issues (e.g. de-prescribing) dominate the de-implementation evidence base, which may limit the transferability of the synthesis findings. Any realist inquiry generates findings that are essentially cumulative and should be developed through further investigation that extends the range of sources into, for example, clinical research and further empirical studies.

Conclusions

This review contributes to our understanding of how de-implementation of low-value procedures and services can be improved within health-care services, through interventions that make professional decision-making more accountable and the prominence of a whole-system approach to de-implementation. Given the whole-system context of de-implementation, a range of different dissemination strategies will be required to engage with different stakeholders, in different ways, to change practice and policy in a timely manner.

Study registration

This study is registered as PROSPERO CRD42017081030.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 2. See the NIHR Journals Library website for further project information.

The effectiveness of behavioral economics-informed interventions on physician behavioral change: A systematic literature review

OBJECTIVE

Interventions informed by behavioral economics have the potential to change behaviors governed by underlying cognitive biases. This has been explored extensively for various use in healthcare including changing patient behavior and, more recently, physician behavior. We aimed to systematically review the literature on the use and effectiveness of behavioral economics-informed interventions in changing physician behavior.

METHOD

We searched Medline, Cochrane Library, EBM Reviews, PsychINFO, EconLit, Business Source Complete and Web of Science for peer-reviewed studies published in English that examined the effectiveness of behavioral economics-informed interventions on physician behavioral change. We included studies of physicians in all care settings and specialties and all types of objectively measured behavioral outcomes. The reporting quality of included studies was appraised using the Effective Public Health Practice Project tool.

RESULTS

We screened 6,439 studies and included 17 studies that met our criteria, involving at least 9,834 physicians. The majority of studies were conducted in the United States, published between 2014 and 2018, and were in the patient safety and quality domain. Reporting quality of included studies included strong (n = 7), moderate (n = 6) and weak (n = 4). Changing default settings and providing social reference points were the most widely studied interventions, with these studies consistently demonstrating their effectiveness in changing physician behavior despite differences in implementation methods among studies. Prescribing behavior was most frequently targeted in included studies, with consistent effectiveness of studied interventions.

CONCLUSION

Changing default settings and providing social reference points were the most frequently studied and consistently effective interventions in changing physician behavior towards guideline-concordant practices. Additional theory-informed research is needed to better understand the mechanisms underlying the effectiveness of these interventions to guide implementation.

Impact of an education and multilevel social comparison–based intervention bundle on use of routine blood tests in hospitalised patients at an academic tertiary care hospital: a controlled pre-intervention post-intervention study

Background

Repetitive inpatient laboratory testing contributes to waste in healthcare. We evaluated an intervention bundle combining education and multilevel social comparison feedback to safely reduce repetitive use of inpatient routine laboratory tests.

Methods

This non-randomised controlled pre-intervention post-intervention study was conducted in four adult hospitals from October 2016 to March 2018. In the medical teaching unit (MTU) of the intervention site, learners received education and aggregate social comparison feedback and attending internists received individual comparison feedback on routine laboratory test utilisation. MTUs of the remaining three sites served as control units. Number and cost of routine laboratory tests ordered per patient-day before and after the intervention was compared with the control units, adjusting for patient factors. Safety endpoints included number of critically abnormal laboratory test results, number of stat laboratory test orders, patient length of stay, transfer rate to the ICU, and 30-day readmission and mortality.

Results

A total of 14 000 patients were included. Pre-intervention and post-intervention groups were similar in age, sex, Charlson Comorbidity Index and length of stay. From the pre-intervention period to the post-intervention period, significantly fewer routine laboratory tests were ordered at the intervention MTU (incidence rate ratio=0.89; 95% CI 0.79 to 1.00; p=0.048) with associated costs savings of $C68 877 (p=0.020) as compared with the control sites. The variability in the ordering pattern of internists at the intervention site also decreased post-intervention. No worsening was noted in the safety endpoints between the pre-intervention and post-intervention period at the intervention unit compared with the controls.

Conclusions

Combination of education and multilevel social comparison feedback significantly and safely led to cost savings through reduced use of routine laboratory tests in hospitalised patients.

Evaluating practice variance among family physicians to identify targets for laboratory utilization management

BACKGROUND

There is widespread variation in testing practice among practitioners, however there has been no objective way to pinpoint target tests for utilization management. We propose to take advantage of inter-physician variance in clinical practice as a quantitative measure to generate lists of potentially misutilized tests.

METHODS

Testing frequencies from a database of clinical testing volumes for outpatients in Calgary, Canada, were obtained for the study period of 2016. For each chemistry, microbiology or hematology test, an arithmetic coefficient of variation (CV) was calculated from family physicians' ordering frequencies.

RESULTS

The mean CV for all 358 tests considered was 219% (95% CI 206-231%) with a range of 52-729%. The highest variance was observed for human T-lymphotropic virus antibody testing and several tests for heavy metal levels (mercury, copper, zinc and chromium). Among the 100 most commonly run tests, high variance was found for several endocrinology tests including cortisol.

CONCLUSIONS

The utility of ranking clinical tests by ordering variance presents a practical approach to evaluate relative variation in physician practice strategy and to identify potential areas of misutilization.