Development of an Educational Strategy and Decision Support Tool to Enhance Appropriate Use of Stress Echocardiography at a Large Academic Medical Center: A Prospective, Pre- and Postintervention Analysis

An initiative to promote value-based stress test selection in primary care and cardiology clinics: A mixed methods evaluation

OBJECTIVES

Exercise stress echocardiograms (stress echos) are overused, whereas exercise stress electrocardiograms (stress ECGs) can be an appropriate, lower-cost substitute. In this post hoc, mixed methods evaluation, we assessed an initiative promoting value-based, guideline-concordant ordering practices in primary care (PC) and cardiology clinics.

METHODS

Change in percent of stress ECGs ordered of all exercise stress tests (stress ECGs and echos) was calculated between three periods: baseline (January 2019-February 2020); Period 1 with reduced stress ECG report turnaround time + PC-targeted education (began June 2020); and Period 2 with the addition of electronic health record-based alternative alert (AA) providing point-of-care clinical decision support. The AA was deployed in two of five PC clinics in July 2020, two additional PC clinics in January 2021, and one of four cardiology clinics in February 2021. Nineteen primary care providers (PCPs) and five cardiologists were interviewed in Period 2.

RESULTS

Clinicians reported reducing ECG report turnaround time was crucial for adoption. PCPs specifically reported that value-based education helped change their practice. In PC, the percent of stress ECGs ordered increased by 38% ± 6% (SE) (p < 0.0001) from baseline to Period 1. Most PCPs identified the AA as the most impactful initiative, yet stress ECG ordering did not change (6% ± 6%; p = 0.34) between Periods 1 and 2. In contrast, cardiologists reportedly relied on their expertise rather than AAs, yet their stress ECGs orders increased from Period 1 to 2 to a larger degree in the cardiology clinic with the AA (12% ± 5%; p = 0.01) than clinics without the AA (6% ± 2%; p = 0.01). The percent of stress ECGs ordered was higher in Period 2 than baseline for both specialties (both p < 0.0001).

CONCLUSIONS

This initiative influenced ordering behaviour in PC and cardiology clinics. However, clinicians' perceptions of the initiative varied between specialties and did not always align with the observed behaviour change.

The Use of Quality Improvement Interventions in Reducing Rarely Appropriate Cardiac Imaging

The use of cardiac imaging has become increasingly prevalent over the last decade. Approximately 10% to 15% of noninvasive cardiac imaging is ordered for rarely appropriate indications. The appropriate use criteria (AUC) for cardiac imaging were issued to decrease unnecessary testing and reduce health care costs. However, it remains unclear whether these efforts have been successful. This meta-analysis evaluates whether AUC quality improvement (QI) interventions effectively reduce inappropriate cardiac imaging. Databases were searched for studies reporting QI intervention effect aiming to reduce rarely appropriate noninvasive cardiac imaging based on AUC. Imaging modalities assessed include transthoracic echocardiography, stress echocardiography, and myocardial perfusion imaging. We searched Ovid MEDLINE, EMBASE, Scopus, Web of Science, Google Scholar, and EBSCO CINAHL. The primary end point was a decrease of rarely appropriate testing. The search was not restricted to time or publication status. The literature search identified 2,391 possible studies, 13 studies and 26,557 patients were included. Mean follow-up was 12 months (1 to 60 months). QI interventions were statistically significant in reducing rarely appropriate tests after the intervention compared with the control group (odds ratio [OR] 0.51, 95% confidence interval [CI] 0.41 to 0.64, p <0.01). The QI interventions were also assessed for persistence based on short-term (<3 months) and long-term (>3 months) efficacy. Both the short-term effect and long-term effect were persistent (OR 0.6, 95% CI 0.47 to 0.77, p <0.01 and OR 0.47, 95% CI 0.37 to 0.61, p <0.01, respectively). AUC QI interventions are associated with the successful decrease of inappropriate noninvasive cardiac testing with these effects persisting over time.

The use of quality improvement interventions in reducing rarely appropriate echocardiograms: A systematic review and meta-analysis

BACKGROUND

The volume of cardiac imaging continues to increase, with many tests performed for rarely appropriate indications. Appropriate use criteria (AUC) documents were published by the American Society of Echocardiography and American College of Cardiology, with quality improvement (QI) interventions developed in various institutions. However, the effectiveness of these interventions has not been assessed in a systematic fashion.

METHODS

We searched Ovid MEDLINE, EMBASE, Scopus, Web of Science, Google Scholar, and EBSCO CINAHL for studies reporting association between cardiac imaging, AUC and QI. The search was not restricted to time or publication status. We selected studies assessing the effect of QI interventions on performance of rarely appropriate echocardiograms. The primary endpoint was reduction of rarely appropriate testing.

RESULTS

Nine studies with 22,070 patients met inclusion criteria. Mean follow up was 15 months (1-60 months). QI interventions resulted in statistically significant reduction in rarely appropriate tests (OR 0.52, 95% CI: .41-.66; p < .01). The effects of QI interventions were analyzed over both the short (<3 months) and long-term (>3 months) post intervention (OR 0.62, 95% CI: .49-.79; p < .01 in the short term, and OR 0.47, 95% CI: .35-.62; p < .01 in the long term). Subgroup analysis of the type of intervention, classified as education tools or decision support tools showed both significantly reduced rarely appropriate testing (OR 0.54, 95% CI: .41-.73; p < .01; OR .47, 95% CI: .36-.61; p < .01). Adding a feedback tool did not change the effect compared to not using a feedback tool (OR 0.49 vs. 0.57, 95% CI: .36-.68 vs. 39-.84; p > .05).

CONCLUSION

QI interventions are associated with a significant reduction in performance of rarely appropriate echocardiography testing, the effects of which persist over time. Both education and decision support tools were effective, while adding feedback tools did not result in further reduction of ordering rarely appropriate studies.

Interventions to reduce low-value imaging - a systematic review of interventions and outcomes

BACKGROUND

It is estimated that 20-50% of all radiological examinations are of low value. Many attempts have been made to reduce the use of low-value imaging. However, the comparative effectiveness of interventions to reduce low-value imaging is unclear. Thus, the objective of this systematic review was to provide an overview and evaluate the outcomes of interventions aimed at reducing low-value imaging.

METHODS

An electronic database search was completed in Medline - Ovid, Embase-Ovid, Scopus, and Cochrane Library for citations between 2010 and 2020. The search was built from medical subject headings for Diagnostic imaging/Radiology, Health service misuse or medical overuse, and Health planning. Keywords were used for the concept of reduction and avoidance. Reference lists of included articles were also hand-searched for relevant citations. Only articles written in English, German, Danish, Norwegian, Dutch, and Swedish were included. The Mixed Methods Appraisal Tool was used to appraise the quality of the included articles. A narrative synthesis of the final included articles was completed.

RESULTS

The search identified 15,659 records. After abstract and full-text screening, 95 studies of varying quality were included in the final analysis, containing 45 studies found through hand-searching techniques. Both controlled and uncontrolled before-and-after studies, time series, chart reviews, and cohort studies were included. Most interventions were aimed at referring physicians. Clinical practice guidelines (n = 28) and education (n = 28) were most commonly evaluated interventions, either alone or in combination with other components. Multi-component interventions were often more effective than single-component interventions showing a reduction in the use of low-value imaging in 94 and 74% of the studies, respectively. The most addressed types of imaging were musculoskeletal (n = 26), neurological (n = 23) and vascular (n = 16) imaging. Seventy-seven studies reported reduced low-value imaging, while 3 studies reported an increase.

CONCLUSIONS

Multi-component interventions that include education were often more effective than single-component interventions. The contextual and cultural factors in the health care systems seem to be vital for successful reduction of low-value imaging. Further research should focus on assessing the impact of the context in interventions reducing low-value imaging and how interventions can be adapted to different contexts.

Implementation of appropriate use criteria for cardiology tests and procedures: a systematic review and meta-analysis

AIMS

The American College of Cardiology appropriate use criteria (AUC) provide clinicians with evidence-informed recommendations for cardiac care. Adopting AUC into clinical workflows may present challenges, and there may be specific implementation strategies that are effective in promoting effective use of AUC. We sought to assess the effect of implementing AUC in clinical practice.

METHODS AND RESULTS

We conducted a meta-analysis of studies found through a systematic search of the MEDLINE, Web of Science, Cochrane, or CINAHL databases. Peer-reviewed manuscripts published after 2005 that reported on the implementation of AUC for a cardiovascular test or procedure were included. The main outcome was to determine if AUC implementation was associated with a reduction in inappropriate/rarely appropriate care. Of the 18 included studies, the majority used pre/post-cohort designs; few (n = 3) were randomized trials. Most studies used multiple strategies (n = 12, 66.7%). Education was the most common individual intervention strategy (n = 13, 72.2%), followed by audit and feedback (n = 8, 44.4%) and computerized physician order entry (n = 6, 33.3%). No studies reported on formal use of stakeholder engagement or 'nudges'. In meta-analysis, AUC implementation was associated with a reduction in inappropriate/rarely appropriate care (odds ratio 0.62, 95% confidence interval 0.49-0.78). Funnel plot suggests the possibility of publication bias.

CONCLUSION

We found most published efforts to implement AUC observed reductions in inappropriate/rarely appropriate care. Studies rarely explored how or why the implementation strategy was effective. Because interventions were infrequently tested in isolation, it is difficult to make observations about their effectiveness as stand-alone strategies.

STUDY REGISTRATION

PROSPERO 2018 CRD42018091602. Available from https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018091602.

Effects of an Electronic Medical Record Intervention on Appropriateness of Transthoracic Echocardiograms: A Prospective Study

BACKGROUND

Transthoracic echocardiograms (TTEs) account for approximately half of U.S. spending on cardiac imaging. We developed an electronic medical record (EMR)-based decision-support algorithm for TTE ordering and hypothesized that it would increase the appropriateness of TTE orders.

METHODS

This prospective observational study was performed at the Veterans Affairs Ann Arbor Healthcare System. From October to December 2016 (preintervention), consecutive TTEs ordered in the inpatient, outpatient, and emergency department settings were included. In May 2017, a decision-support algorithm was incorporated into the EMR, giving immediate feedback to providers. Chart review was performed for TTEs ordered from June to August 2017 (early intervention) and from June to August 2018 (late intervention). Appropriateness was determined based on the 2011 appropriate use criteria for echocardiography.

RESULTS

Appropriate TTE orders increased from 87.6% preintervention to 94.5% at early intervention (z = 0.00018) but decreased to 90.0% at late intervention (z = 0.51, compared with preintervention). Among patients with no previous TTEs in our system, 95.3% of TTEs were appropriate, compared with 87.7% of TTEs for patients with prior TTEs within 30 days prior (odds ratio = 2.85; 95% CI, 1.18-6.31; P = .005).

CONCLUSIONS

The EMR algorithm initially increased the percentage of appropriate TTEs, but this effect decayed over time. Further study is needed to develop EMR-based interventions that will have lasting impacts on provider ordering patterns.

Association Between Physicians' Appropriate Use of Echocardiography and Subsequent Healthcare Use and Outcomes in Patients With Heart Failure

Background

There is little understanding of whether a physician's tendency to order an inappropriate cardiac service is associated with the use of other cardiac services and clinical outcomes in their patients with heart failure (HF).

Methods and Results

We conducted a secondary analysis of 35 Ontario‐based cardiologists who participated in the control arm of the Echo WISELY (Will Inappropriate Scenarios for Echocardiography Lessen Significantly) trial. Transthoracic echocardiograms, ordered during the trial, were classified as rarely appropriate (rA), appropriate, or maybe appropriate on the basis of the 2011 appropriate use criteria. Cardiologists were grouped into tertiles of rA transthoracic echocardiogram ordering frequency: low ordering (bottom tertile), n=11; moderate ordering, n=12; or high ordering (top tertile), n=12. The main outcomes were measures of cardiac service use, including cardiology‐related physician visits, tests, and medications. Among 1677 patients with heart failure and an outpatient visit to 1 of 35 cardiologists, we found no significant association between rA transthoracic echocardiogram ordering frequency (by tertile) and cardiac testing use, although patients of cardiologists in the high ordering group had fewer physician visits, on average, than patients seen by low ordering cardiologists. In addition, patients of cardiologists in the highest rA ordering tertile had significantly lower odds of receiving potentially effective interventions, such as β blockers (odds ratio, 0.62; 95% CI, 0.43–0.89), than the low ordering group.

Conclusions

Although patients of cardiologists who frequently order rA transthoracic echocardiograms do not appear more (or less) likely to have subsequent cardiac tests, these patients have fewer follow‐up visits and lower odds of receiving evidence‐based medications.

Clinical Trial Registration

URL: http://www.clinicaltrials.gov. Unique identifier: NCT02038101.

Factors Influencing Pediatric Outpatient Transthoracic Echocardiography Utilization Before Appropriate Use Criteria Release: A Multicenter Study

BACKGROUND

Although pediatric appropriate use criteria (AUC) for outpatient transthoracic echocardiography (TTE) are available, little is known about TTE utilization patterns before their release. The aims of this study were to determine the relation between AUC and TTE utilization and to identify patient and physician factors associated with discordance between the AUC and clinical practice.

METHODS

A retrospective review of 3,000 initial outpatient pediatric cardiology encounters at six centers was performed. Investigator-determined indications were classified using AUC definitions. Concordance between AUC and TTE utilization was determined. Multivariate analysis was performed to identify patient and physician factors associated with TTE's being performed for rarely appropriate and TTE's not being performed for appropriate indications.

RESULTS

Concordance between AUC and TTE utilization was 88%. TTE was performed for rarely appropriate indications in 9% and was associated with patient age < 3 months, indications of murmur, noninvasive imaging physician subspecialty, and physician volume. No TTE was ordered for appropriate indications in 3% and was associated with indications including prior test result (primarily abnormal electrocardiographic findings), older patients, and physician subspecialty other than generalist or imaging. There was high variability in TTE utilization among centers.

CONCLUSIONS

There was a reasonable degree of concordance between AUC and clinical practice before AUC publication. Several patient and physician factors were associated with discordance with the AUC. These findings should be considered in efforts to disseminate the AUC and in the development of future iterations. The causes for variation among centers deserve further exploration.

Appropriate use of echocardiography and relation to clinical decision making in both inpatients and outpatients in a developing country

BACKGROUND

Use of echocardiography (echo) has exponentially increased in recent decades. Concerned about this scientific society developed appropriate use criteria (AUC). Clinical management still suffers geographical variations, and no objective data are available about echo AUC in developing countries. We aimed to evaluate (1) the appropriateness of referrals and (2) their relation to changes in clinical decision management.

METHODS

Prospective analysis of referrals from January to December 2014. Appropriateness and endpoints analyzed in different time points from medical archives.

ENDPOINTS

(1) change in the diagnosis, (2) indication for another method to complete the diagnosis, (3) change in clinical treatment, (4) indication for a treatment intervention, or (5) no change in management. Descriptive statistical analysis, Fisher's or chi-square tests, and Cox regression used as appropriate (significance if P < .05).

RESULTS

One thousand one hundred referrals were analyzed (55.5 ± 16.1 years, 44.6% male). 80.5% of referrals were appropriate (A), 11.2% "Rarely Appropriate" (RA), and 8.3% "May Be Appropriate" (MBA). Proportion of (A) did not differ between modalities (TTE-80.5% vs TEE-87.7% vs STR-81.2%, P = .67). (A) referrals were more related to clinical decision than (RA)+(MBA) (38.9% [A] vs 15% [RA]+[MBA], P < .001). The most frequent clinical indications of (RA) and (MBA) TTE were reevaluation of ventricular function without clinical change (AUC 10 and 11) and search of infectious endocarditis when low clinical probability (53).

CONCLUSIONS

In a developing country, appropriateness of echo was similar to the United States and Europe. However, a significant proportion of referrals were still (RA) or (MBA), with no effect in clinical management. Controlling referrals 10, 11, and 53 can optimize echo use in developing countries.

Appropriate Use Criteria for Echocardiography: Evolving Applications in the Era of Value-Based Healthcare

The current climate in healthcare is increasingly emphasizing a value-based approach to diagnostic testing. Cardiac imaging, including echocardiography, has been a primary target of ongoing reforms in healthcare delivery and reimbursement. The Appropriate Use Criteria (AUC) for echocardiography is a physician-derived tool intended to guide utilization in optimal patient care. To date, the AUC have primarily been employed solely as justification for reimbursement, though evolving broader applications to guide clinical decision-making suggest a far more valuable role in the delivery of high-quality and high-value healthcare.

Appropriate use criteria for echocardiography in the Netherlands

Introduction

Appropriate use criteria (AUC) for echocardiography based on clinical scenarios were previously published by an American Task Force. We determined whether members of the Dutch Working Group on Echocardiography (WGE) would rate these scenarios in a similar way.

Methods

All 32 members of the WGE were invited to judge clinical scenarios independently using a blanked version of the previously published American version of AUC for echocardiography. During a face-to-face meeting, consensus about the final rating was reached by open discussion for each indication. For reasons of simplicity, the scores were reduced from a 9-point scale to a 3-point scale (indicating an appropriate, uncertain or inappropriate echo indication, respectively).

Results

Nine cardiologist members of the WGE reported their judgment on the echo cases (n = 153). Seventy-one indications were rated as appropriate, 35 were rated as uncertain, and 47 were rated as inappropriate. In 5% of the cases the rating was opposite to that in the original (appropriate compared with inappropriate and vice versa), whereas in 20% judgements differed by 1 level of appropriateness. After the consensus meeting, the appropriateness of 7 (5%) cases was judged differently compared with the original paper.

Conclusions

Echocardiography was rated appropriate when it is applied for an initial diagnosis, a change in clinical status or a change in patient management. However, in about 5% of the listed clinical scenarios, members of the Dutch WGE rated the AUC for echocardiography differently as compared with their American counterparts. Further research is warranted to analyse this decreased external validity.

Electronic supplementary material

The online version of this article (doi: 10.1007/s12471-017-0960-9) contains supplementary material, which is available to authorized users.