Impact of a Health Information Technology–Enabled Appropriate Use Criterion on Utilization of Emergency Department CT for Renal Colic

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

Features and functions of decision support systems for appropriate diagnostic imaging: a scoping review

BACKGROUND

Diagnostic imaging decision support (DI-DS) systems could be effective tools for reducing inappropriate diagnostic imaging examinations. Since effective design and evaluation of these systems requires in-depth understanding of their features and functions, the present study aims to map the existing literature on DI-DS systems to identify features and functions of these systems.

METHODS

The search was performed using Scopus, Embase, PubMed, Web of Science, and Cochrane Central Registry of Controlled Trials (CENTRAL) and was limited to 2000 to 2021. Analytical studies, descriptive studies, reviews and book chapters that explicitly addressed the functions or features of DI-DS systems were included.

RESULTS

A total of 6,046 studies were identified. Out of these, 55 studies met the inclusion criteria. From these, 22 functions and 22 features were identified. Some of the identified features were: visibility, content chunking/grouping, deployed as a multidisciplinary program, clinically valid and relevant feedback, embedding current evidence, and targeted recommendations. And, some of the identified functions were: displaying an appropriateness score, recommending alternative or more appropriate imaging examination(s), providing recommendations for next diagnostic steps, and providing safety alerts.

CONCLUSIONS

The set of features and functions obtained in the present study can provide a basis for developing well-designed DI-DS systems, which could help to improve adherence to diagnostic imaging guidelines, minimize unnecessary costs, and improve the outcome of care through appropriate diagnosis and on-time care delivery.

Applying Natural Language Processing to Textual Data From Clinical Data Warehouses: Systematic Review

BACKGROUND

In recent years, health data collected during the clinical care process have been often repurposed for secondary use through clinical data warehouses (CDWs), which interconnect disparate data from different sources. A large amount of information of high clinical value is stored in unstructured text format. Natural language processing (NLP), which implements algorithms that can operate on massive unstructured textual data, has the potential to structure the data and make clinical information more accessible.

OBJECTIVE

The aim of this review was to provide an overview of studies applying NLP to textual data from CDWs. It focuses on identifying the (1) NLP tasks applied to data from CDWs and (2) NLP methods used to tackle these tasks.

METHODS

This review was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched for relevant articles in 3 bibliographic databases: PubMed, Google Scholar, and ACL Anthology. We reviewed the titles and abstracts and included articles according to the following inclusion criteria: (1) focus on NLP applied to textual data from CDWs, (2) articles published between 1995 and 2021, and (3) written in English.

RESULTS

We identified 1353 articles, of which 194 (14.34%) met the inclusion criteria. Among all identified NLP tasks in the included papers, information extraction from clinical text (112/194, 57.7%) and the identification of patients (51/194, 26.3%) were the most frequent tasks. To address the various tasks, symbolic methods were the most common NLP methods (124/232, 53.4%), showing that some tasks can be partially achieved with classical NLP techniques, such as regular expressions or pattern matching that exploit specialized lexica, such as drug lists and terminologies. Machine learning (70/232, 30.2%) and deep learning (38/232, 16.4%) have been increasingly used in recent years, including the most recent approaches based on transformers. NLP methods were mostly applied to English language data (153/194, 78.9%).

CONCLUSIONS

CDWs are central to the secondary use of clinical texts for research purposes. Although the use of NLP on data from CDWs is growing, there remain challenges in this field, especially with regard to languages other than English. Clinical NLP is an effective strategy for accessing, extracting, and transforming data from CDWs. Information retrieved with NLP can assist in clinical research and have an impact on clinical practice.

Combining text mining with clinical decision support in clinical practice: a scoping review

OBJECTIVE

Combining text mining (TM) and clinical decision support (CDS) could improve diagnostic and therapeutic processes in clinical practice. This review summarizes current knowledge of the TM-CDS combination in clinical practice, including their intended purpose, implementation in clinical practice, and barriers to such implementation.

MATERIALS AND METHODS

A search was conducted in PubMed, EMBASE, and Cochrane Library databases to identify full-text English language studies published before January 2022 with TM-CDS combination in clinical practice.

RESULTS

Of 714 identified and screened unique publications, 39 were included. The majority of the included studies are related to diagnosis (n = 26) or prognosis (n = 11) and used a method that was developed for a specific clinical domain, document type, or application. Most of the studies selected text containing parts of the electronic health record (EHR), such as reports (41%, n = 16) and free-text narratives (36%, n = 14), and 23 studies utilized a tool that had software "developed for the study". In 15 studies, the software source was openly available. In 79% of studies, the tool was not implemented in clinical practice. Barriers to implement these tools included the complexity of natural language, EHR incompleteness, validation and performance of the tool, lack of input from an expert team, and the adoption rate among professionals.

DISCUSSION/CONCLUSIONS

The available evidence indicates that the TM-CDS combination may improve diagnostic and therapeutic processes, contributing to increased patient safety. However, further research is needed to identify barriers to implementation and the impact of such tools in clinical practice.

Effect of an ultrasound-first clinical decision tool in emergency department patients with suspected nephrolithiasis: A randomized trial

INTRODUCTION

Previously, we found that the use of ultrasonography for patients with suspected nephrolithiasis resulted in similar outcomes and less radiation exposure vs. CT scan. In this study, we evaluated the implementation of an ultrasound-first clinical decision support (CDS) tool in patients with suspected nephrolithiasis.

METHODS

This randomized trial was conducted at an academic emergency department (ED). We implemented the ultrasound-first CDS tool, deployed when an ED provider placed a CT order for suspected nephrolithiasis. Providers were randomized to receiving the CDS tool vs. usual care. The primary outcome was receipt of CT during the index ED visit. Secondary outcomes included radiation dose and ED revisit.

RESULTS

64 ED Providers and 254 patients with suspected nephrolithiasis were enrolled from January 2019 through Dec 2020. The US-First CDS tool was deployed for 128 patients and was not deployed for 126 patients. 86.7% of patients in the CDS arm received a CT vs. 94.4% in the usual care arm, resulting in an absolute risk difference of -7.7% (-14.8 to -0.6%). Mean radiation dose in the CDS arm was 6.8 mSv (95% CI 5.7-7.9 mSv) vs. 6.1 mSv (95% CI 5.1-7.1 mSv) in the usual care arm. The CDS arm did not result in increased ED revisits, CT scans, or hospitalizations at 7 or 30 days.

CONCLUSIONS AND RELEVANCE

Implementation of the US-first CDS tool resulted in lower CT use for ED patients with suspected nephrolithiasis. The use of this decision support may improve the evaluation of a common problem in the ED.

TRIAL REGISTRATION

ClinicalTrials.gov#NCT03461536.

Radiation concerns in frequent flyer patients: Should imaging history influence decisions about recurrent imaging?

Radiation risks from diagnostic imaging have captured the attention of patients and medical practitioners alike, yet it remains unclear how these considerations can best be incorporated into clinical decision making. This manuscript presents a framework to consider these issues in a potentially at-risk population, the so called "frequent flyer" patients undergoing a large amount of recurrent imaging over time. Radiation risks from the low-dose exposures of diagnostic imaging are briefly reviewed, as applied to recurrent exposures. Some scenarios are then explored in which it may be helpful to incorporate knowledge of a patient's imaging history. There is no simple or uniformly applicable approach to these challenging and often nuanced clinical decisions. The complexity and variability of the underlying disease states and trajectories argues against alerting mechanisms based on a simple cumulative dose threshold. Awareness of imaging history may instead be beneficial in encouraging physicians and patients to take the long view, and to identify those populations of frequent flyers that might benefit from alternative imaging strategies.

Lessons From the Free-Text Epidemic: Opportunities to Optimize Deployment of Imaging Clinical Decision Support

OBJECTIVE

The Protecting Access to Medicare Act of 2014 requires clinicians to consult Appropriate Use Criteria (AUC) when ordering advanced imaging procedures. Free-text order indications are available when there is no applicable structured indication but are unscored by the AUC. We determined the proportion of free-text indications among all advanced imaging orders and the proportion of free-text indications that could be mapped to a single structured indication.

METHODS

All outpatient advanced diagnostic imaging orders placed in a large multisite health system were recorded after initial AUC deployment (November 20, 2017, to December 19, 2017). Clinicians were prompted upon order entry to select a structured indication or enter a free-text indication. We manually reviewed the two imaging examinations with the highest rate of free-text indications: enhanced CT abdomen/pelvis and unenhanced CT head. Regression analysis examined differences in patient-, imaging-, context-, and provider-level characteristics between scored and unscored examinations.

RESULTS

Among all 39,533 orders for advanced imaging procedures, 59% (23,267 of 39,533) were unscored by the system. The regression model c-statistic (0.50-0.55) demonstrated poor model fit to evaluate for differences between scored and unscored examinations. Free-text indications were found in 71% (16,440 of 23,267) of unscored examinations and 42% (16,440 of 39,533) of all examinations. Manual review of all 1,693 CT abdomen/pelvis and 1,527 CT head examinations with free-text indications revealed that 3,132 free-text indications (97%) could be mapped to a single existing structured indication.

DISCUSSION

Of all initially placed outpatient advanced imaging procedure orders, 42% included free-text indications and 97% of manually reviewed free-text indications could be mapped to a single structured indication.

Radiology Patient Outcome Measures: Impact of a Departmental Pay-for-Performance Initiative on Key Quality and Safety Measures

OBJECTIVE

Assess impact of a multifaceted pay-for-performance (PFP) initiative on radiologists' behavior regarding key quality and safety measures.

METHODS

This institutional review board-approved prospective study was performed at a large, 12-division urban academic radiology department. Radiology patient outcome measures were implemented October 1, 2017, measuring report signature timeliness, critical results communication, and generation of peer-learning communications between radiologists. Subspecialty division-wide and individual radiologist targets were specified, performance was transparently communicated on an intranet dashboard updated daily, and performance was financially incentivized (5% of salary) quarterly. We compared outcomes 12 months pre- versus 12 months post-PFP implementation. Primary outcome was monthly 90th percentile time from scan completion to final report signature (CtoF). Secondary outcomes were percentage timely closed-loop communication of critical results and number of division-wide peer-learning communications. Statistical process control analysis and parallel coordinates charts were used to assess for temporal trends.

RESULTS

In all, 144 radiologists generated 1,255,771 reports (613,273 pre-PFP) during the study period. Monthly 90th percentile CtoF exhibited an absolute decrease of 4.4 hours (from 21.1 to 16.7 hours) and a 20.9% relative decrease post-PFP. Statistical process control analysis demonstrated significant decreases in 90th percentile CtoF post-PFP, sustained throughout the study period (P < .003). Between 95% (119 of 125, July 1, 2018, to September 30, 2018) and 98.4% (126 of 128, October 1, 2017, to December 31, 2017) of radiologists achieved >90% timely closure of critical alerts; all divisions exceeded the target of 90 peer-learning communications each quarter (range: 97-472) after January 1, 2018.

DISCUSSION

Implementation of a multifaceted PFP initiative using well-defined radiology patient outcome measures correlated with measurable improvements in radiologist behavior regarding key quality and safety parameters.

Impact of Radiologist-Driven Change-Order Requests on Outpatient CT and MRI Examinations

PURPOSE

To assess impact of electronic medical record-embedded radiologist-driven change-order request on outpatient CT and MRI examinations.

METHODS

Outpatient CT and MRI requests where an order change was requested by the protocoling radiologist in our tertiary care center, from April 11, 2017, to January 3, 2018, were analyzed. Percentage and categorization of requested order change, provider acceptance of requested change, patient and provider demographics, estimated radiation exposure reduction, and cost were analyzed. P < .05 was used for statistical significance.

RESULTS

In 79,310 outpatient studies in which radiologists determined protocol, change-order requests were higher for MRI (5.2%, 1,283 of 24,553) compared with CT (2.9%, 1,585 of 54,757; P < .001). Provider approval of requested change was equivalent for CT (82%, 1,299 of 1,585) and MRI (82%, 1,052 of 1,283). Change requests driven by improper contrast media utilization were most common and different between CT (76%, 992 of 1,299) and MRI (65%, 688 of 1,052; P < .001). Changing without and with intravenous contrast orders to with contrast only was most common for CT (39%, 505 of 1,299) and with and without intravenous contrast to without contrast only was most common for MRI (26%, 274 of 1,052; P < .001). Of approved changes in CT, 51% (661 of 1,299) resulted in lower radiation exposure. Approved changes frequently resulted in less costly examinations (CT 67% [799 of 1,198], MRI 48% [411 of 863]).

CONCLUSION

Outpatient CT and MRI orders are deemed incorrect in 2.9% to 5% of cases. Radiologist-driven change-order request for CT and MRI are well accepted by ordering providers and decrease radiation exposure associated with imaging.

Patients undergoing recurrent CT exams: assessment of patients with non-malignant diseases, reasons for imaging and imaging appropriateness

OBJECTIVE

To determine percent of patients without malignancy and ≤ 40 years of age with high cumulative radiation doses through recurrent CT exams and assess imaging appropriateness.

METHODS

From the cohort of patients who received cumulative effective dose (CED) of ≥ 100 mSv over a 5-year period, a sub-set was identified with non-malignant disease. The top 50 clinical indications leading to multiple CTs were determined. Clinical decision support (CDS) system scores were analyzed using a widely adopted standard of 1-3 (red) as "not usually appropriate," 4-6 (yellow) "may or may not be appropriate," and 7-9 (green) "usually appropriate." Clinicians reviewed patient records to assess compliance with appropriate use criteria (AUC).

RESULTS

9.6% of patients in our series were with non-malignant conditions and 1.4% with age ≤ 40 years. CDS scores (rounded) were 2% red, 38% yellow, 27% green, and 33% unscored CTs. Clinical society guidelines for CT exams, wherever available, were followed in 87.5 to 100% of cases. AUCs were not available for several clinical indications as also referral guidelines for serial CT imaging. More than half of CT exams were unrelated to follow-up of a primary chronic disease.

CONCLUSIONS

We are faced with a situation wherein patients in age ≤ 40 years require or are thought to require many CT exams over the course of a few years but the radiation risk creates concern. There is a fair number of conditions for which AUC are not available. Suggested solutions include development of CT scanners with lesser radiation dose and further development of appropriateness criteria.

KEY POINTS

We are faced with a situation wherein patients in age group 0-40 years and with non-malignant diagnosis require or are thought to require many CT exams over the course of a few years. More than half of CT exams were unrelated to follow-up of a primary chronic disease. Imaging guidelines and appropriateness use criteria are not available for many conditions. Wherever available, they are for initial work-up and diagnosis and there is a lack of guidance on serial CT imaging.

Randomized Clinical Trial of a Clinical Decision Support Tool for Improving the Appropriateness Scores for Ordering Imaging Studies in Primary and Specialty Care Ambulatory Clinics

OBJECTIVE. The objective of our study was to evaluate whether the use of a clinical decision support (CDS) tool improved the appropriateness scores of orders for advanced imaging in clinical practice. MATERIALS AND METHODS. We used a stepped-wedge, cluster randomized clinical trial to evaluate the effectiveness of a CDS tool in an integrated health care system. Clinicians entered structured indications for each CT and MRI order, and the indications were electronically scored against appropriateness criteria to assign an appropriateness score. We compared the proportion of orders with adjusted appropriateness scores of 7 or greater (on a 1-9 scale) before and after activation of best practice alerts (BPAs) triggered for orders with low or marginal appropriateness scores. Secondary outcomes included the rate per month of orders for advanced imaging and the proportion of orders for which the radiology department requested changes. RESULTS. Between October 2015 and February 2016, 941 clinicians ordered 22,279 CT or MRI studies that met eligibility criteria. Before activation of the BPA, the mean proportion of appropriate orders (adjusted for time and clinic effect) was 77.0% (95% CI, 75.5-78.4%), which increased to 80.1% (95% CI, 78.7-81.5%) after activation (p = 0.001). There was no significant change in the rate of orders per month for advanced imaging. The proportion of order changes requested by the radiology department decreased from 5.7% (95% CI, 5.6-5.9%) before CDS implementation to 5.3% (95% CI, 5.1-5.5%) after CDS implementation (p < 0.001). CONCLUSION. Using an evidence-based CDS tool in clinical practice was associated with a modest but significant improvement in the appropriateness scores of advanced imaging orders.

What Physicians and Health Organizations Should Know About Mandated Imaging Appropriate Use Criteria

The Appropriate Use Criteria Program, enacted by the Centers for Medicare & Medicaid Services in response to the Protecting Access to Medicare Act of 2014 (PAMA), aims to reduce inappropriate and unnecessary imaging by mandating use of clinical decision support (CDS) by all providers who order advanced imaging examinations (magnetic resonance imaging; computed tomography; and nuclear medicine studies, including positron emission tomography). Beginning 1 January 2020, documentation of an interaction with a certified CDS system using approved appropriate use criteria will be required on all Medicare claims for advanced imaging in all emergency department patients and outpatients as a prerequisite for payment. The Appropriate Use Criteria Program will initially cover 8 priority clinical areas, including several (such as headache and low back pain) commonly encountered by internal medicine providers. All providers and organizations that order and provide advanced imaging must understand program requirements and their options for compliance strategies. Substantial resources and planning will be needed to comply with PAMA regulations and avoid unintended negative consequences on workflow and payments. However, robust evidence supporting the desired outcome of reducing inappropriate use of advanced imaging is lacking.