Retrospective Evaluation of Artificial Intelligence Leveraging Free-Text Imaging Order Entry to Facilitate Federally Required Clinical Decision Support

Systematic review of interventions aimed at improving the quality of referrals to radiology

Despite ubiquitous use of medical imaging in daily medical practice, the quality of referrals varies significantly across a variety of practice types and locations. This systematic review summarises studies in the literature that have employed interventions aimed at improving radiology referrals, excluding clinical decision support software. A systematic review of literature was conducted in PubMed, EMBASE, Scopus, and Cochrane. Two reviewers independently identified studies for inclusion. All studies that included interventions with any outcome measure were included. Any irrelevant studies, non-English studies or not retrievable studies were excluded. Studies were grouped into Education, Feedback, Rationing, Penalties, and Other. The outcomes of the studies were summarised and qualitatively analysed due to anticipated heterogeneity. Four thousand six hundred and forty-two studies were identified throughout PubMed, EMBASE, Scopus, and Cochrane. One hundred and eighty-seven duplicates were removed and 4436 abstracts were screened. Two hundred and forty were identified on the first phase of the screening with 167 then excluded for non-relevancy. Seventy-five full studies were included in the final analysis following the addition of 2 additional studies. Fifty-seven studies were grouped into Education, 10 into Feedback, 4 into Rationing, 8 into Penalties, 9 into Other and 11 containing multiple. Eighty-four percent of the studies reported an improvement in the quality of the referrals. Despite a variable rate of quality referrals, there are many interventions that radiology departments across the world can utilise to improve the referral process.

Impact of Predictive Text Clinical Decision Support on Imaging Order Entry in the Emergency Department

PURPOSE

Imaging clinical decision support (CDS) is designed to assist providers in selecting appropriate imaging studies and is now federally required. The aim of this study was to understand the effect of CDS on decisions and workflows in the emergency department (ED).

METHODS

The authors' institution's order entry platform serves up structured indications for imaging orders. Imaging orders are scored by CDS on the basis of appropriate use criteria (AUC). CDS triggers alerts for imaging orders with low AUC scores. Because free text alone cannot be scored by CDS, an artificial intelligence predictive text (AIPT) module was implemented to guide the selection of structured indications when free-text indications are entered. A total of 17,355 imaging orders in the ED over 6 months were retrospectively analyzed.

RESULTS

CDS alerts for low AUC scores were triggered for 3% of all imaging study orders (522 of 17,355). Providers spent an average of 24 seconds interacting with alerts. In 18 of 522 imaging orders with alerts, alternative studies were ordered. After AIPT implementation, the percentage of unscored studies significantly decreased from 81% to 45% (P < .001).

CONCLUSIONS

In a quaternary academic ED, CDS alerts triggered by low AUC scores caused minimal increase in time spent on imaging order entry but had a relatively marginal impact on imaging study selection. AIPT implementation increased the number of scored studies and could potentially enhance CDS effects. CDS implementation enables the collection of novel data regarding which imaging studies receive low AUC scores. Future work could include exploring alternative models of CDS implementation to maximize its impact.

Impact of Artificial Intelligence-Assisted Indication Selection on Appropriateness Order Scoring for Imaging Clinical Decision Support

PURPOSE

The aim of this study was to assess appropriateness scoring and structured order entry after the implementation of an artificial intelligence (AI) tool for analysis of free-text indications.

METHODS

Advanced outpatient imaging orders in a multicenter health care system were recorded 7 months before (March 1, 2020, to September 21, 2020) and after (October 20, 2020, to May 13, 2021) the implementation of an AI tool targeting free-text indications. Clinical decision support score (not appropriate, may be appropriate, appropriate, or unscored) and indication type (structured, free-text, both, or none) were assessed. The χ2 and multivariate logistic regression adjusting for covariables with bootstrapping were used.

RESULTS

In total, 115,079 orders before and 150,950 orders after AI tool deployment were analyzed. The mean patient age was 59.3 ± 15.5 years, and 146,035 (54.9%) were women; 49.9% of orders were for CT, 38.8% for MR, 5.9% for nuclear medicine, and 5.4% for PET. After deployment, scored orders increased to 52% from 30% (P < .001). Orders with structured indications increased to 67.3% from 34.6% (P < .001). On multivariate analysis, orders were more likely to be scored after tool deployment (odds ratio [OR], 2.7, 95% CI, 2.63-2.78; P < .001). Compared with physicians, orders placed by nonphysician providers were less likely to be scored (OR, 0.80; 95% CI, 0.78-0.83; P < .001). MR (OR, 0.84; 95% CI, 0.82-0.87) and PET (OR, 0.12; 95% CI, 0.10-0.13) were less likely to be scored than CT (; P < .001). After AI tool deployment, 72,083 orders (47.8%) remained unscored, 45,186 (62.7%) with free-text-only indications.

CONCLUSIONS

Embedding AI assistance within imaging clinical decision support was associated with increased structured indication orders and independently predicted a higher likelihood of scored orders. However, 48% of orders remained unscored, driven by both provider behavior and infrastructure-related barriers.