Implementation of an Automated Radiology Recommendation-Tracking Engine for Abdominal Imaging Findings of Possible Cancer

Communication of Incidental Imaging Findings on Inpatient Discharge Summaries After Implementation of Electronic Health Record Notification System

OBJECTIVES

Inadequate follow-up of incidental imaging findings (IIFs) can result in poor patient outcomes, patient dissatisfaction, and provider malpractice. At our institution, radiologists flag IIFs during report dictation to trigger electronic health record (EHR) notifications to providers and patients. Nurse coordinators directly contact patients or their primary care physicians (PCPs) regarding IIFs if follow-up is not completed within the recommended time frame. Despite these interventions, many patients and their PCPs remain unaware of IIFs. In an effort to improve awareness of IIFs, we aim to investigate communication of IIFs on inpatient discharge summaries after implementation of our EHR notification system.

METHODS

Inpatient records with IIFs from 2018 to 2021 were retrospectively reviewed to determine type of IIFs, follow-up recommendations, and mention of IIFs on discharge summaries. Nurse coordinators spoke to patients and providers to determine their awareness of IIFs.

RESULTS

Incidental imaging findings were reported in 51% of discharge summaries (711/1383). When nurse coordinators called patients and PCPs regarding IIFs at the time follow-up was due, the patients and PCPs were aware of 79% of IIFs (1096/1383).

CONCLUSIONS

With implementation of EHR notifications to providers regarding IIFs, IIFs were included in 51% of discharge summaries. Lack of inclusion of IIFs on discharge summaries could be related to transitions of care within hospitalization, provider alert fatigue, and many diagnostic testing results to distill. These findings demonstrate the need to improve communication of IIFs, possibly via automating mention of IIFs on discharge summaries, and the need for care coordinators to follow up on IIFs.

Population Health Management in Radiology: Economic Considerations

There is a growing emphasis on population health management (PHM) in the United States, in part because it has the worst health outcomes indices among high-income countries despite spending by far the most on health care. Successful PHM is expected to lead to a healthier population with reduced health care utilization and cost. The role of radiology in PHM is increasingly being recognized, including efforts in care coordination, secondary prevention, and appropriate imaging utilization, among others. To further discuss economic considerations for PHM, we must understand the evolving health care payer environment, which combines fee-for-service and increasingly, an alternative payment model framework developed by the Health Care Payment Learning and Action Network. In considering the term "value-based care," perceived value needs to accrue to those who ultimately pay for care, which is more commonly employers and the government. This perspective drives the design of alternative payment models and thus should be taken into consideration to ensure sustainable practice models.

Informatics Solutions to Mitigate Legal Risk Associated With Communication Failures

Communication failures are a documented cause of malpractice litigation against radiologists. As imaging volumes have increased, and with them the number of findings requiring further workup, radiologists are increasingly expected to communicate with ordering clinicians. However, communication may be unsuccessful for a variety of reasons that expose radiologists to potential malpractice risk. Informatics solutions have the potential to improve communication and decrease this risk. We discuss human-powered, purely automated, and hybrid approaches to closing the communications loop. In addition, we describe the Patient Test Results Information Act (Pennsylvania Act 112) and its implications for closing the loop on noncritical actionable findings.

Use of artificial intelligence in emergency radiology: An overview of current applications, challenges, and opportunities

The value of artificial intelligence (AI) in healthcare has become evident, especially in the field of medical imaging. The accelerated pace and acuity of care in the Emergency Department (ED) has made it a popular target for artificial intelligence-driven solutions. Software that helps better detect, report, and appropriately guide management can ensure high quality patient care while enabling emergency radiologists to better meet the demands of quick turnaround times. Beyond diagnostic applications, AI-based algorithms also have the potential to optimize other important steps within the ED imaging workflow. This review will highlight the different types of AI-based applications currently available for use in the ED, as well as the challenges and opportunities associated with their implementation.

Artificial Intelligence in Diagnostic Radiology: Where Do We Stand, Challenges, and Opportunities

ABSTRACT

Artificial intelligence (AI) is the most revolutionizing development in the health care industry in the current decade, with diagnostic imaging having the greatest share in such development. Machine learning and deep learning (DL) are subclasses of AI that show breakthrough performance in image analysis. They have become the state of the art in the field of image classification and recognition. Machine learning deals with the extraction of the important characteristic features from images, whereas DL uses neural networks to solve such problems with better performance. In this review, we discuss the current applications of machine learning and DL in the field of diagnostic radiology.Deep learning applications can be divided into medical imaging analysis and applications beyond analysis. In the field of medical imaging analysis, deep convolutional neural networks are used for image classification, lesion detection, and segmentation. Also used are recurrent neural networks when extracting information from electronic medical records and to augment the use of convolutional neural networks in the field of image classification. Generative adversarial networks have been explicitly used in generating high-resolution computed tomography and magnetic resonance images and to map computed tomography images from the corresponding magnetic resonance imaging. Beyond image analysis, DL can be used for quality control, workflow organization, and reporting.In this article, we review the most current AI models used in medical imaging research, providing a brief explanation of the various models described in the literature within the past 5 years. Emphasis is placed on the various DL models, as they are the most state-of-art in imaging analysis.

Artificial Intelligence for Quality Improvement in Radiology

Artificial intelligence (AI) and informatics promise to improve the quality and efficiency of diagnostic radiology but will require substantially more standardization and operational coordination to realize and measure those improvements. As radiology steps into the AI-driven future we should work hard to identify the needs and desires of our customers and develop process controls to ensure we are meeting them. Rather than focusing on easy-to-measure turnaround times as surrogates for quality, AI and informatics can support more comprehensive quality metrics, such as ensuring that reports are accurate, readable, and useful to patients and health care providers.

Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers

Background

It is essential for radiologists to communicate actionable findings to the referring clinicians reliably. Natural language processing (NLP) has been shown to help identify free-text radiology reports including actionable findings. However, the application of recent deep learning techniques to radiology reports, which can improve the detection performance, has not been thoroughly examined. Moreover, free-text that clinicians input in the ordering form (order information) has seldom been used to identify actionable reports. This study aims to evaluate the benefits of two new approaches: (1) bidirectional encoder representations from transformers (BERT), a recent deep learning architecture in NLP, and (2) using order information in addition to radiology reports.

Methods

We performed a binary classification to distinguish actionable reports (i.e., radiology reports tagged as actionable in actual radiological practice) from non-actionable ones (those without an actionable tag). 90,923 Japanese radiology reports in our hospital were used, of which 788 (0.87%) were actionable. We evaluated four methods, statistical machine learning with logistic regression (LR) and with gradient boosting decision tree (GBDT), and deep learning with a bidirectional long short-term memory (LSTM) model and a publicly available Japanese BERT model. Each method was used with two different inputs, radiology reports alone and pairs of order information and radiology reports. Thus, eight experiments were conducted to examine the performance.

Results

Without order information, BERT achieved the highest area under the precision-recall curve (AUPRC) of 0.5138, which showed a statistically significant improvement over LR, GBDT, and LSTM, and the highest area under the receiver operating characteristic curve (AUROC) of 0.9516. Simply coupling the order information with the radiology reports slightly increased the AUPRC of BERT but did not lead to a statistically significant improvement. This may be due to the complexity of clinical decisions made by radiologists.

Conclusions

BERT was assumed to be useful to detect actionable reports. More sophisticated methods are required to use order information effectively.

Determining Follow-Up Imaging Study Using Radiology Reports

Radiology reports often contain follow-up imaging recommendations. Failure to comply with these recommendations in a timely manner can lead to delayed treatment, poor patient outcomes, complications, unnecessary testing, lost revenue, and legal liability. The objective of this study was to develop a scalable approach to automatically identify the completion of a follow-up imaging study recommended by a radiologist in a preceding report. We selected imaging-reports containing 559 follow-up imaging recommendations and all subsequent reports from a multi-hospital academic practice. Three radiologists identified appropriate follow-up examinations among the subsequent reports for the same patient, if any, to establish a ground-truth dataset. We then trained an Extremely Randomized Trees that uses recommendation attributes, study meta-data and text similarity of the radiology reports to determine the most likely follow-up examination for a preceding recommendation. Pairwise inter-annotator F-score ranged from 0.853 to 0.868; the corresponding F-score of the classifier in identifying follow-up exams was 0.807. Our study describes a methodology to automatically determine the most likely follow-up exam after a follow-up imaging recommendation. The accuracy of the algorithm suggests that automated methods can be integrated into a follow-up management application to improve adherence to follow-up imaging recommendations. Radiology administrators could use such a system to monitor follow-up compliance rates and proactively send reminders to primary care providers and/or patients to improve adherence.

Automated Detection of Radiology Reports that Require Follow-up Imaging Using Natural Language Processing Feature Engineering and Machine Learning Classification

While radiologists regularly issue follow-up recommendations, our preliminary research has shown that anywhere from 35 to 50% of patients who receive follow-up recommendations for findings of possible cancer on abdominopelvic imaging do not return for follow-up. As such, they remain at risk for adverse outcomes related to missed or delayed cancer diagnosis. In this study, we develop an algorithm to automatically detect free text radiology reports that have a follow-up recommendation using natural language processing (NLP) techniques and machine learning models. The data set used in this study consists of 6000 free text reports from the author's institution. NLP techniques are used to engineer 1500 features, which include the most informative unigrams, bigrams, and trigrams in the training corpus after performing tokenization and Porter stemming. On this data set, we train naive Bayes, decision tree, and maximum entropy models. The decision tree model, with an F1 score of 0.458 and accuracy of 0.862, outperforms both the naive Bayes (F1 score of 0.381) and maximum entropy (F1 score of 0.387) models. The models were analyzed to determine predictive features, with term frequency of n-grams such as "renal neoplasm" and "evalu with enhanc" being most predictive of a follow-up recommendation. Key to maximizing performance was feature engineering that extracts predictive information and appropriate selection of machine learning algorithms based on the feature set.

Impact of PTRIA (Patient Test Result Information Act) on patient follow up management

OBJECTIVE

We aim to study if direct patient notification in accordance with the Patient Test Results Information Act (Act 112) in Pennsylvania leads to decreased loss to follow up and prompt management of actionable imaging findings.

METHODS

For this IRB-approved study, radiology reports were randomly identified using the Nuance mPower™ search engine. The actionable finding group (prior to Act-112) contained 300 patients for which a voice notification was sent by radiologists to alert ordering physicians about significant imaging findings. The PTRIA group (after Act-112) contained 300 patients who were mailed a standardized letter one day after the final report was issued. The electronic medical records were reviewed to evaluate how patients were managed.

RESULTS

There was no difference in loss to follow up rates and time to follow up completion between the two groups. In both groups, 34% of patients were lost to follow up in transition of care from generalists to specialists; 24% cases were lost to follow up when imaging findings were not in the area of the initial ordering provider expertise.

CONCLUSION

The goal of Act 112 is to increase patients' role in the timely management of their significant medical conditions and prevent medical errors, specifically loss to follow up. Our study suggests that presumed patients' awareness does not contribute to improved follow up rates or decreased time to a follow up visit. 13% of patients are lost to follow up in both groups. A tracking system is required to prevent delayed management of the significant findings.

Engaging patients and families in pediatric radiology

While patient and family-centered care (PFCC) is currently a hot topic in medicine, it has long been a specific focus of pediatrics. The concept of PFCC includes a change in culture where physicians and patients move away from paternalism and instead view patients and families as partners in care. Although there are many ways in which adult-focused radiologists can learn from pediatric radiologists as leaders in PFCC, there remain many opportunities for improvement for all radiologists.

Trends of Follow-Up Recommendations Made on Musculoskeletal MRI Reports

OBJECTIVE. The purpose of this article was to analyze trends in follow-up recommendations made on musculoskeletal MRI reports. MATERIALS AND METHODS. An IRB-approved retrospective study identified 790 musculoskeletal MRI reports from our database between January 1, 2016, and January 1, 2018, containing follow-up recommendations made by the interpreting radiologist. Metadata were automatically extracted and classification of the recommendations was performed by manual review. Clinical outcome data were collected from the electronic health record. After exclusion criteria were applied, 654 reports were included in the study. Descriptive statistics, Fisher exact tests, and chi-square tests were used for analysis. RESULTS. Clinicians acknowledged 83% and followed 73% of the recommendations. Follow-up compliance varied with the type of recommendation made: 98% for clinical intervention versus 67% for additional imaging (p < 0.001). Subspecialties acknowledged and followed recommendations at different rates: 92% and 85% for internists versus 76% and 64% for orthopedists (p < 0.001 and p < 0.001), respectively. Patient age, practice setting, radiologist experience, recommendation conditionality, and specified follow-up time intervals made no difference in compliance rate (all p > 0.05). There was no difference in compliance rate among various pathologic findings of concern (p = 0.995). Compliance rate increased significantly after direct communication between the radiologist and clinician compared with when there was no direct communication (93% vs 71%, p < 0.001). Concern for neoplasm comprised the greatest number of unacknowledged recommendations (73%). CONCLUSION. Musculoskeletal MRI recommendations are followed independent of the finding of concern and compliance is lowest for requests of additional imaging. Direct communication improves compliance and may be particularly helpful for orthopedic referrers.

Automated Organ-Level Classification of Free-Text Pathology Reports to Support a Radiology Follow-up Tracking Engine

PURPOSE

To evaluate the performance of machine learning algorithms on organ-level classification of semistructured pathology reports, to incorporate surgical pathology monitoring into an automated imaging recommendation follow-up engine.

MATERIALS AND METHODS

This retrospective study included 2013 pathology reports from patients who underwent abdominal imaging at a large tertiary care center between 2012 and 2018. The reports were labeled by two annotators as relevant to four abdominal organs: liver, kidneys, pancreas and/or adrenal glands, or none. Automated classification methods were compared: simple string matching, random forests, extreme gradient boosting, support vector machines, and two neural network architectures-convolutional neural networks and long short-term memory networks. Three methods from the literature were used to provide interpretability and qualitative validation of the learned network features.

RESULTS

The neural networks performed well on the four-organ classification task (F1 score: 96.3% for convolutional neural network and 96.7% for long short-term memory vs 89.9% for support vector machines, 93.9% for extreme gradient boosting, 82.8% for random forests, and 75.2% for simple string matching). Multiple methods were used to visualize the decision-making process of the network, verifying that the networks used similar heuristics to a human annotator. The neural networks were able to classify, with a high degree of accuracy, pathology reports written in unseen formats, suggesting the networks had learned a generalizable encoding of the salient features.

CONCLUSION

Neural network-based approaches achieve high performance on organ-level pathology report classification, suggesting that it is feasible to use them within automated tracking systems.© RSNA, 2019Supplemental material is available for this article.See also the commentary by Liu in this issue.

Automated Tracking of Follow-Up Imaging Recommendations

OBJECTIVE. Radiology reports often contain follow-up imaging recommendations. Failure to comply with these recommendations in a timely manner can lead to poor patient outcomes, complications, and legal liability. As such, the primary objective of this research was to determine adherence rates to follow-up recommendations. MATERIALS AND METHODS. Radiology-related examination data, including report text, for examinations performed between June 1, 2015, and July 31, 2017, were extracted from the radiology departments at the University of Washington (UW) and Lahey Hospital and Medical Center (LHMC). The UW dataset contained 923,885 examinations, and the LHMC dataset contained 763,059 examinations. A 1-year period was used for detection of imaging recommendations and up to 14-months for the follow-up examination to be performed. RESULTS. On the basis of an algorithm with 97.9% detection accuracy, the follow-up imaging recommendation rate was 11.4% at UW and 20.9% at LHMC. Excluding mammography examinations, the overall follow-up imaging adherence rate was 51.9% at UW (range, 44.4% for nuclear medicine to 63.0% for MRI) and 52.0% at LHMC (range, 30.1% for fluoroscopy to 63.2% for ultrasound) using a matcher algorithm with 76.5% accuracy. CONCLUSION. This study suggests that follow-up imaging adherence rates vary by modality and between sites. Adherence rates can be influenced by various legitimate factors. Having the capability to identify patients who can benefit from patient engagement initiatives is important to improve overall adherence rates. Monitoring of follow-up adherence rates over time and critical evaluation of variation in recommendation patterns across the practice can inform measures to standardize and help mitigate risk.