Predictors of Completion of Clinically Necessary Radiologist-Recommended Follow-Up Imaging: Assessment Using an Automated Closed-Loop Communication and Tracking Tool

Financial Impact of a Radiology Safety Net Program for Resolution of Clinically Necessary Follow-up Imaging Recommendations

OBJECTIVE

Health care safety net (SN) programs can potentially improve patient safety and decrease risk associated with missed or delayed follow-up care, although they require financial resources. This study aimed to assess whether the revenue generated from completion of clinically necessary recommendations for additional imaging (RAI) made possible by an IT-enabled SN program could fund the required additional labor resources.

METHODS

Clinically necessary RAI generated October 21, 2019, to September 24, 2021, were tracked to resolution as of April 13, 2023. A new radiology SN team worked with existing schedulers and care coordinators, performing chart review and patient and provider outreach to ensure RAI resolution. We applied relevant Current Procedural Terminology, version 4 codes of the completed imaging examinations to estimate total revenue. Coprimary outcomes included revenue generated by total performed examinations and estimated revenue attributed to SN involvement. We used Student's t test to compare the secondary outcome, RAI time interval, for higher versus lower revenue-generating modalities.

RESULTS

In all, 24% (3,243) of eligible follow-up recommendations (13,670) required SN involvement. Total estimated revenue generated by performed recommended examinations was $6,116,871, with $980,628 attributed to SN. Net SN-generated revenue per 1.0 full-time equivalent was an estimated $349,768. Greatest proportion of performed examinations were cross-sectional modalities (CT, MRI, PET/CT), which were higher revenue-generating than non-cross-sectional modalities (x-ray, ultrasound, mammography), and had shorter recommendation time frames (153 versus 180 days, P < .001).

DISCUSSION

The revenue generated from completion of RAI facilitated by an IT-enabled quality and safety program supplemented by an SN team can fund the required additional labor resources to improve patient safety. Realizing early revenue may require 5 to 6 months postimplementation.

What Is Diagnostic Excellence? AJR Podcast Series on Diagnostic Excellence and Error, Episode 1

In this episode of the AJR Podcast Series on Diagnostic Excellence and Error, Francis Deng, MD, introduces the concept of diagnostic excellence and its relevance to radiologists. Patient-centered definitions of diagnostic error and conceptualizations of the diagnostic process are discussed.

The Impact of Closed-Loop Imaging on Actionable CT-Detected Breast Findings

PURPOSE

Closed-loop imaging programs (CLIPs) are designed to ensure that patients receive appropriate follow-up, but a review of incidental CT-detected breast findings in the setting of CLIPs has not been performed.

METHODS

A retrospective review was conducted of CT reports at a single academic institution from July 1, 2020, to January 31, 2022, to identify reports with recommendations for breast imaging follow-up. Medical records were reviewed to evaluate patient adherence to follow-up, CLIP intervention, subsequent BI-RADS assessment, and diagnosis. Adherence was defined as diagnostic breast imaging performed within 6 months of the CT recommendation.

RESULTS

Follow-up recommendations for breast imaging were included in CT report impressions for 311 patients. Almost half of patients (47.3% [147 of 311]) underwent follow-up breast imaging within 6 months, yielding breast cancer diagnoses in 12.9% (19 of 147) and a biopsy-proven positive predictive value of 65.5% (19 of 29). Most patients who returned for follow-up within 6 months did so without CLIP intervention. The majority of CT report impressions in the follow-up group (85.0% [125 of 147]) contained specific recommendations for "diagnostic breast imaging." For patients who did not receive follow-up, the CLIP team tracked all cases and intervened in 19.1% (28 of 147). The most common intervention was a phone call and/or fax to the primary care provider. Outpatient CT examination setting and specific recommendation for diagnostic breast imaging were significantly associated with higher follow-up adherence (P < .0001).

CONCLUSIONS

Actionable CT-detected breast findings require follow-up diagnostic breast imaging because of a relevant cancer detection rate of 12.9%. Although many patients return for breast imaging without intervention, almost half of patients did not receive follow-up and may account for a significant number of missed cancer diagnoses. Specific CT recommendation verbiage is associated with higher follow-up adherence, which can be addressed across settings even without CLIPs.

Patient-centered radiology: a roadmap for outpatient imaging

Creating a patient-centered experience is becoming increasingly important for radiology departments around the world. The goal of patient-centered radiology is to ensure that radiology services are sensitive to patients' needs and desires. This article provides a framework for addressing the patient's experience by dividing their imaging journey into three distinct time periods: pre-exam, day of exam, and post-exam. Each time period has aspects that can contribute to patient anxiety. Although there are components of the patient journey that are common in all regions of the world, there are also unique features that vary by location. This paper highlights innovative solutions from different parts of the world that have been introduced in each of these time periods to create a more patient-centered experience. CLINICAL RELEVANCE STATEMENT: Adopting innovative solutions that help patients understand their imaging journey and decrease their anxiety about undergoing an imaging examination are important steps in creating a patient centered imaging experience. KEY POINTS: • Patients often experience anxiety during their imaging journey and decreasing this anxiety is an important component of patient centered imaging. • The patient imaging journey can be divided into three distinct time periods: pre-exam, day of exam, and post-exam. • Although components of the imaging journey are common, there are local differences in different regions of the world that need to be considered when constructing a patient centered experience.

Factors Associated With Timeliness and Equity of Access to Outpatient MRI Examinations

OBJECTIVE

The role of MRI in guiding patients' diagnosis and treatment is increasing. Therefore, timely MRI performance prevents delays that can impact patient care. We assessed the timeliness of performing outpatient MRIs using the socio-ecological model approach and evaluated multilevel factors associated with delays.

METHODS

This institutional review board-approved study included outpatient MRI examinations ordered between October 1, 2021, and December 31, 2022, for performance at a large quaternary care health system. Mean order-to-performed (OtoP) interval (in days) and prolonged OtoP interval (defined as >10 days) for MRI orders with an expected date of 1 day to examination performance were measured. Logistic regression was used to assess patient-level (demographic and social determinants of health), radiology practice-level, and community-level factors associated with prolonged OtoP interval.

RESULTS

There were 126,079 MRI examination orders with expected performance within 1 day placed during the study period (56% of all MRI orders placed). After excluding duplicates, there were 97,160 orders for unique patients. Of the MRI orders, 48% had a prolonged OtoP interval, and mean OtoP interval was 18.5 days. Factors significantly associated with delay in MRI performance included public insurance (odds ratio [OR] = 1.11, P < .001), female gender (OR = 1.11, P < .001), radiology subspecialty (ie, cardiac, OR = 1.71, P < .001), and patients from areas that are most deprived (ie, highest Area Deprivation Index quintile, OR = 1.70, P < .001).

DISCUSSION

Nearly half of outpatient MRI orders were delayed, performed >10 days from the expected date selected by the ordering provider. Addressing multilevel factors associated with such delays may help enhance timeliness and equity of access to MRI examinations, potentially reducing diagnostic errors and treatment delays.

Actionability of Recommendations for Additional Imaging in Head and Neck Radiology

PURPOSE

The aims of this study were to measure the actionability of recommendations for additional imaging (RAIs) in head and neck CT and MRI, for which there is a near complete absence of best practices or guidelines; to identify the most common recommendations; and to assess radiologist factors associated with actionability.

METHODS

All head and neck CT and MRI radiology reports across a multi-institution, multipractice health care system from June 1, 2021, to May 31, 2022, were retrospectively reviewed. The actionability of RAIs was scored using a validated taxonomy. The most common RAIs were identified. Actionability association with radiologist factors (gender, years out of training, fellowship training, practice type) and with trainees was measured using a mixed-effects model.

RESULTS

Two hundred nine radiologists generated 60,543 reports, of which 7.2% (n = 4,382) contained RAIs. Only 3.9% of RAIs (170 of 4,382) were actionable. More than 60% of RAIs were for eight examinations: thyroid ultrasound (14.1%), neck CT (12.6%), brain MRI (6.9%), chest CT (6.5%), neck CT angiography (5.5%), temporal bone CT (5.3%), temporal bone MRI (5.2%), and pituitary MRI (4.6%). Radiologists >23 years out of training (odds ratio, 0.39; 95% confidence interval, 0.15-1.02; P = .05) and community radiologists (odds ratio, 0.53; 95% confidence interval, 0.22-1.31; P = .17) had substantially lower estimated odds of making actionable RAIs than radiologists <7 years out of training and academic radiologists, respectively.

CONCLUSIONS

The studied radiologists rarely made actionable RAIs, which makes it difficult to identify and track clinically necessary RAIs to timely performance. Multifaceted quality improvement initiatives including peer comparisons, clinical decision support at the time of reporting, and the development of evidence-based best practices, may help improve tracking and timely performance of clinically necessary RAIs.

Recommendations for Additional Imaging on Head and Neck Imaging Examinations: Interradiologist Variation and Associated Factors

BACKGROUND. A paucity of relevant guidelines may lead to pronounced variation among radiologists in issuing recommendations for additional imaging (RAI) for head and neck imaging. OBJECTIVE. The purpose of this article was to explore associations of RAI for head and neck imaging examinations with examination, patient, and radiologist factors and to assess the role of individual radiologist-specific behavior in issuing such RAI. METHODS. This retrospective study included 39,200 patients (median age, 58 years; 21,855 women, 17,315 men, 30 with missing sex information) who underwent 39,200 head and neck CT or MRI examinations, interpreted by 61 radiologists, from June 1, 2021, through May 31, 2022. A natural language processing (NLP) tool with manual review of NLP results was used to identify RAI in report impressions. Interradiologist variation in RAI rates was assessed. A generalized mixed-effects model was used to assess associations between RAI and examination, patient, and radiologist factors. RESULTS. A total of 2943 (7.5%) reports contained RAI. Individual radiologist RAI rates ranged from 0.8% to 22.0% (median, 7.1%; IQR, 5.2-10.2%), representing a 27.5-fold difference between minimum and a maximum values and 1.8-fold difference between 25th and 75th percentiles. In multivariable analysis, RAI likelihood was higher for CTA than for CT examinations (OR, 1.32), for examinations that included a trainee in report generation (OR, 1.23), and for patients with self-identified race of Black or African American versus White (OR, 1.25); was lower for male than female patients (OR, 0.90); and was associated with increasing patient age (OR, 1.09 per decade) and inversely associated with radiologist years since training (OR, 0.90 per 5 years). The model accounted for 10.9% of the likelihood of RAI. Of explainable likelihood of RAI, 25.7% was attributable to examination, patient, and radiologist factors; 74.3% was attributable to radiologist-specific behavior. CONCLUSION. Interradiologist variation in RAI rates for head and neck imaging was substantial. RAI appear to be more substantially associated with individual radiologist-specific behavior than with measurable systemic factors. CLINICAL IMPACT. Quality improvement initiatives, incorporating best practices for incidental findings management, may help reduce radiologist preference-sensitive decision-making in issuing RAI for head and neck imaging and associated care variation.

Realizing the Potential of Social Determinants Data: A Scoping Review of Approaches for Screening, Linkage, Extraction, Analysis and Interventions

Background

Social determinants of health (SDoH) like socioeconomics and neighborhoods strongly influence outcomes, yet standardized SDoH data is lacking in electronic health records (EHR), limiting research and care quality.

Methods

We searched PubMed using keywords "SDOH" and "EHR", underwent title/abstract and full-text screening. Included records were analyzed under five domains: 1) SDoH screening and assessment approaches, 2) SDoH data collection and documentation, 3) Use of natural language processing (NLP) for extracting SDoH, 4) SDoH data and health outcomes, and 5) SDoH-driven interventions.

Results

We identified 685 articles, of which 324 underwent full review. Key findings include tailored screening instruments implemented across settings, census and claims data linkage providing contextual SDoH profiles, rule-based and neural network systems extracting SDoH from notes using NLP, connections found between SDoH data and healthcare utilization/chronic disease control, and integrated care management programs executed. However, considerable variability persists across data sources, tools, and outcomes.

Discussion

Despite progress identifying patient social needs, further development of standards, predictive models, and coordinated interventions is critical to fulfill the potential of SDoH-EHR integration. Additional database searches could strengthen this scoping review. Ultimately widespread capture, analysis, and translation of multidimensional SDoH data into clinical care is essential for promoting health equity.

Evaluation of Socioeconomic Disparities in Follow-Up Completion for Incidental Pulmonary Nodules

PURPOSE

The aim of this study was to evaluate the association between census tract-level measures of social vulnerability and residential segregation and incidental pulmonary nodule (IPN) follow-up.

METHODS

This retrospective cohort study included patients with IPNs ≥6 mm in size or multiple subsolid or ground-glass IPNs <6 mm (with nonoptional follow-up recommendations) diagnosed between January 1, 2018, and December 30, 2019, at a large urban tertiary center and followed for ≥2 years. Geographic sociodemographic context was characterized by the 2018 Centers for Disease Control and Prevention Social Vulnerability Index (SVI) and the index of concentration at the extremes (ICE), categorized in quartiles. Multivariable binomial regression models were used, with a primary outcome of inappropriate IPN follow-up (late or no follow-up). Models were also stratified by nodule risk.

RESULTS

The study consisted of 2,492 patients (mean age, 65.6 ± 12.6 years; 1,361 women). Top-quartile SVI patients were more likely to have inappropriate follow-up (risk ratio [RR], 1.24; 95% confidence interval [CI], 1.12-1.36) compared with the bottom quartile; risk was also elevated in top-quartile SVI subcategories of socioeconomic status (RR, 1.23; 95% CI, 1.13-1.34), Minority status and language (RR, 1.24; 95% CI, 1.03-1.48), housing and transportation (RR, 1.13; 95% CI, 1.02-1.26), and ICE (RR, 1.20; 95% CI, 1.11-1.30). Furthermore, top-quartile ICE was associated with greater risk for inappropriate follow-up among high-risk versus lower risk IPNs (RR, 1.33 [95% CI, 1.18-1.50] versus 1.13 [95% CI, 1.02-1.25]), respectively; P for interaction = .017).

CONCLUSIONS

Local social vulnerability and residential segregation are associated with inappropriate IPN follow-up and may inform policy or interventions tailored for neighborhoods.

Challenges in Interpretation of US Breast Findings in the Emergency Setting

Emergencies in breast imaging are infrequent but not rare. Although infectious conditions such as mastitis and breast abscess are the most common breast diseases encountered in acute care settings, other entities that may require additional imaging or different treatment approaches are also seen and include traumatic injury and breast cancer. While mammography is widely available for breast evaluation in outpatient facilities, most emergency departments do not have mammography units. This makes evaluation of patients with breast disease incomplete in the acute care setting and emphasizes the role of appropriate US techniques for interpretation. It also highlights the importance of effective sonographer-to-radiologist communication to ensure patient safety and diagnostic accuracy, especially in an era of increasing adoption of teleradiology. The authors discuss the challenges in image acquisition and remote interpretation that are commonly faced by radiologists when they assess breast anomalies in the emergency setting. They present strategies to overcome these challenges by describing techniques for proper US evaluation, highlighting the importance of sonographer-radiologist communication, defining the goals of the evaluation, reviewing common differential diagnoses, and providing appropriate follow-up recommendations. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.

Development and External Validation of an Artificial Intelligence Model for Identifying Radiology Reports Containing Recommendations for Additional Imaging

BACKGROUND. Reported rates of recommendations for additional imaging (RAIs) in radiology reports are low. Bidirectional encoder representations from transformers (BERT), a deep learning model pretrained to understand language context and ambiguity, has potential for identifying RAIs and thereby assisting large-scale quality improvement efforts. OBJECTIVE. The purpose of this study was to develop and externally validate an artificial intelligence (AI)-based model for identifying radiology reports containing RAIs. METHODS. This retrospective study was performed at a multisite health center. A total of 6300 radiology reports generated at one site from January 1, 2015, to June 30, 2021, were randomly selected and split by 4:1 ratio to create training (n = 5040) and test (n = 1260) sets. A total of 1260 reports generated at the center's other sites (including academic and community hospitals) from April 1 to April 30, 2022, were randomly selected as an external validation group. Referring practitioners and radiologists of varying sub-specialties manually reviewed report impressions for presence of RAIs. A BERT-based technique for identifying RAIs was developed by use of the training set. Performance of the BERT-based model and a previously developed traditional machine learning (TML) model was assessed in the test set. Finally, performance was assessed in the external validation set. The code for the BERT-based RAI model is publicly available. RESULTS. Among a total of 7419 unique patients (4133 women, 3286 men; mean age, 58.8 years), 10.0% of 7560 reports contained RAI. In the test set, the BERT-based model had 94.4% precision, 98.5% recall, and an F1 score of 96.4%. In the test set, the TML model had 69.0% precision, 65.4% recall, and an F1 score of 67.2%. In the test set, accuracy was greater for the BERT-based than for the TML model (99.2% vs 93.1%, p < .001). In the external validation set, the BERT-based model had 99.2% precision, 91.6% recall, an F1 score of 95.2%, and 99.0% accuracy. CONCLUSION. The BERT-based AI model accurately identified reports with RAIs, outperforming the TML model. High performance in the external validation set suggests the potential for other health systems to adapt the model without requiring institution-specific training. CLINICAL IMPACT. The model could potentially be used for real-time EHR monitoring for RAIs and other improvement initiatives to help ensure timely performance of clinically necessary recommended follow-up.

Impact of an Automated Closed-Loop Communication and Tracking Tool on the Rate of Recommendations for Additional Imaging in Thoracic Radiology Reports

OBJECTIVE

Assess the effects of feedback reports and implementing a closed-loop communication system on rates of recommendations for additional imaging (RAIs) in thoracic radiology reports.

METHODS

In this retrospective, institutional review board-approved study at an academic quaternary care hospital, we analyzed 176,498 thoracic radiology reports during a pre-intervention (baseline) period from April 1, 2018, to November 30, 2018; a feedback report only period from December 1, 2018, to September 30, 2019; and a closed-loop communication system plus feedback report (IT intervention) period from October 1, 2019, to December 31, 2020, promoting explicit documentation of rationale, time frame, and imaging modality for RAI, defined as complete RAI. A previously validated natural language processing tool was used to classify reports with an RAI. Primary outcome of rate of RAI was compared using a control chart. Multivariable logistic regression determined factors associated with likelihood of RAI. We also estimated the completeness of RAI in reports comparing IT intervention to baseline using χ2 statistic.

RESULTS

The natural language processing tool classified 3.2% (5,682 of 176,498) reports as having an RAI; 3.5% (1,783 of 51,323) during the pre-intervention period, 3.8% (2,147 of 56,722) during the feedback report only period (odds ratio: 1.1, P = .03), and 2.6% (1,752 of 68,453) during the IT intervention period (odds ratio: 0.60, P < .001). In subanalysis, the proportion of incomplete RAI decreased from 84.0% (79 of 94) during the pre-intervention period to 48.5% (47 of 97) during the IT intervention period (P < .001).

DISCUSSION

Feedback reports alone increased RAI rates, and an IT intervention promoting documentation of complete RAI in addition to feedback reports led to significant reductions in RAI rate, incomplete RAI, and improved overall completeness of the radiology recommendations.