Incidental Findings: A Survey of Radiologists and Emergency Physicians

Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings

OBJECTIVES

To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings.

MATERIALS AND METHODS

Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale.

RESULTS

For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision.

CONCLUSION

GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

Implementing a Streamlined Radiology Workflow to Close the Loop on Incidental Imaging Findings in the Emergency Department

Actionable incidental findings (AIFs) are common imaging findings unrelated to the clinical indication for the imaging test for which follow-up is recommended. Increasing utilization of imaging in the emergency department (ED) in recent years has resulted in more patients with AIFs. When these findings are not properly communicated and followed up upon, there is harm to the patient's health outcome as well as possible increased financial costs for the patient, the health system, and potential litigation. Tracking these findings can be difficult, especially so in a large health system. In this report, we detail our experience implementing a closed-loop AIF program within the ED of 11 satellite hospitals of a large academic health system. Our new workflow streamlined radiologist reporting of AIFs through system macros and by using a standardized form integrated into the dictation software. Upon completion of the form, an automatic email is sent to a dedicated nurse navigator who documented the findings and closed the loop by coordinating follow-up imaging or clinic visits with patients, primary care providers, and specialists. Through the new workflow, a total of 1207 incidental finding reports have been submitted from July 2021 to May 2022. The vast majority of AIFs were identified on CT, and the most common categories included lung nodules, pancreas lesions, liver lesions, and other potentially cancerous lesions. At least 10 new cancers have been detected. We hope this report can help guide other health systems in the design of a closed-loop incidental findings program.

White Paper: Best Practices in the Communication and Management of Actionable Incidental Findings in Emergency Department Imaging

PURPOSE

Actionable incidental findings (AIFs) are common in radiologic imaging. Imaging is commonly performed in emergency department (ED) visits, and AIFs are frequently encountered, but the ED presents unique challenges for communication and follow-up of these findings. The authors formed a multidisciplinary panel to seek consensus regarding best practices in the reporting, communication, and follow-up of AIFs on ED imaging tests.

METHODS

A 15-member panel was formed, nominated by the ACR and American College of Emergency Physicians, to represent radiologists, emergency physicians, patients, and those involved in health care systems and quality. A modified Delphi process was used to identify areas of best practice and seek consensus. The panel identified four areas: (1) report elements and structure, (2) communication of findings with patients, (3) communication of findings with clinicians, and (4) follow-up and tracking systems. A survey was constructed to seek consensus and was anonymously administered in two rounds, with a priori agreement requiring at least 80% consensus. Discussion occurred after the first round, with readministration of questions where consensus was not initially achieved.

RESULTS

Consensus was reached in the four areas identified. There was particularly strong consensus that AIFs represent a system-level issue, with need for approaches that do not depend on individual clinicians or patients to ensure communication and completion of recommended follow-up.

CONCLUSIONS

This multidisciplinary collaboration represents consensus results on best practices regarding the reporting and communication of AIFs in the ED setting.

System-driven longitudinal follow-up of incidental imaging findings

Increasing utilization of cross-sectional imaging has resulted in more clinically significant incidental findings being discovered. However, the current approach for handling these findings is commonly inconsistent and relies greatly on the efforts of individual clinicians. Making sure every actionable incidental finding is handled in a consistent and reliable manner can be difficult, especially for a large health system. We propose an approach to handling incidental findings aimed at improving patient follow-up rates, which involves implementing system-level processes that standardize the reporting of incidental findings, notification of clinicians and the patient, and centralized monitoring of longitudinal patient follow-up. We will lay out a general framework for standardized reporting of incidental findings by the radiologist using software integrated into the daily workflow. This should enable simultaneous notification of the ordering clinician, the patient's primary-care provider, and an incidental findings navigator. The navigator will "close the loop" by working with clinicians to notify the patient of the finding, coordinate patient follow-up, and document the finding and long-term follow-up. We hope this can serve as a basic framework to help large health systems design an incidental findings workflow to improve follow-up rates and reduce patient harm.

Normal blindness: when we Look But Fail To See

Humans routinely miss important information that is 'right in front of our eyes', from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these 'Looked But Failed To See' (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

Incidental Radiology Findings on Computed Tomography Studies in Emergency Department Patients: A Systematic Review and Meta-Analysis

STUDY OBJECTIVE

An incidental finding is defined as a newly discovered mass or lesion detected on imaging performed for an unrelated reason. The identification of an incidental finding may be an opportunity for the early detection of a serious medical condition, including a malignancy. However, little is known about the prevalence of incidental findings in the emergency department (ED) setting and the strategies that can be used to mitigate the risk associated with them in the ED. This study aimed to estimate the overall prevalence of incidental findings and to summarize the currently described measures to mitigate the risks associated with incidental findings.

METHODS

On November 22, 2020, a systematic literature search of PubMed, EMBASE, and Scopus was performed for studies that were published in peer-reviewed journals and reported the prevalence of incidental findings in computed tomography (CT) scans in patients in the ED. Patients who received CT scans that included the head, neck, chest, or abdomen/pelvis were included. The study characteristics, overall prevalence of incidental findings, prevalence of incidental findings by body region, and prespecified subgroups were extracted. The criteria used for risk stratification within individual studies were also extracted. Pooled estimates were calculated using a random-effects meta-analysis.

RESULTS

A total of 1,385 studies were identified, and 69 studies met the inclusion criteria. The included studies represented 147,763 ED encounters or radiology reports across 16 countries, and 83% of studies were observational, cross-sectional studies. A total of 35 studies (50.7%) were in trauma patients. A large degree of heterogeneity was observed across the included studies. The overall pooled prevalence estimate for any incidental finding was 31.3% (95% confidence interval 24.4% to 39.1%). We found great variation in the methods described to mitigate the risk associated with incidental findings, including a lack of standardized risk stratification, inconsistent documentation practices, and only a small subset of studies describing prospective interventions aimed at improving the recognition and management of incidental findings from the ED.

CONCLUSION

In patients in the ED receiving CT scans, incidental findings are commonly encountered across a broad range of ED chief complaints. This review highlights the existence of great heterogeneity in the definitions used to classify incidental findings. Future studies are needed to determine a clinically feasible categorization standard or terminology for commonly encountered incidental findings in the ED setting to standardize classification and documentation.

Novel Quality Measure Set: Closing the Completion Loop on Radiology Follow-up Recommendations for Noncritical Actionable Incidental Findings

BACKGROUND

Care gaps occur when radiology follow-up recommendations are poorly communicated or not completed, resulting in missed or delayed diagnosis potentially leading to worse patient outcomes. This ACR-led initiative assembled a technical expert panel (TEP) to advise development of quality measures intended to improve communication and drive increased completion rates for radiology follow-up recommendations.

MATERIALS AND METHODS

A multistakeholder TEP was assembled to advise the development of quality measures. The project scope, limited to noncritical actionable incidental findings (AIFs), encourages practices to develop and implement systems ensuring appropriate communication and follow-up to completion.

RESULTS

A suite of nine measures were developed: four outcome measures include closing the loop on completion of radiology follow-up recommendations for nonemergent AIFs (with pulmonary nodule and abdominal aortic aneurysm use cases) and overall cancer diagnoses. Five process measures address communication and tracking of AIFs: inclusion of available evidence or guidelines informing the recommendation, communication of AIFs to the practice managing ongoing care, identifying when AIFs have been communicated to the patient, and employing tracking and reminder systems for AIFs.

CONCLUSION

This ACR-led initiative developed a measure set intended to improve patient outcomes by ensuring that AIFs are appropriately communicated and followed up. The intent of these measures is to focus improvement on specific areas in which gaps in communication and AIF follow-up may occur, prompting systems to devote resources that will identify and implement solutions to improve patient care.

Closing the Compliance Loop on Follow-Up Imaging Recommendations: Comparing Radiologists' and Administrators' Attitudes

PURPOSE

To compare non-physician healthcare professional and radiologists' survey responses regarding attitudes and current practices, policies, and procedures related to the follow-up of nonemergent actionable incidental findings (AIF).

MATERIALS AND METHODS

The American College of Radiology (ACR) developed a survey with input from a technical expert panel (TEP). Survey items were developed by TEP members, refined by an ACR market research expert, and were examined for face and construct validity. The survey was distributed among ACR membership and various medical professional organizations. Responses from non-physician responders and radiologists were analyzed and compared using descriptive statistics.

RESULTS

The analysis included 375 responses, 247 from radiologists and 128 from non-physicians. All respondent groups stated that radiology follow-up recommendations are evidence-based. Both respondent groups indicated that there is up to moderate risk associated with AIF follow-up. Both respondent groups similarly favored that the accountability for communicating AIF lies first with the ordering provider, followed by primary care providers, then the patient, and lastly an automated process that is managed by a staff member and/or the radiologist. All respondent groups indicated that tracking processes were more commonly funded by the healthcare system than through the radiology budget.

CONCLUSION

There is alignment between non-physicians and radiologists regarding the implementation of tracking systems that assure completion of radiology follow-up recommendations. Building tracking systems represents an opportunity for multi-disciplinary collaboration to address care transition communication and process gaps.