Catching Those Who Fall Through the Cracks: Integrating a Follow-Up Process for Emergency Department Patients with Incidental Radiologic Findings

Using natural language processing to identify emergency department patients with incidental lung nodules requiring follow-up

OBJECTIVES

For emergency department (ED) patients, lung cancer may be detected early through incidental lung nodules (ILNs) discovered on chest CTs. However, there are significant errors in the communication and follow-up of incidental findings on ED imaging, particularly due to unstructured radiology reports. Natural language processing (NLP) can aid in identifying ILNs requiring follow-up, potentially reducing errors from missed follow-up. We sought to develop an open-access, three-step NLP pipeline specifically for this purpose.

METHODS

This retrospective used a cohort of 26,545 chest CTs performed in three EDs from 2014 to 2021. Randomly selected chest CT reports were annotated by MD raters using Prodigy software to develop a stepwise NLP "pipeline" that first excluded prior or known malignancy, determined the presence of a lung nodule, and then categorized any recommended follow-up. NLP was developed using a RoBERTa large language model on the SpaCy platform and deployed as open-access software using Docker. After NLP development it was applied to 1000 CT reports that were manually reviewed to determine accuracy using accepted NLP metrics of precision (positive predictive value), recall (sensitivity), and F1 score (which balances precision and recall).

RESULTS

Precision, recall, and F1 score were 0.85, 0.71, and 0.77, respectively, for malignancy; 0.87, 0.83, and 0.85 for nodule; and 0.82, 0.90, and 0.85 for follow-up. Overall accuracy for follow-up in the absence of malignancy with a nodule present was 93.3%. The overall recommended follow-up rate was 12.4%, with 10.1% of patients having evidence of known or prior malignancy.

CONCLUSIONS

We developed an accurate, open-access pipeline to identify ILNs with recommended follow-up on ED chest CTs. While the prevalence of recommended follow-up is lower than some prior studies, it more accurately reflects the prevalence of truly incidental findings without prior or known malignancy. Incorporating this tool could reduce errors by improving the identification, communication, and tracking of ILNs.

Clinical Impact of Radiologist's Alert System on Patient Care for High-risk Incidental CT Findings: A Machine Learning-Based Risk Factor Analysis

RATIONALE AND OBJECTIVES

Efficient communication between radiologists and clinicians ordering computed tomography (CT) examinations is crucial for managing high-risk incidental CT findings (ICTFs). Herein, we introduced a Radiologist's Alert and Patient Care Follow-up System (APCFS) for high-risk ICTFs. This study aimed to analyze the ICTFs detected by this system and the factors associated with them.

MATERIALS AND METHODS

This retrospective study was approved by the institutional review board. We analyzed 52,331 CT examinations conducted between 2019 and 2021. In cases where high-risk ICTFs were identified, radiologists utilized APCFS to prompt ordering clinicians for further patient care. We assessed the frequency, affected body organs, presence or absence of therapeutic interventions, and diagnoses of high-risk ICTFs. An automated machine learning platform was employed to analyze the factors associated with high-risk ICTFs.

RESULTS

Among the 52,331 CT examinations, 507 (0.96%) revealed high-risk ICTFs, primarily affecting the lung (18.0%). Of these 507 high-risk ICTFs, 117 (23.1%) underwent therapeutic interventions, while 362 (71.4%) required only follow-up. Of the 117 cases undergoing interventions, 61 (52.1%) required surgery. Of the 219 high-risk ICTFs leading to a confirmed diagnosis, 146 (66.7%) were neoplastic lesions, including 88 (60.3%) malignancies, and 73 (33.3%) were non-neoplastic lesions. The top three risk factors associated with high-risk ICTFs in the regularized logistic regression model were the imaging protocol (especially aortic valve implantation planning protocol), imaging area (especially whole-body imaging), and clinical department (especially cardiology).

CONCLUSION

Utilizing APCFS, high-risk ICTFs were detected in approximately 1% of all CT examinations, likely associated with specific imaging protocols, areas, and clinical departments.

Documentation of incidentally noted hepatic steatosis to emergency department patients: A retrospective study

BACKGROUND

Hepatic steatosis is a common incidental finding on emergency department (ED) imaging studies, occurring in up to 10% of studies, and carries significant long-term morbidity. Frequently considered an unimportant finding, it is unknown how often ED patients are informed of hepatic steatosis. Our objective was to examine hepatic steatosis inclusion in ED discharge materials.

METHODS

Data from discharged patients at an urban academic ED (>90,000 visits) with abdominal imaging (computed tomography or ultrasound) from 2019 to 2022 were screened. Patients with radiology reports documenting hepatic steatosis were included. Two trained data abstractors analyzed discharge materials and coded disclosure of steatosis (present/absent). Data abstraction also noted how the finding was included (e.g., follow-up instructions, provision of radiology report). Factors associated with patient disclosure were examined through regression models including age, race, ethnicity, insurance, number of imaging studies during encounter, type of imaging study, and inclusion of hepatic steatosis in the impression section of radiology report.

RESULTS

Of 10,677 radiology reports, 1209 (11.3%) had documented hepatic steatosis. The mean (±SD) age was 47.1 (±14.1) years; 56.4% were female, 53.5% were White, and 30.7% Hispanic. Only 173 of 1209 patients (14.3%) received any discharge documentation of hepatic steatosis. In 65% of cases where disclosure occurred, the imaging report was pasted verbatim into discharge material. There were no significant differences in discharge documentation by demographic groups. Mention within the radiology report impression (rather than the report body alone) occurred in 73.8% of records and was the strongest predictor of disclosure to patients (adjusted odds ratio 2.18, 95% confidence interval 1.39-3.54).

CONCLUSIONS

Despite the high prevalence of hepatic steatosis in radiology reports, rates of documentation in patient-facing discharge materials are notably low, exposing a possible communication failure with consequences for diagnosis.

Interventions to improve timely cancer diagnosis: an integrative review

Cancer will affect more than one in three U.S. residents in their lifetime, and although the diagnosis will be made efficiently in most of these cases, roughly one in five patients will experience a delayed or missed diagnosis. In this integrative review, we focus on missed opportunities in the diagnosis of breast, lung, and colorectal cancer in the ambulatory care environment. From a review of 493 publications, we summarize the current evidence regarding the contributing factors to missed or delayed cancer diagnosis in ambulatory care, as well as evidence to support possible strategies for intervention. Cancer diagnoses are made after follow-up of a positive screening test or an incidental finding, or most commonly, by following up and clarifying non-specific initial presentations to primary care. Breakdowns and delays are unacceptably common in each of these pathways, representing failures to follow-up on abnormal test results, incidental findings, non-specific symptoms, or consults. Interventions aimed at 'closing the loop' represent an opportunity to improve the timeliness of cancer diagnosis and reduce the harm from diagnostic errors. Improving patient engagement, using 'safety netting,' and taking advantage of the functionality offered through health information technology are all viable options to address these problems.

Development and Psychometric Analysis of a Patient-Reported Measure of Diagnostic Excellence for Emergency and Urgent Care Settings

BACKGROUND

Emergency and urgent care settings face challenges with routinely obtaining performance feedback related to diagnostic care. Patients and their care partners provide an important perspective on the diagnostic process and outcome of care in these settings. We sought to develop and test psychometric properties of Patient-Report to IMprove Diagnostic Excellence in Emergency Department settings (PRIME-ED), a measure of patient-reported diagnostic excellence in these care settings.

METHODS

We developed PRIME-ED based on literature review, expert feedback, and cognitive testing. To assess psychometric properties, we surveyed AmeriSpeak, a probability-based panel that provides sample coverage of approximately 97% of the U.S. household population, in February 2022 to adult patients, or their care partners, who had presented to an emergency department or urgent care facility within the last 30 days. Respondents rated their agreement on a 5-point Likert scale with each of 17 statements across multiple domains of patient-reported diagnostic excellence. Demographics, visit characteristics, and a subset of the Emergency Department Consumer Assessment of Healthcare Providers & Systems were also collected. We conducted psychometric testing for reliability and validity.

RESULTS

Over a thousand (n = 1116) national panelists completed the PRIME-ED survey, of which 58.7% were patients and 40.9% were care partners; 49.6% received care at an emergency department and 49.9% at an urgent care facility. Responses had high internal consistency within 3 patient-reported diagnostic excellence domain groupings: diagnostic process (Cronbach's alpha 0.94), accuracy of diagnosis (0.93), and communication of diagnosis (0.94). Domain groupings were significantly correlated with concurrent Emergency Department Consumer Assessment of Healthcare Providers & Systems items. Factor analyses substantiated 3 domain groupings.

CONCLUSIONS

PRIME-ED has potential as a tool for capturing patient-reported diagnostic excellence in emergency and urgent care.

Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings

OBJECTIVES

To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings.

MATERIALS AND METHODS

Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale.

RESULTS

For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision.

CONCLUSION

GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

Incidental Renal Cysts Found by Point-of-Care Ultrasound: A Retrospective Chart Review

BACKGROUND

Incidental findings are unrelated to a patient's complaint, found on diagnostic imaging, such as point-of-care ultrasound (POCUS). Incidental findings represent potential harms to patients and may lead to increased patient anxiety and health care costs related to downstream testing and surveillance.

STUDY OBJECTIVES

In this study, we aimed to calculate the rate of incidental renal cysts found by POCUS. Further, we hoped to describe how emergency physicians relay the findings to patients. Lastly, we hoped to examine if patients suffered harms in the 12 months following identification of an incidental renal cyst.

METHODS

From our single-center, academic emergency department (ED), we reviewed renal POCUS images from 1000 consecutive adult ED patients to determine if there was a renal cyst. Next, we performed manual chart review to determine if patients were informed of the incidental renal cyst or suffered any patient harms.

RESULTS

We found the prevalence of renal cysts to be 6.5% (95% confidence interval: 4.9%-8.4%). Those with cysts were more likely to be older compared to those without (63 ± 14 vs. 49 ± 15 years of age). Only 8% of patients had evidence that they were informed of their incidental renal cyst. No patients received a biopsy or were diagnosed with renal cell carcinoma or polycystic kidney disease.

CONCLUSION

Incidental renal cysts are common and are more likely to be found in older adults. In our study, physicians infrequently informed patients of their incidental finding.

Navigating Transitions in Oncology Care: From Emergency Department to Outpatient Clinic

OBJECTIVE

This quality improvement project was a collaborative effort with Penn Medicine's emergency department (ED) and oncology nurse navigators (ONNs). The goal of the project was to streamline patient transitions from the ED to the outpatient oncology clinic by developing a standardized referral process. The main objectives were to simplify and automate the referral process using the electronic medical record, improve multidisciplinary communication across the care continuum, ensure timely follow-up, and address barriers to oncology care.

METHODS

The ED providers placed a consult to ONNs. The ONNs reached out to the patient within 48 hours of the consult. They maintained a database of patient referrals and collected information such as patient demographics, reason for referral, insurance, and patient outcomes.

RESULTS

The ED providers referred 204 patients to the ONNs from April 2022 to September 2023. The development of a standardized referral process from the ED to the outpatient oncology clinic proved successful. Of the patients referred, the ONNs facilitated 98 cancer diagnoses and 80 of those patients are receiving oncology care at Penn Medicine. The median time to the patient's first appointments was seven days, diagnosis was 15 days, and treatment initiation occurred within 32 days.

CONCLUSION

The project team achieved their goal of facilitating timely access to oncology care, ensuring continuity, and addressing patient-specific barriers.

IMPLICATIONS FOR NURSING PRACTICE

This quality improvement initiative highlights the ONNs' role in enhancing access and equity in cancer care delivery. The success of the project underscores the ONN's expertise and leadership in addressing healthcare disparities in oncology care. Collaboratively, the teams created a new referral workflow improving care transitions from the ED to the outpatient oncology clinic. The project sets a precedent for optimizing patient care transitions, demonstrating the positive impact of ONNs as key members of the multidisciplinary healthcare team.

The Value of Using a Quality Assurance Follow-Up Team to Address Incidental Findings After Emergency Department or Urgent Care Discharge: A Cost Analysis

BACKGROUND

Incidental finding (IF) follow-up is of critical importance for patient safety and is a source of malpractice risk. Laboratory, imaging, or other types of IFs are often uncovered incidentally and are missed, not addressed, or only result after hospital discharge. Despite a growing IF notification literature, a need remains to study cost-effective non-electronic health record (EHR)-specific solutions that can be used across different types of IFs and EHRs.

OBJECTIVE

The objective of this study was to evaluate the utility and cost-effectiveness of an EHR-independent emergency medicine-based quality assurance (QA) follow-up program in which an experienced nurse reviewed laboratory and imaging studies and ensured appropriate follow-up of results.

METHODS

A QA nurse reviewed preceding-day abnormal studies from a tertiary care hospital, a community hospital, and an urgent care center. Laboratory values outside preset parameters or radiology over-reads resulting in clinically actionable changes triggered contact with an on-call emergency physician to determine an appropriate intervention and its implementation.

RESULTS

Of 104,125 visits with 1,351,212 laboratory studies and 95,000 imaging studies, 6530 visits had IFs, including 2659 laboratory and 4004 imaging results. The most common intervention was contacting a primary care physician (5783 cases [88.6%]). Twenty-one cases resulted in a patient returning to the ED, at an average cost of $28,000 per potential life-/limb-saving intervention.

CONCLUSIONS

Although abnormalities in laboratory results and imaging are often incidental to patient care, a dedicated emergency department QA follow-up program resulted in the identification and communication of numerous laboratory and imaging abnormalities and may result in changes to patients' subsequent clinical course, potentially increasing patient safety.

Medico-legal risk and use of medical directives in the emergency department

PURPOSE

The medico-legal risk associated with application of medical directives in the emergency department (ED) is unknown. The objective of this study was to describe and analyze factors associated with medico-legal risk in cases involving medical directives in the ED.

METHODS

We conducted a descriptive analysis of closed medico-legal cases [hospital complaints, regulatory authority (i.e., College) complaints, and civil legal actions] involving emergency physicians in Canadian EDs involving medical directives (alternate terms including "standing order", "nursing initiated", "nurse initiated", "nursing order", "triage initiated", "triage ordered", "directive", "ED protocol", and "ED's protocol"). We used data from closed cases involving the Canadian Medical Protective Association from January 2016 until December 2021. We abstracted descriptive factors of the cases and used a framework for contributing factors classification.

RESULTS

From 2016 until 2021, 43,332 cases were closed and 1957 involved emergency physicians for which there was medico-legal information available for analysis. In all, 28 involved emergency physicians and medical directives. Situational awareness, team communication, and issues with clinical decision-making were the most important factors contributing to harm and medico-legal risk. Peer experts were critical of physicians not reviewing all results available for patients when initiated through a directive, misinterpreting test results, a less than thorough initial assessment, and of failing to reassess patients or re-order investigations when indicated.

CONCLUSION

Our findings suggest that the medico-legal risk exposure from the use of medical directives in the ED is low. Emergency departments may consider implementing systems to support adherence to medical directive policies, ensure physicians are alerted when medical directives are completed in a timely fashion, and leverage tools to notify the healthcare team when results have not been reviewed.

Expecting the unexpected: incidental findings at a level 1 trauma center

INTRODUCTION

Incidental findings on comprehensive imaging in the adult trauma population occur at rates as high as 54.8%. We sought to determine the incidence of potentially malignant or pre-malignant incidental findings in a high-volume level 1 trauma center and to evaluate follow-up recommendations.

METHODS

This was a retrospective review of all patients with incidental findings on imaging who were admitted to the trauma service at our level 1 trauma center between January 1st, 2014, and October 1st, 2019. A multi-disciplinary team characterized findings as potentially malignant or pre-malignant.

RESULTS

The study included 495 patients who had incidental findings, 410 of whom had potentially malignant or pre-malignant findings on imaging, resulting in a cumulative incidence of 6.6%. The mean age was 65 and 217 (52.9%) patients were male. The majority of "incidentalomas" were discovered on CT imaging (n=665, 98.1%); over half were solid (n=349, 51.5%), while 27.4% were cystic (n=186) in nature. The lungs (n=199, 29.4%), kidneys (n=154, 22.8%), liver (n=74, 10.9%), thyroid gland (n=58, 8.6%), and adrenal glands (n=53, 7.8%) harbored the most incidentalomas. Less than half of patients with incidental findings received specific follow-up recommendations on the radiologist's report (n=150, 39%). Sixty-one percent of patients (n=250) had their incidentalomas detailed in the discharge paperwork.

CONCLUSION

The results of our study suggest that potentially malignant or pre-malignant incidental findings are common among trauma patients. Specific follow-up recommendations were not presented in 61% of the radiology reports, highlighting the need to standardize medical record capture of an incidentaloma to ensure adequate and appropriate follow-up.

White Paper: Best Practices in the Communication and Management of Actionable Incidental Findings in Emergency Department Imaging

PURPOSE

Actionable incidental findings (AIFs) are common in radiologic imaging. Imaging is commonly performed in emergency department (ED) visits, and AIFs are frequently encountered, but the ED presents unique challenges for communication and follow-up of these findings. The authors formed a multidisciplinary panel to seek consensus regarding best practices in the reporting, communication, and follow-up of AIFs on ED imaging tests.

METHODS

A 15-member panel was formed, nominated by the ACR and American College of Emergency Physicians, to represent radiologists, emergency physicians, patients, and those involved in health care systems and quality. A modified Delphi process was used to identify areas of best practice and seek consensus. The panel identified four areas: (1) report elements and structure, (2) communication of findings with patients, (3) communication of findings with clinicians, and (4) follow-up and tracking systems. A survey was constructed to seek consensus and was anonymously administered in two rounds, with a priori agreement requiring at least 80% consensus. Discussion occurred after the first round, with readministration of questions where consensus was not initially achieved.

RESULTS

Consensus was reached in the four areas identified. There was particularly strong consensus that AIFs represent a system-level issue, with need for approaches that do not depend on individual clinicians or patients to ensure communication and completion of recommended follow-up.

CONCLUSIONS

This multidisciplinary collaboration represents consensus results on best practices regarding the reporting and communication of AIFs in the ED setting.