Process improvement for follow-up radiology report recommendations of lung nodules

Incidental pulmonary nodules may lead to a high proportion of early-stage lung cancer: but it requires more than a high CT volume to achieve this

Background

The management of pulmonary nodules plays a critical role in early detection of lung cancer. Computed tomography (CT) has led to a stage-shift towards early-stage lung cancer, but regional differences in survival rates have been reported in Denmark. This study aimed to evaluate whether variations in nodule management among Danish health regions contributed to these differences.

Material and Methods

The Danish Health Data Authority and Danish Lung Cancer Registry provided data on CT usage and lung cancer stage distribution, respectively. Auditing of lung cancer stage IA patient referrals and nodule management of stage IV lung cancer patients was conducted in seven Danish lung cancer investigation centers, covering four of the five Danish health regions. CT scans were performed up to 2 years before the patients' diagnosis from 2019 to 2021.

Results

CT usage has increased steadily in Denmark over the past decade, with a simultaneous increase in the proportion of early-stage lung cancers, particularly stage IA. However, one Danish health region, Region Zealand, exhibited lower rates of early-stage lung cancer and overall survival despite a CT usage roughly similar to that of the other health regions. The audit did not find significant differences in pulmonary nodule management or a higher number of missed nodules by radiologists in this region compared to others.

Conclusion

This study suggests that a high CT scan volume alone is not sufficient for the early detection of lung cancer. Factors beyond hospital management practices, such as patient-related delays in socioeconomically disadvantaged areas, may contribute to regional differences in survival rates. This has implications for future strategies for reducing these differences.

Natural Language Processing for the Identification of Incidental Lung Nodules in Computed Tomography Reports: A Quality Control Tool

PURPOSE

To evaluate the diagnostic performance of a natural language processing (NLP) model in detecting incidental lung nodules (ILNs) in unstructured chest computed tomography (CT) reports.

METHODS

All unstructured consecutive reports of chest CT scans performed at a tertiary hospital between 2020 and 2021 were retrospectively reviewed (n = 21,542) to train the NLP tool. Internal validation was performed using reference readings by two radiologists of both CT scans and reports, using a different external cohort of 300 chest CT scans. Second, external validation was performed in a cohort of all random unstructured chest CT reports from 57 different hospitals conducted in May 2022. A review by the same thoracic radiologists was used as the gold standard. The sensitivity, specificity, and accuracy were calculated.

RESULTS

Of 21,542 CT reports, 484 mentioned at least one ILN (mean age, 71 ± 17.6 [standard deviation] years; women, 52%) and were included in the training set. In the internal validation (n = 300), the NLP tool detected ILN with a sensitivity of 100.0% (95% CI, 97.6 to 100.0), a specificity of 95.9% (95% CI, 91.3 to 98.5), and an accuracy of 98.0% (95% CI, 95.7 to 99.3). In the external validation (n = 977), the NLP tool yielded a sensitivity of 98.4% (95% CI, 94.5 to 99.8), a specificity of 98.6% (95% CI, 97.5 to 99.3), and an accuracy of 98.6% (95% CI, 97.6 to 99.2). Twelve months after the initial reports, 8 (8.60%) patients had a final diagnosis of lung cancer, among which 2 (2.15%) would have been lost to follow-up without the NLP tool.

CONCLUSION

NLP can be used to identify ILNs in unstructured reports with high accuracy, allowing a timely recall of patients and a potential diagnosis of early-stage lung cancer that might have been lost to follow-up.

Revenue and Cost Analysis of a System Utilizing Natural Language Processing and a Nurse Coordinator for Radiology Follow-up Recommendations

Radiology reports often contain recommendations for follow-up imaging, Provider adherence to these radiology recommendations can be incomplete, which may result in patient harm, lost revenue, or litigation. This study sought to perform a revenue assessment of a hybrid natural language processing (NLP) and human follow-up system. Reports generated from January 2020 to April 2021 that were indexed as overdue from follow-up recommendations by mPower Follow-Up Recommendation Algorithm (Nuance Communications Inc., Burlington, MA), were assessed for follow up and revenue. Follow-up exams completed because of the hybrid system were tabulated and given revenue amounts based on Medicare national reimbursement rates. These rates were then summated. A total of n =3011 patients were flagged via the mPower algorithm as having not received a timely follow-up indicated for procedure. Of these, n = 427 required the quality nurse to contact their healthcare provider to place orders. The follow-up imaging of these patients accounted for $62,937.66 of revenue. This revenue was calculated as higher than personnel cost (based on national average quality and safety nurse salary and time allotted on follow-ups). Our results indicate that a hybrid human-artificial intelligence follow-up system can be profitable, while potentially adding to patient safety. Our revenue figure likely significantly underestimates the true revenue obtained at our institution. This was due to the use of Medicare national reimbursement rates to calculate revenue, for the purposes of generalizability.

A Collaborative Emergency Medicine and Radiology Pulmonary Nodule Program: Identification of Associated Efficacy and Outcomes

PURPOSE

Incidental radiologic findings are commonplace, but the episodic nature of emergency department (ED) care makes it challenging to ensure that patients obtain appropriate follow-up. Rates of follow-up range from 30% to 77%, with some studies demonstrating that more than 30% have no follow-up at all. The aim of this study is to describe and analyze the outcomes of a collaborative emergency medicine and radiology initiative to establish a formal workflow for the follow-up of pulmonary nodules identified during ED care.

METHODS

A retrospective analysis was performed of patients referred to the pulmonary nodule program (PNP). Patients were divided into two categories: those with follow-up and those who do not have post-ED follow-up. The primary outcome was determining follow-up rates and outcomes, including patients referred for biopsy. The characteristics of patients who completed follow-up compared with those lost to follow-up were also examined.

RESULTS

A total of 574 patients were referred to the PNP. Initial follow-up was established in 390 (69.1%); 30.8% were considered lost to follow-up, and more than half of these patients did not respond to initial contact. There were minimal differences in characteristics between patients in these two categories. Of the 259 patients who completed PNP follow-up, 26 were referred for biopsy (13%).

CONCLUSIONS

The PNP provided effective transitions of care and potentially improved patient health care. Strategies to further enhance follow-up adherence will provide iterative improvement of the program. The PNP provides an implementation framework for post-ED pulmonary nodule follow-up in other health care systems and can be modified for use with other incidental diagnostic findings.

Research Priorities for Interventions to Address Health Disparities in Lung Nodule Management: An Official American Thoracic Society Research Statement

Background: Lung nodules are common incidental findings, and timely evaluation is critical to ensure diagnosis of localized-stage and potentially curable lung cancers. Rates of guideline-concordant lung nodule evaluation are low, and the risk of delayed evaluation is higher for minoritized groups. Objectives: To summarize the existing evidence, identify knowledge gaps, and prioritize research questions related to interventions to reduce disparities in lung nodule evaluation. Methods: A multidisciplinary committee was convened to review the evidence and identify key knowledge gaps in four domains: 1) research methodology, 2) patient-level interventions, 3) clinician-level interventions, and 4) health system-level interventions. A modified Delphi approach was used to identify research priorities. Results: Key knowledge gaps included 1) a lack of standardized approaches to identify factors associated with lung nodule management disparities, 2) limited data evaluating the role of social determinants of health on disparities in lung nodule management, 3) a lack of certainty regarding the optimal strategy to improve patient-clinician communication and information transmission and/or retention, and 4) a paucity of information on the impact of patient navigators and culturally trained multidisciplinary teams. Conclusions: This statement outlines a research agenda intended to stimulate high-impact studies of interventions to mitigate disparities in lung nodule evaluation. Research questions were prioritized around the following domains: 1) need for methodologic guidelines for conducting research related to disparities in nodule management, 2) evaluating how social determinants of health influence lung nodule evaluation, 3) studying approaches to improve patient-clinician communication, and 4) evaluating the utility of patient navigators and culturally enriched multidisciplinary teams to reduce disparities.

Predictors of Completion of Clinically Necessary Radiologist-Recommended Follow-Up Imaging: Assessment Using an Automated Closed-Loop Communication and Tracking Tool

BACKGROUND. Patients with adverse social determinants of health may be at increased risk of not completing clinically necessary follow-up imaging. OBJECTIVE. The purpose of this study was to use an automated closed-loop communication and tracking tool to identify patient-, referrer-, and imaging-related factors associated with lack of completion of radiologist-recommended follow-up imaging. METHODS. This retrospective study was performed at a single academic health system. A tool for automated communication and tracking of radiologist-recommended follow-up imaging was embedded in the PACS and electronic health record. The tool prompted referrers to record whether they deemed recommendations to be clinically necessary and assessed whether clinically necessary follow-up imaging was pursued. If imaging was not performed within 1 month after the intended completion date, the tool prompted a safety net team to conduct further patient and referrer follow-up. The study included patients for whom a follow-up imaging recommendation deemed clinically necessary by the referrer was entered with the tool from October 21, 2019, through June 30, 2021. The electronic health record was reviewed for documentation of eventual completion of the recommended imaging at the study institution or an outside institution. Multivariable logistic regression analysis was performed to identify factors associated with completion of follow-up imaging. RESULTS. Of 5856 recommendations entered during the study period, the referrer agreed with 4881 recommendations in 4599 patients (2929 women, 1670 men; mean age, 61.3 ± 15.6 years), who formed the study sample. Follow-up was completed for 74.8% (3651/4881) of recommendations. Independent predictors of lower likelihood of completing follow-up imaging included living in a socioeconomically disadvantaged neighborhood according to the area deprivation index (odds ratio [OR], 0.67 [95% CI, 0.54-0.84]), inpatient (OR, 0.25 [95% CI, 0.20-0.32]) or emergency department (OR, 0.09 [95% CI, 0.05-0.15]) care setting, and referrer surgical specialty (OR, 0.70 [95% CI, 0.58-0.84]). Patient age, race and ethnicity, primary language, and insurance status were not independent predictors of completing follow-up (p > .05). CONCLUSION. Socioeconomically disadvantaged patients are at increased risk of not completing recommended follow-up imaging that referrers deem clinically necessary. CLINICAL IMPACT. Initiatives for ensuring completion of follow-up imaging should be aimed at the identified patient groups to reduce disparities in missed and delayed diagnoses.

Utilization of Natural Language Processing Software to Identify Worrisome Pancreatic Lesions

BACKGROUND

Computed tomography (CT) imaging is routinely obtained for diagnostics, especially in trauma and emergency rooms, often identifying incidental findings. We utilized a natural language processing (NLP) algorithm to quantify the incidence of clinically relevant pancreatic lesions in CT imaging.

PATIENTS AND METHODS

We utilized the electronic medical record to perform a retrospective chart review of adult patients admitted for trauma to a level 1 tertiary care center between 2010 and 2020 who underwent abdominal CT imaging. An open-source NLP software was used to identify patients with intrapapillary mucinous neoplasms (IPMN), pancreatic cysts, pancreatic ductal dilation, or pancreatic masses after optimizing the algorithm using a test group of patients who underwent pancreatic surgery.

RESULTS

The algorithm identified pancreatic lesions in 27 of 28 patients who underwent pancreatic surgery and excluded 1 patient who had a pure ampullary mass. The study cohort consisted of 18,769 patients who met our inclusion criteria admitted to the hospital. Of this population, 232 were found to have pancreatic lesions of interest. There were 48 (20.7%) patients with concern for IPMN, pancreatic cysts in 36 (15.5%), concerning masses in 30 (12.9%), traumatic findings in 44 (19.0%), pancreatitis in 41 (17.7%), and ductal abnormalities in 19 (18.2%) patients. Prior pancreatic surgery and other findings were identified in 14 (6.0%) patients.

CONCLUSIONS

In this study, we propose a novel use of NLP software to identify potentially malignant pancreatic lesions annotated in CT imaging performed for other purposes. This methodology can significantly increase the screening and automated referral for the management of precancerous lesions.

Informatics Solutions to Mitigate Legal Risk Associated With Communication Failures

Communication failures are a documented cause of malpractice litigation against radiologists. As imaging volumes have increased, and with them the number of findings requiring further workup, radiologists are increasingly expected to communicate with ordering clinicians. However, communication may be unsuccessful for a variety of reasons that expose radiologists to potential malpractice risk. Informatics solutions have the potential to improve communication and decrease this risk. We discuss human-powered, purely automated, and hybrid approaches to closing the communications loop. In addition, we describe the Patient Test Results Information Act (Pennsylvania Act 112) and its implications for closing the loop on noncritical actionable findings.

Incidental Breast Cancer on Chest CT: Is the Radiology Report Enough?

OBJECTIVE

To determine the frequency of incidental breast findings reported on chest CT for which breast imaging follow-up is recommended, the follow-up adherence rate, and the breast malignancy rate. The relationship between strength of recommendation verbiage and follow-up was also explored.

METHODS

A retrospective review was conducted of chest CT reports from July 1, 2018, to June 30, 2019, to identify those with recommendation for breast imaging follow-up. Patients with recently diagnosed or prior history of breast malignancy were excluded. Medical records were reviewed to evaluate patient adherence to follow-up, subsequent BI-RADS assessment, and diagnosis (if tissue sampling performed). Adherence was defined as diagnostic breast imaging performed within 6 months of CT recommendation. Chi-square and Mann-Whitney U tests were used to determine statistical significance of categorical and continuous variables, respectively.

RESULTS

A follow-up recommendation for breast imaging was included in chest CT reports of 210 patients; 23% (48/210) returned for follow-up breast imaging. All patients assessed as BI-RADS 4 or 5 underwent image-guided biopsy. Incidental breast cancer was diagnosed in 15% (7/48) of patients who underwent follow-up breast imaging as a result of a CT report recommendation and 78% (7/9) of patients undergoing biopsy. There was no significant difference in follow-up adherence when comparing report verbiage strength.

CONCLUSION

It is imperative that incidental breast findings detected on chest CT undergo follow-up breast imaging to establish accurate and timely diagnosis of breast malignancy. Outreach to referring providers and patients may have greater impact on the diagnosis of previously unsuspected breast cancer.

Adherence to Follow-up Testing Recommendations in US Veterans Screened for Lung Cancer, 2015-2019

IMPORTANCE

Lung cancer screening (LCS) can reduce lung cancer mortality with close follow-up and adherence to management recommendations. Little is known about factors associated with adherence to LCS in real-world practice, with data limited to case series from selected LCS programs.

OBJECTIVE

To analyze adherence to follow-up based on standardized follow-up recommendations in a national cohort and to identify factors associated with delayed or absent follow-up.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study was conducted in Veterans Health Administration (VHA) facilities across the US. Veterans were screened for lung cancer between 2015 to 2019 with sufficient follow-up time to receive recommended evaluation. Patient- and facility-level logistic regression analyses were performed. Data were analyzed from November 26, 2019, to December 16, 2020.

MAIN OUTCOMES AND MEASURES

Receipt of the recommended next step after initial LCS according to Lung CT Screening Reporting & Data System (Lung-RADS) category, as captured in VHA or Medicare claims.

RESULTS

Of 28 294 veterans (26 835 [94.8%] men; 21 969 individuals [77.6%] were White; mean [SD] age, 65.2 [5.5] years) who had an initial LCS examination, 17 863 veterans (63.1%) underwent recommended follow-up within the expected timeframe, whereas 3696 veterans (13.1%) underwent late evaluation, and 4439 veterans (15.7%) had no apparent evaluation. Facility-level differences were associated with 9.2% of the observed variation in rates of late or absent evaluation. In multivariable-adjusted models, Black veterans (odds ratio [OR], 1.19 [95% CI, 1.10-1.29]), veterans with posttraumatic stress disorder (OR, 1.13 [95% CI, 1.03-1.23]), veterans with substance use disorders (OR, 1.11 [95% CI, 1.01-1.22]), veterans with lower income (OR, 0.88 [95% CI, 0.79-0.98]), and those living at a greater distance from a VHA facility (OR, 1.06 [95% CI, 1.02-1.10]) were more likely to experience delayed or no follow-up; veterans with higher risk findings (Lung-RADS category 4 vs Lung-RADS category 1: OR, 0.35 [95% CI, 0.28-0.43]) and those screened in high LCS volume facilities (OR, 0.38 [95% CI, 0.21-0.67]) or academic facilities (OR, 0.86 [95% CI, 0.80-0.92]) were less likely to experience delayed or no follow-up. In sensitivity analyses, varying how stringently adherence was defined, expected evaluation ranged from 14 486 veterans (49.7%) under stringent definitions to 20 578 veterans (78.8%) under liberal definitions.

CONCLUSIONS AND RELEVANCE

In this cohort study that captured follow-up care from the integrated VHA health care system and Medicare, less than two-thirds of patients received timely recommended follow-up after initial LCS, with higher risk of delayed or absent follow-up among marginalized populations, such as Black individuals, individuals with mental health disorders, and individuals with low income, that have long experienced disparities in lung cancer outcomes. Future work should focus on identifying facilities that promote high adherence and disseminating successful strategies to promote equity in LCS among marginalized populations.

Improvement in Follow-up Imaging With a Patient Tracking System and Computerized Registry for Lung Nodule Management

PURPOSE

Despite established guidelines, radiologists' recommendations and timely follow-up of incidental lung nodules remain variable. To improve follow-up of nodules, a system using standardized language (tracker phrases) recommending time-based follow-up in chest CT reports, coupled with a computerized registry, was created.

MATERIALS AND METHODS

Data were obtained from the electronic health record and a facility-built electronic lung nodule registry. We evaluated two randomly selected patient cohorts with incidental nodules on chest CT reports: before intervention (September 2008 to March 2011) and after intervention (August 2011 to December 2016). Multivariable logistic regression was used to compare the cohorts for the main outcome of timely follow-up, defined as a subsequent report within 13 months of the initial report.

RESULTS

In all, 410 patients were included in the pretracker cohort versus 626 in the tracker cohort. Before system inception, 30% of CT reports lacked an explicit time-based recommendation for nodule follow-up. The proportion of patients with timely follow-up increased from 46% to 55%, and the proportion of those with no documented follow-up or follow-up beyond 24 months decreased from 48% to 31%. The likelihood of timely follow-up increased 41%, adjusted for high risk for lung cancer and age 65 years or older. After system inception, reports missing a tracker phrase for nodule recommendation averaged 6%, without significant interyear variation.

CONCLUSIONS

Standardized language added to CT reports combined with a computerized registry designed to identify and track patients with incidental lung nodules was associated with improved likelihood of follow-up imaging.

RADAR: A Closed-Loop Quality Improvement Initiative Leveraging A Safety Net Model for Incidental Pulmonary Nodule Management

BACKGROUND

This study was conducted to assess whether patients with incidental pulmonary nodules (IPNs) received timely follow-up care after implementation of a quality improvement (QI) initiative between radiologists and primary care providers (PCPs).

METHODS

A QI study was conducted at an academic medical center for IPNs identified on chest imaging ordered by PCPs, performed between February 1, 2017, and March 31, 2019, and with at least one-year follow-up. A QI initiative, RADAR (Radiology Result Alert and Development of Automated Resolution), was implemented on March 1, 2018, consisting of (1) a novel, electronic communication tool enabling radiologist-generated alerts with time frame and modality for IPN follow-up recommendations, and (2) a safety net team for centralized care coordination to ensure that communication loops were closed. A preintervention IPN cohort was generated through a natural language processing (NLP) algorithm for radiology reports paired with manual chart review. A postintervention IPN cohort was identified using alerts captured in RADAR. The primary outcome was percentage of IPN follow-up alerts resolved on time (defined as receiving follow-up care within the recommended time frame), comparing pre- and postintervention IPN cohorts. Secondary outcomes included agreement between PCPs and radiologists on the recommended follow-up care plan.

RESULTS

A total of 218 IPN alerts were assessed following exclusions: 110 preintervention and 108 postintervention. IPN timely follow-up improved from 64.5% (71/110) to 84.3% (91/108) (p = 0.001). Postintervention, there was 87.0% (94/108) agreement between PCPs and radiologists on the recommended follow-up plan.

CONCLUSION

The RADAR QI initiative was associated with increased timely IPN follow-up. This safety net model may be scaled to other radiology findings and clinical care settings.

A Web Application for Adrenal Incidentaloma Identification, Tracking, and Management Using Machine Learning

BACKGROUND

Incidental radiographic findings, such as adrenal nodules, are commonly identified in imaging studies and documented in radiology reports. However, patients with such findings frequently do not receive appropriate follow-up, partially due to the lack of tools for the management of such findings and the time required to maintain up-to-date lists. Natural language processing (NLP) is capable of extracting information from free-text clinical documents and could provide the basis for software solutions that do not require changes to clinical workflows.

OBJECTIVES

In this manuscript we present (1) a machine learning algorithm we trained to identify radiology reports documenting the presence of a newly discovered adrenal incidentaloma, and (2) the web application and results database we developed to manage these clinical findings.

METHODS

We manually annotated a training corpus of 4,090 radiology reports from across our institution with a binary label indicating whether or not a report contains a newly discovered adrenal incidentaloma. We trained a convolutional neural network to perform this text classification task. Over the NLP backbone we built a web application that allows users to coordinate clinical management of adrenal incidentalomas in real time.

RESULTS

The annotated dataset included 404 positive (9.9%) and 3,686 (90.1%) negative reports. Our model achieved a sensitivity of 92.9% (95% confidence interval: 80.9-97.5%), a positive predictive value of 83.0% (69.9-91.1)%, a specificity of 97.8% (95.8-98.9)%, and an F1 score of 87.6%. We developed a front-end web application based on the model's output.

CONCLUSION

Developing an NLP-enabled custom web application for tracking and management of high-risk adrenal incidentalomas is feasible in a resource constrained, safety net hospital. Such applications can be used by an institution's quality department or its primary care providers and can easily be generalized to other types of clinical findings.

Closing the Loop-A Pilot in Health System Improvement

A significant number of patients are reported to not receive timely completion of their recommended follow-up intervention following the interpretation of their imaging studies, contributing to patient deaths resulting from inaccurate or delayed diagnosis. Though automated critical test notification systems and computerized communication mechanisms currently exist, many institutions are discovering that there continue to be gaps in the completion of follow-up recommendations. Herein, we describe how we developed and implemented a closed-loop program dedicated to identifying such gaps and ensuring patients were aware of and received appropriate follow-up.