Use of a Dual Artificial Intelligence Platform to Detect Unreported Lung Nodules

Artificial intelligence-assisted double reading of chest radiographs to detect clinically relevant missed findings: a two-centre evaluation

OBJECTIVES

To evaluate an artificial intelligence (AI)-assisted double reading system for detecting clinically relevant missed findings on routinely reported chest radiographs.

METHODS

A retrospective study was performed in two institutions, a secondary care hospital and tertiary referral oncology centre. Commercially available AI software performed a comparative analysis of chest radiographs and radiologists' authorised reports using a deep learning and natural language processing algorithm, respectively. The AI-detected discrepant findings between images and reports were assessed for clinical relevance by an external radiologist, as part of the commercial service provided by the AI vendor. The selected missed findings were subsequently returned to the institution's radiologist for final review.

RESULTS

In total, 25,104 chest radiographs of 21,039 patients (mean age 61.1 years ± 16.2 [SD]; 10,436 men) were included. The AI software detected discrepancies between imaging and reports in 21.1% (5289 of 25,104). After review by the external radiologist, 0.9% (47 of 5289) of cases were deemed to contain clinically relevant missed findings. The institution's radiologists confirmed 35 of 47 missed findings (74.5%) as clinically relevant (0.1% of all cases). Missed findings consisted of lung nodules (71.4%, 25 of 35), pneumothoraces (17.1%, 6 of 35) and consolidations (11.4%, 4 of 35).

CONCLUSION

The AI-assisted double reading system was able to identify missed findings on chest radiographs after report authorisation. The approach required an external radiologist to review the AI-detected discrepancies. The number of clinically relevant missed findings by radiologists was very low.

CLINICAL RELEVANCE STATEMENT

The AI-assisted double reader workflow was shown to detect diagnostic errors and could be applied as a quality assurance tool. Although clinically relevant missed findings were rare, there is potential impact given the common use of chest radiography.

KEY POINTS

• A commercially available double reading system supported by artificial intelligence was evaluated to detect reporting errors in chest radiographs (n=25,104) from two institutions. • Clinically relevant missed findings were found in 0.1% of chest radiographs and consisted of unreported lung nodules, pneumothoraces and consolidations. • Applying AI software as a secondary reader after report authorisation can assist in reducing diagnostic errors without interrupting the radiologist's reading workflow. However, the number of AI-detected discrepancies was considerable and required review by a radiologist to assess their relevance.

Perceptions of radiologists on structured reporting for cancer imaging-a survey by the European Society of Oncologic Imaging (ESOI)

OBJECTIVES

To assess radiologists' current use of, and opinions on, structured reporting (SR) in oncologic imaging, and to provide recommendations for a structured report template.

MATERIALS AND METHODS

An online survey with 28 questions was sent to European Society of Oncologic Imaging (ESOI) members. The questionnaire had four main parts: (1) participant information, e.g., country, workplace, experience, and current SR use; (2) SR design, e.g., numbers of sections and fields, and template use; (3) clinical impact of SR, e.g., on report quality and length, workload, and communication with clinicians; and (4) preferences for an oncology-focused structured CT report. Data analysis comprised descriptive statistics, chi-square tests, and Spearman correlation coefficients.

RESULTS

A total of 200 radiologists from 51 countries completed the survey: 57.0% currently utilized SR (57%), with a lower proportion within than outside of Europe (51.0 vs. 72.7%; p = 0.006). Among SR users, the majority observed markedly increased report quality (62.3%) and easier comparison to previous exams (53.5%), a slightly lower error rate (50.9%), and fewer calls/emails by clinicians (78.9%) due to SR. The perceived impact of SR on communication with clinicians (i.e., frequency of calls/emails) differed with radiologists' experience (p < 0.001), and experience also showed low but significant correlations with communication with clinicians (r = - 0.27, p = 0.003), report quality (r = 0.19, p = 0.043), and error rate (r = - 0.22, p = 0.016). Template use also affected the perceived impact of SR on report quality (p = 0.036).

CONCLUSION

Radiologists regard SR in oncologic imaging favorably, with perceived positive effects on report quality, error rate, comparison of serial exams, and communication with clinicians.

CLINICAL RELEVANCE STATEMENT

Radiologists believe that structured reporting in oncologic imaging improves report quality, decreases the error rate, and enables better communication with clinicians. Implementation of structured reporting in Europe is currently below the international level and needs society endorsement.

KEY POINTS

• The majority of oncologic imaging specialists (57% overall; 51% in Europe) use structured reporting in clinical practice. • The vast majority of oncologic imaging specialists use templates (92.1%), which are typically cancer-specific (76.2%). • Structured reporting is perceived to markedly improve report quality, communication with clinicians, and comparison to prior scans.

Diagnostic value of artificial intelligence based on computed tomography (CT) density in benign and malignant pulmonary nodules: a retrospective investigation

Objective

To evaluate the diagnostic value of artificial intelligence (AI) in the detection and management of benign and malignant pulmonary nodules (PNs) using computed tomography (CT) density.

Methods

A retrospective analysis was conducted on the clinical data of 130 individuals diagnosed with PNs based on pathological confirmation. The utilization of AI and physicians has been employed in the diagnostic process of distinguishing benign and malignant PNs. The CT images depicting PNs were integrated into AI-based software. The gold standard for evaluating the accuracy of AI diagnosis software and physician interpretation was the pathological diagnosis.

Results

Out of 226 PNs screened from 130 patients diagnosed by AI and physician reading based on CT, 147 were confirmed by pathology. AI had a sensitivity of 94.69% and radiologists had a sensitivity of 85.40% in identifying PNs. The chi-square analysis indicated that the screening capacity of AI was superior to that of physician reading, with statistical significance (p < 0.05). 195 of the 214 PNs suggested by AI were confirmed pathologically as malignant, and 19 were identified as benign; among the 29 PNs suggested by AI as low risk, 13 were confirmed pathologically as malignant, and 16 were identified as benign. From the physician reading, 193 PNs were identified as malignant, 183 were confirmed malignant by pathology, and 10 appeared benign. Physician reading also identified 30 low-risk PNs, 19 of which were pathologically malignant and 11 benign. The physician readings and AI had kappa values of 0.432 and 0.547, respectively. The physician reading and AI area under curves (AUCs) were 0.814 and 0.798, respectively. Both of the diagnostic techniques had worthy diagnostic value, as indicated by their AUCs of >0.7.

Conclusion

It is anticipated that the use of AI-based CT diagnosis in the detection of PNs would increase the precision in early detection of lung carcinoma, as well as yield more precise evidence for clinical management.

Natural Language Processing for Breast Imaging: A Systematic Review

Natural Language Processing (NLP) has gained prominence in diagnostic radiology, offering a promising tool for improving breast imaging triage, diagnosis, lesion characterization, and treatment management in breast cancer and other breast diseases. This review provides a comprehensive overview of recent advances in NLP for breast imaging, covering the main techniques and applications in this field. Specifically, we discuss various NLP methods used to extract relevant information from clinical notes, radiology reports, and pathology reports and their potential impact on the accuracy and efficiency of breast imaging. In addition, we reviewed the state-of-the-art in NLP-based decision support systems for breast imaging, highlighting the challenges and opportunities of NLP applications for breast imaging in the future. Overall, this review underscores the potential of NLP in enhancing breast imaging care and offers insights for clinicians and researchers interested in this exciting and rapidly evolving field.

Clinical Implementation of a Combined Artificial Intelligence and Natural Language Processing Quality Assurance Program for Pulmonary Nodule Detection in the Emergency Department Setting

OBJECTIVE

This quality assurance study assessed the implementation of a combined artificial intelligence (AI) and natural language processing (NLP) program for pulmonary nodule detection in the emergency department setting. The program was designed to function outside of normal reading workflows to minimize radiologist interruption.

MATERIALS AND METHODS

In all, 19,246 CT examinations including at least some portion of the lung anatomy performed in the emergent setting from October 1, 2021, to June 1, 2022, were processed by the combined AI-NLP program. The program used an AI algorithm trained on 6-mm to 30-mm pulmonary nodules to analyze CT images and an NLP to analyze radiological reports. Cases flagged as negative for pulmonary nodules by the NLP but positive by the AI algorithm were classified as suspected discrepancies. Discrepancies result in secondary review of examinations for possible addenda.

RESULTS

Out of 19,246 CT examinations, 50 examinations (0.26%) resulted in secondary review, and 34 of 50 (68%) reviews resulted in addenda. Of the 34 addenda, 20 patients received instruction for new follow-up imaging. Median time to addendum was 11 hours. The majority of reviews and addenda resulted from missed pulmonary nodules on CT examinations of the abdomen and pelvis.

CONCLUSION

A background quality assurance process using AI and NLP helped improve the detection of pulmonary nodules and resulted in increased numbers of patients receiving appropriate follow-up imaging recommendations. This was achieved without disrupting in-shift radiologist workflow or causing significant delays in patient follow for the diagnosed pulmonary nodule.

Artificial Intelligence (AI) for Lung Nodules, From the AJR Special Series on AI Applications

Interest in artificial intelligence (AI) applications for lung nodules continues to grow among radiologists, particularly with the expanding eligibility criteria and clinical utilization of lung cancer screening CT. AI has been heavily investigated for detecting and characterizing lung nodules and for guiding prognostic assessment. AI tools have also been used for image postprocessing (e.g., rib suppression on radiography or vessel suppression on CT) and for noninterpretive aspects of reporting and workflow, including management of nodule follow-up. Despite growing interest in and rapid development of AI tools and FDA approval of AI tools for pulmonary nodule evaluation, integration into clinical practice has been limited. Challenges to clinical adoption have included concerns about generalizability, regulatory issues, technical hurdles in implementation, and human skepticism. Further validation of AI tools for clinical use and demonstration of benefit in terms of patient-oriented outcomes also are needed. This article provides an overview of potential applications of AI tools in the imaging evaluation of lung nodules and discusses the challenges faced by practices interested in clinical implementation of such tools.

Frequency of Missed Findings on Chest Radiographs (CXRs) in an International, Multicenter Study: Application of AI to Reduce Missed Findings

Background: Missed findings in chest X-ray interpretation are common and can have serious consequences. Methods: Our study included 2407 chest radiographs (CXRs) acquired at three Indian and five US sites. To identify CXRs reported as normal, we used a proprietary radiology report search engine based on natural language processing (mPower, Nuance). Two thoracic radiologists reviewed all CXRs and recorded the presence and clinical significance of abnormal findings on a 5-point scale (1—not important; 5—critical importance). All CXRs were processed with the AI model (Qure.ai) and outputs were recorded for the presence of findings. Data were analyzed to obtain area under the ROC curve (AUC). Results: Of 410 CXRs (410/2407, 18.9%) with unreported/missed findings, 312 (312/410, 76.1%) findings were clinically important: pulmonary nodules (n = 157), consolidation (60), linear opacities (37), mediastinal widening (21), hilar enlargement (17), pleural effusions (11), rib fractures (6) and pneumothoraces (3). AI detected 69 missed findings (69/131, 53%) with an AUC of up to 0.935. The AI model was generalizable across different sites, geographic locations, patient genders and age groups. Conclusion: A substantial number of important CXR findings are missed; the AI model can help to identify and reduce the frequency of important missed findings in a generalizable manner.

Diagnostic study on clinical feasibility of an AI-based diagnostic system as a second reader on mobile CT images: a preliminary result

Background

Artificial intelligence (AI) has breathed new life into the lung nodules detection and diagnosis. However, whether the output information from AI will translate into benefits for clinical workflow or patient outcomes in a real-world setting remains unknown. This study was to demonstrate the feasibility of an AI-based diagnostic system deployed as a second reader in imaging interpretation for patients screened for pulmonary abnormalities in a clinical setting.

Methods

The study included patients from a lung cancer screening program conducted in Sichuan Province, China using a mobile computed tomography (CT) scanner which traveled to medium-size cities between July 10^th, 2020 and September 10^th, 2020. Cases that were suspected to have malignant nodules by junior radiologists, senior radiologists or AI were labeled a high risk (HR) tag as HR-junior, HR-senior and HR-AI, respectively, and included into final analysis. The diagnosis efficacy of the AI was evaluated by calculating negative predictive value and positive predictive value when referring to the senior readers’ final results as the gold standard. Besides, characteristics of the lesions were compared among cases with different HR labels.

Results

In total, 251/3,872 patients (6.48%, male/female: 91/160, median age, 66 years) with HR lung nodules were included. The AI algorithm achieved a negative predictive value of 88.2% [95% confidence interval (CI): 62.2–98.0%] and a positive predictive value of 55.6% (95% CI: 49.0–62.0%). The diagnostic duration was significantly reduced when AI was used as a second reader (223±145.6 vs. 270±143.17 s, P<0.001). The information yielded by AI affected the radiologist’s decision-making in 35/145 cases. Lesions of HR cases had a higher volume [309.9 (214.9–732.5) vs. 141.3 (79.3–380.8) mm³, P<0.001], lower average CT number [−511.0 (−576.5 to −100.5) vs. −191.5 (−487.3 to 22.5), P=0.010], and pure ground glass opacity rather than solid.

Conclusions

The AI algorithm had high negative predictive value but low positive predictive value in diagnosing HR lung lesions in a clinical setting. Deploying AI as a second reader could help avoid missed diagnoses, reduce diagnostic duration, and strengthen diagnostic confidence for radiologists.

Epipericardial Fat Necrosis: A Retrospective Analysis in Japan

Objective Epipericardial fat necrosis (EFN) has been considered to be a rare cause of acute chest pain, and especially important for emergency physicians. Chest computed tomography (CT) is often used for the diagnosis of EFN after excluding life-threatening states, such as acute coronary syndrome and pulmonary embolism. While the proportion of EFN patients who underwent chest CT in emergency departments is being clarified, little is still known about other departments in Japan. To investigate the proportion of EFN patients who underwent chest CT for acute chest pain in various departments. Methods Chest CT performed from January 2015 to July 2020 in Asahikawa Medical University Hospital in Japan was retrospectively analyzed in this study. All images were reviewed by two radiologists. Results There were 373 outpatients identified by a search using the word 'chest pain' who underwent chest CT. Eight patients satisfying the imaging criteria were diagnosed with EFN. The proportions of patients diagnosed with EFN were 10.7%, 4.8%, 2.8%, 0.9% and 0% in the departments of general medicine, cardiovascular surgery, emergency medicine, cardiovascular internal medicine and respiratory medicine, respectively. Only 12.5% of the patients were correctly diagnosed with EFN, and the other patients were treated for musculoskeletal symptoms, acute pericarditis or hypochondriasis. Conclusion EFN is not rare and is often overlooked in various departments. All physicians as well as emergency physicians should consider the possibility of EFN as the cause of pleuritic chest pain.

Development of a Structured Query Language and Natural Language Processing Algorithm to Identify Lung Nodules in a Cancer Centre

Importance: The stratification of indeterminate lung nodules is a growing problem, but the burden of lung nodules on healthcare services is not well-described. Manual service evaluation and research cohort curation can be time-consuming and potentially improved by automation.

Objective: To automate lung nodule identification in a tertiary cancer centre.

Methods: This retrospective cohort study used Electronic Healthcare Records to identify CT reports generated between 31st October 2011 and 24th July 2020. A structured query language/natural language processing tool was developed to classify reports according to lung nodule status. Performance was externally validated. Sentences were used to train machine-learning classifiers to predict concerning nodule features in 2,000 patients.

Results: 14,586 patients with lung nodules were identified. The cancer types most commonly associated with lung nodules were lung (39%), neuro-endocrine (38%), skin (35%), colorectal (33%) and sarcoma (33%). Lung nodule patients had a greater proportion of metastatic diagnoses (45 vs. 23%, p < 0.001), a higher mean post-baseline scan number (6.56 vs. 1.93, p < 0.001), and a shorter mean scan interval (4.1 vs. 5.9 months, p < 0.001) than those without nodules. Inter-observer agreement for sentence classification was 0.94 internally and 0.98 externally. Sensitivity and specificity for nodule identification were 93 and 99% internally, and 100 and 100% at external validation, respectively. A linear-support vector machine model predicted concerning sentence features with 94% accuracy.

Conclusion: We have developed and validated an accurate tool for automated lung nodule identification that is valuable for service evaluation and research data acquisition.