Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings

Can AI generate diagnostic reports for radiologist approval on CXR images? A multi-reader and multi-case observer performance study

BACKGROUND

Accurately detecting a variety of lung abnormalities from heterogenous chest X-ray (CXR) images and writing radiology reports is often difficult and time-consuming.

OBJECTIVE

To access the utility of a novel artificial intelligence (AI) system (MOM-ClaSeg) in enhancing the accuracy and efficiency of radiologists in detecting heterogenous lung abnormalities through a multi-reader and multi-case (MRMC) observer performance study.

METHODS

Over 36,000 CXR images were retrospectively collected from 12 hospitals over 4 months and used as the experiment group and the control group. In the control group, a double reading method is used in which two radiologists interpret CXR to generate a final report, while in the experiment group, one radiologist generates the final reports based on AI-generated reports.

RESULTS

Compared with double reading, the diagnostic accuracy and sensitivity of single reading with AI increases significantly by 1.49% and 10.95%, respectively (P <  0.001), while the difference in specificity is small (0.22%) and without statistical significance (P = 0.255). Additionally, the average image reading and diagnostic time in the experimental group is reduced by 54.70% (P <  0.001).

CONCLUSION

This MRMC study demonstrates that MOM-ClaSeg can potentially serve as the first reader to generate the initial diagnostic reports, with a radiologist only reviewing and making minor modifications (if needed) to arrive at the final decision. It also shows that single reading with AI can achieve a higher diagnostic accuracy and efficiency than double reading.

Detection of pulmonary nodules in chest radiographs: novel cost function for effective network training with purely synthesized datasets

PURPOSE

Many large radiographic datasets of lung nodules are available, but the small and hard-to-detect nodules are rarely validated by computed tomography. Such difficult nodules are crucial for training nodule detection methods. This lack of difficult nodules for training can be addressed by artificial nodule synthesis algorithms, which can create artificially embedded nodules. This study aimed to develop and evaluate a novel cost function for training networks to detect such lesions. Embedding artificial lesions in healthy medical images is effective when positive cases are insufficient for network training. Although this approach provides both positive (lesion-embedded) images and the corresponding negative (lesion-free) images, no known methods effectively use these pairs for training. This paper presents a novel cost function for segmentation-based detection networks when positive-negative pairs are available.

METHODS

Based on the classic U-Net, new terms were added to the original Dice loss for reducing false positives and the contrastive learning of diseased regions in the image pairs. The experimental network was trained and evaluated, respectively, on 131,072 fully synthesized pairs of images simulating lung cancer and real chest X-ray images from the Japanese Society of Radiological Technology dataset.

RESULTS

The proposed method outperformed RetinaNet and a single-shot multibox detector. The sensitivities were 0.688 and 0.507 when the number of false positives per image was 0.2, respectively, with and without fine-tuning under the leave-one-case-out setting.

CONCLUSION

To our knowledge, this is the first study in which a method for detecting pulmonary nodules in chest X-ray images was evaluated on a real clinical dataset after being trained on fully synthesized images. The synthesized dataset is available at https://zenodo.org/records/10648433 .

Novel Deep CNNs Explore Regions, Boundaries, and Residual Learning for COVID-19 Infection Analysis in Lung CT

COVID-19 poses a global health crisis, necessitating precise diagnostic methods for timely containment. However, accurately delineating COVID-19-affected regions in lung CT scans is challenging due to contrast variations and significant texture diversity. In this regard, this study introduces a novel two-stage classification and segmentation CNN approach for COVID-19 lung radiological pattern analysis. A novel Residual-BRNet is developed to integrate boundary and regional operations with residual learning, capturing key COVID-19 radiological homogeneous regions, texture variations, and structural contrast patterns in the classification stage. Subsequently, infectious CT images undergo lesion segmentation using the newly proposed RESeg segmentation CNN in the second stage. The RESeg leverages both average and max-pooling implementations to simultaneously learn region homogeneity and boundary-related patterns. Furthermore, novel pixel attention (PA) blocks are integrated into RESeg to effectively address mildly COVID-19-infected regions. The evaluation of the proposed Residual-BRNet CNN in the classification stage demonstrates promising performance metrics, achieving an accuracy of 97.97%, F1-score of 98.01%, sensitivity of 98.42%, and MCC of 96.81%. Meanwhile, PA-RESeg in the segmentation phase achieves an optimal segmentation performance with an IoU score of 98.43% and a dice similarity score of 95.96% of the lesion region. The framework's effectiveness in detecting and segmenting COVID-19 lesions highlights its potential for clinical applications.

Nodule Detection and Generation on Chest X-Rays: NODE21 Challenge

Pulmonary nodules may be an early manifestation of lung cancer, the leading cause of cancer-related deaths among both men and women. Numerous studies have established that deep learning methods can yield high-performance levels in the detection of lung nodules in chest X-rays. However, the lack of gold-standard public datasets slows down the progression of the research and prevents benchmarking of methods for this task. To address this, we organized a public research challenge, NODE21, aimed at the detection and generation of lung nodules in chest X-rays. While the detection track assesses state-of-the-art nodule detection systems, the generation track determines the utility of nodule generation algorithms to augment training data and hence improve the performance of the detection systems. This paper summarizes the results of the NODE21 challenge and performs extensive additional experiments to examine the impact of the synthetically generated nodule training images on the detection algorithm performance.

Automatic detection of pulmonary embolism on computed tomography pulmonary angiogram scan using a three-dimensional convolutional neural network

OBJECTIVE

To propose a convolutional neural network (EmbNet) for automatic pulmonary embolism detection on computed tomography pulmonary angiogram (CTPA) scans and to assess its diagnostic performance.

METHODS

305 consecutive CTPA scans between January 2019 and December 2021 were enrolled in this study (142 for training, 163 for internal validation), and 250 CTPA scans from a public dataset were used for external validation. The framework comprised a preprocessing step to segment the pulmonary vessels and the EmbNet to detect emboli. Emboli were divided into three location-based subgroups for detailed evaluation: central arteries, lobar branches, and peripheral regions. Ground truth was established by three radiologists.

RESULTS

The EmbNet's per-scan level sensitivity, specificity, positive predictive value (PPV), and negative predictive value were 90.9%, 75.4%, 48.4%, and 97.0% (internal validation) and 88.0%, 70.5%, 42.7%, and 95.9% (external validation). At the per-embolus level, the overall sensitivity and PPV of the EmbNet were 86.0% and 61.3% (internal validation), and 83.5% and 57.5% (external validation). The sensitivity and PPV of central emboli were 89.7% and 52.0% (internal validation), and 94.4% and 43.0% (external validation); of lobar emboli were 95.2% and 76.9% (internal validation), and 93.5% and 72.5% (external validation); and of peripheral emboli were 82.6% and 61.7% (internal validation), and 80.2% and 59.4% (external validation). The average false positive rate was 0.45 false emboli per scan (internal validation) and 0.69 false emboli per scan (external validation).

CONCLUSION

The EmbNet provides high sensitivity across embolus locations, suggesting its potential utility for initial screening in clinical practice.

Pixel-Wise Interstitial Lung Disease Interval Change Analysis: A Quantitative Evaluation Method for Chest Radiographs Using Weakly Supervised Learning

Interstitial lung disease (ILD) is characterized by progressive pathological changes that require timely and accurate diagnosis. The early detection and progression assessment of ILD are important for effective management. This study introduces a novel quantitative evaluation method utilizing chest radiographs to analyze pixel-wise changes in ILD. Using a weakly supervised learning framework, the approach incorporates the contrastive unpaired translation model and a newly developed ILD extent scoring algorithm for more precise and objective quantification of disease changes than conventional visual assessments. The ILD extent score calculated through this method demonstrated a classification accuracy of 92.98% between ILD and normal classes. Additionally, using an ILD follow-up dataset for interval change analysis, this method assessed disease progression with an accuracy of 85.29%. These findings validate the reliability of the ILD extent score as a tool for ILD monitoring. The results of this study suggest that the proposed quantitative method may improve the monitoring and management of ILD.

Radiographic chest wall abnormalities in primary spontaneous pneumothorax identified by artificial intelligence

Primary spontaneous pneumothorax (PSP) primarily affects slim and tall young males. Exploring the etiological link between chest wall structural characteristics and PSP is crucial for advancing treatment methods. In this case-control study, chest computed tomography (CT) images from patients undergoing thoracic surgery, with or without PSP, were analyzed using Artificial Intelligence. Convolutional Neural Network (CNN) model of EfficientNetB3 and InceptionV3 were used with transfer learning on the Imagenet to compare the images of both groups. A heatmap was created on the chest CT scans to enhance interoperability, and the scale-invariant feature transform (SIFT) was adopted to further compare the image level. A total of 2,312 CT images of 26 non-PSP patients and 1,122 CT images of 26 PSP patients were selected. Chest-wall apex pit (CAP) was found in 25 PSP and three non-PSP patients (p < 0.001). The CNN achieved a testing accuracy of 93.47 % in distinguishing PSP from non-PSP based on chest wall features by identifying the existence of CAP. Heatmap analysis demonstrated CNN's precision in targeting the upper chest wall, accurately identifying CAP without undue influence from similar structures, or inappropriately expanding or minimizing the test area. SIFT results indicated a 10.55 % higher mean similarity within the groups compared to between PSP and non-PSP (p < 0.001). In conclusion, distinctive radiographic chest wall configurations were observed in PSP patients, with CAP potentially serving as an etiological factor linked to PSP. This study accentuates the potential of AI-assisted analysis in refining diagnostic approaches and treatment strategies for PSP.

AI for Detection of Tuberculosis: Implications for Global Health

Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity. Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning Supplemental material is available for this article. © RSNA, 2024.

Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.

Attention-Gated Deep-Learning-Based Automatic Digitization of Interstitial Needles in High-Dose-Rate Brachytherapy for Cervical Cancer

Purpose

Deep learning can be used to automatically digitize interstitial needles in high-dose-rate (HDR) brachytherapy for patients with cervical cancer. The aim of this study was to design a novel attention-gated deep-learning model, which may further improve the accuracy of and better differentiate needles.

Methods and Materials

Seventeen patients with cervical cancer with 56 computed tomography-based interstitial HDR brachytherapy plans from the local hospital were retrospectively chosen with the local institutional review board's approval. Among them, 50 plans were randomly selected as the training set and the rest as the validation set. Spatial and channel attention gates (AGs) were added to 3-dimensional convolutional neural networks (CNNs) to highlight needle features and suppress irrelevant regions; this was supposed to facilitate convergence and improve accuracy of automatic needle digitization. Subsequently, the automatically digitized needles were exported to the Oncentra treatment planning system (Elekta Solutions AB, Stockholm, Sweden) for dose evaluation. The geometric and dosimetric accuracy of automatic needle digitization was compared among 3 methods: (1) clinically approved plans with manual needle digitization (ground truth); (2) the conventional deep-learning (CNN) method; and (3) the attention-added deep-learning (CNN + AG) method, in terms of the Dice similarity coefficient (DSC), tip and shaft positioning errors, dose distribution in the high-risk clinical target volume (HR-CTV), organs at risk, and so on.

Results

The attention-gated CNN model was superior to CNN without AGs, with a greater DSC (approximately 94% for CNN + AG vs 89% for CNN). The needle tip and shaft errors of the CNN + AG method (1.1 mm and 1.8 mm, respectively) were also much smaller than those of the CNN method (2.0 mm and 3.3 mm, respectively). Finally, the dose difference for the HR-CTV D90 using the CNN + AG method was much more accurate than that using CNN (0.4% and 1.7%, respectively).

Conclusions

The attention-added deep-learning model was successfully implemented for automatic needle digitization in HDR brachytherapy, with clinically acceptable geometric and dosimetric accuracy. Compared with conventional deep-learning neural networks, attention-gated deep learning may have superior performance and great clinical potential.

Clinical outcomes and actual consequence of lung nodules incidentally detected on chest radiographs by artificial intelligence

This study evaluated how often clinically significant lung nodules were detected unexpectedly on chest radiographs (CXR) by artificial intelligence (AI)-based detection software, and whether co-existing findings can aid in differential diagnosis of lung nodules. Patients (> 18 years old) with AI-detected lung nodules at their first visit from March 2021 to February 2022, except for those in the pulmonology or thoracic surgery departments, were retrospectively included. Three radiologists categorized nodules into malignancy, active inflammation, post-inflammatory sequelae, or "other" groups. Characteristics of the nodule and abnormality scores of co-existing lung lesions were compared. Approximately 1% of patients (152/14,563) had unexpected lung nodules. Among 73 patients with follow-up exams, 69.9% had true positive nodules. Increased abnormality scores for nodules were significantly associated with malignancy (odds ratio [OR] 1.076, P = 0.001). Increased abnormality scores for consolidation (OR 1.033, P = 0.040) and pleural effusion (OR 1.025, P = 0.041) were significantly correlated with active inflammation-type nodules. Abnormality scores for fibrosis (OR 1.036, P = 0.013) and nodules (OR 0.940, P = 0.001) were significantly associated with post-inflammatory sequelae categorization. AI-based lesion-detection software of CXRs in daily practice can help identify clinically significant incidental lung nodules, and referring accompanying lung lesions may help classify the nodule.

Time to change? Present and prospects of hemorrhoidal classification

As a common benign anal condition, the high incidence and recurrence of hemorrhoids pose challenges for both patients and doctors. The classification of hemorrhoids plays a crucial role in assessing, diagnosing, and treating the condition. By using appropriate classification and corresponding treatment strategies, we can achieve higher cure rates and lower recurrence rates of hemorrhoids. Since the introduction of the Miles classification in 1919, various classifications have been developed, which include objective classifications based on anatomical or instrumental assessment and subjective classifications based on symptoms and patient sensations. These classifications aim to accurately evaluate the condition. In this study, we discuss the evaluation values of each classification in terms of their advantages, disadvantages, treatment relevance, reproducibility, practicality, and assessment value. We also analyze the significant and essential factors, principles of use, and components of assessment indicators of hemorrhoidal classification. This study proposes several strategies to address the limitations of current hemorrhoidal assessment methods. All these will provide a reference for the development regarding the assessment and classification of hemorrhoids and clinical diagnosis and management of hemorrhoids.

The diagnostic performance and clinical value of deep learning-based nodule detection system concerning influence of location of pulmonary nodule

BACKGROUND

The deep learning-based nodule detection (DLD) system improves nodule detection performance of observers on chest radiographs (CXRs). However, its performance in different pulmonary nodule (PN) locations remains unknown.

METHODS

We divided the CXR intrathoracic region into non-danger zone (NDZ) and danger zone (DZ). The DZ included the lung apices, paramediastinal areas, and retrodiaphragmatic areas, where nodules could be missed. We used a dataset of 300 CXRs (100 normal and 200 abnormal images with 216 PNs [107 NDZ and 109 DZ nodules]). Eight observers (two thoracic radiologists [TRs], two non-thoracic radiologists [NTRs], and four radiology residents [RRs]) interpreted each radiograph with and without the DLD system. The metric of lesion localization fraction (LLF; the number of correctly localized lesions divided by the total number of true lesions) was used to evaluate the diagnostic performance according to the nodule location.

RESULTS

The DLD system demonstrated a lower LLF for the detection of DZ nodules (64.2) than that of NDZ nodules (83.2, p = 0.008). For DZ nodule detection, the LLF of the DLD system (64.2) was lower than that of TRs (81.7, p < 0.001), which was comparable to that of NTRs (56.4, p = 0.531) and RRs (56.7, p = 0.459). Nonetheless, the LLF of RRs significantly improved from 56.7 to 65.6 using the DLD system (p = 0.021) for DZ nodule detection.

CONCLUSION

The performance of the DLD system was lower in the detection of DZ nodules compared to that of NDZ nodules. Nonetheless, RR performance in detecting DZ nodules improved upon using the DLD system.

CRITICAL RELEVANCE STATEMENT

Despite the deep learning-based nodule detection system's limitations in detecting danger zone nodules, it proves beneficial for less-experienced observers by providing valuable assistance in identifying these nodules, thereby advancing nodule detection in clinical practice.

KEY POINTS

• The deep learning-based nodule detection (DLD) system can improve the diagnostic performance of observers in nodule detection. • The DLD system shows poor diagnostic performance in detecting danger zone nodules. • For less-experienced observers, the DLD system is helpful in detecting danger zone nodules.

Generative Adversarial Network (Generative Artificial Intelligence) in Pediatric Radiology: A Systematic Review

Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1-158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.

Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis

BACKGROUND

Deep learning (DL), a subset of artificial intelligence (AI), has been applied to pneumothorax diagnosis to aid physician diagnosis, but no meta-analysis has been performed.

METHODS

A search of multiple electronic databases through September 2022 was performed to identify studies that applied DL for pneumothorax diagnosis using imaging. Meta-analysis via a hierarchical model to calculate the summary area under the curve (AUC) and pooled sensitivity and specificity for both DL and physicians was performed. Risk of bias was assessed using a modified Prediction Model Study Risk of Bias Assessment Tool.

RESULTS

In 56 of the 63 primary studies, pneumothorax was identified from chest radiography. The total AUC was 0.97 (95% CI 0.96-0.98) for both DL and physicians. The total pooled sensitivity was 84% (95% CI 79-89%) for DL and 85% (95% CI 73-92%) for physicians and the pooled specificity was 96% (95% CI 94-98%) for DL and 98% (95% CI 95-99%) for physicians. More than half of the original studies (57%) had a high risk of bias.

CONCLUSIONS

Our review found the diagnostic performance of DL models was similar to that of physicians, although the majority of studies had a high risk of bias. Further pneumothorax AI research is needed.

A systematic approach to deep learning-based nodule detection in chest radiographs

Lung cancer is a serious disease responsible for millions of deaths every year. Early stages of lung cancer can be manifested in pulmonary lung nodules. To assist radiologists in reducing the number of overseen nodules and to increase the detection accuracy in general, automatic detection algorithms have been proposed. Particularly, deep learning methods are promising. However, obtaining clinically relevant results remains challenging. While a variety of approaches have been proposed for general purpose object detection, these are typically evaluated on benchmark data sets. Achieving competitive performance for specific real-world problems like lung nodule detection typically requires careful analysis of the problem at hand and the selection and tuning of suitable deep learning models. We present a systematic comparison of state-of-the-art object detection algorithms for the task of lung nodule detection. In this regard, we address the critical aspect of class imbalance and and demonstrate a data augmentation approach as well as transfer learning to boost performance. We illustrate how this analysis and a combination of multiple architectures results in state-of-the-art performance for lung nodule detection, which is demonstrated by the proposed model winning the detection track of the Node21 competition. The code for our approach is available at https://github.com/FinnBehrendt/node21-submit.

Estimating the Volume of Nodules and Masses on Serial Chest Radiography Using a Deep-Learning-Based Automatic Detection Algorithm: A Preliminary Study

BACKGROUND

The purpose of this study was to assess the volume of the pulmonary nodules and masses on serial chest X-rays (CXRs) from deep-learning-based automatic detection algorithm (DLAD)-based parameters.

METHODS

In a retrospective single-institutional study, 72 patients, who obtained serial CXRs (n = 147) for pulmonary nodules or masses with corresponding chest CT images as the reference standards, were included. A pre-trained DLAD based on a convolutional neural network was developed to detect and localize nodules using 13,710 radiographs and to calculate a localization map and the derived parameters (e.g., the area and mean probability value of pulmonary nodules) for each CXR, including serial follow-ups. For validation, reference 3D CT volumes were measured semi-automatically. Volume prediction models for pulmonary nodules were established through univariable or multivariable, and linear or non-linear regression analyses with the parameters. A polynomial regression analysis was performed as a method of a non-linear regression model.

RESULTS

Of the 147 CXRs and 208 nodules of 72 patients, the mean volume of nodules or masses was measured as 9.37 ± 11.69 cm³ (mean ± standard deviation). The area and CT volume demonstrated a linear correlation of moderate strength (i.e., R = 0.58, RMSE: 9449.9 mm³ m³ in a linear regression analysis). The area and mean probability values exhibited a strong linear correlation (R = 0.73). The volume prediction performance based on a multivariable regression model was best with a mean probability and unit-adjusted area (i.e.

, RMSE

7975.6 mm³, the smallest among the other variable parameters).

CONCLUSIONS

The prediction model with the area and the mean probability based on the DLAD showed a rather accurate quantitative estimation of pulmonary nodule or mass volume and the change in serial CXRs.

Artificial Intelligence-Based Software with CE Mark for Chest X-ray Interpretation: Opportunities and Challenges

Chest X-ray (CXR) is the most important technique for performing chest imaging, despite its well-known limitations in terms of scope and sensitivity. These intrinsic limitations of CXR have prompted the development of several artificial intelligence (AI)-based software packages dedicated to CXR interpretation. The online database "AI for radiology" was queried to identify CE-marked AI-based software available for CXR interpretation. The returned studies were divided according to the targeted disease. AI-powered computer-aided detection software is already widely adopted in screening and triage for pulmonary tuberculosis, especially in countries with few resources and suffering from high a burden of this disease. AI-based software has also been demonstrated to be valuable for the detection of lung nodules detection, automated flagging of positive cases, and post-processing through the development of digital bone suppression software able to produce digital bone suppressed images. Finally, the majority of available CE-marked software packages for CXR are designed to recognize several findings, with potential differences in sensitivity and specificity for each of the recognized findings.

Effect of Contrast Level and Image Format on a Deep Learning Algorithm for the Detection of Pneumothorax with Chest Radiography

Under the black-box nature in the deep learning model, it is uncertain how the change in contrast level and format affects the performance. We aimed to investigate the effect of contrast level and image format on the effectiveness of deep learning for diagnosing pneumothorax on chest radiographs. We collected 3316 images (1016 pneumothorax and 2300 normal images), and all images were set to the standard contrast level (100%) and stored in the Digital Imaging and Communication in Medicine and Joint Photographic Experts Group (JPEG) formats. Data were randomly separated into 80% of training and 20% of test sets, and the contrast of images in the test set was changed to 5 levels (50%, 75%, 100%, 125%, and 150%). We trained the model to detect pneumothorax using ResNet-50 with 100% level images and tested with 5-level images in the two formats. While comparing the overall performance between each contrast level in the two formats, the area under the receiver-operating characteristic curve (AUC) was significantly different (all p < 0.001) except between 125 and 150% in JPEG format (p = 0.382). When comparing the two formats at same contrast levels, AUC was significantly different (all p < 0.001) except 50% and 100% (p = 0.079 and p = 0.082, respectively). The contrast level and format of medical images could influence the performance of the deep learning model. It is required to train with various contrast levels and formats of image, and further image processing for improvement and maintenance of the performance.

Multicentre external validation of a commercial artificial intelligence software to analyse chest radiographs in health screening environments with low disease prevalence

OBJECTIVES

To externally validate the performance of a commercial AI software program for interpreting CXRs in a large, consecutive, real-world cohort from primary healthcare centres.

METHODS

A total of 3047 CXRs were collected from two primary healthcare centres, characterised by low disease prevalence, between January and December 2018. All CXRs were labelled as normal or abnormal according to CT findings. Four radiology residents read all CXRs twice with and without AI assistance. The performances of the AI and readers with and without AI assistance were measured in terms of area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.

RESULTS

The prevalence of clinically significant lesions was 2.2% (68 of 3047). The AUROC, sensitivity, and specificity of the AI were 0.648 (95% confidence interval [CI] 0.630-0.665), 35.3% (CI, 24.7-47.8), and 94.2% (CI, 93.3-95.0), respectively. AI detected 12 of 41 pneumonia, 3 of 5 tuberculosis, and 9 of 22 tumours. AI-undetected lesions tended to be smaller than true-positive lesions. The readers' AUROCs ranged from 0.534-0.676 without AI and 0.571-0.688 with AI (all p values < 0.05). For all readers, the mean reading time was 2.96-10.27 s longer with AI assistance (all p values < 0.05).

CONCLUSIONS

The performance of commercial AI in these high-volume, low-prevalence settings was poorer than expected, although it modestly boosted the performance of less-experienced readers. The technical prowess of AI demonstrated in experimental settings and approved by regulatory bodies may not directly translate to real-world practice, especially where the demand for AI assistance is highest.

KEY POINTS

• This study shows the limited applicability of commercial AI software for detecting abnormalities in CXRs in a health screening population. • When using AI software in a specific clinical setting that differs from the training setting, it is necessary to adjust the threshold or perform additional training with such data that reflects this environment well. • Prospective test accuracy studies, randomised controlled trials, or cohort studies are needed to examine AI software to be implemented in real clinical practice.

Conventional Versus Artificial Intelligence-Assisted Interpretation of Chest Radiographs in Patients With Acute Respiratory Symptoms in Emergency Department: A Pragmatic Randomized Clinical Trial

OBJECTIVE

It is unknown whether artificial intelligence-based computer-aided detection (AI-CAD) can enhance the accuracy of chest radiograph (CR) interpretation in real-world clinical practice. We aimed to compare the accuracy of CR interpretation assisted by AI-CAD to that of conventional interpretation in patients who presented to the emergency department (ED) with acute respiratory symptoms using a pragmatic randomized controlled trial.

MATERIALS AND METHODS

Patients who underwent CRs for acute respiratory symptoms at the ED of a tertiary referral institution were randomly assigned to intervention group (with assistance from an AI-CAD for CR interpretation) or control group (without AI assistance). Using a commercial AI-CAD system (Lunit INSIGHT CXR, version 2.0.2.0; Lunit Inc.). Other clinical practices were consistent with standard procedures. Sensitivity and false-positive rates of CR interpretation by duty trainee radiologists for identifying acute thoracic diseases were the primary and secondary outcomes, respectively. The reference standards for acute thoracic disease were established based on a review of the patient's medical record at least 30 days after the ED visit.

RESULTS

We randomly assigned 3576 participants to either the intervention group (1761 participants; mean age ± standard deviation, 65 ± 17 years; 978 males; acute thoracic disease in 472 participants) or the control group (1815 participants; 64 ± 17 years; 988 males; acute thoracic disease in 491 participants). The sensitivity (67.2% [317/472] in the intervention group vs. 66.0% [324/491] in the control group; odds ratio, 1.02 [95% confidence interval, 0.70-1.49]; P = 0.917) and false-positive rate (19.3% [249/1289] vs. 18.5% [245/1324]; odds ratio, 1.00 [95% confidence interval, 0.79-1.26]; P = 0.985) of CR interpretation by duty radiologists were not associated with the use of AI-CAD.

CONCLUSION

AI-CAD did not improve the sensitivity and false-positive rate of CR interpretation for diagnosing acute thoracic disease in patients with acute respiratory symptoms who presented to the ED.

Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

A 178-clinical-center experiment of integrating AI solutions for lung pathology diagnosis

In 2020, an experiment testing AI solutions for lung X-ray analysis on a multi-hospital network was conducted. The multi-hospital network linked 178 Moscow state healthcare centers, where all chest X-rays from the network were redirected to a research facility, analyzed with AI, and returned to the centers. The experiment was formulated as a public competition with monetary awards for participating industrial and research teams. The task was to perform the binary detection of abnormalities from chest X-rays. For the objective real-life evaluation, no training X-rays were provided to the participants. This paper presents one of the top-performing AI frameworks from this experiment. First, the framework used two EfficientNets, histograms of gradients, Haar feature ensembles, and local binary patterns to recognize whether an input image represents an acceptable lung X-ray sample, meaning the X-ray is not grayscale inverted, is a frontal chest X-ray, and completely captures both lung fields. Second, the framework extracted the region with lung fields and then passed them to a multi-head DenseNet, where the heads recognized the patient's gender, age and the potential presence of abnormalities, and generated the heatmap with the abnormality regions highlighted. During one month of the experiment from 11.23.2020 to 12.25.2020, 17,888 cases have been analyzed by the framework with 11,902 cases having radiological reports with the reference diagnoses that were unequivocally parsed by the experiment organizers. The performance measured in terms of the area under receiving operator curve (AUC) was 0.77. The AUC for individual diseases ranged from 0.55 for herniation to 0.90 for pneumothorax.

Prediction of oxygen requirement in patients with COVID-19 using a pre-trained chest radiograph xAI model: efficient development of auditable risk prediction models via a fine-tuning approach

Risk prediction requires comprehensive integration of clinical information and concurrent radiological findings. We present an upgraded chest radiograph (CXR) explainable artificial intelligence (xAI) model, which was trained on 241,723 well-annotated CXRs obtained prior to the onset of the COVID-19 pandemic. Mean area under the receiver operating characteristic curve (AUROC) for detection of 20 radiographic features was 0.955 (95% CI 0.938-0.955) on PA view and 0.909 (95% CI 0.890-0.925) on AP view. Coexistent and correlated radiographic findings are displayed in an interpretation table, and calibrated classifier confidence is displayed on an AI scoreboard. Retrieval of similar feature patches and comparable CXRs from a Model-Derived Atlas provides justification for model predictions. To demonstrate the feasibility of a fine-tuning approach for efficient and scalable development of xAI risk prediction models, we applied our CXR xAI model, in combination with clinical information, to predict oxygen requirement in COVID-19 patients. Prediction accuracy for high flow oxygen (HFO) and mechanical ventilation (MV) was 0.953 and 0.934 at 24 h and 0.932 and 0.836 at 72 h from the time of emergency department (ED) admission, respectively. Our CXR xAI model is auditable and captures key pathophysiological manifestations of cardiorespiratory diseases and cardiothoracic comorbidities. This model can be efficiently and broadly applied via a fine-tuning approach to provide fully automated risk and outcome predictions in various clinical scenarios in real-world practice.

Artificial Intelligence in Lung Imaging

Recently, interest and advances in artificial intelligence (AI) including deep learning for medical images have surged. As imaging plays a major role in the assessment of pulmonary diseases, various AI algorithms have been developed for chest imaging. Some of these have been approved by governments and are now commercially available in the marketplace. In the field of chest radiology, there are various tasks and purposes that are suitable for AI: initial evaluation/triage of certain diseases, detection and diagnosis, quantitative assessment of disease severity and monitoring, and prediction for decision support. While AI is a powerful technology that can be applied to medical imaging and is expected to improve our current clinical practice, some obstacles must be addressed for the successful implementation of AI in workflows. Understanding and becoming familiar with the current status and potential clinical applications of AI in chest imaging, as well as remaining challenges, would be essential for radiologists and clinicians in the era of AI. This review introduces the potential clinical applications of AI in chest imaging and also discusses the challenges for the implementation of AI in daily clinical practice and future directions in chest imaging.

Association of Artificial Intelligence-Aided Chest Radiograph Interpretation With Reader Performance and Efficiency

IMPORTANCE

The efficient and accurate interpretation of radiologic images is paramount.

OBJECTIVE

To evaluate whether a deep learning-based artificial intelligence (AI) engine used concurrently can improve reader performance and efficiency in interpreting chest radiograph abnormalities.

DESIGN, SETTING, AND PARTICIPANTS

This multicenter cohort study was conducted from April to November 2021 and involved radiologists, including attending radiologists, thoracic radiology fellows, and residents, who independently participated in 2 observer performance test sessions. The sessions included a reading session with AI and a session without AI, in a randomized crossover manner with a 4-week washout period in between. The AI produced a heat map and the image-level probability of the presence of the referrable lesion. The data used were collected at 2 quaternary academic hospitals in Boston, Massachusetts: Beth Israel Deaconess Medical Center (The Medical Information Mart for Intensive Care Chest X-Ray [MIMIC-CXR]) and Massachusetts General Hospital (MGH).

MAIN OUTCOMES AND MEASURES

The ground truths for the labels were created via consensual reading by 2 thoracic radiologists. Each reader documented their findings in a customized report template, in which the 4 target chest radiograph findings and the reader confidence of the presence of each finding was recorded. The time taken for reporting each chest radiograph was also recorded. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated for each target finding.

RESULTS

A total of 6 radiologists (2 attending radiologists, 2 thoracic radiology fellows, and 2 residents) participated in the study. The study involved a total of 497 frontal chest radiographs-247 from the MIMIC-CXR data set (demographic data for patients were not available) and 250 chest radiographs from MGH (mean [SD] age, 63 [16] years; 133 men [53.2%])-from adult patients with and without 4 target findings (pneumonia, nodule, pneumothorax, and pleural effusion). The target findings were found in 351 of 497 chest radiographs. The AI was associated with higher sensitivity for all findings compared with the readers (nodule, 0.816 [95% CI, 0.732-0.882] vs 0.567 [95% CI, 0.524-0.611]; pneumonia, 0.887 [95% CI, 0.834-0.928] vs 0.673 [95% CI, 0.632-0.714]; pleural effusion, 0.872 [95% CI, 0.808-0.921] vs 0.889 [95% CI, 0.862-0.917]; pneumothorax, 0.988 [95% CI, 0.932-1.000] vs 0.792 [95% CI, 0.756-0.827]). AI-aided interpretation was associated with significantly improved reader sensitivities for all target findings, without negative impacts on the specificity. Overall, the AUROCs of readers improved for all 4 target findings, with significant improvements in detection of pneumothorax and nodule. The reporting time with AI was 10% lower than without AI (40.8 vs 36.9 seconds; difference, 3.9 seconds; 95% CI, 2.9-5.2 seconds; P < .001).

CONCLUSIONS AND RELEVANCE

These findings suggest that AI-aided interpretation was associated with improved reader performance and efficiency for identifying major thoracic findings on a chest radiograph.

Localization-adjusted diagnostic performance and assistance effect of a computer-aided detection system for pneumothorax and consolidation

While many deep-learning-based computer-aided detection systems (CAD) have been developed and commercialized for abnormality detection in chest radiographs (CXR), their ability to localize a target abnormality is rarely reported. Localization accuracy is important in terms of model interpretability, which is crucial in clinical settings. Moreover, diagnostic performances are likely to vary depending on thresholds which define an accurate localization. In a multi-center, stand-alone clinical trial using temporal and external validation datasets of 1,050 CXRs, we evaluated localization accuracy, localization-adjusted discrimination, and calibration of a commercially available deep-learning-based CAD for detecting consolidation and pneumothorax. The CAD achieved image-level AUROC (95% CI) of 0.960 (0.945, 0.975), sensitivity of 0.933 (0.899, 0.959), specificity of 0.948 (0.930, 0.963), dice of 0.691 (0.664, 0.718), moderate calibration for consolidation, and image-level AUROC of 0.978 (0.965, 0.991), sensitivity of 0.956 (0.923, 0.978), specificity of 0.996 (0.989, 0.999), dice of 0.798 (0.770, 0.826), moderate calibration for pneumothorax. Diagnostic performances varied substantially when localization accuracy was accounted for but remained high at the minimum threshold of clinical relevance. In a separate trial for diagnostic impact using 461 CXRs, the causal effect of the CAD assistance on clinicians' diagnostic performances was estimated. After adjusting for age, sex, dataset, and abnormality type, the CAD improved clinicians' diagnostic performances on average (OR [95% CI] = 1.73 [1.30, 2.32]; p < 0.001), although the effects varied substantially by clinical backgrounds. The CAD was found to have high stand-alone diagnostic performances and may beneficially impact clinicians' diagnostic performances when used in clinical settings.

A Novel CovidDetNet Deep Learning Model for Effective COVID-19 Infection Detection Using Chest Radiograph Images

The suspected cases of COVID-19 must be detected quickly and accurately to avoid the transmission of COVID-19 on a large scale. Existing COVID-19 diagnostic tests are slow and take several hours to generate the required results. However, on the other hand, most X-rays or chest radiographs only take less than 15 min to complete. Therefore, we can utilize chest radiographs to create a solution for early and accurate COVID-19 detection and diagnosis to reduce COVID-19 patient treatment problems and save time. For this purpose, CovidDetNet is proposed, which comprises ten learnable layers that are nine convolutional layers and one fully-connected layer. The architecture uses two activation functions: the ReLu activation function and the Leaky Relu activation function and two normalization operations that are batch normalization and cross channel normalization, making it a novel COVID-19 detection model. It is a novel deep learning-based approach that automatically and reliably detects COVID-19 using chest radiograph images. Towards this, a fine-grained COVID-19 classification experiment is conducted to identify and classify chest radiograph images into normal, COVID-19 positive, and pneumonia. In addition, the performance of the proposed novel CovidDetNet deep learning model is evaluated on a standard COVID-19 Radiography Database. Moreover, we compared the performance of our approach with hybrid approaches in which we used deep learning models as feature extractors and support vector machines (SVM) as a classifier. Experimental results on the dataset showed the superiority of the proposed CovidDetNet model over the existing methods. The proposed CovidDetNet outperformed the baseline hybrid deep learning-based models by achieving a high accuracy of 98.40%.

Diagnostic effect of artificial intelligence solution for referable thoracic abnormalities on chest radiography: a multicenter respiratory outpatient diagnostic cohort study

Objectives

We aim ed to evaluate a commercial artificial intelligence (AI) solution on a multicenter cohort of chest radiographs and to compare physicians' ability to detect and localize referable thoracic abnormalities with and without AI assistance.

Methods

In this retrospective diagnostic cohort study, we investigated 6,006 consecutive patients who underwent both chest radiography and CT. We evaluated a commercially available AI solution intended to facilitate the detection of three chest abnormalities (nodule/masses, consolidation, and pneumothorax) against a reference standard to measure its diagnostic performance. Moreover, twelve physicians, including thoracic radiologists, board-certified radiologists, radiology residents, and pulmonologists, assessed a dataset of 230 randomly sampled chest radiographic images. The images were reviewed twice per physician, with and without AI, with a 4-week washout period. We measured the impact of AI assistance on observer's AUC, sensitivity, specificity, and the area under the alternative free-response ROC (AUAFROC).

Results

In the entire set (n = 6,006), the AI solution showed average sensitivity, specificity, and AUC of 0.885, 0.723, and 0.867, respectively. In the test dataset (n = 230), the average AUC and AUAFROC across observers significantly increased with AI assistance (from 0.861 to 0.886; p = 0.003 and from 0.797 to 0.822; p = 0.003, respectively).

Conclusions

The diagnostic performance of the AI solution was found to be acceptable for the images from respiratory outpatient clinics. The diagnostic performance of physicians marginally improved with the use of AI solutions. Further evaluation of AI assistance for chest radiographs using a prospective design is required to prove the efficacy of AI assistance.

Key Points

• AI assistance for chest radiographs marginally improved physicians’ performance in detecting and localizing referable thoracic abnormalities on chest radiographs.

• The detection or localization of referable thoracic abnormalities by pulmonologists and radiology residents improved with the use of AI assistance.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00330-021-08397-5.

Artificial intelligence-assisted decision making for prognosis and drug efficacy prediction in lung cancer patients: a narrative review

Objective

In this review, we aim to present frontier studies in patients with lung cancer as it related to artificial intelligence (AI)-assisted decision-making and summarize the latest advances, challenges and future trend in this field.

Background

Despite increasing survival rate in cancer patients over the last decades, lung cancer remains one of the leading causes of death worldwide. The early diagnosis, accurate evaluation and individualized treatment are vital approaches to improve the survival rate of patients with lung cancer. Thus, decision making based on these approaches requires accuracy and efficiency beyond manpower. Recent advances in AI and precision medicine have provided a fertile environment for the development of AI-based models. These models have the potential to assist radiologists and oncologists in detecting lung cancer, predicting prognosis and developing personalized treatment plans for better outcomes of the patients.

Methods

We searched literature from 2000 through July 31^th, 2021 in Medline/PubMed, the Web of Science, the Cochrane Library, ACM Digital Library, INSPEC and EMBASE. Key words such as “artificial intelligence”, “AI”, “deep learning”, “lung cancer”, “NSCLC”, “SCLC” were combined to identify related literatures. These literatures were then selected by two independent authors. Articles chosen by only one author will be examined by another author to determine whether this article was relative and valuable. The selected literatures were read by all authors and discussed to draw reliable conclusions.

Conclusions

AI, especially for those based on deep learning and radiomics, is capable of assisting clinical decision making from many aspects, for its quantitatively interpretation of patients’ information and its potential to deal with the dynamics, individual differences and heterogeneity of lung cancer. Hopefully, remaining problems such as insufficient data and poor interpretability may be solved to put AI-based models into clinical practice.

Deep learning-based algorithm for lung cancer detection on chest radiographs using the segmentation method

We developed and validated a deep learning (DL)-based model using the segmentation method and assessed its ability to detect lung cancer on chest radiographs. Chest radiographs for use as a training dataset and a test dataset were collected separately from January 2006 to June 2018 at our hospital. The training dataset was used to train and validate the DL-based model with five-fold cross-validation. The model sensitivity and mean false positive indications per image (mFPI) were assessed with the independent test dataset. The training dataset included 629 radiographs with 652 nodules/masses and the test dataset included 151 radiographs with 159 nodules/masses. The DL-based model had a sensitivity of 0.73 with 0.13 mFPI in the test dataset. Sensitivity was lower in lung cancers that overlapped with blind spots such as pulmonary apices, pulmonary hila, chest wall, heart, and sub-diaphragmatic space (0.50-0.64) compared with those in non-overlapped locations (0.87). The dice coefficient for the 159 malignant lesions was on average 0.52. The DL-based model was able to detect lung cancers on chest radiographs, with low mFPI.

Current and emerging artificial intelligence applications in chest imaging: a pediatric perspective

Artificial intelligence (AI) applications for chest radiography and chest CT are among the most developed applications in radiology. More than 40 certified AI products are available for chest radiography or chest CT. These AI products cover a wide range of abnormalities, including pneumonia, pneumothorax and lung cancer. Most applications are aimed at detecting disease, complemented by products that characterize or quantify tissue. At present, none of the thoracic AI products is specifically designed for the pediatric population. However, some products developed to detect tuberculosis in adults are also applicable to children. Software is under development to detect early changes of cystic fibrosis on chest CT, which could be an interesting application for pediatric radiology. In this review, we give an overview of current AI products in thoracic radiology and cover recent literature about AI in chest radiography, with a focus on pediatric radiology. We also discuss possible pediatric applications.

Utility of a Deep Learning Algorithm for Detection of Reticular Opacity on Chest Radiograph in Patients with Interstitial Lung Disease

Background: Deep learning has been heavily explored for pulmonary nodule detection on chest radiograph. Reticular opacity detection in interstitial lung disease (ILD) is challenging and may also benefit from a deep learning algorithm (DLA). Objective: To evaluate the utility of DLA for detection of reticular opacity on chest radiographs in patients with surgically confirmed ILD. Methods: This retrospective study included 197 patients (130 men, 67 women; mean age 62.6±7.6 years) with surgically proven ILD between January 2017 and December 2018 who underwent preoperative chest radiograph and chest CT within a 30-day interval. A total of 197 age- and sex-matched control patients with normal chest radiographs were randomly selected. Commercially available DLA was used to detect lower lobe or subpleural abnormalities; those matching reticular opacity location on CT were deemed true-positives. Six readers (three thoracic radiologists; three residents) independently reviewed radiographs for reticular opacity presence with and without DLA. Interobserver agreement was assessed. Diagnostic performance was compared among interpretations. Subanalysis was performed according to CT-based classification of reticular opacity severity. DLA performance was also assessed in 102 chest radiographs from a different institution (51 with ILD; 51 matched controls). Results: Interobserver agreement was moderate (κ=0.517) for readers alone versus almost perfect (κ=0.870) for readers with DLA. Sensitivity, specificity, and accuracy for reticular opacity for DLA were 98.0%, 99.0%, and 98.5%; for pooled readers alone were 77.3%, 92.3%, and 84.8%; and for readers with DLA were 93.8%, 97.3%, and 95.6%. All metrics were significantly better (all p≤.002) for DLA and for readers with DLA compared with readers alone. Sensitivity for readers without and with DLA were 66.7% and 86.8% in mild disease, 84.2% and 98.8% in moderate disease, and 87.3% and 100.0% in severe disease. DLA exhibited 100.0% accuracy in the cases from the second center. Conclusions: DLA outperformed readers in reticular opacity detection, and use of DLA improved reader performance and interobservser agreement. Benefit of DLA was more notable in terms of sensitivity than specificity and was maintained in mild disease. Clinical Impact: Use of DLA may facilitate detection of reticular opacity on chest radiograph in the early stages of ILD.

Interpretation of Thoracic Radiography Shows Large Discrepancies Depending on the Qualification of the Physician-Quantitative Evaluation of Interobserver Agreement in a Representative Emergency Department Scenario

(1) Background: Chest radiography (CXR) is still a key diagnostic component in the emergency department (ED). Correct interpretation is essential since some pathologies require urgent treatment. This study quantifies potential discrepancies in CXR analysis between radiologists and non-radiology physicians in training with ED experience. (2) Methods: Nine differently qualified physicians (three board-certified radiologists [BCR], three radiology residents [RR], and three non-radiology residents involved in ED [NRR]) evaluated a series of 563 posterior-anterior CXR images by quantifying suspicion for four relevant pathologies: pleural effusion, pneumothorax, pneumonia, and pulmonary nodules. Reading results were noted separately for each hemithorax on a Likert scale (0–4; 0: no suspicion of pathology, 4: safe existence of pathology) adding up to a total of 40,536 reported pathology suspicions. Interrater reliability/correlation and Kruskal–Wallis tests were performed for statistical analysis. (3) Results: While interrater reliability was good among radiologists, major discrepancies between radiologists’ and non-radiologists’ reading results could be observed in all pathologies. Highest overall interrater agreement was found for pneumothorax detection and lowest agreement in raising suspicion for malignancy suspicious nodules. Pleural effusion and pneumonia were often suspected with indifferent choices (1–3). In terms of pneumothorax detection, all readers mainly decided for a clear option (0 or 4). Interrater reliability was usually higher when evaluating the right hemithorax (all pathologies except pneumothorax). (4) Conclusions: Quantified CXR interrater reliability analysis displays a general uncertainty and strongly depends on medical training. NRR can benefit from radiology reporting in terms of time efficiency and diagnostic accuracy. CXR evaluation of long-time trained ED specialists has not been tested.

Prevalence of abnormal cardiovascular magnetic resonance findings in recovered patients from COVID-19: a systematic review and meta-analysis

BACKGROUND

The prevalence of abnormal cardiovascular magnetic resonance (CMR) findings in recovered coronavirus disease 2019 (COVID-19) patients is unclear. This study aimed to investigate the prevalence of abnormal CMR findings in recovered COVID-19 patients.

METHODS

A systematic literature search was performed to identify studies that report the prevalence of abnormal CMR findings in recovered COVID-19 patients. The number of patients with abnormal CMR findings and diagnosis of myocarditis on CMR (based on the Lake Louise criteria) and each abnormal CMR parameter were extracted. Subgroup analyses were performed according to patient characteristics (athletes vs. non-athletes and normal vs. undetermined cardiac enzyme levels). The pooled prevalence and 95% confidence interval (CI) of each CMR finding were calculated. Study heterogeneity was assessed, and meta-regression analysis was performed to investigate factors associated with heterogeneity.

RESULTS

In total, 890 patients from 16 studies were included in the analysis. The pooled prevalence of one or more abnormal CMR findings in recovered COVID-19 patients was 46.4% (95% CI 43.2%-49.7%). The pooled prevalence of myocarditis and late gadolinium enhancement (LGE) was 14.0% (95% CI 11.6%-16.8%) and 20.5% (95% CI 17.7%-23.6%), respectively. Further, heterogeneity was observed (I2 > 50%, p < 0.1). In the subgroup analysis, the pooled prevalence of abnormal CMR findings and myocarditis was higher in non-athletes than in athletes (62.5% vs. 17.1% and 23.9% vs. 2.5%, respectively). Similarly, the pooled prevalence of abnormal CMR findings and LGE was higher in the undetermined than in the normal cardiac enzyme level subgroup (59.4% vs. 35.9% and 45.5% vs. 8.3%, respectively). Being an athlete was a significant independent factor related to heterogeneity in multivariate meta-regression analysis (p < 0.05).

CONCLUSIONS

Nearly half of recovered COVID-19 patients exhibited one or more abnormal CMR findings. Athletes and patients with normal cardiac enzyme levels showed a lower prevalence of abnormal CMR findings than non-athletes and patients with undetermined cardiac enzyme levels. Trial registration The study protocol was registered in the PROSPERO database (registration number: CRD42020225234).

Deep Learning for Detection of Pulmonary Metastasis on Chest Radiographs

Background A computer-aided detection (CAD) system may help surveillance for pulmonary metastasis at chest radiography in situations where there is limited access to CT. Purpose To evaluate whether a deep learning (DL)-based CAD system can improve diagnostic yield for newly visible lung metastasis on chest radiographs in patients with cancer. Materials and Methods A regulatory-approved CAD system for lung nodules was implemented to interpret chest radiographs from patients referred by the medical oncology department in clinical practice. In this retrospective diagnostic cohort study, chest radiographs interpreted with assistance from a CAD system after the implementation (January to April 2019, CAD-assisted interpretation group) and those interpreted before the implementation (September to December 2018, conventional interpretation group) of the CAD system were consecutively included. The diagnostic yield (frequency of true-positive detections) and false-referral rate (frequency of false-positive detections) of formal reports of chest radiographs for newly visible lung metastasis were compared between the two groups using generalized estimating equations. Propensity score matching was performed between the two groups for age, sex, and primary cancer. Results A total of 2916 chest radiographs from 1521 patients (1546 men, 1370 women; mean age, 62 years) and 5681 chest radiographs from 3456 patients (2941 men, 2740 women; mean age, 62 years) were analyzed in the CAD-assisted interpretation and conventional interpretation groups, respectively. The diagnostic yield for newly visible metastasis was higher in the CAD-assisted interpretation group (0.86%, 25 of 2916 [95% CI: 0.58, 1.3] vs 0.32%, 18 of 568 [95% CI: 0.20, 0.50%]; P = .004). The false-referral rate in the CAD-assisted interpretation group (0.34%, 10 of 2916 [95% CI: 0.19, 0.64]) was not inferior to that in the conventional interpretation group (0.25%, 14 of 5681 [95% CI: 0.15, 0.42]) at the noninferiority margin of 0.5% (95% CI of difference: -0.15, 0.35). Conclusion A deep learning-based computer-aided detection system improved the diagnostic yield for newly visible metastasis on chest radiographs in patients with cancer with a similar false-referral rate. © RSNA, 2021 Online supplemental material is available for this article.

LI-RADS treatment response algorithm after first-line DEB-TACE: reproducibility and prognostic value at initial post-treatment CT/MRI

PURPOSE

To evaluate the inter-reader reproducibility and prognostic accuracy of the Liver Imaging Reporting and Data System (LI-RADS) treatment response algorithm (LR-TR) at the time of initial post-treatment evaluation following drug-eluting beads transarterial chemoembolization (DEB-TACE) for hepatocellular carcinoma (HCC).

METHODS

This retrospective study included patients with HCC who underwent first-line DEB-TACE between January 2011 and December 2015. Six readers (three fellowship-trained radiologists and three radiology trainees) independently assessed lesion-level response in up to two treated lesions per LR-TR and modified Response Evaluation Criteria in Solid Tumors (mRECIST)-target criteria, as well as patient-level response per mRECIST-overall criteria, on the initial post-treatment CT/MRI. Inter-reader agreement was calculated by Fleiss' multi-reader κ. We tested whether LR-TR, mRECIST-target, and mRECIST-overall response were associated with overall survival using Kaplan-Meier and Cox proportional hazard model analyses.

RESULTS

A total of 82 patients with 113 treated target lesions were included. Inter-reader agreement was moderate for LR-TR and mRECIST-overall (κ range 0.42-0.57), and substantial for mRECIST-target (κ range 0.62-0.66), among all three reader-groups: all readers, experienced readers, and less-experienced readers. LR-TR and mRECIST-target response were not significantly associated with overall survival regardless of reader experience (P > 0.05). In contrast, mRECIST-overall response was significantly associated with overall survival when assessed by all readers (P = 0.02) and experienced readers (P = 0.03), but not by the less-experienced readers (P = 0.35).

CONCLUSION

Although LR-TR algorithm has moderate inter-reader reproducibility, it alone may not predict overall survival on the initial post-treatment CT/MRI after first-line DEB-TACE for HCC.

Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study

BACKGROUND

Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model.

METHODS

In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior.

FINDINGS

Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings.

INTERPRETATION

This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice.

FUNDING

Annalise.ai.

CT Image Analysis and Clinical Diagnosis of New Coronary Pneumonia Based on Improved Convolutional Neural Network

In this paper, based on the improved convolutional neural network, in-depth analysis of the CT image of the new coronary pneumonia, using the U-Net series of deep neural networks to semantically segment the CT image of the new coronary pneumonia, to obtain the new coronary pneumonia area as the foreground and the remaining areas as the background of the binary image, provides a basis for subsequent image diagnosis. Secondly, the target-detection framework Faster RCNN extracts features from the CT image of the new coronary pneumonia tumor, obtains a higher-level abstract representation of the data, determines the lesion location of the new coronary pneumonia tumor, and gives its bounding box in the image. By generating an adversarial network to diagnose the lesion area of the CT image of the new coronary pneumonia tumor, obtaining a complete image of the new coronary pneumonia, achieving the effect of the CT image diagnosis of the new coronary pneumonia tumor, and three-dimensionally reconstructing the complete new coronary pneumonia model, filling the current the gap in this aspect, provide a basis to produce new coronary pneumonia prosthesis and improve the accuracy of diagnosis.

Automated Radiology Alert System for Pneumothorax Detection on Chest Radiographs Improves Efficiency and Diagnostic Performance

We aimed to set up an Automated Radiology Alert System (ARAS) for the detection of pneumothorax in chest radiographs by a deep learning model, and to compare its efficiency and diagnostic performance with the existing Manual Radiology Alert System (MRAS) at the tertiary medical center. This study retrospectively collected 1235 chest radiographs with pneumothorax labeling from 2013 to 2019, and 337 chest radiographs with negative findings in 2019 were separated into training and validation datasets for the deep learning model of ARAS. The efficiency before and after using the model was compared in terms of alert time and report time. During parallel running of the two systems from September to October 2020, chest radiographs prospectively acquired in the emergency department with age more than 6 years served as the testing dataset for comparison of diagnostic performance. The efficiency was improved after using the model, with mean alert time improving from 8.45 min to 0.69 min and the mean report time from 2.81 days to 1.59 days. The comparison of the diagnostic performance of both systems using 3739 chest radiographs acquired during parallel running showed that the ARAS was better than the MRAS as assessed in terms of sensitivity (recall), area under receiver operating characteristic curve, and F1 score (0.837 vs. 0.256, 0.914 vs. 0.628, and 0.754 vs. 0.407, respectively), but worse in terms of positive predictive value (PPV) (precision) (0.686 vs. 1.000). This study had successfully designed a deep learning model for pneumothorax detection on chest radiographs and set up an ARAS with improved efficiency and overall diagnostic performance.

COVID-19 pneumonia on chest X-rays: Performance of a deep learning-based computer-aided detection system

Chest X-rays (CXRs) can help triage for Coronavirus disease (COVID-19) patients in resource-constrained environments, and a computer-aided detection system (CAD) that can identify pneumonia on CXR may help the triage of patients in those environment where expert radiologists are not available. However, the performance of existing CAD for identifying COVID-19 and associated pneumonia on CXRs has been scarcely investigated. In this study, CXRs of patients with and without COVID-19 confirmed by reverse transcriptase polymerase chain reaction (RT-PCR) were retrospectively collected from four and one institution, respectively, and a commercialized, regulatory-approved CAD that can identify various abnormalities including pneumonia was used to analyze each CXR. Performance of the CAD was evaluated using area under the receiver operating characteristic curves (AUCs), with reference standards of the RT-PCR results and the presence of findings of pneumonia on chest CTs obtained within 24 hours from the CXR. For comparison, 5 thoracic radiologists and 5 non-radiologist physicians independently interpreted the CXRs. Afterward, they re-interpreted the CXRs with corresponding CAD results. The performance of CAD (AUCs, 0.714 and 0.790 against RT-PCR and chest CT, respectively hereinafter) were similar with those of thoracic radiologists (AUCs, 0.701 and 0.784), and higher than those of non-radiologist physicians (AUCs, 0.584 and 0.650). Non-radiologist physicians showed significantly improved performance when assisted with the CAD (AUCs, 0.584 to 0.664 and 0.650 to 0.738). In addition, inter-reader agreement among physicians was also improved in the CAD-assisted interpretation (Fleiss' kappa coefficient, 0.209 to 0.322). In conclusion, radiologist-level performance of the CAD in identifying COVID-19 and associated pneumonia on CXR and enhanced performance of non-radiologist physicians with the CAD assistance suggest that the CAD can support physicians in interpreting CXRs and helping image-based triage of COVID-19 patients in resource-constrained environment.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.

Development and validation of a deep learning algorithm detecting 10 common abnormalities on chest radiographs

We aimed to develop a deep learning algorithm detecting 10 common abnormalities (DLAD-10) on chest radiographs, and to evaluate its impact in diagnostic accuracy, timeliness of reporting and workflow efficacy.

DLAD-10 was trained with 146 717 radiographs from 108 053 patients using a ResNet34-based neural network with lesion-specific channels for 10 common radiological abnormalities (pneumothorax, mediastinal widening, pneumoperitoneum, nodule/mass, consolidation, pleural effusion, linear atelectasis, fibrosis, calcification and cardiomegaly). For external validation, the performance of DLAD-10 on a same-day computed tomography (CT)-confirmed dataset (normal:abnormal 53:147) and an open-source dataset (PadChest; normal:abnormal 339:334) was compared with that of three radiologists. Separate simulated reading tests were conducted on another dataset adjusted to real-world disease prevalence in the emergency department, consisting of four critical, 52 urgent and 146 nonurgent cases. Six radiologists participated in the simulated reading sessions with and without DLAD-10.

DLAD-10 exhibited area under the receiver operating characteristic curve values of 0.895–1.00 in the CT-confirmed dataset and 0.913–0.997 in the PadChest dataset. DLAD-10 correctly classified significantly more critical abnormalities (95.0% (57/60)) than pooled radiologists (84.4% (152/180); p=0.01). In simulated reading tests for emergency department patients, pooled readers detected significantly more critical (70.8% (17/24) versus 29.2% (7/24); p=0.006) and urgent (82.7% (258/312) versus 78.2% (244/312); p=0.04) abnormalities when aided by DLAD-10. DLAD-10 assistance shortened the mean±sd time-to-report critical and urgent radiographs (640.5±466.3 versus 3371.0±1352.5 s and 1840.3±1141.1 versus 2127.1±1468.2 s, respectively; all p<0.01) and reduced the mean±sd interpretation time (20.5±22.8 versus 23.5±23.7 s; p<0.001).

DLAD-10 showed excellent performance, improving radiologists' performance and shortening the reporting time for critical and urgent cases.

A deep learning algorithm detecting 10 common abnormalities was trained with 146 717 images and showed excellent performance on chest radiographs, helping radiologists improve their performance and advance the reporting time for critical or urgent caseshttps://bit.ly/3k8tZ5P

Performance of an AI based CAD system in solid lung nodule detection on chest phantom radiographs compared to radiology residents and fellow radiologists

Background

Despite the decreasing relevance of chest radiography in lung cancer screening, chest radiography is still frequently applied to assess for lung nodules. The aim of the current study was to determine the accuracy of a commercial AI based CAD system for the detection of artificial lung nodules on chest radiograph phantoms and compare the performance to radiologists in training.

Methods

Sixty-one anthropomorphic lung phantoms were equipped with 140 randomly deployed artificial lung nodules (5, 8, 10, 12 mm). A random generator chose nodule size and distribution before a two-plane chest X-ray (CXR) of each phantom was performed. Seven blinded radiologists in training (2 fellows, 5 residents) with 2 to 5 years of experience in chest imaging read the CXRs on a PACS-workstation independently. Results of the software were recorded separately. McNemar test was used to compare each radiologist’s results to the AI-computer-aided-diagnostic (CAD) software in a per-nodule and a per-phantom approach and Fleiss-Kappa was applied for inter-rater and intra-observer agreements.

Results

Five out of seven readers showed a significantly higher accuracy than the AI algorithm. The pooled accuracies of the radiologists in a nodule-based and a phantom-based approach were 0.59 and 0.82 respectively, whereas the AI-CAD showed accuracies of 0.47 and 0.67, respectively. Radiologists’ average sensitivity for 10 and 12 mm nodules was 0.80 and dropped to 0.66 for 8 mm (P=0.04) and 0.14 for 5 mm nodules (P<0.001). The radiologists and the algorithm both demonstrated a significant higher sensitivity for peripheral compared to central nodules (0.66 vs. 0.48; P=0.004 and 0.64 vs. 0.094; P=0.025, respectively). Inter-rater agreements were moderate among the radiologists and between radiologists and AI-CAD software (K’=0.58±0.13 and 0.51±0.1). Intra-observer agreement was calculated for two readers and was almost perfect for the phantom-based (K’=0.85±0.05; K’=0.80±0.02); and substantial to almost perfect for the nodule-based approach (K’=0.83±0.02; K’=0.78±0.02).

Conclusions

The AI based CAD system as a primary reader acts inferior to radiologists regarding lung nodule detection in chest phantoms. Chest radiography has reasonable accuracy in lung nodule detection if read by a radiologist alone and may be further optimized by an AI based CAD system as a second reader.

Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis

Deep learning (DL) has the potential to transform medical diagnostics. However, the diagnostic accuracy of DL is uncertain. Our aim was to evaluate the diagnostic accuracy of DL algorithms to identify pathology in medical imaging. Searches were conducted in Medline and EMBASE up to January 2020. We identified 11,921 studies, of which 503 were included in the systematic review. Eighty-two studies in ophthalmology, 82 in breast disease and 115 in respiratory disease were included for meta-analysis. Two hundred twenty-four studies in other specialities were included for qualitative review. Peer-reviewed studies that reported on the diagnostic accuracy of DL algorithms to identify pathology using medical imaging were included. Primary outcomes were measures of diagnostic accuracy, study design and reporting standards in the literature. Estimates were pooled using random-effects meta-analysis. In ophthalmology, AUC's ranged between 0.933 and 1 for diagnosing diabetic retinopathy, age-related macular degeneration and glaucoma on retinal fundus photographs and optical coherence tomography. In respiratory imaging, AUC's ranged between 0.864 and 0.937 for diagnosing lung nodules or lung cancer on chest X-ray or CT scan. For breast imaging, AUC's ranged between 0.868 and 0.909 for diagnosing breast cancer on mammogram, ultrasound, MRI and digital breast tomosynthesis. Heterogeneity was high between studies and extensive variation in methodology, terminology and outcome measures was noted. This can lead to an overestimation of the diagnostic accuracy of DL algorithms on medical imaging. There is an immediate need for the development of artificial intelligence-specific EQUATOR guidelines, particularly STARD, in order to provide guidance around key issues in this field.

Added Value of Deep Learning-based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study

Background Previous studies assessing the effects of computer-aided detection on observer performance in the reading of chest radiographs used a sequential reading design that may have biased the results because of reading order or recall bias. Purpose To compare observer performance in detecting and localizing major abnormal findings including nodules, consolidation, interstitial opacity, pleural effusion, and pneumothorax on chest radiographs without versus with deep learning-based detection (DLD) system assistance in a randomized crossover design. Materials and Methods This study included retrospectively collected normal and abnormal chest radiographs between January 2016 and December 2017 (https://cris.nih.go.kr/; registration no. KCT0004147). The radiographs were randomized into two groups, and six observers, including thoracic radiologists, interpreted each radiograph without and with use of a commercially available DLD system by using a crossover design with a washout period. Jackknife alternative free-response receiver operating characteristic (JAFROC) figure of merit (FOM), area under the receiver operating characteristic curve (AUC), sensitivity, specificity, false-positive findings per image, and reading times of observers with and without the DLD system were compared by using McNemar and paired t tests. Results A total of 114 normal (mean patient age ± standard deviation, 51 years ± 11; 58 men) and 114 abnormal (mean patient age, 60 years ± 15; 75 men) chest radiographs were evaluated. The radiographs were randomized to two groups: group A (n = 114) and group B (n = 114). Use of the DLD system improved the observers' JAFROC FOM (from 0.90 to 0.95, P = .002), AUC (from 0.93 to 0.98, P = .002), per-lesion sensitivity (from 83% [822 of 990 lesions] to 89.1% [882 of 990 lesions], P = .009), per-image sensitivity (from 80% [548 of 684 radiographs] to 89% [608 of 684 radiographs], P = .009), and specificity (from 89.3% [611 of 684 radiographs] to 96.6% [661 of 684 radiographs], P = .01) and reduced the reading time (from 10-65 seconds to 6-27 seconds, P < .001). The DLD system alone outperformed the pooled observers (JAFROC FOM: 0.96 vs 0.90, respectively, P = .007; AUC: 0.98 vs 0.93, P = .003). Conclusion Observers including thoracic radiologists showed improved performance in the detection and localization of major abnormal findings on chest radiographs and reduced reading time with use of a deep learning-based detection system. © RSNA, 2021 Online supplemental material is available for this article.