Thorax-Net: An Attention Regularized Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography

Development and evaluation of a deep learning-based model for simultaneous detection and localization of rib and clavicle fractures in trauma patients' chest radiographs

Purpose

To develop a rib and clavicle fracture detection model for chest radiographs in trauma patients using a deep learning (DL) algorithm.

Materials and methods

We retrospectively collected 56 145 chest X-rays (CXRs) from trauma patients in a trauma center between August 2008 and December 2016. A rib/clavicle fracture detection DL algorithm was trained using this data set with 991 (1.8%) images labeled by experts with fracture site locations. The algorithm was tested on independently collected 300 CXRs in 2017. An external test set was also collected from hospitalized trauma patients in a regional hospital for evaluation. The receiver operating characteristic curve with area under the curve (AUC), accuracy, sensitivity, specificity, precision, and negative predictive value of the model on each test set was evaluated. The prediction probability on the images was visualized as heatmaps.

Results

The trained DL model achieved an AUC of 0.912 (95% CI 87.8 to 94.7) on the independent test set. The accuracy, sensitivity, and specificity on the given cut-off value are 83.7, 86.8, and 80.4, respectively. On the external test set, the model had a sensitivity of 88.0 and an accuracy of 72.5. While the model exhibited a slight decrease in accuracy on the external test set, it maintained its sensitivity in detecting fractures.

Conclusion

The algorithm detects rib and clavicle fractures concomitantly in the CXR of trauma patients with high accuracy in locating lesions through heatmap visualization.

A deep-learning-based framework for identifying and localizing multiple abnormalities and assessing cardiomegaly in chest X-ray

Accurate identification and localization of multiple abnormalities are crucial steps in the interpretation of chest X-rays (CXRs); however, the lack of a large CXR dataset with bounding boxes severely constrains accurate localization research based on deep learning. We created a large CXR dataset named CXR-AL14, containing 165,988 CXRs and 253,844 bounding boxes. On the basis of this dataset, a deep-learning-based framework was developed to identify and localize 14 common abnormalities and calculate the cardiothoracic ratio (CTR) simultaneously. The mean average precision values obtained by the model for 14 abnormalities reached 0.572-0.631 with an intersection-over-union threshold of 0.5, and the intraclass correlation coefficient of the CTR algorithm exceeded 0.95 on the held-out, multicentre and prospective test datasets. This framework shows an excellent performance, good generalization ability and strong clinical applicability, which is superior to senior radiologists and suitable for routine clinical settings.

Development and validation of open-source deep neural networks for comprehensive chest x-ray reading: a retrospective, multicentre study

BACKGROUND

Artificial intelligence (AI) systems for automated chest x-ray interpretation hold promise for standardising reporting and reducing delays in health systems with shortages of trained radiologists. Yet, there are few freely accessible AI systems trained on large datasets for practitioners to use with their own data with a view to accelerating clinical deployment of AI systems in radiology. We aimed to contribute an AI system for comprehensive chest x-ray abnormality detection.

METHODS

In this retrospective cohort study, we developed open-source neural networks, X-Raydar and X-Raydar-NLP, for classifying common chest x-ray findings from images and their free-text reports. Our networks were developed using data from six UK hospitals from three National Health Service (NHS) Trusts (University Hospitals Coventry and Warwickshire NHS Trust, University Hospitals Birmingham NHS Foundation Trust, and University Hospitals Leicester NHS Trust) collectively contributing 2 513 546 chest x-ray studies taken from a 13-year period (2006-19), which yielded 1 940 508 usable free-text radiological reports written by the contemporary assessing radiologist (collectively referred to as the "historic reporters") and 1 896 034 frontal images. Chest x-rays were labelled using a taxonomy of 37 findings by a custom-trained natural language processing (NLP) algorithm, X-Raydar-NLP, from the original free-text reports. X-Raydar-NLP was trained on 23 230 manually annotated reports and tested on 4551 reports from all hospitals. 1 694 921 labelled images from the training set and 89 238 from the validation set were then used to train a multi-label image classifier. Our algorithms were evaluated on three retrospective datasets: a set of exams sampled randomly from the full NHS dataset reported during clinical practice and annotated using NLP (n=103 328); a consensus set sampled from all six hospitals annotated by three expert radiologists (two independent annotators for each image and a third consultant to facilitate disagreement resolution) under research conditions (n=1427); and an independent dataset, MIMIC-CXR, consisting of NLP-annotated exams (n=252 374).

FINDINGS

X-Raydar achieved a mean AUC of 0·919 (SD 0·039) on the auto-labelled set, 0·864 (0·102) on the consensus set, and 0·842 (0·074) on the MIMIC-CXR test, demonstrating similar performance to the historic clinical radiologist reporters, as assessed on the consensus set, for multiple clinically important findings, including pneumothorax, parenchymal opacification, and parenchymal mass or nodules. On the consensus set, X-Raydar outperformed historical reporter balanced accuracy with significance on 27 of 37 findings, was non-inferior on nine, and inferior on one finding, resulting in an average improvement of 13·3% (SD 13·1) to 0·763 (0·110), including a mean 5·6% (13·2) improvement in critical findings to 0·826 (0·119).

INTERPRETATION

Our study shows that automated classification of chest x-rays under a comprehensive taxonomy can achieve performance levels similar to those of historical reporters and exhibit robust generalisation to external data. The open-sourced neural networks can serve as foundation models for further research and are freely available to the research community.

FUNDING

Wellcome Trust.

Efficient Thorax Disease Classification and Localization Using DCNN and Chest X-ray Images

Thorax disease is a life-threatening disease caused by bacterial infections that occur in the lungs. It could be deadly if not treated at the right time, so early diagnosis of thoracic diseases is vital. The suggested study can assist radiologists in more swiftly diagnosing thorax disorders and in the rapid airport screening of patients with a thorax disease, such as pneumonia. This paper focuses on automatically detecting and localizing thorax disease using chest X-ray images. It provides accurate detection and localization using DenseNet-121 which is foundation of our proposed framework, called Z-Net. The proposed framework utilizes the weighted cross-entropy loss function (W-CEL) that manages class imbalance issue in the ChestX-ray14 dataset, which helped in achieving the highest performance as compared to the previous models. The 112,120 images contained in the ChestX-ray14 dataset (60,412 images are normal, and the rest contain thorax diseases) were preprocessed and then trained for classification and localization. This work uses computer-aided diagnosis (CAD) system that supports development of highly accurate and precise computer-aided systems. We aim to develop a CAD system using a deep learning approach. Our quantitative results show high AUC scores in comparison with the latest research works. The proposed approach achieved the highest mean AUC score of 85.8%. This is the highest accuracy documented in the literature for any related model.

Automatic Diagnosis of Significant Liver Fibrosis From Ultrasound B-Mode Images Using a Handcrafted-Feature-Assisted Deep Convolutional Neural Network

The accurate diagnosis of significant liver fibrosis ( ≥ F2) in patients with chronic liver disease (CLD) is critical, as ≥ F2 is a crucial factor that should be considered in selecting an antiviral therapy for these patients. This article proposes a handcrafted-feature-assisted deep convolutional neural network (HFA-DCNN) that helps radiologists automatically and accurately diagnose significant liver fibrosis from ultrasound (US) brightness (B)-mode images. The HFA-DCNN model has three main branches: one for automatic region of interest (ROI) segmentation in the US images, another for attention deep feature learning from the segmented ROI, and the third for handcrafted feature extraction. The attention deep learning features and handcrafted features are fused in the back end of the model to enable more accurate diagnosis of significant liver fibrosis. The usefulness and effectiveness of the proposed model were validated on a dataset built upon 321 CLD patients with liver fibrosis stages confirmed by pathological evaluations. In a fivefold cross validation (FFCV), the proposed model achieves accuracy, sensitivity, specificity, and area under the receiver-operating-characteristic (ROC) curve (AUC) values of 0.863 (95% confidence interval (CI) 0.820-0.899), 0.879 (95% CI 0.823-0.920), 0.872 (95% CI 0.800-0.925), and 0.925 (95% CI 0.891-0.952), which are significantly better than those obtained by the comparative methods. Given its excellent performance, the proposed HFA-DCNN model can serve as a promising tool for the noninvasive and accurate diagnosis of significant liver fibrosis in CLD patients.

Observer Performance Evaluation of a Deep Learning Model for Multilabel Classification of Active Tuberculosis Lung Zone-Wise Manifestations

Background Chest X-rays (CXRs) are widely used for cost-effective screening of active pulmonary tuberculosis despite their limitations in sensitivity and specificity when interpreted by clinicians or radiologists. To address this issue, computer-aided detection (CAD) algorithms, particularly deep learning architectures based on convolution, have been developed to automate the analysis of radiography imaging. Deep learning algorithms have shown promise in accurately classifying lung abnormalities using chest X-ray images. In this study, we utilized the EfficientNet B4 model, which was pre-trained on ImageNet with 380x380 input dimensions, using its weights for transfer learning, and was modified with a series of components including global average pooling, batch normalization, dropout, and a classifier with 12 image-wise and 44 segment-wise lung zone evaluation classes using sigmoid activation. Objectives Assess the clinical usefulness of our previously created EfficientNet B4 model in identifying lung zone-specific abnormalities related to active tuberculosis through an observer performance test involving a skilled clinician operating in tuberculosis-specific environments. Methods The ground truth was established by a radiologist who examined all sample CXRs to identify lung zone-wise abnormalities. An expert clinician working in tuberculosis-specific settings independently reviewed the same CXR with blinded access to the ground truth. Simultaneously, the CXRs were classified using the EfficientNet B4 model. The clinician's assessments were then compared with the model's predictions, and the agreement between the two was measured using the kappa coefficient, evaluating the model's performance in classifying active tuberculosis manifestations across lung zones. Results The results show a strong agreement (Kappa ≥0.81) seen for lung zone-wise abnormalities of pneumothorax, mediastinal shift, emphysema, fibrosis, calcifications, pleural effusion, and cavity. Substantial agreement (Kappa = 0.61-0.80) for cavity, mediastinal shift, volume loss, and collapsed lungs. The Kappa score for lung zone-wise abnormalities is moderate (0.41-0.60) for 39% of cases. In image-wise agreement, the EfficientNet B4 model's performance ranges from moderate to almost perfect across categories, while in lung zone-wise agreement, it varies from fair to almost perfect. The results show strong agreement between the EfficientNet B4 model and the human reader in detecting lung zone-wise and image-wise manifestations. Conclusion The clinical utility of the EfficientNet B4 models to detect the abnormalities can aid clinicians in primary care settings for screening and triaging tuberculosis where resources are constrained or overburdened.

Multi-Label Local to Global Learning: A Novel Learning Paradigm for Chest X-Ray Abnormality Classification

Deep neural network (DNN) approaches have shown remarkable progress in automatic Chest X-rays classification. However, existing methods use a training scheme that simultaneously trains all abnormalities without considering their learning priority. Inspired by the clinical practice of radiologists progressively recognizing more abnormalities and the observation that existing curriculum learning (CL) methods based on image difficulty may not be suitable for disease diagnosis, we propose a novel CL paradigm, named multi-label local to global (ML-LGL). This approach iteratively trains DNN models on gradually increasing abnormalities within the dataset, i,e, from fewer abnormalities (local) to more ones (global). At each iteration, we first build the local category by adding high-priority abnormalities for training, and the abnormality's priority is determined by our three proposed clinical knowledge-leveraged selection functions. Then, images containing abnormalities in the local category are gathered to form a new training set. The model is lastly trained on this set using a dynamic loss. Additionally, we demonstrate the superiority of ML-LGL from the perspective of the model's initial stability during training. Experimental results on three open-source datasets, PLCO, ChestX-ray14 and CheXpert show that our proposed learning paradigm outperforms baselines and achieves comparable results to state-of-the-art methods. The improved performance promises potential applications in multi-label Chest X-ray classification.

Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis

BACKGROUND

Deep learning (DL), a subset of artificial intelligence (AI), has been applied to pneumothorax diagnosis to aid physician diagnosis, but no meta-analysis has been performed.

METHODS

A search of multiple electronic databases through September 2022 was performed to identify studies that applied DL for pneumothorax diagnosis using imaging. Meta-analysis via a hierarchical model to calculate the summary area under the curve (AUC) and pooled sensitivity and specificity for both DL and physicians was performed. Risk of bias was assessed using a modified Prediction Model Study Risk of Bias Assessment Tool.

RESULTS

In 56 of the 63 primary studies, pneumothorax was identified from chest radiography. The total AUC was 0.97 (95% CI 0.96-0.98) for both DL and physicians. The total pooled sensitivity was 84% (95% CI 79-89%) for DL and 85% (95% CI 73-92%) for physicians and the pooled specificity was 96% (95% CI 94-98%) for DL and 98% (95% CI 95-99%) for physicians. More than half of the original studies (57%) had a high risk of bias.

CONCLUSIONS

Our review found the diagnostic performance of DL models was similar to that of physicians, although the majority of studies had a high risk of bias. Further pneumothorax AI research is needed.

High-Resolution Network with Dynamic Convolution and Coordinate Attention for Classification of Chest X-ray Images

The development of automatic chest X-ray (CXR) disease classification algorithms is significant for diagnosing thoracic diseases. Owing to the characteristics of lesions in CXR images, including high similarity in appearance of the disease, varied sizes, and different occurrence locations, most existing convolutional neural network-based methods have insufficient feature extraction for thoracic lesions and struggle to adapt to changes in lesion size and location. To address these issues, this study proposes a high-resolution classification network with dynamic convolution and coordinate attention (HRCC-Net). In the method, this study suggests a parallel multi-resolution network in which a high-resolution branch acquires essential detailed features of the lesion and multi-resolution feature swapping and fusion to obtain multiple receptive fields to extract complicated disease features adequately. Furthermore, this study proposes dynamic convolution to enhance the network's ability to represent multi-scale information to accommodate lesions of diverse scales. In addition, this study introduces a coordinate attention mechanism, which enables automatic focus on pathologically relevant regions and capturing the variations in lesion location. The proposed method is evaluated on ChestX-ray14 and CheXpert datasets. The average AUC (area under ROC curve) values reach 0.845 and 0.913, respectively, indicating this method's advantages compared with the currently available methods. Meanwhile, with its specificity and sensitivity to measure the performance of medical diagnostic systems, the network can improve diagnostic efficiency while reducing the rate of misdiagnosis. The proposed algorithm has great potential for thoracic disease diagnosis and treatment.

Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

A 178-clinical-center experiment of integrating AI solutions for lung pathology diagnosis

In 2020, an experiment testing AI solutions for lung X-ray analysis on a multi-hospital network was conducted. The multi-hospital network linked 178 Moscow state healthcare centers, where all chest X-rays from the network were redirected to a research facility, analyzed with AI, and returned to the centers. The experiment was formulated as a public competition with monetary awards for participating industrial and research teams. The task was to perform the binary detection of abnormalities from chest X-rays. For the objective real-life evaluation, no training X-rays were provided to the participants. This paper presents one of the top-performing AI frameworks from this experiment. First, the framework used two EfficientNets, histograms of gradients, Haar feature ensembles, and local binary patterns to recognize whether an input image represents an acceptable lung X-ray sample, meaning the X-ray is not grayscale inverted, is a frontal chest X-ray, and completely captures both lung fields. Second, the framework extracted the region with lung fields and then passed them to a multi-head DenseNet, where the heads recognized the patient's gender, age and the potential presence of abnormalities, and generated the heatmap with the abnormality regions highlighted. During one month of the experiment from 11.23.2020 to 12.25.2020, 17,888 cases have been analyzed by the framework with 11,902 cases having radiological reports with the reference diagnoses that were unequivocally parsed by the experiment organizers. The performance measured in terms of the area under receiving operator curve (AUC) was 0.77. The AUC for individual diseases ranged from 0.55 for herniation to 0.90 for pneumothorax.

Deep learning classification of active tuberculosis lung zones wise manifestations using chest X-rays: a multi label approach

Chest X-rays are the most economically viable diagnostic imaging test for active pulmonary tuberculosis screening despite the high sensitivity and low specificity when interpreted by clinicians or radiologists. Computer aided detection (CAD) algorithms, especially convolution based deep learning architecture, have been proposed to facilitate the automation of radiography imaging modalities. Deep learning algorithms have found success in classifying various abnormalities in lung using chest X-ray. We fine-tuned, validated and tested EfficientNetB4 architecture and utilized the transfer learning methodology for multilabel approach to detect lung zone wise and image wise manifestations of active pulmonary tuberculosis using chest X-ray. We used Area Under Receiver Operating Characteristic (AUC), sensitivity and specificity along with 95% confidence interval as model evaluation metrics. We also utilized the visualisation capabilities of convolutional neural networks (CNN), Gradient-weighted Class Activation Mapping (Grad-CAM) as post-hoc attention method to investigate the model and visualisation of Tuberculosis abnormalities and discuss them from radiological perspectives. EfficientNetB4 trained network achieved remarkable AUC, sensitivity and specificity of various pulmonary tuberculosis manifestations in intramural test set and external test set from different geographical region. The grad-CAM visualisations and their ability to localize the abnormalities can aid the clinicians at primary care settings for screening and triaging of tuberculosis where resources are constrained or overburdened.

PCAN: Pixel-wise classification and attention network for thoracic disease classification and weakly supervised localization

Automatic chest X-ray (CXR) disease classification has drawn increasing public attention as CXR is widely used in thoracic disease diagnosis. Existing classification networks typically employ a global average pooling layer to produce the final feature for the subsequent classifier. This limits the classification performance owing to the characteristics of lesions in CXR images, including small relative sizes, varied absolute sizes, and different occurrence locations. In this study, we propose a pixel-wise classification and attention network (PCAN) to simultaneously perform disease classification and weakly supervised localization, which provides interpretability for disease classification. The PCAN comprises a backbone network for extracting mid-level features, a pixel-wise classification branch (pc-branch) for generating pixel-wise diagnoses, and a pixel-wise attention branch (pa-branch) for producing pixel-wise weights. The pc-branch is capable of explicitly detecting small lesions, and the pa-branch is capable of adaptively focusing on different regions when classifying different thoracic diseases. Then, the pixel-wise diagnoses are multiplied with the pixel-wise weights to obtain the disease localization map, which provides the sizes and locations of lesions in a manner of weakly supervised learning. The final image-wise diagnosis is obtained by summing up the disease localization map at the spatial dimension. Comprehensive experiments conducted on the ChestX-ray14 and CheXpert datasets demonstrate the effectiveness of the proposed PCAN, which has great potential for thoracic disease diagnosis and treatment. The source codes are available at https://github.com/fzfs/PCAN.

Using Occlusion-Based Saliency Maps to Explain an Artificial Intelligence Tool in Lung Cancer Screening: Agreement Between Radiologists, Labels, and Visual Prompts

Occlusion-based saliency maps (OBSMs) are one of the approaches for interpreting decision-making process of an artificial intelligence (AI) system. This study explores the agreement among text responses from a cohort of radiologists to describe diagnostically relevant areas on low-dose CT (LDCT) images. It also explores if radiologists' descriptions of cases misclassified by the AI provide a rationale for ruling out the AI's output. The OBSM indicating the importance of different pixels on the final decision made by an AI were generated for 10 benign cases (3 misclassified by the AI tool as malignant) and 10 malignant cases (2 misclassified by the AI tool as benign). Thirty-six radiologists were asked to use radiological vocabulary, typical to reporting LDCT scans, to describe the mapped regions of interest (ROI). The radiologists' annotations were then grouped by using a clustering-based technique. Topics were extracted from the annotations and for each ROI, a percentage of annotations containing each topic were found. Radiologists annotated 17 and 24 unique ROIs on benign and malignant cases, respectively. Agreement on the main label (e.g., "vessel," "nodule") by radiologists was only seen in only in 12% of all areas (5/41 ROI). Topic analyses identified six descriptors which are commonly associated with a lower malignancy likelihood. Eight common topics related to a higher malignancy likelihood were also determined. Occlusion-based saliency maps were used to explain an AI decision-making process to radiologists, who in turn have provided insight into the level of agreement between the AI's decision and radiological lexicon.

AI-CenterNet CXR: An artificial intelligence (AI) enabled system for localization and classification of chest X-ray disease

Machine learning techniques have lately attracted a lot of attention for their potential to execute expert-level clinical tasks, notably in the area of medical image analysis. Chest radiography is one of the most often utilized diagnostic imaging modalities in medical practice, and it necessitates timely coverage regarding the presence of probable abnormalities and disease diagnoses in the images. Computer-aided solutions for the identification of chest illness using chest radiography are being developed in medical imaging research. However, accurate localization and categorization of specific disorders in chest X-ray images is still a challenging problem due to the complex nature of radiographs, presence of different distortions, high inter-class similarities, and intra-class variations in abnormalities. In this work, we have presented an Artificial Intelligence (AI)-enabled fully automated approach using an end-to-end deep learning technique to improve the accuracy of thoracic illness diagnosis. We proposed AI-CenterNet CXR, a customized CenterNet model with an improved feature extraction network for the recognition of multi-label chest diseases. The enhanced backbone computes deep key points that improve the abnormality localization accuracy and, thus, overall disease classification performance. Moreover, the proposed architecture is lightweight and computationally efficient in comparison to the original CenterNet model. We have performed extensive experimentation to validate the effectiveness of the proposed technique using the National Institutes of Health (NIH) Chest X-ray dataset. Our method achieved an overall Area Under the Curve (AUC) of 0.888 and an average IOU of 0.801 to detect and classify the eight types of chest abnormalities. Both the qualitative and quantitative findings reveal that the suggested approach outperforms the existing methods, indicating the efficacy of our approach.

COVID-19 prognosis using limited chest X-ray images

The COrona VIrus Disease 2019 (COVID-19) pandemic is an ongoing global pandemic that has claimed millions of lives till date. Detecting COVID-19 and isolating affected patients at an early stage is crucial to contain its rapid spread. Although accurate, the primary viral test 'Reverse Transcription Polymerase Chain Reaction' (RT-PCR) for COVID-19 diagnosis has an elaborate test kit, and the turnaround time is high. This has motivated the research community to develop CXR based automated COVID-19 diagnostic methodologies. However, COVID-19 being a novel disease, there is no annotated large-scale CXR dataset for this particular disease. To address the issue of limited data, we propose to exploit a large-scale CXR dataset collected in the pre-COVID era and train a deep neural network in a self-supervised fashion to extract CXR specific features. Further, we compute attention maps between the global and the local features of the backbone convolutional network while finetuning using a limited COVID-19 CXR dataset. We empirically demonstrate the effectiveness of the proposed method. We provide a thorough ablation study to understand the effect of each proposed component. Finally, we provide visualizations highlighting the critical patches instrumental to the predictive decision made by our model. These saliency maps are not only a stepping stone towards explainable AI but also aids radiologists in localizing the infected area.

Semantic consistency generative adversarial network for cross-modality domain adaptation in ultrasound thyroid nodule classification

Deep convolutional networks have been widely used for various medical image processing tasks. However, the performance of existing learning-based networks is still limited due to the lack of large training datasets. When a general deep model is directly deployed to a new dataset with heterogeneous features, the effect of domain shifts is usually ignored, and performance degradation problems occur. In this work, by designing the semantic consistency generative adversarial network (SCGAN), we propose a new multimodal domain adaptation method for medical image diagnosis. SCGAN performs cross-domain collaborative alignment of ultrasound images and domain knowledge. Specifically, we utilize a self-attention mechanism for adversarial learning between dual domains to overcome visual differences across modal data and preserve the domain invariance of the extracted semantic features. In particular, we embed nested metric learning in the semantic information space, thus enhancing the semantic consistency of cross-modal features. Furthermore, the adversarial learning of our network is guided by a discrepancy loss for encouraging the learning of semantic-level content and a regularization term for enhancing network generalization. We evaluate our method on a thyroid ultrasound image dataset for benign and malignant diagnosis of nodules. The experimental results of a comprehensive study show that the accuracy of the SCGAN method for the classification of thyroid nodules reaches 94.30%, and the AUC reaches 97.02%. These results are significantly better than the state-of-the-art methods.

Virtual Healthcare Center for COVID-19 Patient Detection Based on Artificial Intelligence Approaches

At the end of 2019, the infectious coronavirus disease (COVID-19) was reported for the first time in Wuhan, and, since then, it has become a public health issue in China and even worldwide. This pandemic has devastating effects on societies and economies around the world, and poor countries and continents are likely to face particularly serious and long-lasting damage, which could lead to large epidemic outbreaks because of the lack of financial and health resources. The increasing number of COVID-19 tests gives more information about the epidemic spread, and this can help contain the spread to avoid more infection. As COVID-19 keeps spreading, medical products, especially those needed to perform blood tests, will become scarce as a result of the high demand and insufficient supply and logistical means. However, technological tests based on deep learning techniques and medical images could be useful in fighting this pandemic. In this perspective, we propose a COVID-19 disease diagnosis (CDD) tool that implements a deep learning technique to provide automatic symptoms checking and COVID-19 detection. Our CDD scheme implements two main steps. First, the patient's symptoms are checked, and the infection probability is predicted. Then, based on the infection probability, the patient's lungs will be diagnosed by an automatic analysis of X-ray or computerized tomography (CT) images, and the presence of the infection will be accordingly confirmed or not. The numerical results prove the efficiency of the proposed scheme by achieving an accuracy value over 90% compared with the other schemes.

Two-Stage Deep Learning Framework for Discrimination between COVID-19 and Community-Acquired Pneumonia from Chest CT scans

COVID-19 stay threatening the health infrastructure worldwide. Computed tomography (CT) was demonstrated as an informative tool for the recognition, quantification, and diagnosis of this kind of disease. It is urgent to design efficient deep learning (DL) approach to automatically localize and discriminate COVID-19 from other comparable pneumonia on lung CT scans. Thus, this study introduces a novel two-stage DL framework for discriminating COVID-19 from community-acquired pneumonia (CAP) depending on the detected infection region within CT slices. Firstly, a novel U-shaped network is presented to segment the lung area where the infection appears. Then, the concept of transfer learning is applied to the feature extraction network to empower the network capabilities in learning the disease patterns. After that, multi-scale information is captured and pooled via an attention mechanism for powerful classification performance. Thirdly, we propose an infection prediction module that use the infection location to guide the classification decision and hence provides interpretable classification decision. Finally, the proposed model was evaluated on public datasets and achieved great segmentation and classification performance outperforming the cutting-edge studies.

A novel MapReduce-based deep convolutional neural network algorithm

Deep convolutional neural networks (DCNNs), with their complex network structure and powerful feature learning and feature expression capabilities, have been remarkable successes in many large-scale recognition tasks. However, with the expectation of memory overhead and response time, along with the increasing scale of data, DCNN faces three non-rival challenges in a big data environment: excessive network parameters, slow convergence, and inefficient parallelism. To tackle these three problems, this paper develops a deep convolutional neural networks optimization algorithm (PDCNNO) in the MapReduce framework. The proposed method first pruned the network to obtain a compressed network in order to effectively reduce redundant parameters. Next, a conjugate gradient method based on modified secant equation (CGMSE) is developed in the Map phase to further accelerate the convergence of the network. Finally, a load balancing strategy based on regulate load rate (LBRLA) is proposed in the Reduce phase to quickly achieve equal grouping of data and thus improving the parallel performance of the system. We compared the PDCNNO algorithm with other algorithms on three datasets, including SVHN, EMNIST Digits, and ISLVRC2012. The experimental results show that our algorithm not only reduces the space and time overhead of network training but also obtains a well-performing speed-up ratio in a big data environment.

Deep Learning for COVID-19 Diagnosis from CT Images

COVID-19, an infectious coronavirus disease, caused a pandemic with countless deaths. From the outset, clinical institutes have explored computed tomography as an effective and complementary screening tool alongside the reverse transcriptase-polymerase chain reaction. Deep learning techniques have shown promising results in similar medical tasks and, hence, may provide solutions to COVID-19 based on medical images of patients. We aim to contribute to the research in this field by: (i) Comparing different architectures on a public and extended reference dataset to find the most suitable; (ii) Proposing a patient-oriented investigation of the best performing networks; and (iii) Evaluating their robustness in a real-world scenario, represented by cross-dataset experiments. We exploited ten well-known convolutional neural networks on two public datasets. The results show that, on the reference dataset, the most suitable architecture is VGG19, which (i) Achieved 98.87% accuracy in the network comparison; (ii) Obtained 95.91% accuracy on the patient status classification, even though it misclassifies some patients that other networks classify correctly; and (iii) The cross-dataset experiments exhibit the limitations of deep learning approaches in a real-world scenario with 70.15% accuracy, which need further investigation to improve the robustness. Thus, VGG19 architecture showed promising performance in the classification of COVID-19 cases. Nonetheless, this architecture enables extensive improvements based on its modification, or even with preprocessing step in addition to it. Finally, the cross-dataset experiments exposed the critical weakness of classifying images from heterogeneous data sources, compatible with a real-world scenario.

A Deep Learning Approach for Segmentation, Classification, and Visualization of 3-D High-Frequency Ultrasound Images of Mouse Embryos

Segmentation and mutant classification of high-frequency ultrasound (HFU) mouse embryo brain ventricle (BV) and body images can provide valuable information for developmental biologists. However, manual segmentation and identification of BV and body requires substantial time and expertise. This article proposes an accurate, efficient and explainable deep learning pipeline for automatic segmentation and classification of the BV and body. For segmentation, a two-stage framework is implemented. The first stage produces a low-resolution segmentation map, which is then used to crop a region of interest (ROI) around the target object and serve as the probability map of the autocontext input for the second-stage fine-resolution refinement network. The segmentation then becomes tractable on high-resolution 3-D images without time-consuming sliding windows. The proposed segmentation method significantly reduces inference time (102.36-0.09 s/volume ≈ 1000× faster) while maintaining high accuracy comparable to previous sliding-window approaches. Based on the BV and body segmentation map, a volumetric convolutional neural network (CNN) is trained to perform a mutant classification task. Through backpropagating the gradients of the predictions to the input BV and body segmentation map, the trained classifier is found to largely focus on the region where the Engrailed-1 (En1) mutation phenotype is known to manifest itself. This suggests that gradient backpropagation of deep learning classifiers may provide a powerful tool for automatically detecting unknown phenotypes associated with a known genetic mutation.

Weighing features of lung and heart regions for thoracic disease classification

BACKGROUND

Chest X-rays are the most commonly available and affordable radiological examination for screening thoracic diseases. According to the domain knowledge of screening chest X-rays, the pathological information usually lay on the lung and heart regions. However, it is costly to acquire region-level annotation in practice, and model training mainly relies on image-level class labels in a weakly supervised manner, which is highly challenging for computer-aided chest X-ray screening. To address this issue, some methods have been proposed recently to identify local regions containing pathological information, which is vital for thoracic disease classification. Inspired by this, we propose a novel deep learning framework to explore discriminative information from lung and heart regions.

RESULT

We design a feature extractor equipped with a multi-scale attention module to learn global attention maps from global images. To exploit disease-specific cues effectively, we locate lung and heart regions containing pathological information by a well-trained pixel-wise segmentation model to generate binarization masks. By introducing element-wise logical AND operator on the learned global attention maps and the binarization masks, we obtain local attention maps in which pixels are are 1 for lung and heart region and 0 for other regions. By zeroing features of non-lung and heart regions in attention maps, we can effectively exploit their disease-specific cues in lung and heart regions. Compared to existing methods fusing global and local features, we adopt feature weighting to avoid weakening visual cues unique to lung and heart regions. Our method with pixel-wise segmentation can help overcome the deviation of locating local regions. Evaluated by the benchmark split on the publicly available chest X-ray14 dataset, the comprehensive experiments show that our method achieves superior performance compared to the state-of-the-art methods.

CONCLUSION

We propose a novel deep framework for the multi-label classification of thoracic diseases in chest X-ray images. The proposed network aims to effectively exploit pathological regions containing the main cues for chest X-ray screening. Our proposed network has been used in clinic screening to assist the radiologists. Chest X-ray accounts for a significant proportion of radiological examinations. It is valuable to explore more methods for improving performance.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.

Part-Aware Mask-Guided Attention for Thorax Disease Classification

Thorax disease classification is a challenging task due to complex pathologies and subtle texture changes, etc. It has been extensively studied for years largely because of its wide application in computer-aided diagnosis. Most existing methods directly learn global feature representations from whole Chest X-ray (CXR) images, without considering in depth the richer visual cues lying around informative local regions. Thus, these methods often produce sub-optimal thorax disease classification performance because they ignore the very informative pathological changes around organs. In this paper, we propose a novel Part-Aware Mask-Guided Attention Network (PMGAN) that learns complementary global and local feature representations from all-organ region and multiple single-organ regions simultaneously for thorax disease classification. Specifically, multiple innovative soft attention modules are designed to progressively guide feature learning toward the global informative regions of whole CXR image. A mask-guided attention module is designed to further search for informative regions and visual cues within the all-organ or single-organ images, where attention is elegantly regularized by automatically generated organ masks and without introducing computation during the inference stage. In addition, a multi-task learning strategy is designed, which effectively maximizes the learning of complementary local and global representations. The proposed PMGAN has been evaluated on the ChestX-ray14 dataset and the experimental results demonstrate its superior thorax disease classification performance against the state-of-the-art methods.

Namiri N, Shah R, Bharadwaj U, M. Link T, D. Bucknor M, Pedoia V, Majumdar S. Automatic Deep Learning-assisted Detection and Grading of Abnormalities in Knee MRI Studies

PURPOSE

To test the hypothesis that artificial intelligence (AI) techniques can aid in identifying and assessing lesion severity in the cartilage, bone marrow, meniscus, and anterior cruciate ligament (ACL) in the knee, improving overall MRI interreader agreement.

MATERIALS AND METHODS

This retrospective study was conducted on 1435 knee MRI studies (n = 294 patients; mean age, 43 years ± 15 [standard deviation]; 153 women) collected within three previous studies (from 2011 to 2014). All MRI studies were acquired using high-spatial-resolution three-dimensional fast-spin-echo CUBE sequence. Three-dimensional convolutional neural networks were developed to detect the regions of interest within MRI studies and grade abnormalities of the cartilage, bone marrow, menisci, and ACL. Evaluation included sensitivity, specificity, and Cohen linear-weighted ĸ. The impact of AI-aided grading in intergrader agreement was assessed on an external dataset.

RESULTS

Binary lesion sensitivity reported for all tissues was between 70% and 88%. Specificity ranged from 85% to 89%. The area under the receiver operating characteristic curve for all tissues ranged from 0.83 to 0.93. Deep learning-assisted intergrader Cohen ĸ agreement significantly improved in 10 of 16 comparisons among two attending physicians and two trainees for all tissues.

CONCLUSION

The three-dimensional convolutional neural network had high sensitivity, specificity, and accuracy for knee-lesion-severity scoring and also increased intergrader agreement when used on an external dataset.Supplemental material is available for this article. Keywords: Bone Marrow, Cartilage, Computer Aided Diagnosis (CAD), Computer Applications-3D, Computer Applications-Detection/Diagnosis, Knee, Ligaments, MR-Imaging, Neural Networks, Observer Performance, Segmentation, Statistics © RSNA, 2021See also the commentary by Li and Chang in this issue.: An earlier incorrect version of this article appeared online. This article was corrected on April 16, 2021.

Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment

Chest radiography is the most common radiographic examination performed in daily clinical practice for the detection of various heart and lung abnormalities. The large amount of data to be read and reported, with more than 100 studies per day for a single radiologist, poses a challenge in consistently maintaining high interpretation accuracy. The introduction of large-scale public datasets has led to a series of novel systems for automated abnormality classification. However, the labels of these datasets were obtained using natural language processed medical reports, yielding a large degree of label noise that can impact the performance. In this study, we propose novel training strategies that handle label noise from such suboptimal data. Prior label probabilities were measured on a subset of training data re-read by 4 board-certified radiologists and were used during training to increase the robustness of the training model to the label noise. Furthermore, we exploit the high comorbidity of abnormalities observed in chest radiography and incorporate this information to further reduce the impact of label noise. Additionally, anatomical knowledge is incorporated by training the system to predict lung and heart segmentation, as well as spatial knowledge labels. To deal with multiple datasets and images derived from various scanners that apply different post-processing techniques, we introduce a novel image normalization strategy. Experiments were performed on an extensive collection of 297,541 chest radiographs from 86,876 patients, leading to a state-of-the-art performance level for 17 abnormalities from 2 datasets. With an average AUC score of 0.880 across all abnormalities, our proposed training strategies can be used to significantly improve performance scores.

Viral Pneumonia Screening on Chest X-Rays Using Confidence-Aware Anomaly Detection

Clusters of viral pneumonia occurrences over a short period may be a harbinger of an outbreak or pandemic. Rapid and accurate detection of viral pneumonia using chest X-rays can be of significant value for large-scale screening and epidemic prevention, particularly when other more sophisticated imaging modalities are not readily accessible. However, the emergence of novel mutated viruses causes a substantial dataset shift, which can greatly limit the performance of classification-based approaches. In this paper, we formulate the task of differentiating viral pneumonia from non-viral pneumonia and healthy controls into a one-class classification-based anomaly detection problem. We therefore propose the confidence-aware anomaly detection (CAAD) model, which consists of a shared feature extractor, an anomaly detection module, and a confidence prediction module. If the anomaly score produced by the anomaly detection module is large enough, or the confidence score estimated by the confidence prediction module is small enough, the input will be accepted as an anomaly case (i.e., viral pneumonia). The major advantage of our approach over binary classification is that we avoid modeling individual viral pneumonia classes explicitly and treat all known viral pneumonia cases as anomalies to improve the one-class model. The proposed model outperforms binary classification models on the clinical X-VIRAL dataset that contains 5,977 viral pneumonia (no COVID-19) cases, 37,393 non-viral pneumonia or healthy cases. Moreover, when directly testing on the X-COVID dataset that contains 106 COVID-19 cases and 107 normal controls without any fine-tuning, our model achieves an AUC of 83.61% and sensitivity of 71.70%, which is comparable to the performance of radiologists reported in the literature.

A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs

Pelvic radiograph (PXR) is essential for detecting proximal femur and pelvis injuries in trauma patients, which is also the key component for trauma survey. None of the currently available algorithms can accurately detect all kinds of trauma-related radiographic findings on PXRs. Here, we show a universal algorithm can detect most types of trauma-related radiographic findings on PXRs. We develop a multiscale deep learning algorithm called PelviXNet trained with 5204 PXRs with weakly supervised point annotation. PelviXNet yields an area under the receiver operating characteristic curve (AUROC) of 0.973 (95% CI, 0.960-0.983) and an area under the precision-recall curve (AUPRC) of 0.963 (95% CI, 0.948-0.974) in the clinical population test set of 1888 PXRs. The accuracy, sensitivity, and specificity at the cutoff value are 0.924 (95% CI, 0.912-0.936), 0.908 (95% CI, 0.885-0.908), and 0.932 (95% CI, 0.919-0.946), respectively. PelviXNet demonstrates comparable performance with radiologists and orthopedics in detecting pelvic and hip fractures.

Ensemble learning based automatic detection of tuberculosis in chest X-ray images using hybrid feature descriptors

Tuberculosis (TB) remains one of the major health problems in modern times with a high mortality rate. While efforts are being made to make early diagnosis accessible and more reliable in high burden TB countries, digital chest radiography has become a popular source for this purpose. However, the screening process requires expert radiologists which may be a potential barrier in developing countries. A fully automatic computer-aided diagnosis system can reduce the need of trained personnel for early diagnosis of TB using chest X-ray images. In this paper, we have proposed a novel TB detection technique that combines hand-crafted features with deep features (convolutional neural network-based) through Ensemble Learning. Handcrafted features were extracted via Gabor Filter and deep features were extracted via pre-trained deep learning models. Two publicly available datasets namely (i) Montgomery and (ii) Shenzhen were used to evaluate the proposed system. The proposed methodology was validated with a k-fold cross-validation scheme. The area under receiver operating characteristics curves of 0.99 and 0.97 were achieved for Shenzhen and Montgomery datasets respectively which shows the superiority of the proposed scheme.

Chest X-Ray image and pathological data based artificial intelligence enabled dual diagnostic method for multi-stage classification of COVID-19 patients

<abstract> <p>The use of Artificial Intelligence (AI) in combination with Internet of Things (IoT) drastically reduces the need to test the COVID samples manually, saving not only time but money and ultimately lives. In this paper, the authors have proposed a novel methodology to identify the COVID-19 patients with an annotated stage to enable the medical staff to manually activate a geo-fence around the subject thus ensuring early detection and isolation. The use of radiography images with pathology data used for COVID-19 identification forms the first-ever contribution by any research group globally. The novelty lies in the correct stage classification of COVID-19 subjects as well. The present analysis would bring this AI Model on the edge to make the facility an IoT-enabled unit. The developed system has been compared and extensively verified thoroughly with those of clinical observations. The significance of radiography imaging for detecting and identification of COVID-19 subjects with severity score tag for stage classification is mathematically established. In a Nutshell, this entire algorithmic workflow can be used not only for predictive analytics but also for prescriptive analytics to complete the entire pipeline from the diagnostic viewpoint of a doctor. As a matter of fact, the authors have used a supervised based learning approach aided by a multiple hypothesis based decision fusion based technique to increase the overall system's accuracy and prediction. The end to end value chain has been put under an IoT based ecosystem to leverage the combined power of AI and IoT to not only detect but also to isolate the coronavirus affected individuals. To emphasize further, the developed AI model predicts the respective categories of a coronavirus affected patients and the IoT system helps the point of care facilities to isolate and prescribe the need of hospitalization for the COVID patients.</p> </abstract>

Triple attention learning for classification of 14 thoracic diseases using chest radiography

Chest X-ray is the most common radiology examinations for the diagnosis of thoracic diseases. However, due to the complexity of pathological abnormalities and lack of detailed annotation of those abnormalities, computer-aided diagnosis (CAD) of thoracic diseases remains challenging. In this paper, we propose the triple-attention learning (A ³ Net) model for this CAD task. This model uses the pre-trained DenseNet-121 as the backbone network for feature extraction, and integrates three attention modules in a unified framework for channel-wise, element-wise, and scale-wise attention learning. Specifically, the channel-wise attention prompts the deep model to emphasize the discriminative channels of feature maps; the element-wise attention enables the deep model to focus on the regions of pathological abnormalities; the scale-wise attention facilitates the deep model to recalibrate the feature maps at different scales. The proposed model has been evaluated on 112,120images in the ChestX-ray14 dataset with the official patient-level data split. Compared to state-of-the-art deep learning models, our model achieves the highest per-class AUC in classifying 13 out of 14 thoracic diseases and the highest average per-class AUC of 0.826 over 14 thoracic diseases.