Triple attention learning for classification of 14 thoracic diseases using chest radiography

Attentional decoder networks for chest X-ray image recognition on high-resolution features

BACKGROUND AND OBJECTIVE

This paper introduces an encoder-decoder-based attentional decoder network to recognize small-size lesions in chest X-ray images. In the encoder-only network, small-size lesions disappear during the down-sampling steps or are indistinguishable in the low-resolution feature maps. To address these issues, the proposed network processes images in the encoder-decoder architecture similar to U-Net families and classifies lesions by globally pooling high-resolution feature maps. However, two challenging obstacles prohibit U-Net families from being extended to classification: (1) the up-sampling procedure consumes considerable resources, and (2) there needs to be an effective pooling approach for the high-resolution feature maps.

METHODS

Therefore, the proposed network employs a lightweight attentional decoder and harmonic magnitude transform. The attentional decoder up-samples the given features with the low-resolution features as the key and value while the high-resolution features as the query. Since multi-scaled features interact, up-sampled features embody global context at a high resolution, maintaining pathological locality. In addition, harmonic magnitude transform is devised for pooling high-resolution feature maps in the frequency domain. We borrow the shift theorem of the Fourier transform to preserve the translation invariant property and further reduce the parameters of the pooling layer by an efficient embedding strategy.

RESULTS

The proposed network achieves state-of-the-art classification performance on the three public chest X-ray datasets, such as NIH, CheXpert, and MIMIC-CXR.

CONCLUSIONS

In conclusion, the proposed efficient encoder-decoder network recognizes small-size lesions well in chest X-ray images by efficiently up-sampling feature maps through an attentional decoder and processing high-resolution feature maps with harmonic magnitude transform. We open-source our implementation at https://github.com/Lab-LVM/ADNet.

Automated thorax disease diagnosis using multi-branch residual attention network

Chest X-ray (CXR) is an extensively utilized radiological modality for supporting the diagnosis of chest diseases. However, existing research approaches suffer from limitations in effectively integrating multi-scale CXR image features and are also hindered by imbalanced datasets. Therefore, there is a pressing need for further advancement in computer-aided diagnosis (CAD) of thoracic diseases. To tackle these challenges, we propose a multi-branch residual attention network (MBRANet) for thoracic disease diagnosis. MBRANet comprises three components. Firstly, to address the issue of inadequate extraction of spatial and positional information by the convolutional layer, a novel residual structure incorporating a coordinate attention (CA) module is proposed to extract features at multiple scales. Next, based on the concept of a Feature Pyramid Network (FPN), we perform multi-scale feature fusion in the following manner. Thirdly, we propose a novel Multi-Branch Feature Classifier (MFC) approach, which leverages the class-specific residual attention (CSRA) module for classification instead of relying solely on the fully connected layer. In addition, the designed BCEWithLabelSmoothing loss function improves the generalization ability and mitigates the problem of class imbalance by introducing a smoothing factor. We evaluated MBRANet on the ChestX-Ray14, CheXpert, MIMIC-CXR, and IU X-Ray datasets and achieved average AUCs of 0.841, 0.895, 0.805, and 0.745, respectively. Our method outperformed state-of-the-art baselines on these benchmark datasets.

Patch-Level Feature Selection for Thoracic Disease Classification by Chest X-ray Images Using Information Bottleneck

Chest X-ray (CXR) examination serves as a widely employed clinical test in medical diagnostics. Many studied have tried to apply artificial intelligence (AI) programs to analyze CXR images. Despite numerous positive outcomes, assessing the applicability of AI models for comprehensive diagnostic support remains a formidable challenge. We observed that, even when AI models exhibit high accuracy on one dataset, their performance may deteriorate when tested on another. To address this issue, we propose incorporating a variational information bottleneck (VIB) at the patch level to enhance the generalizability of diagnostic support models. The VIB introduces a probabilistic model aimed at approximating the posterior distribution of latent variables given input data, thereby enhancing the model's generalization capabilities on unseen data. Unlike the conventional VIB approaches that flatten features and use a re-parameterization trick to sample a new latent feature, our method applies the trick to 2D feature maps. This design allows only important pixels to respond, and the model will select important patches in an image. Moreover, the proposed patch-level VIB seamlessly integrates with various convolutional neural networks, offering a versatile solution to improve performance. Experimental results illustrate enhanced accuracy in standard experiment settings. In addition, the method shows robust improvement when training and testing on different datasets.

A deep-learning-based framework for identifying and localizing multiple abnormalities and assessing cardiomegaly in chest X-ray

Accurate identification and localization of multiple abnormalities are crucial steps in the interpretation of chest X-rays (CXRs); however, the lack of a large CXR dataset with bounding boxes severely constrains accurate localization research based on deep learning. We created a large CXR dataset named CXR-AL14, containing 165,988 CXRs and 253,844 bounding boxes. On the basis of this dataset, a deep-learning-based framework was developed to identify and localize 14 common abnormalities and calculate the cardiothoracic ratio (CTR) simultaneously. The mean average precision values obtained by the model for 14 abnormalities reached 0.572-0.631 with an intersection-over-union threshold of 0.5, and the intraclass correlation coefficient of the CTR algorithm exceeded 0.95 on the held-out, multicentre and prospective test datasets. This framework shows an excellent performance, good generalization ability and strong clinical applicability, which is superior to senior radiologists and suitable for routine clinical settings.

Chest x-ray diagnosis via spatial-channel high-order attention representation learning

Objective. Chest x-ray image representation and learning is an important problem in computer-aided diagnostic area. Existing methods usually adopt CNN or Transformers for feature representation learning and focus on learning effective representations for chest x-ray images. Although good performance can be obtained, however, these works are still limited mainly due to the ignorance of mining the correlations of channels and pay little attention on the local context-aware feature representation of chest x-ray image.Approach. To address these problems, in this paper, we propose a novel spatial-channel high-order attention model (SCHA) for chest x-ray image representation and diagnosis. The proposed network architecture mainly contains three modules, i.e. CEBN, SHAM and CHAM. To be specific, firstly, we introduce a context-enhanced backbone network by employing multi-head self-attention to extract initial features for the input chest x-ray images. Then, we develop a novel SCHA which contains both spatial and channel high-order attention learning branches. For the spatial branch, we develop a novel local biased self-attention mechanism which can capture both local and long-range global dependences of positions to learn rich context-aware representation. For the channel branch, we employ Brownian Distance Covariance to encode the correlation information of channels and regard it as the image representation. Finally, the two learning branches are integrated together for the final multi-label diagnosis classification and prediction.Main results. Experiments on the commonly used datasets including ChestX-ray14 and CheXpert demonstrate that our proposed SCHA approach can obtain better performance when comparing many related approaches.Significance. This study obtains a more discriminative method for chest x-ray classification and provides a technique for computer-aided diagnosis.

Pulmonary nodule detection in x-ray images by feature augmentation and context aggregation

Recent developments in x-ray image based pulmonary nodule detection have achieved remarkable results. However, existing methods are focused on transferring off-the-shelf coarse-grained classification models and fine-grained detection models rather than developing a dedicated framework optimized for nodule detection. In this paper, we propose PN-DetX, which as we know is the first dedicated pulmonary nodule detection framework. PN-DetX incorporates feature fusion and self-attention into x-ray based pulmonary nodule detection tasks, achieving improved detection performance. Specifically, PN-DetX adopts CSPDarknet backbone to extract features, and utilizes feature augmentation module to fuse features from different levels followed by context aggregation module to aggregate semantic information. To evaluate the efficacy of our method, we collect aLArge-scalePulmonaryNOduleDetection dataset,LAPNOD, comprising 2954 x-ray images along with expert-annotated ground truths. As we know, this is the first large-scale chest x-ray pulmonary nodule detection dataset. Experiments demonstrates that our method outperforms baseline by 3.8% mAP and 5.1%AP0.5. The generality of our approach is also evaluated on the publicly available dataset NODE21. We aspire for our method to serve as an inspiration for future research in the field of pulmonary nodule detection. The dataset and codes will be made in public.

Dynamic feature splicing for few-shot rare disease diagnosis

Annotated images for rare disease diagnosis are extremely hard to collect. Therefore, identifying rare diseases under a few-shot learning (FSL) setting is significant. Existing FSL methods transfer useful and global knowledge from base classes with abundant training samples to enrich features of novel classes with few training samples, but still face difficulties when being applied to medical images due to the complex lesion characteristics and large intra-class variance. In this paper, we propose a dynamic feature splicing (DNFS) framework for few-shot rare disease diagnosis. Under DNFS, both low-level features (i.e., the output of three convolutional blocks) and high-level features (i.e., the output of the last fully connected layer) of novel classes are dynamically enriched. We construct the position coherent DNFS (P-DNFS) module to perform low-level feature splicing, where a lesion-oriented Transformer is designed to detect lesion regions. Thus, novel-class channels are replaced by similar base-class channels within the detected lesion regions to achieve disease-related feature enrichment. We also devise a semantic coherent DNFS (S-DNFS) module to perform high-level feature splicing. It explores cross-image channel relations and selects base-class channels with semantic consistency for explicit knowledge transfer. Both low-level and high-level feature splicings are performed dynamically and iteratively. Consequently, abundant spliced features are generated for disease diagnosis, leading to more accurate decision boundary and improved diagnosis performance. Extensive experiments have been conducted on three medical image classification datasets. Our results suggest that the proposed DNFS achieves superior performance against state-of-the-art approaches.

Efficient Thorax Disease Classification and Localization Using DCNN and Chest X-ray Images

Thorax disease is a life-threatening disease caused by bacterial infections that occur in the lungs. It could be deadly if not treated at the right time, so early diagnosis of thoracic diseases is vital. The suggested study can assist radiologists in more swiftly diagnosing thorax disorders and in the rapid airport screening of patients with a thorax disease, such as pneumonia. This paper focuses on automatically detecting and localizing thorax disease using chest X-ray images. It provides accurate detection and localization using DenseNet-121 which is foundation of our proposed framework, called Z-Net. The proposed framework utilizes the weighted cross-entropy loss function (W-CEL) that manages class imbalance issue in the ChestX-ray14 dataset, which helped in achieving the highest performance as compared to the previous models. The 112,120 images contained in the ChestX-ray14 dataset (60,412 images are normal, and the rest contain thorax diseases) were preprocessed and then trained for classification and localization. This work uses computer-aided diagnosis (CAD) system that supports development of highly accurate and precise computer-aided systems. We aim to develop a CAD system using a deep learning approach. Our quantitative results show high AUC scores in comparison with the latest research works. The proposed approach achieved the highest mean AUC score of 85.8%. This is the highest accuracy documented in the literature for any related model.

Improving adversarial robustness of medical imaging systems via adding global attention noise

Recent studies have found that medical images are vulnerable to adversarial attacks. However, it is difficult to protect medical imaging systems from adversarial examples in that the lesion features of medical images are more complex with high resolution. Therefore, a simple and effective method is needed to address these issues to improve medical imaging systems' robustness. We find that the attackers generate adversarial perturbations corresponding to the lesion characteristics of different medical image datasets, which can shift the model's attention to other places. In this paper, we propose global attention noise (GATN) injection, including global noise in the example layer and attention noise in the feature layers. Global noise enhances the lesion features of the medical images, thus keeping the examples away from the sharp areas where the model is vulnerable. The attention noise further locally smooths the model from small perturbations. According to the characteristic of medical image datasets, we introduce Global attention lesion-unrelated noise (GATN-UR) for datasets with unclear lesion boundaries and Global attention lesion-related noise (GATN-R) for datasets with clear lesion boundaries. Extensive experiments on ChestX-ray, Dermatology, and Fundoscopy datasets show that GATN improves the robustness of medical diagnosis models against a variety of powerful attacks and significantly outperforms the existing adversarial defense methods. To be specific, the robust accuracy is 86.66% on ChestX-ray, 72.49% on Dermatology, and 90.17% on Fundoscopy under PGD attack. Under the AA attack, it achieves robust accuracy of 87.70% on ChestX-ray, 66.85% on Dermatology, and 87.83% on Fundoscopy.

Multi-Label Local to Global Learning: A Novel Learning Paradigm for Chest X-Ray Abnormality Classification

Deep neural network (DNN) approaches have shown remarkable progress in automatic Chest X-rays classification. However, existing methods use a training scheme that simultaneously trains all abnormalities without considering their learning priority. Inspired by the clinical practice of radiologists progressively recognizing more abnormalities and the observation that existing curriculum learning (CL) methods based on image difficulty may not be suitable for disease diagnosis, we propose a novel CL paradigm, named multi-label local to global (ML-LGL). This approach iteratively trains DNN models on gradually increasing abnormalities within the dataset, i,e, from fewer abnormalities (local) to more ones (global). At each iteration, we first build the local category by adding high-priority abnormalities for training, and the abnormality's priority is determined by our three proposed clinical knowledge-leveraged selection functions. Then, images containing abnormalities in the local category are gathered to form a new training set. The model is lastly trained on this set using a dynamic loss. Additionally, we demonstrate the superiority of ML-LGL from the perspective of the model's initial stability during training. Experimental results on three open-source datasets, PLCO, ChestX-ray14 and CheXpert show that our proposed learning paradigm outperforms baselines and achieves comparable results to state-of-the-art methods. The improved performance promises potential applications in multi-label Chest X-ray classification.

Development of Debiasing Technique for Lung Nodule Chest X-ray Datasets to Generalize Deep Learning Models

Screening programs for early lung cancer diagnosis are uncommon, primarily due to the challenge of reaching at-risk patients located in rural areas far from medical facilities. To overcome this obstacle, a comprehensive approach is needed that combines mobility, low cost, speed, accuracy, and privacy. One potential solution lies in combining the chest X-ray imaging mode with federated deep learning, ensuring that no single data source can bias the model adversely. This study presents a pre-processing pipeline designed to debias chest X-ray images, thereby enhancing internal classification and external generalization. The pipeline employs a pruning mechanism to train a deep learning model for nodule detection, utilizing the most informative images from a publicly available lung nodule X-ray dataset. Histogram equalization is used to remove systematic differences in image brightness and contrast. Model training is then performed using combinations of lung field segmentation, close cropping, and rib/bone suppression. The resulting deep learning models, generated through this pre-processing pipeline, demonstrate successful generalization on an independent lung nodule dataset. By eliminating confounding variables in chest X-ray images and suppressing signal noise from the bone structures, the proposed deep learning lung nodule detection algorithm achieves an external generalization accuracy of 89%. This approach paves the way for the development of a low-cost and accessible deep learning-based clinical system for lung cancer screening.

Deep learning for pneumothorax diagnosis: a systematic review and meta-analysis

BACKGROUND

Deep learning (DL), a subset of artificial intelligence (AI), has been applied to pneumothorax diagnosis to aid physician diagnosis, but no meta-analysis has been performed.

METHODS

A search of multiple electronic databases through September 2022 was performed to identify studies that applied DL for pneumothorax diagnosis using imaging. Meta-analysis via a hierarchical model to calculate the summary area under the curve (AUC) and pooled sensitivity and specificity for both DL and physicians was performed. Risk of bias was assessed using a modified Prediction Model Study Risk of Bias Assessment Tool.

RESULTS

In 56 of the 63 primary studies, pneumothorax was identified from chest radiography. The total AUC was 0.97 (95% CI 0.96-0.98) for both DL and physicians. The total pooled sensitivity was 84% (95% CI 79-89%) for DL and 85% (95% CI 73-92%) for physicians and the pooled specificity was 96% (95% CI 94-98%) for DL and 98% (95% CI 95-99%) for physicians. More than half of the original studies (57%) had a high risk of bias.

CONCLUSIONS

Our review found the diagnostic performance of DL models was similar to that of physicians, although the majority of studies had a high risk of bias. Further pneumothorax AI research is needed.

High-Resolution Network with Dynamic Convolution and Coordinate Attention for Classification of Chest X-ray Images

The development of automatic chest X-ray (CXR) disease classification algorithms is significant for diagnosing thoracic diseases. Owing to the characteristics of lesions in CXR images, including high similarity in appearance of the disease, varied sizes, and different occurrence locations, most existing convolutional neural network-based methods have insufficient feature extraction for thoracic lesions and struggle to adapt to changes in lesion size and location. To address these issues, this study proposes a high-resolution classification network with dynamic convolution and coordinate attention (HRCC-Net). In the method, this study suggests a parallel multi-resolution network in which a high-resolution branch acquires essential detailed features of the lesion and multi-resolution feature swapping and fusion to obtain multiple receptive fields to extract complicated disease features adequately. Furthermore, this study proposes dynamic convolution to enhance the network's ability to represent multi-scale information to accommodate lesions of diverse scales. In addition, this study introduces a coordinate attention mechanism, which enables automatic focus on pathologically relevant regions and capturing the variations in lesion location. The proposed method is evaluated on ChestX-ray14 and CheXpert datasets. The average AUC (area under ROC curve) values reach 0.845 and 0.913, respectively, indicating this method's advantages compared with the currently available methods. Meanwhile, with its specificity and sensitivity to measure the performance of medical diagnostic systems, the network can improve diagnostic efficiency while reducing the rate of misdiagnosis. The proposed algorithm has great potential for thoracic disease diagnosis and treatment.

Classification of chest X-ray images by incorporation of medical domain knowledge into operation branch networks

BACKGROUND

This study was conducted to alleviate a common difficulty in chest X-ray image diagnosis: The attention region in a convolutional neural network (CNN) does not often match the doctor's point of focus. The method presented herein, which guides the area of attention in CNN to a medically plausible region, can thereby improve diagnostic capabilities.

METHODS

The model is based on an attention branch network, which has excellent interpretability of the classification model. This model has an additional new operation branch that guides the attention region to the lung field and heart in chest X-ray images. We also used three chest X-ray image datasets (Teikyo, Tokushima, and ChestX-ray14) to evaluate the CNN attention area of interest in these fields. Additionally, after devising a quantitative method of evaluating improvement of a CNN's region of interest, we applied it to evaluation of the proposed model.

RESULTS

Operation branch networks maintain or improve the area under the curve to a greater degree than conventional CNNs do. Furthermore, the network better emphasizes reasonable anatomical parts in chest X-ray images.

CONCLUSIONS

The proposed network better emphasizes the reasonable anatomical parts in chest X-ray images. This method can enhance capabilities for image interpretation based on judgment.

Broad learning solution for rapid diagnosis of COVID-19

COVID-19 has put all of humanity in a health dilemma as it spreads rapidly. For many infectious diseases, the delay of detection results leads to the spread of infection and an increase in healthcare costs. COVID-19 diagnostic methods rely on a large number of redundant labeled data and time-consuming data training processes to obtain satisfactory results. However, as a new epidemic, obtaining large clinical datasets is still challenging, which will inhibit the training of deep models. And a model that can really rapidly diagnose COVID-19 at all stages of the model has still not been proposed. To address these limitations, we combine feature attention and broad learning to propose a diagnostic system (FA-BLS) for COVID-19 pulmonary infection, which introduces a broad learning structure to address the slow diagnosis speed of existing deep learning methods. In our network, transfer learning is performed with ResNet50 convolutional modules with fixed weights to extract image features, and the attention mechanism is used to enhance feature representation. After that, feature nodes and enhancement nodes are generated by broad learning with random weights to adaptly select features for diagnosis. Finally, three publicly accessible datasets were used to evaluate our optimization model. It was determined that the FA-BLS model had a 26-130 times faster training speed than deep learning with a similar level of accuracy, which can achieve a fast and accurate diagnosis, achieve effective isolation from COVID-19 and the proposed method also opens up a new method for other types of chest CT image recognition problems.

uRP: An integrated research platform for one-stop analysis of medical images

Introduction

Medical image analysis is of tremendous importance in serving clinical diagnosis, treatment planning, as well as prognosis assessment. However, the image analysis process usually involves multiple modality-specific software and relies on rigorous manual operations, which is time-consuming and potentially low reproducible.

Methods

We present an integrated platform - uAI Research Portal (uRP), to achieve one-stop analyses of multimodal images such as CT, MRI, and PET for clinical research applications. The proposed uRP adopts a modularized architecture to be multifunctional, extensible, and customizable.

Results and Discussion

The uRP shows 3 advantages, as it 1) spans a wealth of algorithms for image processing including semi-automatic delineation, automatic segmentation, registration, classification, quantitative analysis, and image visualization, to realize a one-stop analytic pipeline, 2) integrates a variety of functional modules, which can be directly applied, combined, or customized for specific application domains, such as brain, pneumonia, and knee joint analyses, 3) enables full-stack analysis of one disease, including diagnosis, treatment planning, and prognosis assessment, as well as full-spectrum coverage for multiple disease applications. With the continuous development and inclusion of advanced algorithms, we expect this platform to largely simplify the clinical scientific research process and promote more and better discoveries.

A stereo spatial decoupling network for medical image classification

Deep convolutional neural network (CNN) has made great progress in medical image classification. However, it is difficult to establish effective spatial associations, and always extracts similar low-level features, resulting in redundancy of information. To solve these limitations, we propose a stereo spatial discoupling network (TSDNets), which can leverage the multi-dimensional spatial details of medical images. Then, we use an attention mechanism to progressively extract the most discriminative features from three directions: horizontal, vertical, and depth. Moreover, a cross feature screening strategy is used to divide the original feature maps into three levels: important, secondary and redundant. Specifically, we design a cross feature screening module (CFSM) and a semantic guided decoupling module (SGDM) to model multi-dimension spatial relationships, thereby enhancing the feature representation capabilities. The extensive experiments conducted on multiple open source baseline datasets demonstrate that our TSDNets outperforms previous state-of-the-art models.

Review on chest pathogies detection systems using deep learning techniques

Chest radiography is the standard and most affordable way to diagnose, analyze, and examine different thoracic and chest diseases. Typically, the radiograph is examined by an expert radiologist or physician to decide about a particular anomaly, if exists. Moreover, computer-aided methods are used to assist radiologists and make the analysis process accurate, fast, and more automated. A tremendous improvement in automatic chest pathologies detection and analysis can be observed with the emergence of deep learning. The survey aims to review, technically evaluate, and synthesize the different computer-aided chest pathologies detection systems. The state-of-the-art of single and multi-pathologies detection systems, which are published in the last five years, are thoroughly discussed. The taxonomy of image acquisition, dataset preprocessing, feature extraction, and deep learning models are presented. The mathematical concepts related to feature extraction model architectures are discussed. Moreover, the different articles are compared based on their contributions, datasets, methods used, and the results achieved. The article ends with the main findings, current trends, challenges, and future recommendations.

Attention-based deep learning for breast lesions classification on contrast enhanced spectral mammography: a multicentre study

BACKGROUND

This study aims to develop an attention-based deep learning model for distinguishing benign from malignant breast lesions on CESM.

METHODS

Preoperative CESM images of 1239 patients, which were definitely diagnosed on pathology in a multicentre cohort, were divided into training and validation sets, internal and external test sets. The regions of interest of the breast lesions were outlined manually by a senior radiologist. We adopted three conventional convolutional neural networks (CNNs), namely, DenseNet 121, Xception, and ResNet 50, as the backbone architectures and incorporated the convolutional block attention module (CBAM) into them for classification. The performance of the models was analysed in terms of the receiver operating characteristic (ROC) curve, accuracy, the positive predictive value (PPV), the negative predictive value (NPV), the F1 score, the precision recall curve (PRC), and heat maps. The final models were compared with the diagnostic performance of conventional CNNs, radiomics models, and two radiologists with specialised breast imaging experience.

RESULTS

The best-performing deep learning model, that is, the CBAM-based Xception, achieved an area under the ROC curve (AUC) of 0.970, a sensitivity of 0.848, a specificity of 1.000, and an accuracy of 0.891 on the external test set, which was higher than those of other CNNs, radiomics models, and radiologists. The PRC and the heat maps also indicated the favourable predictive performance of the attention-based CNN model. The diagnostic performance of two radiologists improved with deep learning assistance.

CONCLUSIONS

Using an attention-based deep learning model based on CESM images can help to distinguishing benign from malignant breast lesions, and the diagnostic performance of radiologists improved with deep learning assistance.

Machine Learning Augmented Interpretation of Chest X-rays: A Systematic Review

Limitations of the chest X-ray (CXR) have resulted in attempts to create machine learning systems to assist clinicians and improve interpretation accuracy. An understanding of the capabilities and limitations of modern machine learning systems is necessary for clinicians as these tools begin to permeate practice. This systematic review aimed to provide an overview of machine learning applications designed to facilitate CXR interpretation. A systematic search strategy was executed to identify research into machine learning algorithms capable of detecting >2 radiographic findings on CXRs published between January 2020 and September 2022. Model details and study characteristics, including risk of bias and quality, were summarized. Initially, 2248 articles were retrieved, with 46 included in the final review. Published models demonstrated strong standalone performance and were typically as accurate, or more accurate, than radiologists or non-radiologist clinicians. Multiple studies demonstrated an improvement in the clinical finding classification performance of clinicians when models acted as a diagnostic assistance device. Device performance was compared with that of clinicians in 30% of studies, while effects on clinical perception and diagnosis were evaluated in 19%. Only one study was prospectively run. On average, 128,662 images were used to train and validate models. Most classified less than eight clinical findings, while the three most comprehensive models classified 54, 72, and 124 findings. This review suggests that machine learning devices designed to facilitate CXR interpretation perform strongly, improve the detection performance of clinicians, and improve the efficiency of radiology workflow. Several limitations were identified, and clinician involvement and expertise will be key to driving the safe implementation of quality CXR machine learning systems.

A 178-clinical-center experiment of integrating AI solutions for lung pathology diagnosis

In 2020, an experiment testing AI solutions for lung X-ray analysis on a multi-hospital network was conducted. The multi-hospital network linked 178 Moscow state healthcare centers, where all chest X-rays from the network were redirected to a research facility, analyzed with AI, and returned to the centers. The experiment was formulated as a public competition with monetary awards for participating industrial and research teams. The task was to perform the binary detection of abnormalities from chest X-rays. For the objective real-life evaluation, no training X-rays were provided to the participants. This paper presents one of the top-performing AI frameworks from this experiment. First, the framework used two EfficientNets, histograms of gradients, Haar feature ensembles, and local binary patterns to recognize whether an input image represents an acceptable lung X-ray sample, meaning the X-ray is not grayscale inverted, is a frontal chest X-ray, and completely captures both lung fields. Second, the framework extracted the region with lung fields and then passed them to a multi-head DenseNet, where the heads recognized the patient's gender, age and the potential presence of abnormalities, and generated the heatmap with the abnormality regions highlighted. During one month of the experiment from 11.23.2020 to 12.25.2020, 17,888 cases have been analyzed by the framework with 11,902 cases having radiological reports with the reference diagnoses that were unequivocally parsed by the experiment organizers. The performance measured in terms of the area under receiving operator curve (AUC) was 0.77. The AUC for individual diseases ranged from 0.55 for herniation to 0.90 for pneumothorax.

Survey of Explainable AI Techniques in Healthcare

Artificial intelligence (AI) with deep learning models has been widely applied in numerous domains, including medical imaging and healthcare tasks. In the medical field, any judgment or decision is fraught with risk. A doctor will carefully judge whether a patient is sick before forming a reasonable explanation based on the patient's symptoms and/or an examination. Therefore, to be a viable and accepted tool, AI needs to mimic human judgment and interpretation skills. Specifically, explainable AI (XAI) aims to explain the information behind the black-box model of deep learning that reveals how the decisions are made. This paper provides a survey of the most recent XAI techniques used in healthcare and related medical imaging applications. We summarize and categorize the XAI types, and highlight the algorithms used to increase interpretability in medical imaging topics. In addition, we focus on the challenging XAI problems in medical applications and provide guidelines to develop better interpretations of deep learning models using XAI concepts in medical image and text analysis. Furthermore, this survey provides future directions to guide developers and researchers for future prospective investigations on clinical topics, particularly on applications with medical imaging.

Learning-based landmark detection in pelvis x-rays with attention mechanism: data from the osteoarthritis initiative

Patients with developmental dysplasia of the hip can have this problem throughout their lifetime. The problem is difficult to detect by radiologists throughout x-ray because of an abrasion of anatomical structures. Thus, the landmarks should be automatically and precisely located. In this paper, we propose an attention mechanism of combining multi-dimension information on the basis of separating spatial dimension. The proposed attention mechanism decouples spatial dimension and forms width-channel dimension and height-channel dimension by 1D pooling operations in the height and width of spatial dimension. Then non-local means operations are performed to capture the correlation between long-range pixels in width-channel dimension, as well as that in height-channel dimension at different resolutions. The proposed attention mechanism modules are inserted into the skipped connections of U-Net to form a novel landmark detection structure. This landmark detection method was trained and evaluated through five-fold cross-validation on an open-source dataset, including 524 pelvis x-ray, each containing eight landmarks in pelvis, and achieved excellent performance compared to other landmark detection models. The average point-to-point errors of U-Net, HR-Net, CE-Net, and the proposed network were 3.5651 mm, 3.6118 mm, 3.3914 mm and 3.1350 mm, respectively. The results indicate that the proposed method has the highest detection accuracy. Furthermore, an open-source pelvis dataset is annotated and released for open research.

A Survey on AI Techniques for Thoracic Diseases Diagnosis Using Medical Images

Thoracic diseases refer to disorders that affect the lungs, heart, and other parts of the rib cage, such as pneumonia, novel coronavirus disease (COVID-19), tuberculosis, cardiomegaly, and fracture. Millions of people die every year from thoracic diseases. Therefore, early detection of these diseases is essential and can save many lives. Earlier, only highly experienced radiologists examined thoracic diseases, but recent developments in image processing and deep learning techniques are opening the door for the automated detection of these diseases. In this paper, we present a comprehensive review including: types of thoracic diseases; examination types of thoracic images; image pre-processing; models of deep learning applied to the detection of thoracic diseases (e.g., pneumonia, COVID-19, edema, fibrosis, tuberculosis, chronic obstructive pulmonary disease (COPD), and lung cancer); transfer learning background knowledge; ensemble learning; and future initiatives for improving the efficacy of deep learning models in applications that detect thoracic diseases. Through this survey paper, researchers may be able to gain an overall and systematic knowledge of deep learning applications in medical thoracic images. The review investigates a performance comparison of various models and a comparison of various datasets.

Explainable multi-module semantic guided attention based network for medical image segmentation

Automated segmentation of medical images is crucial for disease diagnosis and treatment planning. Medical image segmentation has been improved based on the convolutional neural networks (CNNs) models. Unfortunately, they are still limited by scenarios in which the segmentation objective has large variations in size, boundary, position, and shape. Moreover, current CNNs have low explainability, restricting their use in clinical decisions. In this paper, we involve substantial use of various attentions in a CNN model and present an explainable multi-module semantic guided attention based network (MSGA-Net) for explainable and highly accurate medical image segmentation, which involves considering the most significant spatial regions, boundaries, scales, and channels. Specifically, we present a multi-scale attention module (MSA) to extract the most salient features at various scales from medical images. Then, we propose a semantic region-guided attention mechanism (SRGA) including location attention (LAM), channel-wise attention (CWA), and edge attention (EA) modules to extract the most important spatial, channel-wise, boundary-related features for interested regions. Moreover, we present a sequence of fine-tuning steps with the SRGA module to gradually weight the significance of interesting regions while simultaneously reducing the noise. In this work, we experimented with three different types of medical images such as dermoscopic images (HAM10000 dataset), multi-organ CT images (CHAOS 2019 dataset), and Brain tumor MRI images (BraTS 2020 dataset). Extensive experiments on all types of medical images revealed that our proposed MSGA-Net substantially increased the overall performance of all metrics over the existing models. Moreover, displaying the attention feature maps has more explainability than state-of-the-art models.

PCAN: Pixel-wise classification and attention network for thoracic disease classification and weakly supervised localization

Automatic chest X-ray (CXR) disease classification has drawn increasing public attention as CXR is widely used in thoracic disease diagnosis. Existing classification networks typically employ a global average pooling layer to produce the final feature for the subsequent classifier. This limits the classification performance owing to the characteristics of lesions in CXR images, including small relative sizes, varied absolute sizes, and different occurrence locations. In this study, we propose a pixel-wise classification and attention network (PCAN) to simultaneously perform disease classification and weakly supervised localization, which provides interpretability for disease classification. The PCAN comprises a backbone network for extracting mid-level features, a pixel-wise classification branch (pc-branch) for generating pixel-wise diagnoses, and a pixel-wise attention branch (pa-branch) for producing pixel-wise weights. The pc-branch is capable of explicitly detecting small lesions, and the pa-branch is capable of adaptively focusing on different regions when classifying different thoracic diseases. Then, the pixel-wise diagnoses are multiplied with the pixel-wise weights to obtain the disease localization map, which provides the sizes and locations of lesions in a manner of weakly supervised learning. The final image-wise diagnosis is obtained by summing up the disease localization map at the spatial dimension. Comprehensive experiments conducted on the ChestX-ray14 and CheXpert datasets demonstrate the effectiveness of the proposed PCAN, which has great potential for thoracic disease diagnosis and treatment. The source codes are available at https://github.com/fzfs/PCAN.

Thoracic Imaging in China: Yesterday, Today, and Tomorrow

Thoracic imaging has been revolutionized through advances in technology and research around the world, and so has China. Thoracic imaging in China has progressed from anatomic observation to quantitative and functional evaluation, from using traditional approaches to using artificial intelligence. This article will review the past, present, and future of thoracic imaging in China, in an attempt to establish new accepted strategies moving forward.

MTU: A multi-tasking U-net with hybrid convolutional learning and attention modules for cancer classification and gland Segmentation in Colon Histopathological Images

A clinically comparable multi-tasking computerized deep U-Net-based model is demonstrated in this paper. It intends to offer clinical gland morphometric information and cancer grade classification to be provided as referential opinions for pathologists in order to abate human errors. It embraces enhanced feature learning capability that aids in extraction of potent multi-scale features; efficacious semantic gap recovery during feature concatenation; and successful interception of resolution-degradation and vanishing gradient problems while performing moderate computations. It is proposed by integrating three unique novel structural components namely Hybrid Convolutional Learning Units in the encoder and decoder, Attention Learning Units in skip connection, and Multi-Scalar Dilated Transitional Unit as the transitional layer in the traditional U-Net architecture. These units are composed of the amalgamated phenomenon of multi-level convolutional learning through conventional, atrous, residual, depth-wise, and point-wise convolutions which are further incorporated with target-specific attention learning and enlarged effectual receptive field size. Also, pre-processing techniques of patch-sampling, augmentation (color and morphological), stain-normalization, etc. are employed to burgeon its generalizability. To build network invariance towards digital variability, exhaustive experiments are conducted using three public datasets (Colorectal Adenocarcinoma Gland (CRAG), Gland Segmentation (GlaS) challenge, and Lung Colon-25000 (LC-25K) dataset)) and then its robustness is verified using an in-house private dataset of Hospital Colon (HosC). For the cancer classification, the proposed model achieved results of Accuracy (CRAG(95%), GlaS(97.5%), LC-25K(99.97%), HosC(99.45%)), Precision (CRAG(0.9678), GlaS(0.9768), LC-25K(1), HosC(1)), F1-score (CRAG(0.968), GlaS(0.977), LC 25K(0.9997), HosC(0.9965)), and Recall (CRAG(0.9677), GlaS(0.9767), LC-25K(0.9994), HosC(0.9931)). For the gland detection and segmentation, the proposed model achieved competitive results of F1-score (CRAG(0.924), GlaS(Test A(0.949), Test B(0.918)), LC-25K(0.916), HosC(0.959)); Object-Dice Index (CRAG(0.959), GlaS(Test A(0.956), Test B(0.909)), LC-25K(0.929), HosC(0.922)), and Object-Hausdorff Distance (CRAG(90.47), GlaS(Test A(23.17), Test B(71.53)), LC-25K(96.28), HosC(85.45)). In addition, the activation mappings for testing the interpretability of the classification decision-making process are reported by utilizing techniques of Local Interpretable Model-Agnostic Explanations, Occlusion Sensitivity, and Gradient-Weighted Class Activation Mappings. This is done to provide further evidence about the model's self-learning capability of the comparable patterns considered relevant by pathologists without any pre-requisite for annotations. These activation mapping visualization outcomes are evaluated by proficient pathologists, and they delivered these images with a class-path validation score of (CRAG(9.31), GlaS(9.25), LC-25K(9.05), and HosC(9.85)). Furthermore, the seg-path validation score of (GlaS (Test A(9.40), Test B(9.25)), CRAG(9.27), LC-25K(9.01), HosC(9.19)) given by multiple pathologists is included for the final segmented outcomes to substantiate the clinical relevance and suitability for facilitation at the clinical level. The proposed model will aid pathologists to formulate an accurate diagnosis by providing a referential opinion during the morphology assessment of histopathology images. It will reduce unintentional human error in cancer diagnosis and consequently will enhance patient survival rate.

Development and validation of an abnormality-derived deep-learning diagnostic system for major respiratory diseases

Respiratory diseases impose a tremendous global health burden on large patient populations. In this study, we aimed to develop DeepMRD^TR, a deep learning-based medical image interpretation system for the diagnosis of major respiratory diseases based on the automated identification of a wide range of radiological abnormalities through computed tomography (CT) and chest X-ray (CXR) from real-world, large-scale datasets. DeepMRD^TR comprises four networks (two CT-Nets and two CXR-Nets) that exploit contrastive learning to generate pre-training parameters that are fine-tuned on the retrospective dataset collected from a single institution. The performance of DeepMRD^TR was evaluated for abnormality identification and disease diagnosis on data from two different institutions: one was an internal testing dataset from the same institution as the training data and the second was collected from an external institution to evaluate the model generalizability and robustness to an unrelated population dataset. In such a difficult multi-class diagnosis task, our system achieved the average area under the receiver operating characteristic curve (AUC) of 0.856 (95% confidence interval (CI):0.843–0.868) and 0.841 (95%CI:0.832–0.887) for abnormality identification, and 0.900 (95%CI:0.872–0.958) and 0.866 (95%CI:0.832–0.887) for major respiratory diseases’ diagnosis on CT and CXR datasets, respectively. Furthermore, to achieve a clinically actionable diagnosis, we deployed a preliminary version of DeepMRD^TR into the clinical workflow, which was performed on par with senior experts in disease diagnosis, with an AUC of 0.890 and a Cohen’s k of 0.746–0.877 at a reasonable timescale; these findings demonstrate the potential to accelerate the medical workflow to facilitate early diagnosis as a triage tool for respiratory diseases which supports improved clinical diagnoses and decision-making.

Glioma survival prediction from whole-brain MRI without tumor segmentation using deep attention network: a multicenter study

OBJECTIVES

To develop and validate a deep learning model for predicting overall survival from whole-brain MRI without tumor segmentation in patients with diffuse gliomas.

METHODS

In this multicenter retrospective study, two deep learning models were built for survival prediction from MRI, including a DeepRisk model built from whole-brain MRI, and an original ResNet model built from expert-segmented tumor images. Both models were developed using a training dataset (n = 935) and an internal tuning dataset (n = 156) and tested on two external test datasets (n = 194 and 150) and a TCIA dataset (n = 121). C-index, integrated Brier score (IBS), prediction error curves, and calibration curves were used to assess the model performance.

RESULTS

In total, 1556 patients were enrolled (age, 49.0 ± 13.1 years; 830 male). The DeepRisk score was an independent predictor and can stratify patients in each test dataset into three risk subgroups. The IBS and C-index for DeepRisk were 0.14 and 0.83 in external test dataset 1, 0.15 and 0.80 in external dataset 2, and 0.16 and 0.77 in TCIA dataset, respectively, which were comparable with those for original ResNet. The AUCs at 6, 12, 24, 26, and 48 months for DeepRisk ranged between 0.77 and 0.94. Combining DeepRisk score with clinicomolecular factors resulted in a nomogram with a better calibration and classification accuracy (net reclassification improvement 0.69, p < 0.001) than the clinical nomogram.

CONCLUSIONS

DeepRisk that obviated the need of tumor segmentation can predict glioma survival from whole-brain MRI and offers incremental prognostic value.

KEY POINTS

• DeepRisk can predict overall survival directly from whole-brain MRI without tumor segmentation. • DeepRisk achieves comparable accuracy in survival prediction with deep learning model built using expert-segmented tumor images. • DeepRisk has independent and incremental prognostic value over existing clinical parameters and IDH mutation status.

Explainable emphysema detection on chest radiographs with deep learning

We propose a deep learning system to automatically detect four explainable emphysema signs on frontal and lateral chest radiographs. Frontal and lateral chest radiographs from 3000 studies were retrospectively collected. Two radiologists annotated these with 4 radiological signs of pulmonary emphysema identified from the literature. A patient with ≥2 of these signs present is considered emphysema positive. Using separate deep learning systems for frontal and lateral images we predict the presence of each of the four visual signs and use these to determine emphysema positivity. The ROC and AUC results on a set of 422 held-out cases, labeled by both radiologists, are reported. Comparison with a black-box model which predicts emphysema without the use of explainable visual features is made on the annotations from both radiologists, as well as the subset that they agreed on. DeLong’s test is used to compare with the black-box model ROC and McNemar’s test to compare with radiologist performance. In 422 test cases, emphysema positivity was predicted with AUCs of 0.924 and 0.946 using the reference standard from each radiologist separately. Setting model sensitivity equivalent to that of the second radiologist, our model has a comparable specificity (p = 0.880 and p = 0.143 for each radiologist respectively). Our method is comparable with the black-box model with AUCs of 0.915 (p = 0.407) and 0.935 (p = 0.291), respectively. On the 370 cases where both radiologists agreed (53 positives), our model achieves an AUC of 0.981, again comparable to the black-box model AUC of 0.972 (p = 0.289). Our proposed method can predict emphysema positivity on chest radiographs as well as a radiologist or a comparable black-box method. It additionally produces labels for four visual signs to ensure the explainability of the result. The dataset is publicly available at https://doi.org/10.5281/zenodo.6373392.

Disease Localization and Severity Assessment in Chest X-Ray Images using Multi-Stage Superpixels Classification

BACKGROUND AND OBJECTIVES

Chest X-ray (CXR) is a non-invasive imaging modality used in the prognosis and management of chronic lung disorders like tuberculosis (TB), pneumonia, coronavirus disease (COVID-19), etc. The radiomic features associated with different disease manifestations assist in detection, localization, and grading the severity of infected lung regions. The majority of the existing computer-aided diagnosis (CAD) system used these features for the classification task, and only a few works have been dedicated to disease-localization and severity scoring. Moreover, the existing deep learning approaches use class activation map and Saliency map, which generate a rough localization. This study aims to generate a compact disease boundary, infection map, and grade the infection severity using proposed multistage superpixel classification-based disease localization and severity assessment framework.

METHODS

The proposed method uses a simple linear iterative clustering (SLIC) technique to subdivide the lung field into small superpixels. Initially, the different radiomic texture and proposed shape features are extracted and combined to train different benchmark classifiers in a multistage framework. Subsequently, the predicted class labels are used to generate an infection map, mark disease boundary, and grade the infection severity. The performance is evaluated using a publicly available Montgomery dataset and validated using Friedman average ranking and Holm and Nemenyi post-hoc procedures.

RESULTS

The proposed multistage classification approach achieved accuracy (ACC)= 95.52%, F-Measure (FM)= 95.48%, area under the curve (AUC)= 0.955 for Stage-I and ACC=85.35%, FM=85.20%, AUC=0.853 for Stage-II using calibration dataset and ACC = 93.41%, FM = 95.32%, AUC = 0.936 for Stage-I and ACC = 84.02%, FM = 71.01%, AUC = 0.795 for Stage-II using validation dataset. Also, the model has demonstrated the average Jaccard Index (JI) of 0.82 and Pearson's correlation coefficient (r) of 0.9589.

CONCLUSIONS

The obtained classification results using calibration and validation dataset confirms the promising performance of the proposed framework. Also, the average JI shows promising potential to localize the disease, and better agreement between radiologist score and predicted severity score (r) confirms the robustness of the method. Finally, the statistical test justified the significance of the obtained results.

COVID-19 prognosis using limited chest X-ray images

The COrona VIrus Disease 2019 (COVID-19) pandemic is an ongoing global pandemic that has claimed millions of lives till date. Detecting COVID-19 and isolating affected patients at an early stage is crucial to contain its rapid spread. Although accurate, the primary viral test 'Reverse Transcription Polymerase Chain Reaction' (RT-PCR) for COVID-19 diagnosis has an elaborate test kit, and the turnaround time is high. This has motivated the research community to develop CXR based automated COVID-19 diagnostic methodologies. However, COVID-19 being a novel disease, there is no annotated large-scale CXR dataset for this particular disease. To address the issue of limited data, we propose to exploit a large-scale CXR dataset collected in the pre-COVID era and train a deep neural network in a self-supervised fashion to extract CXR specific features. Further, we compute attention maps between the global and the local features of the backbone convolutional network while finetuning using a limited COVID-19 CXR dataset. We empirically demonstrate the effectiveness of the proposed method. We provide a thorough ablation study to understand the effect of each proposed component. Finally, we provide visualizations highlighting the critical patches instrumental to the predictive decision made by our model. These saliency maps are not only a stepping stone towards explainable AI but also aids radiologists in localizing the infected area.

Automatic recognition of micronucleus by combining attention mechanism and AlexNet

Background

Micronucleus (MN) is an abnormal fragment in a human cell caused by disorders in the mechanism regulating chromosome segregation. It can be used as a biomarker for genotoxicity, tumor risk, and tumor malignancy. The in vitro micronucleus assay is a commonly used method to detect micronucleus. However, it is time-consuming and the visual scoring can be inconsistent.

Methods

To alleviate this issue, we proposed a computer-aided diagnosis method combining convolutional neural networks and visual attention for micronucleus recognition. The backbone of our model is AlexNet without any dense layers and it is pretrained on the ImageNet dataset. Two attention modules are applied to extract cell image features and generate attention maps highlighting the region of interest to improve the interpretability of the network. Given the problems in the data set, we leverage data augmentation and focal loss to alleviate the impact.

Results

Experiments show that the proposed network yields better performance with fewer parameters. The AP value, F1 value and AUC value reach 0.932, 0.811 and 0.995, respectively.

Conclusion

In conclusion, the proposed network can effectively recognize micronucleus, and it can play an auxiliary role in clinical diagnosis by doctors.

A Lightweight Attention-Based Network towards Distracted Driving Behavior Recognition

Distracted driving is currently a global issue causing fatal traffic crashes and injuries. Although deep learning has achieved significant success in various fields, it still faces the trade-off between computation cost and overall accuracy in the field of distracted driving behavior recognition. This paper addresses this problem and proposes a novel lightweight attention-based (LWANet) network for image classification tasks. To reduce the computation cost and trainable parameters, we replace standard convolution layers with depthwise separable convolutions and optimize the classic VGG16 architecture by 98.16% trainable parameters reduction. Inspired by the attention mechanism in cognitive science, a lightweight inverted residual attention module (IRAM) is proposed to simulate human attention, extract more specific features, and improve the overall accuracy. LWANet achieved an accuracy of 99.37% on Statefarm’s dataset and 98.45% on American University in Cairo’s dataset. With only 1.22 M trainable parameters and a model file size of 4.68 MB, the quantitative experimental results demonstrate that the proposed LWANet obtains state-of-the-art overall performance in deep learning-based distracted driving behavior recognition.

Two-stage segmentation network with feature aggregation and multi-level attention mechanism for multi-modality heart images

Accurate segmentation of cardiac substructures in multi-modality heart images is an important prerequisite for the diagnosis and treatment of cardiovascular diseases. However, the segmentation of cardiac images remains a challenging task due to (1) the interference of multiple targets, (2) the imbalance of sample size. Therefore, in this paper, we propose a novel two-stage segmentation network with feature aggregation and multi-level attention mechanism (TSFM-Net) to comprehensively solve these challenges. Firstly, in order to improve the effectiveness of multi-target features, we adopt the encoder-decoder structure as the backbone segmentation framework and design a feature aggregation module (FAM) to realize the multi-level feature representation (Stage1). Secondly, because the segmentation results obtained from Stage1 are limited to the decoding of single scale feature maps, we design a multi-level attention mechanism (MLAM) to assign more attention to the multiple targets, so as to get multi-level attention maps. We fuse these attention maps and concatenate the output of Stage1 to carry out the second segmentation to get the final segmentation result (Stage2). The proposed method has better segmentation performance and balance on 2017 MM-WHS multi-modality whole heart images than the state-of-the-art methods, which demonstrates the feasibility of TSFM-Net for accurate segmentation of heart images.

Efficient Framework for Detecting COVID-19 and Pneumonia from Chest X-Ray Using Deep Convolutional Network

Recently, the COVID-19 pandemic is considered the most severe infectious disease because of its rapid spreading. Radiologists still lack sufficient knowledge and experience for accurate and fast detecting COVID-19. What exacerbates things is the significant overlap between Pneumonia symptoms and COVID-19, which confuses the radiologists. It’s widely agreed that the early detection of the infected patient increases his likelihood of recovery. Chest X-ray images are considered the cheapest radiology images, and their devices are available widely. This study introduces an effective Deep Convolutional Neural Network (DCNN) called “DeepChest” for fast and accurate detection for both COVID-19 and Pneumonia in chest X-ray images. “DeepChest” runs with a small number of convolutional layers, a small number of max-pooling layers, and a small number of training iterations compared with the recent approaches and the state-of-the-art of DCNN. We conducted the experimental evaluations of the proposed approach on a data set with 7512 chest X-ray images. The proposed approach achieves an accuracy of 96.56% overall, 99.40% in detecting COVID-19, and 99.32% in detecting Pneumonia. In actual practice, the presented approach can be used as a computer-aided diagnosis tool to get accurate results in detecting Pneumonia and COVID-19 in chest X-ray images.

Semantic consistency generative adversarial network for cross-modality domain adaptation in ultrasound thyroid nodule classification

Deep convolutional networks have been widely used for various medical image processing tasks. However, the performance of existing learning-based networks is still limited due to the lack of large training datasets. When a general deep model is directly deployed to a new dataset with heterogeneous features, the effect of domain shifts is usually ignored, and performance degradation problems occur. In this work, by designing the semantic consistency generative adversarial network (SCGAN), we propose a new multimodal domain adaptation method for medical image diagnosis. SCGAN performs cross-domain collaborative alignment of ultrasound images and domain knowledge. Specifically, we utilize a self-attention mechanism for adversarial learning between dual domains to overcome visual differences across modal data and preserve the domain invariance of the extracted semantic features. In particular, we embed nested metric learning in the semantic information space, thus enhancing the semantic consistency of cross-modal features. Furthermore, the adversarial learning of our network is guided by a discrepancy loss for encouraging the learning of semantic-level content and a regularization term for enhancing network generalization. We evaluate our method on a thyroid ultrasound image dataset for benign and malignant diagnosis of nodules. The experimental results of a comprehensive study show that the accuracy of the SCGAN method for the classification of thyroid nodules reaches 94.30%, and the AUC reaches 97.02%. These results are significantly better than the state-of-the-art methods.

Automatic detection of multiple types of pneumonia: Open dataset and a multi-scale attention network

The quick and precise identification of COVID-19 pneumonia, non-COVID-19 viral pneumonia, bacterial pneumonia, mycoplasma pneumonia, and normal lung on chest CT images play a crucial role in timely quarantine and medical treatment. However, manual identification is subject to potential misinterpretations and time-consumption issues owing the visual similarities of pneumonia lesions. In this study, we propose a novel multi-scale attention network (MSANet) based on a bag of advanced deep learning techniques for the automatic classification of COVID-19 and multiple types of pneumonia. The proposed method can automatically pay attention to discriminative information and multi-scale features of pneumonia lesions for better classification. The experimental results show that the proposed MSANet can achieve an overall precision of 97.31%, recall of 96.18%, F1-score of 96.71%, accuracy of 97.46%, and macro-average area under the receiver operating characteristic curve (AUC) of 0.9981 to distinguish between multiple classes of pneumonia. These promising results indicate that the proposed method can significantly assist physicians and radiologists in medical diagnosis. The dataset is publicly available at https://doi.org/10.17632/rf8x3wp6ss.1.

Gender and age detection assist convolutional neural networks in classification of thorax diseases

Conventionally, convolutional neural networks (CNNs) have been used to identify and detect thorax diseases on chest x-ray images. To identify thorax diseases, CNNs typically learn two types of information: disease-specific features and generic anatomical features. CNNs focus on disease-specific features while ignoring the rest of the anatomical features during their operation. There is no evidence that generic anatomical features improve or worsen the performance of convolutional neural networks for thorax disease classification in the current research. As a consequence, the relevance of general anatomical features in boosting the performance of CNNs for thorax disease classification is investigated in this study. We employ a dual-stream CNN model to learn anatomical features before training the model for thorax disease classification. The dual-stream technique is used to compel the model to learn structural information because initial layers of CNNs often learn features of edges and boundaries. As a result, a dual-stream model with minimal layers learns structural and anatomical features as a priority. To make the technique more comprehensive, we first train the model to identify gender and age and then classify thorax diseases using the information acquired. Only when the model learns the anatomical features can it detect gender and age. We also use Non-negative Matrix Factorization (NMF) and Contrast Limited Adaptive Histogram Equalization (CLAHE) to pre-process the training data, which suppresses disease-related information while amplifying general anatomical features, allowing the model to acquire anatomical features considerably faster. Finally, the model that was earlier trained for gender and age detection is retrained for thorax disease classification using original data. The proposed technique increases the performance of convolutional neural networks for thorax disease classification, as per experiments on the Chest X-ray14 dataset. We can also see the significant parts of the image that contribute more for gender, age, and a certain thorax disease by visualizing the features. The proposed study achieves two goals: first, it produces novel gender and age identification results on chest X-ray images that may be used in biometrics, forensics, and anthropology, and second, it highlights the importance of general anatomical features in thorax disease classification. In comparison to state-of-the-art results, the proposed work also produces competitive results.

Factors determining generalization in deep learning models for scoring COVID-CT images

The COVID-19 pandemic has inspired unprecedented data collection and computer vision modelling efforts worldwide, focused on the diagnosis of COVID-19 from medical images. However, these models have found limited, if any, clinical application due in part to unproven generalization to data sets beyond their source training corpus. This study investigates the generalizability of deep learning models using publicly available COVID-19 Computed Tomography data through cross dataset validation. The predictive ability of these models for COVID-19 severity is assessed using an independent dataset that is stratified for COVID-19 lung involvement. Each inter-dataset study is performed using histogram equalization, and contrast limited adaptive histogram equalization with and without a learning Gabor filter. We show that under certain conditions, deep learning models can generalize well to an external dataset with F1 scores up to 86%. The best performing model shows predictive accuracy of between 75% and 96% for lung involvement scoring against an external expertly stratified dataset. From these results we identify key factors promoting deep learning generalization, being primarily the uniform acquisition of training images, and secondly diversity in CT slice position.

Clinical Artificial Intelligence Applications in Radiology: Chest and Abdomen

Organ segmentation, chest radiograph classification, and lung and liver nodule detections are some of the popular artificial intelligence (AI) tasks in chest and abdominal radiology due to the wide availability of public datasets. AI algorithms have achieved performance comparable to humans in less time for several organ segmentation tasks, and some lesion detection and classification tasks. This article introduces the current published articles of AI applied to chest and abdominal radiology, including organ segmentation, lesion detection, classification, and predicting prognosis.

Deep Mining Generation of Lung Cancer Malignancy Models from Chest X-ray Images

Lung cancer is the leading cause of cancer death and morbidity worldwide. Many studies have shown machine learning models to be effective in detecting lung nodules from chest X-ray images. However, these techniques have yet to be embraced by the medical community due to several practical, ethical, and regulatory constraints stemming from the “black-box” nature of deep learning models. Additionally, most lung nodules visible on chest X-rays are benign; therefore, the narrow task of computer vision-based lung nodule detection cannot be equated to automated lung cancer detection. Addressing both concerns, this study introduces a novel hybrid deep learning and decision tree-based computer vision model, which presents lung cancer malignancy predictions as interpretable decision trees. The deep learning component of this process is trained using a large publicly available dataset on pathological biomarkers associated with lung cancer. These models are then used to inference biomarker scores for chest X-ray images from two independent data sets, for which malignancy metadata is available. Next, multi-variate predictive models were mined by fitting shallow decision trees to the malignancy stratified datasets and interrogating a range of metrics to determine the best model. The best decision tree model achieved sensitivity and specificity of 86.7% and 80.0%, respectively, with a positive predictive value of 92.9%. Decision trees mined using this method may be considered as a starting point for refinement into clinically useful multi-variate lung cancer malignancy models for implementation as a workflow augmentation tool to improve the efficiency of human radiologists.

Deep learning for chest X-ray analysis: A survey

Recent advances in deep learning have led to a promising performance in many medical image analysis tasks. As the most commonly performed radiological exam, chest radiographs are a particularly important modality for which a variety of applications have been researched. The release of multiple, large, publicly available chest X-ray datasets in recent years has encouraged research interest and boosted the number of publications. In this paper, we review all studies using deep learning on chest radiographs published before March 2021, categorizing works by task: image-level prediction (classification and regression), segmentation, localization, image generation and domain adaptation. Detailed descriptions of all publicly available datasets are included and commercial systems in the field are described. A comprehensive discussion of the current state of the art is provided, including caveats on the use of public datasets, the requirements of clinically useful systems and gaps in the current literature.