Dense GAN and multi-layer attention based lesion segmentation method for COVID-19 CT images

Hi-gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolutionpGAN

Objective.Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive.Approach.This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning inpGANs, it further enhances the robustness and accuracy of the segmentation map.Main results.Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small.Significance.The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.

A novel denoising method for low-dose CT images based on transformer and CNN

Computed Tomography (CT) has become a mainstream imaging tool in medical diagnosis. However, the issue of increased cancer risk due to radiation exposure has raised public concern. Low-dose computed tomography (LDCT) technique is a CT scan with lower radiation dose than conventional scans. LDCT is used to make a diagnosis of lesions with the smallest dose of x-rays, and is currently mainly used for early lung cancer screening. However, LDCT has severe image noise, and these noises affect adversely the quality of medical images and thus the diagnosis of lesions. In this paper, we propose a novel LDCT image denoising method based on transformer combined with convolutional neural network (CNN). The encoder part of the network is based on CNN, which is mainly used to extract the image detail information. In the decoder part, we propose a dual-path transformer block (DPTB), which extracts the features of input of the skip connection and the features of input of the previous level through two paths respectively. DPTB can better restore the detail and structure information of the denoised image. In order to pay more attention to the key regions of the feature images extracted at the shallow level of the network, we also propose a multi-feature spatial attention block (MSAB) in the skip connection part. Experimental studies are conducted, and comparisons with the state-of-the-art networks are made, and the results demonstrate that the developed method can effectively remove the noise in CT images and improve the image quality in the evaluation metrics of peak signal to noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) and is superior to the state-of-the-art models. Our method achieved 28.9720 of PSNR, 0.8595 of SSIM and 14.8657 of RMSE on the Mayo Clinic LDCT Grand Challenge dataset. For different noise level σ (15, 35, and 55) on the QIN_LUNG_CT dataset, our proposed also achieved better performances.

A Review Paper about Deep Learning for Medical Image Analysis

Medical imaging refers to the process of obtaining images of internal organs for therapeutic purposes such as discovering or studying diseases. The primary objective of medical image analysis is to improve the efficacy of clinical research and treatment options. Deep learning has revamped medical image analysis, yielding excellent results in image processing tasks such as registration, segmentation, feature extraction, and classification. The prime motivations for this are the availability of computational resources and the resurgence of deep convolutional neural networks. Deep learning techniques are good at observing hidden patterns in images and supporting clinicians in achieving diagnostic perfection. It has proven to be the most effective method for organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Many deep learning approaches have been published to analyze medical images for various diagnostic purposes. In this paper, we review the work exploiting current state-of-the-art deep learning approaches in medical image processing. We begin the survey by providing a synopsis of research works in medical imaging based on convolutional neural networks. Second, we discuss popular pretrained models and general adversarial networks that aid in improving convolutional networks' performance. Finally, to ease direct evaluation, we compile the performance metrics of deep learning models focusing on COVID-19 detection and child bone age prediction.

COVID-19 CT ground-glass opacity segmentation based on attention mechanism threshold

The ground glass opacity (GGO) of the lung is one of the essential features of COVID-19. The GGO in computed tomography (CT) images has various features and low-intensity contrast between the GGO and edge structures. These problems pose significant challenges for segmenting the GGO. To tackle these problems, we propose a new threshold method for accurate segmentation of GGO. Specifically, we offer a framework for adjusting the threshold parameters according to the image contrast. Three functions include Attention mechanism threshold, Contour equalization, and Lung segmentation (ACL). The lung is divided into three areas using the attention mechanism threshold. Further, the segmentation parameters of the attention mechanism thresholds of the three parts are adaptively adjusted according to the image contrast. Only the segmentation regions restricted by the lung segmentation results are retained. Extensive experiments on four COVID datasets show that ACL can segment GGO images at low contrast well. Compared with the state-of-the-art methods, the similarity Dice of the ACL segmentation results is improved by 8.9%, the average symmetry surface distance ASD is reduced by 23%, and the required computational power F L O P s are only 0.09% of those of deep learning models. For GGO segmentation, ACL is more lightweight, and the accuracy is higher. Code will be released at https://github.com/Lqs-github/ACL.

A survey on deep learning applied to medical images: from simple artificial neural networks to generative models

Deep learning techniques, in particular generative models, have taken on great importance in medical image analysis. This paper surveys fundamental deep learning concepts related to medical image generation. It provides concise overviews of studies which use some of the latest state-of-the-art models from last years applied to medical images of different injured body areas or organs that have a disease associated with (e.g., brain tumor and COVID-19 lungs pneumonia). The motivation for this study is to offer a comprehensive overview of artificial neural networks (NNs) and deep generative models in medical imaging, so more groups and authors that are not familiar with deep learning take into consideration its use in medicine works. We review the use of generative models, such as generative adversarial networks and variational autoencoders, as techniques to achieve semantic segmentation, data augmentation, and better classification algorithms, among other purposes. In addition, a collection of widely used public medical datasets containing magnetic resonance (MR) images, computed tomography (CT) scans, and common pictures is presented. Finally, we feature a summary of the current state of generative models in medical image including key features, current challenges, and future research paths.

COVID-19 ground-glass opacity segmentation based on fuzzy c-means clustering and improved random walk algorithm

Accurate segmentation of ground-glass opacity (GGO) is an important premise for doctors to judge COVID-19. Aiming at the problem of mis-segmentation for GGO segmentation methods, especially the problem of adhesive GGO connected with chest wall or blood vessel, this paper proposes an accurate segmentation of GGO based on fuzzy c-means (FCM) clustering and improved random walk algorithm. The innovation of this paper is to construct a Markov random field (MRF) with adaptive spatial information by using the spatial gravity Model and the spatial structural characteristics, which is introduced into the FCM model to automatically balance the insensitivity to noise and preserve the effectiveness of image edge details to improve the clustering accuracy of image. Then, the coordinate values of nodes and seed points in the image are combined with the spatial distance, and the geodesic distance is added to redefine the weight. According to the edge density of the image, the weight of the grayscale and the spatial feature in the weight function is adaptively calculated. In order to reduce the influence of edge noise on GGO segmentation, an adaptive snowfall model is proposed to preprocess the image, which can suppress the noise without losing the edge information. In this paper, CT images of different types of COVID-19 are selected for segmentation experiments, and the experimental results are compared with the traditional segmentation methods and several SOTA methods. The results suggest that the paper method can be used for the auxiliary diagnosis of COVID-19, so as to improve the work efficiency of doctors.

A fast lightweight network for the discrimination of COVID-19 and pulmonary diseases

With the outbreak of COVID-19 and the increasing number of infections worldwide, there has been a noticeable deficiency in healthcare provided by medical professionals. To cope with this situation, computational methods can be used in different steps of COVID-19 handling. The first step is to accurately and rapidly diagnose infected persons, because the time taken for the diagnosis is among the crucial factors to save human lives. This paper proposes a computationally fast network for the diagnosis of COVID-19 and pulmonary diseases, which can be used in telemedicine. The proposed network is called DLNet because it jointly encodes local binary patterns along with filter outputs of discrete cosine transform (DCT). The first layer in DLNet is the convolution layer in which the input image is convolved using DCT filters. Then, to avoid over-fitting, a binary hashing procedure is performed by fusing responses of different filters into a unique feature map. This map is used to generate block-wise histograms by binding local binary codes of the input image and the map values. We normalize these histograms to improve the robustness of the network against illumination changes. Experiments conducted on a public dataset demonstrate the rapidity and effectiveness of DLNet, where an average accuracy, sensitivity, and specificity of 98.86%, 98.06, and 99.24% have been achieved, respectively. Moreover, the proposed network has shown high tolerance to the missing parts in the medical image, which makes it suitable for the telemedicine scenario.

How much BiGAN and CycleGAN-learned hidden features are effective for COVID-19 detection from CT images? A comparative study

Bidirectional generative adversarial networks (BiGANs) and cycle generative adversarial networks (CycleGANs) are two emerging machine learning models that, up to now, have been used as generative models, i.e., to generate output data sampled from a target probability distribution. However, these models are also equipped with encoding modules, which, after weakly supervised training, could be, in principle, exploited for the extraction of hidden features from the input data. At the present time, how these extracted features could be effectively exploited for classification tasks is still an unexplored field. Hence, motivated by this consideration, in this paper, we develop and numerically test the performance of a novel inference engine that relies on the exploitation of BiGAN and CycleGAN-learned hidden features for the detection of COVID-19 disease from other lung diseases in computer tomography (CT) scans. In this respect, the main contributions of the paper are twofold. First, we develop a kernel density estimation (KDE)-based inference method, which, in the training phase, leverages the hidden features extracted by BiGANs and CycleGANs for estimating the (a priori unknown) probability density function (PDF) of the CT scans of COVID-19 patients and, then, in the inference phase, uses it as a target COVID-PDF for the detection of COVID diseases. As a second major contribution, we numerically evaluate and compare the classification accuracies of the implemented BiGAN and CycleGAN models against the ones of some state-of-the-art methods, which rely on the unsupervised training of convolutional autoencoders (CAEs) for attaining feature extraction. The performance comparisons are carried out by considering a spectrum of different training loss functions and distance metrics. The obtained classification accuracies of the proposed CycleGAN-based (resp., BiGAN-based) models outperform the corresponding ones of the considered benchmark CAE-based models of about 16% (resp., 14%).

SRV-GAN: A generative adversarial network for segmenting retinal vessels

In the field of ophthalmology, retinal diseases are often accompanied by complications, and effective segmentation of retinal blood vessels is an important condition for judging retinal diseases. Therefore, this paper proposes a segmentation model for retinal blood vessel segmentation. Generative adversarial networks (GANs) have been used for image semantic segmentation and show good performance. So, this paper proposes an improved GAN. Based on R2U-Net, the generator adds an attention mechanism, channel and spatial attention, which can reduce the loss of information and extract more effective features. We use dense connection modules in the discriminator. The dense connection module has the characteristics of alleviating gradient disappearance and realizing feature reuse. After a certain amount of iterative training, the generated prediction map and label map can be distinguished. Based on the loss function in the traditional GAN, we introduce the mean squared error. By using this loss, we ensure that the synthetic images contain more realistic blood vessel structures. The values of area under the curve (AUC) in the retinal blood vessel pixel segmentation of the three public data sets DRIVE, CHASE-DB1 and STARE of the proposed method are 0.9869, 0.9894 and 0.9885, respectively. The indicators of this experiment have improved compared to previous methods.

Combating COVID-19 Using Generative Adversarial Networks and Artificial Intelligence for Medical Images: Scoping Review

Background

Research on the diagnosis of COVID-19 using lung images is limited by the scarcity of imaging data. Generative adversarial networks (GANs) are popular for synthesis and data augmentation. GANs have been explored for data augmentation to enhance the performance of artificial intelligence (AI) methods for the diagnosis of COVID-19 within lung computed tomography (CT) and X-ray images. However, the role of GANs in overcoming data scarcity for COVID-19 is not well understood.

Objective

This review presents a comprehensive study on the role of GANs in addressing the challenges related to COVID-19 data scarcity and diagnosis. It is the first review that summarizes different GAN methods and lung imaging data sets for COVID-19. It attempts to answer the questions related to applications of GANs, popular GAN architectures, frequently used image modalities, and the availability of source code.

Methods

A search was conducted on 5 databases, namely PubMed, IEEEXplore, Association for Computing Machinery (ACM) Digital Library, Scopus, and Google Scholar. The search was conducted from October 11-13, 2021. The search was conducted using intervention keywords, such as “generative adversarial networks” and “GANs,” and application keywords, such as “COVID-19” and “coronavirus.” The review was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) guidelines for systematic and scoping reviews. Only those studies were included that reported GAN-based methods for analyzing chest X-ray images, chest CT images, and chest ultrasound images. Any studies that used deep learning methods but did not use GANs were excluded. No restrictions were imposed on the country of publication, study design, or outcomes. Only those studies that were in English and were published from 2020 to 2022 were included. No studies before 2020 were included.

Results

This review included 57 full-text studies that reported the use of GANs for different applications in COVID-19 lung imaging data. Most of the studies (n=42, 74%) used GANs for data augmentation to enhance the performance of AI techniques for COVID-19 diagnosis. Other popular applications of GANs were segmentation of lungs and superresolution of lung images. The cycleGAN and the conditional GAN were the most commonly used architectures, used in 9 studies each. In addition, 29 (51%) studies used chest X-ray images, while 21 (37%) studies used CT images for the training of GANs. For the majority of the studies (n=47, 82%), the experiments were conducted and results were reported using publicly available data. A secondary evaluation of the results by radiologists/clinicians was reported by only 2 (4%) studies.

Conclusions

Studies have shown that GANs have great potential to address the data scarcity challenge for lung images in COVID-19. Data synthesized with GANs have been helpful to improve the training of the convolutional neural network (CNN) models trained for the diagnosis of COVID-19. In addition, GANs have also contributed to enhancing the CNNs’ performance through the superresolution of the images and segmentation. This review also identified key limitations of the potential transformation of GAN-based methods in clinical applications.

A transfer learning based deep learning model to diagnose covid-19 CT scan images

To save the life of human beings during the pandemic conditions we need an effective automated method to deal with this situation. In pandemic conditions when the available resources becomes insufficient to handle the patient's load, then we needed some fast and reliable method which analyse the patient medical data with high efficiency and accuracy within time limitations. In this manuscript, an effective and efficient method is proposed for exact diagnosis of the patient whether it is coronavirus disease-2019 (covid-19) positive or negative with the help of deep learning. To find the correct diagnosis with high accuracy we use pre-processed segmented images for the analysis with deep learning. In the first step the X-ray image or computed tomography (CT) of a covid-19 infected person is analysed with various schemes of image segmentation like simple thresholding at 0.3, simple thresholding at 0.6, multiple thresholding (between 26-230) and Otsu's algorithm. On comparative analysis of all these methods, it is found that the Otsu's algorithm is a simple and optimum scheme to improve the segmented outcome of binary image for the diagnosis point of view. Otsu's segmentation scheme gives more precise values in comparison to other methods on the scale of various image quality parameters like accuracy, sensitivity, f-measure, precision, and specificity. For image classification here we use Resnet-50, MobileNet and VGG-16 models of deep learning which gives accuracy 70.24%, 72.95% and 83.18% respectively with non-segmented CT scan images and 75.08%, 80.12% and 99.28% respectively with Otsu's segmented CT scan images. On a comparative study we find that the VGG-16 models with CT scan image segmented with Otsu's segmentation gives very high accuracy of 99.28%. On the basis of the diagnosis of the patient firstly we go for an arterial blood gas (ABG) analysis and then on the behalf of this diagnosis and ABG report, the severity level of the patient can be decided and according to this severity level, proper treatment protocols can be followed immediately to save the patient's life. Compared with the existing works, our deep learning based novel method reduces the complexity, takes much less time and has a greater accuracy for exact diagnosis of coronavirus disease-2019 (covid-19).

Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN)

Deep learning has gained immense attention from researchers in medicine, especially in medical imaging. The main bottleneck is the unavailability of sufficiently large medical datasets required for the good performance of deep learning models. This paper proposes a new framework consisting of one variational autoencoder (VAE), two generative adversarial networks, and one auxiliary classifier to artificially generate realistic-looking skin lesion images and improve classification performance. We first train the encoder-decoder network to obtain the latent noise vector with the image manifold’s information and let the generative adversarial network sample the input from this informative noise vector in order to generate the skin lesion images. The use of informative noise allows the GAN to avoid mode collapse and creates faster convergence. To improve the diversity in the generated images, we use another GAN with an auxiliary classifier, which samples the noise vector from a heavy-tailed student t-distribution instead of a random noise Gaussian distribution. The proposed framework was named TED-GAN, with T from the t-distribution and ED from the encoder-decoder network which is part of the solution. The proposed framework could be used in a broad range of areas in medical imaging. We used it here to generate skin lesion images and have obtained an improved classification performance on the skin lesion classification task, rising from 66% average accuracy to 92.5%. The results show that TED-GAN has a better impact on the classification task because of its diverse range of generated images due to the use of a heavy-tailed t-distribution.