Attribute-guided image generation of three-dimensional computed tomography images of lung nodules using a generative adversarial network

A comparative analysis of different augmentations for brain images

Deep learning (DL) requires a large amount of training data to improve performance and prevent overfitting. To overcome these difficulties, we need to increase the size of the training dataset. This can be done by augmentation on a small dataset. The augmentation approaches must enhance the model's performance during the learning period. There are several types of transformations that can be applied to medical images. These transformations can be applied to the entire dataset or to a subset of the data, depending on the desired outcome. In this study, we categorize data augmentation methods into four groups: Absent augmentation, where no modifications are made; basic augmentation, which includes brightness and contrast adjustments; intermediate augmentation, encompassing a wider array of transformations like rotation, flipping, and shifting in addition to brightness and contrast adjustments; and advanced augmentation, where all transformation layers are employed. We plan to conduct a comprehensive analysis to determine which group performs best when applied to brain CT images. This evaluation aims to identify the augmentation group that produces the most favorable results in terms of improving model accuracy, minimizing diagnostic errors, and ensuring the robustness of the model in the context of brain CT image analysis.

Intelligent Stroke Disease Prediction Model Using Deep Learning Approaches

Stroke is a high morbidity and mortality disease that poses a serious threat to people's health. Early recognition of the various warning signs of stroke is necessary so that timely clinical intervention can help reduce the severity of stroke. Deep neural networks have powerful feature representation capabilities and can automatically learn discriminant features from large amounts of data. This paper uses a range of physiological characteristic parameters and collaborates with deep neural networks, such as the Wasserstein generative adversarial networks with gradient penalty and regression network, to construct a stroke prediction model. Firstly, to address the problem of imbalance between positive and negative samples in the stroke public data set, we performed positive sample data augmentation and utilized WGAN-GP to generate stroke data with high fidelity and used it for the training of the prediction network model. Then, the relationship between observable physiological characteristic parameters and the predicted risk of suffering a stroke was modeled as a nonlinear mapping transformation, and a stroke prediction model based on a deep regression network was designed. Finally, the proposed method is compared with commonly used machine learning-based classification algorithms such as decision tree, random forest, support vector machine, and artificial neural networks. The prediction results of the proposed method are optimal in the comprehensive measurement index F. Further ablation experiments also show that the designed prediction model has certain robustness and can effectively predict stroke diseases.

ADGAN: Attribute-Driven Generative Adversarial Network for Synthesis and Multiclass Classification of Pulmonary Nodules

Lung cancer is the leading cause of cancer-related deaths worldwide. According to the American Cancer Society, early diagnosis of pulmonary nodules in computed tomography (CT) scans can improve the five-year survival rate up to 70% with proper treatment planning. In this article, we propose an attribute-driven Generative Adversarial Network (ADGAN) for synthesis and multiclass classification of Pulmonary Nodules. A self-attention U-Net (SaUN) architecture is proposed to improve the generation mechanism of the network. The generator is designed with two modules, namely, self-attention attribute module (SaAM) and a self-attention spatial module (SaSM). SaAM generates a nodule image based on given attributes whereas SaSM specifies the nodule region of the input image to be altered. A reconstruction loss along with an attention localization loss (AL) is used to produce an attention map prioritizing the nodule regions. To avoid resemblance between a generated image and a real image, we further introduce an adversarial loss containing a regularization term based on KL divergence. The discriminator part of the proposed model is designed to achieve the multiclass nodule classification task. Our proposed approach is validated over two challenging publicly available datasets, namely LIDC-IDRI and LUNGX. Exhaustive experimentation on these two datasets clearly indicate that we have achieved promising classification accuracy as compared to other state-of-the-art methods.

Research related to the diagnosis of prostate cancer based on machine learning medical images: A review

BACKGROUND

Prostate cancer is currently the second most prevalent cancer among men. Accurate diagnosis of prostate cancer can provide effective treatment for patients and greatly reduce mortality. The current medical imaging tools for screening prostate cancer are mainly MRI, CT and ultrasound. In the past 20 years, these medical imaging methods have made great progress with machine learning, especially the rise of deep learning has led to a wider application of artificial intelligence in the use of image-assisted diagnosis of prostate cancer.

METHOD

This review collected medical image processing methods, prostate and prostate cancer on MR images, CT images, and ultrasound images through search engines such as web of science, PubMed, and Google Scholar, including image pre-processing methods, segmentation of prostate gland on medical images, registration between prostate gland on different modal images, detection of prostate cancer lesions on the prostate.

CONCLUSION

Through these collated papers, it is found that the current research on the diagnosis and staging of prostate cancer using machine learning and deep learning is in its infancy, and most of the existing studies are on the diagnosis of prostate cancer and classification of lesions, and the accuracy is low, with the best results having an accuracy of less than 0.95. There are fewer studies on staging. The research is mainly focused on MR images and much less on CT images, ultrasound images.

DISCUSSION

Machine learning and deep learning combined with medical imaging have a broad application prospect for the diagnosis and staging of prostate cancer, but the research in this area still has more room for development.

Medical image data augmentation: techniques, comparisons and interpretations

Designing deep learning based methods with medical images has always been an attractive area of research to assist clinicians in rapid examination and accurate diagnosis. Those methods need a large number of datasets including all variations in their training stages. On the other hand, medical images are always scarce due to several reasons, such as not enough patients for some diseases, patients do not want to allow their images to be used, lack of medical equipment or equipment, inability to obtain images that meet the desired criteria. This issue leads to bias in datasets, overfitting, and inaccurate results. Data augmentation is a common solution to overcome this issue and various augmentation techniques have been applied to different types of images in the literature. However, it is not clear which data augmentation technique provides more efficient results for which image type since different diseases are handled, different network architectures are used, and these architectures are trained and tested with different numbers of data sets in the literature. Therefore, in this work, the augmentation techniques used to improve performances of deep learning based diagnosis of the diseases in different organs (brain, lung, breast, and eye) from different imaging modalities (MR, CT, mammography, and fundoscopy) have been examined. Also, the most commonly used augmentation methods have been implemented, and their effectiveness in classifications with a deep network has been discussed based on quantitative performance evaluations. Experiments indicated that augmentation techniques should be chosen carefully according to image types.

Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in image synthesis, Generative Adversarial Networks (GANs), and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

Multi-scale dense selective network based on border modeling for lung nodule segmentation

PURPOSE

Accurate quantification of pulmonary nodules helps physicians to accurately diagnose and treat lung cancer. We try to improve the segmentation efficiency of irregular nodules while maintaining the segmentation accuracy of simple types of nodules.

METHODS

In this paper, we obtain the unique edge part of pulmonary nodules and process it as a single branch stream, i.e., border stream, to explicitly model the nodule edge information. We propose a multi-scale dense selective network based on border modeling (BorDenNet). Its overall framework consists of a dual-branch encoder-decoder, which achieves parallel processing of classical image stream and border stream. We design a dense attention module to facilitate a strongly coupled status of feature images to focus on key regions of pulmonary nodules. Then, during the process of model decoding, the multi-scale selective attention module is proposed to establish long-range correlation relationships between different scale features, which further achieves finer feature discrimination and spatial recovery. We introduce border context enhancement module to mutually fuse and enhance the edge-related voxel features contained in the image stream and border stream and finally achieve the accurate segmentation of pulmonary nodules.

RESULTS

We evaluate the BorDenNet rigorously on the lung public dataset LIDC-IDRI. For the segmentation of the target nodules, the average Dice score is 92.78[Formula: see text], the average sensitivity is 91.37[Formula: see text], and the average Hausdorff distance is 3.06 mm. We further test on a private dataset from Shanxi Provincial People's Hospital, which verifies the excellent generalization of BorDenNet. Our BorDenNet relatively improves the segmentation efficiency for multi-type nodules such as adherent pulmonary nodules and ground-glass pulmonary nodules.

CONCLUSION

Accurate segmentation of irregular pulmonary nodules can obtain important clinical parameters, which can be used as a guide for clinicians and improve clinical efficiency.

Synthesizing 3D Lung CT scans with Generative Adversarial Networks

In the healthcare domain, datasets are often private and lack large amounts of samples, making it difficult to cope with the inherent patient data heterogeneity. As an attempt to mitigate data scarcity, generative models are being used due to their ability to produce new data, using a dataset as a reference. However, synthesis studies often rely on a 2D representation of data, a seriously limited form of information when it comes to lung computed tomography scans where, for example, pathologies like nodules can manifest anywhere in the organ. Here, we develop a 3D Progressive Growing Generative Adversarial Network capable of generating thoracic CT volumes at a resolution of 128³, and analyze the model outputs through a quantitative metric (3D Muli-Scale Structural Similarity) and a Visual Turing Test. Clinical relevance - This paper is a novel application of the 3D PGGAN model to synthesize CT lung scans. This preliminary study focuses on synthesizing the entire volume of the lung rather than just the lung nodules. The synthesized data represent an attempt to mitigate data scarcity which is one of the major limitations to create learning models with good generalization in healthcare.

A GAN based approach for inferring progression trajectories of costal cartilage calcification from cross-sectional data at image level

BACKGROUND

Costal cartilage calcification (CCC) increases with age and presents differently for men and women. In individuals, however, the cross-sectional studies that show such trends do not reveal the geometric trajectories through which calcification might accumulate across a lifetime. Generative adversarial networks have the potential to reveal such trajectories from cross-sectional data by learning population trends and synthesizing individualized images at progressive levels of calcification.

METHODS

Chest wall mid-surface CT images with normalized cartilage morphologies were produced for 379 subjects aged 6 to 90, and labeled by sex and calcification severity. A conditional GAN with added loss terms to favor one-way accumulation of CCC was trained using organized image batches. GAN performance was assessed by comparing the distributions of images between the training and synthetic groups.

RESULTS

Synthetic images generated from a common seed for a given sex and at successive calcification severity levels showed incremental and regional growth of calcification sites. CCC patterns for synthetic male and female images matched known sex-based differences, and individual CCC growth in synthetic images was consistent with previously observed population trends. These trends in the synthetic images were also quantified by structural similarity scores. Synthetic images generated from different input seeds further showed individual variance in specific regions and trajectories of CCC accumulation.

CONCLUSION

This study inferred individual progression of CCC accumulation from uncalcified to severely calcified using cross-sectional image data. This information can inform computational models of the changing chest wall biomechanics with age, and the GAN-based technique shows potential for inferring longitudinal data from population trends in other clinical areas.

Generative Adversarial Networks in Medical Image augmentation: A review

OBJECT

With the development of deep learning, the number of training samples for medical image-based diagnosis and treatment models is increasing. Generative Adversarial Networks (GANs) have attracted attention in medical image processing due to their excellent image generation capabilities and have been widely used in data augmentation. In this paper, a comprehensive and systematic review and analysis of medical image augmentation work are carried out, and its research status and development prospects are reviewed.

METHOD

This paper reviews 105 medical image augmentation related papers, which mainly collected by ELSEVIER, IEEE Xplore, and Springer from 2018 to 2021. We counted these papers according to the parts of the organs corresponding to the images, and sorted out the medical image datasets that appeared in them, the loss function in model training, and the quantitative evaluation metrics of image augmentation. At the same time, we briefly introduce the literature collected in three journals and three conferences that have received attention in medical image processing.

RESULT

First, we summarize the advantages of various augmentation models, loss functions, and evaluation metrics. Researchers can use this information as a reference when designing augmentation tasks. Second, we explore the relationship between augmented models and the amount of the training set, and tease out the role that augmented models may play when the quality of the training set is limited. Third, the statistical number of papers shows that the development momentum of this research field remains strong. Furthermore, we discuss the existing limitations of this type of model and suggest possible research directions.

CONCLUSION

We discuss GAN-based medical image augmentation work in detail. This method effectively alleviates the challenge of limited training samples for medical image diagnosis and treatment models. It is hoped that this review will benefit researchers interested in this field.

Lung Cancer Segmentation With Transfer Learning: Usefulness of a Pretrained Model Constructed From an Artificial Dataset Generated Using a Generative Adversarial Network

Purpose: The purpose of this study was to develop and evaluate lung cancer segmentation with a pretrained model and transfer learning. The pretrained model was constructed from an artificial dataset generated using a generative adversarial network (GAN).

Materials and Methods: Three public datasets containing images of lung nodules/lung cancers were used: LUNA16 dataset, Decathlon lung dataset, and NSCLC radiogenomics. The LUNA16 dataset was used to generate an artificial dataset for lung cancer segmentation with the help of the GAN and 3D graph cut. Pretrained models were then constructed from the artificial dataset. Subsequently, the main segmentation model was constructed from the pretrained models and the Decathlon lung dataset. Finally, the NSCLC radiogenomics dataset was used to evaluate the main segmentation model. The Dice similarity coefficient (DSC) was used as a metric to evaluate the segmentation performance.

Results: The mean DSC for the NSCLC radiogenomics dataset improved overall when using the pretrained models. At maximum, the mean DSC was 0.09 higher with the pretrained model than that without it.

Conclusion: The proposed method comprising an artificial dataset and a pretrained model can improve lung cancer segmentation as confirmed in terms of the DSC metric. Moreover, the construction of the artificial dataset for the segmentation using the GAN and 3D graph cut was found to be feasible.

CMMCSegNet: Cross-Modality Multicascade Indirect LGE Segmentation on Multimodal Cardiac MR

Since Late-Gadolinium Enhancement (LGE) of cardiac magnetic resonance (CMR) visualizes myocardial infarction, and the balanced-Steady State Free Precession (bSSFP) cine sequence can capture cardiac motions and present clear boundaries; multimodal CMR segmentation has played an important role in the assessment of myocardial viability and clinical diagnosis, while automatic and accurate CMR segmentation still remains challenging due to a very small amount of labeled LGE data and the relatively low contrasts of LGE. The main purpose of our work is to learn the real/fake bSSFP modality with ground truths to indirectly segment the LGE modality of cardiac MR by using a proposed cross-modality multicascade framework: cross-modality translation network and automatic segmentation network, respectively. In the segmentation stage, a novel multicascade pix2pix network is designed to segment the fake bSSFP sequence obtained from a cross-modality translation network. Moreover, we propose perceptual loss measuring features between ground truth and prediction, which are extracted from the pretrained vgg network in the segmentation stage. We evaluate the performance of the proposed method on the multimodal CMR dataset and verify its superiority over other state-of-the-art approaches under different network structures and different types of adversarial losses in terms of dice accuracy in testing. Therefore, the proposed network is promising for Indirect Cardiac LGE Segmentation in clinical applications.