Semantic Segmentation of White Matter in FDG-PET Using Generative Adversarial Network

Longitudinal imaging of murine atherosclerosis with 2-deoxy-2-[18F]fluoro-D-glucose and [18F]-sodium fluoride in genetically modified Apolipoprotein E knock-out and wild type mice

In a longitudinal design, four arterial segments in mice were followed by positron emission tomography/computed tomography (PET/CT) imaging. We aimed to determine how the tracers reflected the development of atherosclerosis via the uptake of 2-deoxy-2-[18F]fluoro-D-glucose ([18F]FDG) for imaging inflammation and [18F]-sodium fluoride (Na[18F]F) for imaging active microcalcification in a murine model of atherosclerosis. Apolipoprotein E knock-out (ApoE) mice and C57 BL/6NtaC (B6) mice were divided into four groups. They received either normal chow (N = 7, ApoE mice and N = 6, B6 mice) for 32 weeks or a high-fat diet (N = 6, ApoEHFD mice and N = 9, B6HFD mice) for 32 weeks. The mice were scanned with [18F]FDG and Na[18F]F using a dedicated small animal PET/CT scanner at three timepoints. The tracer uptakes in four aortic segments (abdominal aorta, aortic arch, ascending aorta, and thoracic aorta) were measured and reported as SUVmax values. The uptake of [18F]FDG (SUVmax: 5.7 ± 0.5 vs 1.9 ± 0.2, 230.3%, p =  < 0.0001) and Na[18F]F (SUVmax: 9.6 ± 1.8 vs 4.0 ± 0.3, 175%, p = 0.007) was significantly increased in the abdominal aorta of ApoEHFD mice at Week 32 compared to baseline abdominal aorta values of ApoEHFD mice. [18F]FDG uptake in the aortic arch, ascending aorta and the thoracic aorta of B6HFD mice at Week 32 showed a robust resemblance to the abdominal aorta uptake whereas the Na[18F]F uptake only resembled in the thoracic aorta of B6HFD mice at Week 32 compared to the abdominal aorta. The uptake of both [18F]FDG and Na[18F]F increased as the disease progressed over time, and the abdominal aorta provided a robust measure across mouse strain and diet. Therefore, it seems to be the preferred region for image readout. For [18F]FDG-PET, both B6 and ApoE mice provide valuable information and either mouse strain may be used in preclinical cardiovascular studies, whereas for Na[18F]F -PET, ApoE mice should be preferred.

MMGan: a multimodal MR brain tumor image segmentation method

Computer-aided diagnosis has emerged as a rapidly evolving field, garnering increased attention in recent years. At the forefront of this field is the segmentation of lesions in medical images, which is a critical preliminary stage in subsequent treatment procedures. Among the most challenging tasks in medical image analysis is the accurate and automated segmentation of brain tumors in various modalities of brain tumor MRI. In this article, we present a novel end-to-end network architecture called MMGan, which combines the advantages of residual learning and generative adversarial neural networks inspired by classical generative adversarial networks. The segmenter in the MMGan network, which has a U-Net architecture, is constructed using a deep residual network instead of the conventional convolutional neural network. The dataset used for this study is the BRATS dataset from the Brain Tumor Segmentation Challenge at the Medical Image Computing and Computer Assisted Intervention Society. Our proposed method has been extensively tested, and the results indicate that this MMGan framework is more efficient and stable for segmentation tasks. On BRATS 2019, the segmentation algorithm improved accuracy and sensitivity in whole tumor, tumor core, and enhanced tumor segmentation. Particularly noteworthy is the higher dice score of 0.86 achieved by our proposed method in tumor core segmentation, surpassing those of stateof-the-art models. This study improves the accuracy and sensitivity of the tumor segmentation task, which we believe is significant for medical image analysis. And it should be further improved by replacing different loss functions such as cross-entropy loss function and other methods.

A transformer-based generative adversarial network for brain tumor segmentation

Brain tumor segmentation remains a challenge in medical image segmentation tasks. With the application of transformer in various computer vision tasks, transformer blocks show the capability of learning long-distance dependency in global space, which is complementary to CNNs. In this paper, we proposed a novel transformer-based generative adversarial network to automatically segment brain tumors with multi-modalities MRI. Our architecture consists of a generator and a discriminator, which is trained in min-max game progress. The generator is based on a typical "U-shaped" encoder-decoder architecture, whose bottom layer is composed of transformer blocks with Resnet. Besides, the generator is trained with deep supervision technology. The discriminator we designed is a CNN-based network with multi-scale L 1 loss, which is proved to be effective for medical semantic image segmentation. To validate the effectiveness of our method, we conducted exclusive experiments on BRATS2015 dataset, achieving comparable or better performance than previous state-of-the-art methods. On additional datasets, including BRATS2018 and BRATS2020, experimental results prove that our technique is capable of generalizing successfully.

Generative adversarial networks and its applications in the biomedical image segmentation: a comprehensive survey

Recent advancements with deep generative models have proven significant potential in the task of image synthesis, detection, segmentation, and classification. Segmenting the medical images is considered a primary challenge in the biomedical imaging field. There have been various GANs-based models proposed in the literature to resolve medical segmentation challenges. Our research outcome has identified 151 papers; after the twofold screening, 138 papers are selected for the final survey. A comprehensive survey is conducted on GANs network application to medical image segmentation, primarily focused on various GANs-based models, performance metrics, loss function, datasets, augmentation methods, paper implementation, and source codes. Secondly, this paper provides a detailed overview of GANs network application in different human diseases segmentation. We conclude our research with critical discussion, limitations of GANs, and suggestions for future directions. We hope this survey is beneficial and increases awareness of GANs network implementations for biomedical image segmentation tasks.

Applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging: A review

PURPOSE

This paper reviews recent applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging. Recent advances in Deep Learning (DL) and GANs catalysed the research of their applications in medical imaging modalities. As a result, several unique GAN topologies have emerged and been assessed in an experimental environment over the last two years.

METHODS

The present work extensively describes GAN architectures and their applications in PET imaging. The identification of relevant publications was performed via approved publication indexing websites and repositories. Web of Science, Scopus, and Google Scholar were the major sources of information.

RESULTS

The research identified a hundred articles that address PET imaging applications such as attenuation correction, de-noising, scatter correction, removal of artefacts, image fusion, high-dose image estimation, super-resolution, segmentation, and cross-modality synthesis. These applications are presented and accompanied by the corresponding research works.

CONCLUSION

GANs are rapidly employed in PET imaging tasks. However, specific limitations must be eliminated to reach their full potential and gain the medical community's trust in everyday clinical practice.

Generative Adversarial Networks in Brain Imaging: A Narrative Review

Artificial intelligence (AI) is expected to have a major effect on radiology as it demonstrated remarkable progress in many clinical tasks, mostly regarding the detection, segmentation, classification, monitoring, and prediction of diseases. Generative Adversarial Networks have been proposed as one of the most exciting applications of deep learning in radiology. GANs are a new approach to deep learning that leverages adversarial learning to tackle a wide array of computer vision challenges. Brain radiology was one of the first fields where GANs found their application. In neuroradiology, indeed, GANs open unexplored scenarios, allowing new processes such as image-to-image and cross-modality synthesis, image reconstruction, image segmentation, image synthesis, data augmentation, disease progression models, and brain decoding. In this narrative review, we will provide an introduction to GANs in brain imaging, discussing the clinical potential of GANs, future clinical applications, as well as pitfalls that radiologists should be aware of.

Advancing diagnostic performance and clinical applicability of deep learning-driven generative adversarial networks for Alzheimer's disease

Alzheimer's disease (AD) is a neurodegenerative disease that severely affects the activities of daily living in aged individuals, which typically needs to be diagnosed at an early stage. Generative adversarial networks (GANs) provide a new deep learning method that show good performance in image processing, while it remains to be verified whether a GAN brings benefit in AD diagnosis. The purpose of this research is to systematically review psychoradiological studies on the application of a GAN in the diagnosis of AD from the aspects of classification of AD state and AD-related image processing compared with other methods. In addition, we evaluated the research methodology and provided suggestions from the perspective of clinical application. Compared with other methods, a GAN has higher accuracy in the classification of AD state and better performance in AD-related image processing (e.g. image denoising and segmentation). Most studies used data from public databases but lacked clinical validation, and the process of quantitative assessment and comparison in these studies lacked clinicians' participation, which may have an impact on the improvement of generation effect and generalization ability of the GAN model. The application value of GANs in the classification of AD state and AD-related image processing has been confirmed in reviewed studies. Improvement methods toward better GAN architecture were also discussed in this paper. In sum, the present study demonstrated advancing diagnostic performance and clinical applicability of GAN for AD, and suggested that the future researchers should consider recruiting clinicians to compare the algorithm with clinician manual methods and evaluate the clinical effect of the algorithm.

Segmentation of white matter hyperintensities on 18F-FDG PET/CT images with a generative adversarial network

PURPOSE

White matter hyperintensities (WMH) are typically segmented using MRI because WMH are hardly visible on ¹⁸F-FDG PET/CT. This retrospective study was conducted to segment WMH and estimate their volumes from ¹⁸F-FDG PET with a generative adversarial network (W_hyperGAN).

METHODS

We selected patients whose interval between MRI and FDG PET/CT scans was within 3 months, from January 2017 to December 2018, and classified them into mild, moderate, and severe groups by following the semiquantitative rating method of Fazekas. For each group, 50 patients were selected, and of them, we randomly selected 35 patients for training and 15 for testing. WMH were automatically segmented from FLAIR MRI with manual adjustment. Patches of WMH were extracted from ¹⁸F-FDG PET and segmented MRI. W_hyperGAN was compared with H-DenseUnet, a deep learning method widely used for segmentation tasks, for segmentation performance based on the dice similarity coefficient (DSC), recall, and average volume differences (AVD). For volume estimation, the predicted WMH volumes from PET were compared with ground truth volumes.

RESULTS

The DSC values were associated with WMH volumes on MRI. For volumes >60 mL, the DSC values were 0.751 for W_hyperGAN and 0.564 for H-DenseUnet. For volumes ≤60 mL, the DSC values rapidly decreased as the volume decreased (0.362 for W_hyperGAN vs. 0.237 for H-DenseUnet). For recall, W_hyperGAN achieved the highest value in the severe group (0.579 for W_hyperGAN vs. 0.509 for H-DenseUnet). For AVD, W_hyperGAN achieved the lowest score in the severe group (0.494 for W_hyperGAN vs. 0.941 for H-DenseUnet). For the WMH volume estimation, W_hyperGAN performed better than H-DenseUnet and yielded excellent correlation coefficients (r = 0.998, 0.983, and 0.908 in the severe, moderate, and mild group).

CONCLUSIONS

Although limited by visual analysis, the W_hyperGAN based can be used to automatically segment and estimate volumes of WMH from ¹⁸F-FDG PET/CT. This would increase the usefulness of ¹⁸F-FDG PET/CT for the evaluation of WMH in patients with cognitive impairment.

Application and Construction of Deep Learning Networks in Medical Imaging

Deep learning (DL) approaches are part of the machine learning (ML) subfield concerned with the development of computational models to train artificial intelligence systems. DL models are characterized by automatically extracting high-level features from the input data to learn the relationship between matching datasets. Thus, its implementation offers an advantage over common ML methods that often require the practitioner to have some domain knowledge of the input data to select the best latent representation. As a result of this advantage, DL has been successfully applied within the medical imaging field to address problems, such as disease classification and tumor segmentation for which it is difficult or impossible to determine which image features are relevant. Therefore, taking into consideration the positive impact of DL on the medical imaging field, this article reviews the key concepts associated with its evolution and implementation. The sections of this review summarize the milestones related to the development of the DL field, followed by a description of the elements of deep neural network and an overview of its application within the medical imaging field. Subsequently, the key steps necessary to implement a supervised DL application are defined, and associated limitations are discussed.

Assessment of Total-Body Atherosclerosis by PET/Computed Tomography

Atherosclerotic burden has become the focus of cardiovascular risk assessment. PET/computed tomography (CT) imaging with the tracers 18F-fluorodeoxyglucose and 18F-sodium fluoride shows arterial wall inflammation and microcalcification, respectively. Arterial uptake of both tracers is modestly age dependent. 18F-sodium fluoride uptake is consistently associated with risk factors and more easily measured in the heart. Because of extremely high sensitivity, ultrashort acquisition, and minimal radiation to the patient, total-body PET/CT provides unique opportunities for atherosclerosis imaging: disease screening and delayed and repeat imaging with global disease scoring and parametric imaging to better characterize the atherosclerosis of individual patients.