TumorGAN: A Multi-Modal Data Augmentation Framework for Brain Tumor Segmentation

Development of brain tumor radiogenomic classification using GAN-based augmentation of MRI slices in the newly released gazi brains dataset

Significant progress has been made recently with the contribution of technological advances in studies on brain cancer. Regarding this, identifying and correctly classifying tumors is a crucial task in the field of medical imaging. The disease-related tumor classification problem, on which deep learning technologies have also become a focus, is very important in the diagnosis and treatment of the disease. The use of deep learning models has shown promising results in recent years. However, the sparsity of ground truth data in medical imaging or inconsistent data sources poses a significant challenge for training these models. The utilization of StyleGANv2-ADA is proposed in this paper for augmenting brain MRI slices to enhance the performance of deep learning models. Specifically, augmentation is applied solely to the training data to prevent any potential leakage. The StyleGanv2-ADA model is trained with the Gazi Brains 2020, BRaTS 2021, and Br35h datasets using the researchers' default settings. The effectiveness of the proposed method is demonstrated on datasets for brain tumor classification, resulting in a notable improvement in the overall accuracy of the model for brain tumor classification on all the Gazi Brains 2020, BraTS 2021, and Br35h datasets. Importantly, the utilization of StyleGANv2-ADA on the Gazi Brains 2020 Dataset represents a novel experiment in the literature. The results show that the augmentation with StyleGAN can help overcome the challenges of working with medical data and the sparsity of ground truth data. Data augmentation employing the StyleGANv2-ADA GAN model yielded the highest overall accuracy for brain tumor classification on the BraTS 2021 and Gazi Brains 2020 datasets, together with the BR35H dataset, achieving 75.18%, 99.36%, and 98.99% on the EfficientNetV2S models, respectively. This study emphasizes the potency of GANs for augmenting medical imaging datasets, particularly in brain tumor classification, showcasing a notable increase in overall accuracy through the integration of synthetic GAN data on the used datasets.

Convolutional Neural Network-Machine Learning Model: Hybrid Model for Meningioma Tumour and Healthy Brain Classification

This paper presents a hybrid study of convolutional neural networks (CNNs), machine learning (ML), and transfer learning (TL) in the context of brain magnetic resonance imaging (MRI). The anatomy of the brain is very complex; inside the skull, a brain tumour can form in any part. With MRI technology, cross-sectional images are generated, and radiologists can detect the abnormalities. When the size of the tumour is very small, it is undetectable to the human visual system, necessitating alternative analysis using AI tools. As is widely known, CNNs explore the structure of an image and provide features on the SoftMax fully connected (SFC) layer, and the classification of the items that belong to the input classes is established. Two comparison studies for the classification of meningioma tumours and healthy brains are presented in this paper: (i) classifying MRI images using an original CNN and two pre-trained CNNs, DenseNet169 and EfficientNetV2B0; (ii) determining which CNN and ML combination yields the most accurate classification when SoftMax is replaced with three ML models; in this context, Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM) were proposed. In a binary classification of tumours and healthy brains, the EfficientNetB0-SVM combination shows an accuracy of 99.5% on the test dataset. A generalisation of the results was performed, and overfitting was prevented by using the bagging ensemble method.

Brain tumor recognition by an optimized deep network utilizing ammended grasshopper optimization

Brain tumors are abnormal cell masses that can get originated in the brain spread from other organs. They can be categorized as either malignant (cancerous) or benign (noncancerous), and their growth rates and locations can impact the functioning of the nerve system. The timely detection of brain tumors is crucial for effective treatment and prognosis. In this study, a new approach has been proposed for diagnosing brain tumors using deep learning and a meta-heuristic algorithm. The method involves three main steps: (1) extracting features from brain MRI images using AlexNet, (2) reducing the complexity of AlexNet by employing an Extreme Learning Machine (ELM) network as a classification layer, and (3) fine-tuning the parameters of the ELM network using an Amended Grasshopper Optimization Algorithm (AGOA). The performance of the method has been evaluated on a publicly available dataset consisting of 20 patients with newly diagnosed glioblastoma that is compared with several state-of-the-art techniques. Experimental results demonstrate that the method achieves the highest accuracy, precision, specificity, F1-score, sensitivity, and MCC with values of 0.96, 0.94, 0.96, 0.96, 0.94, and 0.90, respectively. Furthermore, the robustness and stability of the method have been illustrated when subjected to different levels of noise and image resolutions. The proposed approach offers a rapid, accurate, and dependable diagnosis of brain tumors and holds potential for application in other medical image analysis tasks.

An Efficient Brain Tumor Segmentation Method Based on Adaptive Moving Self-Organizing Map and Fuzzy K-Mean Clustering

Brain tumors in Magnetic resonance image segmentation is challenging research. With the advent of a new era and research into machine learning, tumor detection and segmentation generated significant interest in the research world. This research presents an efficient tumor detection and segmentation technique using an adaptive moving self-organizing map and Fuzzyk-mean clustering (AMSOM-FKM). The proposed method mainly focused on tumor segmentation using extraction of the tumor region. AMSOM is an artificial neural technique whose training is unsupervised. This research utilized the online Kaggle Brats-18 brain tumor dataset. This dataset consisted of 1691 images. The dataset was partitioned into 70% training, 20% testing, and 10% validation. The proposed model was based on various phases: (a) removal of noise, (b) selection of feature attributes, (c) image classification, and (d) tumor segmentation. At first, the MR images were normalized using the Wiener filtering method, and the Gray level co-occurrences matrix (GLCM) was used to extract the relevant feature attributes. The tumor images were separated from non-tumor images using the AMSOM classification approach. At last, the FKM was used to distinguish the tumor region from the surrounding tissue. The proposed AMSOM-FKM technique and existing methods, i.e., Fuzzy-C-means and K-mean (FMFCM), hybrid self-organization mapping-FKM, were implemented over MATLAB and compared based on comparison parameters, i.e., sensitivity, precision, accuracy, and similarity index values. The proposed technique achieved more than 10% better results than existing methods.

A synthetic data generation system for myalgic encephalomyelitis/chronic fatigue syndrome questionnaires

Artificial intelligence or machine-learning-based models have proven useful for better understanding various diseases in all areas of health science. Myalgic Encephalomyelitis or chronic fatigue syndrome (ME/CFS) lacks objective diagnostic tests. Some validated questionnaires are used for diagnosis and assessment of disease progression. The availability of a sufficiently large database of these questionnaires facilitates research into new models that can predict profiles that help to understand the etiology of the disease. A synthetic data generator provides the scientific community with databases that preserve the statistical properties of the original, free of legal restrictions, for use in research and education. The initial databases came from the Vall Hebron Hospital Specialized Unit in Barcelona, Spain. 2522 patients diagnosed with ME/CFS were analyzed. Their answers to questionnaires related to the symptoms of this complex disease were used as training datasets. They have been fed for deep learning algorithms that provide models with high accuracy [0.69-0.81]. The final model requires SF-36 responses and returns responses from HAD, SCL-90R, FIS8, FIS40, and PSQI questionnaires. A highly reliable and easy-to-use synthetic data generator is offered for research and educational use in this disease, for which there is currently no approved treatment.

Artificial Intelligence-Assisted Classification of Gliomas Using Whole Slide Images

CONTEXT.—

Glioma is the most common primary brain tumor in adults. The diagnosis and grading of different pathological subtypes of glioma is essential in treatment planning and prognosis.

OBJECTIVE.—

To propose a deep learning-based approach for the automated classification of glioma histopathology images. Two classification methods, the ensemble method based on 2 binary classifiers and the multiclass method using a single multiclass classifier, were implemented to classify glioma images into astrocytoma, oligodendroglioma, and glioblastoma, according to the 5th edition of the World Health Organization classification of central nervous system tumors, published in 2021.

DESIGN.—

We tested 2 different deep neural network architectures (VGG19 and ResNet50) and extensively validated the proposed approach based on The Cancer Genome Atlas data set (n = 700). We also studied the effects of stain normalization and data augmentation on the glioma classification task.

RESULTS.—

With the binary classifiers, our model could distinguish astrocytoma and oligodendroglioma (combined) from glioblastoma with an accuracy of 0.917 (area under the curve [AUC] = 0.976) and astrocytoma from oligodendroglioma (accuracy = 0.821, AUC = 0.865). The multiclass method (accuracy = 0.861, AUC = 0.961) outperformed the ensemble method (accuracy = 0.847, AUC = 0.933) with the best performance displayed by the ResNet50 architecture.

CONCLUSIONS.—

With the high performance of our model (>80%), the proposed method can assist pathologists and physicians to support examination and differential diagnosis of glioma histopathology images, with the aim to expedite personalized medical care.

Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification

Multi-modal (i.e., visible, near-infrared, and thermal-infrared) vehicle re-identification has good potential to search vehicles of interest in low illumination. However, due to the fact that different modalities have varying imaging characteristics, a proper multi-modal complementary information fusion is crucial to multi-modal vehicle re-identification. For that, this paper proposes a progressively hybrid transformer (PHT). The PHT method consists of two aspects: random hybrid augmentation (RHA) and a feature hybrid mechanism (FHM). Regarding RHA, an image random cropper and a local region hybrider are designed. The image random cropper simultaneously crops multi-modal images of random positions, random numbers, random sizes, and random aspect ratios to generate local regions. The local region hybrider fuses the cropped regions to let regions of each modal bring local structural characteristics of all modalities, mitigating modal differences at the beginning of feature learning. Regarding the FHM, a modal-specific controller and a modal information embedding are designed to effectively fuse multi-modal information at the feature level. Experimental results show the proposed method wins the state-of-the-art method by a larger 2.7% mAP on RGBNT100 and a larger 6.6% mAP on RGBN300, demonstrating that the proposed method can learn multi-modal complementary information effectively.

Interpretable Multi-modal Image Registration Network Based on Disentangled Convolutional Sparse Coding

Multi-modal image registration aims to spatially align two images from different modalities to make their feature points match with each other. Captured by different sensors, the images from different modalities often contain many distinct features, which makes it challenging to find their accurate correspondences. With the success of deep learning, many deep networks have been proposed to align multi-modal images, however, they are mostly lack of interpretability. In this paper, we first model the multi-modal image registration problem as a disentangled convolutional sparse coding (DCSC) model. In this model, the multi-modal features that are responsible for alignment (RA features) are well separated from the features that are not responsible for alignment (nRA features). By only allowing the RA features to participate in the deformation field prediction, we can eliminate the interference of the nRA features to improve the registration accuracy and efficiency. The optimization process of the DCSC model to separate the RA and nRA features is then turned into a deep network, namely Interpretable Multi-modal Image Registration Network (InMIR-Net). To ensure the accurate separation of RA and nRA features, we further design an accompanying guidance network (AG-Net) to supervise the extraction of RA features in InMIR-Net. The advantage of InMIR-Net is that it provides a universal framework to tackle both rigid and non-rigid multi-modal image registration tasks. Extensive experimental results verify the effectiveness of our method on both rigid and non-rigid registrations on various multi-modal image datasets, including RGB/depth images, RGB/near-infrared (NIR) images, RGB/multi-spectral images, T1/T2 weighted magnetic resonance (MR) images and computed tomography (CT)/MR images. The codes are available at https://github.com/lep990816/Interpretable-Multi-modal-Image-Registration.

Data augmentation for medical imaging: A systematic literature review

Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.

The multi-level classification network (MCN) with modified residual U-Net for ischemic stroke lesions segmentation from ATLAS

Ischemic and hemorrhagic strokes are two major types of internal brain injury. 3D brain MRI is suggested by neurologists to examine the brain. Manual examination of brain MRI is very sensitive and time-consuming task. However, automatic diagnosis can assist doctors in this regard. Anatomical Tracings of Lesions After Stroke (ATLAS) is publicly available dataset for research experiments. One of the major issues in medical imaging is class imbalance. Similarly, pixel representation of stroke lesion is less than 1% in ATLAS. Second major challenge in this dataset is inter-class similarity. A multi-level classification network (MCN) is proposed for segmentation of ischemic stroke lesions. MCN consists of three cascaded discrete networks. The first network designed to reduce the slice level class imbalance, where a classifier model is trained to extract the slices of stroke lesions from a whole brain MRI volume. The interclass similarity cause to produce false positives in segmented output. Therefore, all extracted stroke slices were divided into overlapping patches (64 × 64) and carried to the second network. The task associated with second network is to classify the patches comprises of stroke lesion. The third network is a 2D modified residual U-Net that segments out the stroke lesions from the patches extracted by the second network. MCN achieved 0.754 mean dice score on test dataset which is higher than the other state-of-the-art methods on the same dataset.

Deep learning in neuroimaging data analysis: Applications, challenges, and solutions

Methods for the analysis of neuroimaging data have advanced significantly since the beginning of neuroscience as a scientific discipline. Today, sophisticated statistical procedures allow us to examine complex multivariate patterns, however most of them are still constrained by assuming inherent linearity of neural processes. Here, we discuss a group of machine learning methods, called deep learning, which have drawn much attention in and outside the field of neuroscience in recent years and hold the potential to surpass the mentioned limitations. Firstly, we describe and explain the essential concepts in deep learning: the structure and the computational operations that allow deep models to learn. After that, we move to the most common applications of deep learning in neuroimaging data analysis: prediction of outcome, interpretation of internal representations, generation of synthetic data and segmentation. In the next section we present issues that deep learning poses, which concerns multidimensionality and multimodality of data, overfitting and computational cost, and propose possible solutions. Lastly, we discuss the current reach of DL usage in all the common applications in neuroimaging data analysis, where we consider the promise of multimodality, capability of processing raw data, and advanced visualization strategies. We identify research gaps, such as focusing on a limited number of criterion variables and the lack of a well-defined strategy for choosing architecture and hyperparameters. Furthermore, we talk about the possibility of conducting research with constructs that have been ignored so far or/and moving toward frameworks, such as RDoC, the potential of transfer learning and generation of synthetic data.

Particle Swarm Optimization and Two-Way Fixed-Effects Analysis of Variance for Efficient Brain Tumor Segmentation

Simple Summary

Segmentation of brain tumor images from magnetic resonance imaging (MRI) is a challenging topic in medical image analysis. The brain tumor can take many shapes, and MRI images vary considerably in intensity, making lesion detection difficult for radiologists. This paper proposes a three-step approach to solving this problem: (1) pre-processing, based on morphological operations, is applied to remove the skull bone from the image; (2) the particle swarm optimization (PSO) algorithm, with a two-way fixed-effects analysis of variance (ANOVA)-based fitness function, is used to find the optimal block containing the brain lesion; (3) the K-means clustering algorithm is adopted, to classify the detected block as tumor or non-tumor. An extensive experimental analysis, including visual and statistical evaluations, was conducted, using two MRI databases: a private database provided by the Kouba imaging center—Algiers (KICA)—and the multimodal brain tumor segmentation challenge (BraTS) 2015 database. The results show that the proposed methodology achieved impressive performance, compared to several competing approaches.

Abstract

Segmentation of brain tumor images, to refine the detection and understanding of abnormal masses in the brain, is an important research topic in medical imaging. This paper proposes a new segmentation method, consisting of three main steps, to detect brain lesions using magnetic resonance imaging (MRI). In the first step, the parts of the image delineating the skull bone are removed, to exclude insignificant data. In the second step, which is the main contribution of this study, the particle swarm optimization (PSO) technique is applied, to detect the block that contains the brain lesions. The fitness function, used to determine the best block among all candidate blocks, is based on a two-way fixed-effects analysis of variance (ANOVA). In the last step of the algorithm, the K-means segmentation method is used in the lesion block, to classify it as a tumor or not. A thorough evaluation of the proposed algorithm was performed, using: (1) a private MRI database provided by the Kouba imaging center—Algiers (KICA); (2) the multimodal brain tumor segmentation challenge (BraTS) 2015 database. Estimates of the selected fitness function were first compared to those based on the sum-of-absolute-differences (SAD) dissimilarity criterion, to demonstrate the efficiency and robustness of the ANOVA. The performance of the optimized brain tumor segmentation algorithm was then compared to the results of several state-of-the-art techniques. The results obtained, by using the Dice coefficient, Jaccard distance, correlation coefficient, and root mean square error (RMSE) measurements, demonstrated the superiority of the proposed optimized segmentation algorithm over equivalent techniques.

Dielectric Breast Phantoms by Generative Adversarial Network

In order to conduct the research of machine-learning (ML) based microwave breast imaging (MBI), a large number of digital dielectric breast phantoms that can be used as training data (ground truth) are required but are difficult to be achieved from practice. Although a few dielectric breast phantoms have been developed for research purpose, the number and the diversity are limited and is far inadequate to develop a robust ML algorithm for MBI. This paper presents a neural network method to generate 2D virtual breast phantoms that are similar to the real ones, which can be used to develop ML-based MBI in the future. The generated phantoms are similar but are different from those used in training. Each phantom consists of several images with each representing the distribution of a dielectric parameter in the breast map. Statistical analysis was performed over 10,000 generated phantoms to investigate the performance of the generative network. With the generative network, one may generate unlimited number of breast images with more variations, so the ML-based MBI will be more ready to deploy.

Generative adversarial networks and its applications in the biomedical image segmentation: a comprehensive survey

Recent advancements with deep generative models have proven significant potential in the task of image synthesis, detection, segmentation, and classification. Segmenting the medical images is considered a primary challenge in the biomedical imaging field. There have been various GANs-based models proposed in the literature to resolve medical segmentation challenges. Our research outcome has identified 151 papers; after the twofold screening, 138 papers are selected for the final survey. A comprehensive survey is conducted on GANs network application to medical image segmentation, primarily focused on various GANs-based models, performance metrics, loss function, datasets, augmentation methods, paper implementation, and source codes. Secondly, this paper provides a detailed overview of GANs network application in different human diseases segmentation. We conclude our research with critical discussion, limitations of GANs, and suggestions for future directions. We hope this survey is beneficial and increases awareness of GANs network implementations for biomedical image segmentation tasks.

Brain tumor image generation using an aggregation of GAN models with style transfer

In the recent past, deep learning-based models have achieved tremendous success in computer vision-related tasks with the help of large-scale annotated datasets. An interesting application of deep learning is synthetic data generation, especially in the domain of medical image analysis. The need for such a task arises due to the scarcity of original data. Class imbalance is another reason for applying data augmentation techniques. Generative Adversarial Networks (GANs) are beneficial for synthetic image generation in various fields. However, stand-alone GANs may only fetch the localized features in the latent representation of an image, whereas combining different GANs might understand the distributed features. To this end, we have proposed AGGrGAN, an aggregation of three base GAN models-two variants of Deep Convolutional Generative Adversarial Network (DCGAN) and a Wasserstein GAN (WGAN) to generate synthetic MRI scans of brain tumors. Further, we have applied the style transfer technique to enhance the image resemblance. Our proposed model efficiently overcomes the limitation of data unavailability and can understand the information variance in multiple representations of the raw images. We have conducted all the experiments on the two publicly available datasets - the brain tumor dataset and the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2020 dataset. Results show that the proposed model can generate fine-quality images with maximum Structural Similarity Index Measure (SSIM) scores of 0.57 and 0.83 on the said two datasets.

Multiresolution Aggregation Transformer UNet Based on Multiscale Input and Coordinate Attention for Medical Image Segmentation

The latest medical image segmentation methods uses UNet and transformer structures with great success. Multiscale feature fusion is one of the important factors affecting the accuracy of medical image segmentation. Existing transformer-based UNet methods do not comprehensively explore multiscale feature fusion, and there is still much room for improvement. In this paper, we propose a novel multiresolution aggregation transformer UNet (MRA-TUNet) based on multiscale input and coordinate attention for medical image segmentation. It realizes multiresolution aggregation from the following two aspects: (1) On the input side, a multiresolution aggregation module is used to fuse the input image information of different resolutions, which enhances the input features of the network. (2) On the output side, an output feature selection module is used to fuse the output information of different scales to better extract coarse-grained information and fine-grained information. We try to introduce a coordinate attention structure for the first time to further improve the segmentation performance. We compare with state-of-the-art medical image segmentation methods on the automated cardiac diagnosis challenge and the 2018 atrial segmentation challenge. Our method achieved average dice score of 0.911 for right ventricle (RV), 0.890 for myocardium (Myo), 0.961 for left ventricle (LV), and 0.923 for left atrium (LA). The experimental results on two datasets show that our method outperforms eight state-of-the-art medical image segmentation methods in dice score, precision, and recall.

A Novel Data Augmentation-Based Brain Tumor Detection Using Convolutional Neural Network

Brain tumor is a severe cancer and a life-threatening disease. Thus, early detection is crucial in the process of treatment. Recent progress in the field of deep learning has contributed enormously to the health industry medical diagnosis. Convolutional neural networks (CNNs) have been intensively used as a deep learning approach to detect brain tumors using MRI images. Due to the limited dataset, deep learning algorithms and CNNs should be improved to be more efficient. Thus, one of the most known techniques used to improve model performance is Data Augmentation. This paper presents a detailed review of various CNN architectures and highlights the characteristics of particular models such as ResNet, AlexNet, and VGG. After that, we provide an efficient method for detecting brain tumors using magnetic resonance imaging (MRI) datasets based on CNN and data augmentation. Evaluation metrics values of the proposed solution prove that it succeeded in being a contribution to previous studies in terms of both deep architectural design and high detection success.

Generative Adversarial Networks in Brain Imaging: A Narrative Review

Artificial intelligence (AI) is expected to have a major effect on radiology as it demonstrated remarkable progress in many clinical tasks, mostly regarding the detection, segmentation, classification, monitoring, and prediction of diseases. Generative Adversarial Networks have been proposed as one of the most exciting applications of deep learning in radiology. GANs are a new approach to deep learning that leverages adversarial learning to tackle a wide array of computer vision challenges. Brain radiology was one of the first fields where GANs found their application. In neuroradiology, indeed, GANs open unexplored scenarios, allowing new processes such as image-to-image and cross-modality synthesis, image reconstruction, image segmentation, image synthesis, data augmentation, disease progression models, and brain decoding. In this narrative review, we will provide an introduction to GANs in brain imaging, discussing the clinical potential of GANs, future clinical applications, as well as pitfalls that radiologists should be aware of.

Super-Resolution Generative Adversarial Network Based on the Dual Dimension Attention Mechanism for Biometric Image Super-Resolution

There exist many types of intelligent security sensors in the environment of the Internet of Things (IoT) and cloud computing. Among them, the sensor for biometrics is one of the most important types. Biometric sensors capture the physiological or behavioral features of a person, which can be further processed with cloud computing to verify or identify the user. However, a low-resolution (LR) biometrics image causes the loss of feature details and reduces the recognition rate hugely. Moreover, the lack of resolution negatively affects the performance of image-based biometric technology. From a practical perspective, most of the IoT devices suffer from hardware constraints and the low-cost equipment may not be able to meet various requirements, particularly for image resolution, because it asks for additional storage to store high-resolution (HR) images, and a high bandwidth to transmit the HR image. Therefore, how to achieve high accuracy for the biometric system without using expensive and high-cost image sensors is an interesting and valuable issue in the field of intelligent security sensors. In this paper, we proposed DDA-SRGAN, which is a generative adversarial network (GAN)-based super-resolution (SR) framework using the dual-dimension attention mechanism. The proposed model can be trained to discover the regions of interest (ROI) automatically in the LR images without any given prior knowledge. The experiments were performed on the CASIA-Thousand-v4 and the CelebA datasets. The experimental results show that the proposed method is able to learn the details of features in crucial regions and achieve better performance in most cases.

State-of-the-Art Challenges and Perspectives in Multi-Organ Cancer Diagnosis via Deep Learning-Based Methods

Thus far, the most common cause of death in the world is cancer. It consists of abnormally expanding areas that are threatening to human survival. Hence, the timely detection of cancer is important to expanding the survival rate of patients. In this survey, we analyze the state-of-the-art approaches for multi-organ cancer detection, segmentation, and classification. This article promptly reviews the present-day works in the breast, brain, lung, and skin cancer domain. Afterwards, we analytically compared the existing approaches to provide insight into the ongoing trends and future challenges. This review also provides an objective description of widely employed imaging techniques, imaging modality, gold standard database, and related literature on each cancer in 2016-2021. The main goal is to systematically examine the cancer diagnosis systems for multi-organs of the human body as mentioned. Our critical survey analysis reveals that greater than 70% of deep learning researchers attain promising results with CNN-based approaches for the early diagnosis of multi-organ cancer. This survey includes the extensive discussion part along with current research challenges, possible solutions, and prospects. This research will endow novice researchers with valuable information to deepen their knowledge and also provide the room to develop new robust computer-aid diagnosis systems, which assist health professionals in bridging the gap between rapid diagnosis and treatment planning for cancer patients.

A conventional-to-spectral CT image translation augmentation workflow for robust contrast injection-independent organ segmentation

PURPOSE

In cardiovascular imaging, the numerous contrast injection protocols used to enhance structures make it difficult to gather training datasets for deep learning applications supporting diverse protocols. Moreover, creating annotations on non-contrast scans is extremely tedious. Recently, spectral CT's virtual-non-contrast images (VNC) have been used as data augmentation to train segmentation networks performing on enhanced and true-non-contrast (TNC) scans alike, while improving results on protocols absent of their training dataset. However, spectral data are not widely available, making it difficult to gather specific datasets for each task. As a solution, we present a data augmentation workflow based on a trained image translation network, to bring spectral-like augmentation to any conventional CT dataset.

METHOD

The HU-to-spectral image translation network (HUSpecNet) was first trained to generate VNC from HU images, using an unannotated spectral dataset of 1830 patients. It was then tested on a second dataset of 300 spectral CT scans, by comparing generated VNC (VNC_DL ) to their true counterparts. To illustrate and compare our workflow's efficiency with true spectral augmentation, HUSpecNet was applied to a third dataset of 112 spectral scans to generate VNC_DL along HU and VNC images. Three different 3D networks (U-Net, X-Net, U-Net++) were trained for multi-label heart segmentation, following four augmentation strategies. As baselines, trainings were performed on contrasted images without (HUonly) and with conventional gray-values augmentation (HUaug). Then, the same networks were trained using a proportion of contrasted and VNC/VNC_DL images (TrueSpec/GenSpec). Each training strategy applied to each architecture was evaluated using Dice coefficients on a fourth multi-centric multi-vendor single-energy CT dataset of 121 patients, including different contrast injection protocols and unenhanced scans. The U-Net++ results were further explored with distance metrics on every label.

RESULTS

Tested on 300 full scans, our HUSpectNet translation network shows a mean absolute error of 6.70±2.83 HU between VNC_DL and VNC, while peak-signal-to-noise-ratio reaches 43.89 dB. GenSpec and TrueSpec show very close results regardless of the protocol and used architecture: mean Dice coefficients (DSC_mean ) are equal with a margin of 0.006, ranging from 0.879 to 0.938. Their performances significantly increase on TNC scans (p-values<0.017 for all architectures) compared to HUonly and HUaug, with DSC_mean of 0.448/0.770/0.879/0.885 for HUonly/HUaug/TrueSpec/GenSpec using the Unet++ architecture. Significant improvements are also noted for all architectures on chest-abdominal-pelvic scans (p-values<0.007) compared to HUonly and for pulmonary embolism scans (p-values<0.039) compared to HUaug. Using Unet++, DSC_mean reaches 0.892/0.901/0.903 for HUonly/TrueSpec/GenSpec on pulmonary embolism scans and 0.872/0.896/0.896 for HUonly/TrueSpec/GenSpec on chest-abdominal-pelvic scans.

CONCLUSION

Using the proposed workflow, we trained versatile heart segmentation networks on a dataset of conventional enhanced CT scans, providing robust predictions on both enhanced scans with different contrast injection protocols and TNC scans. The performances obtained were not significantly inferior to training the model on a genuine spectral CT dataset, regardless of the architecture implemented. Using a general-purpose conventional-to-spectral CT translation network as data augmentation could therefore contribute to reducing data collection and annotation requirements for machine learning-based CT studies, while extending their range of application. This article is protected by copyright. All rights reserved.

Enhancing magnetic resonance imaging-driven Alzheimer's disease classification performance using generative adversarial learning

BACKGROUND

Generative adversarial networks (GAN) can produce images of improved quality but their ability to augment image-based classification is not fully explored. We evaluated if a modified GAN can learn from magnetic resonance imaging (MRI) scans of multiple magnetic field strengths to enhance Alzheimer's disease (AD) classification performance.

METHODS

T1-weighted brain MRI scans from 151 participants of the Alzheimer's Disease Neuroimaging Initiative (ADNI), who underwent both 1.5-Tesla (1.5-T) and 3-Tesla imaging at the same time were selected to construct a GAN model. This model was trained along with a three-dimensional fully convolutional network (FCN) using the generated images (3T*) as inputs to predict AD status. Quality of the generated images was evaluated using signal to noise ratio (SNR), Blind/Referenceless Image Spatial Quality Evaluator (BRISQUE) and Natural Image Quality Evaluator (NIQE). Cases from the Australian Imaging, Biomarker & Lifestyle Flagship Study of Ageing (AIBL, n = 107) and the National Alzheimer's Coordinating Center (NACC, n = 565) were used for model validation.

RESULTS

The 3T*-based FCN classifier performed better than the FCN model trained using the 1.5-T scans. Specifically, the mean area under curve increased from 0.907 to 0.932, from 0.934 to 0.940, and from 0.870 to 0.907 on the ADNI test, AIBL, and NACC datasets, respectively. Additionally, we found that the mean quality of the generated (3T*) images was consistently higher than the 1.5-T images, as measured using SNR, BRISQUE, and NIQE on the validation datasets.

CONCLUSION

This study demonstrates a proof of principle that GAN frameworks can be constructed to augment AD classification performance and improve image quality.

Enhanced Region Growing for Brain Tumor MR Image Segmentation

A brain tumor is one of the foremost reasons for the rise in mortality among children and adults. A brain tumor is a mass of tissue that propagates out of control of the normal forces that regulate growth inside the brain. A brain tumor appears when one type of cell changes from its normal characteristics and grows and multiplies abnormally. The unusual growth of cells within the brain or inside the skull, which can be cancerous or non-cancerous has been the reason for the death of adults in developed countries and children in under developing countries like Ethiopia. The studies have shown that the region growing algorithm initializes the seed point either manually or semi-manually which as a result affects the segmentation result. However, in this paper, we proposed an enhanced region-growing algorithm for the automatic seed point initialization. The proposed approach's performance was compared with the state-of-the-art deep learning algorithms using the common dataset, BRATS2015. In the proposed approach, we applied a thresholding technique to strip the skull from each input brain image. After the skull is stripped the brain image is divided into 8 blocks. Then, for each block, we computed the mean intensities and from which the five blocks with maximum mean intensities were selected out of the eight blocks. Next, the five maximum mean intensities were used as a seed point for the region growing algorithm separately and obtained five different regions of interest (ROIs) for each skull stripped input brain image. The five ROIs generated using the proposed approach were evaluated using dice similarity score (DSS), intersection over union (IoU), and accuracy (Acc) against the ground truth (GT), and the best region of interest is selected as a final ROI. Finally, the final ROI was compared with different state-of-the-art deep learning algorithms and region-based segmentation algorithms in terms of DSS. Our proposed approach was validated in three different experimental setups. In the first experimental setup where 15 randomly selected brain images were used for testing and achieved a DSS value of 0.89. In the second and third experimental setups, the proposed approach scored a DSS value of 0.90 and 0.80 for 12 randomly selected and 800 brain images respectively. The average DSS value for the three experimental setups was 0.86.

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

Every year thousands of patients are diagnosed with a glioma, a type of malignant brain tumor. MRI plays an essential role in the diagnosis and treatment assessment of these patients. Neural networks show great potential to aid physicians in the medical image analysis. This study investigated the creation of synthetic brain T1-weighted (T1), post-contrast T1-weighted (T1CE), T2-weighted (T2), and T2 Fluid Attenuated Inversion Recovery (Flair) MR images. These synthetic MR (synMR) images were assessed quantitatively with four metrics. The synMR images were also assessed qualitatively by an authoring physician with notions that synMR possessed realism in its portrayal of structural boundaries but struggled to accurately depict tumor heterogeneity. Additionally, this study investigated the synMR images created by generative adversarial network (GAN) to overcome the lack of annotated medical image data in training U-Nets to segment enhancing tumor, whole tumor, and tumor core regions on gliomas. Multiple two-dimensional (2D) U-Nets were trained with original BraTS data and differing subsets of the synMR images. Dice similarity coefficient (DSC) was used as the loss function during training as well a quantitative metric. Additionally, Hausdorff Distance 95% CI (HD) was used to judge the quality of the contours created by these U-Nets. The model performance was improved in both DSC and HD when incorporating synMR in the training set. In summary, this study showed the ability to generate high quality Flair, T2, T1, and T1CE synMR images using GAN. Using synMR images showed encouraging results to improve the U-Net segmentation performance and shows potential to address the scarcity of annotated medical images.

Improving Individual Brain Age Prediction Using an Ensemble Deep Learning Framework

Brain age is an imaging-based biomarker with excellent feasibility for characterizing individual brain health and may serve as a single quantitative index for clinical and domain-specific usage. Brain age has been successfully estimated using extensive neuroimaging data from healthy participants with various feature extraction and conventional machine learning (ML) approaches. Recently, several end-to-end deep learning (DL) analytical frameworks have been proposed as alternative approaches to predict individual brain age with higher accuracy. However, the optimal approach to select and assemble appropriate input feature sets for DL analytical frameworks remains to be determined. In the Predictive Analytics Competition 2019, we proposed a hierarchical analytical framework which first used ML algorithms to investigate the potential contribution of different input features for predicting individual brain age. The obtained information then served as a priori knowledge for determining the input feature sets of the final ensemble DL prediction model. Systematic evaluation revealed that ML approaches with multiple concurrent input features, including tissue volume and density, achieved higher prediction accuracy when compared with approaches with a single input feature set [Ridge regression: mean absolute error (MAE) = 4.51 years, R ² = 0.88; support vector regression, MAE = 4.42 years, R ² = 0.88]. Based on this evaluation, a final ensemble DL brain age prediction model integrating multiple feature sets was constructed with reasonable computation capacity and achieved higher prediction accuracy when compared with ML approaches in the training dataset (MAE = 3.77 years; R ² = 0.90). Furthermore, the proposed ensemble DL brain age prediction model also demonstrated sufficient generalizability in the testing dataset (MAE = 3.33 years). In summary, this study provides initial evidence of how-to efficiency for integrating ML and advanced DL approaches into a unified analytical framework for predicting individual brain age with higher accuracy. With the increase in large open multiple-modality neuroimaging datasets, ensemble DL strategies with appropriate input feature sets serve as a candidate approach for predicting individual brain age in the future.

Synthesis of Prostate MR Images for Classification Using Capsule Network-Based GAN Model

Prostate cancer remains a major health concern among elderly men. Deep learning is a state-of-the-art technique for MR image-based prostate cancer diagnosis, but one of major bottlenecks is the severe lack of annotated MR images. The traditional and Generative Adversarial Network (GAN)-based data augmentation methods cannot ensure the quality and the diversity of generated training samples. In this paper, we have proposed a novel GAN model for synthesis of MR images by utilizing its powerful ability in modeling the complex data distributions. The proposed model is designed based on the architecture of deep convolutional GAN. To learn the more equivariant representation of images that is robust to the changes in the pose and spatial relationship of objects in the images, the capsule network is applied to replace CNN used in the discriminator of regular GAN. Meanwhile, the least squares loss has been adopted for both the generator and discriminator in the proposed GAN to address the vanishing gradient problem of sigmoid cross entropy loss function in regular GAN. Extensive experiments are conducted on the simulated and real MR images. The results demonstrate that the proposed capsule network-based GAN model can generate more realistic and higher quality MR images than the compared GANs. The quantitative comparisons show that among all evaluated models, the proposed GAN generally achieves the smallest Kullback-Leibler divergence values for image generation task and provides the best classification performance when it is introduced into the deep learning method for image classification task.