Generative adversarial networks in medical image segmentation: A review

B-mode ultrasound-based CAD by learning using privileged information with dual-level missing modality completion

Learning using privileged information (LUPI) has shown its effectiveness to improve the B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) by transferring knowledge from the elasticity ultrasound (EUS). However, LUPI only performs transfer learning between the paired data with shared labels, and cannot handle the scenario of modality imbalance. In order to conduct the supervised transfer learning between the paired ultrasound data together with the additional single-modal BUS images, a novel multi-view LUPI algorithm with Dual-Level Modality Completion, named DLMC-LUPI, is proposed to improve the performance of BUS-based CAD. The DLMC-LUPI implements both image-level and feature-level (dual-level) completions of missing EUS modality, and then performs multi-view LUPI for knowledge transfer. Specifically, in the dual-level modality completion stage, a variational autoencoder (VAE) model for feature generation and a novel generative adversarial network (VAE-based GAN) model for image generation are sequentially trained. The proposed VAE-based GAN can improve the synthesis quality of EUS images by adopting the features generated by VAE from the BUS images as the model constrain to make the features generated from the synthesized EUS images more similar to them. In the multi-view LUPI stage, two feature vectors are generated from the real or pseudo images as two source domains, and then fed them to the multi-view support vector machine plus classifier for model training. The experiments on two ultrasound datasets indicate that the DLMC-LUPI outperforms all the compared algorithms, and it can effectively improve the performance of single-modal BUS-based CAD.

Insights and Considerations in Development and Performance Evaluation of Generative Adversarial Networks (GANs): What Radiologists Need to Know

The rapid development of deep learning in medical imaging has significantly enhanced the capabilities of artificial intelligence while simultaneously introducing challenges, including the need for vast amounts of training data and the labor-intensive tasks of labeling and segmentation. Generative adversarial networks (GANs) have emerged as a solution, offering synthetic image generation for data augmentation and streamlining medical image processing tasks through models such as cGAN, CycleGAN, and StyleGAN. These innovations not only improve the efficiency of image augmentation, reconstruction, and segmentation, but also pave the way for unsupervised anomaly detection, markedly reducing the reliance on labeled datasets. Our investigation into GANs in medical imaging addresses their varied architectures, the considerations for selecting appropriate GAN models, and the nuances of model training and performance evaluation. This paper aims to provide radiologists who are new to GAN technology with a thorough understanding, guiding them through the practical application and evaluation of GANs in brain imaging with two illustrative examples using CycleGAN and pixel2style2pixel (pSp)-combined StyleGAN. It offers a comprehensive exploration of the transformative potential of GANs in medical imaging research. Ultimately, this paper strives to equip radiologists with the knowledge to effectively utilize GANs, encouraging further research and application within the field.

Artificial intelligence for treatment delivery: image-guided radiotherapy

Radiation therapy (RT) is a highly digitized field relying heavily on computational methods and, as such, has a high affinity for the automation potential afforded by modern artificial intelligence (AI). This is particularly relevant where imaging is concerned and is especially so during image-guided RT (IGRT). With the advent of online adaptive RT (ART) workflows at magnetic resonance (MR) linear accelerators (linacs) and at cone-beam computed tomography (CBCT) linacs, the need for automation is further increased. AI as applied to modern IGRT is thus one area of RT where we can expect important developments in the near future. In this review article, after outlining modern IGRT and online ART workflows, we cover the role of AI in CBCT and MRI correction for dose calculation, auto-segmentation on IGRT imaging, motion management, and response assessment based on in-room imaging.

A review: artificial intelligence in image-guided spinal surgery

INTRODUCTION

Due to the complex anatomy of the spine and the intricate surgical procedures involved, spinal surgery demands a high level of technical expertise from surgeons. The clinical application of image-guided spinal surgery has significantly enhanced lesion visualization, reduced operation time, and improved surgical outcomes.

AREAS COVERED

This article reviews the latest advancements in deep learning and artificial intelligence in image-guided spinal surgery, aiming to provide references and guidance for surgeons, engineers, and researchers involved in this field.

EXPERT OPINION

Our analysis indicates that image-guided spinal surgery, augmented by artificial intelligence, outperforms traditional spinal surgery techniques. Moving forward, it is imperative to collect a more expansive dataset to further ensure the procedural safety of such surgeries. These insights carry significant implications for the integration of artificial intelligence in the medical field, ultimately poised to enhance the proficiency of surgeons and improve surgical outcomes.

Semi-supervised multi-modal medical image segmentation with unified translation

The two major challenges to deep-learning-based medical image segmentation are multi-modality and a lack of expert annotations. Existing semi-supervised segmentation models can mitigate the problem of insufficient annotations by utilizing a small amount of labeled data. However, most of these models are limited to single-modal data and cannot exploit the complementary information from multi-modal medical images. A few semi-supervised multi-modal models have been proposed recently, but they have rigid structures and require additional training steps for each modality. In this work, we propose a novel flexible method, semi-supervised multi-modal medical image segmentation with unified translation (SMSUT), and a unique semi-supervised procedure that can leverage multi-modal information to improve the semi-supervised segmentation performance. Our architecture capitalizes on unified translation to extract complementary information from multi-modal data which compels the network to focus on the disparities and salient features among each modality. Furthermore, we impose constraints on the model at both pixel and feature levels, to cope with the lack of annotation information and the diverse representations within semi-supervised multi-modal data. We introduce a novel training procedure tailored for semi-supervised multi-modal medical image analysis, by integrating the concept of conditional translation. Our method has a remarkable ability for seamless adaptation to varying numbers of distinct modalities in the training data. Experiments show that our model exceeds the semi-supervised segmentation counterparts in the public datasets which proves our network's high-performance capabilities and the transferability of our proposed method. The code of our method will be openly available at https://github.com/Sue1347/SMSUT-MedicalImgSegmentation.

Hi-gMISnet: generalized medical image segmentation using DWT based multilayer fusion and dual mode attention into high resolutionpGAN

Objective.Automatic medical image segmentation is crucial for accurately isolating target tissue areas in the image from background tissues, facilitating precise diagnoses and procedures. While the proliferation of publicly available clinical datasets led to the development of deep learning-based medical image segmentation methods, a generalized, accurate, robust, and reliable approach across diverse imaging modalities remains elusive.Approach.This paper proposes a novel high-resolution parallel generative adversarial network (pGAN)-based generalized deep learning method for automatic segmentation of medical images from diverse imaging modalities. The proposed method showcases better performance and generalizability by incorporating novel components such as partial hybrid transfer learning, discrete wavelet transform (DWT)-based multilayer and multiresolution feature fusion in the encoder, and a dual mode attention gate in the decoder of the multi-resolution U-Net-based GAN. With multi-objective adversarial training loss functions including a unique reciprocal loss for enforcing cooperative learning inpGANs, it further enhances the robustness and accuracy of the segmentation map.Main results.Experimental evaluations conducted on nine diverse publicly available medical image segmentation datasets, including PhysioNet ICH, BUSI, CVC-ClinicDB, MoNuSeg, GLAS, ISIC-2018, DRIVE, Montgomery, and PROMISE12, demonstrate the proposed method's superior performance. The proposed method achieves mean F1 scores of 79.53%, 88.68%, 82.50%, 93.25%, 90.40%, 94.19%, 81.65%, 98.48%, and 90.79%, respectively, on the above datasets, surpass state-of-the-art segmentation methods. Furthermore, our proposed method demonstrates robust multi-domain segmentation capabilities, exhibiting consistent and reliable performance. The assessment of the model's proficiency in accurately identifying small details indicates that the high-resolution generalized medical image segmentation network (Hi-gMISnet) is more precise in segmenting even when the target area is very small.Significance.The proposed method provides robust and reliable segmentation performance on medical images, and thus it has the potential to be used in a clinical setting for the diagnosis of patients.

A lightweight bladder tumor segmentation method based on attention mechanism

In the endoscopic images of bladder, accurate segmentation of different grade bladder tumor from blurred boundary regions and highly variable shapes is of great significance for doctors' diagnosis and patients' later treatment. We propose a nested attentional feature fusion segmentation network (NAFF-Net) based on the encoder-decoder structure formed by the combination of weighted pyramid pooling module (WPPM) and nested attentional feature fusion (NAFF). Among them, WPPM applies the cascade of atrous convolution to enhance the overall perceptual field while introducing adaptive weights to optimize multi-scale feature extraction, NAFF integrates deep semantic information into shallow feature maps, effectively focusing on edge and detail information in bladder tumor images. Additionally, a weighted mixed loss function is constructed to alleviate the impact of imbalance between positive and negative sample distribution on segmentation accuracy. Experiments illustrate the proposed NAFF-Net achieves better segmentation results compared to other mainstream models, with a MIoU of 84.05%, MPrecision of 91.52%, MRecall of 90.81%, and F1-score of 91.16%, and also achieves good results on the public datasets Kvasir-SEG and CVC-ClinicDB. Compared to other models, NAFF-Net has a smaller number of parameters, which is a significant advantage in model deployment.

A recurrent positional encoding circular attention mechanism network for biomedical image segmentation

Deep-learning-based medical image segmentation techniques can assist doctors in disease diagnosis and rapid treatment. However, existing medical image segmentation models do not fully consider the dependence between feature segments in the feature extraction process, and the correlated features can be further extracted. Therefore, a recurrent positional encoding circular attention mechanism network (RPECAMNet) is proposed based on relative positional encoding for medical image segmentation. Multiple residual modules are used to extract the primary features of the medical images, which are thereafter converted into one-dimensional data for relative positional encoding. The recursive former is used to further extract features from medical images, and decoding is performed using deconvolution. An adaptive loss function is designed to train the model and achieve accurate medical-image segmentation. Finally, the proposed model is used to conduct comparative experiments on the synapse and self-constructed kidney datasets to verify the accuracy of the proposed model for medical image segmentation.

Generative adversarial networks in dental imaging: a systematic review

OBJECTIVES

This systematic review on generative adversarial network (GAN) architectures for dental image analysis provides a comprehensive overview to readers regarding current GAN trends in dental imagery and potential future applications.

METHODS

Electronic databases (PubMed/MEDLINE, Scopus, Embase, and Cochrane Library) were searched to identify studies involving GANs for dental image analysis. Eighteen full-text articles describing the applications of GANs in dental imagery were reviewed. Risk of bias and applicability concerns were assessed using the QUADAS-2 tool.

RESULTS

GANs were used for various imaging modalities, including two-dimensional and three-dimensional images. In dental imaging, GANs were utilized for tasks such as artifact reduction, denoising, and super-resolution, domain transfer, image generation for augmentation, outcome prediction, and identification. The generated images were incorporated into tasks such as landmark detection, object detection and classification. Because of heterogeneity among the studies, a meta-analysis could not be conducted. Most studies (72%) had a low risk of bias in all four domains. However, only three (17%) studies had a low risk of applicability concerns.

CONCLUSIONS

This extensive analysis of GANs in dental imaging highlighted their broad application potential within the dental field. Future studies should address limitations related to the stability, repeatability, and overall interpretability of GAN architectures. By overcoming these challenges, the applicability of GANs in dentistry can be enhanced, ultimately benefiting the dental field in its use of GANs and artificial intelligence.

Impact of bias field correction on 0.35 T pelvic MR images: evaluation on generative adversarial network-based OARs' auto-segmentation and visual grading assessment

Purpose

Magnetic resonance imaging (MRI)-guided radiotherapy enables adaptive treatment plans based on daily anatomical changes and accurate organ visualization. However, the bias field artifact can compromise image quality, affecting diagnostic accuracy and quantitative analyses. This study aims to assess the impact of bias field correction on 0.35 T pelvis MRIs by evaluating clinical anatomy visualization and generative adversarial network (GAN) auto-segmentation performance.

Materials and methods

3D simulation MRIs from 60 prostate cancer patients treated on MR-Linac (0.35 T) were collected and preprocessed with the N4ITK algorithm for bias field correction. A 3D GAN architecture was trained, validated, and tested on 40, 10, and 10 patients, respectively, to auto-segment the organs at risk (OARs) rectum and bladder. The GAN was trained and evaluated either with the original or the bias-corrected MRIs. The Dice similarity coefficient (DSC) and 95th percentile Hausdorff distance (HD95th) were computed for the segmented volumes of each patient. The Wilcoxon signed-rank test assessed the statistical difference of the metrics within OARs, both with and without bias field correction. Five radiation oncologists blindly scored 22 randomly chosen patients in terms of overall image quality and visibility of boundaries (prostate, rectum, bladder, seminal vesicles) of the original and bias-corrected MRIs. Bennett's S score and Fleiss' kappa were used to assess the pairwise interrater agreement and the interrater agreement among all the observers, respectively.

Results

In the test set, the GAN trained and evaluated on original and bias-corrected MRIs showed DSC/HD95th of 0.92/5.63 mm and 0.92/5.91 mm for the bladder and 0.84/10.61 mm and 0.83/9.71 mm for the rectum. No statistical differences in the distribution of the evaluation metrics were found neither for the bladder (DSC: p = 0.07; HD95th: p = 0.35) nor for the rectum (DSC: p = 0.32; HD95th: p = 0.63). From the clinical visual grading assessment, the bias-corrected MRI resulted mostly in either no change or an improvement of the image quality and visualization of the organs' boundaries compared with the original MRI.

Conclusion

The bias field correction did not improve the anatomy visualization from a clinical point of view and the OARs' auto-segmentation outputs generated by the GAN.

Applications of AI in multi-modal imaging for cardiovascular disease

Data for healthcare is diverse and includes many different modalities. Traditional approaches to Artificial Intelligence for cardiovascular disease were typically limited to single modalities. With the proliferation of diverse datasets and new methods in AI, we are now able to integrate different modalities, such as magnetic resonance scans, computerized tomography scans, echocardiography, x-rays, and electronic health records. In this paper, we review research from the last 5 years in applications of AI to multi-modal imaging. There have been many promising results in registration, segmentation, and fusion of different magnetic resonance imaging modalities with each other and computer tomography scans, but there are still many challenges that need to be addressed. Only a few papers have addressed modalities such as x-ray, echocardiography, or non-imaging modalities. As for prediction or classification tasks, there have only been a couple of papers that use multiple modalities in the cardiovascular domain. Furthermore, no models have been implemented or tested in real world cardiovascular clinical settings.

ExS-GAN: Synthesizing Anti-Forensics Images via Extra Supervised GAN

So far, researchers have proposed many forensics tools to protect the authenticity and integrity of digital information. However, with the explosive development of machine learning, existing forensics tools may compromise against new attacks anytime. Hence, it is always necessary to investigate anti-forensics to expose the vulnerabilities of forensics tools. It is beneficial for forensics researchers to develop new tools as countermeasures. To date, one of the potential threats is the generative adversarial networks (GANs), which could be employed for fabricating or forging falsified data to attack forensics detectors. In this article, we investigate the anti-forensics performance of GANs by proposing a novel model, the ExS-GAN, which features an extra supervision system. After training, the proposed model could launch anti-forensics attacks on various manipulated images. Evaluated by experiments, the proposed method could achieve high anti-forensics performance while preserving satisfying image quality. We also justify the proposed extra supervision via an ablation study.

K. SK, Jayaraman S. Automatic foot ulcer segmentation using conditional generative adversarial network (AFSegGAN): A wound management system

Effective wound care is essential to prevent further complications, promote healing, and reduce the risk of infection and other health issues. Chronic wounds, particularly in older adults, patients with disabilities, and those with pressure, venous, or diabetic foot ulcers, cause significant morbidity and mortality. Due to the positive trend in the number of individuals with chronic wounds, particularly among the growing elderly and diabetes populations, it is imperative to develop novel technologies and practices for the best practice clinical management of chronic wounds to minimize the potential health and economic burdens on society. As wound care is managed in hospitals and community care, it is crucial to have quantitative metrics like wound boundary and morphological features. The traditional visual inspection technique is purely subjective and error-prone, and digitization provides an appealing alternative. Various deep-learning models have earned confidence; however, their accuracy primarily relies on the image quality, the dataset size to learn the features, and experts' annotation. This work aims to develop a wound management system that automates wound segmentation using a conditional generative adversarial network (cGAN) and estimate the wound morphological parameters. AFSegGAN was developed and validated on the MICCAI 2021-foot ulcer segmentation dataset. In addition, we use adversarial loss and patch-level comparison at the discriminator network to improve the segmentation performance and balance the GAN network training. Our model outperformed state-of-the-art methods with a Dice score of 93.11% and IoU of 99.07%. The proposed wound management system demonstrates its abilities in wound segmentation and parameter estimation, thereby reducing healthcare workers' efforts to diagnose or manage wounds and facilitating remote healthcare.

MUE-CoT: multi-scale uncertainty entropy-aware co-training framework for left atrial segmentation

Objective.Accurate left atrial segmentation is the basis of the recognition and clinical analysis of atrial fibrillation. Supervised learning has achieved some competitive segmentation results, but the high annotation cost often limits its performance. Semi-supervised learning is implemented from limited labeled data and a large amount of unlabeled data and shows good potential in solving practical medical problems.Approach. In this study, we proposed a collaborative training framework for multi-scale uncertain entropy perception (MUE-CoT) and achieved efficient left atrial segmentation from a small amount of labeled data. Based on the pyramid feature network, learning is implemented from unlabeled data by minimizing the pyramid prediction difference. In addition, novel loss constraints are proposed for co-training in the study. The diversity loss is defined as a soft constraint so as to accelerate the convergence and a novel multi-scale uncertainty entropy calculation method and a consistency regularization term are proposed to measure the consistency between prediction results. The quality of pseudo-labels cannot be guaranteed in the pre-training period, so a confidence-dependent empirical Gaussian function is proposed to weight the pseudo-supervised loss.Main results.The experimental results of a publicly available dataset and an in-house clinical dataset proved that our method outperformed existing semi-supervised methods. For the two datasets with a labeled ratio of 5%, the Dice similarity coefficient scores were 84.94% ± 4.31 and 81.24% ± 2.4, the HD95values were 4.63 mm ± 2.13 and 3.94 mm ± 2.72, and the Jaccard similarity coefficient scores were 74.00% ± 6.20 and 68.49% ± 3.39, respectively.Significance.The proposed model effectively addresses the challenges of limited data samples and high costs associated with manual annotation in the medical field, leading to enhanced segmentation accuracy.

High-Density Functional Near-Infrared Spectroscopy and Machine Learning for Visual Perception Quantification

The main application scenario for wearable sensors involves the generation of data and monitoring metrics. fNIRS (functional near-infrared spectroscopy) allows the nonintrusive monitoring of human visual perception. The quantification of visual perception by fNIRS facilitates applications in engineering-related fields. This study designed a set of experimental procedures to effectively induce visible alterations and to quantify visual perception in conjunction with the acquisition of Hbt (total hemoglobin), Hb (hemoglobin), and HbO2 (oxygenated hemoglobin) data obtained from HfNIRS (high-density functional near-infrared spectroscopy). Volunteers completed the visual task separately in response to different visible changes in the simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visual difference, and the time point of the task change. This study consisted of one simulated scene, two visual variations, and four visual tasks. The simulation scene featured a car driving location. The visible change suggested that the brightness and saturation of the car operator interface would change. The visual task represented the completion of the layout, color, design, and information questions answered in response to the visible change. This study collected data from 29 volunteers. The volunteers completed the visual task separately in response to different visual changes in the same simulated scene. HfNIRS recorded the changes in Hbt, Hb, and HbO2 during the study, the time point of the visible difference, and the time point of the task change. The data analysis methods in this study comprised a combination of channel dimensionality reduction, feature extraction, task classification, and score correlation. Channel downscaling: This study used the data of 15 channels in HfNIRS to calculate the mutual information between different channels to set a threshold, and to retain the data of the channels that were higher than those of the mutual information. Feature extraction: The statistics derived from the visual task, including time, mean, median, variance, extreme variance, kurtosis, bias, information entropy, and approximate entropy were computed. Task classification: This study used the KNN (K-Nearest Neighbors) algorithm to classify different visual tasks and to calculate the accuracy, precision, recall, and F1 scores. Scoring correlation: This study matched the visual task scores with the fluctuations of Hbt, Hb, and HbO2 and observed the changes in Hbt, Hb, and HbO2 under different scoring levels. Mutual information was used to downscale the channels, and seven channels were retained for analysis under each visual task. The average accuracy was 96.3% ± 1.99%; the samples that correctly classified the visual task accounted for 96.3% of the total; and the classification accuracy was high. By analyzing the correlation between the scores on different visual tasks and the fluctuations of Hbt, Hb, and HbO2, it was found that the higher the score, the more obvious, significant, and higher the fluctuations of Hbt, Hb, and HbO2. Experiments found that changes in visual perception triggered changes in Hbt, Hb, and HbO2. HfNIRS combined with Hbt, Hb, and HbO2 recorded by machine learning algorithms can effectively quantify visual perception. However, the related research in this paper still needs to be further refined, and the mathematical relationship between HfNIRS and visual perception needs to be further explored to realize the quantitative study of subjective and objective visual perception supported by the mathematical relationship.

Evaluating the performance of generative adversarial network-synthesized periapical images in classifying C-shaped root canals

This study evaluated the performance of generative adversarial network (GAN)-synthesized periapical images for classifying C-shaped root canals, which are challenging to diagnose because of their complex morphology. GANs have emerged as a promising technique for generating realistic images, offering a potential solution for data augmentation in scenarios with limited training datasets. Periapical images were synthesized using the StyleGAN2-ADA framework, and their quality was evaluated based on the average Frechet inception distance (FID) and the visual Turing test. The average FID was found to be 35.353 (± 4.386) for synthesized C-shaped canal images and 25.471 (± 2.779) for non C-shaped canal images. The visual Turing test conducted by two radiologists on 100 randomly selected images revealed that distinguishing between real and synthetic images was difficult. These results indicate that GAN-synthesized images exhibit satisfactory visual quality. The classification performance of the neural network, when augmented with GAN data, showed improvements compared with using real data alone, and could be advantageous in addressing data conditions with class imbalance. GAN-generated images have proven to be an effective data augmentation method, addressing the limitations of limited training data and computational resources in diagnosing dental anomalies.

Placental Vessel Segmentation Using Pix2pix Compared to U-Net

Computer-assisted technologies have made significant progress in fetoscopic laser surgery, including placental vessel segmentation. However, the intra- and inter-procedure variabilities in the state-of-the-art segmentation methods remain a significant hurdle. To address this, we investigated the use of conditional generative adversarial networks (cGANs) for fetoscopic image segmentation and compared their performance with the benchmark U-Net technique for placental vessel segmentation. Two deep-learning models, U-Net and pix2pix (a popular cGAN model), were trained and evaluated using a publicly available dataset and an internal validation set. The overall results showed that the pix2pix model outperformed the U-Net model, with a Dice score of 0.80 [0.70; 0.86] versus 0.75 [0.0.60; 0.84] (p-value < 0.01) and an Intersection over Union (IoU) score of 0.70 [0.61; 0.77] compared to 0.66 [0.53; 0.75] (p-value < 0.01), respectively. The internal validation dataset further validated the superiority of the pix2pix model, achieving Dice and IoU scores of 0.68 [0.53; 0.79] and 0.59 [0.49; 0.69] (p-value < 0.01), respectively, while the U-Net model obtained scores of 0.53 [0.49; 0.64] and 0.49 [0.17; 0.56], respectively. This study successfully compared U-Net and pix2pix models for placental vessel segmentation in fetoscopic images, demonstrating improved results with the cGAN-based approach. However, the challenge of achieving generalizability still needs to be addressed.

Multichannel One-Dimensional Data Augmentation with Generative Adversarial Network

Data augmentation is one of the most important problems in deep learning. There have been many algorithms proposed to solve this problem, such as simple noise injection, the generative adversarial network (GAN), and diffusion models. However, to the best of our knowledge, these works mainly focused on computer vision-related tasks, and there have not been many proposed works for one-dimensional data. This paper proposes a GAN-based data augmentation for generating multichannel one-dimensional data given single-channel inputs. Our architecture consists of multiple discriminators that adapt deep convolution GAN (DCGAN) and patchGAN to extract the overall pattern of the multichannel generated data while also considering the local information of each channel. We conducted an experiment with website fingerprinting data. The result for the three channels' data augmentation showed that our proposed model obtained FID scores of 0.005,0.017,0.051 for each channel, respectively, compared to 0.458,0.551,0.521 when using the vanilla GAN.

CHSNet: Automatic lesion segmentation network guided by CT image features for acute cerebral hemorrhage

Stroke is a cerebrovascular disease that can lead to severe sequelae such as hemiplegia and mental retardation with a mortality rate of up to 40%. In this paper, we proposed an automatic segmentation network (CHSNet) to segment the lesions in cranial CT images based on the characteristics of acute cerebral hemorrhage images, such as high density, multi-scale, and variable location, and realized the three-dimensional (3D) visualization and localization of the cranial lesions after the segmentation was completed. To enhance the feature representation of high-density regions, and capture multi-scale and up-down information on the target location, we constructed a convolutional neural network with encoding-decoding backbone, Res-RCL module, Atrous Spatial Pyramid Pooling, and Attention Gate. We collected images of 203 patients with acute cerebral hemorrhage, constructed a dataset containing 5998 cranial CT slices, and conducted comparative and ablation experiments on the dataset to verify the effectiveness of our model. Our model achieved the best results on both test sets with different segmentation difficulties, test1: Dice = 0.918, IoU = 0.853, ASD = 0.476, RVE = 0.113; test2: Dice = 0.716, IoU = 0.604, ASD = 5.402, RVE = 1.079. Based on the segmentation results, we achieved 3D visualization and localization of hemorrhage in CT images of stroke patients. The study has important implications for clinical adjuvant diagnosis.

Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks

Segmentation of tumor regions in H &E-stained slides is an important task for a pathologist while diagnosing different types of cancer, including oral squamous cell carcinoma (OSCC). Histological image segmentation is often constrained by the availability of labeled training data since labeling histological images is a highly skilled, complex, and time-consuming task. Thus, data augmentation strategies become essential to train convolutional neural networks models to overcome the overfitting problem when only a few training samples are available. This paper proposes a new data augmentation strategy, named Random Composition Augmentation (RCAug), to train fully convolutional networks (FCN) to segment OSCC tumor regions in H &E-stained histological images. Given the input image and their corresponding label, a pipeline with a random composition of geometric, distortion, color transfer, and generative image transformations is executed on the fly. Experimental evaluations were performed using an FCN-based method to segment OSCC regions through a set of different data augmentation transformations. By using RCAug, we improved the FCN-based segmentation method from 0.51 to 0.81 of intersection-over-union (IOU) in a whole slide image dataset and from 0.65 to 0.69 of IOU in a tissue microarray images dataset.

Retinal Vascular Image Segmentation Using Improved UNet Based on Residual Module

In recent years, deep learning technology for clinical diagnosis has progressed considerably, and the value of medical imaging continues to increase. In the past, clinicians evaluated medical images according to their individual expertise. In contrast, the application of artificial intelligence technology for automatic analysis and diagnostic assistance to support clinicians in evaluating medical information more efficiently has become an important trend. In this study, we propose a machine learning architecture designed to segment images of retinal blood vessels based on an improved U-Net neural network model. The proposed model incorporates a residual module to extract features more effectively, and includes a full-scale skip connection to combine low level details with high-level features at different scales. The results of an experimental evaluation show that the model was able to segment images of retinal vessels accurately. The proposed method also outperformed several existing models on the benchmark datasets DRIVE and ROSE, including U-Net, ResUNet, U-Net3+, ResUNet++, and CaraNet.

An Improved Co-Training and Generative Adversarial Network (Diff-CoGAN) for Semi-Supervised Medical Image Segmentation

Semi-supervised learning is a technique that utilizes a limited set of labeled data and a large amount of unlabeled data to overcome the challenges of obtaining a perfect dataset in deep learning, especially in medical image segmentation. The accuracy of the predicted labels for the unlabeled data is a critical factor that affects the training performance, thus reducing the accuracy of segmentation. To address this issue, a semi-supervised learning method based on the Diff-CoGAN framework was proposed, which incorporates co-training and generative adversarial network (GAN) strategies. The proposed Diff-CoGAN framework employs two generators and one discriminator. The generators work together by providing mutual information guidance to produce predicted maps that are more accurate and closer to the ground truth. To further improve segmentation accuracy, the predicted maps are subjected to an intersection operation to identify a high-confidence region of interest, which reduces boundary segmentation errors. The predicted maps are then fed into the discriminator, and the iterative process of adversarial training enhances the generators’ ability to generate more precise maps, while also improving the discriminator’s ability to distinguish between the predicted maps and the ground truth. This study conducted experiments on the Hippocampus and Spleen images from the Medical Segmentation Decathlon (MSD) dataset using three semi-supervised methods: co-training, semi-GAN, and Diff-CoGAN. The experimental results demonstrated that the proposed Diff-CoGAN approach significantly enhanced segmentation accuracy compared to the other two methods by benefiting on the mutual guidance of the two generators and the adversarial training between the generators and discriminator. The introduction of the intersection operation prior to the discriminator also further reduced boundary segmentation errors.

CellTranspose: Few-shot Domain Adaptation for Cellular Instance Segmentation

Automated cellular instance segmentation is a process utilized for accelerating biological research for the past two decades, and recent advancements have produced higher quality results with less effort from the biologist. Most current endeavors focus on completely cutting the researcher out of the picture by generating highly generalized models. However, these models invariably fail when faced with novel data, distributed differently than the ones used for training. Rather than approaching the problem with methods that presume the availability of large amounts of target data and computing power for retraining, in this work we address the even greater challenge of designing an approach that requires minimal amounts of new annotated data as well as training time. We do so by designing specialized contrastive losses that leverage the few annotated samples very efficiently. A large set of results show that 3 to 5 annotations lead to models with accuracy that: 1) significantly mitigate the covariate shift effects; 2) matches or surpasses other adaptation methods; 3) even approaches methods that have been fully retrained on the target distribution. The adaptation training is only a few minutes, paving a path towards a balance between model performance, computing requirements and expert-level annotation needs.

E-DU: Deep neural network for multimodal medical image segmentation based on semantic gap compensation

BACKGROUND

U-Net includes encoder, decoder and skip connection structures. It has become the benchmark network in medical image segmentation. However, the direct fusion of low-level and high-level convolution features with semantic gaps by traditional skip connections may lead to problems such as fuzzy generated feature maps and target region segmentation errors.

OBJECTIVE

We use spatial enhancement filtering technology to compensate for the semantic gap and propose an enhanced dense U-Net (E-DU), aiming to apply it to multimodal medical image segmentation to improve the segmentation performance and efficiency.

METHODS

Before combining encoder and decoder features, we replace the traditional skip connection with a multiscale denoise enhancement (MDE) module. The encoder features need to be deeply convolved by the spatial enhancement filter and then combined with the decoder features. We propose a simple and efficient deep full convolution network structure E-DU, which can not only fuse semantically various features but also denoise and enhance the feature map.

RESULTS

We performed experiments on medical image segmentation datasets with seven image modalities and combined MDE with various baseline networks to perform ablation studies. E-DU achieved the best segmentation results on evaluation indicators such as DSC on the U-Net family, with DSC values of 97.78, 97.64, 95.31, 94.42, 94.93, 98.85, and 98.38 (%), respectively. The addition of the MDE module to the attention mechanism network improves segmentation performance and efficiency, reflecting its generalization performance. In comparison to advanced methods, our method is also competitive.

CONCLUSION

Our proposed MDE module has a good segmentation effect and operating efficiency, and it can be easily extended to multiple modal medical segmentation datasets. Our idea and method can achieve clinical multimodal medical image segmentation and make full use of image information to provide clinical decision support. It has great application value and promotion prospects.

Introducing AI to the molecular tumor board: one direction toward the establishment of precision medicine using large-scale cancer clinical and biological information

Since U.S. President Barack Obama announced the Precision Medicine Initiative in his New Year's State of the Union address in 2015, the establishment of a precision medicine system has been emphasized worldwide, particularly in the field of oncology. With the advent of next-generation sequencers specifically, genome analysis technology has made remarkable progress, and there are active efforts to apply genome information to diagnosis and treatment. Generally, in the process of feeding back the results of next-generation sequencing analysis to patients, a molecular tumor board (MTB), consisting of experts in clinical oncology, genetic medicine, etc., is established to discuss the results. On the other hand, an MTB currently involves a large amount of work, with humans searching through vast databases and literature, selecting the best drug candidates, and manually confirming the status of available clinical trials. In addition, as personalized medicine advances, the burden on MTB members is expected to increase in the future. Under these circumstances, introducing cutting-edge artificial intelligence (AI) technology and information and communication technology to MTBs while reducing the burden on MTB members and building a platform that enables more accurate and personalized medical care would be of great benefit to patients. In this review, we introduced the latest status of elemental technologies that have potential for AI utilization in MTB, and discussed issues that may arise in the future as we progress with AI implementation.

FetalGAN: Automated Segmentation of Fetal Functional Brain MRI Using Deep Generative Adversarial Learning and Multi-Scale 3D U-Net

An important step in the preprocessing of resting state functional magnetic resonance images (rs-fMRI) is the separation of brain from non-brain voxels. Widely used imaging tools such as FSL's BET2 and AFNI's 3dSkullStrip accomplish this task effectively in children and adults. In fetal functional brain imaging, however, the presence of maternal tissue around the brain coupled with the non-standard position of the fetal head limit the usefulness of these tools. Accurate brain masks are thus generated manually, a time-consuming and tedious process that slows down preprocessing of fetal rs-fMRI. Recently, deep learning-based segmentation models such as convolutional neural networks (CNNs) have been increasingly used for automated segmentation of medical images, including the fetal brain. Here, we propose a computationally efficient end-to-end generative adversarial neural network (GAN) for segmenting the fetal brain. This method, which we call FetalGAN, yielded whole brain masks that closely approximated the manually labeled ground truth. FetalGAN performed better than 3D U-Net model and BET2: FetalGAN, Dice score = 0.973 ± 0.013, precision = 0.977 ± 0.015; 3D U-Net, Dice score = 0.954 ± 0.054, precision = 0.967 ± 0.037; BET2, Dice score = 0.856 ± 0.084, precision = 0.758 ± 0.113. FetalGAN was also faster than 3D U-Net and the manual method (7.35 s vs. 10.25 s vs. ∼5 min/volume). To the best of our knowledge, this is the first successful implementation of 3D CNN with GAN on fetal fMRI brain images and represents a significant advance in fully automating processing of rs-MRI images.

Facial skin disease prediction using StarGAN v2 and transfer learning

Deep learning algorithms have become the most prominent methods for medical image analysis over the past years, leading to enhanced performances in various medical applications. In this paper, we focus on applying intelligent skin disease detection to face images, where the crucial challenge is the low availability of training data. To achieve high disease detection and classification success rates, we adapt the state-of-the-art StarGAN v2 network to augment images of faces and combine it with a transfer learning approach. The experimental results show that the classification accuracies of transfer learning models are in the range of 77.46–99.80% when trained on datasets that are extended with StarGAN v2 augmented data.