Generative adversarial network-based synthesis of contrast-enhanced MR images from precontrast images for predicting histological characteristics in breast cancer

Objective. Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a sensitive tool for assessing breast cancer by analyzing tumor blood flow, but it requires gadolinium-based contrast agents, which carry risks such as brain retention and astrocyte migration. Contrast-free MRI is thus preferable for patients with renal impairment or who are pregnant. This study aimed to investigate the feasibility of generating contrast-enhanced MR images from precontrast images and to evaluate the potential use of synthetic images in diagnosing breast cancer.Approach. This retrospective study included 322 women with invasive breast cancer who underwent preoperative DCE-MRI. A generative adversarial network (GAN) based postcontrast image synthesis (GANPIS) model with perceptual loss was proposed to generate contrast-enhanced MR images from precontrast images. The quality of the synthesized images was evaluated using the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The diagnostic performance of the generated images was assessed using a convolutional neural network to predict Ki-67, luminal A and histological grade with the area under the receiver operating characteristic curve (AUC). The patients were divided into training (n= 200), validation (n= 60), and testing sets (n= 62).Main results. Quantitative analysis revealed strong agreement between the generated and real postcontrast images in the test set, with PSNR and SSIM values of 36.210 ± 2.670 and 0.988 ± 0.006, respectively. The generated postcontrast images achieved AUCs of 0.918 ± 0.018, 0.842 ± 0.028 and 0.815 ± 0.019 for predicting the Ki-67 expression level, histological grade, and luminal A subtype, respectively. These results showed a significant improvement compared to the use of precontrast images alone, which achieved AUCs of 0.764 ± 0.031, 0.741 ± 0.035, and 0.797 ± 0.021, respectively.Significance. This study proposed a GAN-based MR image synthesis method for breast cancer that aims to generate postcontrast images from precontrast images, allowing the use of contrast-free images to simulate kinetic features for improved diagnosis.

A Stage-Wise Residual Attention Generation Adversarial Network for Mandibular Defect Repairing and Reconstruction

Surgical reconstruction of mandibular defects is a clinical routine manner for the rehabilitation of patients with deformities. The mandible plays a crucial role in maintaining the facial contour and ensuring the speech and mastication functions. The repairing and reconstruction of mandible defects is a significant yet challenging task in oral-maxillofacial surgery. Currently, the mainly available methods are traditional digitalized design methods that suffer from substantial artificial operations, limited applicability and high reconstruction error rates. An automated, precise, and individualized method is imperative for maxillofacial surgeons. In this paper, we propose a Stage-wise Residual Attention Generative Adversarial Network (SRA-GAN) for mandibular defect reconstruction. Specifically, we design a stage-wise residual attention mechanism for generator to enhance the extraction capability of mandibular remote spatial information, making it adaptable to various defects. For the discriminator, we propose a multi-field perceptual network, consisting of two parallel discriminators with different perceptual fields, to reduce the cumulative reconstruction errors. Furthermore, we design a self-encoder perceptual loss function to ensure the correctness of mandibular anatomical structures. The experimental results on a novel custom-built mandibular defect dataset demonstrate that our method has a promising prospect in clinical application, achieving the best Dice Similarity Coefficient (DSC) of 94.238% and 95% Hausdorff Distance (HD95) of 4.787.

Enhancing molecular design efficiency: Uniting language models and generative networks with genetic algorithms

This study examines the effectiveness of generative models in drug discovery, material science, and polymer science, aiming to overcome constraints associated with traditional inverse design methods relying on heuristic rules. Generative models generate synthetic data resembling real data, enabling deep learning model training without extensive labeled datasets. They prove valuable in creating virtual libraries of molecules for material science and facilitating drug discovery by generating molecules with specific properties. While generative adversarial networks (GANs) are explored for these purposes, mode collapse restricts their efficacy, limiting novel structure variability. To address this, we introduce a masked language model (LM) inspired by natural language processing. Although LMs alone can have inherent limitations, we propose a hybrid architecture combining LMs and GANs to efficiently generate new molecules, demonstrating superior performance over standalone masked LMs, particularly for smaller population sizes. This hybrid LM-GAN architecture enhances efficiency in optimizing properties and generating novel samples.

Application of Self-Attention Generative Adversarial Network for Electromagnetic Imaging in Half-Space

In this paper, we introduce a novel artificial intelligence technique with an attention mechanism for half-space electromagnetic imaging. A dielectric object in half-space is illuminated by TM (transverse magnetic) waves. Since measurements can only be made in the upper space, the measurement angle will be limited. As a result, we apply a back-propagation scheme (BPS) to generate an initial guessed image from the measured scattered fields for scatterer buried in the lower half-space. This process can effectively reduce the high nonlinearity of the inverse scattering problem. We further input the guessed images into the generative adversarial network (GAN) and the self-attention generative adversarial network (SAGAN), respectively, to compare the reconstruction performance. Numerical results prove that both SAGAN and GAN can reconstruct dielectric objects and the MNIST dataset under same measurement conditions. Our analysis also reveals that SAGAN is able to reconstruct electromagnetic images more accurately and efficiently than GAN.

Maximum Classifier Discrepancy Generative Adversarial Network for Jointly Harmonizing Scanner Effects and Improving Reproducibility of Downstream Tasks

OBJECTIVE

Multi-site collaboration is essential for overcoming small-sample problems when exploring reproducible biomarkers in MRI studies. However, various scanner-specific factors dramatically reduce the cross-scanner replicability. Moreover, existing harmony methods mostly could not guarantee the improved performance of downstream tasks.

METHODS

we proposed a new multi-scanner harmony framework, called 'maximum classifier discrepancy generative adversarial network', or MCD-GAN, for removing scanner effects in the original feature space while preserving substantial biological information for downstream tasks. Specifically, the adversarial generative network was utilized for persisting the structural layout of each sample, and the maximum classifier discrepancy module was introduced for regulating GAN generators by incorporating the downstream tasks.

RESULTS

We compared the MCD-GAN with other state-of-the-art data harmony approaches (e.g., ComBat, CycleGAN) on simulated data and the Adolescent Brain Cognitive Development (ABCD) dataset. Results demonstrate that MCD-GAN outperformed other approaches in improving cross-scanner classification performance while preserving the anatomical layout of the original images.

SIGNIFICANCE

To the best of our knowledge, the proposed MCD-GAN is the first generative model which incorporates downstream tasks while harmonizing, and is a promising solution for facilitating cross-site reproducibility in various tasks such as classification and regression.

A systematic literature review: deep learning techniques for synthetic medical image generation and their applications in radiotherapy

The aim of this systematic review is to determine whether Deep Learning (DL) algorithms can provide a clinically feasible alternative to classic algorithms for synthetic Computer Tomography (sCT). The following categories are presented in this study: ∙ MR-based treatment planning and synthetic CT generation techniques. ∙ Generation of synthetic CT images based on Cone Beam CT images. ∙ Low-dose CT to High-dose CT generation. ∙ Attenuation correction for PET images. To perform appropriate database searches, we reviewed journal articles published between January 2018 and June 2023. Current methodology, study strategies, and results with relevant clinical applications were analyzed as we outlined the state-of-the-art of deep learning based approaches to inter-modality and intra-modality image synthesis. This was accomplished by contrasting the provided methodologies with traditional research approaches. The key contributions of each category were highlighted, specific challenges were identified, and accomplishments were summarized. As a final step, the statistics of all the cited works from various aspects were analyzed, which revealed that DL-based sCTs have achieved considerable popularity, while also showing the potential of this technology. In order to assess the clinical readiness of the presented methods, we examined the current status of DL-based sCT generation.

Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept

[18F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

GAN-based metal artifacts region inpainting in brain MRI imaging with reflective registration

BACKGROUND AND OBJECTIVE

Metallic magnetic resonance imaging (MRI) implants can introduce magnetic field distortions, resulting in image distortion, such as bulk shifts and signal-loss artifacts. Metal Artifacts Region Inpainting Network (MARINet), using the symmetry of brain MRI images, has been developed to generate normal MRI images in the image domain and improve image quality.

METHODS

T1-weighted MRI images containing or located near the teeth of 100 patients were collected. A total of 9000 slices were obtained after data augmentation. Then, MARINet based on U-Net with a dual-path encoder was employed to inpaint the artifacts in MRI images. The input of MARINet contains the original image and the flipped registered image, with partial convolution used concurrently. Subsequently, we compared PConv with partial convolution, and GConv with gated convolution, SDEdit using a diffusion model for inpainting the artifact region of MRI images. The mean absolute error (MAE) and peak signal-to-noise ratio (PSNR) for the mask were used to compare the results of these methods. In addition, the artifact masks of clinical MRI images were inpainted by physicians.

RESULTS

MARINet could directly and effectively inpaint the incomplete MRI images generated by masks in the image domain. For the test results of PConv, GConv, SDEdit, and MARINet, the masked MAEs were 0.1938, 0.1904, 0.1876, and 0.1834, respectively, and the masked PSNRs were 17.39, 17.40, 17.49, and 17.60 dB, respectively. The visualization results also suggest that the network can recover the tissue texture, alveolar shape, and tooth contour. Additionally, for clinical artifact MRI images, MARINet completed the artifact region inpainting task more effectively when compared with other models.

CONCLUSIONS

By leveraging the quasi-symmetry of brain MRI images, MARINet can directly and effectively inpaint the metal artifacts in MRI images in the image domain, restoring the tooth contour and detail, thereby enhancing the image quality.

Structurally constrained and pathology-aware convolutional transformer generative adversarial network for virtual histology staining of human coronary optical coherence tomography images

Significance

There is a significant need for the generation of virtual histological information from coronary optical coherence tomography (OCT) images to better guide the treatment of coronary artery disease (CAD). However, existing methods either require a large pixel-wise paired training dataset or have limited capability to map pathological regions.

Aim

The aim of this work is to generate virtual histological information from coronary OCT images, without a pixel-wise paired training dataset while capable of providing pathological patterns.

Approach

We design a structurally constrained, pathology-aware, transformer generative adversarial network, namely structurally constrained pathology-aware convolutional transformer generative adversarial network (SCPAT-GAN), to generate virtual stained H&E histology from OCT images. We quantitatively evaluate the quality of virtual stained histology images by measuring the Fréchet inception distance (FID) and perceptual hash value (PHV). Moreover, we invite experienced pathologists to evaluate the virtual stained images. Furthermore, we visually inspect the virtual stained image generated by SCPAT-GAN. Also, we perform an ablation study to validate the design of the proposed SCPAT-GAN. Finally, we demonstrate 3D virtual stained histology images.

Results

Compared to previous research, the proposed SCPAT-GAN achieves better FID and PHV scores. The visual inspection suggests that the virtual histology images generated by SCPAT-GAN resemble both normal and pathological features without artifacts. As confirmed by the pathologists, the virtual stained images have good quality compared to real histology images. The ablation study confirms the effectiveness of the combination of proposed pathological awareness and structural constraining modules.

Conclusions

The proposed SCPAT-GAN is the first to demonstrate the feasibility of generating both normal and pathological patterns without pixel-wisely supervised training. We expect the SCPAT-GAN to assist in the clinical evaluation of treating the CAD by providing 2D and 3D histopathological visualizations.

Remaining Useful Life Estimation of MoSi2 Heating Element in a Pusher Kiln Process

The critical challenge of estimating the Remaining Useful Life (RUL) of MoSi2 heating elements utilized in pusher kiln processes is to enhance operational efficiency and minimize downtime in industrial applications. MoSi2 heating elements are integral components in high-temperature environments, playing a pivotal role in achieving optimal thermal performance. However, prolonged exposure to extreme conditions leads to degradation, necessitating precise RUL predictions for proactive maintenance strategies. Since insufficient failure experience deals with Predictive Maintenance (PdM) in real-life scenarios, a Generative Adversarial Network (GAN) generates specific training data as failure experiences. The Remaining Useful Life (RUL) is the duration of the equipment's operation before repair or replacement, often measured in days, miles, or cycles. Machine learning models are trained using historical data encompassing various operational scenarios and degradation patterns. The RUL prediction model is determined through training, hyperparameter tuning, and comparisons based on the machine-learning model, such as Long Short-Term Memory (LSTM) or Support Vector Regression (SVR). As a result, SVR reflects the actual resistance variation, achieving the R-Square (R2) of 0.634, better than LSTM. From a safety perspective, SVR offers high prediction accuracy and sufficient time to schedule maintenance plans.

Multi-Object Trajectory Prediction Based on Lane Information and Generative Adversarial Network

Nowadays, most trajectory prediction algorithms have difficulty simulating actual traffic behavior, and there is still a problem of large prediction errors. Therefore, this paper proposes a multi-object trajectory prediction algorithm based on lane information and foresight information. A Hybrid Dilated Convolution module based on the Channel Attention mechanism (CA-HDC) is developed to extract features, which improves the lane feature extraction in complicated environments and solves the problem of poor robustness of the traditional PINet. A lane information fusion module and a trajectory adjustment module based on the foresight information are developed. A socially acceptable trajectory with Generative Adversarial Networks (S-GAN) is developed to reduce the error of the trajectory prediction algorithm. The lane detection accuracy in special scenarios such as crowded, shadow, arrow, crossroad, and night are improved on the CULane dataset. The average F1-measure of the proposed lane detection has been increased by 4.1% compared to the original PINet. The trajectory prediction test based on D2-City indicates that the average displacement error of the proposed trajectory prediction algorithm is reduced by 4.27%, and the final displacement error is reduced by 7.53%. The proposed algorithm can achieve good results in lane detection and multi-object trajectory prediction tasks.

ECG data enhancement method using generate adversarial networks based on Bi-LSTM and CBAM

Objective.The classification performance of electrocardiogram (ECG) classification algorithms is easily affected by data imbalance, which often leads to poor model prediction performance for a few classes and a consequent decrease in the overall performance of the model.Approach.To address this problem, this paper proposed an ECG data augmentation method based on a generative adversarial network (GAN) that combines bidirectional long short-term memory (Bi-LSTM) networks and convolutional block attention mechanism (CBAM) to improve the overall performance of ECG classification models. In this paper, we used two ECG databases, namely the MIT-BIH arrhythmia (MIT-BIH-AR) database and the Chinese cardiovascular disease database (CCDD). The quality of the ECG signals produced by the generated models was assessed using the percent relative difference, root mean square error, Frechet distance, dynamic time warping (DTW), and Pearson correlation metrics. In addition, we also validated the impact of our proposed data augmentation method on ECG classification performance on MIT-BIH-AR database and CCDD.Main results.On the MIT-BIH-AR database, the overall accuracy of the data-enhanced balanced dataset was improved to 99.46% for 15 types of heartbeat classification task. On the CCDD, which focuses on the detection of ventricular precession (PVC), the overall accuracy of PVC detection improved to 99.15% after performing data enhancement.Significance.The experimental results indicate that the data augmentation method proposed in this paper can further improve the ECG classification performance.

Smart Grid Security: An Effective Hybrid CNN-Based Approach for Detecting Energy Theft Using Consumption Patterns

In Internet of Things-based smart grids, smart meters record and report a massive number of power consumption data at certain intervals to the data center of the utility for load monitoring and energy management. Energy theft is a big problem for smart meters and causes non-technical losses. Energy theft attacks can be launched by malicious consumers by compromising the smart meters to report manipulated consumption data for less billing. It is a global issue causing technical and financial damage to governments and operators. Deep learning-based techniques can effectively identify consumers involved in energy theft through power consumption data. In this study, a hybrid convolutional neural network (CNN)-based energy-theft-detection system is proposed to detect data-tampering cyber-attack vectors. CNN is a commonly employed method that automates the extraction of features and the classification process. We employed CNN for feature extraction and traditional machine learning algorithms for classification. In this work, honest data were obtained from a real dataset. Six attack vectors causing data tampering were utilized. Tampered data were synthetically generated through these attack vectors. Six separate datasets were created for each attack vector to design a specialized detector tailored for that specific attack. Additionally, a dataset containing all attack vectors was also generated for the purpose of designing a general detector. Furthermore, the imbalanced dataset problem was addressed through the application of the generative adversarial network (GAN) method. GAN was chosen due to its ability to generate new data closely resembling real data, and its application in this field has not been extensively explored. The data generated with GAN ensured better training for the hybrid CNN-based detector on honest and malicious consumption patterns. Finally, the results indicate that the proposed general detector could classify both honest and malicious users with satisfactory accuracy.

Generative artificial intelligence in drug discovery: basic framework, recent advances, challenges, and opportunities

There are two main ways to discover or design small drug molecules. The first involves fine-tuning existing molecules or commercially successful drugs through quantitative structure-activity relationships and virtual screening. The second approach involves generating new molecules through de novo drug design or inverse quantitative structure-activity relationship. Both methods aim to get a drug molecule with the best pharmacokinetic and pharmacodynamic profiles. However, bringing a new drug to market is an expensive and time-consuming endeavor, with the average cost being estimated at around $2.5 billion. One of the biggest challenges is screening the vast number of potential drug candidates to find one that is both safe and effective. The development of artificial intelligence in recent years has been phenomenal, ushering in a revolution in many fields. The field of pharmaceutical sciences has also significantly benefited from multiple applications of artificial intelligence, especially drug discovery projects. Artificial intelligence models are finding use in molecular property prediction, molecule generation, virtual screening, synthesis planning, repurposing, among others. Lately, generative artificial intelligence has gained popularity across domains for its ability to generate entirely new data, such as images, sentences, audios, videos, novel chemical molecules, etc. Generative artificial intelligence has also delivered promising results in drug discovery and development. This review article delves into the fundamentals and framework of various generative artificial intelligence models in the context of drug discovery via de novo drug design approach. Various basic and advanced models have been discussed, along with their recent applications. The review also explores recent examples and advances in the generative artificial intelligence approach, as well as the challenges and ongoing efforts to fully harness the potential of generative artificial intelligence in generating novel drug molecules in a faster and more affordable manner. Some clinical-level assets generated form generative artificial intelligence have also been discussed in this review to show the ever-increasing application of artificial intelligence in drug discovery through commercial partnerships.

A Variational Autoencoder Cascade Generative Adversarial Network for Scalable 3D Object Generation and Reconstruction

Generative Adversarial Networks (GANs) for 3D volume generation and reconstruction, such as shape generation, visualization, automated design, real-time simulation, and research applications, are receiving increased amounts of attention in various fields. However, challenges such as limited training data, high computational costs, and mode collapse issues persist. We propose combining a Variational Autoencoder (VAE) and a GAN to uncover enhanced 3D structures and introduce a stable and scalable progressive growth approach for generating and reconstructing intricate voxel-based 3D shapes. The cascade-structured network involves a generator and discriminator, starting with small voxel sizes and incrementally adding layers, while subsequently supervising the discriminator with ground-truth labels in each newly added layer to model a broader voxel space. Our method enhances the convergence speed and improves the quality of the generated 3D models through stable growth, thereby facilitating an accurate representation of intricate voxel-level details. Through comparative experiments with existing methods, we demonstrate the effectiveness of our approach in evaluating voxel quality, variations, and diversity. The generated models exhibit improved accuracy in 3D evaluation metrics and visual quality, making them valuable across various fields, including virtual reality, the metaverse, and gaming.

MFGAN: Multimodal Fusion for Industrial Anomaly Detection Using Attention-Based Autoencoder and Generative Adversarial Network

Anomaly detection plays a critical role in ensuring safe, smooth, and efficient operation of machinery and equipment in industrial environments. With the wide deployment of multimodal sensors and the rapid development of Internet of Things (IoT), the data generated in modern industrial production has become increasingly diverse and complex. However, traditional methods for anomaly detection based on a single data source cannot fully utilize multimodal data to capture anomalies in industrial systems. To address this challenge, we propose a new model for anomaly detection in industrial environments using multimodal temporal data. This model integrates an attention-based autoencoder (AAE) and a generative adversarial network (GAN) to capture and fuse rich information from different data sources. Specifically, the AAE captures time-series dependencies and relevant features in each modality, and the GAN introduces adversarial regularization to enhance the model's ability to reconstruct normal time-series data. We conduct extensive experiments on real industrial data containing both measurements from a distributed control system (DCS) and acoustic signals, and the results demonstrate the performance superiority of the proposed model over the state-of-the-art TimesNet for anomaly detection, with an improvement of 5.6% in F1 score.

Using a generative adversarial network to generate synthetic MRI images for multi-class automatic segmentation of brain tumors

Challenging tasks such as lesion segmentation, classification, and analysis for the assessment of disease progression can be automatically achieved using deep learning (DL)-based algorithms. DL techniques such as 3D convolutional neural networks are trained using heterogeneous volumetric imaging data such as MRI, CT, and PET, among others. However, DL-based methods are usually only applicable in the presence of the desired number of inputs. In the absence of one of the required inputs, the method cannot be used. By implementing a generative adversarial network (GAN), we aim to apply multi-label automatic segmentation of brain tumors to synthetic images when not all inputs are present. The implemented GAN is based on the Pix2Pix architecture and has been extended to a 3D framework named Pix2PixNIfTI. For this study, 1,251 patients of the BraTS2021 dataset comprising sequences such as T1w, T2w, T1CE, and FLAIR images equipped with respective multi-label segmentation were used. This dataset was used for training the Pix2PixNIfTI model for generating synthetic MRI images of all the image contrasts. The segmentation model, namely DeepMedic, was trained in a five-fold cross-validation manner for brain tumor segmentation and tested using the original inputs as the gold standard. The inference of trained segmentation models was later applied to synthetic images replacing missing input, in combination with other original images to identify the efficacy of generated images in achieving multi-class segmentation. For the multi-class segmentation using synthetic data or lesser inputs, the dice scores were observed to be significantly reduced but remained similar in range for the whole tumor when compared with evaluated original image segmentation (e.g. mean dice of synthetic T2w prediction NC, 0.74 ± 0.30; ED, 0.81 ± 0.15; CET, 0.84 ± 0.21; WT, 0.90 ± 0.08). A standard paired t-tests with multiple comparison correction were performed to assess the difference between all regions (p < 0.05). The study concludes that the use of Pix2PixNIfTI allows us to segment brain tumors when one input image is missing.

A Fault Diagnosis Method for 5G Cellular Networks Based on Knowledge and Data Fusion

As 5G networks become more complex and heterogeneous, the difficulty of network operation and maintenance forces mobile operators to find new strategies to stay competitive. However, most existing network fault diagnosis methods rely on manual testing and time stacking, which suffer from long optimization cycles and high resource consumption. Therefore, we herein propose a knowledge- and data-fusion-based fault diagnosis algorithm for 5G cellular networks from the perspective of big data and artificial intelligence. The algorithm uses a generative adversarial network (GAN) to expand the data set collected from real network scenarios to balance the number of samples under different network fault categories. In the process of fault diagnosis, a naive Bayesian model (NBM) combined with domain expert knowledge is firstly used to pre-diagnose the expanded data set and generate a topological association graph between the data with solid engineering significance and interpretability. Then, as the pre-diagnostic prior knowledge, the topological association graph is fed into the graph convolutional neural network (GCN) model simultaneously with the training data set for model training. We use a data set collected by Minimization of Drive Tests under real network scenarios in Lu'an City, Anhui Province, in August 2019. The simulation results demonstrate that the algorithm outperforms other traditional models in fault detection and diagnosis tasks, achieving an accuracy of 90.56% and a macro F1 score of 88.41%.

Development of a Non-Contact Sensor System for Converting 2D Images into 3D Body Data: A Deep Learning Approach to Monitor Obesity and Body Shape in Individuals in Their 20s and 30s

This study demonstrates how to generate a three-dimensional (3D) body model through a small number of images and derive body values similar to the actual values using generated 3D body data. In this study, a 3D body model that can be used for body type diagnosis was developed using two full-body pictures of the front and side taken with a mobile phone. For data training, 400 3D body datasets (male: 200, female: 200) provided by Size Korea were used, and four models, i.e., 3D recurrent reconstruction neural network, point cloud generative adversarial network, skinned multi-person linear model, and pixel-aligned impact function for high-resolution 3D human digitization, were used. The models proposed in this study were analyzed and compared. A total of 10 men and women were analyzed, and their corresponding 3D models were verified by comparing 3D body data derived from 2D image inputs with those obtained using a body scanner. The model was verified through the difference between 3D data derived from the 2D image and those derived using an actual body scanner. Unlike the 3D generation models that could not be used to derive the body values in this study, the proposed model was successfully used to derive various body values, indicating that this model can be implemented to identify various body types and monitor obesity in the future.

Utilizing deep learning techniques to improve image quality and noise reduction in preclinical low-dose PET images in the sinogram domain

BACKGROUND

Low-dose positron emission tomography (LD-PET) imaging is commonly employed in preclinical research to minimize radiation exposure to animal subjects. However, LD-PET images often exhibit poor quality and high noise levels due to the low signal-to-noise ratio. Deep learning (DL) techniques such as generative adversarial networks (GANs) and convolutional neural network (CNN) have the capability to enhance the quality of images derived from noisy or low-quality PET data, which encodes critical information about radioactivity distribution in the body.

PURPOSE

Our objective was to optimize the image quality and reduce noise in preclinical PET images by utilizing the sinogram domain as input for DL models, resulting in improved image quality as compared to LD-PET images.

METHODS

A GAN and CNN model were utilized to predict high-dose (HD) preclinical PET sinograms from the corresponding LD preclinical PET sinograms. In order to generate the datasets, experiments were conducted on micro-phantoms, animal subjects (rats), and virtual simulations. The quality of DL-generated images was weighted by performing the following quantitative measures: structural similarity index measure (SSIM), root mean squared error (RMSE), peak signal-to-noise ratio (PSNR), signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR). Additionally, DL input and output were both subjected to a spatial resolution calculation of full width half maximum (FWHM) and full width tenth maximum (FWTM). DL outcomes were then compared with the conventional denoising algorithms such as non-local means (NLM), block-matching, and 3D filtering (BM3D).

RESULTS

The DL models effectively learned image features and produced high-quality images, as reflected in the quantitative metrics. Notably, the FWHM and FWTM values of DL PET images exhibited significantly improved accuracy compared to LD, NLM, and BM3D PET images, and just as precise as HD PET images. The MSE loss underscored the excellent performance of the models, indicating that the models performed well. To further improve the training, the generator loss (G loss) was increased to a value higher than the discriminator loss (D loss), thereby achieving convergence in the GAN model.

CONCLUSIONS

The sinograms generated by the GAN network closely resembled real HD preclinical PET sinograms and were more realistic than LD. There was a noticeable improvement in image quality and noise factor in the predicted HD images. Importantly, DL networks did not fully compromise the spatial resolution of the images.

Enhancer Recognition: A Transformer Encoder-Based Method with WGAN-GP for Data Augmentation

Enhancers are located upstream or downstream of key deoxyribonucleic acid (DNA) sequences in genes and can adjust the transcription activity of neighboring genes. Identifying enhancers and determining their functions are important for understanding gene regulatory networks and expression regulatory mechanisms. However, traditional enhancer recognition relies on manual feature engineering, which is time-consuming and labor-intensive, making it difficult to perform large-scale recognition analysis. In addition, if the original dataset is too small, there is a risk of overfitting. In recent years, emerging methods, such as deep learning, have provided new insights for enhancing identification. However, these methods also present certain challenges. Deep learning models typically require a large amount of high-quality data, and data acquisition demands considerable time and resources. To address these challenges, in this paper, we propose a data-augmentation method based on generative adversarial networks to solve the problem of small datasets. Moreover, we used regularization methods such as weight decay to improve the generalizability of the model and alleviate overfitting. The Transformer encoder was used as the main component to capture the complex relationships and dependencies in enhancer sequences. The encoding layer was designed based on the principle of k-mers to preserve more information from the original DNA sequence. Compared with existing methods, the proposed approach made significant progress in enhancing the accuracy and strength of enhancer identification and prediction, demonstrating the effectiveness of the proposed method. This paper provides valuable insights for enhancer analysis and is of great significance for understanding gene regulatory mechanisms and studying disease correlations.

Impact of data synthesis strategies for the classification of craniosynostosis

Introduction

Photogrammetric surface scans provide a radiation-free option to assess and classify craniosynostosis. Due to the low prevalence of craniosynostosis and high patient restrictions, clinical data are rare. Synthetic data could support or even replace clinical data for the classification of craniosynostosis, but this has never been studied systematically.

Methods

We tested the combinations of three different synthetic data sources: a statistical shape model (SSM), a generative adversarial network (GAN), and image-based principal component analysis for a convolutional neural network (CNN)-based classification of craniosynostosis. The CNN is trained only on synthetic data but is validated and tested on clinical data.

Results

The combination of an SSM and a GAN achieved an accuracy of 0.960 and an F1 score of 0.928 on the unseen test set. The difference to training on clinical data was smaller than 0.01. Including a second image modality improved classification performance for all data sources.

Conclusions

Without a single clinical training sample, a CNN was able to classify head deformities with similar accuracy as if it was trained on clinical data. Using multiple data sources was key for a good classification based on synthetic data alone. Synthetic data might play an important future role in the assessment of craniosynostosis.

Noise Reduction for a Virtual Grid Using a Generative Adversarial Network in Breast X-ray Images

In this study, we aimed to address the issue of noise amplification after scatter correction when using a virtual grid in breast X-ray images. To achieve this, we suggested an algorithm for estimating noise level and developed a noise reduction algorithm based on generative adversarial networks (GANs). Synthetic scatter in breast X-ray images were collected using Sizgraphy equipment and scatter correction was performed using dedicated software. After scatter correction, we determined the level of noise using noise-level function plots and trained a GAN using 42 noise combinations. Subsequently, we obtained the resulting images and quantitatively evaluated their quality by measuring the contrast-to-noise ratio (CNR), coefficient of variance (COV), and normalized noise-power spectrum (NNPS). The evaluation revealed an improvement in the CNR by approximately 2.80%, an enhancement in the COV by 12.50%, and an overall improvement in the NNPS across all frequency ranges. In conclusion, the application of our GAN-based noise reduction algorithm effectively reduced noise and demonstrated the acquisition of improved-quality breast X-ray images.

A Fault Detection Approach Based on One-Sided Domain Adaptation and Generative Adversarial Networks for Railway Door Systems

Fault detection using the domain adaptation technique is one of the more promising methods of solving the domain shift problem, and has therefore been intensively investigated in recent years. However, the domain adaptation method still has elements of impracticality: firstly, domain-specific decision boundaries are not taken into consideration, which often results in poor performance near the class boundary; and secondly, information on the source domain needs to be exploited with priority over information on the target domain, as the source domain can provide a rich dataset. Thus, the real-world implementations of this approach are still scarce. In order to address these issues, a novel fault detection approach based on one-sided domain adaptation for real-world railway door systems is proposed. An anomaly detector created using label-rich source domain data is used to generate distinctive source latent features, and the target domain features are then aligned toward the source latent features in a one-sided way. The performance and sensitivity analyses show that the proposed method is more accurate than alternative methods, with an F1 score of 97.9%, and is the most robust against variation in the input features. The proposed method also bridges the gap between theoretical domain adaptation research and tangible industrial applications. Furthermore, the proposed approach can be applied to conventional railway components and various electro-mechanical actuators. This is because the motor current signals used in this study are primarily obtained from the controller or motor drive, which eliminates the need for extra sensors.

A framework for immunofluorescence image augmentation and classification based on unsupervised attention mechanism

Autoimmune encephalitis (AE) is a common neurological disorder. As a standard method for neuroautoantibody detection, pathologists use tissue matrix assays (TBA) for initial disease screening. In this study, microscopic fluorescence imaging was combined with deep learning to improve AE diagnostic accuracy. Due to the inter-class imbalance of medical data, we propose an innovative generative adversarial network supplemented with attention mechanisms to highlight key regions in images to synthesize high-quality fluorescence images. However, securing annotated medical data is both time-consuming and costly. To circumvent this problem, we employ a self-supervised learning approach that utilizes unlabeled fluorescence data to support downstream classification tasks. To better understand the fluorescence properties in the data, we introduce a multichannel input convolutional neural network that adds additional channels of fluorescence intensity. This study builds an AE immunofluorescence dataset and obtains the classification accuracy of 88.5% using our method, thus confirming the effectiveness of the proposed method.

AI Hyperrealism: Why AI Faces Are Perceived as More Real Than Human Ones

Recent evidence shows that AI-generated faces are now indistinguishable from human faces. However, algorithms are trained disproportionately on White faces, and thus White AI faces may appear especially realistic. In Experiment 1 (N = 124 adults), alongside our reanalysis of previously published data, we showed that White AI faces are judged as human more often than actual human faces-a phenomenon we term AI hyperrealism. Paradoxically, people who made the most errors in this task were the most confident (a Dunning-Kruger effect). In Experiment 2 (N = 610 adults), we used face-space theory and participant qualitative reports to identify key facial attributes that distinguish AI from human faces but were misinterpreted by participants, leading to AI hyperrealism. However, the attributes permitted high accuracy using machine learning. These findings illustrate how psychological theory can inform understanding of AI outputs and provide direction for debiasing AI algorithms, thereby promoting the ethical use of AI.

Deep-learning predicted PET can be subtracted from the true clinical fluorodeoxyglucose PET co-registered to MRI to identify the epileptogenic zone in focal epilepsy

OBJECTIVE

Normal interictal [18 F]FDG-PET can be predicted from the corresponding T1w MRI with Generative Adversarial Networks (GANs). A technique we call SIPCOM (Subtraction Interictal PET Co-registered to MRI) can then be used to compare epilepsy patients' predicted and clinical PET. We assessed the ability of SIPCOM to identify the Resection Zone (RZ) in patients with drug-resistant epilepsy (DRE) with reference to visual and statistical parametric mapping (SPM) analysis.

METHODS

Patients with complete presurgical work-up and subsequent SEEG and cortectomy were included. RZ localisation, the reference region, was assigned to one of eighteen anatomical brain regions. SIPCOM was implemented using healthy controls to train a GAN. To compare, the clinical PET coregistered to MRI was visually assessed by two trained readers, and a standard SPM analysis was performed.

RESULTS

Twenty patients aged 17-50 (32 ± 7.8) years were included, 14 (70%) with temporal lobe epilepsy (TLE). Eight (40%) were MRI-negative. After surgery, 14 patients (70%) had a good outcome (Engel I-II). RZ localisation rate was 60% with SIPCOM vs 35% using SPM (P = 0.015) and vs 85% using visual analysis (P = 0.54). Results were similar for Engel I-II patients, the RZ localisation rate was 64% with SIPCOM vs 36% with SPM. With SIPCOM localisation was correct in 67% in MRI-positive vs 50% in MRI-negative patients, and 64% in TLE vs 43% in extra-TLE. The average number of false-positive clusters was 2.2 ± 1.3 using SIPCOM vs 2.3 ± 3.1 using SPM. All RZs localized with SPM were correctly localized with SIPCOM. In one case, PET and MRI were visually reported as negative, but both SIPCOM and SPM localized the RZ.

SIGNIFICANCE

SIPCOM performed better than the reference computer-assisted method (SPM) for RZ detection in a group of operated DRE patients. SIPCOM's impact on epilepsy management needs to be prospectively validated.

3DGAUnet: 3D Generative Adversarial Networks with a 3D U-Net Based Generator to Achieve the Accurate and Effective Synthesis of Clinical Tumor Image Data for Pancreatic Cancer

Pancreatic ductal adenocarcinoma (PDAC) presents a critical global health challenge, and early detection is crucial for improving the 5-year survival rate. Recent medical imaging and computational algorithm advances offer potential solutions for early diagnosis. Deep learning, particularly in the form of convolutional neural networks (CNNs), has demonstrated success in medical image analysis tasks, including classification and segmentation. However, the limited availability of clinical data for training purposes continues to represent a significant obstacle. Data augmentation, generative adversarial networks (GANs), and cross-validation are potential techniques to address this limitation and improve model performance, but effective solutions are still rare for 3D PDAC, where the contrast is especially poor, owing to the high heterogeneity in both tumor and background tissues. In this study, we developed a new GAN-based model, named 3DGAUnet, for generating realistic 3D CT images of PDAC tumors and pancreatic tissue, which can generate the inter-slice connection data that the existing 2D CT image synthesis models lack. The transition to 3D models allowed the preservation of contextual information from adjacent slices, improving efficiency and accuracy, especially for the poor-contrast challenging case of PDAC. PDAC's challenging characteristics, such as an iso-attenuating or hypodense appearance and lack of well-defined margins, make tumor shape and texture learning challenging. To overcome these challenges and improve the performance of 3D GAN models, our innovation was to develop a 3D U-Net architecture for the generator, to improve shape and texture learning for PDAC tumors and pancreatic tissue. Thorough examination and validation across many datasets were conducted on the developed 3D GAN model, to ascertain the efficacy and applicability of the model in clinical contexts. Our approach offers a promising path for tackling the urgent requirement for creative and synergistic methods to combat PDAC. The development of this GAN-based model has the potential to alleviate data scarcity issues, elevate the quality of synthesized data, and thereby facilitate the progression of deep learning models, to enhance the accuracy and early detection of PDAC tumors, which could profoundly impact patient outcomes. Furthermore, the model has the potential to be adapted to other types of solid tumors, hence making significant contributions to the field of medical imaging in terms of image processing models.

Manipulation Direction: Evaluating Text-Guided Image Manipulation Based on Similarity between Changes in Image and Text Modalities

At present, text-guided image manipulation is a notable subject of study in the vision and language field. Given an image and text as inputs, these methods aim to manipulate the image according to the text, while preserving text-irrelevant regions. Although there has been extensive research to improve the versatility and performance of text-guided image manipulation, research on its performance evaluation is inadequate. This study proposes Manipulation Direction (MD), a logical and robust metric, which evaluates the performance of text-guided image manipulation by focusing on changes between image and text modalities. Specifically, we define MD as the consistency of changes between images and texts occurring before and after manipulation. By using MD to evaluate the performance of text-guided image manipulation, we can comprehensively evaluate how an image has changed before and after the image manipulation and whether this change agrees with the text. Extensive experiments on Multi-Modal-CelebA-HQ and Caltech-UCSD Birds confirmed that there was an impressive correlation between our calculated MD scores and subjective scores for the manipulated images compared to the existing metrics.

A multi-scale unified model of human mobility in urban agglomerations

Understanding human mobility patterns is vital for the coordinated development of cities in urban agglomerations. Existing mobility models can capture single-scale travel behavior within or between cities, but the unified modeling of multi-scale human mobility in urban agglomerations is still analytically and computationally intractable. In this study, by simulating people's mental representations of physical space, we decompose and model the human travel choice process as a cascaded multi-class classification problem. Our multi-scale unified model, built upon cascaded deep neural networks, can predict human mobility in world-class urban agglomerations with thousands of regions. By incorporating individual memory features and population attractiveness features extracted by a graph generative adversarial network, our model can simultaneously predict multi-scale individual and population mobility patterns within urban agglomerations. Our model serves as an exemplar framework for reproducing universal-scale laws of human mobility across various spatial scales, providing vital decision support for urban settings of urban agglomerations.

High-fidelity direct contrast synthesis from magnetic resonance fingerprinting

PURPOSE

This work was aimed at proposing a supervised learning-based method that directly synthesizes contrast-weighted images from the Magnetic Resonance Fingerprinting (MRF) data without performing quantitative mapping and spin-dynamics simulations.

METHODS

To implement our direct contrast synthesis (DCS) method, we deploy a conditional generative adversarial network (GAN) framework with a multi-branch U-Net as the generator and a multilayer CNN (PatchGAN) as the discriminator. We refer to our proposed approach as N-DCSNet. The input MRF data are used to directly synthesize T1-weighted, T2-weighted, and fluid-attenuated inversion recovery (FLAIR) images through supervised training on paired MRF and target spin echo-based contrast-weighted scans. The performance of our proposed method is demonstrated on in vivo MRF scans from healthy volunteers. Quantitative metrics, including normalized root mean square error (nRMSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID), were used to evaluate the performance of the proposed method and compare it with others.

RESULTS

In-vivo experiments demonstrated excellent image quality with respect to that of simulation-based contrast synthesis and previous DCS methods, both visually and according to quantitative metrics. We also demonstrate cases in which our trained model is able to mitigate the in-flow and spiral off-resonance artifacts typically seen in MRF reconstructions, and thus more faithfully represent conventional spin echo-based contrast-weighted images.

CONCLUSION

We present N-DCSNet to directly synthesize high-fidelity multicontrast MR images from a single MRF acquisition. This method can significantly decrease examination time. By directly training a network to generate contrast-weighted images, our method does not require any model-based simulation and therefore can avoid reconstruction errors due to dictionary matching and contrast simulation (code available at:https://github.com/mikgroup/DCSNet).

Spatial-Intensity Transforms for Medical Image-to-Image Translation

Image-to-image translation has seen major advances in computer vision but can be difficult to apply to medical images, where imaging artifacts and data scarcity degrade the performance of conditional generative adversarial networks. We develop the spatial-intensity transform (SIT) to improve output image quality while closely matching the target domain. SIT constrains the generator to a smooth spatial transform (diffeomorphism) composed with sparse intensity changes. SIT is a lightweight, modular network component that is effective on various architectures and training schemes. Relative to unconstrained baselines, this technique significantly improves image fidelity, and our models generalize robustly to different scanners. Additionally, SIT provides a disentangled view of anatomical and textural changes for each translation, making it easier to interpret the model's predictions in terms of physiological phenomena. We demonstrate SIT on two tasks: predicting longitudinal brain MRIs in patients with various stages of neurodegeneration, and visualizing changes with age and stroke severity in clinical brain scans of stroke patients. On the first task, our model accurately forecasts brain aging trajectories without supervised training on paired scans. On the second task, it captures associations between ventricle expansion and aging, as well as between white matter hyperintensities and stroke severity. As conditional generative models become increasingly versatile tools for visualization and forecasting, our approach demonstrates a simple and powerful technique for improving robustness, which is critical for translation to clinical settings. Source code is available at github.com/clintonjwang/spatial-intensity-transforms.

Bi2O2Se-Based Bimode Noise Generator for the Application of Generative Adversarial Networks

In the emerging technology, the generative aversive networks (GANs), randomness, and unpredictability of inputting noises are the keys to the uniqueness, diversity, robustness, and security of the generated images. Compared with deterministic software-based noise generation, hardware-based noise generation introduces physical entropy sources, such as electronic and photonic noises, to add unpredictability. In this study, bimode Bi2O2Se-based noise generators have been demonstrated for the application of GANs. Harnessing its ultrahigh carrier mobility, excellent air stability, marvelous optoelectronic performance, as well as the unique surface resistive switching effect and defect locations in the energy diagram, Bi2O2Se provides a good material platform to easily integrate with multiple device architectures for generating noises in different physical sources. The noise of the black current mode in a photodetector architecture and the random telegraph noise in a memristor mode were measured, characterized, compared, and analyzed. A method of Markov chain equipped with K-means clustering was carried out to calculate the discrete noise states and the transition probability matrix between them. To evaluate the generated properties of the GANs based on the hardware noise source, the inception score and Fréchet inception distance were evaluated.

A Generative Adversarial Network to Synthesize 3D Magnetohydrodynamic Distortions for Electrocardiogram Analyses Applied to Cardiac Magnetic Resonance Imaging

Recently, deep learning (DL) models have been increasingly adopted for automatic analyses of medical data, including electrocardiograms (ECGs). Large, available ECG datasets, generally of high quality, often lack specific distortions, which could be helpful for enhancing DL-based algorithms. Synthetic ECG datasets could overcome this limitation. A generative adversarial network (GAN) was used to synthesize realistic 3D magnetohydrodynamic (MHD) distortion templates, as observed during magnetic resonance imaging (MRI), and then added to available ECG recordings to produce an augmented dataset. Similarity metrics, as well as the accuracy of a DL-based R-peak detector trained with and without data augmentation, were used to evaluate the effectiveness of the synthesized data. Three-dimensional MHD distortions produced by the proposed GAN were similar to the measured ones used as input. The precision of a DL-based R-peak detector, tested on actual unseen data, was significantly enhanced by data augmentation; its recall was higher when trained with augmented data. Using synthesized MHD-distorted ECGs significantly improves the accuracy of a DL-based R-peak detector, with a good generalization capacity. This provides a simple and effective alternative to collecting new patient data. DL-based algorithms for ECG analyses can suffer from bias or gaps in training datasets. Using a GAN to synthesize new data, as well as metrics to evaluate its performance, can overcome the scarcity issue of data availability.

2S-BUSGAN: A Novel Generative Adversarial Network for Realistic Breast Ultrasound Image with Corresponding Tumor Contour Based on Small Datasets

Deep learning (DL) models in breast ultrasound (BUS) image analysis face challenges with data imbalance and limited atypical tumor samples. Generative Adversarial Networks (GAN) address these challenges by providing efficient data augmentation for small datasets. However, current GAN approaches fail to capture the structural features of BUS and generated images lack structural legitimacy and are unrealistic. Furthermore, generated images require manual annotation for different downstream tasks before they can be used. Therefore, we propose a two-stage GAN framework, 2s-BUSGAN, for generating annotated BUS images. It consists of the Mask Generation Stage (MGS) and the Image Generation Stage (IGS), generating benign and malignant BUS images using corresponding tumor contours. Moreover, we employ a Feature-Matching Loss (FML) to enhance the quality of generated images and utilize a Differential Augmentation Module (DAM) to improve GAN performance on small datasets. We conduct experiments on two datasets, BUSI and Collected. Moreover, results indicate that the quality of generated images is improved compared with traditional GAN methods. Additionally, our generated images underwent evaluation by ultrasound experts, demonstrating the possibility of deceiving doctors. A comparative evaluation showed that our method also outperforms traditional GAN methods when applied to training segmentation and classification models. Our method achieved a classification accuracy of 69% and 85.7% on two datasets, respectively, which is about 3% and 2% higher than that of the traditional augmentation model. The segmentation model trained using the 2s-BUSGAN augmented datasets achieved DICE scores of 75% and 73% on the two datasets, respectively, which were higher than the traditional augmentation methods. Our research tackles imbalanced and limited BUS image data challenges. Our 2s-BUSGAN augmentation method holds potential for enhancing deep learning model performance in the field.

MRAβ: A multimodal MRI-derived amyloid-β biomarker for Alzheimer's disease

Florbetapir 18 F (AV45), a highly sensitive and specific positron emission tomographic (PET) molecular biomarker binding to the amyloid-β of Alzheimer's disease (AD), is constrained by radiation and cost. We sought to combat it by combining multimodal magnetic resonance imaging (MRI) images and a collaborative generative adversarial networks model (CollaGAN) to develop a multimodal MRI-derived Amyloid-β (MRAβ) biomarker. We collected multimodal MRI and PET AV45 data of 380 qualified participants from the ADNI dataset and 64 subjects from OASIS3 dataset. A five-fold cross-validation CollaGAN were applied to generate MRAβ. In the ADNI dataset, we found MRAβ could characterize the subject-level AV45 spatial variations in both AD and mild cognitive impairment (MCI). Voxel-wise two-sample t-tests demonstrated amyloid-β depositions identified by MRAβ in AD and MCI were significantly higher than healthy controls (HCs) in widespread cortices (p < .05, corrected) and were much similar to those by AV45 (r > .92, p < .001). Moreover, a 3D ResNet classifier demonstrated that MRAβ was comparable to AV45 in discriminating AD from HC in both the ADNI and OASIS3 datasets, and in discriminate MCI from HC in ADNI. Finally, we found MRAβ could mimic cortical hyper-AV45 in HCs who later converted to MCI (r = .79, p < .001) and was comparable to AV45 in discriminating them from stable HC (p > .05). In summary, our work illustrates that MRAβ synthesized by multimodal MRI could mimic the cerebral amyloid-β depositions like AV45 and lends credence to the feasibility of advancing MRI toward molecular-explainable biomarkers.

DRGAN: Dense Residual Generative Adversarial Network for Image Enhancement in an Underwater Autonomous Driving Device

Underwater autonomous driving devices, such as autonomous underwater vehicles (AUVs), rely on visual sensors, but visual images tend to produce color aberrations and a high turbidity due to the scattering and absorption of underwater light. To address these issues, we propose the Dense Residual Generative Adversarial Network (DRGAN) for underwater image enhancement. Firstly, we adopt a multi-scale feature extraction module to obtain a range of information and increase the receptive field. Secondly, a dense residual block is proposed, to realize the interaction of image features and ensure stable connections in the feature information. Multiple dense residual modules are connected from beginning to end to form a cyclic dense residual network, producing a clear image. Finally, the stability of the network is improved via adjustment to the training with multiple loss functions. Experiments were conducted using the RUIE and Underwater ImageNet datasets. The experimental results show that our proposed DRGAN can remove high turbidity from underwater images and achieve color equalization better than other methods.

Generation of synthetic EEG data for training algorithms supporting the diagnosis of major depressive disorder

Introduction

Major depressive disorder (MDD) is the most common mental disorder worldwide, leading to impairment in quality and independence of life. Electroencephalography (EEG) biomarkers processed with machine learning (ML) algorithms have been explored for objective diagnoses with promising results. However, the generalizability of those models, a prerequisite for clinical application, is restricted by small datasets. One approach to train ML models with good generalizability is complementing the original with synthetic data produced by generative algorithms. Another advantage of synthetic data is the possibility of publishing the data for other researchers without risking patient data privacy. Synthetic EEG time-series have not yet been generated for two clinical populations like MDD patients and healthy controls.

Methods

We first reviewed 27 studies presenting EEG data augmentation with generative algorithms for classification tasks, like diagnosis, for the possibilities and shortcomings of recent methods. The subsequent empirical study generated EEG time-series based on two public datasets with 30/28 and 24/29 subjects (MDD/controls). To obtain baseline diagnostic accuracies, convolutional neural networks (CNN) were trained with time-series from each dataset. The data were synthesized with generative adversarial networks (GAN) consisting of CNNs. We evaluated the synthetic data qualitatively and quantitatively and finally used it for re-training the diagnostic model.

Results

The reviewed studies improved their classification accuracies by between 1 and 40% with the synthetic data. Our own diagnostic accuracy improved up to 10% for one dataset but not significantly for the other. We found a rich repertoire of generative models in the reviewed literature, solving various technical issues. A major shortcoming in the field is the lack of meaningful evaluation metrics for synthetic data. The few studies analyzing the data in the frequency domain, including our own, show that only some features can be produced truthfully.

Discussion

The systematic review combined with our own investigation provides an overview of the available methods for generating EEG data for a classification task, their possibilities, and shortcomings. The approach is promising and the technical basis is set. For a broad application of these techniques in neuroscience research or clinical application, the methods need fine-tuning facilitated by domain expertise in (clinical) EEG research.

Sample Generation Method Based on Variational Modal Decomposition and Generative Adversarial Network (VMD-GAN) for Chemical Oxygen Demand (COD) Detection Using Ultraviolet Visible Spectroscopy

Ultraviolet visible spectroscopy can realize the detection of chemical oxygen demand (COD), especially for low concentration levels due to its high sensitivity, but the issue of insufficient real water sample data has always been a challenge owing to the low probability of occurrence of actual water pollution events. However, in existing methods, generated absorption spectra do not conform to actual situations as the former neglect the actual spectral characteristics. On the other hand, the diversity and complexity are restricted because the information in one-dimensional data is not enough for direct spectral generation. This study proposed a spectral sample generation method based on the variational modal decomposition and generative adversarial network (VMD-GAN). First, the VMD algorithm was utilized to separate principal components and residuals of absorption spectra. Among them, the GAN was used to generate new principal components to ensure that the major spectral characteristics of actual water samples are not lost. The corresponding residuals were then obtained by adjusting the parameters of a three-order Gaussian fitting function, which is more beneficial than the direct use of GAN in the aspect of diversity and complexity. Based on the spectral reconstruction with new principal components and residuals, various absorption spectra were generated more coincident with actual situations. Finally, the effectiveness of this method was evaluated by establishing regression models and predicting COD for actual water samples. In all, the insufficient water sample data can be expanded for a better performance in modeling and analysis of water pollution using the proposed method.

A Social Recommendation Model Based on Basic Spatial Mapping and Bilateral Generative Adversarial Networks

Social recommender systems are expected to improve recommendation quality by incorporating social information when there is little user-item interaction data. Therefore, how to effectively fuse interaction information and social information becomes a hot research topic in social recommendation, and how to mine and exploit the heterogeneous information in the interaction and social space becomes the key to improving recommendation performance. In this paper, we propose a social recommendation model based on basic spatial mapping and bilateral generative adversarial networks (MBSGAN). First, we propose to map the base space to the interaction and social space, respectively, in order to overcome the issue of heterogeneous information fusion in two spaces. Then, we construct bilateral generative adversarial networks in both interaction space and social space. Specifically, two generators are used to select candidate samples that are most similar to user feature vectors, and two discriminators are adopted to distinguish candidate samples from high-quality positive and negative examples obtained from popularity sampling, so as to learn complex information in the two spaces. Finally, the effectiveness of the proposed MBSGAN model is verified by comparing it with both eight social recommendation models and six models based on generative adversarial networks on four public datasets, Douban, FilmTrust, Ciao, and Epinions.

Deep Learning-Based Delayed PET Image Synthesis from Corresponding Early Scanned PET for Dosimetry Uptake Estimation

The acquisition of in vivo radiopharmaceutical distribution through imaging is time-consuming due to dosimetry, which requires the subject to be scanned at several time points post-injection. This study aimed to generate delayed positron emission tomography images from early images using a deep-learning-based image generation model to mitigate the time cost and inconvenience. Eighteen healthy participants were recruited and injected with [18F]Fluorodeoxyglucose. A paired image-to-image translation model, based on a generative adversarial network (GAN), was used as the generation model. The standardized uptake value (SUV) mean of the generated image of each organ was compared with that of the ground-truth. The least square GAN and perceptual loss combinations displayed the best performance. As the uptake time of the early image became closer to that of the ground-truth image, the translation performance improved. The SUV mean values of the nominated organs were estimated reasonably accurately for the muscle, heart, liver, and spleen. The results demonstrate that the image-to-image translation deep learning model is applicable for the generation of a functional image from another functional image acquired from normal subjects, including predictions of organ-wise activity for specific normal organs.

Metal Particle Detection by Integration of a Generative Adversarial Network and Electrical Impedance Tomography (GAN-EIT) for a Wet-Type Gravity Vibration Separator

The minor copper (Cu) particles among major aluminum (Al) particles have been detected by means of an integration of a generative adversarial network and electrical impedance tomography (GAN-EIT) for a wet-type gravity vibration separator (WGS). This study solves the problem of blurred EIT reconstructed images by proposing a GAN-EIT integration system for Cu detection in WGS. GAN-EIT produces two types of images of various Cu positions among major Al particles, which are (1) the photo-based GAN-EIT images, where blurred EIT reconstructed images are enhanced by GAN based on a full set of photo images, and (2) the simulation-based GAN-EIT images. The proposed metal particle detection by GAN-EIT is applied in experiments under static conditions to investigate the performance of the metal detection method under single-layer conditions with the variation of the position of Cu particles. As a quantitative result, the images of detected Cu by GAN-EIT ψ̿GAN in different positions have higher accuracy as compared to σ*EIT. In the region of interest (ROI) covered by the developed linear sensor, GAN-EIT successfully reduces the Cu detection error of conventional EIT by 40% while maintaining a minimum signal-to-noise ratio (SNR) of 60 [dB]. In conclusion, GAN-EIT is capable of improving the detailed features of the reconstructed images to visualize the detected Cu effectively.

Prediction of the COVID disease using lung CT images by Deep Learning algorithm: DETS-optimized Resnet 101 classifier

As a result of the COVID-19 (coronavirus) disease due to SARS-CoV2 becoming a pandemic, it has spread over the globe. It takes time to evaluate the results of the laboratory tests because of the rising number of cases each day. Therefore, there are restrictions in terms of both therapy and findings. A clinical decision-making system with predictive algorithms is needed to alleviate the pressure on healthcare systems via Deep Learning (DL) algorithms. With the use of DL and chest scans, this research intends to determine COVID-19 patients by utilizing the Transfer Learning (TL)-based Generative Adversarial Network (Pix 2 Pix-GAN). Moreover, the COVID-19 images are then classified as either positive or negative using a Duffing Equation Tuna Swarm (DETS)-optimized Resnet 101 classifier trained on synthetic and real images from the Kaggle lung CT Covid dataset. Implementation of the proposed technique is done using MATLAB simulations. Besides, is evaluated via accuracy, precision, F1-score, recall, and AUC. Experimental findings show that the proposed prediction model identifies COVID-19 patients with 97.2% accuracy, a recall of 95.9%, and a specificity of 95.5%, which suggests the proposed predictive model can be utilized to forecast COVID-19 infection by medical specialists for clinical prediction research and can be beneficial to them.

A novel fine-tuned deep-learning-based multi-class classifier for severity of paddy leaf diseases

Introduction

Paddy leaf diseases have a catastrophic influence on the quality and quantity of paddy grain production. The detection and identification of the intensity of various paddy infections are critical for high-quality crop production.

Methods

In this paper, infections in paddy leaves are considered for the identification of illness severity. The dataset contains both primary and secondary data. The four online repositories used for secondary data resources are Mendeley, GitHub, Kaggle and UCI. The size of the dataset is 4,068 images. The dataset is first pre-processed using ImageDataGenerator. Then, a generative adversarial network (GAN) is used to increase the dataset size exponentially. The disease severity calculation for the infected leaf is performed using a number of segmentation methods. To determine paddy infection, a deep learning-based hybrid approach is proposed that combines the capabilities of a convolutional neural network (CNN) and support vector machine (SVM). The severity levels are determined with the assistance of a domain expert. Four degrees of disease severity (mild, moderate, severe, and profound) are considered.

Results

Three infections are considered in the categorization of paddy leaf diseases: bacterial blight, blast, and leaf smut. The model predicted the paddy disease type and intensity with a 98.43% correctness rate. The loss rate is 41.25%.

Discussion

The findings show that the proposed method is reliable and effective for identifying the four levels of severity of bacterial blight, blast, and leaf smut infections in paddy crops. The proposed model performed better than the existing CNN and SVM classification models.

Patch-wise 3D segmentation quality assessment combining reconstruction and regression networks

Purpose

General deep-learning (DL)-based semantic segmentation methods with expert level accuracy may fail in 3D medical image segmentation due to complex tissue structures, lack of large datasets with ground truth, etc. For expeditious diagnosis, there is a compelling need to predict segmentation quality without ground truth. In some medical imaging applications, maintaining the quality of segmentation is crucial to the localized regions where disease is prevalent rather than just globally maintaining high-average segmentation quality. We propose a DL framework to identify regions of segmentation inaccuracies by combining a 3D generative adversarial network (GAN) and a convolutional regression network.

Approach

Our approach is methodologically based on the learned ability to reconstruct the original images identifying the regions of location-specific segmentation failures, in which the reconstruction does not match the underlying original image. We use conditional GAN to reconstruct input images masked by the segmentation results. The regression network is trained to predict the patch-wise Dice similarity coefficient (DSC), conditioned on the segmentation results. The method relies directly on the extracted segmentation related features and does not need to use ground truth during the inference phase to identify erroneous regions in the computed segmentation.

Results

We evaluated the proposed method on two public datasets: osteoarthritis initiative 4D (3D + time) knee MRI (knee-MR) and 3D non-small cell lung cancer CT (lung-CT). For the patch-wise DSC prediction, we observed the mean absolute errors of 0.01 and 0.04 with the independent standard for the knee-MR and lung-CT data, respectively.

Conclusions

This method shows promising results in localizing the erroneous segmentation regions that may aid the downstream analysis of disease diagnosis and prognosis prediction.

High-Accuracy Maize Disease Detection Based on Attention Generative Adversarial Network and Few-Shot Learning

This study addresses the problem of maize disease detection in agricultural production, proposing a high-accuracy detection method based on Attention Generative Adversarial Network (Attention-GAN) and few-shot learning. The method introduces an attention mechanism, enabling the model to focus more on the significant parts of the image, thereby enhancing model performance. Concurrently, data augmentation is performed through Generative Adversarial Network (GAN) to generate more training samples, overcoming the difficulties of few-shot learning. Experimental results demonstrate that this method surpasses other baseline models in accuracy, recall, and mean average precision (mAP), achieving 0.97, 0.92, and 0.95, respectively. These results validate the high accuracy and stability of the method in handling maize disease detection tasks. This research provides a new approach to solving the problem of few samples in practical applications and offers valuable references for subsequent research, contributing to the advancement of agricultural informatization and intelligence.

Synthetic whole-slide image tile generation with gene expression profile-infused deep generative models

In this work, we propose an approach to generate whole-slide image (WSI) tiles by using deep generative models infused with matched gene expression profiles. First, we train a variational autoencoder (VAE) that learns a latent, lower-dimensional representation of multi-tissue gene expression profiles. Then, we use this representation to infuse generative adversarial networks (GANs) that generate lung and brain cortex tissue tiles, resulting in a new model that we call RNA-GAN. Tiles generated by RNA-GAN were preferred by expert pathologists compared with tiles generated using traditional GANs, and in addition, RNA-GAN needs fewer training epochs to generate high-quality tiles. Finally, RNA-GAN was able to generalize to gene expression profiles outside of the training set, showing imputation capabilities. A web-based quiz is available for users to play a game distinguishing real and synthetic tiles: https://rna-gan.stanford.edu/, and the code for RNA-GAN is available here: https://github.com/gevaertlab/RNA-GAN.

Real-Time Detection Algorithm for Kiwifruit Canker Based on a Lightweight and Efficient Generative Adversarial Network

Disease diagnosis and control play important roles in agriculture and crop protection. Traditional methods of identifying plant disease rely primarily on human vision and manual inspection, which are subjective, have low accuracy, and make it difficult to estimate the situation in real time. At present, an intelligent detection technology based on computer vision is becoming an increasingly important tool used to monitor and control crop disease. However, the use of this technology often requires the collection of a substantial amount of specialized data in advance. Due to the seasonality and uncertainty of many crop pathogeneses, as well as some rare diseases or rare species, such data requirements are difficult to meet, leading to difficulties in achieving high levels of detection accuracy. Here, we use kiwifruit trunk bacterial canker (Pseudomonas syringae pv. actinidiae) as an example and propose a high-precision detection method to address the issue mentioned above. We introduce a lightweight and efficient image generative model capable of generating realistic and diverse images of kiwifruit trunk disease and expanding the original dataset. We also utilize the YOLOv8 model to perform disease detection; this model demonstrates real-time detection capability, taking only 0.01 s per image. The specific contributions of this study are as follows: (1) a depth-wise separable convolution is utilized to replace part of ordinary convolutions and introduce noise to improve the diversity of the generated images; (2) we propose the GASLE module by embedding a GAM, adjust the importance of different channels, and reduce the loss of spatial information; (3) we use an AdaMod optimizer to increase the convergence of the network; and (4) we select a real-time YOLOv8 model to perform effect verification. The results of this experiment show that the Fréchet Inception Distance (FID) of the proposed generative model reaches 84.18, having a decrease of 41.23 compared to FastGAN and a decrease of 2.1 compared to ProjectedGAN. The mean Average Precision (mAP@0.5) on the YOLOv8 network reaches 87.17%, which is nearly 17% higher than that of the original algorithm. These results substantiate the effectiveness of our generative model, providing a robust strategy for image generation and disease detection in plant kingdoms.

SUGAN: A Stable U-Net Based Generative Adversarial Network

As one of the representative models in the field of image generation, generative adversarial networks (GANs) face a significant challenge: how to make the best trade-off between the quality of generated images and training stability. The U-Net based GAN (U-Net GAN), a recently developed approach, can generate high-quality synthetic images by using a U-Net architecture for the discriminator. However, this model may suffer from severe mode collapse. In this study, a stable U-Net GAN (SUGAN) is proposed to mainly solve this problem. First, a gradient normalization module is introduced to the discriminator of U-Net GAN. This module effectively reduces gradient magnitudes, thereby greatly alleviating the problems of gradient instability and overfitting. As a result, the training stability of the GAN model is improved. Additionally, in order to solve the problem of blurred edges of the generated images, a modified residual network is used in the generator. This modification enhances its ability to capture image details, leading to higher-definition generated images. Extensive experiments conducted on several datasets show that the proposed SUGAN significantly improves over the Inception Score (IS) and Fréchet Inception Distance (FID) metrics compared with several state-of-the-art and classic GANs. The training process of our SUGAN is stable, and the quality and diversity of the generated samples are higher. This clearly demonstrates the effectiveness of our approach for image generation tasks. The source code and trained model of our SUGAN have been publicly released.

Adversarial Deep Transfer Learning in Fault Diagnosis: Progress, Challenges, and Future Prospects

Deep Transfer Learning (DTL) signifies a novel paradigm in machine learning, merging the superiorities of deep learning in feature representation with the merits of transfer learning in knowledge transference. This synergistic integration propels DTL to the forefront of research and development within the Intelligent Fault Diagnosis (IFD) sphere. While the early DTL paradigms, reliant on fine-tuning, demonstrated effectiveness, they encountered considerable obstacles in complex domains. In response to these challenges, Adversarial Deep Transfer Learning (ADTL) emerged. This review first categorizes ADTL into non-generative and generative models. The former expands upon traditional DTL, focusing on the efficient transference of features and mapping relationships, while the latter employs technologies such as Generative Adversarial Networks (GANs) to facilitate feature transformation. A thorough examination of the recent advancements of ADTL in the IFD field follows. The review concludes by summarizing the current challenges and future directions for DTL in fault diagnosis, including issues such as data imbalance, negative transfer, and adversarial training stability. Through this cohesive analysis, this review aims to offer valuable insights and guidance for the optimization and implementation of ADTL in real-world industrial scenarios.