Review on Generative Adversarial Networks: Focusing on Computer Vision and Its Applications

Progress in the application of artificial intelligence in molecular generation models based on protein structure

The molecular generation models based on protein structures represent a cutting-edge research direction in artificial intelligence-assisted drug discovery. This article aims to comprehensively summarize the research methods and developments by analyzing a series of novel molecular generation models predicated on protein structures. Initially, we categorize the molecular generation models based on protein structures and highlight the architectural frameworks utilized in these models. Subsequently, we detail the design and implementation of protein structure-based molecular generation models by introducing different specific examples. Lastly, we outline the current opportunities and challenges encountered in this field, intending to offer guidance and a referential framework for developing and studying new models in related fields in the future.

Application progress of deep generative models in de novo drug design

The deep molecular generative model has recently become a research hotspot in pharmacy. This paper analyzes a large number of recent reports and reviews these models. In the central part of this paper, four compound databases and two molecular representation methods are compared. Five model architectures and applications for deep molecular generative models are emphatically introduced. Three evaluation metrics for model evaluation are listed. Finally, the limitations and challenges in this field are discussed to provide a reference and basis for developing and researching new models published in future.

Toward Unbiased High-Quality Portraits through Latent-Space Evaluation

Images, texts, voices, and signals can be synthesized by latent spaces in a multidimensional vector, which can be explored without the hurdles of noise or other interfering factors. In this paper, we present a practical use case that demonstrates the power of latent space in exploring complex realities such as image space. We focus on DaVinciFace, an AI-based system that explores the StyleGAN2 space to create a high-quality portrait for anyone in the style of the Renaissance genius Leonardo da Vinci. The user enters one of their portraits and receives the corresponding Da Vinci-style portrait as an output. Since most of Da Vinci's artworks depict young and beautiful women (e.g., "La Belle Ferroniere", "Beatrice de' Benci"), we investigate the ability of DaVinciFace to account for other social categorizations, including gender, race, and age. The experimental results evaluate the effectiveness of our methodology on 1158 portraits acting on the vector representations of the latent space to produce high-quality portraits that retain the facial features of the subject's social categories, and conclude that sparser vectors have a greater effect on these features. To objectively evaluate and quantify our results, we solicited human feedback via a crowd-sourcing campaign. Analysis of the human feedback showed a high tolerance for the loss of important identity features in the resulting portraits when the Da Vinci style is more pronounced, with some exceptions, including Africanized individuals.

Interactive design generation and optimization from generative adversarial networks in spatial computing

This paper focuses on exploring the application possibilities and optimization problems of Generative Adversarial Networks (GANs) in spatial computing to improve design efficiency and creativity and achieve a more intelligent design process. A method for icon generation is proposed, and a basic architecture for icon generation is constructed. A system with generation and optimization capabilities is constructed to meet various requirements in spatial design by introducing the concept of interactive design and the characteristics of requirement conditions. Next, the generated icons can effectively maintain diversity and innovation while meeting the conditional features by integrating multi-feature recognition modules into the discriminator and optimizing the structure of conditional features. The experiment uses publicly available icon datasets, including LLD-Icon and Icons-50. The icon shape generated by the model proposed here is more prominent, and the color of colored icons can be more finely controlled. The Inception Score (IS) values under different models are compared, and it is found that the IS value of the proposed model is 7.05, which is higher than that of other GAN models. The multi-feature icon generation model based on Auxiliary Classifier GANs performs well in presenting multiple feature representations of icons. After introducing multi-feature recognition modules into the network model, the peak error of the recognition network is only 2.000 in the initial stage, while the initial error of the ordinary GAN without multi-feature recognition modules is as high as 5.000. It indicates that the improved model effectively helps the discriminative network recognize the core information of icon images more quickly. The research results provide a reference basis for achieving more efficient and innovative interactive space design.

Machine Learning Methods for Small Data Challenges in Molecular Science

Small data are often used in scientific and engineering research due to the presence of various constraints, such as time, cost, ethics, privacy, security, and technical limitations in data acquisition. However, big data have been the focus for the past decade, small data and their challenges have received little attention, even though they are technically more severe in machine learning (ML) and deep learning (DL) studies. Overall, the small data challenge is often compounded by issues, such as data diversity, imputation, noise, imbalance, and high-dimensionality. Fortunately, the current big data era is characterized by technological breakthroughs in ML, DL, and artificial intelligence (AI), which enable data-driven scientific discovery, and many advanced ML and DL technologies developed for big data have inadvertently provided solutions for small data problems. As a result, significant progress has been made in ML and DL for small data challenges in the past decade. In this review, we summarize and analyze several emerging potential solutions to small data challenges in molecular science, including chemical and biological sciences. We review both basic machine learning algorithms, such as linear regression, logistic regression (LR), k-nearest neighbor (KNN), support vector machine (SVM), kernel learning (KL), random forest (RF), and gradient boosting trees (GBT), and more advanced techniques, including artificial neural network (ANN), convolutional neural network (CNN), U-Net, graph neural network (GNN), Generative Adversarial Network (GAN), long short-term memory (LSTM), autoencoder, transformer, transfer learning, active learning, graph-based semi-supervised learning, combining deep learning with traditional machine learning, and physical model-based data augmentation. We also briefly discuss the latest advances in these methods. Finally, we conclude the survey with a discussion of promising trends in small data challenges in molecular science.

Facilitating COVID recognition from X-rays with computer vision models and transfer learning

Multimedia data plays an important role in medicine and healthcare since EHR (Electronic Health Records) entail complex images and videos for analyzing patient data. In this article, we hypothesize that transfer learning with computer vision can be adequately harnessed on such data, more specifically chest X-rays, to learn from a few images for assisting accurate, efficient recognition of COVID. While researchers have analyzed medical data (including COVID data) using computer vision models, the main contributions of our study entail the following. Firstly, we conduct transfer learning using a few images from publicly available big data on chest X-rays, suitably adapting computer vision models with data augmentation. Secondly, we aim to find the best fit models to solve this problem, adjusting the number of samples for training and validation to obtain the minimum number of samples with maximum accuracy. Thirdly, our results indicate that combining chest radiography with transfer learning has the potential to improve the accuracy and timeliness of radiological interpretations of COVID in a cost-effective manner. Finally, we outline applications of this work during COVID and its recovery phases with future issues for research and development. This research exemplifies the use of multimedia technology and machine learning in healthcare.

Image Generation and Recognition for Railway Surface Defect Detection

Railway defects can result in substantial economic and human losses. Among all defects, surface defects are the most common and prominent type, and various optical-based non-destructive testing (NDT) methods have been employed to detect them. In NDT, reliable and accurate interpretation of test data is vital for effective defect detection. Among the many sources of errors, human errors are the most unpredictable and frequent. Artificial intelligence (AI) has the potential to address this challenge; however, the lack of sufficient railway images with diverse types of defects is the major obstacle to training the AI models through supervised learning. To overcome this obstacle, this research proposes the RailGAN model, which enhances the basic CycleGAN model by introducing a pre-sampling stage for railway tracks. Two pre-sampling techniques are tested for the RailGAN model: image-filtration, and U-Net. By applying both techniques to 20 real-time railway images, it is demonstrated that U-Net produces more consistent results in image segmentation across all images and is less affected by the pixel intensity values of the railway track. Comparison of the RailGAN model with U-Net and the original CycleGAN model on real-time railway images reveals that the original CycleGAN model generates defects in the irrelevant background, while the RailGAN model produces synthetic defect patterns exclusively on the railway surface. The artificial images generated by the RailGAN model closely resemble real cracks on railway tracks and are suitable for training neural-network-based defect identification algorithms. The effectiveness of the RailGAN model can be evaluated by training a defect identification algorithm with the generated dataset and applying it to real defect images. The proposed RailGAN model has the potential to improve the accuracy of NDT for railway defects, which can ultimately lead to increased safety and reduced economic losses. The method is currently performed offline, but further study is planned to achieve real-time defect detection in the future.

A scoping review on deep learning for next-generation RNA-Seq. data analysis

In the last decade, transcriptome research adopting next-generation sequencing (NGS) technologies has gathered incredible momentum amongst functional genomics scientists, particularly amongst clinical/biomedical research groups. The progressive enfoldment/adoption of NGS technologies has incited an abundance of next-generation transcriptomic data harbouring an opulence of new knowledge in public databases. Nevertheless, knowledge discovery from these next-generation RNA-Seq. data analysis necessitates extensive bioinformatics know-how besides elaborate data analysis software packages consistent with the type and context of data analysis. Several reliability and reproducibility concerns continue to impede RNA-Seq. data analysis. Characteristic challenges comprise of data quality, hardware and networking provisions, selection and prioritisation of data analysis tools, and yet significantly implementing of robust machine learning algorithms for maximised exploitation of these experimental transcriptomic data. Over the years, numerous machine learning algorithms have been implemented for improved transcriptomic data analysis executing predominantly shallow learning approaches. More recently, deep learning algorithms are becoming more mainstream, and enactment for next-generation RNA-Seq. data analysis could be revolutionary in the coming years in the biomedical domain. In this scoping review, we attempt to determine the existing literature's size and potential nature in deep learning and NGS RNA-Seq. data analysis. An analysis of the contemporary topics of next-generation RNA-Seq. data analysis based on deep learning algorithms is critically reviewed, emphasising open-source resources.

Steel Strip Defect Sample Generation Method Based on Fusible Feature GAN Model under Few Samples

Due to the shortage of defect samples and the high cost of labelling during the process of hot-rolled strip production in the metallurgical industry, it is difficult to obtain a large quantity of defect data with diversity, which seriously affects the identification accuracy of different types of defects on the steel surface. To address the problem of insufficient defect sample data in the task of strip steel defect identification and classification, this paper proposes the Strip Steel Surface Defect-ConSinGAN (SDE-ConSinGAN) model for strip steel defect identification which is based on a single-image model trained by the generative adversarial network (GAN) and which builds a framework of image-feature cutting and splicing. The model aims to reduce training time by dynamically adjusting the number of iterations for different training stages. The detailed defect features of training samples are highlighted by introducing a new size-adjustment function and increasing the channel attention mechanism. In addition, real image features will be cut and synthesized to obtain new images with multiple defect features for training. The emergence of new images is able to richen generated samples. Eventually, the generated simulated samples can be directly used in deep-learning-based automatic classification of surface defects in cold-rolled thin strips. The experimental results show that, when SDE-ConSinGAN is used to enrich the image dataset, the generated defect images have higher quality and more diversity than the current methods do.

Generative models and Bayesian inversion using Laplace approximation

The Bayesian approach to solving inverse problems relies on the choice of a prior. This critical ingredient allows expert knowledge or physical constraints to be formulated in a probabilistic fashion and plays an important role for the success of the inference. Recently, Bayesian inverse problems were solved using generative models as highly informative priors. Generative models are a popular tool in machine learning to generate data whose properties closely resemble those of a given database. Typically, the generated distribution of data is embedded in a low-dimensional manifold. For the inverse problem, a generative model is trained on a database that reflects the properties of the sought solution, such as typical structures of the tissue in the human brain in magnetic resonance imaging. The inference is carried out in the low-dimensional manifold determined by the generative model that strongly reduces the dimensionality of the inverse problem. However, this procedure produces a posterior that does not admit a Lebesgue density in the actual variables and the accuracy attained can strongly depend on the quality of the generative model. For linear Gaussian models, we explore an alternative Bayesian inference based on probabilistic generative models; this inference is carried out in the original high-dimensional space. A Laplace approximation is employed to analytically derive the prior probability density function required, which is induced by the generative model. Properties of the resulting inference are investigated. Specifically, we show that derived Bayes estimates are consistent, in contrast to the approach in which the low-dimensional manifold of the generative model is employed. The MNIST data set is used to design numerical experiments that confirm our theoretical findings. It is shown that the approach proposed can be advantageous when the information contained in the data is high and a simple heuristic is considered for the detection of this case. Finally, the pros and cons of both approaches are discussed.

Deep-HAR: an ensemble deep learning model for recognizing the simple, complex, and heterogeneous human activities

The recognition of human activities has become a dominant emerging research problem and widely covered application areas in surveillance, wellness management, healthcare, and many more. In real life, the activity recognition is a challenging issue because human beings are often performing the activities not only simple but also complex and heterogeneous in nature. Most of the existing approaches are addressing the problem of recognizing only simple straightforward activities (e.g. walking, running, standing, sitting, etc.). Recognizing the complex and heterogeneous human activities are a challenging research problem whereas only a limited number of existing works are addressing this issue. In this paper, we proposed a novel Deep-HAR model by ensembling the Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) for recognizing the simple, complex, and heterogeneous type activities. Here, the CNNs are used for extracting the features whereas RNNs are used for finding the useful patterns in time-series sequential data. The activities recognition performance of the proposed model was evaluated using three different publicly available datasets, namely WISDM, PAMAP2, and KU-HAR. Through extensive experiments, we have demonstrated that the proposed model performs well in recognizing all types of activities and has achieved an accuracy of 99.98%, 99.64%, and 99.98% for simple, complex, and heterogeneous activities respectively.

Evaluating the Influence of ipRGCs on Color Discrimination

To investigate the influence of intrinsically photosensitive retinal ganglion cells (ipRGCs) on color discrimination, it is necessary to create two metameric light stimuli (metameric ipRGC stimuli) with the same amount of cone and rod stimulation, but different amounts of ipRGC stimulation. However, since the spectral sensitivity functions of cones and rods overlap with those of ipRGCs in a wavelength band, it has been difficult to independently control the amount of stimulation of ipRGCs only. In this study, we first propose a method for calculating metameric ipRGC stimulation based on the orthogonal basis functions of human photoreceptor cells. Then, we clarify the controllable range of metameric ipRGC stimulation within a color gamut. Finally, to investigate the color discrimination by metameric ipRGC stimuli, we conduct subjective evaluation experiments on 24 chromaticity coordinates using a multispectral projector. The results reveal a correlation between differences in the amount of ipRGC stimulation and differences in color appearance, indicating that ipRGCs may influence color discrimination.

Unveiling OASIS family as a key player in hypoxia-ischemia cases induced by cocaine using generative adversarial networks

Repeated cocaine use poses many serious health risks to users. One of the risks is hypoxia and ischemia (HI). To restore the biological system against HI, complex biological mechanisms operate at the gene level. Despite the complexity of biological mechanisms, there are common denominator genes that play pivotal roles in various defense systems. Among these genes, the cAMP response element-binding (Creb) protein contributes not only to various aspects of drug-seeking behavior and drug reward, but also to protective mechanisms. However, it is still unclear which Creb members are key players in the protection of cocaine-induced HI conditions. Herein, using one of the state-of-the-art deep learning methods, the generative adversarial network, we revealed that the OASIS family, one of the Creb family, is a key player in various defense mechanisms such as angiogenesis and unfolded protein response against the HI state by unveiling hidden mRNA expression profiles. Furthermore, we identified mysterious kinases in the OASIS family and are able to explain why the prefrontal cortex and hippocampus are vulnerable to HI at the genetic level.

Review for Examining the Oxidation Process of the Moon Using Generative Adversarial Networks: Focusing on Landscape of Moon

Japan Aerospace Exploration Agency (JAXA) has collected and studied the data observed by the lunar probe, SELenological and ENgineering Explorer (SELENE), from 2007 to 2017. JAXA discovered that the oxygen of the upper atmosphere of the Earth is transported to the moon by the tail of the magnetic field. However, this research is still in progress, and more data are needed to clarify the oxidation process. Therefore, this paper supplements the insufficient observation data by using Generative Adversarial Networks (GAN) and proposes a review paper focusing on the methodology, enhancing the level of completion of the preceding research, and the trend of examining the oxidation process and landscape of the moon. We propose using Anokhin’s Conditionally-Independent Pixel Synthesis (CIPS) as a model to be used in future experiments as a result of the review. CIPS can generate pixels independently for each color value, and since it uses a Multi-Layer Perceptron (MLP) network rather than spatial convolutions, there is a significant advantage in scalability. It is concluded that the proposed methodology will save time and costs of the existing research in progress and will help reveal the causal relationship more clearly.

An Intelligent Hybrid–Integrated System Using Speech Recognition and a 3D Display for Early Childhood Education

In the past few years, people’s attitudes toward early childhood education (PAUD) have undergone a complete transformation. Personalized and intelligent communication methods are highly praised, which also promotes the further focus on timely and effective human–computer interaction. Since traditional English learning that relies on parents consumes more time and energy and is prone to errors and omissions, this paper proposes a system based on a convolution neural network (CNN) and automatic speech recognition (ASR) to achieve an integrated process of object recognition, intelligent speech interaction, and synchronization of learning records in children’s education. Compared with platforms described in the literature, not only does it shoot objects in the real-life environment to obtain English words, their pronunciation, and example sentences corresponding to them, but also it combines the technique of a three-dimensional display to help children learn abstract words. At the same time, the cloud database summarizes and tracks the learning progress by a horizontal comparison, which makes it convenient for parents to figure out the situation. The performance evaluation of image and speech recognition demonstrates that the overall accuracy remains above 96%. Through comprehensive experiments in different scenarios, we prove that the platform is suitable for children as an auxiliary method and cultivates their interest in learning English.