Generalized backpropagation algorithm for training second-order neural networks

Fusion of Audio and Vibration Signals for Bearing Fault Diagnosis Based on a Quadratic Convolution Neural Network

In this paper, a quadratic convolution neural network (QCNN) using both audio and vibration signals is utilized for bearing fault diagnosis. Specifically, to make use of multi-modal information for bearing fault diagnosis, the audio and vibration signals are first fused together using a 1 × 1 convolution. Then, a quadratic convolution neural network is applied for the fusion feature extraction. Finally, a decision module is designed for fault classification. The proposed method utilizes the complementary information of audio and vibration signals, and is insensitive to noise. The experimental results show that the accuracy of the proposed method can achieve high accuracies for both single and multiple bearing fault diagnosis in the noisy situations. Moreover, the combination of two-modal data helps improve the performance under all conditions.

Superiority of quadratic over conventional neural networks for classification of gaussian mixture data

To enrich the diversity of artificial neurons, a type of quadratic neurons was proposed previously, where the inner product of inputs and weights is replaced by a quadratic operation. In this paper, we demonstrate the superiority of such quadratic neurons over conventional counterparts. For this purpose, we train such quadratic neural networks using an adapted backpropagation algorithm and perform a systematic comparison between quadratic and conventional neural networks for classificaiton of Gaussian mixture data, which is one of the most important machine learning tasks. Our results show that quadratic neural networks enjoy remarkably better efficacy and efficiency than conventional neural networks in this context, and potentially extendable to other relevant applications.

ISSM-ELM - a guide star selection for a small-FOV star sensor based on the improved SSM and extreme learning machine

The construction of a guide star catalog is crucial for a star sensor to achieve accurate star map recognition and attitude determination. At present, the methods of a guide star catalog for a large field of view (FOV) star sensor have been relatively mature. However, for a small-FOV star sensor, there are still certain problems, such as a large storage capacity of a guide star catalog, uneven distribution of stars, and easy occurrence of voids. To address these problems, we propose a construction method of a small-FOV star sensor guide star catalog based on the combination of the improved spherical spiral method (ISSM) and extreme learning machine (ELM) named the ISSM-ELM. First, a spiral reference point is used as an optical axis pointing of the star sensor, and the guide stars are preliminarily screened based on the star-diagonal distance between the star and the reference point, and the star-density and magnitude characteristics of the guide star. Then the ELM is used to supplement the guide star empty sky area to construct an integrity guide star catalog. The experimental results demonstrate that the proposed method can reduce the storage capacity of the guide star catalog, and improve its uniformity, integrity, and average brightness.

Development of artificial neural networks software for arsenic adsorption from an aqueous environment

Arsenic contamination is a global problem, as it affects the health of millions of people. For this study, data-driven artificial neural network (ANN) software was developed to predict and validate the removal of As(V) from an aqueous solution using graphene oxide (GO) under various experimental conditions. A reliable model for wastewater treatment is essential in order to predict its overall performance and to provide an idea of how to control its operation. This model considered the adsorption process parameters (initial concentration, adsorbent dosage, pH, and residence time) as the input variables and arsenic removal as the only output. The ANN model predicted the adsorption efficiency with high accuracy for both training and testing datasets, when compared with the available response surface methodology (RSM) model. Based on the best model synaptic weights, user-friendly ANN software was created to predict and analyze arsenic removal as a function of adsorption process parameters. We developed various graphical user interfaces (GUI) for easy use of the developed model. Thus, a researcher can efficiently operate the software without an understanding of programming or artificial neural networks. Sensitivity analysis and quantitative estimation were carried out to study the function of adsorption process parameter variables on As(V) removal efficiency, using the GUI of the model. The model prediction shows that the adsorbent dosages, initial concentration, and pH are the most influential parameters. The efficiency was increased as the adsorbent dosages increased, decreasing with initial concentration and pH. The result show that the pH 2.0-5.0 is optimal for adsorbent efficiency (%).

A review on Deep Learning approaches for low-dose Computed Tomography restoration

Computed Tomography (CT) is a widely use medical image modality in clinical medicine, because it produces excellent visualizations of fine structural details of the human body. In clinical procedures, it is desirable to acquire CT scans by minimizing the X-ray flux to prevent patients from being exposed to high radiation. However, these Low-Dose CT (LDCT) scanning protocols compromise the signal-to-noise ratio of the CT images because of noise and artifacts over the image space. Thus, various restoration methods have been published over the past 3 decades to produce high-quality CT images from these LDCT images. More recently, as opposed to conventional LDCT restoration methods, Deep Learning (DL)-based LDCT restoration approaches have been rather common due to their characteristics of being data-driven, high-performance, and fast execution. Thus, this study aims to elaborate on the role of DL techniques in LDCT restoration and critically review the applications of DL-based approaches for LDCT restoration. To achieve this aim, different aspects of DL-based LDCT restoration applications were analyzed. These include DL architectures, performance gains, functional requirements, and the diversity of objective functions. The outcome of the study highlights the existing limitations and future directions for DL-based LDCT restoration. To the best of our knowledge, there have been no previous reviews, which specifically address this topic.

Deep Face Rectification for 360° Dual-Fisheye Cameras

Rectilinear face recognition models suffer from severe performance degradation when applied to fisheye images captured by 360° back-to-back dual fisheye cameras. We propose a novel face rectification method to combat the effect of fisheye image distortion on face recognition. The method consists of a classification network and a restoration network specifically designed to handle the non-linear property of fisheye projection. The classification network classifies an input fisheye image according to its distortion level. The restoration network takes a distorted image as input and restores the rectilinear geometric structure of the face. The performance of the proposed method is tested on an end-to-end face recognition system constructed by integrating the proposed rectification method with a conventional rectilinear face recognition system. The face verification accuracy of the integrated system is 99.18% when tested on images in the synthetic Labeled Faces in the Wild (LFW) dataset and 95.70% for images in a real image dataset, resulting in an average accuracy improvement of 6.57% over the conventional face recognition system. For face identification, the average improvement over the conventional face recognition system is 4.51%.

Cancer classification from microarray data for genomic disorder research using optimal discriminant independent component analysis and kernel extreme learning machine

One of the challenging tasks in the medicinal field is genomic disorder investigation and its classification from the microarray dataset. The microarray dataset reorganization and its classification is more complex and expensive in the biomedical research area due to the larger number of features in the microarray dataset. In this paper, we construct a hybrid feature selection method such as t test, Fisher ration, and Bayesian logistic regression to select genes and that reduce the time cost. Based on the features, the top-ranked features are selected via the best hybrid rank method. Thereafter, the features are extracted using the modified firefly optimization-based discriminant independent component analysis (MF-DICA). Especially, the modified firefly optimization algorithm is capable of improving the search efficiency of DICA. From the high dimensional microarray dataset, MF-DICA is used to obtain the best features within the entire search space. The kernel extreme learning machine classifies the gene features depending upon the most relevant class. Experimentally, six datasets namely Leukemia dataset, Diffuse Larger B-cell Lymphomas, Lung cancer, Breast cancer, Prostate tumor, and Colon dataset are chosen to evaluate the performance of proposed approaches. Finally, the experimental data demonstrate that the proposed method is well suitable to classify the microarray data.

Quadratic Autoencoder (Q-AE) for Low-Dose CT Denoising

Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power of quadratic neurons in popular network architectures, simulating human-like learning in the form of "quadratic-neuron-based deep learning". Our prior theoretical studies have shown important merits of quadratic neurons and networks in representation, efficiency, and interpretability. In this paper, we use quadratic neurons to construct an encoder-decoder structure, referred as the quadratic autoencoder, and apply it to low-dose CT denoising. The experimental results on the Mayo low-dose CT dataset demonstrate the utility and robustness of quadratic autoencoder in terms of image denoising and model efficiency. To our best knowledge, this is the first time that the deep learning approach is implemented with a new type of neurons and demonstrates a significant potential in the medical imaging field.

Universal approximation with quadratic deep networks

Recently, deep learning has achieved huge successes in many important applications. In our previous studies, we proposed quadratic/second-order neurons and deep quadratic neural networks. In a quadratic neuron, the inner product of a vector of data and the corresponding weights in a conventional neuron is replaced with a quadratic function. The resultant quadratic neuron enjoys an enhanced expressive capability over the conventional neuron. However, how quadratic neurons improve the expressing capability of a deep quadratic network has not been studied up to now, preferably in relation to that of a conventional neural network. Specifically, we ask four basic questions in this paper: (1) for the one-hidden-layer network structure, is there any function that a quadratic network can approximate much more efficiently than a conventional network? (2) for the same multi-layer network structure, is there any function that can be expressed by a quadratic network but cannot be expressed with conventional neurons in the same structure? (3) Does a quadratic network give a new insight into universal approximation? (4) To approximate the same class of functions with the same error bound, could a quantized quadratic network have a lower number of weights than a quantized conventional network? Our main contributions are the four interconnected theorems shedding light upon these four questions and demonstrating the merits of a quadratic network in terms of expressive efficiency, unique capability, compact architecture and computational capacity respectively.