Projection-based fast learning fully complex-valued relaxation neural network

Adaptive CL-BFGS Algorithms for Complex-Valued Neural Networks

Complex-valued limited-memory BFGS (CL-BFGS) algorithm is efficient for the training of complex-valued neural networks (CVNNs). As an important parameter, the memory size represents the number of saved vector pairs and would essentially affect the performance of the algorithm. However, the determination of a suitable memory size for the CL-BFGS algorithm remains challenging. To deal with this issue, an adaptive method is proposed in which the memory size is allowed to vary during the iteration process. Basically, at each iteration, with the help of multistep quasi-Newton method, an appropriate memory size is chosen from a variable set {1,2, ... , M} by approximating complex Hessian matrix as close as possible. To reduce the computational complexity and ensure desired performance, the upper bound M is adjustable according to the moving average of memory sizes found in previous iterations. The proposed adaptive CL-BFGS (ACL-BFGS) algorithm can be efficiently applied for the training of CVNNs. Moreover, it is suggested to take multiple memory sizes to construct the search direction, which further improves the performance of the ACL-BFGS algorithm. Experimental results on some benchmark problems including the pattern classification, complex function approximation, and nonlinear channel equalization problems are given to illustrate the advantages of the developed algorithms over some previous ones.

Circular Complex-Valued GMDH-Type Neural Network for Real-Valued Classification Problems

Recently, applications of complex-valued neural networks (CVNNs) to real-valued classification problems have attracted significant attention. However, most existing CVNNs are black-box models with poor explanation performance. This study extends the real-valued group method of data handling (RGMDH)-type neural network to the complex field and constructs a circular complex-valued group method of data handling (C-CGMDH)-type neural network, which is a white-box model. First, a complex least squares method is proposed for parameter estimation. Second, a new complex-valued symmetric regularity criterion is constructed with a logarithmic function to represent explicitly the magnitude and phase of the actual and predicted complex output to evaluate and select the middle candidate models. Furthermore, the property of this new complex-valued external criterion is proven to be similar to that of the real external criterion. Before training this model, a circular transformation is used to transform the real-valued input features to the complex field. Twenty-five real-valued classification data sets from the UCI Machine Learning Repository are used to conduct the experiments. The results show that both RGMDH and C-CGMDH models can select the most important features from the complete feature space through a self-organizing modeling process. Compared with RGMDH, the C-CGMDH model converges faster and selects fewer features. Furthermore, its classification performance is statistically significantly better than the benchmark complex-valued and real-valued models. Regarding time complexity, the C-CGMDH model is comparable with other models in dealing with the data sets that have few features. Finally, we demonstrate that the GMDH-type neural network can be interpretable.

Inverse Problem of Ultrasound Beamforming With Sparsity Constraints and Regularization

Ultrasound (US) beamforming is the process of reconstructing an image from acquired echo traces on several transducer elements. Typical beamforming approaches, such as delay-and-sum, perform simple projection operations, while techniques using statistical information also exist, e.g., adaptive, phase coherence, delay-multiply-and-sum, and sparse coding approaches. Inspired by the feasibility and success of inverse problem (IP) formulations in several image reconstruction problems, such as computed tomography, we herein devise an IP approach for US beamforming. We define a linear forward model for the synthesis of the beamformed image, and solve its IP thanks to several intuitive and physics-based constraints and regularization terms proposed. These reflect the prior knowledge about the spectra of carrier signal and spatial coherence of modulated signal. These constraints admit effective formulation through sparse representations. Our proposed method was evaluated for plane-wave imaging (PWI) that transmits unfocused waves, enabling high frame rates with large field of view at the expense of much lower image quality with conventional beamforming techniques. Results are evaluated in numerical simulations, as well as tissue-mimicking phantoms and in vivo data provided by PWI challenge in medical US. The best results achieved by our proposed techniques are 0.39-mm full-width at half-maximum for spatial resolution and 16.3-dB contrast-to-noise ratio, using a single plane-wave transmit.

A New Feature Ensemble with a Multistage Classification Scheme for Breast Cancer Diagnosis

A new and effective feature ensemble with a multistage classification is proposed to be implemented in a computer-aided diagnosis (CAD) system for breast cancer diagnosis. A publicly available mammogram image dataset collected during the Image Retrieval in Medical Applications (IRMA) project is utilized to verify the suggested feature ensemble and multistage classification. In achieving the CAD system, feature extraction is performed on the mammogram region of interest (ROI) images which are preprocessed by applying a histogram equalization followed by a nonlocal means filtering. The proposed feature ensemble is formed by concatenating the local configuration pattern-based, statistical, and frequency domain features. The classification process of these features is implemented in three cases: a one-stage study, a two-stage study, and a three-stage study. Eight well-known classifiers are used in all cases of this multistage classification scheme. Additionally, the results of the classifiers that provide the top three performances are combined via a majority voting technique to improve the recognition accuracy on both two- and three-stage studies. A maximum of 85.47%, 88.79%, and 93.52% classification accuracies are attained by the one-, two-, and three-stage studies, respectively. The proposed multistage classification scheme is more effective than the single-stage classification for breast cancer diagnosis.

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Complex gradient methods have been widely used in learning theory, and typically aim to optimize real-valued functions of complex variables. The stepsize of complex gradient learning methods (CGLMs) is a positive number, and little is known about how a complex stepsize would affect the learning process. To this end, we undertake a comprehensive analysis of CGLMs with a complex stepsize, including the search space, convergence properties, and the dynamics near critical points. Furthermore, several adaptive stepsizes are derived by extending the Barzilai-Borwein method to the complex domain, in order to show that the complex stepsize is superior to the corresponding real one in approximating the information in the Hessian. A numerical example is presented to support the analysis.

Complete stability of delayed recurrent neural networks with Gaussian activation functions

This paper addresses the complete stability of delayed recurrent neural networks with Gaussian activation functions. By means of the geometrical properties of Gaussian function and algebraic properties of nonsingular M-matrix, some sufficient conditions are obtained to ensure that for an n-neuron neural network, there are exactly 3^k equilibrium points with 0≤k≤n, among which 2^k and 3^k-2^k equilibrium points are locally exponentially stable and unstable, respectively. Moreover, it concludes that all the states converge to one of the equilibrium points; i.e., the neural networks are completely stable. The derived conditions herein can be easily tested. Finally, a numerical example is given to illustrate the theoretical results.

Multistability of complex-valued neural networks with discontinuous activation functions

In this paper, based on the geometrical properties of the discontinuous activation functions and the Brouwer's fixed point theory, the multistability issue is tackled for the complex-valued neural networks with discontinuous activation functions and time-varying delays. To address the network with discontinuous functions, Filippov solution of the system is defined. Through rigorous analysis, several sufficient criteria are obtained to assure the existence of 25ⁿ equilibrium points. Among them, 9ⁿ points are locally stable and 16ⁿ-9ⁿ equilibrium points are unstable. Furthermore, to enlarge the attraction basins of the 9ⁿ equilibrium points, some mild conditions are imposed. Finally, one numerical example is provided to illustrate the effectiveness of the obtained results.

Learning dynamics of a single polar variable complex-valued neuron

This letter investigates the characteristics of the complex-valued neuron model with parameters represented by polar coordinates (called polar variable complex-valued neuron). The parameters of the polar variable complex-valued neuron are unidentifiable. The plateau phenomenon can occur during learning of the polar variable complex-valued neuron. Furthermore, computer simulations suggest that a single polar variable complex-valued neuron has the following characteristics in the case of using the steepest gradient-descent method with square error: (1) unidentifiable parameters (singular points) degrade the learning speed and (2) a plateau can occur during learning. When the weight is attracted to the singular point, the learning tends to become stuck. However, computer simulations also show that the steepest gradient-descent method with amplitude-phase error and the complex-valued natural gradient method could reduce the effects of the singular points. The learning dynamics near singular points depends on the error functions and the training algorithms used.