Fully complex conjugate gradient-based neural networks using Wirtinger calculus framework: Deterministic convergence and its application

Adaptive CL-BFGS Algorithms for Complex-Valued Neural Networks

Complex-valued limited-memory BFGS (CL-BFGS) algorithm is efficient for the training of complex-valued neural networks (CVNNs). As an important parameter, the memory size represents the number of saved vector pairs and would essentially affect the performance of the algorithm. However, the determination of a suitable memory size for the CL-BFGS algorithm remains challenging. To deal with this issue, an adaptive method is proposed in which the memory size is allowed to vary during the iteration process. Basically, at each iteration, with the help of multistep quasi-Newton method, an appropriate memory size is chosen from a variable set {1,2, ... , M} by approximating complex Hessian matrix as close as possible. To reduce the computational complexity and ensure desired performance, the upper bound M is adjustable according to the moving average of memory sizes found in previous iterations. The proposed adaptive CL-BFGS (ACL-BFGS) algorithm can be efficiently applied for the training of CVNNs. Moreover, it is suggested to take multiple memory sizes to construct the search direction, which further improves the performance of the ACL-BFGS algorithm. Experimental results on some benchmark problems including the pattern classification, complex function approximation, and nonlinear channel equalization problems are given to illustrate the advantages of the developed algorithms over some previous ones.

Prescribed-Time Convergent Adaptive ZNN for Time-Varying Matrix Inversion under Harmonic Noise

Harmonic noises widely exist in industrial fields and always affect the computational accuracy of neural network models. The existing original adaptive zeroing neural network (OAZNN) model can effectively suppress harmonic noises. Nevertheless, the OAZNN model’s convergence rate only stays at the exponential convergence, that is, its convergence speed is usually greatly affected by the initial state. Consequently, to tackle the above issue, this work combines the dynamic characteristics of harmonic signals with prescribed-time convergence activation function, and proposes a prescribed-time convergent adaptive ZNN (PTCAZNN) for solving time-varying matrix inverse problem (TVMIP) under harmonic noises. Owing to the nonlinear activation function used having the ability to reject noises itself and the adaptive term also being able to compensate the influence of noises, the PTCAZNN model can realize double noise suppression. More importantly, the theoretical analysis of PTCAZNN model with prescribed-time convergence and robustness performance is provided. Finally, by varying a series of conditions such as the frequency of single harmonic noise, the frequency of multi-harmonic noise, and the initial value and the dimension of the matrix, the comparative simulation results further confirm the effectiveness and superiority of the PTCAZNN model.

A global neural network learning machine: Coupled integer and fractional calculus operator with an adaptive learning scheme

Find the global optimal solution of the model is one promising research topic in computational intelligent community. Dependent on analogies to natural processes, the evolutionary swarm intelligent algorithms are widely used for solving global optimization problems which directed by the fitness values. In this paper, we propose one efficient fractional global learning machine (Fragmachine) which includes two stages (descending and ascending) to determine the optimal search path. The neural network model is used to approach the given fitness value. Specifically, for the descending stage, the integer gradient of the network output with respect the current location is employed to find the next descending point, while for the ascending stage, the fractional gradient is implemented to climb and escape from the local optimal point. We further propose one adaptive learning rate during training which relies on both the current gradient (integer or fractional) information and the fitness value. Finally, a series of numerical experiments verify the effectiveness of the proposed algorithm, Fragmachine.

H∞ estimation for stochastic semi-Markovian switching CVNNs with missing measurements and mode-dependent delays

This article is devoted to the H_∞ estimation problem for stochastic semi-Markovian switching complex-valued neural networks subject to incomplete measurement outputs, where the time-varying delay also depends on another semi-Markov process. A sequence of random variables with known statistical property is introduced to depict the missing measurement phenomenon. Based on the generalized Itoˆ's formula in complex form concerning with the semi-Markovian systems, complex-valued reciprocal convex inequality as well as intensive stochastic analysis method, some mode-dependent sufficient conditions are presented guaranteeing the estimation error system to be exponentially mean-square stable with a prespecified H_∞ disturbance attenuation level. In addition, the mode-dependent estimator gain matrices are appropriately designed according to the feasible solutions of certain complex matrix inequalities. In the end, one numerical example is provided to illustrate effectiveness of the theoretical results.

A training algorithm with selectable search direction for complex-valued feedforward neural networks

This paper focuses on presenting an efficient training algorithm for complex-valued feedforward neural networks by utilizing a tree structure. The basic idea of the proposed algorithm is that, by introducing a set of direction factors, distinctive search directions are available to be selected at each iteration such that the objective function is reduced as much as possible. Compared with some well-known training algorithms, one of the advantages of our algorithm is that the determination of search direction is of great flexibility and thus more accurate solution is obtained with faster convergence speed. Experimental simulations on pattern recognition, channel equalization and complex function approximation are provided to verify the effectiveness and applications of the proposed algorithm.