Adaptive quasi-Newton algorithm for source extraction via CCA approach

A global neural network learning machine: Coupled integer and fractional calculus operator with an adaptive learning scheme

Find the global optimal solution of the model is one promising research topic in computational intelligent community. Dependent on analogies to natural processes, the evolutionary swarm intelligent algorithms are widely used for solving global optimization problems which directed by the fitness values. In this paper, we propose one efficient fractional global learning machine (Fragmachine) which includes two stages (descending and ascending) to determine the optimal search path. The neural network model is used to approach the given fitness value. Specifically, for the descending stage, the integer gradient of the network output with respect the current location is employed to find the next descending point, while for the ascending stage, the fractional gradient is implemented to climb and escape from the local optimal point. We further propose one adaptive learning rate during training which relies on both the current gradient (integer or fractional) information and the fitness value. Finally, a series of numerical experiments verify the effectiveness of the proposed algorithm, Fragmachine.

Sequential Independent Component Analysis Density Estimation

A problem of multivariate probability density function estimation by exploiting linear independent components analysis (ICA) is addressed. Historically, ICA density estimation was initially proposed under the name projection pursuit density estimation (PPDE) and two basic methods, named forward and backward, were published. We derive a modification of the forward PPDE method, which avoids a computationally demanding optimization involving Monte Carlo sampling of the original method. The results of the experiments show that the proposed method presents an attractive choice for density estimation, which is pronounced for a small number of training observations. Under such conditions, our method usually outperforms model-based Gaussian mixture model. We also found that our method obtained better results than the backward PPDE methods in the situation of nonfactorizable underlying density functions. The proposed method has demonstrated a competitive performance compared with the support vector machine and the extreme learning machine in some real classification tasks.

An Adaptive Self-Stabilizing Algorithm for Minor Generalized Eigenvector Extraction and Its Convergence Analysis

Generalized eigendecomposition, which extracts the generalized eigenvector from a matrix pencil, is a powerful tool and has been widely used in many fields, such as data classification and blind source separation. First, to extract the minor generalized eigenvector (MGE), we propose a deterministic discrete-time (DDT) system. Unlike some existing systems, the proposed DDT system does not need to normalize the weight vector in each iteration, since the weight vectors in the proposed DDT system are self-stabilizing. Second, we propose an adaptive algorithm corresponding to the proposed DDT system. Moreover, we study the dynamic behavior and convergence properties of the proposed DDT system and prove that the weight vector must converge to the direction of the MGE of a matrix pencil under some mild conditions. Numerical simulations show that the proposed algorithm has a better performance in terms of convergence speed and estimation accuracy than some existing algorithms. Finally, we conduct two experiments on real data sets to demonstrate its practicability.

A Novel Unified and Self-Stabilizing Algorithm for Generalized Eigenpairs Extraction

Generalized eigendecomposition problem has been widely employed in many signal processing applications. In this paper, we propose a unified and self-stabilizing algorithm, which is able to extract the first principal and minor generalized eigenvectors of a matrix pencil of two vector sequences adaptively. Furthermore, we extend the proposed algorithm to extract multiple generalized eigenvectors. The performance analysis shows that only the desired equilibrium point of the proposed algorithm is stable and all others are (unstable) repellers or saddle points. Convergence analysis based on the deterministic discrete-time approach shows that, for a step size within a certain range, the norm of the principal/minor state vector converges to a fixed value that relates to the corresponding principal/minor generalized eigenvalue. Thus, the proposed algorithm is a generalized eigenpairs (eigenvectors and eigenvalues) extraction algorithm. Finally, the simulation experiments are carried to further demonstrate the efficiency of the proposed algorithm.