Descent algorithms on oblique manifold for source-adaptive ICA contrast

A Dimensionality-Reducible Operational Optimal Control for Wastewater Treatment Process

Operational optimal control (OOC) is an essential component of wastewater treatment process (WWTP). The control variables usually are high-dimensional, nonlinear, and strongly coupled, which can easily fail traditional optimization control methods. Mathematically, these operational variables usually are in the unknown low-dimensional space embedded in the high-dimensional space. Therefore, the OOC problem of WWTP can be resolved as an optimization challenge involving low-dimensional space, and the unknown low-dimensional space is presented in the form of a set of controlled variables in a high-dimensional space, which is normal in real-world industries. Here, a dimension-reducible data-driven optimization control framework for WWTP is proposed. Considering the difficulty in elucidating the whole space of set points, a neural network is designed to approximate the constraint relationship between control variables. The search process is based on optimization methods in low-dimensional space embedded into Euclidean spaces. Furthermore, the convergence of the process is ensured via mathematical analysis. Finally, the experimental simulation of wastewater treatment revealed that this approach is effective for an optimal solution in control systems.

The Real Eigenpairs of Symmetric Tensors and Its Application to Independent Component Analysis

It has been proved that the determination of independent components (ICs) in the independent component analysis (ICA) can be attributed to calculating the eigenpairs of high-order statistical tensors of the data. However, previous works can only obtain approximate solutions, which may affect the accuracy of the ICs. In addition, the number of ICs would need to be set manually. Recently, an algorithm based on semidefinite programming (SDP) has been proposed, which utilizes the first-order gradient information of the Lagrangian function and can obtain all the accurate real eigenpairs. In this article, for the first time, we introduce this into the ICA field, which tends to further improve the accuracy of the ICs. Note that the number of eigenpairs of symmetric tensors is usually larger than the number of ICs, indicating that the results directly obtained by SDP are redundant. Thus, in practice, it is necessary to introduce second-order derivative information to identify local extremum solutions. Therefore, originating from the SDP method, we present a new modified version, called modified SDP (MSDP), which incorporates the concept of the projected Hessian matrix into SDP and, thus, can intellectually exclude redundant ICs and select true ICs. Some cases that have been tested in the experiments demonstrate its effectiveness. Experiments on the image/sound blind separation and real multi/hyperspectral image also show its superiority in improving the accuracy of ICs and automatically determining the number of ICs. In addition, the results on hyperspectral simulation and real data also demonstrate that MSDP is also capable of dealing with cases, where the number of features is less than the number of ICs.

Broad Coverage Precoding for 3D Massive MIMO with Huge Uniform Planar Arrays

In this paper, we propose a novel broad coverage precoder design for three-dimensional (3D) massive multi-input multi-output (MIMO) equipped with huge uniform planar arrays (UPAs). The desired two-dimensional (2D) angle power spectrum is assumed to be separable. We use the per-antenna constant power constraint and the semi-unitary constraint which are widely used in the literature. For normal broad coverage precoder design, the dimension of the optimization space is the product of the number of antennas at the base station (BS) and the number of transmit streams. With the proposed method, the design of the high-dimensional precoding matrices is reduced to that of a set of low-dimensional orthonormal vectors, and of a pair of low-dimensional vectors. The dimensions of the vectors in the set and the pair are the number of antennas per column and per row of the UPA, respectively. We then use optimization methods to generate the set of orthonormal vectors and the pair of vectors, respectively. Finally, simulation results show that the proposed broad coverage precoding matrices achieve nearly the same performance as the normal broad coverage precoder with much lower computational complexity.

Lie Group Methods in Blind Signal Processing

This paper deals with the use of Lie group methods to solve optimization problems in blind signal processing (BSP), including Independent Component Analysis (ICA) and Independent Subspace Analysis (ISA). The paper presents the theoretical fundamentals of Lie groups and Lie algebra, the geometry of problems in BSP as well as the basic ideas of optimization techniques based on Lie groups. Optimization algorithms based on the properties of Lie groups are characterized by the fact that during optimization motion, they ensure permanent bonding with a search space. This property is extremely significant in terms of the stability and dynamics of optimization algorithms. The specific geometry of problems such as ICA and ISA along with the search space homogeneity enable the use of optimization techniques based on the properties of the Lie groups O(n) and SO(n). An interesting idea is that of optimization motion in one-parameter commutative subalgebras and toral subalgebras that ensure low computational complexity and high-speed algorithms.

Design, Analysis, and Representation of Novel Five-Step DTZD Algorithm for Time-Varying Nonlinear Optimization

Continuous-time and discrete-time forms of Zhang dynamics (ZD) for time-varying nonlinear optimization have been developed recently. In this paper, a novel discrete-time ZD (DTZD) algorithm is proposed and investigated based on the previous research. Specifically, the DTZD algorithm for time-varying nonlinear optimization is developed by adopting a new Taylor-type difference rule. This algorithm is a five-step iteration process, and thus, is referred to as the five-step DTZD algorithm in this paper. Theoretical analysis and results of the proposed five-step DTZD algorithm are presented to highlight its excellent computational performance. The geometric representation of the proposed algorithm for time-varying nonlinear optimization is also provided. Comparative numerical results are illustrated with four examples to substantiate the efficacy and superiority of the proposed five-step DTZD algorithm for time-varying nonlinear optimization compared with the previous DTZD algorithms.

Discrete-Time Zhang Neural Network for Online Time-Varying Nonlinear Optimization With Application to Manipulator Motion Generation

In this brief, a discrete-time Zhang neural network (DTZNN) model is first proposed, developed, and investigated for online time-varying nonlinear optimization (OTVNO). Then, Newton iteration is shown to be derived from the proposed DTZNN model. In addition, to eliminate the explicit matrix-inversion operation, the quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method is introduced, which can effectively approximate the inverse of Hessian matrix. A DTZNN-BFGS model is thus proposed and investigated for OTVNO, which is the combination of the DTZNN model and the quasi-Newton BFGS method. In addition, theoretical analyses show that, with step-size h=1 and/or with zero initial error, the maximal residual error of the DTZNN model has an O(τ(2)) pattern, whereas the maximal residual error of the Newton iteration has an O(τ) pattern, with τ denoting the sampling gap. Besides, when h ≠ 1 and h ∈ (0,2) , the maximal steady-state residual error of the DTZNN model has an O(τ(2)) pattern. Finally, an illustrative numerical experiment and an application example to manipulator motion generation are provided and analyzed to substantiate the efficacy of the proposed DTZNN and DTZNN-BFGS models for OTVNO.

Range-based ICA using a nonsmooth quasi-newton optimizer for electroencephalographic source localization in focal epilepsy

Independent component analysis (ICA) aims at separating a multivariate signal into independent nongaussian signals by optimizing a contrast function with no knowledge on the mixing mechanism. Despite the availability of a constellation of contrast functions, a Hartley-entropy-based ICA contrast endowed with the discriminacy property makes it an appealing choice as it guarantees the absence of mixing local optima. Fueled by an outstanding source separation performance of this contrast function in practical instances, a succession of optimization techniques has recently been adopted to solve the ICA problem. Nevertheless, the nondifferentiability of the considered contrast restricts the choice of the optimizer to the class of derivative-free methods. On the contrary, this letter concerns a Riemannian quasi-Newton scheme involving an explicit expression for the gradient to optimize the contrast function that is differentiable almost everywhere. Furthermore, the inexact line search insisting on the weak Wolfe condition and a terminating criterion befitting the partly smooth function optimization have been generalized to manifold settings, leaving the previous results intact. The investigations with diversified images and the electroencephalographic (EEG) data acquired from 45 focal epileptic subjects demonstrate the efficacy of our approach in terms of computational savings and reliable EEG source localization, respectively. Additional experimental results are available in the online supplement.

Nonsmooth ICA contrast minimization using a Riemannian Nelder-Mead method

This brief concerns the design and application of a Riemannian Nelder-Mead algorithm to minimize a Hartley-entropybased contrast function to reliably estimate the sources from their mixtures. Despite its nondifferentiability, the contrast function is endowed with attractive properties such as discriminacy, and hence warrants an effort to be effectively handled by a derivative-free optimizer. Aside from tailoring the Nelder-Mead technique to the constraint set, namely, oblique manifold, the source separation results attained in an empirical study with quasi-correlated synthetic signals and digital images are presented, which favor the proposed method on a comparative basis.

Spherical mesh adaptive direct search for separating quasi-uncorrelated sources by range-based independent component analysis

It is seemingly paradoxical to the classical definition of the independent component analysis (ICA), that in reality, the true sources are often not strictly uncorrelated. With this in mind, this letter concerns a framework to extract quasi-uncorrelated sources with finite supports by optimizing a range-based contrast function under unit-norm constraints (to handle the inherent scaling indeterminacy of ICA) but without orthogonality constraints. Albeit the appealing contrast properties of the range-based function (e.g., the absence of mixing local optima), the function is not differentiable everywhere. Unfortunately, there is a dearth of literature on derivative-free optimizers that effectively handle such a nonsmooth yet promising contrast function. This is the compelling reason for the design of a nonsmooth optimization algorithm on a manifold of matrices having unit-norm columns with the following objectives: to ascertain convergence to a Clarke stationary point of the contrast function and adhere to the necessary unit-norm constraints more naturally. The proposed nonsmooth optimization algorithm crucially relies on the design and analysis of an extension of the mesh adaptive direct search (MADS) method to handle locally Lipschitz objective functions defined on the sphere. The applicability of the algorithm in the ICA domain is demonstrated with simulations involving natural, face, aerial, and texture images.