Insights Into Algorithms for Separable Nonlinear Least Squares Problems

Online Identification of Nonlinear Systems With Separable Structure

Separable nonlinear models (SNLMs) are of great importance in system modeling, signal processing, and machine learning because of their flexible structure and excellent description of nonlinear behaviors. The online identification of such models is quite challenging, and previous related work usually ignores the special structure where the estimated parameters can be partitioned into a linear and a nonlinear part. In this brief, we propose an efficient first-order recursive algorithm for SNLMs by introducing the variable projection (VP) step. The proposed algorithm utilizes the recursive least-squares method to eliminate the linear parameters, resulting in a reduced function. Then, the stochastic gradient descent (SGD) algorithm is employed to update the parameters of the reduced function. By considering the tight coupling relationship between linear parameters and nonlinear parameters, the proposed first-order VP algorithm is more efficient and robust than the traditional SGD algorithm and alternating optimization algorithm. More importantly, since the proposed algorithm just uses the first-order information, it is easier to apply it to large-scale models. Numerical results on examples of different sizes confirm the effectiveness and efficiency of the proposed algorithm.

A Novel Symmetrical Peak Fitting Method Based on Improved WOA Algorithm for the Analysis of Microchip Electrophoresis Signals

The problem of overlapping peaks has been a challenge in microchip electrophoresis (ME) signal analysis. However, traditional peak fitting algorithms have difficulty analyzing overlapping peaks due to the high dependence on the starting point. In this study, we propose a symmetrical peak fitting method named the tent-mapped whale optimization algorithm and Levenberg–Marquardt (TWOALM), which combines a whale optimization algorithm (WOA) improved by one of the most commonly used chaotic maps, the tent map and the Levenberg–Marquardt (LM) algorithm. Specifically, we first derive the fitted model for the overlapping peaks, showing that it is separable nonlinear least squares, significantly reducing the number of parameters to be optimized. Second, we integrate the tent map into the WOA, which improves the convergence speed of the peak fitting algorithm. Finally, we propose an efficient peak-fitting algorithm that combines the improved WOA and LM. The advantage of the proposed algorithm is that it is significantly faster than WOA and significantly more accurate than the LM algorithm. The results of fitting the synthetic peaks and ME signals showed that the combination of the chaotic map-based WOA algorithm and the LM algorithm can significantly improve the peak fitting performance and provide an effective solution for the analysis of overlapping peaks.

Two-stage extended recursive gradient algorithm for locally linear RBF-based autoregressive models with colored noises

A novel parameter identification method for locally linear radial basis function-based autoregressive models in presence of colored noises is proposed in this paper. Taking advantage of the global nonlinear and local linear structural characteristics of the models, two dynamical criterion functions are constructed based on the separated parameters to realize the dynamical acquisition and utilization of the entire process data. Two recursive gradient sub-algorithms are derived for estimating the separated parameters by using the nonlinear gradient optimization. To coordinate the associated variables existing in the sub-algorithms and to estimate the unmeasurable noise terms, we combine the sub-algorithms and propose a two-stage extended recursive gradient (2S-ERG) algorithm. In addition, an extended recursive gradient algorithm is given as a comparison. The feasibility of the 2S-ERG algorithm is validated by numerical simulations.