A global–local optimization approach to parameter estimation of RBF-type models

Term Selection for a Class of Separable Nonlinear Models

In this paper, we consider the term selection problem for a class of separable nonlinear models. The strategy is a two-step process in which the nonlinear parameters of the model are first optimized by a variable projection method, and then the least absolute shrinkage and selection operator are adopted to obtain a sparse solution by picking out the critical terms automatically. This process may be repeated several times. The proposed algorithm is tested on parameter estimation problems for an exponential model and a neural network-based model. The numerical results show that the proposed algorithm can pick out the appropriate terms from the overparameterized model and the obtained parsimonious model performs better than other methods.

A Regularized SNPOM for Stable Parameter Estimation of RBF-AR(X) Model

Recently, the radial basis function (RBF) network-style coefficients AutoRegressive (with exogenous inputs) [RBF-AR(X)] model identified by the structured nonlinear parameter optimization method (SNPOM) has attracted considerable interest because of its significant performance in nonlinear system modeling. However, this promising technique may occasionally confront the problem that the parameters are divergent in the optimization process, which may be a potential issue ignored by most researchers. In this paper, a regularized SNPOM, together with the regularization parameter detection technique, is presented to estimate the parameters of RBF-AR(X) models. This approach first separates the parameters of an RBF-AR(X) model into a linear parameters set and a nonlinear parameters set, and then combines a gradient-based nonlinear optimization algorithm for estimating the nonlinear parameters and the regularized least squares method for estimating the linear parameters. Several examples demonstrate that the proposed approach is effective to cope with the potential unstable problem in the parameters search process, and may also yield better or similar multistep forecasting accuracy and better robustness than the previous method.

A variable projection approach for efficient estimation of RBF-ARX model

The radial basis function network-based autoregressive with exogenous inputs (RBF-ARX) models have much more linear parameters than nonlinear parameters. Taking advantage of this special structure, a variable projection algorithm is proposed to estimate the model parameters more efficiently by eliminating the linear parameters through the orthogonal projection. The proposed method not only substantially reduces the dimension of parameter space of RBF-ARX model but also results in a better-conditioned problem. In this paper, both the full Jacobian matrix of Golub and Pereyra and the Kaufman's simplification are used to test the performance of the algorithm. An example of chaotic time series modeling is presented for the numerical comparison. It clearly demonstrates that the proposed approach is computationally more efficient than the previous structured nonlinear parameter optimization method and the conventional Levenberg-Marquardt algorithm without the parameters separated. Finally, the proposed method is also applied to a simulated nonlinear single-input single-output process, a time-varying nonlinear process and a real multiinput multioutput nonlinear industrial process to illustrate its usefulness.