A Dual Filter Based on Radial Basis Function Neural Networks and Kalman Filters with Application to Numerical Wave Prediction Models

The aim of this study is to introduce and evaluate a dual filter that combines Radial Basis Function neural networks and Kalman filters to enhance the accuracy of numerical wave prediction models. Unlike the existing methods, which focus solely on systematic errors, the proposed framework concurrently targets both systematic and non-systematic parts of forecast errors, significantly reducing the bias and variability in significant wave height predictions. The produced filter is self-adaptive, identifying optimal Radial Basis Function network configurations through an automated process involving various network parameters tuning. The produced computational system is assessed using a time-window procedure applied across divergent time periods and regions in the Aegean Sea and the Pacific Ocean. The results reveal a consistent performance, outperforming classic Kalman filters with an average reduction of 53% in bias and 28% in RMSE, underlining the dual filter's potential as a robust post-processing tool for environmental simulations.

Online Identification of Nonlinear Systems With Separable Structure

Separable nonlinear models (SNLMs) are of great importance in system modeling, signal processing, and machine learning because of their flexible structure and excellent description of nonlinear behaviors. The online identification of such models is quite challenging, and previous related work usually ignores the special structure where the estimated parameters can be partitioned into a linear and a nonlinear part. In this brief, we propose an efficient first-order recursive algorithm for SNLMs by introducing the variable projection (VP) step. The proposed algorithm utilizes the recursive least-squares method to eliminate the linear parameters, resulting in a reduced function. Then, the stochastic gradient descent (SGD) algorithm is employed to update the parameters of the reduced function. By considering the tight coupling relationship between linear parameters and nonlinear parameters, the proposed first-order VP algorithm is more efficient and robust than the traditional SGD algorithm and alternating optimization algorithm. More importantly, since the proposed algorithm just uses the first-order information, it is easier to apply it to large-scale models. Numerical results on examples of different sizes confirm the effectiveness and efficiency of the proposed algorithm.

Deep learning based model predictive controller on a magnetic levitation ball system

The magnetic levitation (maglev) ball system is a prototypical Single-Input-Single-Output (SISO) system, characterized by its pronounced nonlinearity, rapid response, and open-loop instability. It serves as the basis for many industrial devices. For describing the dynamics of the maglev ball system precisely in the pseudo linear model, the long short-term memory (LSTM) based auto-regressive model with exogenous input variables (LSTM-ARX) is proposed. Firstly, the LSTM network is modified by incorporating the auto-regressive structure with respect to sequence input, allowing it to deduce a locally linearized model without the need for Taylor expansion. Then, the LSTM-ARX model is transformed into a linear parameter varying (LPV) state space model, and upon this foundation, a model predictive controller (MPC) is proposed. Specifically, when deducing the MPC, the deep learning-based model is linearized by fixing its state input at the current state, so that the nonlinear, non-convex optimization problem can be converted to a finite-horizon quadratic programming problem, thereby deriving the explicit form of MPC. To further enhance the efficiency of the controller in real-time control tasks, a predictive functional controller (PFC) is proposed. It employs multiple nonlinear functions to fit the control sequence, thereby reducing the number of decision variables of the on-line optimization problem in MPC. The proposed controller was successfully applied to the real-time control of the maglev ball system. Simulation and real-time control experiments have validated the improvement in transient performance and efficiency of the LSTM-ARX model-based PFC (LSTM-ARX-PFC).

Two-stage extended recursive gradient algorithm for locally linear RBF-based autoregressive models with colored noises

A novel parameter identification method for locally linear radial basis function-based autoregressive models in presence of colored noises is proposed in this paper. Taking advantage of the global nonlinear and local linear structural characteristics of the models, two dynamical criterion functions are constructed based on the separated parameters to realize the dynamical acquisition and utilization of the entire process data. Two recursive gradient sub-algorithms are derived for estimating the separated parameters by using the nonlinear gradient optimization. To coordinate the associated variables existing in the sub-algorithms and to estimate the unmeasurable noise terms, we combine the sub-algorithms and propose a two-stage extended recursive gradient (2S-ERG) algorithm. In addition, an extended recursive gradient algorithm is given as a comparison. The feasibility of the 2S-ERG algorithm is validated by numerical simulations.

Insights Into Algorithms for Separable Nonlinear Least Squares Problems

Separable nonlinear least squares (SNLLS) problems have attracted interest in a wide range of research fields such as machine learning, computer vision, and signal processing. During the past few decades, several algorithms, including the joint optimization algorithm, alternated least squares (ALS) algorithm, embedded point iterations (EPI) algorithm, and variable projection (VP) algorithms, have been employed for solving SNLLS problems in the literature. The VP approach has been proven to be quite valuable for SNLLS problems and the EPI method has been successful in solving many computer vision tasks. However, no clear explanations about the intrinsic relationships of these algorithms have been provided in the literature. In this paper, we give some insights into these algorithms for SNLLS problems. We derive the relationships among different forms of the VP algorithms, EPI algorithm and ALS algorithm. In addition, the convergence and robustness of some algorithms are investigated. Moreover, the analysis of the VP algorithm generates a negative answer to Kaufman's conjecture. Numerical experiments on the image restoration task, fitting the time series data using the radial basis function network based autoregressive (RBF-AR) model, and bundle adjustment are given to compare the performance of different algorithms.

RBF-ARX model-based fast robust MPC approach to an inverted pendulum

In general, the online computation burden of robust model predictive control (RMPC) is very heavy, and the mechanical model of a plant, which is used in RMPC, is hard to obtain precisely in real industry. These issues may largely restrict the applicability of RMPC in real applications. This paper proposes a RBF-ARX (state-dependent Auto-Regressive model with eXogenous input and Radial Basis Function network type coefficients) model-based efficient robust predictive control (RBF-ARX-ERPC) approach to an inverted pendulum system, which is a complete and systematic method for designing robust MPC controller because it integrates the RBF-ARX modeling method and a fast RMPC approach. First, based on the offline identified RBF-ARX model without offset term, two convex polytopic sets are constructed to wrap the globally nonlinear behavior of the system. Then, the optimization problem of implementing a quasi-min-max MPC algorithm including several linear matrix inequalities (LMIs) is formulated, and it is solved offline to synthesize a sequence of explicit control laws that correspond to a sequence of asymptotically stable invariant ellipsoids, of which all the optimization results are stored in a look-up table. During the online real-time control, the controller only needs to carry out a simple state-vector computation and bisection search. The proposed approach is applied to an actual linear one-stage inverted pendulum (LOSIP), which is a fast-responding and nonlinear plant. The real-time control experiments demonstrate the effectiveness of the proposed RBF-ARX model-based efficient RMPC approach.

Modified Gram-Schmidt Method-Based Variable Projection Algorithm for Separable Nonlinear Models

Separable nonlinear models are very common in various research fields, such as machine learning and system identification. The variable projection (VP) approach is efficient for the optimization of such models. In this paper, we study various VP algorithms based on different matrix decompositions. Compared with the previous method, we use the analytical expression of the Jacobian matrix instead of finite differences. This improves the efficiency of the VP algorithms. In particular, based on the modified Gram-Schmidt (MGS) method, a more robust implementation of the VP algorithm is introduced for separable nonlinear least-squares problems. In numerical experiments, we compare the performance of five different implementations of the VP algorithm. Numerical results show the efficiency and robustness of the proposed MGS method-based VP algorithm.

A Regularized SNPOM for Stable Parameter Estimation of RBF-AR(X) Model

Recently, the radial basis function (RBF) network-style coefficients AutoRegressive (with exogenous inputs) [RBF-AR(X)] model identified by the structured nonlinear parameter optimization method (SNPOM) has attracted considerable interest because of its significant performance in nonlinear system modeling. However, this promising technique may occasionally confront the problem that the parameters are divergent in the optimization process, which may be a potential issue ignored by most researchers. In this paper, a regularized SNPOM, together with the regularization parameter detection technique, is presented to estimate the parameters of RBF-AR(X) models. This approach first separates the parameters of an RBF-AR(X) model into a linear parameters set and a nonlinear parameters set, and then combines a gradient-based nonlinear optimization algorithm for estimating the nonlinear parameters and the regularized least squares method for estimating the linear parameters. Several examples demonstrate that the proposed approach is effective to cope with the potential unstable problem in the parameters search process, and may also yield better or similar multistep forecasting accuracy and better robustness than the previous method.

Generalized Hybrid Constructive Learning Algorithm for Multioutput RBF Networks

An efficient generalized hybrid constructive (GHC) learning algorithm for multioutput radial basis function (RBF) networks is proposed to obtain a compact network with good generalization capability. By this algorithm, one can train the adjustable parameters and determine the optimal network structure simultaneously. First, an initialization method based on the growing and pruning algorithm is utilized to select the important initial hidden neurons and candidate ones. Then, by introducing a generalized hidden matrix, a structured parameter optimization algorithm is presented to train multioutput RBF network with fixed size, which combines Levenberg-Marquardt (LM) algorithm with least-square method together. Beginning from an appropriate number of hidden neurons, new neurons chosen from the candidates are added one by one each time when the training entraps into local minima. By incorporating an improved incremental constructive scheme, the training is built on previous results after adding new neurons such that the GHC learning algorithm avoids a trial-and-error procedure. Furthermore, based on the improved computation for LM training, the memory limitation problem is solved. The computational complexity analysis and experimental results demonstrate that better performance is efficiently achieved by this algorithm.

A Hybrid Constructive Algorithm for Single-Layer Feedforward Networks Learning

Single-layer feedforward networks (SLFNs) have been proven to be a universal approximator when all the parameters are allowed to be adjustable. It is widely used in classification and regression problems. The SLFN learning involves two tasks: determining network size and training the parameters. Most current algorithms could not be satisfactory to both sides. Some algorithms focused on construction and only tuned part of the parameters, which may not be able to achieve a compact network. Other gradient-based optimization algorithms focused on parameters tuning while the network size has to be preset by the user. Therefore, trial-and-error approach has to be used to search the optimal network size. Because results of each trial cannot be reused in another trial, it costs much computation. In this paper, a hybrid constructive (HC)algorithm is proposed for SLFN learning, which can train all the parameters and determine the network size simultaneously. At first, by combining Levenberg-Marquardt algorithm and least-square method, a hybrid algorithm is presented for training SLFN with fixed network size. Then,with the hybrid algorithm, an incremental constructive scheme is proposed. A new randomly initialized neuron is added each time when the training entrapped into local minima. Because the training continued on previous results after adding new neurons, the proposed HC algorithm works efficiently. Several practical problems were given for comparison with other popular algorithms. The experimental results demonstrated that the HC algorithm worked more efficiently than those optimization methods with trial and error, and could achieve much more compact SLFN than those construction algorithms.

A variable projection approach for efficient estimation of RBF-ARX model

The radial basis function network-based autoregressive with exogenous inputs (RBF-ARX) models have much more linear parameters than nonlinear parameters. Taking advantage of this special structure, a variable projection algorithm is proposed to estimate the model parameters more efficiently by eliminating the linear parameters through the orthogonal projection. The proposed method not only substantially reduces the dimension of parameter space of RBF-ARX model but also results in a better-conditioned problem. In this paper, both the full Jacobian matrix of Golub and Pereyra and the Kaufman's simplification are used to test the performance of the algorithm. An example of chaotic time series modeling is presented for the numerical comparison. It clearly demonstrates that the proposed approach is computationally more efficient than the previous structured nonlinear parameter optimization method and the conventional Levenberg-Marquardt algorithm without the parameters separated. Finally, the proposed method is also applied to a simulated nonlinear single-input single-output process, a time-varying nonlinear process and a real multiinput multioutput nonlinear industrial process to illustrate its usefulness.

Incorporating significant amino acid pairs and protein domains to predict RNA splicing-related proteins with functional roles

Machinery of pre-mRNA splicing is carried out through the interaction of RNA sequence elements and a variety of RNA splicing-related proteins (SRPs) (e.g. spliceosome and splicing factors). Alternative splicing, which is an important post-transcriptional regulation in eukaryotes, gives rise to multiple mature mRNA isoforms, which encodes proteins with functional diversities. However, the regulation of RNA splicing is not yet fully elucidated, partly because SRPs have not yet been exhaustively identified and the experimental identification is labor-intensive. Therefore, we are motivated to design a new method for identifying SRPs with their functional roles in the regulation of RNA splicing. The experimentally verified SRPs were manually curated from research articles. According to the functional annotation of Splicing Related Gene Database, the collected SRPs were further categorized into four functional groups including small nuclear Ribonucleoprotein, Splicing Factor, Splicing Regulation Factor and Novel Spliceosome Protein. The composition of amino acid pairs indicates that there are remarkable differences among four functional groups of SRPs. Then, support vector machines (SVMs) were utilized to learn the predictive models for identifying SRPs as well as their functional roles. The cross-validation evaluation presents that the SVM models trained with significant amino acid pairs and functional domains could provide a better predictive performance. In addition, the independent testing demonstrates that the proposed method could accurately identify SRPs in mammals/plants as well as effectively distinguish between SRPs and RNA-binding proteins. This investigation provides a practical means to identifying potential SRPs and a perspective for exploring the regulation of RNA splicing.

Modeling and control approach to a distinctive quadrotor helicopter

The referenced quadrotor helicopter in this paper has a unique configuration. It is more complex than commonly used quadrotors because of its inaccurate parameters, unideal symmetrical structure and unknown nonlinear dynamics. A novel method was presented to handle its modeling and control problems in this paper, which adopts a MIMO RBF neural nets-based state-dependent ARX (RBF-ARX) model to represent its nonlinear dynamics, and then a MIMO RBF-ARX model-based global LQR controller is proposed to stabilize the quadrotor's attitude. By comparing with a physical model-based LQR controller and an ARX model-set-based gain scheduling LQR controller, superiority of the MIMO RBF-ARX model-based control approach was confirmed. This successful application verified the validity of the MIMO RBF-ARX modeling method to the quadrotor helicopter with complex nonlinearity.

Using Radial Basis Function Networks for Function Approximation and Classification

The radial basis function (RBF) network has its foundation in the conventional approximation theory. It has the capability of universal approximation. The RBF network is a popular alternative to the well-known multilayer perceptron (MLP), since it has a simpler structure and a much faster training process. In this paper, we give a comprehensive survey on the RBF network and its learning. Many aspects associated with the RBF network, such as network structure, universal approimation capability, radial basis functions, RBF network learning, structure optimization, normalized RBF networks, application to dynamic system modeling, and nonlinear complex-valued signal processing, are described. We also compare the features and capability of the two models.

Construction of tunable radial basis function networks using orthogonal forward selection

An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO misclassification rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstrated using examples taken from both regression and classification applications.

Hepatitis C virus contact map prediction based on binary encoding strategy

Inter-residue contact map is an important two-dimensional representation of protein spatial structure, and has much potential application in the area of understanding protein fold mechanism. In the present note, a 19-bit binary input encoding strategy, integrating with residue pair conformational features (possible residue pairwise, residue classification, secondary structure, sequence length, and sequence separation information), is proposed for the purpose of capturing mapping relationship of protein sequence. Simulation results on a set of 61 hepatitis C virus (HCV) retrieved from the protein data bank (PDB) demonstrate that the proposed encoding scheme could precisely capture conformational patterns within HCV protein sequence. This promising result could provide some useful insights into the nature of HCV protein fold mechanism.

A Hybrid Forward Algorithm for RBF Neural Network Construction

This paper proposes a novel hybrid forward algorithm (HFA) for the construction of radial basis function (RBF) neural networks with tunable nodes. The main objective is to efficiently and effectively produce a parsimonious RBF neural network that generalizes well. In this study, it is achieved through simultaneous network structure determination and parameter optimization on the continuous parameter space. This is a mixed integer hard problem and the proposed HFA tackles this problem using an integrated analytic framework, leading to significantly improved network performance and reduced memory usage for the network construction. The computational complexity analysis confirms the efficiency of the proposed algorithm, and the simulation results demonstrate its effectiveness.

Improved optimization of soft-partition-weighted-sum filters and their application to image restoration

Soft-partition-weighted-sum (Soft-PWS) filters are a class of spatially adaptive moving-window filters for signal and image restoration. Their performance is shown to be promising. However, optimization of the Soft-PWS filters has received only limited attention. Earlier work focused on a stochastic-gradient method that is computationally prohibitive in many applications. We describe a novel radial basis function interpretation of the Soft-PWS filters and present an efficient optimization procedure. We apply the filters to the problem of noise reduction. The experimental results show that the Soft-PWS filter outperforms the standard partition-weighted-sum filter and the Wiener filter.