Modeling of nonlinear nonstationary dynamic systems with a novel class of artificial neural networks

Nonlinear System Identification of Neural Systems from Neurophysiological Signals

The human nervous system is one of the most complicated systems in nature. Complex nonlinear behaviours have been shown from the single neuron level to the system level. For decades, linear connectivity analysis methods, such as correlation, coherence and Granger causality, have been extensively used to assess the neural connectivities and input-output interconnections in neural systems. Recent studies indicate that these linear methods can only capture a certain amount of neural activities and functional relationships, and therefore cannot describe neural behaviours in a precise or complete way. In this review, we highlight recent advances in nonlinear system identification of neural systems, corresponding time and frequency domain analysis, and novel neural connectivity measures based on nonlinear system identification techniques. We argue that nonlinear modelling and analysis are necessary to study neuronal processing and signal transfer in neural systems quantitatively. These approaches can hopefully provide new insights to advance our understanding of neurophysiological mechanisms underlying neural functions. These nonlinear approaches also have the potential to produce sensitive biomarkers to facilitate the development of precision diagnostic tools for evaluating neurological disorders and the effects of targeted intervention.

Modeling of batch processes using explicitly time-dependent artificial neural networks

A neural network architecture incorporating time dependency explicitly, proposed recently, for modeling nonlinear nonstationary dynamic systems is further developed in this paper, and three alternate configurations are proposed to represent the dynamics of batch chemical processes. The first configuration consists of L subnets, each having M inputs representing the past samples of process inputs and output; each subnet has a hidden layer with polynomial activation function; the outputs of the hidden layer are combined and acted upon by an explicitly time-dependent modulation function. The outputs of all the subnets are summed to obtain the output prediction. In the second configuration, additional weights are incorporated to obtain a more generalized model. In the third configuration, the subnets are eliminated by incorporating an additional hidden layer consisting of L nodes. Backpropagation learning algorithm is formulated for each of the proposed neural network configuration to determine the weights, the polynomial coefficients, and the modulation function parameters. The modeling capability of the proposed neural network configuration is evaluated by employing it to represent the dynamics of a batch reactor in which a consecutive reaction takes place. The results show that all the three time-varying neural networks configurations are able to represent the batch reactor dynamics accurately, and it is found that the third configuration is exhibiting comparable or better performance over the other two configurations while requiring much smaller number of parameters. The modeling ability of the third configuration is further validated by applying to modeling a semibatch polymerization reactor challenge problem. This paper illustrates that the proposed approach can be applied to represent dynamics of any batch/semibatch process.

Online modeling with tunable RBF network

In this paper, we propose a novel online modeling algorithm for nonlinear and nonstationary systems using a radial basis function (RBF) neural network with a fixed number of hidden nodes. Each of the RBF basis functions has a tunable center vector and an adjustable diagonal covariance matrix. A multi-innovation recursive least square (MRLS) algorithm is applied to update the weights of RBF online, while the modeling performance is monitored. When the modeling residual of the RBF network becomes large in spite of the weight adaptation, a node identified as insignificant is replaced with a new node, for which the tunable center vector and diagonal covariance matrix are optimized using the quantum particle swarm optimization (QPSO) algorithm. The major contribution is to combine the MRLS weight adaptation and QPSO node structure optimization in an innovative way so that it can track well the local characteristic in the nonstationary system with a very sparse model. Simulation results show that the proposed algorithm has significantly better performance than existing approaches.

Spectral analysis for nonstationary and nonlinear systems: a discrete-time-model-based approach

A new frequency-domain analysis framework for nonlinear time-varying systems is introduced based on parametric time-varying nonlinear autoregressive with exogenous input models. It is shown how the time-varying effects can be mapped to the generalized frequency response functions (FRFs) to track nonlinear features in frequency, such as intermodulation and energy transfer effects. A new mapping to the nonlinear output FRF is also introduced. A simulated example and the application to intracranial electroencephalogram data are used to illustrate the theoretical results.

Tracking temporal evolution of nonlinear dynamics in hippocampus using time-varying volterra kernels

Hippocampus and other parts of the cortex are not stationary, but change as a function of time and experience. The goal of this study is to apply adaptive modeling techniques to the tracking of multiple-input, multiple-output (MIMO) nonlinear dynamics underlying spike train transformations across brain subregions, e.g. CA3 and CA1 of the hippocampus. A stochastic state point process adaptive filter will be used to track the temporal evolutions of both feedforward and feedback kernels in the natural flow of multiple behavioral events.

Nonstationary modeling of neural population dynamics

A stochastic state point-process adaptive filter was used to track the temporal evolution of several simulated nonlinear dynamical systems. The estimated Laguerre coefficients and Laguerre poles were used to reconstruct the feedforward and feedback kernels in the system. Simulations showed that the proposed method could track the actual underlying changes of nonlinear kernels using spike input and spike output information alone. The estimated models also converge quickly to the actual models after abrupt step changes in kernels. The proposed method can be used to track the functional input-output properties of neural systems as a result of learning, changes in context, aging or other factors in the natural flow of behavioral events.

Representation of time-varying nonlinear systems with time-varying principal dynamic modes

System identification of nonlinear time-varying (TV) systems has been a daunting task, as the number of parameters required for accurate identification is often larger than the number of data points available, and scales with the number of data points. Further, a 3-D graphical representation of TV second-order nonlinear dynamics without resorting to taking slices along one of the four axes has been a significant challenge to date. In this paper, we present a TV principal dynamic mode (TVPDM) method which overcomes these deficiencies. The TVPDM, by design, reduces one dimension, and by projecting PDM coefficients onto a set of basis functions, both nonstationary and nonlinear dynamics can be characterized. Another significant advantage of the TVPDM is its ability to discriminate the signal from noise dynamics, and provided that signal dynamics are orthogonal to each other, it has the capability to separate them. The efficacy of the proposed method is demonstrated with computer simulation examples comprised of various forms of nonstationarity and nonlinearity. The application of the TVPDM to the human heart rate and arterial blood pressure data during different postures is also presented and the results reveal significant nonstationarity even for short-term data recordings. The newly developed method has the potential to be a very useful tool for characterizing nonlinear TV systems, which has been a significant, challenging problem to date.

Nonlinear dynamic modeling of spike train transformations for hippocampal-cortical prostheses

One of the fundamental principles of cortical brain regions, including the hippocampus, is that information is represented in the ensemble firing of populations of neurons, i.e., spatio-temporal patterns of electrophysiological activity. The hippocampus has long been known to be responsible for the formation of declarative, or fact-based, memories. Damage to the hippocampus disrupts the propagation of spatio-temporal patterns of activity through hippocampal internal circuitry, resulting in a severe anterograde amnesia. Developing a neural prosthesis for the damaged hippocampus requires restoring this multiple-input, multiple-output transformation of spatio-temporal patterns of activity. Because the mechanisms underlying synaptic transmission and generation of electrical activity in neurons are inherently nonlinear, any such prosthesis must be based on a nonlinear multiple-input, multiple-output model. In this paper, we have formulated the transformational process of multi-site propagation of spike activity between two subregions of the hippocampus (CA3 and CA1) as the identification of a multiple-input, multiple-output (MIMO) system, and proposed that it can be decomposed into a series of multiple-input, single-output (MISO) systems. Each MISO system is modeled as a physiologically plausible structure that consists of 1) linear/nonlinear feedforward Volterra kernels modeling synaptic transmission and dendritic integration, 2) a linear feedback Volterra kernel modeling spike-triggered after-potentials, 3) a threshold for spike generation, 4) a summation process for somatic integration, and 5) a noise term representing intrinsic neuronal noise and the contributions of unobserved inputs. Input and output spike trains were recorded from hippocampal CA3 and CA1 regions of rats performing a spatial delayed-nonmatch-to-sample memory task that requires normal hippocampal function. Kernels were expanded with Laguerre basis functions and estimated using a maximum-likelihood method. Complexity of the feedforward kernel was progressively increased to capture higher-order system nonlinear dynamics. Results showed higher prediction accuracies as kernel complexity increased. Self-kernels describe the nonlinearities within each input. Cross-kernels capture the nonlinear interaction between inputs. Second- and third-order nonlinear models were found to successfully predict the CA1 output spike distribution based on CA3 input spike trains. First-order, linear models were shown to be insufficient.