ReNFuzz-LF: A Recurrent Neurofuzzy System for Short-Term Load Forecasting

A neurofuzzy system is proposed for short-term electric load forecasting. The fuzzy rule base of ReNFuzz-LF consists of rules with dynamic consequent parts that are small-scale recurrent neural networks with one hidden layer, whose neurons have local output feedback. The particular representation maintains the local learning nature of the typical static fuzzy model, since the dynamic consequent parts of the fuzzy rules can be considered as subsystems operating at the subspaces defined by the fuzzy premise parts, and they are interconnected through the defuzzification part. The Greek power system is examined, and hourly based predictions are extracted for the whole year. The recurrent nature of the forecaster leads to the use of a minimal set of inputs, since the temporal relations of the electric load time-series are identified without any prior knowledge of the appropriate past load values being necessary. An extensive simulation analysis is conducted, and the forecaster’s performance is evaluated using appropriate metrics (APE, RMSE, forecast error duration curve). ReNFuzz-LF performs efficiently, attaining an average percentage error of 1.35% and an average yearly absolute error of 86.3 MW. Finally, the performance of the proposed forecaster is compared to a series of Computational Intelligence based models, such that the learning characteristics of ReNFuzz-LF are highlighted.

A Vibration Control Method for Hybrid-Structured Flexible Manipulator Based on Sliding Mode Control and Reinforcement Learning

The hybrid-structured flexible manipulator has a complex structure and strong coupling between state variables. Meanwhile, the natural frequency of the hybrid-structured flexible manipulator varies with the motion of the telescopic joint, so it is difficult to suppress the vibration quickly. In this article, the tip state signal of the hybrid-structured flexible manipulator is decomposed into elastic vibration signal and tip vibration equilibrium position signal, and a combined control method is proposed to improve tip positioning accuracy and trajectory tracking accuracy. In the proposed combined control method, an improved nominal model-based sliding mode controller (NMBSMC) is used as the main controller to output the driving torque, and an actor-critic-based reinforcement learning controller (ACBRLC) is used as an auxiliary controller to output small compensation torque. The improved NMBSMC can be divided into a nominal model-based sliding mode robust controller and a practical model-based integral sliding mode controller. Two sliding mode controllers with different structures make full use of the mathematical model and the measured data of the actual system to improve the vibration equilibrium position tracking accuracy. The ACBRLC uses the tip elastic vibration signal and the prioritized experience replay method to obtain the small reverse compensation torque, which is superimposed with the output of the NMBSMC to suppress tip vibration and improve the positioning accuracy of the hybrid-structured flexible manipulator. Finally, several groups of experiments are designed to verify the effectiveness and robustness of the proposed combined control method.

Dynamic System Identification and Prediction Using a Self-Evolving Takagi–Sugeno–Kang-Type Fuzzy CMAC Network

This study proposes a Self-evolving Takagi-Sugeno-Kang-type Fuzzy Cerebellar Model Articulation Controller (STFCMAC) for solving identification and prediction problems. The proposed STFCMAC model uses the hypercube firing strength for generating external loops and internal feedback. A differentiable Gaussian function is used in the fuzzy hypercube cell of the proposed model, and a linear combination function of the model inputs is used as the output of the proposed model. The learning process of the STFCMAC is initiated using an empty hypercube base. Fuzzy hypercube cells are generated through structure learning, and the related parameters are adjusted by a gradient descent algorithm. The proposed STFCMAC network has some advantages that are summarized as follows: (1) the model automatically selects the parameters of the memory structure, (2) it requires few fuzzy hypercube cells, and (3) it performs identification and prediction adaptively and effectively.

Interval Type-2 Fuzzy Neural Networks for Chaotic Time Series Prediction: A Concise Overview

Chaotic time series widely exists in nature and society (e.g., meteorology, physics, economics, etc.), which usually exhibits seemingly unpredictable features due to its inherent nonstationary and high complexity. Thankfully, multifarious advanced approaches have been developed to tackle the prediction issues, such as statistical methods, artificial neural networks (ANNs), and support vector machines. Among them, the interval type-2 fuzzy neural network (IT2FNN), which is a synergistic integration of fuzzy logic systems and ANNs, has received wide attention in the field of chaotic time series prediction. This paper begins with the structural features and superiorities of IT2FNN. Moreover, chaotic characters identification and phase-space reconstruction matters for prediction are presented. In addition, we also offer a comprehensive review of state-of-the-art applications of IT2FNN, with an emphasis on chaotic time series prediction and summarize their main contributions as well as some hardware implementations for computation speedup. Finally, this paper trends and extensions of this field, along with an outlook of future challenges are revealed. The primary objective of this paper is to serve as a tutorial or referee for interested researchers to have an overall picture on the current developments and identify their potential research direction to further investigation.

Temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems

In this paper, a novel temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems is proposed. The proposed structure consists of recurrent hidden neurons having weighted self-feedback loops and a weighted linear feed-through from the input layer directly to the output layer neuron(s). The dynamic back-propagation algorithm is developed and used for updating the parameters of the proposed structure. To improve the performance of learning algorithm, discrete Lyapunov stability method is used to develop an adaptive learning rate scheme. This scheme ensures the faster convergence of the parameters and maintains the stability of the system. A total of 5 complex nonlinear systems are used to test and compare the performance of the proposed network with other neural network structures. The disturbance rejection tests are also carried out to check whether the proposed scheme is able to handle the external disturbance/noise signals effects or not. The obtained results show the efficacy of the proposed method.

Design of a heart rate controller for treadmill exercise using a recurrent fuzzy neural network

BACKGROUND AND OBJECTIVE

In this study, we developed a computer controlled treadmill system using a recurrent fuzzy neural network heart rate controller (RFNNHRC). Treadmill speeds and inclines were controlled by corresponding control servo motors. The RFNNHRC was used to generate the control signals to automatically control treadmill speed and incline to minimize the user heart rate deviations from a preset profile.

METHODS

The RFNNHRC combines a fuzzy reasoning capability to accommodate uncertain information and an artificial recurrent neural network learning process that corrects for treadmill system nonlinearities and uncertainties. Treadmill speeds and inclines are controlled by the RFNNHRC to achieve minimal heart rate deviation from a pre-set profile using adjustable parameters and an on-line learning algorithm that provides robust performance against parameter variations. The on-line learning algorithm of RFNNHRC was developed and implemented using a dsPIC 30F4011 DSP.

RESULTS

Application of the proposed control scheme to heart rate responses of runners resulted in smaller fluctuations than those produced by using proportional integra control, and treadmill speeds and inclines were smoother. The present experiments demonstrate improved heart rate tracking performance with the proposed control scheme.

CONCLUSIONS

The RFNNHRC scheme with adjustable parameters and an on-line learning algorithm was applied to a computer controlled treadmill system with heart rate control during treadmill exercise. Novel RFNNHRC structure and controller stability analyses were introduced. The RFNNHRC were tuned using a Lyapunov function to ensure system stability. The superior heart rate control with the proposed RFNNHRC scheme was demonstrated with various pre-set heart rates.

A Novel Interactively Recurrent Self-Evolving Fuzzy CMAC and Its Classification Applications

In this paper, an Interactively Recurrent Self-evolving Fuzzy Cerebellar Model Articulation Controller (IRSFCMAC) model is developed for solving classification problems. The proposed IRSFCMAC classifier consists of internal feedback and external loops, which are generated by the hypercube cell firing strength to itself and other hypercube cells. The learning process of the IRSFCMAC gets started with an empty hypercube base, and then all of hypercube cells are generated and learned online via structure and parameter learning, respectively. The structure learning algorithm is based on the degree measure to determine the number of hypercube cells. The parameter learning algorithm, based on the gradient descent method, adjusts the shapes of the membership functions and the corresponding fuzzy weights of the IRSFCMAC. Finally, the proposed IRSFCMAC model is tested by four benchmark classification problems. Experimental results show that the proposed IRSFCMAC model has superior performance than traditional FCMAC and other models.

INDIRECT ADAPTIVE CONTROL OF NONLINEAR SYSTEMS BASED ON BILINEAR NEURO-FUZZY APPROXIMATION

In this paper, we investigate the indirect adaptive regulation problem of unknown affine in the control nonlinear systems. The proposed approach consists of choosing an appropriate system approximation model and a proper control law, which will regulate the system under the certainty equivalence principle. The main difference from other relevant works of the literature lies in the proposal of a potent approximation model that is bilinear with respect to the tunable parameters. To deploy the bilinear model, the components of the nonlinear plant are initially approximated by Fuzzy subsystems. Then, using appropriately defined fuzzy rule indicator functions, the initial dynamical fuzzy system is translated to a dynamical neuro-fuzzy model, where the indicator functions are replaced by High Order Neural Networks (HONNS), trained by sampled system data. The fuzzy output partitions of the initial fuzzy components are also estimated based on sampled data. This way, the parameters to be estimated are the weights of the HONNs and the centers of the output partitions, both arranged in matrices of appropriate dimensions and leading to a matrix to matrix bilinear parametric model. Based on the bilinear parametric model and the design of appropriate control law we use a Lyapunov stability analysis to obtain parameter adaptation laws and to regulate the states of the system. The weight updating laws guarantee that both the identification error and the system states reach zero exponentially fast, while keeping all signals in the closed loop bounded. Moreover, introducing a method of "concurrent" parameter hopping, the updating laws are modified so that the existence of the control signal is always assured. The main characteristic of the proposed approach is that the a priori experts information required by the identification scheme is extremely low, limited to the knowledge of the signs of the centers of the fuzzy output partitions. Therefore, the proposed scheme is not vulnerable to initial design assumptions. Simulations on selected examples of well-known benchmarks illustrate the potency of the method.

Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network

This paper presents a novel recurrent fuzzy neural network, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for prediction and identification of dynamic systems. The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN is composed of a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNNs learning starts with an empty rule base and all of the rules are generated and learned online through a simultaneous structure and parameter learning. An on-line clustering algorithm is effective in generating fuzzy rules. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN for the prediction and identification of dynamic plants and compare it to other well-known recurrent FNNs. The proposed model obtains enhanced performance results.

A DISCRETE-TIME RECURRENT NEUROFUZZY NETWORK FOR BLACK-BOX MODELING OF INSULIN DYNAMICS IN DIABETIC TYPE-1 PATIENTS

In this work we present a data-driven modeling of the insulin dynamics in different in silico patients using a recurrent neural network with output feedback. The inputs for the identification is the rate of insulin (μU / dl / min) applied to the patient, and blood glucose concentration. The output is insulin concentration (μU / ml) present in the blood stream. Once completed the off-line modeling, this model could be used for on-line monitoring of the insulin concentration for a better treatment. The learning law of the recurrent neural network is inspired by adaptive observer theory, and proven to be convergent in the parameters and stable in the Lyapunov sense, even with only 13 samples available. Simulation results are shown to validate the presented modeling.

DYNAMICAL RECURRENT NEURO-FUZZY IDENTIFICATION SCHEMES EMPLOYING SWITCHING PARAMETER HOPPING

In this paper we analyze the identification problem which consists of choosing an appropriate identification model and adjusting its parameters according to some adaptive law, such that the response of the model to an input signal (or a class of input signals), approximates the response of the real system for the same input. For identification models we use fuzzy-recurrent high order neural networks. High order networks are expansions of the first-order Hopfield and Cohen-Grossberg models that allow higher order interactions between neurons. The underlying fuzzy model is of Mamdani type assuming a standard defuzzification procedure such as the weighted average. Learning laws are proposed which ensure that the identification error converges to zero exponentially fast or to a residual set when a modeling error is applied. There are two core ideas in the proposed method: (1) Several high order neural networks are specialized to work around fuzzy centers, separating in this way the system into neuro-fuzzy subsystems, and (2) the use of a novel method called switching parameter hopping against the commonly used projection in order to restrict the weights and avoid drifting to infinity.

Multi Groups Cooperation based Symbiotic Evolution for TSK-type Neuro-Fuzzy Systems Design

In this paper, a TSK-type neuro-fuzzy system with multi groups cooperation based symbiotic evolution method (TNFS-MGCSE) is proposed. The TNFS-MGCSE is developed from symbiotic evolution. The symbiotic evolution is different from traditional GAs (genetic algorithms) that each chromosome in symbiotic evolution represents a rule of fuzzy model. The MGCSE is different from the traditional symbiotic evolution; with a population in MGCSE is divided to several groups. Each group formed by a set of chromosomes represents a fuzzy rule and cooperate with other groups to generate the better chromosomes by using the proposed cooperation based crossover strategy (CCS). In this paper, the proposed TNFS-MGCSE is used to evaluate by numerical examples (Mackey-Glass chaotic time series and sunspot number forecasting). The performance of the TNFS-MGCSE achieves excellently with other existing models in the simulations.

Black-box identification of a class of nonlinear systems by a recurrent neurofuzzy network

This brief presents a structure for black-box identification based on continuous-time recurrent neurofuzzy networks for a class of dynamic nonlinear systems. The proposed network catches the dynamics of a system by generating its own states, using only input and output measurements of the system. The training algorithm is based on adaptive observer theory, the stability of the network, the convergence of the training algorithm, and the ultimate bound on the identification error as well as the parameter error are established. Experimental results are included to illustrate the effectiveness of the proposed method.

Gene regulatory networks modelling using a dynamic evolutionary hybrid

Background

Inference of gene regulatory networks is a key goal in the quest for understanding fundamental cellular processes and revealing underlying relations among genes. With the availability of gene expression data, computational methods aiming at regulatory networks reconstruction are facing challenges posed by the data's high dimensionality, temporal dynamics or measurement noise. We propose an approach based on a novel multi-layer evolutionary trained neuro-fuzzy recurrent network (ENFRN) that is able to select potential regulators of target genes and describe their regulation type.

Results

The recurrent, self-organizing structure and evolutionary training of our network yield an optimized pool of regulatory relations, while its fuzzy nature avoids noise-related problems. Furthermore, we are able to assign scores for each regulation, highlighting the confidence in the retrieved relations. The approach was tested by applying it to several benchmark datasets of yeast, managing to acquire biologically validated relations among genes.

Conclusions

The results demonstrate the effectiveness of the ENFRN in retrieving biologically valid regulatory relations and providing meaningful insights for better understanding the dynamics of gene regulatory networks.

The algorithms and methods described in this paper have been implemented in a Matlab toolbox and are available from: http://bioserver-1.bioacademy.gr/DataRepository/Project_ENFRN_GRN/.

ANNEALED CHAOTIC LEARNING FOR TIME SERIES PREDICTION IN IMPROVED NEURO-FUZZY NETWORK WITH FEEDBACKS

A new version of neuro-fuzzy system of feedbacks with chaotic dynamics is proposed in this work. Unlike the conventional neuro-fuzzy, improved neuro-fuzzy system with feedbacks is better able to handle temporal data series. By introducing chaotic dynamics into the feedback neuro-fuzzy system, the system has richer and more flexible dynamics to search for near-optimal solutions. In the experimental results, performance and effectiveness of the presented approach are evaluated by using benchmark data series. Comparison with other existing methods shows the proposed method for the neuro-fuzzy feedback is able to predict the time series accurately.

Hierarchical singleton-type recurrent neural fuzzy networks for noisy speech recognition

This paper proposes noisy speech recognition using hierarchical singleton-type recurrent neural fuzzy networks (HSRNFNs). The proposed HSRNFN is a hierarchical connection of two singleton-type recurrent neural fuzzy networks (SRNFNs), where one is used for noise filtering and the other for recognition. The SRNFN is constructed by recurrent fuzzy if-then rules with fuzzy singletons in the consequences, and their recurrent properties make them suitable for processing speech patterns with temporal characteristics. In n words recognition, n SRNFNs are created for modeling n words, where each SRNFN receives the current frame feature and predicts the next one of its modeling word. The prediction error of each SRNFN is used as recognition criterion. In filtering, one SRNFN is created, and each SRNFN recognizer is connected to the same SRNFN filter, which filters noisy speech patterns in the feature domain before feeding them to the SRNFN recognizer. Experiments with Mandarin word recognition under different types of noise are performed. Other recognizers, including multilayer perceptron (MLP), time-delay neural networks (TDNNs), and hidden Markov models (HMMs), are also tested and compared. These experiments and comparisons demonstrate good results with HSRNFN for noisy speech recognition tasks.

Multifeedback-layer neural network

The architecture and training procedure of a novel recurrent neural network (RNN), referred to as the multifeedback-layer neural network (MFLNN), is described in this paper. The main difference of the proposed network compared to the available RNNs is that the temporal relations are provided by means of neurons arranged in three feedback layers, not by simple feedback elements, in order to enrich the representation capabilities of the recurrent networks. The feedback layers provide local and global recurrences via nonlinear processing elements. In these feedback layers, weighted sums of the delayed outputs of the hidden and of the output layers are passed through certain activation functions and applied to the feedforward neurons via adjustable weights. Both online and offline training procedures based on the backpropagation through time (BPTT) algorithm are developed. The adjoint model of the MFLNN is built to compute the derivatives with respect to the MFLNN weights which are then used in the training procedures. The Levenberg-Marquardt (LM) method with a trust region approach is used to update the MFLNN weights. The performance of the MFLNN is demonstrated by applying to several illustrative temporal problems including chaotic time series prediction and nonlinear dynamic system identification, and it performed better than several networks available in the literature.