A recurrent self-organizing neural fuzzy inference network

EEG-Based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and Their Applications

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment. Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications. EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas. In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years. Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals. Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications. Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.

EMG-Based 3D Hand Motor Intention Prediction for Information Transfer from Human to Robot

(1) Background: Three-dimensional (3-D) hand position is one of the kinematic parameters that can be inferred from Electromyography (EMG) signals. The inferred parameter is used as a communication channel in human–robot collaboration applications. Although its application from the perspective of rehabilitation and assistive technologies are widely studied, there are few papers on its application involving healthy subjects such as intelligent manufacturing and skill transfer. In this regard, for tasks associated with complex hand trajectories without the consideration of the degree of freedom (DOF), the prediction of 3-D hand position from EMG signal alone has not been addressed. (2) Objective: The primary aim of this study is to propose a model to predict human motor intention that can be used as information from human to robot. Therefore, the prediction of a 3-D hand position directly from the EMG signal for complex trajectories of hand movement, without the direct consideration of joint movements, is studied. In addition, the effects of slow and fast motions on the accuracy of the prediction model are analyzed. (3) Methods: This study used the EMG signal that is collected from the upper limb of healthy subjects, and the position signal of the hand while the subjects manipulate complex trajectories. We considered and analyzed two types of tasks with complex trajectories, each with quick and slow motions. A recurrent fuzzy neural network (RFNN) model was constructed to predict the 3-D position of the hand from the features of EMG signals alone. We used the Pearson correlation coefficient (CC) and normalized root mean square error (NRMSE) as performance metrics. (4) Results: We found that 3-D hand positions of the complex movement can be predicted with the mean performance of CC = 0.85 and NRMSE = 0.105. The 3-D hand position can be predicted well within a future time of 250 ms, from the EMG signal alone. Even though tasks performed under quick motion had a better prediction performance; the statistical difference in the accuracy of prediction between quick and slow motion was insignificant. Concerning the prediction model, we found that RFNN has a good performance in decoding for the time-varying system. (5) Conclusions: In this paper, irrespective of the speed of the motion, the 3-D hand position is predicted from the EMG signal alone. The proposed approach can be used in human–robot collaboration applications to enhance the natural interaction between a human and a robot.

Development and Analysis of Neural Networks Realized in the Presence of Granular Data

In this article, we propose a design and evaluation framework of granular neural networks realized in the presence of information granules. Neural networks realized in this manner are able to process both nonnumerical data, such as information granules as well as numerical data. Information granules are meaningful and semantically sound entities formed by organizing existing knowledge and available experimental data. The directional nature of mapping between the input and output data needs to be considered when building information granules. The development of neural networks advocated in this article is realized as a two-phase process. First, a collection of information granules is formed through granulation of numeric data in the input and output spaces. Second, neural networks are constructed on the basis of information granules rather than original (numeric) data. The proposed method leads to the construction of neural networks in a completely new way. In comparison with traditional (numeric) neural networks, the networks developed in the presence of granular data require shorter learning time. They also produce the results (outputs) that are information granules rather than numeric entities. The quality of granular outputs generated by our neural networks is evaluated in terms of the coverage and specificity criteria that are pertinent to the characterization of the information granules.

Chaotic Synchronization Using a Self-Evolving Recurrent Interval Type-2 Petri Cerebellar Model Articulation Controller

In this manuscript, the synchronization of four-dimensional (4D) chaotic systems with uncertain parameters using a self-evolving recurrent interval type-2 Petri cerebellar model articulation controller is studied. The design of the synchronization control system is comprised of a recurrent interval type-2 Petri cerebellar model articulation controller and a fuzzy compensation controller. The proposed network structure can automatically generate new rules or delete unnecessary rules based on the self-evolving algorithm. Furthermore, the gradient-descent method is applied to adjust the proposed network parameters. Through Lyapunov stability analysis, bounded system stability is guaranteed. Finally, the effectiveness of the proposed controller is illustrated using numerical simulations of 4D chaotic systems.

Fuzzy jump wavelet neural network based on rule induction for dynamic nonlinear system identification with real data applications

Aim

Fuzzy wavelet neural network (FWNN) has proven to be a promising strategy in the identification of nonlinear systems. The network considers both global and local properties, deals with imprecision present in sensory data, leading to desired precisions. In this paper, we proposed a new FWNN model nominated “Fuzzy Jump Wavelet Neural Network” (FJWNN) for identifying dynamic nonlinear-linear systems, especially in practical applications.

Methods

The proposed FJWNN is a fuzzy neural network model of the Takagi-Sugeno-Kang type whose consequent part of fuzzy rules is a linear combination of input regressors and dominant wavelet neurons as a sub-jump wavelet neural network. Each fuzzy rule can locally model both linear and nonlinear properties of a system. The linear relationship between the inputs and the output is learned by neurons with linear activation functions, whereas the nonlinear relationship is locally modeled by wavelet neurons. Orthogonal least square (OLS) method and genetic algorithm (GA) are respectively used to purify the wavelets for each sub-JWNN. In this paper, fuzzy rule induction improves the structure of the proposed model leading to less fuzzy rules, inputs of each fuzzy rule and model parameters. The real-world gas furnace and the real electromyographic (EMG) signal modeling problem are employed in our study. In the same vein, piecewise single variable function approximation, nonlinear dynamic system modeling, and Mackey–Glass time series prediction, ratify this method superiority. The proposed FJWNN model is compared with the state-of-the-art models based on some performance indices such as RMSE, RRSE, Rel ERR%, and VAF%.

Results

The proposed FJWNN model yielded the following results: RRSE (mean±std) of 10e-5±6e-5 for piecewise single-variable function approximation, RMSE (mean±std) of 2.6–4±2.6e-4 for the first nonlinear dynamic system modelling, RRSE (mean±std) of 1.59e-3±0.42e-3 for Mackey–Glass time series prediction, RMSE of 0.3421 for gas furnace modelling and VAF% (mean±std) of 98.24±0.71 for the EMG modelling of all trial signals, indicating a significant enhancement over previous methods.

Conclusions

The FJWNN demonstrated promising accuracy and generalization while moderating network complexity. This improvement is due to applying main useful wavelets in combination with linear regressors and using fuzzy rule induction. Compared to the state-of-the-art models, the proposed FJWNN yielded better performance and, therefore, can be considered a novel tool for nonlinear system identification.

Temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems

In this paper, a novel temporally local recurrent radial basis function network for modeling and adaptive control of nonlinear systems is proposed. The proposed structure consists of recurrent hidden neurons having weighted self-feedback loops and a weighted linear feed-through from the input layer directly to the output layer neuron(s). The dynamic back-propagation algorithm is developed and used for updating the parameters of the proposed structure. To improve the performance of learning algorithm, discrete Lyapunov stability method is used to develop an adaptive learning rate scheme. This scheme ensures the faster convergence of the parameters and maintains the stability of the system. A total of 5 complex nonlinear systems are used to test and compare the performance of the proposed network with other neural network structures. The disturbance rejection tests are also carried out to check whether the proposed scheme is able to handle the external disturbance/noise signals effects or not. The obtained results show the efficacy of the proposed method.

Identification and control of dynamical systems using different architectures of recurrent fuzzy system

Fuzzy logic based systems are very widely used for modeling and control of complex non-linear, plants. Fuzzy systems require the knowledge about the structure of the dynamic plant in order to achieve fruitful results. Recurrent Fuzzy systems (RFS) are a variation of fuzzy systems and have the ability to model and control dynamic plants without using the information about the structure of the plant. This paper presents identification and control of non-linear dynamical systems using two different architectures of recurrent fuzzy system (RFS). It highlights the importance of RFS over the conventional type-1 fuzzy based system. The objective of system identification as well as control has been achieved using both the architectures of RFS and the simulation results clearly show their efficiency. This paper also highlights yet another advantage of RFS over the conventional type-1 fuzzy systems which comes into light when dealing with higher order systems. The paper explains how the computational complexity can be greatly reduced by using RFS for higher order dynamical systems. A comparative analysis between the conventional type-1 fuzzy system and the two recurrent fuzzy systems has also been performed.

Efficient Approach for RLS Type Learning in TSK Neural Fuzzy Systems

This paper presents an efficient approach for the use of recursive least square (RLS) learning algorithm in Takagi-Sugeno-Kang neural fuzzy systems. In the use of RLS, reduced covariance matrix, of which the off-diagonal blocks defining the correlation between rules are set to zeros, may be employed to reduce computational burden. However, as reported in the literature, the performance of such an approach is slightly worse than that of using the full covariance matrix. In this paper, we proposed a so-called enhanced local learning concept in which a threshold is considered to stop learning for those less fired rules. It can be found from our experiments that the proposed approach can have better performances than that of using the full covariance matrix. Enhanced local learning method can be more active on the structure learning phase. Thus, the method not only can stop the update for insufficiently fired rules to reduce disturbances in self-constructing neural fuzzy inference network but also raises the learning speed on structure learning phase by using a large backpropagation learning constant.

Granular neural networks: concepts and development schemes

In this paper, we introduce a concept of a granular neural network and develop its comprehensive design process. The proposed granular network is formed on the basis of a given (numeric) neural network whose structure is augmented by the formation of granular connections (being realized as intervals) spanned over the numeric ones. Owing to its simplicity of the underlying processing, the interval connections become an appealing alternative of information granules to clarify the main idea. We introduce a concept of information granularity and its quantification (viewed as a level of information granularity). Being treated as an essential design asset, the assumed level of information granularity is distributed (allocated) among the connections of the network in several different ways so that certain performance index becomes maximized. Due to the high dimensionality nature of some protocols of allocation of information granularity and the nature of the allocation process itself, single-objective versions of particle swarm optimization is considered a suitable optimization vehicle. As we are concerned with the granular output of the network, which has to be evaluated with regard to the numeric target of data, two criteria are considered; namely, coverage of numeric data and specificity of information granules (intervals). A series of numeric studies completed for synthetic data and data coming from the machine learning and StatLib repositories provide a useful insight into the effectiveness of the proposed algorithm.

Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network

This paper presents a novel recurrent fuzzy neural network, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for prediction and identification of dynamic systems. The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN is composed of a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNNs learning starts with an empty rule base and all of the rules are generated and learned online through a simultaneous structure and parameter learning. An on-line clustering algorithm is effective in generating fuzzy rules. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN for the prediction and identification of dynamic plants and compare it to other well-known recurrent FNNs. The proposed model obtains enhanced performance results.

Gene networks inference from expression data using a recurrent neuro-fuzzy approach

The reverse engineering paradigm is given increasing attention in computational molecular biology lately. One of the goals is to understand how gene regulatory networks (complex systems of genes, proteins and other molecules) function and interact to carry out specific cell functions. We present an approach for inferring the complex causal relationships among genes from microarray experimental data based on a recurrent neuro-fuzzy method. The method derives information on the gene interactions in a highly interpretable form (fuzzy rules) and takes into account dynamical aspects of genes regulation through its recurrent structure. We tested our approach on a set of genes known to be highly regulated during the yeast cell-cycle. The retrieved gene interactions correspond to the ones validated by previous biological studies, while our method surpasses previous computational techniques that attempted gene networks reconstruction, being able to retrieve significantly more biologically valid relationships among genes. At the same time, our method is able to predict time series for the expression of the genes based on the information extracted from a training subset of the data. The results prove highly accurate prediction capability.

Gene regulatory network from microarray data of colon cancer patients using TSK-type recurrent neural fuzzy network

In this work we applied a TSK-type recurrent neural fuzzy approach to extract regulatory relationship among genes and reconstruct gene regulatory network from microarray data. The identified signature has captured the regulatory relationship among 27 differentially expressed genes from microarray dataset. We applied three different methods viz., feed forward neural fuzzy, modified genetic algorithm and recurrent neural fuzzy, on the same data set for the inference of GRNs and the results obtained are almost comparable. In all tested cases, TRNFN identified more biologically meaningful relations. We found that 87.8% of the total interactions extracted by TRNFN are correct in accordance with the biological knowledge. Our analysis resulted in 2 major outcomes. First, upregulated genes are regulated by more genes than downregulated genes. Second, tumor activators activate other tumor activators and suppress tumor suppressers strongly in the disease environment. These findings will help to elucidate the common molecular mechanism of colon cancer, and provide new insights into cancer diagnostics, prognostics and therapy.

Multi Groups Cooperation based Symbiotic Evolution for TSK-type Neuro-Fuzzy Systems Design

In this paper, a TSK-type neuro-fuzzy system with multi groups cooperation based symbiotic evolution method (TNFS-MGCSE) is proposed. The TNFS-MGCSE is developed from symbiotic evolution. The symbiotic evolution is different from traditional GAs (genetic algorithms) that each chromosome in symbiotic evolution represents a rule of fuzzy model. The MGCSE is different from the traditional symbiotic evolution; with a population in MGCSE is divided to several groups. Each group formed by a set of chromosomes represents a fuzzy rule and cooperate with other groups to generate the better chromosomes by using the proposed cooperation based crossover strategy (CCS). In this paper, the proposed TNFS-MGCSE is used to evaluate by numerical examples (Mackey-Glass chaotic time series and sunspot number forecasting). The performance of the TNFS-MGCSE achieves excellently with other existing models in the simulations.

Gene regulatory networks modelling using a dynamic evolutionary hybrid

Background

Inference of gene regulatory networks is a key goal in the quest for understanding fundamental cellular processes and revealing underlying relations among genes. With the availability of gene expression data, computational methods aiming at regulatory networks reconstruction are facing challenges posed by the data's high dimensionality, temporal dynamics or measurement noise. We propose an approach based on a novel multi-layer evolutionary trained neuro-fuzzy recurrent network (ENFRN) that is able to select potential regulators of target genes and describe their regulation type.

Results

The recurrent, self-organizing structure and evolutionary training of our network yield an optimized pool of regulatory relations, while its fuzzy nature avoids noise-related problems. Furthermore, we are able to assign scores for each regulation, highlighting the confidence in the retrieved relations. The approach was tested by applying it to several benchmark datasets of yeast, managing to acquire biologically validated relations among genes.

Conclusions

The results demonstrate the effectiveness of the ENFRN in retrieving biologically valid regulatory relations and providing meaningful insights for better understanding the dynamics of gene regulatory networks.

The algorithms and methods described in this paper have been implemented in a Matlab toolbox and are available from: http://bioserver-1.bioacademy.gr/DataRepository/Project_ENFRN_GRN/.