New Dynamical Optimal Learning for Linear Multilayer FNN

A new learning algorithm for a fully connected neuro-fuzzy inference system

A traditional neuro-fuzzy system is transformed into an equivalent fully connected three layer neural network (NN), namely, the fully connected neuro-fuzzy inference systems (F-CONFIS). The F-CONFIS differs from traditional NNs by its dependent and repeated weights between input and hidden layers and can be considered as the variation of a kind of multilayer NN. Therefore, an efficient learning algorithm for the F-CONFIS to cope these repeated weights is derived. Furthermore, a dynamic learning rate is proposed for neuro-fuzzy systems via F-CONFIS where both premise (hidden) and consequent portions are considered. Several simulation results indicate that the proposed approach achieves much better accuracy and fast convergence.

Simplified interval type-2 fuzzy neural networks

This paper describes a self-evolving interval type-2 fuzzy neural network (FNN) for various applications. As type-1 fuzzy systems cannot effectively handle uncertainties in information within the knowledge base, we propose a simple interval type-2 FNN, which uses interval type-2 fuzzy sets in the premise and the Takagi-Sugeno-Kang (TSK) type in the consequent of the fuzzy rule. The TSK-type consequent of fuzzy rule is a linear combination of exogenous input variables. Given an initially empty the rule-base, all rules are generated with on-line type-2 fuzzy clustering. Instead of the time-consuming K-M iterative procedure, the design factors ql and qr are learned to adaptively adjust the upper and lower positions on the left and right limit outputs, using the parameter update rule based on a gradient descent algorithm. Simulation results demonstrate that our approach yields fewer test errors and less computational complexity than other type-2 FNNs.

On Adaptive Learning Rate That Guarantees Convergence in Feedforward Networks

This paper investigates new learning algorithms (LF I and LF II) based on Lyapunov function for the training of feedforward neural networks. It is observed that such algorithms have interesting parallel with the popular backpropagation (BP) algorithm where the fixed learning rate is replaced by an adaptive learning rate computed using convergence theorem based on Lyapunov stability theory. LF II, a modified version of LF I, has been introduced with an aim to avoid local minima. This modification also helps in improving the convergence speed in some cases. Conditions for achieving global minimum for these kind of algorithms have been studied in detail. The performances of the proposed algorithms are compared with BP algorithm and extended Kalman filtering (EKF) on three bench-mark function approximation problems: XOR, 3-bit parity, and 8-3 encoder. The comparisons are made in terms of number of learning iterations and computational time required for convergence. It is found that the proposed algorithms (LF I and II) are much faster in convergence than other two algorithms to attain same accuracy. Finally, the comparison is made on a complex two-dimensional (2-D) Gabor function and effect of adaptive learning rate for faster convergence is verified. In a nutshell, the investigations made in this paper help us better understand the learning procedure of feedforward neural networks in terms of adaptive learning rate, convergence speed, and local minima.

Extension neural network-type 2 and its applications

A supervised learning pattern classifier, called the extension neural network (ENN), has been described in a recent paper. In this sequel, the unsupervised learning pattern clustering sibling called the extension neural network type 2 (ENN-2) is proposed. This new neural network uses an extension distance (ED) to measure the similarity between data and the cluster center. It does not require an initial guess of the cluster center coordinates, nor of the initial number of clusters. The clustering process is controlled by a distanced parameter and by a novel extension distance. It shows the same capability as human memory systems to keep stability and plasticity characteristics at the same time, and it can produce meaningful weights after learning. Moreover, the structure of the proposed ENN-2 is simpler and the learning time is shorter than traditional neural networks. Experimental results from five different examples, including three benchmark data sets and two practical applications, verify the effectiveness and applicability of the proposed work.