A sequential learning algorithm for self-adaptive resource allocation network classifier

SEFRON: A New Spiking Neuron Model With Time-Varying Synaptic Efficacy Function for Pattern Classification

This paper presents a new time-varying long-term Synaptic Efficacy Function-based leaky-integrate-and-fire neuRON model, referred to as SEFRON and its supervised learning rule for pattern classification problems. The time-varying synaptic efficacy function is represented by a sum of amplitude modulated Gaussian distribution functions located at different times. For a given pattern, the SEFRON's learning rule determines the changes in the amplitudes of weights at selected presynaptic spike times by minimizing a new error function reflecting the differences between the desired and actual postsynaptic firing times. Similar to the gamma-aminobutyric acid-switch phenomenon observed in a biological neuron that switches between excitatory and inhibitory postsynaptic potentials based on the physiological needs, the time-varying synapse model proposed in this paper allows the synaptic efficacy (weight) to switch signs in a continuous manner. The computational power and the functioning of SEFRON are first illustrated using a binary pattern classification problem. The detailed performance comparisons of a single SEFRON classifier with other spiking neural networks (SNNs) are also presented using four benchmark data sets from the UCI machine learning repository. The results clearly indicate that a single SEFRON provides a similar generalization performance compared to other SNNs with multiple layers and multiple neurons.

Multilayer Extreme Learning Machine With Subnetwork Nodes for Representation Learning

The extreme learning machine (ELM), which was originally proposed for "generalized" single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of clustering, regression, and classification. It presents competitive accuracy with superb efficiency in many applications. However, ELM with subnetwork nodes architecture has not attracted much research attentions. Recently, many methods have been proposed for supervised/unsupervised dimension reduction or representation learning, but these methods normally only work for one type of problem. This paper studies the general architecture of multilayer ELM (ML-ELM) with subnetwork nodes, showing that: 1) the proposed method provides a representation learning platform with unsupervised/supervised and compressed/sparse representation learning and 2) experimental results on ten image datasets and 16 classification datasets show that, compared to other conventional feature learning methods, the proposed ML-ELM with subnetwork nodes performs competitively or much better than other feature learning methods.

A Structure-Adaptive Hybrid RBF-BP Classifier with an Optimized Learning Strategy

This paper presents a structure-adaptive hybrid RBF-BP (SAHRBF-BP) classifier with an optimized learning strategy. SAHRBF-BP is composed of a structure-adaptive RBF network and a BP network of cascade, where the number of RBF hidden nodes is adjusted adaptively according to the distribution of sample space, the adaptive RBF network is used for nonlinear kernel mapping and the BP network is used for nonlinear classification. The optimized learning strategy is as follows: firstly, a potential function is introduced into training sample space to adaptively determine the number of initial RBF hidden nodes and node parameters, and a form of heterogeneous samples repulsive force is designed to further optimize each generated RBF hidden node parameters, the optimized structure-adaptive RBF network is used for adaptively nonlinear mapping the sample space; then, according to the number of adaptively generated RBF hidden nodes, the number of subsequent BP input nodes can be determined, and the overall SAHRBF-BP classifier is built up; finally, different training sample sets are used to train the BP network parameters in SAHRBF-BP. Compared with other algorithms applied to different data sets, experiments show the superiority of SAHRBF-BP. Especially on most low dimensional and large number of data sets, the classification performance of SAHRBF-BP outperforms other training SLFNs algorithms.

Adaptive Modulation for DFIG and STATCOM With High-Voltage Direct Current Transmission

This paper develops an adaptive modulation approach for power system control based on the approximate/adaptive dynamic programming method, namely, the goal representation heuristic dynamic programming (GrHDP). In particular, we focus on the fault recovery problem of a doubly fed induction generator (DFIG)-based wind farm and a static synchronous compensator (STATCOM) with high-voltage direct current (HVDC) transmission. In this design, the online GrHDP-based controller provides three adaptive supplementary control signals to the DFIG controller, STATCOM controller, and HVDC rectifier controller, respectively. The mechanism is to observe the system states and their derivatives and then provides supplementary control to the plant according to the utility function. With the GrHDP design, the controller can adaptively develop an internal goal representation signal according to the observed power system states, therefore, to achieve more effective learning and modulating. Our control approach is validated on a wind power integrated benchmark system with two areas connected by HVDC transmission lines. Compared with the classical direct HDP and proportional integral control, our GrHDP approach demonstrates the improved transient stability under system faults. Moreover, experiments under different system operating conditions with signal transmission delays are also carried out to further verify the effectiveness and robustness of the proposed approach.

Data Partition Learning With Multiple Extreme Learning Machines

As demonstrated earlier, the learning accuracy of the single-layer-feedforward-network (SLFN) is generally far lower than expected, which has been a major bottleneck for many applications. In fact, for some large real problems, it is accepted that after tremendous learning time (within finite epochs), the network output error of SLFN will stop or reduce increasingly slowly. This report offers an extreme learning machine (ELM)-based learning method, referred to as the parent-offspring progressive learning method. The proposed method works by separating the data points into various parts, and then multiple ELMs learn and identify the clustered parts separately. The key advantages of the proposed algorithms as compared to the traditional supervised methods are twofold. First, it extends the ELM learning method from a single neural network to a multinetwork learning system, as the proposed multiELM method can approximate any target continuous function and classify disjointed regions. Second, the proposed method tends to deliver a similar or much better generalization performance than other learning methods. All the methods proposed in this paper are tested on both artificial and real datasets.

A SEQUENTIAL LEARNING METHOD FOR DETECTION AND CLASSIFICATION OF EXUDATES IN RETINAL IMAGES TO ASSESS DIABETIC RETINOPATHY

One of the greatest concerns to the personnel in the current health care sector is the severe progression of diabetes. People can often have diabetes and be completely unaware as the symptoms seem harmless when they are seen on their own. Diabetic retinopathy (DR) is an eye disease that is associated with long-standing diabetes. Retinopathy can occur with all types of diabetes and can lead to blindness if left untreated. The conventional method followed by ophthalmologists is the regular testing of the retina. As this method takes time and energy of the ophthalmologists, a new feature-based automated technique for classification and detection of exudates in color fundus image is proposed in this paper. This method reduces the work of the professionals while examining every fundus image rather than only on abnormal image. The exudates are detected from the color fundus image by applying a few pre-processing techniques that remove the optic disk and similar blood vessels using morphological operations. The pre-processed image was then applied for feature extraction and these features were utilized for classification purpose. In this paper, a novel classification technique such as self-adaptive resource allocation network (SRAN) and meta-cognitive neural network (McNN) classifier is employed for classification of images as exudates, their severity and nonexudates. SRAN classifier makes use of self-adaptive thresholds to choose the appropriate training samples and removes the redundant samples to prevent over-training. These selected samples are availed to improve the classification performance. McNN classifier employs human-like meta-cognition to regulate the sequential learning process. The meta-cognitive component controls the learning process in the cognitive component by deciding what-to-learn, when-to-learn and how-to-learn. It is therefore evident that the implementation of human meta-cognitive learning principle improves efficient learning.

Projection-based fast learning fully complex-valued relaxation neural network

This paper presents a fully complex-valued relaxation network (FCRN) with its projection-based learning algorithm. The FCRN is a single hidden layer network with a Gaussian-like sech activation function in the hidden layer and an exponential activation function in the output layer. For a given number of hidden neurons, the input weights are assigned randomly and the output weights are estimated by minimizing a nonlinear logarithmic function (called as an energy function) which explicitly contains both the magnitude and phase errors. A projection-based learning algorithm determines the optimal output weights corresponding to the minima of the energy function by converting the nonlinear programming problem into that of solving a set of simultaneous linear algebraic equations. The resultant FCRN approximates the desired output more accurately with a lower computational effort. The classification ability of FCRN is evaluated using a set of real-valued benchmark classification problems from the University of California, Irvine machine learning repository. Here, a circular transformation is used to transform the real-valued input features to the complex domain. Next, the FCRN is used to solve three practical problems: a quadrature amplitude modulation channel equalization, an adaptive beamforming, and a mammogram classification. Performance results from this paper clearly indicate the superior classification/approximation performance of the FCRN.

Sequential projection-based metacognitive learning in a radial basis function network for classification problems

In this paper, we present a sequential projection-based metacognitive learning algorithm in a radial basis function network (PBL-McRBFN) for classification problems. The algorithm is inspired by human metacognitive learning principles and has two components: a cognitive component and a metacognitive component. The cognitive component is a single-hidden-layer radial basis function network with evolving architecture. The metacognitive component controls the learning process in the cognitive component by choosing the best learning strategy for the current sample and adapts the learning strategies by implementing self-regulation. In addition, sample overlapping conditions and past knowledge of the samples in the form of pseudosamples are used for proper initialization of new hidden neurons to minimize the misclassification. The parameter update strategy uses projection-based direct minimization of hinge loss error. The interaction of the cognitive component and the metacognitive component addresses the what-to-learn, when-to-learn, and how-to-learn human learning principles efficiently. The performance of the PBL-McRBFN is evaluated using a set of benchmark classification problems from the University of California Irvine machine learning repository. The statistical performance evaluation on these problems proves the superior performance of the PBL-McRBFN classifier over results reported in the literature. Also, we evaluate the performance of the proposed algorithm on a practical Alzheimer's disease detection problem. The performance results on open access series of imaging studies and Alzheimer's disease neuroimaging initiative datasets, which are obtained from different demographic regions, clearly show that PBL-McRBFN can handle a problem with change in distribution.

HUMAN ACTION RECOGNITION USING META-COGNITIVE NEURO-FUZZY INFERENCE SYSTEM

We propose a sequential Meta-Cognitive learning algorithm for Neuro-Fuzzy Inference System (McFIS) to efficiently recognize human actions from video sequence. Optical flow information between two consecutive image planes can represent actions hierarchically from local pixel level to global object level, and hence are used to describe the human action in McFIS classifier. McFIS classifier and its sequential learning algorithm is developed based on the principles of self-regulation observed in human meta-cognition. McFIS decides on what-to-learn, when-to-learn and how-to-learn based on the knowledge stored in the classifier and the information contained in the new training samples. The sequential learning algorithm of McFIS is controlled and monitored by the meta-cognitive components which uses class-specific, knowledge based criteria along with self-regulatory thresholds to decide on one of the following strategies: (i) Sample deletion (ii) Sample learning and (iii) Sample reserve. Performance of proposed McFIS based human action recognition system is evaluated using benchmark Weizmann and KTH video sequences. The simulation results are compared with well known SVM classifier and also with state-of-the-art action recognition results reported in the literature. The results clearly indicates McFIS action recognition system achieves better performances with minimal computational effort.

Metacognitive Learning in a Fully Complex-Valued Radial Basis Function Neural Network

Recent studies on human learning reveal that self-regulated learning in a metacognitive framework is the best strategy for efficient learning. As the machine learning algorithms are inspired by the principles of human learning, one needs to incorporate the concept of metacognition to develop efficient machine learning algorithms. In this letter we present a metacognitive learning framework that controls the learning process of a fully complex-valued radial basis function network and is referred to as a metacognitive fully complex-valued radial basis function (Mc-FCRBF) network. Mc-FCRBF has two components: a cognitive component containing the FC-RBF network and a metacognitive component, which regulates the learning process of FC-RBF. In every epoch, when a sample is presented to Mc-FCRBF, the metacognitive component decides what to learn, when to learn, and how to learn based on the knowledge acquired by the FC-RBF network and the new information contained in the sample. The Mc-FCRBF learning algorithm is described in detail, and both its approximation and classification abilities are evaluated using a set of benchmark and practical problems. Performance results indicate the superior approximation and classification performance of Mc-FCRBF compared to existing methods in the literature.

Identification of brain regions responsible for Alzheimer's disease using a Self-adaptive Resource Allocation Network

In this paper, we present a novel approach for the identification of brain regions responsible for Alzheimer's disease using the Magnetic Resonance (MR) images. The approach incorporates the recently developed Self-adaptive Resource Allocation Network (SRAN) for Alzheimer's disease classification using voxel-based morphometric features of MR images. SRAN classifier uses a sequential learning algorithm, employing self-adaptive thresholds to select the appropriate training samples and discard redundant samples to prevent over-training. These selected training samples are then used to evolve the network architecture efficiently. Since, the number of features extracted from the MR images is large, a feature selection scheme (to reduce the number of features needed) using an Integer-Coded Genetic Algorithm (ICGA) in conjunction with the SRAN classifier (referred to here as the ICGA-SRAN classifier) have been developed. In this study, different healthy/Alzheimer's disease patient's MR images from the Open Access Series of Imaging Studies data set have been used for the performance evaluation of the proposed ICGA-SRAN classifier. We have also compared the results of the ICGA-SRAN classifier with the well-known Support Vector Machine (SVM) and Extreme Learning Machine (ELM) classifiers. The study results clearly show that the ICGA-SRAN classifier produces a better generalization performance with a smaller number of features, lower misclassification rate and a compact network. The ICGA-SRAN selected features clearly indicate that the variations in the gray matter volume in the parahippocampal gyrus and amygdala brain regions may be good indicators of the onset of Alzheimer's disease in normal persons.