Sequential projection-based metacognitive learning in a radial basis function network for classification problems

Self-Organizing RBF Neural Network Using an Adaptive Gradient Multiobjective Particle Swarm Optimization

One of the major obstacles in using radial basis function (RBF) neural networks is the convergence toward local minima instead of the global minima. For this reason, an adaptive gradient multiobjective particle swarm optimization (AGMOPSO) algorithm is designed to optimize both the structure and parameters of RBF neural networks in this paper. First, the AGMOPSO algorithm, based on a multiobjective gradient method and a self-adaptive flight parameters mechanism, is developed to improve the computation performance. Second, the AGMOPSO-based self-organizing RBF neural network (AGMOPSO-SORBF) can optimize the parameters (centers, widths, and weights), as well as determine the network size. The goal of AGMOPSO-SORBF is to find a tradeoff between the accuracy and the complexity of RBF neural networks. Third, the convergence analysis of AGMOPSO-SORBF is detailed to ensure the prerequisite of any successful applications. Finally, the merits of our proposed approach are verified on multiple numerical examples. The results indicate that the proposed AGMOPSO-SORBF achieves much better generalization capability and compact network structure than some other existing methods.

Radial Basis Functions With Adaptive Input and Composite Trend Representation for Portfolio Selection

We propose a set of novel radial basis functions with adaptive input and composite trend representation (AICTR) for portfolio selection (PS). Trend representation of asset price is one of the main information to be exploited in PS. However, most state-of-the-art trend representation-based systems exploit only one kind of trend information and lack effective mechanisms to construct a composite trend representation. The proposed system exploits a set of RBFs with multiple trend representations, which improves the effectiveness and robustness in price prediction. Moreover, the input of the RBFs automatically switches to the best trend representation according to the recent investing performance of different price predictions. We also propose a novel objective to combine these RBFs and select the portfolio. Extensive experiments on six benchmark data sets (including a new challenging data set that we propose) from different real-world stock markets indicate that the proposed RBFs effectively combine different trend representations and AICTR achieves state-of-the-art investing performance and risk control. Besides, AICTR withstands the reasonable transaction costs and runs fast; hence, it is applicable to real-world financial environments.

Sequential Independent Component Analysis Density Estimation

A problem of multivariate probability density function estimation by exploiting linear independent components analysis (ICA) is addressed. Historically, ICA density estimation was initially proposed under the name projection pursuit density estimation (PPDE) and two basic methods, named forward and backward, were published. We derive a modification of the forward PPDE method, which avoids a computationally demanding optimization involving Monte Carlo sampling of the original method. The results of the experiments show that the proposed method presents an attractive choice for density estimation, which is pronounced for a small number of training observations. Under such conditions, our method usually outperforms model-based Gaussian mixture model. We also found that our method obtained better results than the backward PPDE methods in the situation of nonfactorizable underlying density functions. The proposed method has demonstrated a competitive performance compared with the support vector machine and the extreme learning machine in some real classification tasks.

Solving Classification Problems Using Projection-Based Learning Algorithm with Fuzzy Radial Basis Function Neural Network

Radial basis function (RBF) is combined with fuzzy C-means algorithms and its learning process made by projection-based learning (PBL) has been proposed in this paper, which is pointed out as PBL-fuzzy radial basis function (PBL-FRBF). The proposed method PBL-FRBF is producing good performances by selecting appropriate center and its width in order to achieve it by unsupervised classification algorithms instead of random selection. The PBL decreases the learning time, finds optimum output weight by its energy function and prefers smallest amount of samples for testing. Performance analysis is evaluated by benchmark datasets for classification problem taken from the UCI machine learning repository. The performance of the proposed PBL-FRBF has produced superior results when compared with FRBF and RBF for classification problems.

Haze Removal Using Radial Basis Function Networks for Visibility Restoration Applications

Restoration of visibility in hazy images is the first relevant step of information analysis in many outdoor computer vision applications. To this aim, the restored image must feature clear visibility with sufficient brightness and visible edges, while avoiding the production of noticeable artifacts. In this paper, we propose a haze removal approach based on the radial basis function (RBF) through artificial neural networks dedicated to effectively removing haze formation while retaining not only the visible edges but also the brightness of restored images. Unlike traditional haze-removal methods that consist of single atmospheric veils, the multiatmospheric veil is generated and then dynamically learned by the neurons of the proposed RBF networks according to the scene complexity. Through this process, more visible edges are retained in the restored images. Subsequently, the activation function during the testing process is employed to represent the brightness of the restored image. We compare the proposed method with the other state-of-the-art haze-removal methods and report experimental results in terms of qualitative and quantitative evaluations for benchmark color images captured in typical hazy weather conditions. The experimental results demonstrate that the proposed method is able to produce brighter and more vivid haze-free images with more visible edges than can the other state-of-the-art methods.

Spiking Neural Classifier with Lumped Dendritic Nonlinearity and Binary Synapses: A Current Mode VLSI Implementation and Analysis

We present a neuromorphic current mode implementation of a spiking neural classifier with lumped square law dendritic nonlinearity. It has been shown previously in software simulations that such a system with binary synapses can be trained with structural plasticity algorithms to achieve comparable classification accuracy with fewer synaptic resources than conventional algorithms. We show that even in real analog systems with manufacturing imperfections (CV of 23.5% and 14.4% for dendritic branch gains and leaks respectively), this network is able to produce comparable results with fewer synaptic resources. The chip fabricated in [Formula: see text]m complementary metal oxide semiconductor has eight dendrites per cell and uses two opposing cells per class to cancel common-mode inputs. The chip can operate down to a [Formula: see text] V and dissipates 19 nW of static power per neuronal cell and [Formula: see text] 125 pJ/spike. For two-class classification problems of high-dimensional rate encoded binary patterns, the hardware achieves comparable performance as software implementation of the same with only about a 0.5% reduction in accuracy. On two UCI data sets, the IC integrated circuit has classification accuracy comparable to standard machine learners like support vector machines and extreme learning machines while using two to five times binary synapses. We also show that the system can operate on mean rate encoded spike patterns, as well as short bursts of spikes. To the best of our knowledge, this is the first attempt in hardware to perform classification exploiting dendritic properties and binary synapses.

An Adaptive-PSO-Based Self-Organizing RBF Neural Network

In this paper, a self-organizing radial basis function (SORBF) neural network is designed to improve both accuracy and parsimony with the aid of adaptive particle swarm optimization (APSO). In the proposed APSO algorithm, to avoid being trapped into local optimal values, a nonlinear regressive function is developed to adjust the inertia weight. Furthermore, the APSO algorithm can optimize both the network size and the parameters of an RBF neural network simultaneously. As a result, the proposed APSO-SORBF neural network can effectively generate a network model with a compact structure and high accuracy. Moreover, the analysis of convergence is given to guarantee the successful application of the APSO-SORBF neural network. Finally, multiple numerical examples are presented to illustrate the effectiveness of the proposed APSO-SORBF neural network. The results demonstrate that the proposed method is more competitive in solving nonlinear problems than some other existing SORBF neural networks.

Improving Neural-Network Classifiers Using Nearest Neighbor Partitioning

This paper presents a nearest neighbor partitioning method designed to improve the performance of a neural-network classifier. For neural-network classifiers, usually the number, positions, and labels of centroids are fixed in partition space before training. However, that approach limits the search for potential neural networks during optimization; the quality of a neural network classifier is based on how clear the decision boundaries are between classes. Although attempts have been made to generate floating centroids automatically, these methods still tend to generate sphere-like partitions and cannot produce flexible decision boundaries. We propose the use of nearest neighbor classification in conjunction with a neural-network classifier. Instead of being bound by sphere-like boundaries (such as the case with centroid-based methods), the flexibility of nearest neighbors increases the chance of finding potential neural networks that have arbitrarily shaped boundaries in partition space. Experimental results demonstrate that the proposed method exhibits superior performance on accuracy and average f-measure.

Training Radial Basis Function Neural Networks for Classification via Class-Specific Clustering

In training radial basis function neural networks (RBFNNs), the locations of Gaussian neurons are commonly determined by clustering. Training inputs can be clustered on a fully unsupervised manner (input clustering), or some supervision can be introduced, for example, by concatenating the input vectors with weighted output vectors (input-output clustering). In this paper, we propose to apply clustering separately for each class (class-specific clustering). The idea has been used in some previous works, but without evaluating the benefits of the approach. We compare the class-specific, input, and input-output clustering approaches in terms of classification performance and computational efficiency when training RBFNNs. To accomplish this objective, we apply three different clustering algorithms and conduct experiments on 25 benchmark data sets. We show that the class-specific approach significantly reduces the overall complexity of the clustering, and our experimental results demonstrate that it can also lead to a significant gain in the classification performance, especially for the networks with a relatively few Gaussian neurons. Among other applied clustering algorithms, we combine, for the first time, a dynamic evolutionary optimization method, multidimensional particle swarm optimization, and the class-specific clustering to optimize the number of cluster centroids and their locations.

Learning Subspace-Based RBFNN Using Coevolutionary Algorithm for Complex Classification Tasks

Many real-world classification problems are characterized by samples of a complex distribution in the input space. The classification accuracy is determined by intrinsic properties of all samples in subspaces of features. This paper proposes a novel algorithm for the construction of radial basis function neural network (RBFNN) classifier based on subspace learning. In this paper, feature subspaces are obtained for every hidden node of the RBFNN during the learning process. The connection weights between the input layer and the hidden layer are adjusted to produce various subspaces with dominative features for different hidden nodes. The network structure and dominative features are encoded in two subpopulations that are cooperatively coevolved using the coevolutionary algorithm to achieve a better global optimality for the estimated RBFNN. Experimental results illustrate that the proposed algorithm is able to obtain RBFNN models with both better classification accuracy and simpler network structure when compared with other learning algorithms. Thus, the proposed model provides a more flexible and efficient approach to complex classification tasks by employing the local characteristics of samples in subspaces.

An Empirical Analysis of Evolved Radial Basis Function Networks and Support Vector Machines with Mixture of Kernels

Classification is one of the most fundamental and formidable tasks in many domains including biomedical. In biomedical domain, the distributions of data in most of the datasets into predefined number of classes is significantly different (i.e., the classes are distributed unevenly). Many mathematical, statistical, and machine learning approaches have been developed for classification of biomedical datasets with a varying degree of success. This paper attempts to analyze the empirical performance of two forefront machine learning algorithms particularly designed for classification problem by adding some novelty to address the problem of imbalanced dataset. The evolved radial basis function network with novel kernel and support vector machine with mixture of kernels are suitably designed for the purpose of classification of imbalanced dataset. The experimental outcome shows that both algorithms are promising compared to simple radial basis function neural networks and support vector machine, respectively. However, on an average, support vector machine with mixture kernels is better than evolved radial basis function neural networks.

Hardware-amenable structural learning for spike-based pattern classification using a simple model of active dendrites

This letter presents a spike-based model that employs neurons with functionally distinct dendritic compartments for classifying high-dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron the capacity to perform a large number of input-output mappings. The model uses sparse synaptic connectivity, where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin-enhancing learning algorithm inspired by the mechanism of structural plasticity in biological neurons. The learning algorithm groups correlated synaptic inputs on the same dendritic branch. Since the learning results in modified connection patterns, it can be incorporated into current event-based neuromorphic systems with little overhead. This work also presents a branch-specific spike-based version of this structural plasticity rule. The proposed model is evaluated on benchmark binary classification problems, and its performance is compared against that achieved using support vector machine and extreme learning machine techniques. Our proposed method attains comparable performance while using 10% to 50% less in computational resource than the other reported techniques.

A Novel Latin hypercube algorithm via translational propagation

Metamodels have been widely used in engineering design to facilitate analysis and optimization of complex systems that involve computationally expensive simulation programs. The accuracy of metamodels is directly related to the experimental designs used. Optimal Latin hypercube designs are frequently used and have been shown to have good space-filling and projective properties. However, the high cost in constructing them limits their use. In this paper, a methodology for creating novel Latin hypercube designs via translational propagation and successive local enumeration algorithm (TPSLE) is developed without using formal optimization. TPSLE algorithm is based on the inspiration that a near optimal Latin Hypercube design can be constructed by a simple initial block with a few points generated by algorithm SLE as a building block. In fact, TPSLE algorithm offers a balanced trade-off between the efficiency and sampling performance. The proposed algorithm is compared to two existing algorithms and is found to be much more efficient in terms of the computation time and has acceptable space-filling and projective properties.

A sequential optimization sampling method for metamodels with radial basis functions

Metamodels have been widely used in engineering design to facilitate analysis and optimization of complex systems that involve computationally expensive simulation programs. The accuracy of metamodels is strongly affected by the sampling methods. In this paper, a new sequential optimization sampling method is proposed. Based on the new sampling method, metamodels can be constructed repeatedly through the addition of sampling points, namely, extrema points of metamodels and minimum points of density function. Afterwards, the more accurate metamodels would be constructed by the procedure above. The validity and effectiveness of proposed sampling method are examined by studying typical numerical examples.

An adaptive learning rate for RBFNN using time-domain feedback analysis

Radial basis function neural networks are used in a variety of applications such as pattern recognition, nonlinear identification, control and time series prediction. In this paper, the learning algorithm of radial basis function neural networks is analyzed in a feedback structure. The robustness of the learning algorithm is discussed in the presence of uncertainties that might be due to noisy perturbations at the input or to modeling mismatch. An intelligent adaptation rule is developed for the learning rate of RBFNN which gives faster convergence via an estimate of error energy while giving guarantee to the l₂ stability governed by the upper bounding via small gain theorem. Simulation results are presented to support our theoretical development.