A Maximally Split and Relaxed ADMM for Regularized Extreme Learning Machines

SSGCNet: A Sparse Spectra Graph Convolutional Network for Epileptic EEG Signal Classification

In this article, we propose a sparse spectra graph convolutional network (SSGCNet) for epileptic electroencephalogram (EEG) signal classification. The goal is to develop a lightweighted deep learning model while retaining a high level of classification accuracy. To do so, we propose a weighted neighborhood field graph (WNFG) to represent EEG signals. The WNFG reduces redundant edges between graph nodes and has lower graph generation time and memory usage than the baseline solution. The sequential graph convolutional network is further developed from a WNFG by combining sparse weight pruning and the alternating direction method of multipliers (ADMM). Compared with the state-of-the-art method, our method has the same classification accuracy on the Bonn public dataset and the spikes and slow waves (SSW) clinical real dataset when the connection rate is ten times smaller.

An Accelerated Maximally Split ADMM for a Class of Generalized Ridge Regression

Ridge regression (RR) has been commonly used in machine learning, but is facing computational challenges in big data applications. To meet the challenges, this article develops a highly parallel new algorithm, i.e., an accelerated maximally split alternating direction method of multipliers (A-MS-ADMM), for a class of generalized RR (GRR) that allows different regularization factors for different regression coefficients. Linear convergence of the new algorithm along with its convergence ratio is established. Optimal parameters of the algorithm for the GRR with a particular set of regularization factors are derived, and a selection scheme of the algorithm parameters for the GRR with general regularization factors is also discussed. The new algorithm is then applied in the training of single-layer feedforward neural networks. Experimental results on performance validation on real-world benchmark datasets for regression and classification and comparisons with existing methods demonstrate the fast convergence, low computational complexity, and high parallelism of the new algorithm.

Tensor based stacked fuzzy neural network for efficient data regression

Random vector functional link and extreme learning machine have been extended by the type-2 fuzzy sets with vector stacked methods, this extension leads to a new way to use tensor to construct learning structure for the type-2 fuzzy sets-based learning framework. In this paper, type-2 fuzzy sets-based random vector functional link, type-2 fuzzy sets-based extreme learning machine and Tikhonov-regularized extreme learning machine are fused into one network, a tensor way of stacking data is used to incorporate the nonlinear mappings when using type-2 fuzzy sets. In this way, the network could learn the sub-structure by three sub-structures' algorithms, which are merged into one tensor structure via the type-2 fuzzy mapping results. To the stacked single fuzzy neural network, the consequent part parameters learning is implemented by unfolding tensor-based matrix regression. The newly proposed stacked single fuzzy neural network shows a new way to design the hybrid fuzzy neural network with the higher order fuzzy sets and higher order data structure. The effective of the proposed stacked single fuzzy neural network are verified by the classical testing benchmarks and several statistical testing methods.

Mixture Correntropy-Based Kernel Extreme Learning Machines

Kernel-based extreme learning machine (KELM), as a natural extension of ELM to kernel learning, has achieved outstanding performance in addressing various regression and classification problems. Compared with the basic ELM, KELM has a better generalization ability owing to no needs of the number of hidden nodes given beforehand and random projection mechanism. Since KELM is derived under the minimum mean square error (MMSE) criterion for the Gaussian assumption of noise, its performance may deteriorate under the non-Gaussian cases, seriously. To improve the robustness of KELM, this article proposes a mixture correntropy-based KELM (MC-KELM), which adopts the recently proposed maximum mixture correntropy criterion as the optimization criterion, instead of using the MMSE criterion. In addition, an online sequential version of MC-KELM (MCOS-KELM) is developed to deal with the case that the data arrive sequentially (one-by-one or chunk-by-chunk). Experimental results on regression and classification data sets are reported to validate the performance superiorities of the new methods.

Hierarchical One-Class Classifier With Within-Class Scatter-Based Autoencoders

Autoencoding is a vital branch of representation learning in deep neural networks (DNNs). The extreme learning machine-based autoencoder (ELM-AE) has been recently developed and has gained popularity for its fast learning speed and ease of implementation. However, the ELM-AE uses random hidden node parameters without tuning, which may generate meaningless encoded features. In this brief, we first propose a within-class scatter information constraint-based AE (WSI-AE) that minimizes both the reconstruction error and the within-class scatter of the encoded features. We then build stacked WSI-AEs into a one-class classification (OCC) algorithm based on the hierarchical regularized least-squared method. The effectiveness of our approach was experimentally demonstrated in comparisons with several state-of-the-art AEs and OCC algorithms. The evaluations were performed on several benchmark data sets.

Regularized correntropy criterion based semi-supervised ELM

Along with the explosive growing of data, semi-supervised learning attracts increasing attention in the past years due to its powerful capability in labeling unlabeled data and knowledge mining. As an emerging method, the semi-supervised extreme learning machine (SSELM), that builds on ELM, has been developed for data classification and shown superiorities in learning efficiency and accuracy. However, the optimization of SSELM as well as most of the other ELMs is generally based on the mean square error (MSE) criterion, which has been shown less effective in dealing with non-Gaussian noises. In this paper, a robust regularized correntropy criterion based SSELM (RC-SSELM) is developed. The optimization of the output weight matrix of RC-SSELM is derived by the fixed-point iteration based approach. A convergent analysis of the proposed RC-SSELM is presented based on the half-quadratic optimization technique. Experimental results on 4 synthetic datasets and 13 benchmark UCI datasets are provided to show the superiorities of the proposed RC-SSELM over SSELM and other state-of-the-art methods.