Laplacian Welsch Regularization for Robust Semisupervised Learning

Regularly Truncated M-Estimators for Learning With Noisy Labels

The sample selection approach is very popular in learning with noisy labels. As deep networks "learn pattern first", prior methods built on sample selection share a similar training procedure: the small-loss examples can be regarded as clean examples and used for helping generalization, while the large-loss examples are treated as mislabeled ones and excluded from network parameter updates. However, such a procedure is arguably debatable from two folds: (a) it does not consider the bad influence of noisy labels in selected small-loss examples; (b) it does not make good use of the discarded large-loss examples, which may be clean or have meaningful information for generalization. In this paper, we propose regularly truncated M-estimators (RTME) to address the above two issues simultaneously. Specifically, RTME can alternately switch modes between truncated M-estimators and original M-estimators. The former can adaptively select small-losses examples without knowing the noise rate and reduce the side-effects of noisy labels in them. The latter makes the possibly clean examples but with large losses involved to help generalization. Theoretically, we demonstrate that our strategies are label-noise-tolerant. Empirically, comprehensive experimental results show that our method can outperform multiple baselines and is robust to broad noise types and levels.

A regularized orthogonal activated inverse-learning neural network for regression and classification with outliers

A novel regularized orthogonal activated inverse-learning (ROAIL) neural network is proposed and investigated for reducing the impact of outliers in regression and classification fields. The proposed ROAIL network does not require extensive iterative computations. Instead, it can achieve the desired results with a single step of computation, allowing for the efficient acquisition of network weights. By extending the Gegenbauer polynomials to a multi-variate version, and integrating the ℓ2 regularization and Welsch loss function into the orthogonal activated inverse-learning framework, two forms of ROAIL are obtained, i.e., ℓ2 norm ROAIL (ℓ2-ROAIL) and Welsch-ROAIL (W-ROAIL). ℓ2-ROAIL neural network is proposed to minimize the empirical and structural risk simultaneously since taking the structural risk as a part of loss function can effectively reduce the complexity of the model and thus improve the generalization ability. W-ROAIL neural network further improves the robustness of the ℓ2-ROAIL neural network by replacing the original two-norm in loss function with Welsch function. The Welsch function can determine the weights of each sample according to its output error, and influence of outliers could be weakened since the weights of outliers would be reduced. Both regression and classification experiments show that W-ROAIL neural network has strong ability to suppress the influence of outliers.

Enhanced Group Sparse Regularized Nonconvex Regression for Face Recognition

Regression analysis based methods have shown strong robustness and achieved great success in face recognition. In these methods, convex l₁-norm and nuclear norm are usually utilized to approximate the l₀-norm and rank function. However, such convex relaxations may introduce a bias and lead to a suboptimal solution. In this paper, we propose a novel Enhanced Group Sparse regularized Nonconvex Regression (EGSNR) method for robust face recognition. An upper bounded nonconvex function is introduced to replace l₁-norm for sparsity, which alleviates the bias problem and adverse effects caused by outliers. To capture the characteristics of complex errors, we propose a mixed model by combining γ-norm and matrix γ-norm induced from the nonconvex function. Furthermore, an l_2,γ-norm based regularizer is designed to directly seek the interclass sparsity or group sparsity instead of traditional l_2,1-norm. The locality of data, i.e., the distance between the query sample and multi-subspaces, is also taken into consideration. This enhanced group sparse regularizer enables EGSNR to learn more discriminative representation coefficients. Comprehensive experiments on several popular face datasets demonstrate that the proposed EGSNR outperforms the state-of-the-art regression based methods for robust face recognition.

Diffusion adaptive filtering algorithm based on the Fair cost function

To better perform distributed estimation, this paper, by combining the Fair cost function and adapt-then-combine scheme at all distributed network nodes, a novel diffusion adaptive estimation algorithm is proposed from an M-estimator perspective, which is called the diffusion Fair (DFair) adaptive filtering algorithm. The stability of the mean estimation error and the computational complexity of the DFair are theoretically analyzed. Compared with the robust diffusion LMS (RDLMS), diffusion Normalized Least Mean M-estimate (DNLMM), diffusion generalized correntropy logarithmic difference (DGCLD), and diffusion probabilistic least mean square (DPLMS) algorithms, the simulation experiment results show that the DFair algorithm is more robust to input signals and impulsive interference. In conclusion, Theoretical analysis and simulation results show that the DFair algorithm performs better when estimating an unknown linear system in the changeable impulsive interference environments.

L2,1-Extreme Learning Machine: An Efficient Robust Classifier for Tumor Classification

With the development of cancer research, various gene expression datasets containing cancer information show an explosive growth trend. In addition, due to the continuous maturity of single-cell RNA sequencing (scRNA-seq) technology, the protein information and pedigree information of a single cell are also continuously mined. It is a technical problem of how to classify these high-dimensional data correctly. In recent years, Extreme Learning Machine (ELM) has been widely used in the field of supervised learning and unsupervised learning. However, the traditional ELM does not consider the robustness of the method. To improve the robustness of ELM, in this paper, a novel ELM method based on L_2,1-norm named L_2,1-Extreme Learning Machine (L_2,1 -ELM) has been proposed. The method introduces L_2,1-norm on loss function to improve the robustness, and minimizes the influence of noise and outliers. Firstly, we evaluate the new method on five UCI datasets. The experiment results prove that our method can achieve competitive results. Next, the novel method is applied to the problem of classification of cancer samples and single-cell RNA sequencing datasets. The experimental results on The Cancer Genome Atlas (TCGA) datasets and scRNA-seq datasets prove that ELM and its variants has great potential in the classification of cancer samples.