Jiang B, Ding C. Revisiting L
2,1-Norm Robustness With Vector Outlier Regularization.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2020;
31:5624-5629. [PMID:
32011268 DOI:
10.1109/tnnls.2020.2964297]
[Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
In many real-world applications, data usually contain outliers. One popular approach is to use the L2,1 -norm function as a robust loss/error function. However, the robustness of the L2,1 -norm function is not well understood so far. In this brief, we propose a new vector outlier regularization (VOR) framework to understand and analyze the robustness of the L2,1 -norm function. Our VOR function defines a data point to be the outlier if it is outside a threshold with respect to a theoretical prediction, and regularizes it, i.e., pull it back to the threshold line. Thus, in the VOR function, how far an outlier lies away from its theoretical predicted value does not affect the final regularization and analysis results. One important aspect of the VOR function is that it has an equivalent continuous formulation, based on which we can prove that the L2,1 -norm function is the limiting case of the proposed VOR function. Based on this theoretical result, we thus provide a new and intuitive explanation for the robustness property of the L2,1 -norm function. As an example, we use the VOR function to matrix factorization and propose a VOR principal component analysis (PCA) (VORPCA). We show some benefits of VORPCA on data reconstruction and clustering tasks.
Collapse