1
|
Wang J, Xie F, Nie F, Li X. Generalized and Robust Least Squares Regression. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:7006-7020. [PMID: 36264726 DOI: 10.1109/tnnls.2022.3213594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
As a simple yet effective method, least squares regression (LSR) is extensively applied for data regression and classification. Combined with sparse representation, LSR can be extended to feature selection (FS) as well, in which l1 regularization is often applied in embedded FS algorithms. However, because the loss function is in the form of squared error, LSR and its variants are sensitive to noises, which significantly degrades the effectiveness and performance of classification and FS. To cope with the problem, we propose a generalized and robust LSR (GRLSR) for classification and FS, which is made up of arbitrary concave loss function and the l2,p -norm regularization term. Meanwhile, an iterative algorithm is applied to efficiently deal with the nonconvex minimization problem, in which an additional weight to suppress the effect of noises is added to each data point. The weights can be automatically assigned according to the error of the samples. When the error is large, the value of the corresponding weight is small. It is this mechanism that allows GRLSR to reduce the impact of noises and outliers. According to the different formulations of the concave loss function, four specific methods are proposed to clarify the essence of the framework. Comprehensive experiments on corrupted datasets have proven the advantage of the proposed method.
Collapse
|
2
|
Zhang J, Lin Y, Jiang M, Li S, Tang Y, Long J, Weng J, Tan KC. Fast Multilabel Feature Selection via Global Relevance and Redundancy Optimization. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:5721-5734. [PMID: 36215379 DOI: 10.1109/tnnls.2022.3208956] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Information theoretical-based methods have attracted a great attention in recent years and gained promising results for multilabel feature selection (MLFS). Nevertheless, most of the existing methods consider a heuristic way to the grid search of important features, and they may also suffer from the issue of fully utilizing labeling information. Thus, they are probable to deliver a suboptimal result with heavy computational burden. In this article, we propose a general optimization framework global relevance and redundancy optimization (GRRO) to solve the learning problem. The main technical contribution in GRRO is a formulation for MLFS while feature relevance, label relevance (i.e., label correlation), and feature redundancy are taken into account, which can avoid repetitive entropy calculations to obtain a global optimal solution efficiently. To further improve the efficiency, we extend GRRO to filter out inessential labels and features, thus facilitating fast MLFS. We call the extension as GRROfast, in which the key insights are twofold: 1) promising labels and related relevant features are investigated to reduce ineffective calculations in terms of features, even labels and 2) the framework of GRRO is reconstructed to generate the optimal result with an ensemble. Moreover, our proposed algorithms have an excellent mechanism for exploiting the inherent properties of multilabel data; specifically, we provide a formulation to enhance the proposal with label-specific features. Extensive experiments clearly reveal the effectiveness and efficiency of our proposed algorithms.
Collapse
|
3
|
Sheikhpour R, Berahmand K, Forouzandeh S. Hessian-based semi-supervised feature selection using generalized uncorrelated constraint. Knowl Based Syst 2023. [DOI: 10.1016/j.knosys.2023.110521] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
|
4
|
An in-depth and contrasting survey of meta-heuristic approaches with classical feature selection techniques specific to cervical cancer. Knowl Inf Syst 2023. [DOI: 10.1007/s10115-022-01825-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
5
|
Sheikhpour R. A local spline regression-based framework for semi-supervised sparse feature selection. Knowl Based Syst 2023. [DOI: 10.1016/j.knosys.2023.110265] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
|
6
|
Fan M, Zhang X, Hu J, Gu N, Tao D. Adaptive Data Structure Regularized Multiclass Discriminative Feature Selection. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2022; 33:5859-5872. [PMID: 33882003 DOI: 10.1109/tnnls.2021.3071603] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Feature selection (FS), which aims to identify the most informative subset of input features, is an important approach to dimensionality reduction. In this article, a novel FS framework is proposed for both unsupervised and semisupervised scenarios. To make efficient use of data distribution to evaluate features, the framework combines data structure learning (as referred to as data distribution modeling) and FS in a unified formulation such that the data structure learning improves the results of FS and vice versa. Moreover, two types of data structures, namely the soft and hard data structures, are learned and used in the proposed FS framework. The soft data structure refers to the pairwise weights among data samples, and the hard data structure refers to the estimated labels obtained from clustering or semisupervised classification. Both of these data structures are naturally formulated as regularization terms in the proposed framework. In the optimization process, the soft and hard data structures are learned from data represented by the selected features, and then, the most informative features are reselected by referring to the data structures. In this way, the framework uses the interactions between data structure learning and FS to select the most discriminative and informative features. Following the proposed framework, a new semisupervised FS (SSFS) method is derived and studied in depth. Experiments on real-world data sets demonstrate the effectiveness of the proposed method.
Collapse
|
7
|
Semi-supervised feature selection based on pairwise constraint-guided dual space latent representation learning and double sparse graphs discriminant. APPL INTELL 2022. [DOI: 10.1007/s10489-022-04040-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
8
|
Lai J, Chen H, Li T, Yang X. Adaptive graph learning for semi-supervised feature selection with redundancy minimization. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2022.07.102] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
9
|
Lai J, Chen H, Li W, Li T, Wan J. Semi-supervised feature selection via adaptive structure learning and constrained graph learning. Knowl Based Syst 2022. [DOI: 10.1016/j.knosys.2022.109243] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
10
|
Chen Z, Liu Y, Zhang Y, Jin R, Tao J, Chen L. Low-rank sparse feature selection with incomplete labels for Alzheimer's disease progression prediction. Comput Biol Med 2022; 147:105705. [PMID: 35717935 DOI: 10.1016/j.compbiomed.2022.105705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 05/16/2022] [Accepted: 06/04/2022] [Indexed: 11/29/2022]
Abstract
BACKGROUND How to predict the cognitive performance of Alzheimer's disease (AD) and identify the informative neuroimaging markers is essential for timely treatment and possible delay of the disease. However, incomplete labeled samples and noises in neuroimaging data pose challenges to building reliable and robust prediction models. In this paper, we present a model named Low-rank Sparse Feature Selection with Incomplete Labels (LSFSIL) for predicting cognitive performance and identifying informative neuroimaging markers with MRI data and incomplete cognitive scores. METHOD We propose a sparse matrix decomposition method to decompose the incomplete cognitive score matrix into two parts for recovering missing scores and utilizing incomplete labeled data. The former is the recovered cognitive score matrix without missing values. To make the recovered scores close to the real ones, a manifold regularizer is devised to fit the label distribution for capturing the label correlations locally. The latter is a ℓ1-norm regularized matrix which represents the associated errors. Next, a low-rank regression model that regards the recovered matrix as the target is developed to increase the robustness to noises and outliers. Besides, ℓ2,1-norm is introduced into the objective function as a sparse regularization to identify the important features. RESULTS Experimental results demonstrate that LSFSIL achieves higher performance and outperforms several state-of-the-art feature selection approaches. Moreover, the neuroimaging markers selected by LSFSIL are consistent with the previous AD studies. CONCLUSIONS LSFSIL is effective in informative neuroimaging marker identification for cognitive performance prediction with incomplete labeled data.
Collapse
Affiliation(s)
- Zhi Chen
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Yongguo Liu
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China.
| | - Yun Zhang
- Knowledge and Data Engineering Laboratory of Chinese Medicine, School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, China
| | - Rongjiang Jin
- College of Health Preservation and Rehabilitation, Chengdu University of Traditional Chinese Medicine, Chengdu, 610075, China
| | - Jing Tao
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, 350122, China
| | - Lidian Chen
- College of Rehabilitation Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, 350122, China
| |
Collapse
|
11
|
Robust dual-graph regularized and minimum redundancy based on self-representation for semi-supervised feature selection. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.03.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
12
|
Ni J, Huang Z, Hu Y, Lin C. A two-stage embedding model for recommendation with multimodal auxiliary information. Inf Sci (N Y) 2022. [DOI: 10.1016/j.ins.2021.09.006] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
13
|
Wu X, Chen H, Li T, Wan J. Semi-supervised feature selection with minimal redundancy based on local adaptive. APPL INTELL 2021. [DOI: 10.1007/s10489-021-02288-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
14
|
Sparse robust multiview feature selection via adaptive-weighting strategy. INT J MACH LEARN CYB 2021. [DOI: 10.1007/s13042-021-01453-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
15
|
Cao J, Wang R, Jia Y, Zhang X, Wang S, Kwong S. No-reference image quality assessment for contrast-changed images via a semi-supervised robust PCA model. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.07.052] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
16
|
Wu X, Jiang B, Yu K, Chen H. Separation and recovery Markov boundary discovery and its application in EEG-based emotion recognition. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.04.071] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
17
|
Wang J, Liang J, Cui J, Liang J. Semi-supervised learning with mixed-order graph convolutional networks. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.05.057] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
18
|
Ensemble of feature selection algorithms: a multi-criteria decision-making approach. INT J MACH LEARN CYB 2021. [DOI: 10.1007/s13042-021-01347-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|