1
|
Feature selection for distance-based regression: An umbrella review and a one-shot wrapper. Neurocomputing 2022. [DOI: 10.1016/j.neucom.2022.11.023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
2
|
Pan M, Sun Z, Wang C, Cao G. A multi-label feature selection method based on an approximation of interaction information. INTELL DATA ANAL 2022. [DOI: 10.3233/ida-215985] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
High-dimensional multi-label data is widespread in practical applications, which brings great challenges to the research field of pattern recognition and machine learning. Many feature selection algorithms have been proposed in recent years, among which the filtering feature selection algorithm is the most popular one because of its simplicity. Therefore, filtering feature selection has become a hot research topic, especially the multi-label feature selection algorithm based on mutual information. In the algorithm, the computation cost of high dimensional mutual information is expensive. How to approximate high order mutual information based on low order mutual information has become a major research direction. To our best knowledge, all existing feature selection algorithms that consider the label correlation will increase the computational cost greatly. Therefore, this paper proposes an approximation method of three-dimensional interaction information, which is applied to the calculation of correlation and redundancy. It can take the correlation of labels into account and don’t increase the computation cost significantly at the same time. Experiments analysis results show that the proposed method is effective.
Collapse
|
3
|
Zaman EAK, Mohamed A, Ahmad A. Feature selection for online streaming high-dimensional data: A state-of-the-art review. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.109355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
4
|
|
5
|
|
6
|
|
7
|
Zhang P, Sheng J, Gao W, Hu J, Li Y. Multi-label feature selection method based on dynamic weight. Soft comput 2022. [DOI: 10.1007/s00500-021-06664-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
8
|
Basgalupp M, Cerri R, Schietgat L, Triguero I, Vens C. Beyond global and local multi-target learning. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2021.08.022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
9
|
Paul D, Jain A, Saha S, Mathew J. Multi-objective PSO based online feature selection for multi-label classification. Knowl Based Syst 2021. [DOI: 10.1016/j.knosys.2021.106966] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
|
10
|
A novel binary many-objective feature selection algorithm for multi-label data classification. INT J MACH LEARN CYB 2021. [DOI: 10.1007/s13042-021-01291-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
11
|
Cost-sensitive feature selection on multi-label data via neighborhood granularity and label enhancement. APPL INTELL 2021. [DOI: 10.1007/s10489-020-01993-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
12
|
Zhang P, Gao W, Hu J, Li Y. Multi-label feature selection based on the division of label topics. Inf Sci (N Y) 2021. [DOI: 10.1016/j.ins.2020.12.036] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
13
|
Multi-Label Feature Selection Based on High-Order Label Correlation Assumption. ENTROPY 2020; 22:e22070797. [PMID: 33286568 PMCID: PMC7517369 DOI: 10.3390/e22070797] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/14/2020] [Revised: 07/12/2020] [Accepted: 07/20/2020] [Indexed: 11/17/2022]
Abstract
Multi-label data often involve features with high dimensionality and complicated label correlations, resulting in a great challenge for multi-label learning. Feature selection plays an important role in multi-label learning to address multi-label data. Exploring label correlations is crucial for multi-label feature selection. Previous information-theoretical-based methods employ the strategy of cumulative summation approximation to evaluate candidate features, which merely considers low-order label correlations. In fact, there exist high-order label correlations in label set, labels naturally cluster into several groups, similar labels intend to cluster into the same group, different labels belong to different groups. However, the strategy of cumulative summation approximation tends to select the features related to the groups containing more labels while ignoring the classification information of groups containing less labels. Therefore, many features related to similar labels are selected, which leads to poor classification performance. To this end, Max-Correlation term considering high-order label correlations is proposed. Additionally, we combine the Max-Correlation term with feature redundancy term to ensure that selected features are relevant to different label groups. Finally, a new method named Multi-label Feature Selection considering Max-Correlation (MCMFS) is proposed. Experimental results demonstrate the classification superiority of MCMFS in comparison to eight state-of-the-art multi-label feature selection methods.
Collapse
|
14
|
Mishra NK, Singh PK. FS-MLC: Feature selection for multi-label classification using clustering in feature space. Inf Process Manag 2020. [DOI: 10.1016/j.ipm.2020.102240] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
|
15
|
Sechidis K, Spyromitros-Xioufis E, Vlahavas I. Information Theoretic Multi-Target Feature Selection via Output Space Quantization. ENTROPY 2019. [PMCID: PMC7515384 DOI: 10.3390/e21090855] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
A key challenge in information theoretic feature selection is to estimate mutual information expressions that capture three desirable terms—the relevancy of a feature with the output, the redundancy and the complementarity between groups of features. The challenge becomes more pronounced in multi-target problems, where the output space is multi-dimensional. Our work presents an algorithm that captures these three desirable terms and is suitable for the well-known multi-target prediction settings of multi-label/dimensional classification and multivariate regression. We achieve this by combining two ideas—deriving low-order information theoretic approximations for the input space and using quantization algorithms for deriving low-dimensional approximations of the output space. Under the above framework we derive a novel criterion, Group-JMI-Rand, which captures various high-order target interactions. In an extensive experimental study we showed that our suggested criterion achieves competing performance against various other information theoretic feature selection criteria suggested in the literature.
Collapse
Affiliation(s)
- Konstantinos Sechidis
- Department of Computer Science, Aristotle University, 54124 Thessaloniki, Greece
- School of Computer Science, University of Manchester, Manchester M13 9PL, UK
- Correspondence:
| | | | - Ioannis Vlahavas
- Department of Computer Science, Aristotle University, 54124 Thessaloniki, Greece
| |
Collapse
|
16
|
A Feature Selection Method for Multi-Label Text Based on Feature Importance. APPLIED SCIENCES-BASEL 2019. [DOI: 10.3390/app9040665] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Multi-label text classification refers to a text divided into multiple categories simultaneously, which corresponds to a text associated with multiple topics in the real world. The feature space generated by text data has the characteristics of high dimensionality and sparsity. Feature selection is an efficient technology that removes useless and redundant features, reduces the dimension of the feature space, and avoids dimension disaster. A feature selection method for multi-label text based on feature importance is proposed in this paper. Firstly, multi-label texts are transformed into single-label texts using the label assignment method. Secondly, the importance of each feature is calculated using the method based on Category Contribution (CC). Finally, features with higher importance are selected to construct the feature space. In the proposed method, the feature importance is calculated from the perspective of the category, which ensures the selected features have strong category discrimination ability. Specifically, the contributions of the features to each category from two aspects of inter-category and intra-category are calculated, then the importance of the features is obtained with the combination of them. The proposed method is tested on six public data sets and the experimental results are good, which demonstrates the effectiveness of the proposed method.
Collapse
|
17
|
Sun Z, Zhang J, Dai L, Li C, Zhou C, Xin J, Li S. Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2018.10.047] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
|
18
|
|
19
|
|
20
|
|
21
|
Huang R, Jiang W, Sun G. Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recognit Lett 2018. [DOI: 10.1016/j.patrec.2018.08.021] [Citation(s) in RCA: 73] [Impact Index Per Article: 12.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
22
|
Arnaiz-González Á, Díez-Pastor JF, Rodríguez JJ, García-Osorio C. Local sets for multi-label instance selection. Appl Soft Comput 2018. [DOI: 10.1016/j.asoc.2018.04.016] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
23
|
Charte F, Rivera AJ, Charte D, del Jesus MJ, Herrera F. Tips, guidelines and tools for managing multi-label datasets: The mldr.datasets R package and the Cometa data repository. Neurocomputing 2018. [DOI: 10.1016/j.neucom.2018.02.011] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
24
|
A PSO-based multi-objective multi-label feature selection method in classification. Sci Rep 2017; 7:376. [PMID: 28336938 PMCID: PMC5428503 DOI: 10.1038/s41598-017-00416-0] [Citation(s) in RCA: 56] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2016] [Accepted: 02/23/2017] [Indexed: 11/08/2022] Open
Abstract
Feature selection is an important data preprocessing technique in multi-label classification. Although a large number of studies have been proposed to tackle feature selection problem, there are a few cases for multi-label data. This paper studies a multi-label feature selection algorithm using an improved multi-objective particle swarm optimization (PSO), with the purpose of searching for a Pareto set of non-dominated solutions (feature subsets). Two new operators are employed to improve the performance of the proposed PSO-based algorithm. One operator is adaptive uniform mutation with action range varying over time, which is used to extend the exploration capability of the swarm; another is a local learning strategy, which is designed to exploit the areas with sparse solutions in the search space. Moreover, the idea of the archive, and the crowding distance are applied to PSO for finding the Pareto set. Finally, experiments verify that the proposed algorithm is a useful approach of feature selection for multi-label classification problem.
Collapse
|
25
|
Efficient Multi-Label Feature Selection Using Entropy-Based Label Selection. ENTROPY 2016. [DOI: 10.3390/e18110405] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|