Neighborhood rough set based heterogeneous feature subset selection

A dynamic attribute reduction algorithm based on relative neighborhood discernibility degree

This paper addresses the current existence of attribute reduction algorithms for incomplete hybrid decision-making systems, including low attribute reduction efficiency, low classification accuracy and lack of consideration of unlabeled data types. To address these issues, this paper first redefines the weakly labeled relative neighborhood discernibility degree and develops a non-dynamic attribute reduction algorithm. In addition, this paper proposes an incremental update mechanism for weakly tagged relative neighborhood discernibility degree and introduces a new dynamic attribute reduction algorithm for increasing the set of objects based on it. Meanwhile, this paper also compares and analyses the improved algorithm proposed in this study with two existing attribute reduction algorithms using 8 data sets in the UCI database. The results show that the dynamic attribute reduction algorithm proposed in this paper achieves higher attribute reduction efficiency and classification accuracy, which further validates the effectiveness of the algorithm proposed in this paper.

Feature selection based on neighborhood rough sets and Gini index

Neighborhood rough set is considered an essential approach for dealing with incomplete data and inexact knowledge representation, and it has been widely applied in feature selection. The Gini index is an indicator used to evaluate the impurity of a dataset and is also commonly employed to measure the importance of features in feature selection. This article proposes a novel feature selection methodology based on these two concepts. In this methodology, we present the neighborhood Gini index and the neighborhood class Gini index and then extensively discuss their properties and relationships with attributes. Subsequently, two forward greedy feature selection algorithms are developed using these two metrics as a foundation. Finally, to comprehensively evaluate the performance of the algorithm proposed in this article, comparative experiments were conducted on 16 UCI datasets from various domains, including industry, food, medicine, and pharmacology, against four classical neighborhood rough set-based feature selection algorithms. The experimental results indicate that the proposed algorithm improves the average classification accuracy on the 16 datasets by over 6%, with improvements exceeding 10% in five. Furthermore, statistical tests reveal no significant differences between the proposed algorithm and the four classical neighborhood rough set-based feature selection algorithms. However, the proposed algorithm demonstrates high stability, eliminating most redundant or irrelevant features effectively while enhancing classification accuracy. In summary, the algorithm proposed in this article outperforms classical neighborhood rough set-based feature selection algorithms.

Large-Scale Meta-Heuristic Feature Selection Based on BPSO Assisted Rough Hypercuboid Approach

The selection of prominent features for building more compact and efficient models is an important data preprocessing task in the field of data mining. The rough hypercuboid approach is an emerging technique that can be applied to eliminate irrelevant and redundant features, especially for the inexactness problem in approximate numerical classification. By integrating the meta-heuristic-based evolutionary search technique, a novel global search method for numerical feature selection is proposed in this article based on the hybridization of the rough hypercuboid approach and binary particle swarm optimization (BPSO) algorithm, namely RH-BPSO. To further alleviate the issue of high computational cost when processing large-scale datasets, parallelization approaches for calculating the hybrid feature evaluation criteria are presented by decomposing and recombining hypercuboid equivalence partition matrix via horizontal data partitioning. A distributed meta-heuristic optimized rough hypercuboid feature selection (DiRH-BPSO) algorithm is thus developed and embedded in the Apache Spark cloud computing model. Extensive experimental results indicate that RH-BPSO is promising and can significantly outperform the other representative feature selection algorithms in terms of classification accuracy, the cardinality of the selected feature subset, and execution efficiency. Moreover, experiments on distributed-memory multicore clusters show that DiRH-BPSO is significantly faster than its sequential counterpart and is perfectly capable of completing large-scale feature selection tasks that fail on a single node due to memory constraints. Parallel scalability and extensibility analysis also demonstrate that DiRH-BPSO could scale out and extend well with the growth of computational nodes and the volume of data.

RST: Rough Set Transformer for Point Cloud Learning

Point cloud data generated by LiDAR sensors play a critical role in 3D sensing systems, with applications encompassing object classification, part segmentation, and point cloud recognition. Leveraging the global learning capacity of dot product attention, transformers have recently exhibited outstanding performance in point cloud learning tasks. Nevertheless, existing transformer models inadequately address the challenges posed by uncertainty features in point clouds, which can introduce errors in the dot product attention mechanism. In response to this, our study introduces a novel global guidance approach to tolerate uncertainty and provide a more reliable guidance. We redefine the granulation and lower-approximation operators based on neighborhood rough set theory. Furthermore, we introduce a rough set-based attention mechanism tailored for point cloud data and present the rough set transformer (RST) network. Our approach utilizes granulation concepts derived from token clusters, enabling us to explore relationships between concepts from an approximation perspective, rather than relying on specific dot product functions. Empirically, our work represents the pioneering fusion of rough set theory and transformer networks for point cloud learning. Our experimental results, including point cloud classification and segmentation tasks, demonstrate the superior performance of our method. Our method establishes concepts based on granulation generated from clusters of tokens. Subsequently, relationships between concepts can be explored from an approximation perspective, instead of relying on specific dot product or addition functions. Empirically, our work represents the pioneering fusion of rough set theory and transformer networks for point cloud learning. Our experimental results, including point cloud classification and segmentation tasks, demonstrate the superior performance of our method.

Relative Fuzzy Rough Approximations for Feature Selection and Classification

Fuzzy rough set (FRS) theory is generally used to measure the uncertainty of data. However, this theory cannot work well when the class density of a data distribution differs greatly. In this work, a relative distance measure is first proposed to fit the mentioned data distribution. Based on the measure, a relative FRS model is introduced to remedy the mentioned imperfection of classical FRSs. Then, the positive region, negative region, and boundary region are defined to measure the uncertainty of data with the relative FRSs. Besides, a relative fuzzy dependency is defined to evaluate the importance of features to decision. With the proposed feature evaluation, we propose a feature selection algorithm and design a classifier based on the maximal positive region. The classification principle is that an unlabeled sample will be classified into the class corresponding to the maximal degree of the positive region. Experimental results show the relative fuzzy dependency is an effective and efficient measure for evaluating features, and the proposed feature selection algorithm presents better performance than some classical algorithms. Besides, it also shows the proposed classifier can achieve slightly better performance than the KNN classifier, which demonstrates that the maximal positive region-based classifier is effective and feasible.

A self-adaptive quantum equilibrium optimizer with artificial bee colony for feature selection

Feature selection (FS) is a popular data pre-processing technique in machine learning to extract the optimal features to maintain or increase the classification accuracy of the dataset, which is a combinatorial optimization problem, requiring a powerful optimizer to obtain the optimum subset. The equilibrium optimizer (EO) is a recent physical-based metaheuristic algorithm with good performance for various optimization problems, but it may encounter premature or the local convergence in feature selection. This work presents a self-adaptive quantum EO with artificial bee colony for feature selection, named SQEOABC. In the proposed algorithm, the quantum theory and the self-adaptive mechanism are employed into the updating rule of EO to enhance convergence, and the updating mechanism from the artificial bee colony is also incorporated into EO to achieve appropriate FS solutions. In the experiments, 25 benchmark datasets from the UCI repository are investigated to verify SQEOABC, which is compared with several state-of-the-art metaheuristic algorithms and the variants of EO. The statistical results of fitness values and accuracy demonstrate that SQEOABC has better performance than the compared algorithms and the variants of EO. Finally, a real-world FS problem from COVID-19 illustrates the effectiveness and superiority of SQEOABC.

A novel neighborhood rough set-based feature selection method and its application to biomarker identification of schizophrenia

Feature selection can disclose biomarkers of mental disorders that have unclear biological mechanisms. Although neighborhood rough set (NRS) has been applied to discover important sparse features, it has hardly ever been utilized in neuroimaging-based biomarker identification, probably due to the inadequate feature evaluation metric and incomplete information provided under a single-granularity. Here, we propose a new NRS-based feature selection method and successfully identify brain functional connectivity biomarkers of schizophrenia (SZ) using functional magnetic resonance imaging (fMRI) data. Specifically, we develop a new weighted metric based on NRS combined with information entropy to evaluate the capacity of features in distinguishing different groups. Inspired by multi-granularity information maximization theory, we further take advantage of the complementary information from different neighborhood sizes via a multi-granularity fusion to obtain the most discriminative and stable features. For validation, we compare our method with six popular feature selection methods using three public omics datasets as well as resting-state fMRI data of 393 SZ patients and 429 healthy controls. Results show that our method obtained higher classification accuracies on both omics data (100.0%, 88.6%, and 72.2% for three omics datasets, respectively) and fMRI data (93.9% for main dataset, and 76.3% and 83.8% for two independent datasets, respectively). Moreover, our findings reveal biologically meaningful substrates of SZ, notably involving the connectivity between the thalamus and superior temporal gyrus as well as between the postcentral gyrus and calcarine gyrus. Taken together, we propose a new NRS-based feature selection method that shows the potential of exploring effective and sparse neuroimaging-based biomarkers of mental disorders.

Class-specific attribute reducts based on neighborhood rough sets

Attribute reduction based on rough sets is an effective approach of data learning in intelligent systems, and it has two basic types. Traditional classification-based attribute reducts mainly complete the classification task, while recent class-specific reducts directly realize the class-pattern recognition. Neighborhood rough sets have the covering-structure extension and data-diversity applicability, but their attribute reducts concern only the neighborhood classification-based reducts. This paper proposes class-specific attribute reducts based on neighborhood rough sets, so as to promote the optimal identification and robust processing of specific classes. At first, neighborhood class-specific reducts are defined, and their basic properties and heuristic algorithms are acquired by granulation monotonicity. Then, hierarchical relationships between the neighborhood classification-based and class-specific reducts are analyzed, and mutual derivation algorithms are designed. Finally, the theoretical constructions and mutual relationships are effectively verified by both decision table examples and data set experiments. The neighborhood class-specific reducts robustly extend the existing class-specific reducts, and they also provide a hierarchical mechanism for the neighborhood classification-based reducts, thus facilitating wide applications of class-pattern processing.

Multi-Label Attribute Reduction Based on Neighborhood Multi-Target Rough Sets

The rough set model has two symmetry approximations called upper approximation and lower approximation, which correspond to a concept’s intension and extension, respectively. Multi-label learning enforces the rough set model, which wants to be applied considering the correlations among labels, while the target concept should not be limited to only one. This paper proposes a multi-target model considering label correlation (Neighborhood Multi-Target Rough Sets, NMTRS) and proposes an attribute reduction approach based on NMTRS. First, some definitions of NMTRS are introduced. Second, some properties of NMTRS are discussed. Third, some discussion about the attribute significance measure is given. Fourth, the attribute reduction approaches based on NMTRS are proposed. Finally, the efficiency and validity of the designed algorithms are verified by experiments. The experiments show that our algorithm shows considerable performance when compared to state-of-the-art approaches.

A Novel Feature Selection Method for High-Dimensional Mixed Decision Tables

Attribute reduction, also called feature selection, is one of the most important issues of rough set theory, which is regarded as a vital preprocessing step in pattern recognition, machine learning, and data mining. Nowadays, high-dimensional mixed and incomplete data sets are very common in real-world applications. Certainly, the selection of a promising feature subset from such data sets is a very interesting, but challenging problem. Almost all of the existing methods generated a cover on the space of objects to determine important features. However, some tolerance classes in the cover are useless for the computational process. Thus, this article introduces a new concept of stripped neighborhood covers to reduce unnecessary tolerance classes from the original cover. Based on the proposed stripped neighborhood cover, we define a new reduct in mixed and incomplete decision tables, and then design an efficient heuristic algorithm to find this reduct. For each loop in the main loop of the proposed algorithm, we use an error measure to select an optimal feature and put it into the selected feature subset. Besides, to deal more efficiently with high-dimensional data sets, we also determine redundant features after each loop and remove them from the candidate feature subset. For the purpose of verifying the performance of the proposed algorithm, we carry out experiments on data sets downloaded from public data sources to compare with existing state-of-the-art algorithms. Experimental results showed that our algorithm outperforms compared algorithms, especially in classification accuracy.

An Ensemble Framework to Forest Optimization Based Reduct Searching

Essentially, the solution to an attribute reduction problem can be viewed as a reduct searching process. Currently, among various searching strategies, meta-heuristic searching has received extensive attention. As a new emerging meta-heuristic approach, the forest optimization algorithm (FOA) is introduced to the problem solving of attribute reduction in this study. To further improve the classification performance of selected attributes in reduct, an ensemble framework is also developed: firstly, multiple reducts are obtained by FOA and data perturbation, and the structure of those multiple reducts is symmetrical, which indicates that no order exists among those reducts; secondly, multiple reducts are used to execute voting classification over testing samples. Finally, comprehensive experiments on over 20 UCI datasets clearly validated the effectiveness of our framework: it is not only beneficial to output reducts with superior classification accuracies and classification stabilities but also suitable for data pre-processing with noise. This improvement work we have performed makes the FOA obtain better benefits in the data processing of life, health, medical and other fields.

A deep fuzzy model for diagnosis of COVID-19 from CT images

From early 2020, a novel coronavirus disease pneumonia has shown a global "pandemic" trend at an extremely fast speed. Due to the magnitude of its harm, it has become a major global public health event. In the face of dramatic increase in the number of patients with COVID-19, the need for quick diagnosis of suspected cases has become particularly critical. Therefore, this paper constructs a fuzzy classifier, which aims to detect infected subjects by observing and analyzing the CT images of suspected patients. Firstly, a deep learning algorithm is used to extract the low-level features of CT images in the COVID-CT dataset. Subsequently, we analyze the extracted feature information with attribute reduction algorithm to obtain features with high recognition. Then, some key features are selected as the input for the fuzzy diagnosis model to the training model. Finally, several images in the dataset are used as the test set to test the trained fuzzy classifier. The obtained accuracy rate is 94.2%, and the F1-score is 93.8%. Experimental results show that, compared with the deep learning diagnosis methods widely used in medical image analysis, the proposed fuzzy model improves the accuracy and efficiency of diagnosis, which consequently helps to curb the spread of COVID-19.