Heuristic-based feature selection for rough set approach

Improved EAV-Based Algorithm for Decision Rules Construction

In this article, we present a modification of the algorithm based on EAV (entity-attribute-value) model, for induction of decision rules, utilizing novel approach for attribute ranking. The selection of attributes used as premises of decision rules, is an important stage of the process of rules induction. In the presented approach, this task is realized using ranking of attributes based on standard deviation of attributes' values per decision classes, which is considered as a distinguishability level. The presented approach allows to work not only with numerical values of attributes but also with categorical ones. For this purpose, an additional step of data transformation into a matrix format has been proposed. It allows to transform data table into a binary one with proper equivalents of categorical values of attributes and ensures independence of the influence of the attribute selection function from the data type of variables. The motivation for the proposed method is the development of an algorithm which allows to construct rules close to optimal ones in terms of length, while maintaining enough good classification quality. The experiments presented in the paper have been performed on data sets from UCI ML Repository, comparing results of the proposed approach with three selected greedy heuristics for induction of decision rules, taking into consideration classification accuracy and length and support of constructed rules. The obtained results show that for the most part of datasests, the average length of rules obtained for 80% of best attributes from the ranking is very close to values obtained for the whole set of attributes. In case of classification accuracy, for 50% of considered datasets, results obtained for 80% of best attributes from the ranking are higher or the same as results obtained for the whole set of attributes.

Pruning Decision Rules by Reduct-Based Weighting and Ranking of Features

Methods and techniques of feature selection support expert domain knowledge in the search for attributes, which are the most important for a task. These approaches can also be used in the process of closer tailoring of the obtained solutions when dimensionality reduction is aimed not only at variables but also at learners. The paper reports on research where attribute rankings were employed to filter induced decision rules. The rankings were constructed through the proposed weighting factor based on the concept of decision reducts-a feature reduction mechanism embedded in the rough set theory. Classical rough sets operate only in discrete input space by indiscernibility relation. Replacing it with dominance enables processing real-valued data. Decision reducts were found for both numeric and discrete attributes, transformed by selected discretisation approaches. The calculated ranking scores were used to control the selection of decision rules. The performance of the resulting rule classifiers was observed for the entire range of rejected variables, for decision rules with conditions on continuous values, discretised conditions, and also inferred from discrete data. The predictive powers were analysed and compared to detect existing trends. The experiments show that for all variants of the rule sets, not only was dimensionality reduction possible, but also predictions were improved, which validated the proposed methodology.

Unsupervised feature selection based on incremental forward iterative Laplacian score

Feature selection facilitates intelligent information processing, and the unsupervised learning of feature selection has become important. In terms of unsupervised feature selection, the Laplacian score (LS) provides a powerful measurement and optimization method, and good performance has been achieved using the recent forward iterative Laplacian score (FILS) algorithm. However, there is still room for advancement. The aim of this paper is to improve the FILS algorithm, and thus, feature significance (SIG) is mainly introduced to develop a high-quality selection method, i.e., the incremental forward iterative Laplacian score (IFILS) algorithm. Based on the modified LS, the metric difference in the incremental feature process motivates SIG. Therefore, SIG offers a dynamic characterization by considering initial and terminal states, and it promotes the current FILS measurement on only the terminal state. Then, both the modified LS and integrated SIG acquire granulation nonmonotonicity and uncertainty, especially on incremental feature chains, and the corresponding verification is achieved by completing examples and experiments. Furthermore, a SIG-based incremental criterion of minimum selection is designed to choose optimization features, and thus, the IFILS algorithm is naturally formulated to implement unsupervised feature selection. Finally, an in-depth comparison of the IFILS algorithm with the FILS algorithm is achieved using data experiments on multiple datasets, including a nominal dataset of COVID-19 surveillance. As validated by the experimental results, the IFILS algorithm outperforms the FILS algorithm and achieves better classification performance.