Mutual information criterion for feature selection from incomplete data

Evaluating the utility of identity-by-descent segment numbers for relatedness inference via information theory and classification

Despite decades of methods development for classifying relatives in genetic studies, pairwise relatedness methods' recalls are above 90% only for first through third-degree relatives. The top-performing approaches, which leverage identity-by-descent segments, often use only kinship coefficients, while others, including estimation of recent shared ancestry (ERSA), use the number of segments relatives share. To quantify the potential for using segment numbers in relatedness inference, we leveraged information theory measures to analyze exact (i.e. produced by a simulator) identity-by-descent segments from simulated relatives. Over a range of settings, we found that the mutual information between the relatives' degree of relatedness and a tuple of their kinship coefficient and segment number is on average 4.6% larger than between the degree and the kinship coefficient alone. We further evaluated identity-by-descent segment number utility by building a Bayes classifier to predict first through sixth-degree relationships using different feature sets. When trained and tested with exact segments, the inclusion of segment numbers improves the recall by between 0.28% and 3% for second through sixth-degree relatives. However, the recalls improve by less than 1.8% per degree when using inferred segments, suggesting limitations due to identity-by-descent detection accuracy. Last, we compared our Bayes classifier that includes segment numbers with both ERSA and IBIS and found comparable recalls, with the Bayes classifier and ERSA slightly outperforming each other across different degrees. Overall, this study shows that identity-by-descent segment numbers can improve relatedness inference, but errors from current SNP array-based detection methods yield dampened signals in practice.

An Accelerating Reduction Approach for Incomplete Decision Table Using Positive Approximation Set

Due to the explosive growth of data collected by various sensors, it has become a difficult problem determining how to conduct feature selection more efficiently. To address this problem, we offer a fresh insight into rough set theory from the perspective of a positive approximation set. It is found that a granularity domain can be used to characterize the target knowledge, because of its form of a covering with respect to a tolerance relation. On the basis of this fact, a novel heuristic approach ARIPA is proposed to accelerate representative reduction algorithms for incomplete decision table. As a result, ARIPA in classical rough set model and ARIPA-IVPR in variable precision rough set model are realized respectively. Moreover, ARIPA is adopted to improve the computational efficiency of two existing state-of-the-art reduction algorithms. To demonstrate the effectiveness of the improved algorithms, a variety of experiments utilizing four UCI incomplete data sets are conducted. The performances of improved algorithms are compared with those of original ones as well. Numerical experiments justify that our accelerating approach enhances the existing algorithms to accomplish the reduction task more quickly. In some cases, they fulfill attribute reduction even more stably than the original algorithms do.

Land Cover Classification with Multispectral LiDAR Based on Multi-Scale Spatial and Spectral Feature Selection

The distribution of land cover has an important impact on climate, environment, and public policy planning. The Optech Titan multispectral LiDAR system provides new opportunities and challenges for land cover classification, but the better application of spectral and spatial information of multispectral LiDAR data is a problem to be solved. Therefore, we propose a land cover classification method based on multi-scale spatial and spectral feature selection. The public data set of Tobermory Port collected by the Optech Titan multispectral airborne laser scanner was used as research data, and the data was manually divided into eight categories. The method flow is divided into four steps: neighborhood point selection, spatial–spectral feature extraction, feature selection, and classification. First, the K-nearest neighborhood is used to select the neighborhood points for the multispectral LiDAR point cloud data. Additionally, the spatial and spectral features under the multi-scale neighborhood (K = 20, 50, 100, 150) are extracted. The Equalizer Optimization algorithm is used to perform feature selection on multi-scale neighborhood spatial–spectral features, and a feature subset is obtained. Finally, the feature subset is input into the support vector machine (SVM) classifier for training. Using only small training samples (about 0.5% of the total data) to train the SVM classifier, 91.99% overall accuracy (OA), 93.41% average accuracy (AA) and 0.89 kappa coefficient were obtained in study area. Compared with the original information’s classification result, the OA, AA and kappa coefficient increased by 15.66%, 8.7% and 0.19, respectively. The results show that the constructed spatial–spectral features and the application of the Equalizer Optimization algorithm for feature selection are effective in land cover classification with Titan multispectral LiDAR point data.

Evaluating the impact of multivariate imputation by MICE in feature selection

Handling missing values is a crucial step in preprocessing data in Machine Learning. Most available algorithms for analyzing datasets in the feature selection process and classification or estimation process analyze complete datasets. Consequently, in many cases, the strategy for dealing with missing values is to use only instances with full data or to replace missing values with a mean, mode, median, or a constant value. Usually, discarding missing samples or replacing missing values by means of fundamental techniques causes bias in subsequent analyzes on datasets.

AIM

Demonstrate the positive impact of multivariate imputation in the feature selection process on datasets with missing values.

RESULTS

We compared the effects of the feature selection process using complete datasets, incomplete datasets with missingness rates between 5 and 50%, and imputed datasets by basic techniques and multivariate imputation. The feature selection algorithms used are well-known methods. The results showed that the datasets imputed by multivariate imputation obtained the best results in feature selection compared to datasets imputed by basic techniques or non-imputed incomplete datasets.

CONCLUSIONS

Considering the results obtained in the evaluation, applying multivariate imputation by MICE reduces bias in the feature selection process.

Handling incomplete data classification using imputed feature selected bagging (IFBag) method

Almost all real-world datasets contain missing values. Classification of data with missing values can adversely affect the performance of a classifier if not handled correctly. A common approach used for classification with incomplete data is imputation. Imputation transforms incomplete data with missing values to complete data. Single imputation methods are mostly less accurate than multiple imputation methods which are often computationally much more expensive. This study proposes an imputed feature selected bagging (IFBag) method which uses multiple imputation, feature selection and bagging ensemble learning approach to construct a number of base classifiers to classify new incomplete instances without any need for imputation in testing phase. In bagging ensemble learning approach, data is resampled multiple times with substitution, which can lead to diversity in data thus resulting in more accurate classifiers. The experimental results show the proposed IFBag method is considerably fast and gives 97.26% accuracy for classification with incomplete data as compared to common methods used.

An Efficient Alert Aggregation Method Based on Conditional Rough Entropy and Knowledge Granularity

With the emergence of network security issues, various security devices that generate a large number of logs and alerts are widely used. This paper proposes an alert aggregation scheme that is based on conditional rough entropy and knowledge granularity to solve the problem of repetitive and redundant alert information in network security devices. Firstly, we use conditional rough entropy and knowledge granularity to determine the attribute weights. This method can determine the different important attributes and their weights for different types of attacks. We can calculate the similarity value of two alerts by weighting based on the results of attribute weighting. Subsequently, the sliding time window method is used to aggregate the alerts whose similarity value is larger than a threshold, which is set to reduce the redundant alerts. Finally, the proposed scheme is applied to the CIC-IDS 2018 dataset and the DARPA 98 dataset. The experimental results show that this method can effectively reduce the redundant alerts and improve the efficiency of data processing, thus providing accurate and concise data for the next stage of alert fusion and analysis.

An Efficient Uncertainty Measure-based Attribute Reduction Approach for Interval-valued Data with Missing Values

Attribute reduction plays an important role in knowledge discovery and data mining. Confronted with data characterized by the interval and missing values in many data analysis tasks, it is interesting to research the attribute reduction for interval-valued data with missing values. Uncertainty measures can supply efficient viewpoints, which help us to disclose the substantive characteristics of such data. Therefore, this paper addresses the attribute reduction problem based on uncertainty measure for interval-valued data with missing values. At first, an uncertainty measure is provided for measuring candidate attributes, and then an efficient attribute reduction algorithm is developed for the interval-valued data with missing values. To improve the efficiency of attribute reduction, the objects that fall within the positive region are deleted from the whole object set in the process of selecting attributes. Finally, experimental results demonstrate that the proposed algorithm can find a subset of attributes in much shorter time than existing attribute reduction algorithms without losing the classification performance.

Double-Granule Conditional-Entropies Based on Three-Level Granular Structures

Rough set theory is an important approach for data mining, and it refers to Shannon's information measures for uncertainty measurements. The existing local conditional-entropies have both the second-order feature and application limitation. By improvements of hierarchical granulation, this paper establishes double-granule conditional-entropies based on three-level granular structures (i.e., micro-bottom, meso-middle, macro-top ), and then investigates the relevant properties. In terms of the decision table and its decision classification, double-granule conditional-entropies are proposed at micro-bottom by the dual condition-granule system. By virtue of successive granular summation integrations, they hierarchically evolve to meso-middle and macro-top, to respectively have part and complete condition-granulations. Then, the new measures acquire their number distribution, calculation algorithm, three bounds, and granulation non-monotonicity at three corresponding levels. Finally, the hierarchical constructions and achieved properties are effectively verified by decision table examples and data set experiments. Double-granule conditional-entropies carry the second-order characteristic and hierarchical granulation to deepen both the classical entropy system and local conditional-entropies, and thus they become novel uncertainty measures for information processing and knowledge reasoning.

Multivariate Analysis of Data Sets with Missing Values: An Information Theory-Based Reliability Function

Missing values in complex biological data sets have significant impacts on our ability to correctly detect and quantify interactions in biological systems and to infer relationships accurately. In this article, we propose a useful metaphor to show that information theory measures, such as mutual information and interaction information, can be employed directly for evaluating multivariable dependencies even if data contain some missing values. The metaphor is that of thinking of variable dependencies as information channels between and among variables. In this view, missing data can be thought of as noise that reduces the channel capacity in predictable ways. We extract the available information in the data even if there are missing values and use the notion of channel capacity to assess the reliability of the result. This avoids the common practice—in the absence of prior knowledge of random imputation—of eliminating samples entirely, thus losing the information they can provide. We show how this reliability function can be implemented for pairs of variables, and generalize it for an arbitrary number of variables. Illustrations of the reliability functions for several cases are provided using simulated data.

Urban air quality, meteorology and traffic linkages: Evidence from a sixteen-day particulate matter pollution event in December 2015, Beijing

A heavy 16-day pollution episode occurred in Beijing from December 19, 2015 to January 3, 2016. The mean daily AQI and PM_2.5 were 240.44 and 203.6μg/m³. We analyzed the spatiotemporal characteristics of air pollutants, meteorology and road space speed during this period, then extended to reveal the combined effects of traffic restrictions and meteorology on urban air quality with observational data and a multivariate mutual information model. Results of spatiotemporal analysis showed that five pollution stages were identified with remarkable variation patterns based on evolution of PM_2.5 concentration and weather conditions. Southern sites (DX, YDM and DS) experienced heavier pollution than northern ones (DL, CP and WL). Stage P2 exhibited combined functions of meteorology and traffic restrictions which were delayed peak-clipping effects on PM_2.5. Mutual information values of Air quality-Traffic-Meteorology (ATM-MI) revealed that additive functions of traffic restrictions, suitable relative humidity and temperature were more effective on the removal of fine particles and CO than NO₂.

Relevance popularity: A term event model based feature selection scheme for text classification

Feature selection is a practical approach for improving the performance of text classification methods by optimizing the feature subsets input to classifiers. In traditional feature selection methods such as information gain and chi-square, the number of documents that contain a particular term (i.e. the document frequency) is often used. However, the frequency of a given term appearing in each document has not been fully investigated, even though it is a promising feature to produce accurate classifications. In this paper, we propose a new feature selection scheme based on a term event Multinomial naive Bayes probabilistic model. According to the model assumptions, the matching score function, which is based on the prediction probability ratio, can be factorized. Finally, we derive a feature selection measurement for each term after replacing inner parameters by their estimators. On a benchmark English text datasets (20 Newsgroups) and a Chinese text dataset (MPH-20), our numerical experiment results obtained from using two widely used text classifiers (naive Bayes and support vector machine) demonstrate that our method outperformed the representative feature selection methods.