Considering diversity and accuracy simultaneously for ensemble pruning

Uncertainty-aware selecting for an ensemble of deep food recognition models

Deep learning is a machine learning technique that has revolutionized the research community due to its impressive results on various real-life problems. Recently, ensembles of Convolutional Neural Networks (CNN) have proven to achieve high robustness and accuracy in numerous computer vision challenges. As expected, the more models we add to the ensemble, the better performance we can obtain, but, in contrast, more computer resources are needed. Hence, the importance of deciding how many models to use and which models to select from a pool of trained models is huge. From the latter, a common strategy in deep learning is to select the models randomly or according to the results on the validation set. However, in this way models are chosen based on individual performance ignoring how they are expected to work together. Alternatively, to ensure a better complement between models, an exhaustive search can be used by evaluating the performance of several ensemble models based on different numbers and combinations of trained models. Nevertheless, this may result in being high computationally expensive. Considering that epistemic uncertainty analysis has recently been successfully employed to understand model learning, we aim to analyze whether an uncertainty-aware epistemic method can help us decide which groups of CNN models may work best. The method was validated on several food datasets and with different CNN architectures. In most cases, our proposal outperforms the results by a statistically significant range with respect to the baseline techniques and is much less computationally expensive compared to the brute-force search.

R-HEFS: Rough set based heterogeneous ensemble feature selection method for medical data classification

Feature selection is one of the trustworthy processes of dimensionality reduction technique to select a subset of relevant and non-redundant features from large datasets. Ensemble feature selection (EFS) approach is a recent technique aiming at accumulating diversity in the subset of selected features. It improves the performance of learning algorithms and obtains more stable and robust results. In this paper, a novel rough set theory (RST) based heterogeneous EFS method (R-HEFS) is proposed for selecting the less redundant and highly relevant features during the aggregation of diverse feature subsets by applying the feature-class, feature-feature rough dependency and feature-significance measures. In R-HEFS five state-of-the-art RST based filter methods are used as a base feature selectors. Experiments are carried out on 10 benchmark medical datasets collected from the UCI repository. For the imputation of the missing values and discretization of the continuous features, k nearest neighbor (kNN) imputation method and RST based discretization techniques are applied. The effectiveness of the proposed R-HEFS method is evaluated and analyzed by using four benchmark classifiers viz., Naïve Bayes (NB), random forest (RF), support vector machine (SVM), and AdaBoost. The proposed R-HEFS method turns out to be effective by removing the non-relevant and redundant features during the process of aggregation of base feature selectors and it assists to increase the classification accuracy. Out of 10 different medical datasets, on 7 datasets, R-HEFS has achieved better average classification accuracy. So, the overall results strongly suggest that the proposed R-HEFS method can reduce the dimension of large medical datasets and may help the physicians or medical experts to diagnose (classify) different diseases with lesser computational complexities.

A novel ensemble pruning approach based on information exchange glowworm swarm optimization and complementarity measure

Ensemble pruning has been widely used for enhancing classification ability employing a smaller number of classifiers. Ensemble pruning extracts a part of classifiers with good overall performance to form the final ensemble. Diversity and accuracy of classifiers are of vital importance for a successful ensemble. It is hard for the members in one ensemble to achieve both good diversity and high accuracy, simultaneously, because there is a tradeoff between them. Existing works usually search for the tradeoff in terms of diversity measures, or find it utilizing heuristic algorithms, which cannot gain the exact solution without exhaustive search. To address the above issue, a novel ensemble pruning method based on information exchange glowworm swarm optimization and complementarity measure, abbreviated EPIECM, is proposed using the combination of information exchange glowworm swarm optimization (IEGSO) and complementarity measure (COM). Firstly, multiple generated classifiers are utilized to construct a pool of learners who perform diversely. Secondly, COM is employed to pre-prune the classifiers with poor comprehensive performance, and the pre-pruned ensemble is formed utilizing the retaining classifiers. Finally, the optimal subset of classifiers is combined from the remaining constituents after pre-pruning with IEGSO. Empirical results on 27 UCI datasets indicate that EPIECM significantly outperforms other techniques.

A Novel Framework with High Diagnostic Sensitivity for Lung Cancer Detection by Electronic Nose

The electronic nose (e-nose) system is a newly developing detection technology for its advantages of non-invasiveness, simple operation, and low cost. However, lung cancer screening through e-nose requires effective pattern recognition frameworks. Existing frameworks rely heavily on hand-crafted features and have relatively low diagnostic sensitivity. To handle these problems, gated recurrent unit based autoencoder (GRU-AE) is adopted to automatically extract features from temporal and high-dimensional e-nose data. Moreover, we propose a novel margin and sensitivity based ordering ensemble pruning (MSEP) model for effective classification. The proposed heuristic model aims to reduce missed diagnosis rate of lung cancer patients while maintaining a high rate of overall identification. In the experiments, five state-of-the-art classification models and two popular dimensionality reduction methods were involved for comparison to demonstrate the validity of the proposed GRU-AE-MSEP framework, through 214 collected breath samples measured by e-nose. Experimental results indicated that the proposed intelligent framework achieved high sensitivity of 94.22%, accuracy of 93.55%, and specificity of 92.80%, thereby providing a new practical means for wide disease screening by e-nose in medical scenarios.

A Semi-Boosted Nested Model With Sensitivity-Based Weighted Binarization for Multi-Domain Network Intrusion Detection

Effective network intrusion detection techniques are required to thwart evolving cybersecurity threats. Historically, traditional enterprise networks have been researched extensively in this regard. However, the cyber threat landscape has grown to include wireless networks. In this article, the authors present a novel model that can be trained on completely different feature sets and applied to two distinct intrusion detection applications: traditional enterprise networks and 802.11 wireless networks. This is the first method that demonstrates superior performance in both aforementioned applications. The model is based on a one-versus-all binary framework comprising multiple nested sub-ensembles. To provide good generalization ability, each sub-ensemble contains a collection of sub-learners, and only a portion of the sub-learners implement boosting. A class weight based on the sensitivity metric (true-positive rate), learned from the training data only, is assigned to the sub-ensembles of each class. The use of pruning to remove sub-learners that do not contribute to or have an adverse effect on overall system performance is investigated as well. The results demonstrate that the proposed system can achieve exceptional performance in applications to both traditional enterprise intrusion detection and 802.11 wireless intrusion detection.