Effective feature selection scheme using mutual information

A Novel Variable Selection Method Based on Binning-Normalized Mutual Information for Multivariate Calibration

Variable (wavelength) selection is essential in the multivariate analysis of near-infrared spectra to improve model performance and provide a more straightforward interpretation. This paper proposed a new variable selection method named binning-normalized mutual information (B-NMI) based on information entropy theory. "Data binning" was applied to reduce the effects of minor measurement errors and increase the features of near-infrared spectra. "Normalized mutual information" was employed to calculate the correlation between each wavelength and the reference values. The performance of B-NMI was evaluated by two experimental datasets (ideal ternary solvent mixture dataset, fluidized bed granulation dataset) and two public datasets (gasoline octane dataset, corn protein dataset). Compared with classic methods of backward and interval PLS (BIPLS), variable importance projection (VIP), correlation coefficient (CC), uninformative variables elimination (UVE), and competitive adaptive reweighted sampling (CARS), B-NMI not only selected the most featured wavelengths from the spectra of complex real-world samples but also improved the stability and robustness of variable selection results.

Some Applications of ANN to Solar Radiation Estimation and Forecasting for Energy Applications

In solar energy, the knowledge of solar radiation is very important for the integration of energy systems in building or electrical networks. Global horizontal irradiation (GHI) data are rarely measured over the world, thus an artificial neural network (ANN) model was built to calculate this data from more available ones. For the estimation of 5-min GHI, the normalized root mean square error (nRMSE) of the 6-inputs model is 19.35%. As solar collectors are often tilted, a second ANN model was developed to transform GHI into global tilted irradiation (GTI), a difficult task due to the anisotropy of scattering phenomena in the atmosphere. The GTI calculation from GHI was realized with an nRMSE around 8% for the optimal configuration. These two models estimate solar data at time, t, from other data measured at the same time, t. For an optimal management of energy, the development of forecasting tools is crucial because it allows anticipation of the production/consumption balance; thus, ANN models were developed to forecast hourly direct normal (DNI) and GHI irradiations for a time horizon from one hour (h+1) to six hours (h+6). The forecasting of hourly solar irradiation from h+1 to h+6 using ANN was realized with an nRMSE from 22.57% for h+1 to 34.85% for h+6 for GHI and from 38.23% for h+1 to 61.88% for h+6 for DNI.

New perspectives on multilocus ancestry informativeness

We present an axiomatic approach for multilocus informativeness measures for determining the amount of information that a set of polymorphic genetic markers provides about individual ancestry. We then reveal several surprising properties of a decision-theoretic based measure that is consistent with the set of proposed criteria for multilocus informativeness. In particular, these properties highlight the interplay between information originating from population priors and the information extractable from the population genetic variants. This analysis then reveals a certain deficiency of mutual information based multilocus informativeness measures when such population priors are incorporated. Finally, we analyse and quantify the inevitable inherent decrease in informativeness due to learning from finite population samples.

Optimization of a multilayer neural network by using minimal redundancy maximal relevance-partial mutual information clustering with least square regression

In this paper, an optimized multilayer feed-forward network (MLFN) is developed to construct a soft sensor for controlling naphtha dry point. To overcome the two main flaws in the structure and weight of MLFNs, which are trained by a back-propagation learning algorithm, minimal redundancy maximal relevance-partial mutual information clustering (mPMIc) integrated with least square regression (LSR) is proposed to optimize the MLFN. The mPMIc can determine the location of hidden layer nodes using information in the hidden and output layers, as well as remove redundant hidden layer nodes. These selected nodes are highly related to output data, but are minimally correlated with other hidden layer nodes. The weights between the selected hidden layer nodes and output layer are then updated through LSR. When the redundant nodes from the hidden layer are removed, the ideal MLFN structure can be obtained according to the test error results. In actual applications, the naphtha dry point must be controlled accurately because it strongly affects the production yield and the stability of subsequent operational processes. The mPMIc-LSR MLFN with a simple network size performs better than other improved MLFN variants and existing efficient models.

A novel feature selection method and its application

In this paper, a novel feature selection method based on rough sets and mutual information is proposed. The dependency of each feature guides the selection, and mutual information is employed to reduce the features which do not favor addition of dependency significantly. So the dependency of the subset found by our method reaches maximum with small number of features. Since our method evaluates both definitive relevance and uncertain relevance by a combined selection criterion of dependency and class-based distance metric, the feature subset is more relevant than other rough sets based methods. As a result, the subset is near optimal solution. In order to verify the contribution, eight different classification applications are employed. Our method is also employed on a real Alzheimer's disease dataset, and finds a feature subset where classification accuracy arrives at 81.3%. Those present results verify the contribution of our method.

Tree-structured feature extraction using mutual information

One of the most informative measures for feature extraction (FE) is mutual information (MI). In terms of MI, the optimal FE creates new features that jointly have the largest dependency on the target class. However, obtaining an accurate estimate of a high-dimensional MI as well as optimizing with respect to it is not always easy, especially when only small training sets are available. In this paper, we propose an efficient tree-based method for FE in which at each step a new feature is created by selecting and linearly combining two features such that the MI between the new feature and the class is maximized. Both the selection of the features to be combined and the estimation of the coefficients of the linear transform rely on estimating 2-D MIs. The estimation of the latter is computationally very efficient and robust. The effectiveness of our method is evaluated on several real-world data sets. The results show that the classification accuracy obtained by the proposed method is higher than that achieved by other FE methods.