Variable selection in classification of environmental soil samples for partial least square and neural network models

Artificial neural network modeling in environmental radioactivity studies - A review

The development of nuclear technologies has directed environmental radioactivity research toward continuously improving existing and developing new models for different interpolation, optimization, and classification tasks. Due to their adaptability to new data without knowing the actual modeling function, artificial neural networks (ANNs) are extensively used to resolve the tasks for which the application of traditional statistical methods has not provided an adequate response. This study presents an overview of ANN-based modeling in environmental radioactivity studies, including identifying and quantifying radionuclides, predicting their migration in the environment, mapping their distribution, optimizing measurement methodologies, monitoring processes in nuclear plants, and real-time data analysis. Special attention is paid to highlighting the scope of the different case studies and discussing the techniques used in model development over time. The performances of ANNs are evaluated from the perspective of prediction accuracy, emphasizing the advantages and limitations encountered in their use. The most critical elements in model optimization were identified as network structure, selection of input parameters, the properties of input data set, and applied learning algorithm. The analysis of strategies and methods for improving the performance of ANNs has shown that developing integrated and hybrid artificial intelligent tools could provide a new path in environmental radioactivity modeling toward more reliable outcomes and higher accuracy predictions. The review highlights the potential of neural networks and challenges in their application in environmental radioactivity studies and proposes directions for future research.

Artificial neural networks combined multi-wavelength transmission spectrum feature extraction for sensitive identification of waterborne bacteria

Present research is focused on the rapid and accurate identification of bacterial species based on artificial neural networks combined with spectral data processing technology. The spectra of different bacterial species in the logarithmic growth phase were obtained. Model input features were extracted from the raw spectra using signal processing techniques, including normalization, principal component analysis (PCA) and area-based feature value extraction. The identification models based on artificial neural network of back propagation neural networks (BPNN), generalized regression neural networks (GRNN) and probabilistic neural networks (PNN) were developed using the extracted features in order to ascertain whether the different species of bacteria could be differentiated. The performance of developed models and its corresponding signal processing techniques is tested by the recognition accuracy of validation set and test set, and model error. The maximum recognition accuracy of normalized spectrum combined with BPNN was 95.5% (error: 10%, test accuracy: 100%). The total recognition accuracy of PCA-reduced features (200-400 nm) combined with GRNN resulted in 96.3%~96.8% (error: 3.3%~6.7%, test accuracy: 97.5%~100%). While the overall recognition accuracy of area-based features combined with GRNN reached 97.3% with test accuracy of 100% (model error: 5.0%). Choosing of model and signal processing techniques has a positive influence on improving classification accuracy, so as to make it possible to realize the rapid detection and online monitoring of waterborne microbial contamination.

Tools based on multivariate statistical analysis for classification of soil and groundwater in Apulian agricultural sites

In this paper, the results obtained from multivariate statistical techniques such as PCA (Principal component analysis) and LDA (Linear discriminant analysis) applied to a wide soil data set are presented. The results have been compared with those obtained on a groundwater data set, whose samples were collected together with soil ones, within the project "Improvement of the Regional Agro-meteorological Monitoring Network (2004-2007)". LDA, applied to soil data, has allowed to distinguish the geographical origin of the sample from either one of the two macroaeras: Bari and Foggia provinces vs Brindisi, Lecce e Taranto provinces, with a percentage of correct prediction in cross validation of 87%. In the case of the groundwater data set, the best classification was obtained when the samples were grouped into three macroareas: Foggia province, Bari province and Brindisi, Lecce and Taranto provinces, by reaching a percentage of correct predictions in cross validation of 84%. The obtained information can be very useful in supporting soil and water resource management, such as the reduction of water consumption and the reduction of energy and chemical (nutrients and pesticides) inputs in agriculture.

Soil type recognition as improved by genetic algorithm-based variable selection using near infrared spectroscopy and partial least squares discriminant analysis

Soil types have traditionally been determined by soil physical and chemical properties, diagnostic horizons and pedogenic processes based on a given classification system. This is a laborious and time consuming process. Near infrared (NIR) spectroscopy can comprehensively characterize soil properties, and may provide a viable alternative method for soil type recognition. Here, we presented a partial least squares discriminant analysis (PLSDA) method based on the NIR spectra for the accurate recognition of the types of 230 soil samples collected from farmland topsoils (0-10 cm), representing 5 different soil classes (Albic Luvisols, Haplic Luvisols, Chernozems, Eutric Cambisols and Phaeozems) in northeast China. We found that the PLSDA had an internal validation accuracy of 89% and external validation accuracy of 83% on average, while variable selection with the genetic algorithm (GA and GA-PLSDA) improved this to 92% and 93%. Our results indicate that the GA variable selection technique can significantly improve the accuracy rate of soil type recognition using NIR spectroscopy, suggesting that the proposed methodology is a promising alternative for recognizing soil types using NIR spectroscopy.

Predictive-property-ranked variable reduction with final complexity adapted models in partial least squares modeling for multiple responses

For partial least-squares regression with one response (PLS1), many variable-reduction methods have been developed. However, only a few address the case of multiple-response partial-least-squares (PLS2) modeling. The calibration performance of PLS1 can be improved by elimination of uninformative variables. Many variable-reduction methods are based on various PLS-model-related parameters, called predictor-variable properties. Recently, an important adaptation, in which the model complexity is optimized, was introduced in these methods. This method was called Predictive-Property-Ranked Variable Reduction with Final Complexity Adapted Models, denoted as PPRVR-FCAM or simply FCAM. In this study, variable reduction for PLS2 models, using an adapted FCAM method, FCAM-PLS2, is investigated. The utility and effectiveness of four new predictor-variable properties, derived from the multiple response PLS2 regression coefficients, are studied for six data sets consisting of ultraviolet-visible (UV-vis) spectra, near-infrared (NIR) spectra, NMR spectra, and two simulated sets, one with correlated and one with uncorrelated responses. The four properties include the mean of the absolute values as well as the norm of the PLS2 regression coefficients and their significances. The four properties were found to be applicable by the FCAM-PLS2 method for variable reduction. The predictive abilities of models resulting from the four properties are similar. The norm of the PLS2 regression coefficients has the best selective abilities, low numbers of variables with an informative meaning to the responses are retained. The significance of the mean of the PLS2 regression coefficients is found to be the least-selective property.

A Partial Least Squares based algorithm for parsimonious variable selection

Background

In genomics, a commonly encountered problem is to extract a subset of variables out of a large set of explanatory variables associated with one or several quantitative or qualitative response variables. An example is to identify associations between codon-usage and phylogeny based definitions of taxonomic groups at different taxonomic levels. Maximum understandability with the smallest number of selected variables, consistency of the selected variables, as well as variation of model performance on test data, are issues to be addressed for such problems.

Results

We present an algorithm balancing the parsimony and the predictive performance of a model. The algorithm is based on variable selection using reduced-rank Partial Least Squares with a regularized elimination. Allowing a marginal decrease in model performance results in a substantial decrease in the number of selected variables. This significantly improves the understandability of the model. Within the approach we have tested and compared three different criteria commonly used in the Partial Least Square modeling paradigm for variable selection; loading weights, regression coefficients and variable importance on projections. The algorithm is applied to a problem of identifying codon variations discriminating different bacterial taxa, which is of particular interest in classifying metagenomics samples. The results are compared with a classical forward selection algorithm, the much used Lasso algorithm as well as Soft-threshold Partial Least Squares variable selection.

Conclusions

A regularized elimination algorithm based on Partial Least Squares produces results that increase understandability and consistency and reduces the classification error on test data compared to standard approaches.

Counter propagation artificial neural network categorical models for prediction of carcinogenicity for non-congeneric chemicals

One of the main goals of the new chemical regulation REACH (Registration, Evaluation and Authorization of Chemicals) is to fill the gaps on the toxicological properties of chemicals that affect human health. Carcinogenicity is one of the endpoints under consideration. The information obtained from (quantitative) structure-activity relationship ((Q)SAR) models is accepted as an alternative solution to avoid expensive and time-consuming animal tests. The reported results were obtained within the framework of the European project 'Computer Assisted Evaluation of industrial chemical Substances According to Regulations (CAESAR)'. In this article, we demonstrate intermediate results for counter propagation artificial neural network (CP ANN) models for the prediction category of the carcinogenic potency using two-dimensional (2D) descriptors from different software programs. A total of 805 non-congeneric chemicals were extracted from the Carcinogenic Potency Database (CPDBAS). The resulting models had prediction accuracies for internal (training) and external (test) sets as high as 91-93% and 68-70%, respectively. The sensitivity and specificity of the test set were 69-73 and 63-72% correspondingly. High specificity is critical in models for regulatory use that are aimed at ensuring public safety. Thus, the errors that give rise to false negatives are much more relevant. We discuss how we can increase the number of correctly predicted carcinogens using the correlation between the threshold and the values of the sensitivity and specificity.

Variable selection using iterative reformulation of training set models for discrimination of samples: application to gas chromatography/mass spectrometry of mouse urinary metabolites

The paper discusses variable selection as used in large metabolomic studies, exemplified by mouse urinary gas chromatography of 441 mice in three experiments to detect the influence of age, diet, and stress on their chemosignal. Partial least squares discriminant analysis (PLS-DA) was applied to obtain class models, using a procedure of 20,000 iterations including the bootstrap for model optimization and random splits into test and training sets for validation. Variables are selected using PLS regression coefficients on the training set using an optimized number of components obtained from the bootstrap. The variables are ranked in order of significance, and the overall optimal variables are selected as those that appear as highly significant over 100 different test and training set splits. Cost/benefit analysis of performing the model on a reduced number of variables is also illustrated. This paper provides a strategy for properly validated methods for determining which variables are most significant for discriminating between two groups in large metabolomic data sets avoiding the common pitfall of overfitting if variables are selected on a combined training and test set and also taking into account that different variables may be selected each time the samples are split into training and test sets using iterative procedures.

Chemically driven variable selection by focused multimodal genetic algorithms in mid-IR spectra

Four genetic-algorithm-based approaches to variable selection in spectral data sets are presented. They range from a pure black-box approach to a chemically driven one. The latter uses a fitness function that takes into account not only typical parameters like the number of errors when classifying a training set but also the chemical interpretability of the selected variables. In order to cope with the fact that multiple solutions may be acceptable, a multimodal genetic algorithm (GA) is employed and the most satisfactory solution selected. The multimodal GA uses two populations (denominated "hybrid two populations" GA or HTP-GA): a classical population, from which potential solutions emerge, and a new population, which maintains diversity in the search space (as required by multimodal problems). Results show that the HTP-GA approach improves the chemical understanding of the selected solution (compared to other GA approaches) and that the classification capabilities of the approach are still good. All of the GA strategies for variable selection were compared with a classical parametric technique, Procrustes rotation, which does not consider interpretability.

Classification of soil samples according to geographic origin using gamma-ray spectrometry and pattern recognition methods

Multivariate data analysis methods were used to recognize and classify soils of unknown geographic origin. A total of 103 soil samples were differentiated into classes, according to regions in Serbia and Montenegro from which they were collected. Their radionuclide (226Ra, 238U, 235U, 40K, 134Cs, 137Cs, 232Th and 7Be) activities detected by gamma-ray spectrometry were then used as the inputs in different pattern recognition methods. For the classification of soil samples using eight selected radionuclides, the prediction ability of linear discriminant analysis (LDA), k-nearest neighbours (kNN), soft independent modelling of class analogy (SIMCA) and artificial neural network (ANN) were 82.8%, 88.6%, 60.0% and 92.1%, respectively.

Classification of soil samples according to their geographic origin using gamma-ray spectrometry and principal component analysis

A principal component analysis (PCA) was used for classification of soil samples from different locations in Serbia and Montenegro. Based on activities of radionuclides ((226)Ra, (238)U, (235)U, (40)K, (134)Cs, (137)Cs, (232)Th and (7)Be) detected by gamma-ray spectrometry, the classification of soils according to their geographical origin was performed. Application of PCA to our experimental data resulted in satisfactory classification rate (86.0% correctly classified samples). The obtained results indicate that gamma-ray spectrometry in conjunction with PCA is a viable tool for soil classification.

VSMP: a novel variable selection and modeling method based on the prediction

The use of numerous descriptors that are indicative of molecular structure and topology is becoming more common in quantitative structure-activity relationship (QSAR). How to choose the adequate descriptors for QSAR studies is important but difficult because there are no absolute rules to govern this choice. A variety of variable selection techniques including stepwise, partial least squares/principal component analysis (PLS/PCA), neural network, and evolutionary algorithm such as genetic algorithm have been applied to this common problem. All-subsets regression (ASR) is capable of finding out the best variable subset from among a large pool. In this paper, a novel variable selection and modeling method based on the prediction, for short VSMP, has been developed. Here two controllable parameters, the interrelation coefficient between the pairs of the independent variables (r(int)) and the correlation coefficient (q(2)) obtained using the leave-one-out (LOO) cross-validation technique, are introduced into the ASR to improve its performances. This technique differs from the other variable selection procedures related to the ASR by two main features: (1) The search of various optimal subset search is controlled by the statistic q(2) or root-mean-square error (RMSEP) in the LOO cross-validation step rather than the correlation coefficient obtained in the modeling step (r(2)). (2) The searching speed of all optimal subsets is expedited by the statistic r(int) together with q(2). A comparison of the results of the VSMP applied to the Selwood data set (n = 31 compounds, m = 53 descriptors) with those obtained from alternative algorithms shows the good performance of the technique.