Variable selection in wavelet regression models

Feature selection using distributions of orthogonal PLS regression vectors in spectral data

Feature selection, which is important for successful analysis of chemometric data, aims to produce parsimonious and predictive models. Partial least squares (PLS) regression is one of the main methods in chemometrics for analyzing multivariate data with input X and response Y by modeling the covariance structure in the X and Y spaces. Recently, orthogonal projections to latent structures (OPLS) has been widely used in processing multivariate data because OPLS improves the interpretability of PLS models by removing systematic variation in the X space not correlated to Y. The purpose of this paper is to present a feature selection method of multivariate data through orthogonal PLS regression (OPLSR), which combines orthogonal signal correction with PLS. The presented method generates empirical distributions of features effects upon Y in OPLSR vectors via permutation tests and examines the significance of the effects of the input features on Y. We show the performance of the proposed method using a simulation study in which a three-layer network structure exists in compared with the false discovery rate method. To demonstrate this method, we apply it to both real-life NIR spectra data and mass spectrometry data.

Multiscale and Multi-Granularity Process Analytics: A Review

As Industry 4.0 makes its course into the Chemical Processing Industry (CPI), new challenges emerge that require an adaptation of the Process Analytics toolkit. In particular, two recurring classes of problems arise, motivated by the growing complexity of systems on one hand, and increasing data throughput (i.e., the product of two well-known “V’s” from Big Data: Volume × Velocity) on the other. More specifically, as enabling IT technologies (IoT, smart sensors, etc.) enlarge the focus of analysis from the unit level to the entire plant or even to the supply chain level, the existence of relevant dynamics at multiple scales becomes a common pattern; therefore, multiscale methods are called for and must be applied in order to avoid biased analysis towards a certain scale, compromising the benefits from the balanced exploitation of the information content at all scales. Also, these same enabling technologies currently collect large volumes of data at high-sampling rates, creating a flood of digital information that needs to be properly handled; optimal data aggregation provides an efficient solution to this challenge, leading to the emergence of multi-granularity frameworks. In this article, an overview is presented on multiscale and multi-granularity methods that are likely to play an important role in the future of Process Analytics with respect to several common activities, such as data integration/fusion, de-noising, process monitoring and predictive modelling, among others.

Variable selection in discriminant partial least-squares analysis

Variable selection enhances the understanding and interpretability of multivariate classification models. A new chemometric method based on the selection of the most important variables in discriminant partial least-squares (VS-DPLS) analysis is described. The suggested method is a simple extension of DPLS where a small number of elements in the weight vector w is retained for each factor. The optimal number of DPLS factors is determined by cross-validation. The new algorithm is applied to four different high-dimensional spectral data sets with excellent results. Spectral profiles from Fourier transform infrared spectroscopy and pyrolysis mass spectrometry are used. To investigate the uniqueness of the selected variables an iterative VS-DPLS procedure is performed. At each iteration, the previously found selected variables are removed to see if a new VS-DPLS classification model can be constructed using a different set of variables. In this manner, it is possible to determine regions rather than individual variables that are important for a successful classification.

Does history repeat itself? Wavelets and the phylodynamics of influenza A

Unprecedented global surveillance of viruses will result in massive sequence data sets that require new statistical methods. These data sets press the limits of Bayesian phylogenetics as the high-dimensional parameters that comprise a phylogenetic tree increase the already sizable computational burden of these techniques. This burden often results in partitioning the data set, for example, by gene, and inferring the evolutionary dynamics of each partition independently, a compromise that results in stratified analyses that depend only on data within a given partition. However, parameter estimates inferred from these stratified models are likely strongly correlated, considering they rely on data from a single data set. To overcome this shortfall, we exploit the existing Monte Carlo realizations from stratified Bayesian analyses to efficiently estimate a nonparametric hierarchical wavelet-based model and learn about the time-varying parameters of effective population size that reflect levels of genetic diversity across all partitions simultaneously. Our methods are applied to complete genome influenza A sequences that span 13 years. We find that broad peaks and trends, as opposed to seasonal spikes, in the effective population size history distinguish individual segments from the complete genome. We also address hypotheses regarding intersegment dynamics within a formal statistical framework that accounts for correlation between segment-specific parameters.

Variable selection using iterative reformulation of training set models for discrimination of samples: application to gas chromatography/mass spectrometry of mouse urinary metabolites

The paper discusses variable selection as used in large metabolomic studies, exemplified by mouse urinary gas chromatography of 441 mice in three experiments to detect the influence of age, diet, and stress on their chemosignal. Partial least squares discriminant analysis (PLS-DA) was applied to obtain class models, using a procedure of 20,000 iterations including the bootstrap for model optimization and random splits into test and training sets for validation. Variables are selected using PLS regression coefficients on the training set using an optimized number of components obtained from the bootstrap. The variables are ranked in order of significance, and the overall optimal variables are selected as those that appear as highly significant over 100 different test and training set splits. Cost/benefit analysis of performing the model on a reduced number of variables is also illustrated. This paper provides a strategy for properly validated methods for determining which variables are most significant for discriminating between two groups in large metabolomic data sets avoiding the common pitfall of overfitting if variables are selected on a combined training and test set and also taking into account that different variables may be selected each time the samples are split into training and test sets using iterative procedures.

Variable interaction network based variable selection for multivariate calibration

Multivariate calibration problems often involve the identification of a meaningful subset of variables, from a vast number of variables for better prediction of output variables. A new graph theoretic method based on partial correlations (variable interaction network-VIN) is proposed. Many well studied representative calibration datasets spanning different application domains are selected for investigating the performance. Partial least squares (PLS) regression models combined with variable selection techniques are employed for benchmarking the performance. Subsets of variables with different number of variables are retained for the final analysis after VIN selection and progressive prediction accuracies are used for comparison. VIN-PLS results show significant improvement in prediction efficiencies and variable subset optimization. Improvement of up to 45% over existing methods with significantly fewer variables is achieved using the new method. Advantages of VIN based variable selection are highlighted.

A method for determination of COD in a domestic wastewater treatment plant by using near-infrared reflectance spectrometry of seston

This paper proposes a method for determination of chemical oxygen demand (COD) in domestic wastewater. The proposed method is based on near-infrared reflectance (NIRR) measurements of seston collected from wastewater samples by filtration. The analysis does not require any special reagent, catalyst or solvent. Inherent baseline and noise features present in NIRR spectra are removed by a Savitzky-Golay derivative procedure followed by wavelet denoising. The resulting wavelet approximation coefficients are used for partial-least-squares modelling and subsequent prediction of COD values in new samples. The model is calibrated by using COD values obtained according to the American Public Health Association (APHA) reference method. The proposed method is applied to effluent samples from the anaerobic ponds of the Mangabeira municipal wastewater treatment plant in the city of João Pessoa (Paraíba, Brazil). By comparing the NIRR prediction results with the APHA reference values, a root-mean-square error of prediction (RMSEP) of 19 mg O2 L(-1) and a correlation of 0.97 were obtained. Such results are deemed adequate in view of the joint estimate of the standard error of the reference method, which was calculated as 21 mg O2 L(-1).

Screening analysis of river seston downstream of an effluent discharge point using near-infrared reflectance spectrometry and wavelet-based spectral region selection

A methodology for screening analysis of river seston downstream of an industry effluent by using near-infrared reflectance spectrometry was developed. A wavelet transform (WT)-based strategy is used to select a spectral region in which the effect of the effluent on the optical properties of the seston is more evident. The methodology was applied to samples from the River Mumbaba in northeast Brazil. Four sites were monitored: two upstream (1 and 2), one at the discharge point of the effluent (3), and another downstream (4). Soft Independent Modelling of Class Analogies (SIMCA) models were built for site 1 and were then applied to the classification of samples from sites 2 and 4. The results reveal that the WT-based spectral region selection is essential to ensure good sensitivity and specificity with respect to the detection of events associated to the effluent discharges at site 3. In fact, the changes in site 4 caused by the effluent are masked by other environmental factors when the full spectrum is employed.

Wavelet multiscale regression from the perspective of data fusion: new conceptual approaches

Wavelet regression is a very promising technique for modern multivariate calibration and calibration transfer. Multiscale analysis of wavelet scales provides a connection between wavelet regression and data fusion. In this paper, current wavelet regression methods are reviewed from the novel perspective of data fusion. Illustrated by analysis of a public domain near-infrared dataset, the advantages and drawbacks of these methods are examined. For wavelet regression, the non-uniformity of the wavelet components, the multiscale nature of the signal, and the prevention of information leakage are crucial issues that will be addressed.

Maximum likelihood principal components regression on wavelet-compressed data

Maximum likelihood principal component regression (MLPCR) is an errors-in-variables method used to accommodate measurement error information when building multivariate calibration models. A hindrance of MLPCR has been the substantial demand on computational resources sometimes made by the algorithm, especially for certain types of error structures. Operations on these large matrices are memory intensive and time consuming, especially when techniques such as cross-validation are used. This work describes the use of wavelet transforms (WT) as a data compression method for MLPCR. It is shown that the error covariance matrix in the wavelet and spectral domains are related through a two-dimensional WT. This allows the user to account for any effects of the wavelet transform on spectral and error structures. The wavelet transform can be applied to MLPCR when using either the full error covariance matrix or the smaller pooled error covariance matrix. Simulated and experimental near-infrared data sets are used to demonstrate the benefits of using wavelets with the MLPCR algorithm. In all cases, significant compression can be obtained while maintaining favorable predictive ability. Considerable time savings were also attained, with improvements ranging from a factor of 2 to a factor of 720. Using the WT-compressed data in MLPCR gave a reduction in prediction errors compared to using the raw data in MLPCR. An analogous reduction in prediction errors was not always seen when using PCR.

Simultaneous Spectrophotometric Determination of p-Benzoquinone and Chloranil after Microcrystalline Naphthalene Extraction by Using Genetic Algorithm-Based Wavelength Selection-Partial Least Squares Regression

Microcrystalline naphthalene extraction has been used for the preconcentration of p-benzoquinone and tetrachloro-p-benzoquinone (chloranil), after their reaction by aniline, and later simultaneous spectrophotometric analysis by genetic algorithm-partial least squares (GA-PLS) calibration. The chemical variables affecting the analytical performance of the methodology were studied and optimized. Under the optimum conditions i.e., [aniline] = 0.05 M and [naphthalene] = 2.2% (w/v), preconcentration of 25 ml of sample solution permitted the detection of 0.32 and 0.23 microg ml(-1) for p-benzoquinone and chloranil, respectively. The predictive abilities of partial least squares regression (PLS) and genetic algorithm-partial least squares regression (GA-PLS) were examined for simultaneous determination of two quinones. The GA-PLS shows superiority over other PLS methods due to the wavelength selection in PLS calibration using a genetic algorithm without loss of prediction capacity, provides useful information about the chemical system.

A Multiscale Wavelet Data Treatment for Reliable Localization of Inflection Points for Analytical Purposes

Instrumental analysis techniques that employ measurements based on inflection points may have their accuracy compromised due to the need for signal differentiation, which is very sensitive to instrumental noise. This paper presents a strategy for localizing inflection points that exploits the multiscale processing capability of the Wavelet Transform and avoids the need for explicit signal differentiation. The strategy is illustrated in simulated examples and also in a real analytical problem involving the determination of Pb and Cd by potentiometric stripping analysis. In this application, the results were in good agreement with the expected values and were slightly better than those obtained from the first derivative of the curves after smoothing by a Windowed Fourier Transform.

Haar transform analysis of photon time-of-flight measurements for quantification of optical properties in scattering media

A method to independently quantify the absorption and the scattering properties of samples based on the analysis of the Haar transform (HT) of photon time-of-flight (TOF) distributions is described. A series of reflectance photon TOF measurements were acquired from absorbing/scattering milk samples of known composition (0 < mu(a) < 0.025 mm(-1); 100 < mu(s) < 250 mm(-1)). The HT of the profiles was calculated, and the regression based on the most parsimonious subset of wavelets was determined by the genetic algorithm (GA). In addition, the utility of computing the logarithm of the profiles or of the absolute value of the wavelet coefficients before the GA was studied. Results show that the absorption coefficient could be estimated with a coefficient of variation (C.V.) of 6.7% and an r2 of 0.99 by use of the log of selected wavelets of frequency less than 800 MHz. Scattering coefficients were estimated with a C.V. of 2.3% and an r2 of 0.99 with the log of wavelets of frequency less than 400 MHz. The above results suggest that a simplified instrument based on low-frequency switches could be developed to quantify the optical properties of highly scattering media.

A linear semi-infinite programming strategy for constructing optimal wavelet transforms in multivariate calibration problems

A novel strategy for the optimization of wavelet transforms with respect to the statistics of the data set in multivariate calibration problems is proposed. The optimization follows a linear semi-infinite programming formulation, which does not display local maxima problems and can be reproducibly solved with modest computational effort. After the optimization, a variable selection algorithm is employed to choose a subset of wavelet coefficients with minimal collinearity. The selection allows the building of a calibration model by direct multiple linear regression on the wavelet coefficients. In an illustrative application involving the simultaneous determination of Mn, Mo, Cr, Ni, and Fe in steel samples by ICP-AES, the proposed strategy yielded more accurate predictions than PCR, PLS, and nonoptimized wavelet regression.

Use of the statistical properties of the wavelet-transform coefficients for optimization of integration time in Fourier transform spectrometry

We show that an analysis of the mean and variance of discrete wavelet coefficients of coaveraged time-domain interferograms can be used as a specification for determining when to stop coaveraging. We also show that, if a prediction model built in the wavelet domain is used to determine the composition of unknown samples, a stopping criterion for the coaveraging process can be developed with respect to the uncertainty tolerated in the prediction.

Spike removal and denoising of Raman spectra by wavelet transform methods

Wavelet decompositions of Raman spectra were investigated with respect to their usability for spike removal and denoising of the raw data. It could be shown that those operations should be performed sequentially. Suppression of spikes is not straightforwardly possible by wavelet transformation; however, the wavelet transform may be used to recognize the spikes by their first level detail coefficients. Spike locations could be projected from the details to the approximations and, further, to appropriate locations of the original spectrum. After spike recognition, those regions will be replaced by interpolated values. To complete processing, denoising is performed with the despiked spectrum by repeated application of wavelet transform methods.

Multivariate curve resolution of wavelet and Fourier compressed spectra

The multivariate curve resolution method SIMPLe to use Interactive Self-Modeling Mixture Analysis (SIMPLISMA) was applied to Fourier and wavelet compressed ion-mobility spectra. The spectra obtained from the SIMPLISMA model were transformed back to their original representation, that is, uncompressed format. SIMPULSMA was able to model the same pure variables for the partial wavelet transform, although for the Fourier and complete wavelet transforms, satisfactory pure variables and models were not obtained. Data were acquired from two samples and two different ion mobility spectrometry (IMS) sensors. The first sample was thermally desorbed sodium gamma-hydroxybutyrate (GHB), and the second sample was a liquid mixture of dicyclohexylamine (DCHA) and diethylmethylphosphonate (DEMP). The spectra were compressed to 6.3% of their original size. SIMPLISMA was applied to the compressed data in the Fourier and wavelet domains. An alternative method of normalizing SIMPLISMA spectra was devised that removes variation in scale between SIMPLISMA results obtained from uncompressed and compressed data. SIMPLISMA was able to accurately extract the spectral features and concentration profiles directly from daublet compressed IMS data at a compression ratio of 93.7% with root-mean-square errors of reconstruction < 3%. The daublet wavelet filters were selected, because they worked well when compared to coiflet and symmlet. The effects of the daublet filter width and compression ratio were evaluated with respect to reconstruction errors of the data sets and SIMPLISMA spectra. For these experiments, the daublet 14 filter performed well for the two data sets.

Multivariate statistical monitoring of batch processes: an industrial case study of fermentation supervision

This article describes the development of Multivariate Statistical Process Control (MSPC) procedures for monitoring batch processes and demonstrates its application with respect to industrial tylosin biosynthesis. Currently, the main fermentation phase is monitored using univariate statistical process control principles implemented within the G2 real-time expert system package. This development addresses integrating various process stages into a monitoring system and observing interactions among individual variables through the use of multivariate projection methods. The benefits of this approach will be discussed from an industrial perspective.