Chemometric methods for extracting information from temperature-dependent near-infrared spectra

Reconstructing Randomly Masked Spectra Helps DNNs Identify Discriminant Wavenumbers

Nondestructive detection methods, based on vibrational spectroscopy, are vitally important in a wide range of applications including industrial chemistry, pharmacy and national defense. Recently, deep learning has been introduced into vibrational spectroscopy showing great potential. Different from images, text, etc. that offer large labeled data sets, vibrational spectroscopic data is very limited, which requires novel concepts beyond transfer and meta learning. To tackle this, we propose a task-enhanced augmentation network (TeaNet). The key component of TeaNet is a reconstruction module that inputs randomly masked spectra and outputs reconstructed samples that are similar to the original ones, but include additional variations learned from the domain. These augmented samples are used to train the classification model. The reconstruction and prediction parts are trained simultaneously, end-to-end with back-propagation. Results on both synthetic and real-world datasets verified the superiority of the proposed method. In the most difficult synthetic scenarios TeaNet outperformed CNN by 17%. We visualized and analysed the neuron responses of TeaNet and CNN, and found that TeaNet's ability to identify discriminant wavenumbers was excellent compared to CNN. Our approach is general and can be easily adapted to other domains, offering a solution to more accurate and interpretable few-shot learning.

Raman spectroscopy combined with partial least squares (PLS) based on hybrid spectral preprocessing and backward interval PLS (biPLS) for quantitative analysis of four PAHs in oil sludge

Polycyclic aromatic hydrocarbons (PAHs) contained in a large amount of oily sludge produced in petroleum and petrochemical production has become one of the main environmental protection concerns in the industry. The accurate determination of PAHs is of great significance in the field of petroleum geochemistry and environmental protection. In this study, Raman spectroscopy combined with partial least squares (PLS) based on different hybrid spectral preprocessing methods and variable selection strategies was proposed for quantitative analysis of phenanthrene, fluoranthrene, fluorene and naphthalene (Phe, Flt, Flu and Nap) in oil sludge. At first, PAHs in oily sludge was extracted by solid-liquid extraction with methanol as extractant, and Raman spectra of 21 oily sludge samples were collected by portable Raman spectrometer. And then, the influence of first derivative (D1st), wavelet transform (WT) and their hybrid spectral preprocessing on the predictive performance of the PLS calibration model was discussed. Thirdly, biPLS (backward interval partial least squares) was used to optimize the input variables before and after the hybrid spectral preprocessing methods, and the influence of biPLS and the hybrid spectral preprocessing sequence on the predictive performance of the PLS calibration model was discussed. Finally, the predictive performance of the PLS calibration model was optimized according to the results of leave-one-out cross-validation (LOOCV) method. The results show that the biPLS-D1st-WT-PLS calibration model established by using biPLS first to select the characteristic variables, followed by hybrid spectral preprocessing of the characteristic variables, has better prediction performance for Flt (determination coefficient of prediction (R2P) = 0.9987, and the mean relative error of prediction (MREP) = 0.0606). For Phe, Flu and Nap, the WT-biPLS-PLS calibration model has a better predictive effect (R2P are 0.9995, 0.9996 and 0.9983, and MREP are 0.0426, 0.0719 and 0.0497, respectively). In general, portable Raman spectroscopy combined with PLS calibration model based on different hybrid spectral preprocessing and variable selection strategies has achieved good prediction results for quantitative analysis of four PAHs in oily sludge. It is a new strategy to firstly select the characteristic variables of the original spectra, and secondly to preprocess the characteristic variables by the hybrid spectral preprocessing, which will provide a new idea for the establishment of quantitative analysis methods for PAHs in oily sludge.

Structures of water on the surface of anatase TiO2 studied by diffuse reflectance near-infrared spectroscopy

Investigating the structures of water on metal oxides is helpful for understanding the mechanism of the adsorptions in the presence of water. In this work, the structures of adsorbed water molecules on anatase TiO₂ (101) were studied by diffuse reflectance near-infrared spectroscopy (DR-NIRS). With resolution enhanced spectrum by continuous wavelet transform (CWT), the spectral features of adsorbed water at different sites were found. In the spectrum of dried TiO₂ powder, there is only the spectral feature of the water adsorbed at 5-coordinated titanium atoms (Ti_5c). With the increase of the adsorbed water, the spectral feature of the water at 2-coordinated oxygen atoms (O_2c) emerges first, and then that of the water interacting with the adsorbed water can be observed. When adenosine triphosphate (ATP) was adsorbed on TiO₂, the intensity of the peaks related to the adsorbed water decreases, indicating that the adsorbed water is replaced by ATP due to the strong affinity to Ti_5c. Therefore, there is a clear correlation between the peak intensity of the adsorbed water and the adsorbed quantity of ATP. Water can be a NIR spectroscopic probe to detect the quantity of the adsorbed ATP. A partial least squares (PLS) model was established to predict the content of adsorbed ATP by the spectral peaks of water. The recoveries of validation samples are in the range of 92.00-114.96% with the relative standard deviations (RSDs) in a range of 2.13-5.82%.

Mechanism of Binding of Polyproline to Ice via Interfacial Water: An Experimental and Theoretical Study

The molecular mechanism underlying inhibition of ice growth by polyproline (PPro), a minimal antifreeze glycoprotein mimic, remains unclear. In this work, the change in the structure of water during the growth of ice in PPro solutions was investigated using a combination of near-infrared spectroscopy and molecular dynamics (MD) simulations. The results show that only high concentrations of PPro solutions can effectively inhibit ice growth, as indicated by the variation in the spectral intensity of ice with time. When PPro exhibits an antifreeze effect, the spectral intensity of hydrated water associated with PPro in a solution is weakened. The experiments and MD simulations reveal that the quantity of the interfacial water between the ice crystal and the hydrophobic groups of PPro progressively reaches a plateau. Most significantly, we present clear evidence that the stable existence of this interfacial water is critical for the antifreeze activity of PPro.

Ultra-high resolution near-infrared spectrum by wavelet packet transform revealing the hydrogen bond interactions

Resolution is always an obstacle to analyzing the fine structure of a spectrum. The problem is particularly serious in the analysis of the near-infrared (NIR) spectra of aqueous solutions, because the spectrum is generally composed of overlapping broad peaks making the understanding of the structures and the interactions notoriously difficult. In this work, wavelet packet transform (WPT) was adopted to enhance the resolution of the NIR spectra of aqueous mixtures. Due to the microscopic ability of WPT in both position and frequency, the fine details of a spectrum can be observed in the spectral components of different frequencies obtained by WPT decomposition. Ultra-high resolution spectrum can be obtained from the high-frequency component representing the spectral features. Spectral features of different hydrogen-bonded OH, as well as the OH in HOH and HOD, were identified from the high-resolution NIR spectra of water and heavy water mixtures and validated by the variation of the spectral intensity with the mole ratio of H₂O and D₂O. The high-resolution spectrum was further applied in analyzing the interaction of amine and water. The spectral features of the hydrogen bonding between CH/NH in tert-butylamine (TBA) and OH in water were observed. The structures of CH bonded to one water molecule, and the structures of NH connecting with one and two water molecules were identified.

Aquaphotomics—Exploring Water Molecular Systems in Nature

Since its birth in 2005, when introduced by Prof [...]

Aquaphotomic Study of Effects of Different Mixing Waters on the Properties of Cement Mortar

The mixing water used for cement concrete has a significant effect on the physical properties of the material after hardening; however, other than the upper limit for the mixed impurities, not enough consideration has been given to the functions and characteristics of water at the molecular level. In this study, we investigated the effect of four different types of water (two spring-, mineral waters, tap water and distilled water) on the drying shrinkage of the hardened cement by comparing the material properties of the concrete specimens and analyzing the molecular structure of the water and cement mortar using aquaphotomics. The near infrared (NIR) spectra of waters used for mixing were acquired in the transmittance mode using a high-precision, high-accuracy benchtop spectrometer in the range of 400-2500 nm, with the 0.5 nm step. The NIR spectra of cement paste and mortar were measured in 6.2 nm increments in the wavelength range of 950 nm to 1650 nm using a portable spectrometer. The measurements of cement paste and mortar were performed on Day 0 (immediately after mixing, cement paste), 1 day, 3 days, 7 days, and 28 days after mixing (cement mortar). The spectral data were analyzed according to the aquaphotomics' multivariate analysis protocol, which involved exploration of raw and preprocessed spectra, exploratory analysis, discriminating analysis and aquagrams. The results of the aquaphotomics' analysis were interpreted together with the results of thermal and drying shrinkage measurements. Together, the findings clearly demonstrated that the thermal and drying shrinkage properties of the hardened cement material differed depending on the water used. Better mechanical properties were found to be a result of using mineral waters for cement mixing despite minute differences in the chemical content. In addition, the aquaphotomic characterization of the molecular structure of waters and cement mortar during the initial hydration reaction demonstrated the possibility to predict the characteristics of hardened cement at a very early stage. This provided the rationale to propose a novel evaluation method based on aquaphotomics for non-invasive evaluation and monitoring of cement mortar.

Water as a Probe for Standardization of Near-Infrared Spectra by Mutual-Individual Factor Analysis

The standardization of near-infrared (NIR) spectra is essential in practical applications, because various instruments are generally employed. However, standardization is challenging due to numerous perturbations, such as the instruments, testing environments, and sample compositions. In order to explain the spectral changes caused by the various perturbations, a two-step standardization technique was presented in this work called mutual–individual factor analysis (MIFA). Taking advantage of the sensitivity of a water probe to perturbations, the spectral information from a water spectral region was gradually divided into mutual and individual parts. With aquaphotomics expertise, it can be found that the mutual part described the overall spectral features among instruments, whereas the individual part depicted the difference of component structural changes in the sample caused by operation and the measurement conditions. Furthermore, the spectral difference was adjusted by the coefficients in both parts. The effectiveness of the method was assessed by using two NIR datasets of corn and wheat, respectively. The results showed that the standardized spectra can be successfully predicted by using the partial least squares (PLS) models developed with the spectra from the reference instrument. Consequently, the MIFA offers a viable solution to standardize the spectra obtained from several instruments when measurements are affected by multiple factors.

Analyzing the Water Confined in Hydrogel Using Near-Infrared Spectroscopy

Analysis of the confined water in hydrogels is essential for understanding the chemical and physical properties. Methods to quantify the content and study the structure of water in hydrogel using near-infrared (NIR) spectroscopy were proposed. The NIR spectra of poly-N,N-dimethylacrylamide (PDMAA) hydrogel with different water contents were measured at different temperatures. A partial least squares (PLS) model was established using the spectra of the samples with water content (w_h) from 0.9 to 387.6%. Continuous wavelet transform (CWT) was adopted to calculate the resolution enhanced spectra from which the spectral features of water species with free OH (S₀) and with one or two hydrogen bonds (S₁ and S₂) was obtained. The variation of these water species with water content suggests the existence of the water molecules bonding to NH groups by one hydrogen bond (S_1NH) and hydrating the CH groups of the polymer network and bulk-like water. Moreover, the variation of water structures with temperature shows that the release of bulk-like water occurs in the phase transition of the hydrogel, but the S_1NH and the hydration water stay unchanged. The former explains the sudden volume shrinkage for the phase transition and the latter may be the reason for the shape memory effect in the repeated swelling and deswelling of hydrogels.

Chemometrics: An Excavator in Temperature-Dependent Near-Infrared Spectroscopy

Temperature-dependent near-infrared (NIR) spectroscopy has been developed and taken as a powerful technique for analyzing the structure of water and the interactions in aqueous systems. Due to the overlapping of the peaks in NIR spectra, it is difficult to obtain the spectral features showing the structures and interactions. Chemometrics, therefore, is adopted to improve the spectral resolution and extract spectral information from the temperature-dependent NIR spectra for structural and quantitative analysis. In this review, works on chemometric studies for analyzing temperature-dependent NIR spectra were summarized. The temperature-induced spectral features of water structures can be extracted from the spectra with the help of chemometrics. Using the spectral variation of water with the temperature, the structural changes of small molecules, proteins, thermo-responsive polymers, and their interactions with water in aqueous solutions can be demonstrated. Furthermore, quantitative models between the spectra and the temperature or concentration can be established using the spectral variations of water and applied to determine the compositions in aqueous mixtures.

Water Spectral Patterns Reveals Similarities and Differences in Rice Germination and Induced Degenerated Callus Development

In vivo monitoring of rice (Oryza sativa L.) seed germination and seedling growth under general conditions in closed Petri dishes containing agar base medium at room temperature (temperature = 24.5 ± 1 °C, relative humidity = 76 ± 7% (average ± standard deviation)), and induced degenerated callus formation with plant growth regulator, were performed using short-wavelength near-infrared spectroscopy and aquaphotomics over A period of 26 days. The results of spectral analysis suggest changes in water absorbances due to the production of common metabolites, as well as increases in biomass and the sizes of the samples. Quantitative models built to predict the day of the development provided better accuracy for rice seedlings growth compared to callus formation. Eight common water bands were identified as presenting prominent changes in the absorbance pattern. The water matrix of only rice seedlings showed three developmental stages: firstly expressing a predominantly weakly hydrogen-bonded state, then a more strongly hydrogen-bonded state, and then, again, a weakly hydrogen-bonded state at the end. In rice callus induction and proliferation, no similar change in water absorbance pattern was observed. The presented findings indicate the potential of aquaphotomics for the in vivo detection of degeneration in cell development.

Quantitative analysis of polycyclic aromatic hydrocarbons in soil by infrared spectroscopy combined with hybrid variable selection strategy and partial least squares

Infrared spectroscopy (IR) combined with multivariate calibration technology can be used as a potential method to quantitative analysis of polycyclic aromatic hydrocarbons (PAHs) in soil, which provides a rapid data support for soil risk assessment. However, IR spectrum contains lots of useless information, its predictive performance is poor. Variable selection is an effective strategy to eliminate irrelevant wavelengths and enhance predictive performance. In this study, IR combined with partial least squares (PLS) was proposed to quantify anthracene and fluoranthene in soil. In order to improve the predictive performance of the PLS calibration model, the synergy interval PLS (siPLS) method was first used for "rough selection" to select feature bands; on this basis, "fine selection" was performed to extract the feature variables. In "fine selection", three different feature variables selection methods, such as successive projection algorithm (SPA), genetic algorithm (GA), and particle swarm optimization (PSO), were compared for their performance in extracting effective variables. The results show that the siPLS-GA calibration model receive a lowest root mean square error (RMSE) and a largest determination coefficient (R²). Results of external validation demonstrate an excellent predictive performance of siPLS-GA calibration model, with the R² = 0.9830, RMSE = 0.5897 mg/g and R² = 0.9849, RMSE = 0.4739 mg/g for anthracene and fluoranthene, respectively. In summary, siPLS combined with GA can accurately extract the effective information of the target substance and improve the predictive performance of the PLS calibration model based on IR spectroscopy.

Three-level simultaneous component analysis for analyzing the near-infrared spectra of aqueous solutions under multiple perturbations

Quantitative analysis under various perturbations is a difficult problem because the analytical signal changes with different factors. In this work, three-level simultaneous component analysis (3-MSCA) was used for analyzing the near-infrared (NIR) spectra of aqueous solutions under different perturbations. The spectral data of aqueous proline solutions at different pH, concentration and temperature were measured, and a three-level model was built to describe the effects of the three perturbations on the spectra, respectively. The first level model describes the change of the spectra with pH, from which significant aggregation of proline was observed around the isoelectric point. The second and third level model show the spectral change with concentration and temperature, respectively, and the spectral feature has a very good linear relationship with the corresponding influencing factors. Therefore, the pH and concentration scores can be used as the calibration curve for quantitative analysis of the pH and the content of proline, and the temperature scores can be used to predict the temperature of the solutions. In addition, the structural change of water molecules under different conditions is obtained from the loadings. A decline of the bulk water was found with the increase of concentration, implying an ascending trend of the bonded water due to the interaction of proline and water. The dissociation of water clusters with the increase of temperature is also displayed.

Interaction between tau and water during the induced aggregation revealed by near-infrared spectroscopy

Near-infrared (NIR) spectra are sensitive to the variation of water structure. In this work, NIR spectroscopy was used to investigate the variation of hydration water during the aggregation of R2/wt induced by heparin. The osmolytes, urea and trehalose, were used to slow down and speed up the aggregation. The spectra of R2/wt aqueous solution obtained by NIR spectroscopy at 37 °C were adopted to analyze the structure of water during the aggregation. The spectral features of different water species were observed by principal component analysis (PCA) from the resolution enhanced NIR spectra. The existence of the water molecules with one and two hydrogen bonds around the NH and CH groups of R2/wt, respectively, was suggested, and the variation of the hydrogen-bonded water was found to be an indicator to monitor the process of aggregation. Then, the variation of the water species during the aggregation was analyzed by two-dimensional correlation spectroscopy. The water hydrogen bonded with NH group was found to change earlier than the water around the hydrophobic groups. The results suggest that β-sheet forms though the hydrogen bonds of amide groups in the early stage of the aggregation, and the destruction of the hydrogen bond network of the water around the side chains maybe the main reason for the formation of the ordered amyloid fibers.

Understanding the complexity of the structures in alcohol solutions by temperature-dependent near-infrared spectroscopy

For understanding the structures and the hydrogen bonding in alcohol solutions, the changes of the structures and hydrogen bonding with temperature were studied by temperature-dependent near-infrared (NIR) spectroscopy. The spectral features of eight alcohol species including the monomer, dimer and linear or cyclic aggregates (trimer, tetramer and polymer) were found from the resolution-enhanced spectra calculated by continuous wavelet transform. The changes of the eight species with concentration and temperature were analyzed using the intensity variation of the corresponding spectral features and two-dimensional correlation NIR spectroscopy. The aggregates were found to form at a very low concentration and the stability of the seven aggregates with temperature was found in an order of cyclic tetramer > linear polymer > linear tetramer > cyclic trimer > linear trimer > cyclic polymer > dimer. Furthermore, the formation of the aggregates was found to be affected by the chain length. The increase of the chain length is beneficial for the formation of cyclic tetramer and polymer due to the hydrophobic effect, but is an adverse effect for the formation of linear polymer.

Aquaphotomics-From Innovative Knowledge to Integrative Platform in Science and Technology

Aquaphotomics is a young scientific discipline based on innovative knowledge of water molecular network, which as an intrinsic part of every aqueous system is being shaped by all of its components and the properties of the environment. With a high capacity for hydrogen bonding, water molecules are extremely sensitive to any changes the system undergoes. In highly aqueous systems-especially biological-water is the most abundant molecule. Minute changes in system elements or surroundings affect multitude of water molecules, causing rearrangements of water molecular network. Using light of various frequencies as a probe, the specifics of water structure can be extracted from the water spectrum, indirectly providing information about all the internal and external elements influencing the system. The water spectral pattern hence becomes an integrative descriptor of the system state. Aquaphotomics and the new knowledge of water originated from the field of near infrared spectroscopy. This technique resulted in significant findings about water structure-function relationships in various systems contributing to a better understanding of basic life phenomena. From this foundation, aquaphotomics started integration with other disciplines into systematized science from which a variety of applications ensued. This review will present the basics of this emerging science and its technological potential.

Water as a probe for serum-based diagnosis by temperature- dependent near-infrared spectroscopy

Diagnosis based on the compositional variation of biological liquids such as serum has drawn much attention. For exploring the potential diagnostic information from serum samples, temperature-dependent near-infrared (NIR) spectroscopy was developed to obtain the spectral change of water reflecting the interactions in serum solution, and chemometric methods were employed to mine the information from the temperature-dependent NIR spectra. The spectra of 68 healthy controls, 42 patients with the type II diabetes and 16 patients with coronary heart disease were measured and analyzed by chemometric calculations. Continuous wavelet transform (CWT) was used to enhance the resolution of the spectra. From the processed spectra, the information of non-hydrogen-bonded (NHB), weakly hydrogen-bonded (WHB) and strongly hydrogen-bonded (SHB) water species was observed. For explaining the variation of the spectra with temperature, two-dimensional (2D) correlation analysis was adopted. A clear difference in SHB/NHB ratio in the synchronous maps was found between the spectra of the patients and the controls. 86.8% of the controls and 98.3% of the patients can be correctly identified. Furthermore, combining the maps of the synchronous and asynchronous analysis, the correlation between SHB and WHB water species was discovered to have an ability to discriminate the patients of diabetes and heart disease with an accuracy of 83.7% and 75.0%, respectively. Therefore, water may be a probe for providing diagnostic information by temperature-dependent NIR spectroscopy.