A Novel Pre-Processing Algorithm Based on the Wavelet Transform for Raman Spectrum

A three-stage deep learning-based training frame for spectra baseline correction

For spectrometers, baseline drift seriously affects the measurement and quantitative analysis of spectral data. Deep learning has recently emerged as a powerful method for baseline correction. However, the dependence on vast amounts of paired data and the difficulty in obtaining spectral data limit the performance and development of deep learning-based methods. Therefore, we solve these problems from the network architecture and training framework. For the network architecture, a Learned Feature Fusion (LFF) module is designed to improve the performance of U-net, and a three-stage training frame is proposed to train this network. Specifically, the LFF module is designed to adaptively integrate features from different scales, greatly improving the performance of U-net. For the training frame, stage 1 uses airPLS to ameliorate the problem of vast amounts of paired data, stage 2 uses synthetic spectra to further ease reliance on real spectra, and stage 3 uses contrastive learning to reduce the gap between synthesized and real spectra. The experiments show that the proposed method is a powerful tool for baseline correction and possesses potential for application in spectral quantitative analysis.

Deep neural network: As the novel pipelines in multiple preprocessing for Raman spectroscopy

Raman spectroscopy is a kind of vibrational method that can rapidly and non-invasively gives chemical structural information with the Raman spectrometer. Despite its technical advantages, in practical application scenarios, Raman spectroscopy often suffers from interference, such as noises and baseline drifts, resulting in the inability to acquire high-quality Raman spectroscopy signals, which brings challenges to subsequent spectral analysis. The commonly applied spectral preprocessing methods, such as Savitzky-Golay smooth and wavelet transform, can only perform corresponding single-item processing and require manual intervention to carry out a series of tedious trial parameters. Especially, each scheme can only be used for a specific data set. In recent years, the development of deep neural networks has provided new solutions for intelligent preprocessing of spectral data. In this paper, we first creatively started from the basic mechanism of spectral signal generation and constructed a mathematical model of the Raman spectral signal. By counting the noise parameters of the real system, we generated a simulation dataset close to the output of the real system, which alleviated the dependence on data during deep learning training. Due to the powerful nonlinear fitting ability of the neural network, fully connected network model is constructed to complete the baseline estimation task simply and quickly. Then building the Unet model can effectively achieve spectral denoising, and combining it with baseline estimation can realize intelligent joint processing. Through the simulation dataset experiment, it is proved that compared with the classic method, the method proposed in this paper has obvious advantages, which can effectively improve the signal quality and further ensure the accuracy of the peak intensity. At the same time, when the proposed method is applied to the actual system, it also achieves excellent performance compared with the common method, which indirectly indicates the effectiveness of the Raman signal simulation model. The research presented in this paper offers a variety of efficient pipelines for the intelligent processing of Raman spectroscopy, which can adapt to the requirements of different tasks while providing a new idea for enhancing the quality of Raman spectroscopy signals.

A double sliding-window method for baseline correction and noise estimation for Raman spectra of microplastics

When measuring microplastics of environmental samples, additives and attachment of biological materials may result in strong fluorescence in Raman spectra, which increases difficulty for imaging, identification, and quantification. Although there are several baseline correction methods available, user intervention is usually needed, which is not feasible for automated processes. In current study, a double sliding-window (DSW) method was proposed to estimate the baseline and standard deviation of noise. Simulated spectra and experimental spectra were used to evaluate the performance in comparison with two popular and widely used methods. Validation with simulated spectra and spectra of environmental samples showed that DSW method can accurately estimate the standard deviation of spectral noise. DSW method also showed better performance than compared methods when handling spectra of low signal-to-noise ratio (SNR) and elevated baselines. Therefore, DSW method is a useful approach for preprocessing Raman spectra of environmental samples and automated processes.

A Wavelet Derivative Spectrum Length Method of TFBG Sensor Demodulation

Fibre optic sensors using tilted fibre Bragg grating (TFBG) have high sensitivity for refractive index measurements. In order to achieve good metrological parameters of the measurement, an appropriate method of spectrum demodulation must be used. The method proposed in the article is an improvement of the spectral length algorithm. The spectral length parameter is treated as the sum of the derivative filter responses. In the original version, the first difference of spectrum elements was used, while this article proposes to use the wavelet transform to calculate the numerical derivative approximation. The advantage of this solution is an easy way to select the level of smoothing filtration by changing the scale parameter. The derivation is appropriate even for a relatively low signal-to-noise level. The approximation of the spectral length by the derivative calculated using the wavelet transform eliminates the high-frequency noise of the optical signal. The absolute value of determined spectral derivatives after significant smoothing can be used to estimate the wavelength of the decay of modes. After analyzing experimental data and performing calculations, it turns out that this is a linear method with better resolution than the contour length algorithm.

Quantitative analysis of phenanthrene in soil by fluorescence spectroscopy coupled with the CARS-PLS model†

Polycyclic aromatic hydrocarbons (PAHs) are typical organic pollutants in soil and are teratogenic and carcinogenic. Therefore, rapid and accurate analysis of PAHs in soil can provide a theoretical basis and data support for soil contamination risk assessment. In this work, a fluorescence spectroscopy technique combined with partial least squares (PLS) was proposed for rapid quantitative analysis of phenanthrene (PHE) in soil. At first, the fluorescence spectra of 29 soil samples with different concentrations (0.3–10 mg g⁻¹) of PHE were collected by RF-5301 PC fluorescence spectrophotometer. Secondly, the effects of different spectral preprocessing methods were investigated on the prediction performance of the PLS calibration model. And then, the influence of competitive adaptive reweighted sampling (CARS) wavelength points on the prediction performance of PLS calibration model was discussed. Finally, according to the selected wavelength points, a quantitative analytical model for PHE content in soil was constructed using the PLS calibration method. To further explore the predictive performance of the CARS-PLS calibration model, the predictive results were compared with those of the RAW spectrum-partial least squares calibration model (RAW-PLS) and the wavelet transform-standard normal variation (WT-SNV) calibration model. The CARS-PLS calibration model showed the optimal predictive performance and its coefficient of determination of cross-validation (R_cv²) and root mean square error of 10-fold cross-validation (RMSEcv) were 0.9957 and 18.98%, respectively. The coefficient of determination of prediction set (R_p²) and root mean square error of prediction set (RMSEp) were 0.9963 and 16.13%, respectively. Hence, the CARS algorithm based on fluorescence spectrum coupled with PLS can give a rapid and accurate quantitative analysis of the PHE content in soil.

Fluorescence spectroscopy coupled with CARS-PLS model is successfully used for the rapid quantitative analysis of phenanthrene in soil.

Baseline Correction Algorithm Based on Catastrophe Point Detection and Lipschitz Exponent’s Analysis

In order to quantitatively analyze the proportions of independent components in mixtures, it is necessary to extract line spectra corresponding to those components from the spectrum signal of some mixture and evaluate the amplitude of the spectral lines. Multiple factors cause the drift and tilt of a spectrum signal’s baseline, such as environment noises, instrument bias, and sample size, which affect the identification and quantitative analysis of the line spectra superimposed on the baseline. Therefore, the baseline of a spectrum signal should be removed before the line spectra are identified. A baseline correction algorithm based on Catastrophe Point detection and Lipschitz exponent’s analysis is proposed in this paper. With the algorithm, the strong spectral lines are identified and removed, and then the spectral baseline is evaluated without the interference of strong spectrum signals. First, catastrophe points are located based on the local modulus maxima theory of wavelet coefficients. Second, according to the Lipschitz exponent theory, the strong spectral peaks’ regions are identified and removed by a smoothing filter. Then the slowly varying spectrum is segmented adaptively and fitted by the least square fitting method. After the segments are attached and the boundaries are smoothed, the baseline of the spectrum is acquired and extracted finally. The algorithm is more accurate than classical ones because identifying the baseline is implemented after strong peaks are removed, so their influences to baseline extracting are eliminated. The results of experiments show that the algorithm is accurately performed for the spectrum signal of a gas mixture, [Formula: see text].

Estimation of metal elements content in soil using x-ray fluorescence based on multilayer perceptron

X-ray fluorescence (XRF) is widely used to rapidly detect heavy metals in soil. Spectra processing has been an important research topic to improve accuracy. In this study, 80 soil samples were analyzed by XRF under indoor conditions, where different preprocessing and quantitative analysis methods were compared in terms of prediction accuracy. Denoising algorithms were used to preprocess the soil spectra before establishing prediction models for As, Pb, Cu, Cr, and Cd in soil. The influence of denoising methods on the prediction effects of different models was compared and discussed. The results on five heavy metal spectra show that the proper spectral preprocessing method can effectively improve the prediction performance of the model. The multilayer perceptron model provides promising analysis and modeling for the five metal elements. The determination coefficients (R²) of the models were 0.857, 0.976, 0.977, 0.995, and 0.886, respectively. The proposed method provides the potential to support accurate quantitation by XRF analysis.

Baseline correction method based on improved adaptive iteratively reweighted penalized least squares for the x-ray fluorescence spectrum

To solve the problem of baseline drift in the detection of soil samples by x-ray fluorescence spectrum, an improved adaptive iterative weighted penalized least squares (IairPLS) method is proposed to estimate the baseline of x-ray fluorescence spectrum signals. We improved the original exponential weight function to solve the problem of baseline underestimation caused by adaptive iterative weighted penalized least squares. The improved function effectively reduces the risk of baseline underestimation and speeds up the weighting process, achieving good results. In this paper, the MC simulation spectrum and soil real analysis spectrum are used to verify the performance of the algorithm. Finally, the algorithm is compared with previous penalized least squares methods (asymmetric least squares, adaptive iterative reweighted penalized least squares, and multiple constrained reweighted penalized least squares), with the results showing that the proposed method has the least root-mean-square error after baseline correction for optimal smoothing parameters λ and the best relative error of baseline estimation accuracy. Meanwhile, the IairPLS method can effectively improve the quantitative analysis ability of the x-ray fluorescence spectrum. The proposed method can be successfully applied to the actual x-ray fluorescence spectrum, which provides a powerful basis for quantitative analysis.

Baseline correction method based on improved asymmetrically reweighted penalized least squares for the Raman spectrum

We present a baseline correction method based on improved asymmetrically reweighted penalized least squares (IarPLS) for the Raman spectrum. This method utilizes a new S-type function to reduce the risk of baseline overestimation and speed up the reweighting process. Simulated spectra with different levels of noise and measured spectra with strong fluorescence background from different samples are used to validate the performance of the proposed algorithm. Considering the drawbacks of the weighting rules for the asymmetrically reweighted penalized least squares (arPLS) method, we adapt an inverse square root unit (ISRU) function, which performs well in baseline correction. Compared with previous penalized least squares methods, such as asymmetric least squares, adaptive iteratively reweighted penalized least squares, and arPLS, experiments with the simulated Raman spectra have confirmed that the proposed method yields better outcomes. Experiments with the measured Raman spectra show that the IarPLS method can improve real Raman spectra within 20 ms. The results show that the proposed method can be successfully applied to the practical Raman spectrum as a strong basis for quantitative analysis.

Adaptive monostable stochastic resonance for processing UV absorption spectrum of nitric oxide

When ultraviolet (UV) absorption spectroscopy technology is used for nitric oxide (NO) detection, the background noise will directly affect the accuracy of concentration inversion, especially in low concentrations. Traditional processing methods attempt to eliminate background noise, which damages the absorption spectrum characteristics. However, stochastic resonance (SR) can utilize the noise to extract a weak characteristic signal. This paper reports a monostable stochastic resonance (MSR) model for processing an UV NO absorption spectrum. By analyzing the characteristics of UV absorption spectrum of NO, the evaluation indexes were constructed, thereby an adaptive MSR method was designed for parameter optimization. The numerical simulation confirmed the absorbance peak can be amplified and spectral signal-to-noise ratio (SNR) can be in the stable range of the proposed method, when noise intensity increased. Finally, this experiment obtained a NO detection limit (3σ) of 1.456 ppm and the maximum relative deviation of concentration is 6.32% by this proposed method, which is satisfactory for processing of the UV NO absorption spectrum.

Quantitative visualization of subcellular lignocellulose revealing the mechanism of alkali pretreatment to promote methane production of rice straw

BACKGROUND

As a renewable carbon source, biomass energy not only helps in resolving the management problems of lignocellulosic wastes, but also helps to alleviate the global climate change by controlling environmental pollution raised by their generation on a large scale. However, the bottleneck problem of extensive production of biofuels lies in the filamentous crystal structure of cellulose and the embedded connection with lignin in biomass that leads to poor accessibility, weak degradation and digestion by microorganisms. Some pretreatment methods have shown significant improvement of methane yield and production rate, but the promotion mechanism has not been thoroughly studied. Revealing the temporal and spatial effects of pretreatment on lignocellulose will greatly help deepen our understanding of the optimization mechanism of pretreatment, and promote efficient utilization of lignocellulosic biomass. Here, we propose an approach for qualitative, quantitative, and location analysis of subcellular lignocellulosic changes induced by alkali treatment based on label-free Raman microspectroscopy combined with chemometrics.

RESULTS

Firstly, the variations of rice straw induced by alkali treatment were characterized by the Raman spectra, and the Raman fingerprint characteristics for classification of rice straw were captured. Then, a label-free Raman chemical imaging strategy was executed to obtain subcellular distribution of the lignocellulose, in the strategy a serious interference of plant tissues' fluorescence background was effectively removed. Finally, the effects of alkali pretreatment on the subcellular spatial distribution of lignocellulose in different types of cells were discovered.

CONCLUSIONS

The results demonstrated the mechanism of alkali treatment that promotes methane production in rice straw through anaerobic digestion by means of a systemic study of the evidence from the macroscopic measurement and Raman microscopic quantitative and localization two-angle views. Raman chemical imaging combined with chemometrics could nondestructively realize qualitative, quantitative, and location analysis of the lignocellulose of rice straw at a subcellular level in a label-free way, which was beneficial to optimize pretreatment for the improvement of biomass conversion efficiency and promote extensive utilization of biofuel.