Automatic baseline recognition for the correction of large sets of spectra using continuous wavelet transform and iterative fitting

Origins of Baseline Drift and Distortion in Fourier Transform Spectra

The spectrum scanned by a Fourier transform spectrometer (FTIR) often has a baseline drift. However, baseline distortion rarely occurs in a laboratory owing to the insignificant effects of environmental vibrations and electromagnetic factors. Even if it occurs, the distorted spectrum can be manually eliminated. However, in a complex environment, especially after the long-term operation of a spectrometer, the scanned spectrum may be distorted to different degrees. Herein, the origins of spectral baseline drifts and distortions are analyzed and simulated using MATLAB; furthermore, a baseline correction method based on the baseline-type model is proposed. The results of experiments performed on the methane spectrum confirm that the proposed method outperformed the improved modified multi-polynomial fitting and iterative averaging methods.

Fast Burst-Sparsity Learning-Based Baseline Correction (FBSL-BC) Algorithm for Signals of Analytical Instruments

Baseline correction is a critical step for eliminating the interference of baseline drift in spectroscopic analysis. The recently proposed sparse Bayesian learning (SBL)-based method can significantly improve the baseline correction performance. However, it has at least two disadvantages: (i) it works poorly for large-scale datasets and (ii) it completely ignores the burst-sparsity structure of the sparse representation of the pure spectrum. In this paper, we present a new fast burst-sparsity learning method for baseline correction to overcome these shortcomings. The first novelty of the proposed method is to jointly adopt a down-sampling strategy and construct a multiple measurements block-sparse recovery problem with the down-sampling sequences. The down-sampling strategy can significantly reduce the dimension of the spectrum; while jointly exploiting the block sparsity among the down-sampling sequences avoids losing the information contained in the original spectrum. The second novelty of the proposed method is introducing the pattern-coupled prior into the SBL framework to characterize the inherent burst-sparsity in the sparse representation of spectrum. As illustrated in the paper, burst-sparsity commonly occurs in peak zones with more denser nonzero coefficients. Properly utilizing such burst-sparsity can further enhance the baseline correction performance. Results on both simulated and real datasets (such as FT-IR, Raman spectrum, and chromatography) verify the substantial improvement, in terms of estimation accuracy and computational complexity.

Spectral preprocessing to improve accuracy of quantitative detection of elemental Cr in austenitic stainless steel by laser-induced breakdown spectroscopy

The production and consumption of austenitic stainless steel account for about 70% of stainless steel worldwide. The content of chromium (Cr) must be accurately detected and controlled to form a stable austenite structure and obtain strong properties in production. Laser-induced breakdown spectroscopy (LIBS) can be used to detect the Cr content of austenitic stainless steel in a complex production process. However, LIBS signals may be weak and unstable because the experimental signals are seriously affected by noise, self-absorption, the matrix effect, and the instability of the shot-to-shot signal, rendering the quantitative detection results inaccurate and unstable. The spectral-preprocessing methods of baseline correction and denoising can improve the accuracy of quantitative detection of LIBS. An improved segmented Hermite cubic-interpolation method is proposed herein to correct the baseline offset and produce baseline signals that are smooth and convergent (to overcome the Runge phenomenon). Empirical mode decomposition (EMD) based on the wavelet method is proposed to remove LIBS noise; this is done by exploiting the adaptivity of EMD to refine the wavelet-scaling coefficients. Compared with other denoising methods, the proposed method has good denoising evaluation indices and stability and, thus, effectively removes the noise. To verify detection accuracy, the internal standard quantitative method is used to detect the Cr content, and a cyclic-inversion prediction method is designed to verify detection stability. The results show that the correlation coefficient of the calibration curve is improved, the root-mean-square error is reduced, and the average relative error of the predicted Cr content decreases from 10.46% to 3.858%.

Baseline correction using a deep-learning model combining ResNet and UNet

A baseline correction method based on deep-learning model is proposed, which combines ResNet and UNet. Compared with the traditional methods, this method has relatively excellent performance.

An Automatic Baseline Correction Method Based on the Penalized Least Squares Method

Baseline drift spectra are used for quantitative and qualitative analysis, which can easily lead to inaccurate or even wrong results. Although there are several baseline correction methods based on penalized least squares, they all have one or more parameters that must be optimized by users. For this purpose, an automatic baseline correction method based on penalized least squares is proposed in this paper. The algorithm first linearly expands the ends of the spectrum signal, and a Gaussian peak is added to the expanded range. Then, the whole spectrum is corrected by the adaptive smoothness parameter penalized least squares (asPLS) method, that is, by turning the smoothing parameter λ of asPLS to obtain a different root-mean-square error (RMSE) in the extended range, the optimal λ is selected with minimal RMSE. Finally, the baseline of the original signal is well estimated by asPLS with the optimal λ. The paper concludes with the experimental results on the simulated spectra and measured infrared spectra, demonstrating that the proposed method can automatically deal with different types of baseline drift.

Edge-pixel referencing suppresses correlated baseline noise in heterodyned spectroscopies

Referencing schemes are commonly used in heterodyned spectroscopies to mitigate correlated baseline noise arising from shot-to-shot fluctuations of the local oscillator. Although successful, these methods rely on careful pixel-to-pixel matching between the two spectrographs. A recent scheme introduced by Feng et al. [Opt. Express 27(15), 20323-20346 (2019)] employed a correlation matrix to allow free mapping between dissimilar spectrographs, leading to the first demonstration of floor noise limited detection on a multichannel array used in heterodyned spectroscopy. In addition to their primary results using a second reference spectrometer, Feng et al. briefly demonstrated the flexibility of their method by referencing to same-array pixels at the two spectral edges (i.e., edge-pixel referencing). We present a comprehensive study of this approach, which we term edge-pixel referencing, including optimization of the approach, assessment of the performance, and determination of the effects of background responses. We show that, within some limitations, the distortions due to background signals will not affect the 2D IR line shape or amplitude and can be mitigated by band narrowing of the pump beams. We also show that the performance of edge-pixel referencing is comparable to that of referencing to a second spectrometer in terms of noise suppression and that the line shapes and amplitudes of the spectral features are, within the measurement error, identical. Altogether, these results demonstrate that edge-pixel referencing is a powerful approach for noise suppression in heterodyned spectroscopies, which requires no new hardware and, so, can be implemented as a software solution for anyone performing heterodyned spectroscopy with multichannel array detectors already.

All-fiber SERS sensing with a depressed double cladding fiber probe embedded in a microfluidic chip

A fiber surface-enhanced Raman scattering (SERS) probe based on the depressed double cladding fiber (DDCF) is demonstrated. Due to the depressed cladding, the optical field distribution of the DDCF is more approachable to the outer surface than that of a no core fiber without depressed cladding, improving the excitation efficiency for generating Raman scattered light. The cladding diameter of the DDCF is only 80 µm, which is smaller than the core diameter (105 µm) of the input multimode fiber (MMF). This small cladding diameter ensures a well collection coupling from the DDCF to the MMF. The impact of the DDCF length on the performance of the probe is investigated. Raman spectra of Rhodamine 6G (R6G) are measured using the DDCF-based SERS probe, which is embedded in a microfluidic chip. The all-fiber optical path configuration realizes an all-fiber Raman detection system that is stable, portable, and convenient to operate. Such results confirm the applicability of the DDCF-based fiber probe in the all-fiber SERS detection area and indicate that the proposed SERS probe renders a new scheme for optimal design of the fiber SERS probe structure.

Inline integration of offset MMF-capillary-MMF structure as a portable and compact fiber-optic surface-enhanced Raman scattering microfluidic chip

A novel fiber-optic surface-enhanced Raman scattering (SERS) microfluidic chip integrated with an embedded Raman probe is presented and demonstrated. The Raman probe consists of the offset-multimode-fiber (MMF)-capillary-MMF (OMCM) structure and SERS substrate. The probe is embedded in the microfluidic channel to form a compact and portable chip. The chip is employed with a fiber coupler in an all-fiber detection system, which has a good stability compared to the free space focusing by the conventional confocal Raman microscope. The excitation light transmits from both ends of the OMCM probe into the capillary, and the generated Raman scattered signals are collected by two MMFs simultaneously. Experimental results of the Rhodamine 6G (R6G) detection show that the Raman signal intensity increases in a linear pattern at ∼1509  cm^-1 with the increase of R6G concentrations. This kind of chip is compact, integrated, and miniaturized for the Raman signal detection. Furthermore, it can be fabricated easily in large quantities at cost, rendering promising applications.

Collaborative Penalized Least Squares for Background Correction of Multiple Raman Spectra

Although Raman spectroscopy has been widely used as a noninvasive analytical tool in various applications, backgrounds in Raman spectra impair its performance in quantitative analysis. Many algorithms have been proposed to separately correct the background spectrum by spectrum. However, in real applications, there are commonly multiple spectra collected from the close locations of a sample or from the same analyte with different concentrations. These spectra are strongly correlated and provide valuable information for more robust background correction. Herein, we propose two new strategies to remove background for a set of related spectra collaboratively. Based on weighted penalized least squares, the new approaches will use the fused weights from multiple spectra or the weights from the average spectrum to estimate the background of each spectrum in the set. Background correction results from both simulated and real experimental data demonstrate that the proposed collaborative approaches outperform traditional algorithms which process spectra individually.

Study on baseline correction methods for the Fourier transform infrared spectra with different signal-to-noise ratios

Removing the baseline from the spectra, which are measured by a Fourier transform infrared spectrometer (FTIR), is an important preprocessing step for further spectra analysis such as quantitative and qualitative analysis. An automatic baseline correction method named iterative averaging, which is based on the basic knowledge of moving average, is presented. We also compared it to other methods, such as rubber band, adaptive iterative reweight penalized least squares, automatic iterative moving average, and morphological weighted penalized least squares, using simulated and experimental spectra with different signal-to-noise ratios (SNRs) to evaluate the performance of these methods by performance metrics and to select an appropriate method to analyze FTIR spectra. Performance metrics such as root-mean-square error, goodness-of-fit coefficient, and chi-square are calculated. The iterative averaging method achieves the best results, which are judged by performance metrics values, when it is applied to the FTIR spectra with different SNRs. It also can correct the baseline of the FTIR spectra automatically, and improve the capability and adaptability of the unsupervised online analysis of the FTIR system effectively.

Automatic baseline correction method for the open-path Fourier transform infrared spectra by using simple iterative averaging

It's necessary to remove the baseline from the spectra, which measured by open-path Fourier transform infrared spectrometry, for further spectral analysis such as qualitative and quantitative analysis. An automatic baseline correction method, the Iterative Averaging method, is presented. Baseline corrected by this method is accurate, and it also shows more precise than other methods when it is applied to Fourier Transform Infrared experimental spectra and simulated data. This method solves the key technology of the real-time on-line spectral analysis of OP-FTIR and improves the capability and adaptability of the unsupervised on-line system effectively.

Multivariate Analysis of Mixed Lipid Aggregate Phase Transitions Monitored Using Raman Spectroscopy

The phase behavior of aqueous 1,2-dimyristoyl-sn-glycero-3-phosphorylcholine (DMPC)/1,2-dihexanoyl-sn-glycero-3-phosphocholine (DHPC) mixtures between 8.0 ℃ and 41.0 ℃ were monitored using Raman spectroscopy. Temperature-dependent Raman matrices were assembled from series of spectra and subjected to multivariate analysis. The consensus of pseudo-rank estimation results is that seven to eight components account for the temperature-dependent changes observed in the spectra. The spectra and temperature response profiles of the mixture components were resolved by applying a variant of the non-negative matrix factorization (NMF) algorithm described by Lee and Seung (1999). The rotational ambiguity of the data matrix was reduced by augmenting the original temperature-dependent spectral matrix with its cumulative counterpart, i.e., the matrix formed by successive integration of the spectra across the temperature index (columns). Successive rounds of constrained NMF were used to isolate component spectra from a significant fluorescence background. Five major components exhibiting varying degrees of gel and liquid crystalline lipid character were resolved. Hydrogen-bonded water networks exhibiting varying degrees of organization are associated with the lipid components. Spectral parameters were computed to compare the chain conformation, packing, and hydration indicated by the resolved spectra. Based on spectral features and relative amounts of the components observed, four components reflect long chain lipid response. The fifth component could reflect the response of the short chain lipid, DHPC, but there were no definitive spectral features confirming this assignment. A minor component of uncertain assignment that exhibits a striking response to the DMPC pre-transition and chain melting transition also was recovered. While none of the spectra resolved exhibit features unequivocally attributable to a specific aggregate morphology or step in the gelation process, the results are consistent with the evolution of mixed phase bicelles (nanodisks) and small amounts of worm-like DMPC/DHPC aggregates, and perhaps DHPC micelles, at low temperature to suspensions of branched and entangled worm-like aggregates above the DMPC gel phase transition and perforated multi-lamellar aggregates at high temperature.

Salient space detection algorithm for signal extraction from contaminated and distorted spectrum

The proposed algorithm minimizes the influence of baseline distortion and exhibits good anti-noise ability and high real-time performance.

Background Subtraction of Raman Spectra Based on Iterative Polynomial Smoothing

In this paper, a novel background subtraction algorithm is presented that can automatically recover Raman signal. This algorithm is based on an iterative polynomial smoothing method that highly reduces the need for experience and a priori knowledge. First, a polynomial filter is applied to smooth the input spectrum (the input spectrum is just an original spectrum at the first iteration). The output curve of the filter divides the original spectrum into two parts, top and bottom. Second, a proportion is calculated between the lowest point of the signal in the bottom part and the highest point of the signal in the top part. The proportion is a key index that decides whether to go into a new iteration. If a new iteration is needed, the minimum value between the output curve and the original spectrum forms a new curve that goes into the same filter in the first step and continues as another iteration until no more iteration is needed to finally get the background of the original spectrum. Results from the simulation experiments not only show that the iterative polynomial smoothing algorithm achieves good performance, processing time, cost, and accuracy of recovery, but also prove that the algorithm adapts to different background types and a large signal-to-noise ratio range. Furthermore, real measured Raman spectra of organic mixtures and non-organic samples are used to demonstrate the application of the algorithm.

Development and integration of block operations for data invariant automation of digital preprocessing and analysis of biological and biomedical Raman spectra

High-throughput information extraction from large numbers of Raman spectra is becoming an increasingly taxing problem due to the proliferation of new applications enabled using advances in instrumentation. Fortunately, in many of these applications, the entire process can be automated, yielding reproducibly good results with significant time and cost savings. Information extraction consists of two stages, preprocessing and analysis. We focus here on the preprocessing stage, which typically involves several steps, such as calibration, background subtraction, baseline flattening, artifact removal, smoothing, and so on, before the resulting spectra can be further analyzed. Because the results of some of these steps can affect the performance of subsequent ones, attention must be given to the sequencing of steps, the compatibility of these sequences, and the propensity of each step to generate spectral distortions. We outline here important considerations to effect full automation of Raman spectral preprocessing: what is considered full automation; putative general principles to effect full automation; the proper sequencing of processing and analysis steps; conflicts and circularities arising from sequencing; and the need for, and approaches to, preprocessing quality control. These considerations are discussed and illustrated with biological and biomedical examples reflecting both successful and faulty preprocessing.

Improved Savitzky-Golay-method-based fluorescence subtraction algorithm for rapid recovery of Raman spectra

In this paper, we propose an improved subtraction algorithm for rapid recovery of Raman spectra that can substantially reduce the computation time. This algorithm is based on an improved Savitzky-Golay (SG) iterative smoothing method, which involves two key novel approaches: (a) the use of the Gauss-Seidel method and (b) the introduction of a relaxation factor into the iterative procedure. By applying a novel successive relaxation (SG-SR) iterative method to the relaxation factor, additional improvement in the convergence speed over the standard Savitzky-Golay procedure is realized. The proposed improved algorithm (the RIA-SG-SR algorithm), which uses SG-SR-based iteration instead of Savitzky-Golay iteration, has been optimized and validated with a mathematically simulated Raman spectrum, as well as experimentally measured Raman spectra from non-biological and biological samples. The method results in a significant reduction in computing cost while yielding consistent rejection of fluorescence and noise for spectra with low signal-to-fluorescence ratios and varied baselines. In the simulation, RIA-SG-SR achieved 1 order of magnitude improvement in iteration number and 2 orders of magnitude improvement in computation time compared with the range-independent background-subtraction algorithm (RIA). Furthermore the computation time of the experimentally measured raw Raman spectrum processing from skin tissue decreased from 6.72 to 0.094 s. In general, the processing of the SG-SR method can be conducted within dozens of milliseconds, which can provide a real-time procedure in practical situations.