A novel NIR spectral calibration method: Sparse coefficients wavelength selection and regression (SCWR)

Characteristic wavelength optimization for partial least squares regression using improved flower pollination algorithm

Wavelength selection is crucial to the success of near-infrared (NIR) spectroscopy analysis as it considerably improves the generalization of the multivariate model and reduces model complexity. This study proposes a new wavelength selection method, interval flower pollination algorithm (iFPA), for spectral variable selection in the partial least squares regression (PLSR) model. The proposed iFPA consists of three phases. First, the flower pollination algorithm is applied to search for informative spectral variables, followed by variable elimination. Subsequently, the iFPA performs a local search to determine the best continuous interval spectral variables. The interpretability of the selected variables is assessed on three public NIR datasets (corn, diesel and soil datasets). Performance comparison with other competing wavelength selection methods shows that the iFPA used in conjunction with the PLSR model gives better prediction performance, with the root mean square error of prediction values of 0.0096-0.0727, 0.0015-3.9717 and 1.3388-29.1144 are obtained for various responses in corn, diesel and soil datasets, respectively.

Non-Destructive Detection of the Freshness of Air-Modified Mutton Based on Near-Infrared Spectroscopy

Monitoring and identifying the freshness levels of meat holds significant importance in the field of food safety as it directly relates to human dietary safety. Traditional packaging methods for lamb meat quality assessment present issues such as cumbersome operations and irreversible damage. This research proposes a quality assessment method for modified atmosphere packaging lamb meat using near-infrared spectroscopy and multi-parameter fusion. Fresh lamb meat quality is taken as the research subject, comparing various physicochemical indicators and near-infrared spectroscopic information under different temperatures (4 °C and 10 °C) and different modified atmosphere packaging combinations. Through precision parameter comparison, rebound and TVB-N values are selected as the modeling parameters. Six spectral preprocessing methods (multi-scatter calibration, MSC; standard normal variate transformation, SNV; normalization; Savitzky-Golay smoothing, SG; Savitzky-Golay 1 derivative, SG-1st; and Savitzky-Golay 2 derivative, SG-2nd), and three feature wavelength selection methods (competitive adaptive reweighted sampling, CARS; successive projections algorithm, SPA; and uninformative variable elimination, UVE) are compared. Partial least squares (PLS) and support vector machine (SVM) are used to construct prediction models for chilled fresh lamb meat quality. The results show that when rebound is used as a parameter, the SG-2nd-SPA-PLSR model has the highest accuracy, with a determination coefficient R²p of 0.94 for the prediction set. When TVB-N is used as a parameter, the MSC-UVE-SVM model has the highest accuracy, with an R²p of 0.95 for the prediction set. In conclusion, the use of near-infrared spectroscopic analysis enables rapid and non-destructive prediction and evaluation of lamb meat freshness, including its textural characteristics and TVB-N content under different modified atmosphere packaging. This study provides a theoretical basis and technical support for further encapsulating the models into portable devices and developing portable near-infrared spectrometers to rapidly determine lamb meat freshness.

Flexible Hyperspectral Anomaly Detection Using Weighted Nuclear Norm

It has been demonstrated that nuclear-norm-based low-rank representation is capable of modeling cluttered backgrounds in hyperspectral images (HSIs) for robust anomaly detection. However, minimizing the nuclear norm regularizes each singular value equally during rank reduction, which restricts the capacity and flexibility of modeling the major structures of the background. To address this problem, we propose detection of anomaly pixels in HSIs using the weighted nuclear norm, which can preserve the major singular values during rank reduction. We present a down-up sampling scheme to remove plausible anomaly pixels from the image as much as possible and learn a robust principal component analysis (PCA) background dictionary. From a dictionary, we develop a weighted nuclear-norm minimization model to represent the background with a low-rank coefficients matrix that can be effectively optimized using the standard alternating direction method of multipliers (ADMM). Due to the flexible modeling capacity using the weighted nuclear norm, anomaly pixels can be distinguished from the background with the reconstruction error. The experimental results on two real HSIs datasets demonstrate the effectiveness of the proposed method for anomaly detection.

Prediction of essential oil content in spearmint (Mentha spicata) via near-infrared hyperspectral imaging and chemometrics

Spearmint (Mentha spicata L.) is grown for its essential oil (EO), which find use in food, beverage, fragrance and other industries. The current study explores the ability of near infrared hyperspectral imaging (HSI) (935 to 1720 nm) to predict, in a rapid, nondestructive manner, the essential oil content of dried spearmint (0.2 to 2.6% EO). Spectral values of spearmint samples varied considerably with spatial coordinates, and so the use of averaging the spectral values of a surface scan was warranted. Data preprocessing was done with Multiplicative Scatter Correction (MSC) or Standard Normal Variate (SNV). Selection of spectral input variables was done with Least Absolute Shrinkage and Selection Operator (LASSO), Principal Component Analysis (PCA) or Partial Least Squares (PLS). Regression was executed with linear regression (LASSO, PLS regression, PCA regression), Support Vector Machine (SVM) regression, and Multilayer Perceptron (MLP). The best prediction of EO concentration was achieved with the combination of MSC or SNV preprocessing, PLS dimension reduction, and MLP regression (1 hidden layer with 6 nodes), achieving a good prediction with a ratio of performance to deviation (RPD) of 2.84 ± 0.07, an R² of prediction of 0.863 ± 0.008, and a RMSE of prediction of 0.219 ± 0.005% EO. These results show that NIR-HSI is a viable method for rapid, nondestructive analysis of EO concentration. Future work should explore the use of NIR in the visible spectrum, the use of HSI for determining EO in other plant materials and the potential of HSI to determine individual compounds in these solid plant/food matrices.

Practical Online Characterization of the Properties of Hydrocracking Bottom Oil via Near-Infrared Spectroscopy

Providing real-time information on the chemical properties of hydrocracking bottom oil (HBO) as the feedstock for ethylene cracker while minimizing processing time, is important to improve the real-time optimization of ethylene production. In this study, a novel approach for estimating the properties of HBO samples was developed on the basis of near-infrared (NIR) spectra. The main noise and extreme samples in the spectral data were removed by combining discrete wavelet transform with principal component analysis and Hotelling’s T2 test. Kernel partial least squares (KPLS) regression was utilized to account for the nonlinearities between NIR data and the chemical properties of HBO. Compared with the principal component regression, partial least squares regression, and artificial neural network, the KPLS model had a better performance of obtaining acceptable values of root mean square error of prediction (RMSEP) and mean absolute relative error (MARE). All RMSEP and MARE values of density, Bureau of Mines correlation index, paraffins, isoparaffins, and naphthenes were less than 1.0 and 3.0, respectively. The accuracy of the industrial NIR online measurement system during consecutive running periods in predicting the chemical properties of HBO was satisfactory. The yield of high value-added products increased by 0.26 percentage points and coil outlet temperature decreased by 0.25 °C, which promoted economic benefits of the ethylene cracking process and boosted industrial reform from automation to digitization and intelligence.

Non-Destructive Detection of Meat Quality Based on Multiple Spectral Dimension Reduction Methods by Near-Infrared Spectroscopy

The potential of four dimension reduction methods for near-infrared spectroscopy was investigated, in terms of predicting the protein, fat, and moisture contents in lamb meat. With visible/near-infrared spectroscopy at 400-1050 nm and 900-1700 nm, respectively, calibration models using partial least squares regression (PLSR) or multiple linear regression (MLR) between spectra and quality parameters were established and compared. The MLR prediction models for all three quality parameters based on the wavelengths selected by stepwise regression achieved the best results in the spectral region of 400-1050 nm. As for the spectral region of 900-1700 nm, the PLSR prediction model based on the raw spectra or high-correlation spectra achieved better results. The results of this study indicate that sampling interval shortening and of peak-to-trough jump features are worthy of further study, due to their great potential in explaining the quality parameters.

Ensemble calibration model of near-infrared spectroscopy based on functional data analysis

As a nondestructive detection technology, near-infrared spectroscopy has been widely applied in various fields. With the wide application of near-infrared spectroscopy, the research on data processing has attracted more attention. Different from the existing discrete data model and based on the functional data analysis method, an ensemble calibration model FDA-EM-PLS (functional data analysis-ensemble learning-partial least squares) of near-infrared spectroscopy is proposed in this paper. Firstly, the near-infrared spectroscopy of each sample is divided into several intervals, and the functional data analysis is carried out on each interval. Then, the samples are clustered according to the generated functions, which can not only reduce the influence of noise, but also provide a theoretical basis for selecting variables. Further, Monte Carlo sampling is used to generate training subsets from clustering samples for ensemble learning, which not only solves the problem of small samples, but also improves the robustness of the model. The relevant experimental results show that the absolute relative error of FDA-EM-PLS for the corn and soil data are both less than 10%.

Fast Burst-Sparsity Learning-Based Baseline Correction (FBSL-BC) Algorithm for Signals of Analytical Instruments

Baseline correction is a critical step for eliminating the interference of baseline drift in spectroscopic analysis. The recently proposed sparse Bayesian learning (SBL)-based method can significantly improve the baseline correction performance. However, it has at least two disadvantages: (i) it works poorly for large-scale datasets and (ii) it completely ignores the burst-sparsity structure of the sparse representation of the pure spectrum. In this paper, we present a new fast burst-sparsity learning method for baseline correction to overcome these shortcomings. The first novelty of the proposed method is to jointly adopt a down-sampling strategy and construct a multiple measurements block-sparse recovery problem with the down-sampling sequences. The down-sampling strategy can significantly reduce the dimension of the spectrum; while jointly exploiting the block sparsity among the down-sampling sequences avoids losing the information contained in the original spectrum. The second novelty of the proposed method is introducing the pattern-coupled prior into the SBL framework to characterize the inherent burst-sparsity in the sparse representation of spectrum. As illustrated in the paper, burst-sparsity commonly occurs in peak zones with more denser nonzero coefficients. Properly utilizing such burst-sparsity can further enhance the baseline correction performance. Results on both simulated and real datasets (such as FT-IR, Raman spectrum, and chromatography) verify the substantial improvement, in terms of estimation accuracy and computational complexity.

Rapid Estimation of Potato Quality Parameters by a Portable Near-Infrared Spectroscopy Device

The aim of the present work was to determine the main quality parameters on tuber potato using a portable near-infrared spectroscopy device (MicroNIR). Potato tubers protected by the Protected Geographical Indication (PGI “Patata de Galicia”, Spain) were analyzed both using chemical methods of reference and also using the NIR methodology for the determination of important parameters for tuber commercialization, such as dry matter and reducing sugars. MicroNIR technology allows for the attainment/estimation of dry matter and reducing sugars in the warehouses by directly measuring the tubers without a chemical treatment and destruction of samples. The principal component analysis and modified partial least squares regression method were used to develop the NIR calibration model. The best determination coefficients obtained for dry matter and reducing sugars were of 0.72 and 0.55, respectively, and with acceptable standard errors of cross-validation. Near-infrared spectroscopy was established as an effective tool to obtain prediction equations of these potato quality parameters. At the same time, the efficiency of portable devices for taking instantaneous measurements of crucial quality parameters is useful for potato processors.

Recent developments in vibrational spectral analyses for dynamically assessing and monitoring food dehydration processes

Dehydration is one of the most widely used food processing techniques, which is sophisticated in nature. Rapid and accurate prediction of dehydration performance and its effects on product quality is still a difficult task. Traditional analytical methods for evaluating food dehydration processes are laborious, time-consuming and destructive, and they are not suitable for online applications. On the other hand, vibrational spectral techniques coupled with chemometrics have emerged as a rapid and noninvasive tool with excellent potential for online evaluation and control of the dehydration process to improve final dried food quality. In the current review, the fundamental of food dehydration and five types of vibrational spectral techniques, and spectral data processing methods are introduced. Critical overtones bands related to dehydration attributes in the near-infrared (NIR) region and the state-of-the-art applications of vibrational spectral analyses in evaluating food quality attributes as affected by dehydration processes are summarized. Research investigations since 2010 on using vibrational spectral technologies combined with chemometrics to continuously monitor food quality attributes during dehydration processes are also covered in this review.

Wavelength calibration methods in laser wavelength measurement

At present, accurate wavelength calibration plays an important role in laser spectrum measurements. Although the wavelength calibration methods have been investigated for a long time, there are no techniques that are particularly designed for laser spectral calibration to the best of our knowledge. A mathematical model for calibrating a pulse laser wavelength is first established, to the best of our knowledge. According to the analysis formula of dispersion aberration, a flat-field concave grating in the near-infrared band is designed. Then, a wavelength calibration model based on concave grating spectroscopy is proposed. Through adjusting the spectra of each pixel, we design a calibration algorithm based on the cubic spline interpolation and kernel regression methods. By compensating and interpolating spectral data, accurate wavelengths are obtained. Finally, some experiments verify the calibration performance of the proposed method. Meanwhile, the uncertainty of measurement is also analyzed.

Wheat Kernel Variety Identification Based on a Large Near-Infrared Spectral Dataset and a Novel Deep Learning-Based Feature Selection Method

Near-infrared (NIR) hyperspectroscopy becomes an emerging nondestructive sensing technology for inspection of crop seeds. A large spectral dataset of more than 140,000 wheat kernels in 30 varieties was prepared for classification. Feature selection is a critical segment in large spectral data analysis. A novel convolutional neural network-based feature selector (CNN-FS) was proposed to screen out deeply target-related spectral channels. A convolutional neural network with attention (CNN-ATT) framework was designed for one-dimension data classification. Popular machine learning models including support vector machine (SVM) and partial least square discrimination analysis were used as the benchmark classifiers. Features selected by conventional feature selection algorithms were considered for comparison. Results showed that the designed CNN-ATT produced a higher performance than the compared classifier. The proposed CNN-FS found a subset of features, which made a better representation of raw dataset than conventional selectors did. The CNN-ATT achieved an accuracy of 93.01% using the full spectra and keep its high precision (90.20%) by training on the 60-channel features obtained via the CNN-FS method. The proposed methods have great potential for handling the analyzing tasks on other large spectral datasets. The proposed feature selection structure can be extended to design other new model-based selectors. The combination of NIR hyperspectroscopic technology and the proposed models has great potential for automatic nondestructive classification of single wheat kernels.