NIR spectroscopy coupled with chemometric algorithms for the prediction of cadmium content in rice samples

Machine learning methods in near infrared spectroscopy for predicting sensory traits in sweetpotatoes

It has been established that near infrared (NIR) spectroscopy has the potential of estimating sensory traits given the direct spectral responses that these properties have in the NIR region. In sweetpotato, sensory and texture traits are key for improving acceptability of the crop for food security and nutrition. Studies have statistically modelled the levels of NIR spectroscopy sensory characteristics using partial least squares (PLS) regression methods. To improve prediction accuracy, there are many advanced techniques, which could enhance modelling of fresh (wet and un-processed) samples or nonlinear dependence relationships. Performance of different quantitative prediction models for sensory traits developed using different machine learning methods were compared. Overall, results show that linear methods; linear support vector machine (L-SVM), principal component regression (PCR) and PLS exhibited higher mean R2 values than other statistical methods. For all the 27 sensory traits, calibration models using L-SVM and PCR has slightly higher overall R2 (x¯ = 0.33) compared to PLS (x¯ = 0.32) and radial-based SVM (NL-SVM; x¯= 0.30). The levels of orange color intensity were the best predicted by all the calibration models (R2 = 0.87 - 0.89). The elastic net linear regression (ENR) and tree-based methods; extreme gradient boost (XGBoost) and random forest (RF) performed worse than would be expected but could possibly be improved with increased sample size. Lower average R2 values were observed for calibration models of ENR (x¯ = 0.26), XGBoost (x¯ = 0.26) and RF (x¯ = 0.22). The overall RMSE in calibration models was lower in PCR models (X = 0.82) compared to L-SVM (x¯ = 0.86) and PLS (x¯ = 0.90). ENR, XGBoost and RF also had higher RMSE (x¯ = 0.90 - 0.92). Effective wavelengths selection using the interval partial least-squares regression (iPLS), improved the performance of the models but did not perform as good as the PLS. SNV pre-treatment was useful in improving model performance.

E-eye and FT-NIR combined with multivariate algorithms to rapidly evaluate the dynamic changes in the quality of Gastrodia elata during steaming process

Gastrodia elata (GE) is traditionally subjected to steaming, and steaming duration plays a crucially important role in determining GE quality. This study examined the variations in bioactive components during the steaming process and proposed the utilization of electronic eye and Fourier Transform near-infrared (FT-NIR) spectroscopy for quality assessment. The findings revealed that the levels of parishin E parishin B, parishin A, and gastrodin initially rose and subsequently declined, while 4-Hydroxybenzyl alcohol exhibited a rapid decrease followed by stabilization. With prolonged steaming, the brightness of GE decreased, while the red and yellow tones became more pronounced and the color saturation increased. FT-NIR divided the steaming process into three stages: 0 min (raw GE), 0-9 min (partially steamed GE), and 9-30 min (fully steamed GE), and the partial least squares regression models effectively predicted the levels of five components. Overall, this study provided valuable insights into quality control in food processing.

A comprehensive "quality-quantity-activity" approach based on portable near-infrared spectrometer and membership function analysis to systematically evaluate spice quality: Cinnamomum cassia as an example

Spices have long been popular worldwide. Besides serving as aromatic and flavorful food and cooking ingredients, many spices exhibit notable bioactivity. Quality evaluation methods are essential for ensuring the quality and flavor of spices. However, existing methods typically focus on the content of particular components or certain aspects of bioactivity. For a systematic evaluation of spice quality, we herein propose a comprehensive "quality-quantity-activity" approach based on portable near-infrared spectrometer and membership function analysis. Cinnamomum cassia was used as a representative example to illustrate this approach. Near-infrared spectroscopy and chemometric methods were combined to predict the geographical origin, cinnamaldehyde content, ash content, antioxidant activity, and integrated membership function value. All the optimal prediction models displayed good predictive ability (correlation coefficient of prediction > 0.9, residual predictive deviation > 2.1). The proposed approach can provide a valuable reference for the rapid and comprehensive quality evaluation of spices.

Spectraformer: deep learning model for grain spectral qualitative analysis based on transformer structure

This study delves into the use of compact near-infrared spectroscopy instruments for distinguishing between different varieties of barley, chickpeas, and sorghum, addressing a vital need in agriculture for precise crop variety identification. This identification is crucial for optimizing crop performance in diverse environmental conditions and enhancing food security and agricultural productivity. We also explore the potential application of transformer models in near-infrared spectroscopy and conduct an in-depth evaluation of the impact of data preprocessing and machine learning algorithms on variety classification. In our proposed spectraformer multi-classification model, we successfully differentiated 24 barley varieties, 19 chickpea varieties, and ten sorghum varieties, with the highest accuracy rates reaching 85%, 95%, and 86%, respectively. These results demonstrate that small near-infrared spectroscopy instruments are cost-effective and efficient tools with the potential to advance research in various identification methods, but also underscore the value of transformer models in near-infrared spectroscopy classification. Furthermore, we delve into the discussion of the influence of data preprocessing on the performance of deep learning models compared to traditional machine learning models, providing valuable insights for future research in this field.

Raman spectroscopy combined with partial least squares (PLS) based on hybrid spectral preprocessing and backward interval PLS (biPLS) for quantitative analysis of four PAHs in oil sludge

Polycyclic aromatic hydrocarbons (PAHs) contained in a large amount of oily sludge produced in petroleum and petrochemical production has become one of the main environmental protection concerns in the industry. The accurate determination of PAHs is of great significance in the field of petroleum geochemistry and environmental protection. In this study, Raman spectroscopy combined with partial least squares (PLS) based on different hybrid spectral preprocessing methods and variable selection strategies was proposed for quantitative analysis of phenanthrene, fluoranthrene, fluorene and naphthalene (Phe, Flt, Flu and Nap) in oil sludge. At first, PAHs in oily sludge was extracted by solid-liquid extraction with methanol as extractant, and Raman spectra of 21 oily sludge samples were collected by portable Raman spectrometer. And then, the influence of first derivative (D1st), wavelet transform (WT) and their hybrid spectral preprocessing on the predictive performance of the PLS calibration model was discussed. Thirdly, biPLS (backward interval partial least squares) was used to optimize the input variables before and after the hybrid spectral preprocessing methods, and the influence of biPLS and the hybrid spectral preprocessing sequence on the predictive performance of the PLS calibration model was discussed. Finally, the predictive performance of the PLS calibration model was optimized according to the results of leave-one-out cross-validation (LOOCV) method. The results show that the biPLS-D1st-WT-PLS calibration model established by using biPLS first to select the characteristic variables, followed by hybrid spectral preprocessing of the characteristic variables, has better prediction performance for Flt (determination coefficient of prediction (R2P) = 0.9987, and the mean relative error of prediction (MREP) = 0.0606). For Phe, Flu and Nap, the WT-biPLS-PLS calibration model has a better predictive effect (R2P are 0.9995, 0.9996 and 0.9983, and MREP are 0.0426, 0.0719 and 0.0497, respectively). In general, portable Raman spectroscopy combined with PLS calibration model based on different hybrid spectral preprocessing and variable selection strategies has achieved good prediction results for quantitative analysis of four PAHs in oily sludge. It is a new strategy to firstly select the characteristic variables of the original spectra, and secondly to preprocess the characteristic variables by the hybrid spectral preprocessing, which will provide a new idea for the establishment of quantitative analysis methods for PAHs in oily sludge.

Application of Visible/Near-Infrared Spectroscopy and Hyperspectral Imaging with Machine Learning for High-Throughput Plant Heavy Metal Stress Phenotyping: A Review

Heavy metal pollution is becoming a prominent stress on plants. Plants contaminated with heavy metals undergo changes in external morphology and internal structure, and heavy metals can accumulate through the food chain, threatening human health. Detecting heavy metal stress on plants quickly, accurately, and nondestructively helps to achieve precise management of plant growth status and accelerate the breeding of heavy metal-resistant plant varieties. Traditional chemical reagent-based detection methods are laborious, destructive, time-consuming, and costly. The internal and external structures of plants can be altered by heavy metal contamination, which can lead to changes in plants' absorption and reflection of light. Visible/near-infrared (V/NIR) spectroscopy can obtain plant spectral information, and hyperspectral imaging (HSI) can obtain spectral and spatial information in simple, speedy, and nondestructive ways. These 2 technologies have been the most widely used high-throughput phenotyping technologies of plants. This review summarizes the application of V/NIR spectroscopy and HSI in plant heavy metal stress phenotype analysis as well as introduces the method of combining spectroscopy with machine learning approaches for high-throughput phenotyping of plant heavy metal stress, including unstressed and stressed identification, stress types identification, stress degrees identification, and heavy metal content estimation. The vegetation indexes, full-range spectra, and feature bands identified by different plant heavy metal stress phenotyping methods are reviewed. The advantages, limitations, challenges, and prospects of V/NIR spectroscopy and HSI for plant heavy metal stress phenotyping are discussed. Further studies are needed to promote the research and application of V/NIR spectroscopy and HSI for plant heavy metal stress phenotyping.

Application of spectroscopic techniques combined with chemometrics to the authenticity and quality attributes of rice

Rice is a staple food for two-thirds of the world's population and is grown in over a hundred countries around the world. Due to its large scale, it is vulnerable to adulteration. In addition, the quality attribute of rice is an important factor affecting the circulation and price, which is also paid more and more attention. The combination of spectroscopy and chemometrics enables rapid detection of authenticity and quality attributes in rice. This article described the application of seven spectroscopic techniques combined with chemometrics to the rice industry. For a long time, near-infrared spectroscopy and linear chemometric methods (e.g., PLSR and PLS-DA) have been widely used in the rice industry. Although some studies have achieved good accuracy, with models in many studies having greater than 90% accuracy. However, higher accuracy and stability were more likely to be obtained using multiple spectroscopic techniques, nonlinear chemometric methods, and key wavelength selection algorithms. Future research should develop larger rice databases to include more rice varieties and larger amounts of rice depending on the type of rice, and then combine various spectroscopic techniques, nonlinear chemometric methods, and key wavelength selection algorithms. This article provided a reference for a more efficient and accurate determination of rice quality and authenticity.

Ultra-Sensitive Determination of Cadmium in Food and Water by Flame-AAS after a New Polyvinyl Benzyl Xanthate as an Adsorbent Based Vortex Assisted Dispersive Solid-Phase Microextraction: Multivariate Optimization

Background: Cadmium (Cd) is a very toxic and carcinogenic heavy metal even at low levels and it is naturally present in water as well as in food. Methods: A new polyvinyl benzyl xanthate (PvbXa) was synthesized and used as a new adsorbent in this work. It contains pendant sulfide groups on the main polystyryl chain. Using this new adsorbent, PvbXa, a vortex-assisted dispersive solid-phase microextraction (VA-dSPµE) procedure was developed for the determination of cadmium from food and water samples via flame atomic absorption spectrophotometry (FAAS). Synthesized PvbXa was characterized by 1H Nuclear magnetic resonance (NMR) Spectroscopy, Fourier Transform Infrared Spectroscopy (FTIR), and X-ray Photoelectron Spectroscopy (XPS). The different parameters of pH, sample volume, mixing type and time, sorbent amount, and eluent time were optimized using standard analytical methods. Results: The optimized method for assessment of Cd in food and water samples shows good reliability. The optimum conditions were found to be a 0.20-150 µg L-1 linear range, 0.06 µg L-1 LOD, 0.20 µg L-1 LOQ, 4.3 RSD %, and a preconcentration factor of 160. Conclusions: The statistically experimental variables were utilized using a central composite design (CCD). The present method is a low-cost, simple, sensitive, and very effective tool for the recovery of Cd.

Rapid evaluation of Radix Paeoniae Alba and its processed products by near-infrared spectroscopy combined with multivariate algorithms

It is well known that the processing method of herbal medicine has a complex impact on the active components and clinical efficacy, which is difficult to measure. As a representative herb medicine with diverse processing methods, Radix Paeoniae Alba (RPA) and its processed products differ greatly in clinical efficacy. However, in some cases, different processed products are confused for use in clinical practice. Therefore, it is necessary to strictly control the quality of RPA and its processed products. Giving that the time-consuming and laborious operation of traditional quality control methods, a comprehensive strategy of near-infrared (NIR) spectroscopy combined with multivariate algorithms was proposed. This strategy has the advantages of being rapid and non-destructive, not only qualitatively distinguishing RPA and various processed products but also enabling quantitative prediction of five bioactive components. Qualitatively, the subspace clustering algorithm successfully differentiated RPA and three processed products, with an accuracy rate of 97.1%; quantitatively, interval combination optimization (ICO), competitive adaptive reweighted sampling (CARS), and competitive adaptive reweighted sampling combined with successive projections algorithm (CARS-SPA) were used to optimize the PLS model, and satisfactory results were obtained in terms of wavelength selection. In conclusion, it is feasible to use NIR spectroscopy to rapidly evaluate the effect of processing methods on the quality of RPA, which provides a meaningful reference for quality control of other herbal medicines with numerous processing methods.

Measurement of nitrogen content in rice plant using near infrared spectroscopy combined with different PLS algorithms

Nitrogen plays an important role in rice growth, and determination of nitrogen content in rice plants is of great significance in assessing plant nutritional status and allowing precision cultivation. Traditional chemical methods for determining nitrogen content have the disadvantages of destructive sampling and lengthy analysis times. Here, the feasibility of rapid nitrogen content analysis by near-infrared (NIR) spectroscopy of rice plants was studied. Spectral data from 447 rice samples at several growth stages were used to establish a predictive model. Different spectral preprocessing methods and characteristic selection methods were compared, such as interval partial least-squares (iPLS), synergy interval partial least-squares (SiPLS), and moving-window partial least-squares (mwPLS). The SiPLS method exhibited better performance than mwPLS or iPLS. Specifically, the combination of four subintervals (7, 26, 27, and 28), with characteristic bands at 5299-4451 cm^-1 and 10445-10423 cm^-1, resulted in the best model. The optimal SiPLS model had a correlation coefficient of 0.9533 and a root mean square error of prediction (RMSEP) of 0.1952 on the prediction set. Compared to using the full spectra, using SiPLS reduced the number of characteristics by 87 % in the model, and RMSEP was reduced from 0.2284 to 0.1952. The results demonstrate that NIR spectroscopy combined with the SiPLS algorithm can be applied to quickly determine nitrogen content in rice plants. This study provides a technical framework to guide future precision agriculture efforts with respect to nitrogen application.

Near infrared spectroscopy quantification based on Bi-LSTM and transfer learning for new scenarios

This study proposed a deep transfer learning methodology based on an improved Bi-directional Long Short-Term Memory (Bi-LSTM) network for the first time to address the near infrared spectroscopy (NIR) model transfer issue between samples. We tested its effectiveness on two datasets of manure and polyglutamic acid (γ-PGA) solution, respectively. First, the optimal primary Bi-LSTM networks for cattle manure and the first batch of γ-PGA were developed by ablation experiments and both proved to outperform one-dimensional convolutional neural network (1D-CNN), Partial Least Square (PLS) and Extreme Learning Machine (ELM) models. Then, two types of transfer learning approaches were carried out to determine model transferability to non-homologous samples. For poultry manure and the second batch of γ-PGA, the obtained predicting results verified that the second approach of fine-tuning Bi-LSTM layers and re-training FC layers transcended the first approach of fixing Bi-LSTM layers and only re-training FC layers by reducing the RMSEP_target of 23.4275% and 50.7343%, respectively. Finally, comparisons with fine-tuning 1D-CNN and other traditional model transfer methods further identified the superiority of the proposed methodology with exceeding accuracy and smaller variation, which decreased RMSEP_target of poultry manure and the second batch of γ-PGA of 7.2832% and 48.1256%, 67.1117% and 80.6924% when compared to that acquired by fine-tuning 1D-CNN, Tradaboost-ELM and CCA-PLS which were the best of five traditional methods, respectively. The study demonstrates the potential of the Fine-tuning-Bi-LSTM enabled NIR technology to be used as a simple, cost effective and reliable detection tool for a wide range of applications under various new scenarios.

Development of multi-disturbance bagging Extreme Learning Machine method for cadmium content prediction of rape leaf using hyperspectral imaging technology

Exploring the cadmium (Cd) pollution in rape is of great significance to food safety and consumer health. In this study, a rapid, nondestructive and accurate method for the determination of Cd content in rape leaves based on hyperspectral imaging (HSI) technology was proposed. The spectral data of rape leaves under different Cd stress from 431 nm to 962 nm were collected by visible-near infrared HSI spectrometer. In order to improve the robustness and accuracy of the regression model, a machine learning algorithm was proposed, named multi-disturbance bagging Extreme Learning Machine (MdbaggingELM). The prediction models of Cd content in rape leaves based on MdbaggingELM and ELM-based method (ELM and baggingELM) were established and compared. The results showed that the model of the proposed MdbaggingELM method performed significantly in the prediction of Cd content, with Rp² of 0.9830 and RMSEP of 2.8963 mg/kg. The results confirmed that MdbaggingELM is an efficient regression algorithm, which played a positive role in enhancing the stability and the prediction effect of the model. The combination of MdbaggingELM and HSI technology has great potential in the detection of Cd content in rape leaves.

Rapid Determination of Cellulose and Hemicellulose Contents in Corn Stover Using Near-Infrared Spectroscopy Combined with Wavelength Selection

The contents of cellulose and hemicellulose (C and H) in corn stover (CS) have an important influence on its biochemical transformation and utilization. To rapidly detect the C and H contents in CS by near-infrared spectroscopy (NIRS), the characteristic wavelength selection algorithms of backward partial least squares (BIPLS), competitive adaptive reweighted sampling (CARS), BIPLS combined with CARS, BIPLS combined with a genetic simulated annealing algorithm (GSA), and CARS combined with a GSA were used to select the wavelength variables (WVs) for C and H, and the corresponding regression correction models were established. The results showed that five wavelength selection algorithms could effectively eliminate irrelevant redundant WVs, and their modeling performance was significantly superior to that of the full spectrum. Through comparison and analysis, it was found that CARS combined with GSA had the best comprehensive performance; the predictive root mean squared errors of the C and H regression model were 0.786% and 0.893%, and the residual predictive deviations were 3.815 and 12.435, respectively. The wavelength selection algorithm could effectively improve the accuracy of the quantitative analysis of C and H contents in CS by NIRS, providing theoretical support for the research and development of related online detection equipment.

Genetic algorithm based artificial neural network and partial least squares regression methods to predict of breakdown voltage for transformer oils samples in power industry using ATR-FTIR spectroscopy

The current study proposes a novel analytical method for calculating the breakdown voltage (BV) of transformer oil samples considered as a significant method to assess the safe operation of power industry. Transformer oil samples can be analyzed using the Attenuated total reflection Fourier transform infrared (ATR-FTIR) spectroscopy combined with multivariate calibration methods. The partial least squares regression (PLSR) back propagation-artificial neural network (BP-ANN) methods and a genetic algorithm (GA) for variable selection are used to predict and assess breakdown voltage in transformer oil samples from various Iranian transformer oils. As a result, the root mean square error (RMSE) and correlation coefficient for the training and test sets of oil samples are also calculated. In the GA-PLS-R method, the squared correlation coefficient (R²pred) and root mean square prediction error (RMSEP) are 0.9437 and 2.6835, respectively. GA-BP-ANN, on the other hand, had a lower RMSEP value (0.2874) and a higher R²pred function (0.9891). Considering the complexity of transformer oil samples, the performance of GA-BP-ANN has resulted in an efficient approach for predicting breakdown voltage; consequently, it can be effectively used as a new method for quantitative breakdown voltage analysis of samples to evaluate the health of transformer oil. .

Near infrared spectroscopy combined with chemometrics for quantitative analysis of corn oil in edible blend oil

In this study, near infrared (NIR) spectroscopy combined with chemometrics was used for the quantitative analysis of corn oil in binary to hexanary edible blend oil. Sesame oil, soybean oil, rice oil, sunflower oil and peanut oil were mixed with corn oil subsequently to form binary, ternary, quaternary, quinary and hexanary blend oil datasets. NIR spectra for the five order blend oil datasets were measured in a transmittance mode in the range of 12000-4000 cm^-1. Partial least square (PLS) was used to build models for the five datasets. Six spectral preprocessing methods and their combinations were investigated to improve the prediction performance. Furthermore, the optimal preprocessing-PLS models were further optimized by uninformative variable elimination (UVE), Monte Carlo uninformative variable elimination (MCUVE) and randomization test (RT) variable selection methods. The optimal models acquire root mean square error of prediction (RMSEP) of 1.7299, 2.2089, 2.3742, 2.5608 and 2.6858 for binary, ternary, quaternary, quinary and hexanary blend oil datasets, respectively. The determination coefficients of prediction set (R²_P) and residual predictive deviations (RPDs) for the five datasets are all above 0.93 and 3. Results show that the prediction accuracy is gradually decreased with the increasing of mixture order of blend oil. However, with proper spectral preprocessing and variable selection, the optimal models present good prediction accuracy even for the higher order blend oil. It demonstrates that NIR technology is feasible for determining the pure oil contents in binary to hexanary blend oil.