Application of Hyperspectral Imaging and Deep Learning for Robust Prediction of Sugar and pH Levels in Wine Grape Berries

Prediction performance and reliability evaluation of three ginsenosides in Panax ginseng using hyperspectral imaging combined with a novel ensemble chemometric model

Panax ginseng C. A. Meyer (PG) is a health-promoting food, and its ginsenosides (Rb1, Rg1, Re) content, as the quality indicator, is affected by the planting modes (garden or forest ginsengs) and years. Effective prediction of this content remains to be investigated. In this study, hyperspectral (HSI) combined with ensemble model (CGRU-GPR) including the convolutional neural network (CNN), gate recurrent unit (GRU), and Gaussian process regression (GPR) realized a comprehensive evaluation of the prediction performance and predictive uncertainty. With effective wavelengths, the proposed CGRU-GPR model improved operation efficiency and obtained satisfactory prediction results with relative percent deviation (RPD) values all higher than 2.70 in three ginsenosides. Meanwhile, the interval prediction with a high prediction interval coverage probability (PICP) of 0.97 - 1.0 and a low mean width percentage (MWP) of 0.7 - 1.66 indicated a low prediction uncertainty. This study provides a rapid and reliable method for predicting ginsenosides contents in PG.

A Rehabilitation of Pixel-Based Spectral Reconstruction from RGB Images

Recently, many deep neural networks (DNN) have been proposed to solve the spectral reconstruction (SR) problem: recovering spectra from RGB measurements. Most DNNs seek to learn the relationship between an RGB viewed in a given spatial context and its corresponding spectra. Significantly, it is argued that the same RGB can map to different spectra depending on the context with respect to which it is seen and, more generally, that accounting for spatial context leads to improved SR. However, as it stands, DNN performance is only slightly better than the much simpler pixel-based methods where spatial context is not used. In this paper, we present a new pixel-based algorithm called A++ (an extension of the A+ sparse coding algorithm). In A+, RGBs are clustered, and within each cluster, a designated linear SR map is trained to recover spectra. In A++, we cluster the spectra instead in an attempt to ensure neighboring spectra (i.e., spectra in the same cluster) are recovered by the same SR map. A polynomial regression framework is developed to estimate the spectral neighborhoods given only the RGB values in testing, which in turn determines which mapping should be used to map each testing RGB to its reconstructed spectrum. Compared to the leading DNNs, not only does A++ deliver the best results, it is parameterized by orders of magnitude fewer parameters and has a significantly faster implementation. Moreover, in contradistinction to some DNN methods, A++ uses pixel-based processing, which is robust to image manipulations that alter the spatial context (e.g., blurring and rotations). Our demonstration on the scene relighting application also shows that, while SR methods, in general, provide more accurate relighting results compared to the traditional diagonal matrix correction, A++ provides superior color accuracy and robustness compared to the top DNN methods.

Estimation of Sugar Content in Wine Grapes via In Situ VNIR-SWIR Point Spectroscopy Using Explainable Artificial Intelligence Techniques

Spectroscopy is a widely used technique that can contribute to food quality assessment in a simple and inexpensive way. Especially in grape production, the visible and near infrared (VNIR) and the short-wave infrared (SWIR) regions are of great interest, and they may be utilized for both fruit monitoring and quality control at all stages of maturity. The aim of this work was the quantitative estimation of the wine grape ripeness, for four different grape varieties, by using a highly accurate contact probe spectrometer that covers the entire VNIR-SWIR spectrum (350-2500 nm). The four varieties under examination were Chardonnay, Malagouzia, Sauvignon-Blanc, and Syrah and all the samples were collected over the 2020 and 2021 harvest and pre-harvest phenological stages (corresponding to stages 81 through 89 of the BBCH scale) from the vineyard of Ktima Gerovassiliou located in Northern Greece. All measurements were performed in situ and a refractometer was used to measure the total soluble solids content (°Brix) of the grapes, providing the ground truth data. After the development of the grape spectra library, four different machine learning algorithms, namely Partial Least Squares regression (PLS), Random Forest regression, Support Vector Regression (SVR), and Convolutional Neural Networks (CNN), coupled with several pre-treatment methods were applied for the prediction of the °Brix content from the VNIR-SWIR hyperspectral data. The performance of the different models was evaluated using a cross-validation strategy with three metrics, namely the coefficient of the determination (R2), the root mean square error (RMSE), and the ratio of performance to interquartile distance (RPIQ). High accuracy was achieved for Malagouzia, Sauvignon-Blanc, and Syrah from the best models developed using the CNN learning algorithm (R2>0.8, RPIQ≥4), while a good fit was attained for the Chardonnay variety from SVR (R2=0.63, RMSE=2.10, RPIQ=2.24), proving that by using a portable spectrometer the in situ estimation of the wine grape maturity could be provided. The proposed methodology could be a valuable tool for wine producers making real-time decisions on harvest time and with a non-destructive way.

Irradiance Independent Spectrum Reconstruction from Camera Signals Using the Interpolation Method

The spectrum of light captured by a camera can be reconstructed using the interpolation method. The reconstructed spectrum is a linear combination of the reference spectra, where the weighting coefficients are calculated from the signals of the pixel and the reference samples by interpolation. This method is known as the look-up table (LUT) method. It is irradiance-dependent due to the dependence of the reconstructed spectrum shape on the sample irradiance. Since the irradiance can vary in field applications, an irradiance-independent LUT (II-LUT) method is required to recover spectral reflectance. This paper proposes an II-LUT method to interpolate the spectrum in the normalized signal space. Munsell color chips irradiated with D65 were used as samples. Example cameras are a tricolor camera and a quadcolor camera. Results show that the proposed method can achieve the irradiance independent spectrum reconstruction and computation time saving at the expense of the recovered spectral reflectance error. Considering that the irradiance variation will introduce additional errors, the actual mean error using the II-LUT method might be smaller than that of the ID-LUT method. It is also shown that the proposed method outperformed the weighted principal component analysis method in both accuracy and computation speed.

Spectral Reflectance Recovery from the Quadcolor Camera Signals Using the Interpolation and Weighted Principal Component Analysis Methods

The recovery of surface spectral reflectance using the quadcolor camera was numerically studied. Assume that the RGB channels of the quadcolor camera are the same as the Nikon D5100 tricolor camera. The spectral sensitivity of the fourth signal channel was tailored using a color filter. Munsell color chips were used as reflective surfaces. When the interpolation method or the weighted principal component analysis (wPCA) method is used to reconstruct spectra, using the quadcolor camera can effectively reduce the mean spectral error of the test samples compared to using the tricolor camera. Except for computation time, the interpolation method outperforms the wPCA method in spectrum reconstruction. A long-pass optical filter can be applied to the fourth channel for reducing the mean spectral error. A short-pass optical filter can be applied to the fourth channel for reducing the mean color difference, but the mean spectral error will be larger. Due to the small color difference, the quadcolor camera using an optimized short-pass filter may be suitable as an imaging colorimeter. It was found that an empirical design rule to keep the color difference small is to reduce the error in fitting the color-matching functions using the camera spectral sensitivity functions.

Detection of Pesticide Residue Level in Grape Using Hyperspectral Imaging with Machine Learning

Rapid and accurate detection of pesticide residue levels can help to prevent the harm of pesticide residue. This study used visible/near-infrared (Vis-NIR) (376–1044 nm) and near-infrared (NIR) (915–1699 nm) hyperspectral imaging systems (HISs) to detect the level of pesticide residues. Three different varieties of grapes were sprayed with four levels of pesticides. Logistic regression (LR), support vector machine (SVM), random forest (RF), convolutional neural network (CNN), and residual neural network (ResNet) models were used to build classification models for pesticide residue levels. The saliency maps of CNN and ResNet were conducted to visualize the contribution of wavelengths. Overall, the results of NIR spectra performed better than those of Vis-NIR spectra. For Vis-NIR spectra, the best model was ResNet, with the accuracy of over 93%. For NIR spectra, LR was the best, with the accuracy of over 97%, but SVM, CNN, and ResNet also showed closed and fine results. The saliency map of CNN and ResNet presented similar and closed ranges of crucial wavelengths. Overall results indicated deep learning performed better than conventional machine learning. The study showed that the use of hyperspectral imaging technology combined with machine learning can effectively detect the level of pesticide residues in grapes.

Identification of micro- and nanoplastics released from medical masks using hyperspectral imaging and deep learning

In this study, dark-field microscopy-based hyperspectral imaging augmented with deep learning data analysis was applied for effective visualisation, detection and identification of microplastics released from polypropylene medical masks.

On the Optimization of Regression-Based Spectral Reconstruction

Spectral reconstruction (SR) algorithms attempt to recover hyperspectral information from RGB camera responses. Recently, the most common metric for evaluating the performance of SR algorithms is the Mean Relative Absolute Error (MRAE)-an ℓ1 relative error (also known as percentage error). Unsurprisingly, the leading algorithms based on Deep Neural Networks (DNN) are trained and tested using the MRAE metric. In contrast, the much simpler regression-based methods (which actually can work tolerably well) are trained to optimize a generic Root Mean Square Error (RMSE) and then tested in MRAE. Another issue with the regression methods is-because in SR the linear systems are large and ill-posed-that they are necessarily solved using regularization. However, hitherto the regularization has been applied at a spectrum level, whereas in MRAE the errors are measured per wavelength (i.e., per spectral channel) and then averaged. The two aims of this paper are, first, to reformulate the simple regressions so that they minimize a relative error metric in training-we formulate both ℓ2 and ℓ1 relative error variants where the latter is MRAE-and, second, we adopt a per-channel regularization strategy. Together, our modifications to how the regressions are formulated and solved leads to up to a 14% increment in mean performance and up to 17% in worst-case performance (measured with MRAE). Importantly, our best result narrows the gap between the regression approaches and the leading DNN model to around 8% in mean accuracy.