Variable Selection in Visible and Near-Infrared Spectral Analysis for Noninvasive Determination of Soluble Solids Content of ‘Ya’ Pear

Nondestructive detection of SSC in multiple pear (Pyrus pyrifolia Nakai) cultivars using Vis-NIR spectroscopy coupled with the Grad-CAM method

Vis-NIR spectroscopy coupled with chemometric models is frequently used for pear soluble solid content (SSC) prediction. However, the model robustness is challenged by the variations in pear cultivars. This study explored the feasibility of developing universal models for predicting SSC of multiple pear varieties to improve the model's generalizability. The mature fruits of 6 pear cultivars with green skin (Pyrus pyrifolia Nakai cv. 'Cuiyu', 'Sucui No.1' and 'Cuiguan') and brown skin (Pyrus pyrifolia Nakai cv. 'Hosui','Syusui' and 'Wakahikari') were used to establish single-cultivar models and multi-cultivar universal models using convolutional neural network (CNN), partial least square (PLS), and support vector regression (SVR) approaches. Multi-cultivar universal models were built using full spectra and important variables extracted by gradient-weighted class activation mapping (Grad-CAM), respectively. The universal models based on important variables obtained satisfactory performances with RMSEPs of 0.76, 0.59, 0.80, 1.64, 0.98, and 1.03°Brix on 6 cultivars, respectively.

Discrimination of New and Aged Seeds Based on On-Line Near-Infrared Spectroscopy Technology Combined with Machine Learning

The harvest year of maize seeds has a significant impact on seed vitality and maize yield. Therefore, it is vital to identify new seeds. In this study, an on-line near-infrared (NIR) spectra collection device (899-1715 nm) was designed and employed for distinguishing maize seeds harvested in different years. Compared with least squares support vector machine (LS-SVM), k-nearest neighbor (KNN), and extreme learning machine (ELM), the partial least squares discriminant analysis (PLS-DA) model has the optimal recognition performance for maize seed harvest years. Six different preprocessing methods, including Savitzky-Golay smoothing (SGS), standard normal variate transformation (SNV), multiplicative scatter correction (MSC), Savitzky-Golay 1 derivative (SG-D1), Savitzky-Golay 2 derivative (SG-D2), and normalization (Norm), were used to improve the quality of the spectra. The Monte Carlo cross-validation uninformative variable elimination (MC-UVE), competitive adaptive reweighted sampling (CARS), bootstrapping soft shrinkage (BOSS), successive projections algorithm (SPA), and their combinations were used to obtain effective wavelengths and decrease spectral dimensionality. The MC-UVE-BOSS-PLS-DA model achieved the classification with an accuracy of 88.75% using 93 features based on Norm preprocessed spectral data. This study showed that the self-designed NIR collection system could be used to identify the harvested years of maize seed.

Robustness of calibration model for prediction of lignin content in different batches of snow pears based on NIR spectroscopy

Snow pear is very popular in southwest China thanks to its fruit texture and potential medicinal value. Lignin content (LC) plays a direct and negative role (higher concentration and larger size of stone cells lead to thicker pulp and deterioration of the taste) in determining the fruit texture of snow pears as well as consumer purchasing decisions of fresh pears. In this study, we assessed the robustness of a calibration model for predicting LC in different batches of snow pears using a portable near-infrared (NIR) spectrometer, with the range of 1033-2300 nm. The average NIR spectra at nine different measurement positions of snow pear samples purchased at four different periods (batch A, B, C and D) were collected. We developed a standard normal variate transformation (SNV)-genetic algorithm (GA) -the partial least square regression (PLSR) model (master model A) - to predict LC in batch A of snow pear samples based on 80 selected effective wavelengths, with a higher correlation coefficient of prediction set (Rp) of 0.854 and a lower root mean square error of prediction set (RMSEP) of 0.624, which we used as the prediction model to detect LC in three other batches of snow pear samples. The performance of detecting the LC of batch B, C, and D samples by the master model A directly was poor, with lower Rp and higher RMSEP. The independent semi-supervision free parameter model enhancement (SS-FPME) method and the sequential SS-FPME method were used and compared to update master model A to predict the LC of snow pears. For the batch B samples, the predictive ability of the updated model (Ind-model AB) was improved, with an Rp of 0.837 and an RMSEP of 0.614. For the batch C samples, the performance of the Seq-model ABC was improved greatly, with an Rp of 0.952 and an RMSEP of 0.383. For the batch D samples, the performance of the Seq-model ABCD was also improved, with an Rp of 0.831 and an RMSEP of 0.309. Therefore, the updated model based on supervision and learning of new batch samples by the sequential SS-FPME method could improve the robustness and migration ability of the model used to detect the LC of snow pears and provide technical support for the development and practical application of portable detection device.

Prediction Models for the Content of Calcium, Boron and Potassium in the Fruit of 'Huangguan' Pears Established by Using Near-Infrared Spectroscopy

It has been proved that the imbalance of the proportion of elements of 'Huangguan' pears in the pulp and peel, especially calcium, boron and potassium, may be important factors that can seriously affect the pears' appearance quality and economic benefits. The objective of this study was to predict the content of calcium, boron and potassium in the pulp and peel of 'Huangguan' pears nondestructively and conveniently by using near-infrared spectroscopy (900-1700 nm) technology. Firstly, 12 algorithms were used to preprocess the original spectral data. Then, based on the original and preprocessed spectral data, full-band prediction models were established by using Partial Least Squares Regression and Gradient Boosting Regression Tree. Finally, the characteristic wavelengths were extracted by Genetic Algorithms to establish the characteristic wavelength prediction models. According to the prediction results, the value of the determination coefficient of the prediction sets of the best prediction models for the three elements all reached ideal levels, and the values of their Relative analysis error also showed high levels. Therefore, the micro near-infrared spectrometer based on machine learning can predict the content of calcium, boron and potassium in the pulp and peel of 'Huangguan' pears accurately and quickly. The results also provide an important scientific theoretical basis for further research on the degradation of the quality of 'Huangguan' pears caused by a lack of nutrients.

Model development and update of portable NIRS instrument for assessment of internal quality attributes of two navel orange varieties

The variation of fruit among batches influences the performance of the portable near infrared spectroscopy (NIRS) instrument and then determines the success or failure for practical application in fruit industry. Model development and update methods were investigated for determining soluble solids contents (SSC) and titrable acidity (TA) of navel orange. The pretreatment and variable selection methods were explored for building partial least square regression (PLSR) models. The best models, developed by the combination of second derivative (2D) and variable sorting for normalization (VSN), could predict SSC but not TA. The root mean square error of prediction (RMSEP), coefficient of determination for prediction (Rp2) and ratio of prediction to deviation (RPD) for SSC were 0.66 °Brix, 0.66 and 1.73. Model maintain methods of model update (MU) and slope and bias correction (SBC) achieved the best results in predicting SSC for two external validation sets with Rp2, RMSEP and RPD of 0.54, 0.83 °Brix, 1.60 and 0.52, 0.83 °Brix, 1.65, respectively. The results suggested model development and update with MU and SBC could improve the robustness of the portable NIRS instrument in predicting SSC of navel orange.

Determination of the Soluble Solids Content in Korla Fragrant Pears Based on Visible and Near-Infrared Spectroscopy Combined With Model Analysis and Variable Selection

The non-destructive detection of soluble solids content (SSC) in fruit by near-infrared (NIR) spectroscopy has a good application prospect. At present, the application of portable devices is more common. The construction of an accurate and stable prediction model is the key for the successful application of the device. In this study, the visible and near-infrared (Vis/NIR) spectra of Korla fragrant pears were collected by a commercial portable measurement device. Different pretreatment methods were used to preprocess the raw spectra, and the partial least squares (PLS) model was constructed to predict the SSC of pears for the determination of the appropriate pretreatment method. Subsequently, PLS and least squares support vector machine (LS-SVM) models were constructed based on the preprocessed full spectra. A new combination (BOSS-SPA) of bootstrapping soft shrinkage (BOSS) and successive projections algorithm (SPA) was used for variable selection. For comparison, single BOSS and SPA were also used for variable selection. Finally, three types of models, namely, PLS, LS-SVM, and multiple linear regression (MLR), were constructed based on different input variables. Comparing the prediction performance of all models, it showed that the BOSS-SPA-PLS model based on 17 variables obtained the best SSC assessment ability with r _p of 0.94 and RMSEP of 0.27 °Brix. The overall result indicated that portable measurement with Vis/NIR spectroscopy can be used for the detection of SSC in Korla fragrant pears.

Identification of Maize with Different Moldy Levels Based on Catalase Activity and Data Fusion of Hyperspectral Images

Maize is susceptible to mold infection during growth and storage due to its large embryo and high moisture content. Therefore, it is essential to distinguish the moldy sample from healthy groups to prevent the spread of mold and avoid huger economic losses. Catalase is a metabolite in the growth of microorganisms; hence, all maize samples were accurately divided into four moldy grades (health, mild, moderate, and severe levels) by determining their catalase activity. The visible and shortwave near-infrared (Vis-SWNIR) and longwave near-infrared (LWNIR) hyperspectral images were investigated to jointly identify the moldy levels of maize. Spectra and texture information of each maize sample were extracted and used to build the classification models of maize with different moldy levels in pixel-level fusion and feature-level fusion. The result showed that the feature-level fusion of spectral and texture within Vis-SWNIR and LWNIR regions achieved the best results, overall prediction accuracy reached 95.00% for each moldy level, all healthy maize was correctly classified, and none of the moldy samples were misclassified as healthy level. This study illustrated that two hyperspectral image systems, with complementary spectral ranges, combined with feature selection and data fusion strategies, could be used synergistically to improve the classification accuracy of maize with different moldy levels.

Rapid and nondestructive determination of sorghum purity combined with deep forest and near-infrared hyperspectral imaging

This study combined hyperspectral imaging (HSI) and deep forest (DF) to develop a reliable model for conducting a rapid and nondestructive determination of sorghum purity. Isolated forest (IF) algorithm and principal component analysis (PCA) were used to remove the abnormal data of sorghum grains. Competitive adaptive reweighted sampling (CARS) algorithm and successive projections algorithm (SPA) were combined and used to extract the characteristic wavelengths. Gray-level co-occurrence matrix (GLCM) was used to extract the textural features. DF models were established based on the different types of data. Specifically, the DF models established using the characteristic spectra produced the best recognition results: the average correct recognition rate (CRR) of the models was greater than 91%. In addition, the average CRR of validation set Ⅰ was 88.89%. These results show that a combination of HSI and DF could be used for the rapid and nondestructive determination of sorghum purity.

Online Detection of Watercore Apples by Vis/NIR Full-Transmittance Spectroscopy Coupled with ANOVA Method

Watercore is an internal physiological disorder affecting the quality and price of apples. Rapid and non-destructive detection of watercore is of great significance to improve the commercial value of apples. In this study, the visible and near infrared (Vis/NIR) full-transmittance spectroscopy combined with analysis of variance (ANOVA) method was used for online detection of watercore apples. At the speed of 0.5 m/s, the effects of three different orientations (O1, O2, and O3) on the discrimination results of watercore apples were evaluated, respectively. It was found that O3 orientation was the most suitable for detecting watercore apples. One-way ANOVA was used to select the characteristic wavelengths. The least squares-support vector machine (LS-SVM) model with two characteristic wavelengths obtained good performance with the success rates of 96.87% and 100% for watercore and healthy apples, respectively. In addition, full-spectrum data was also utilized to determine the optimal two-band ratio for the discrimination of watercore apples by ANOVA method. Study showed that the threshold discrimination model established based on O3 orientation had the same detection accuracy as the optimal LS-SVM model for samples in the prediction set. Overall, full-transmittance spectroscopy combined with the ANOVA method was feasible to online detect watercore apples, and the threshold discrimination model based on two-band ratio showed great potential for detection of watercore apples.

Nondestructive Analysis of Internal Quality in Pears with a Self-Made Near-Infrared Spectrum Detector Combined with Multivariate Data Processing

The consumption of pears has increased, thanks not only to their delicious and juicy flavor, but also their rich nutritional value. Traditional methods of detecting internal qualities (e.g., soluble solid content (SSC), titratable acidity (TA), and taste index (TI)) of pears are reliable, but they are destructive, time-consuming, and polluting. It is necessary to detect internal qualities of pears rapidly and nondestructively by using near-infrared (NIR) spectroscopy. In this study, we used a self-made NIR spectrum detector with an improved variable selection algorithm, named the variable stability and cluster analysis algorithm (VSCAA), to establish a partial least squares regression (PLSR) model to detect SSC content in snow pears. VSCAA is a variable selection method based on the combination of variable stability and cluster analysis to select the infrared spectrum variables. To reflect the advantages of VSCAA, we compared the classical variable selection methods (synergy interval partial least squares (SiPLS), genetic algorithm (GA), successive projections algorithm (SPA), and bootstrapping soft shrinkage (BOSS)) to extract useful wavelengths. The PLSR model, based on the useful variables selected by SiPLS-VSCAA, was optimal for measuring SSC in pears, and the correlation coefficient of calibration (Rc), root mean square error of cross validation (RMSECV), correlation coefficient of prediction (Rp), root mean square error of prediction (RMSEP), and residual predictive deviation (RPD) were 0.942, 0.198%, 0.936, 0.222%, and 2.857, respectively. Then, we applied these variable selection methods to select the characteristic wavelengths for measuring the TA content and TI value in snow pears. The prediction PLSR models, based on the variables selected by GA-BOSS to measure TA and that by GA-VSCAA to detect TI, were the best models, and the Rc, RMSECV, Rp and RPD were 0.931, 0.124%, 0.912, 0.151%, and 2.434 and 0.968, 0.080%, 0.968, 0.089%, and 3.775, respectively. The results showed that the self-made NIR-spectrum detector based on a portable NIR spectrometer with multivariate data processing was a good tool for rapid and nondestructive analysis of internal quality in pears.

Development of a novel wavelength selection method VCPA-PLS for robust quantification of soluble solids in tomato by on-line diffuse reflectance NIR

This work was attempted to evaluate the feasibility of a constructed on-line NIR platform coupled with efficient algorithms for rapid and robust quantification of quality parameter in cherry tomato. Specifically, a system was developed based on shortwave NIR spectroscopy for on-line quality inspection of cherry tomatoes. The spectra were recorded in diffuse reflectance mode from 900 to 1700 nm, and the conveyor belt speed was fixed to five samples per second. Three novel methods, namely variable combination population analysis (VCPA), uninformative variable elimination (UVE) and competitive adaptive reweighed sampling algorithm (CARS) were coupled with partial least square (PLS) for selecting optimal dataset, and modeling. The obtained results showed that under the optimal tuning parameters (N = 100, k = 500, ω = 14, σ = 10%), a total of 512 original variables, only 9 variables (1.75%) were extracted by VCPA. Subsequently, VCPA-PLS yielded outstanding performance in predicting soluble solid content in cherry tomatoes, with a higher correlation coefficient (R_P = 0.9053), and lower root mean square errors (RMSEP = 0.382) in prediction set. This methodology demonstrated the versatile potential of the proposed installation coupled with VCPA methods for on-line detection of total soluble solids in cherry tomatoes.

Non-Destructive Detection of Strawberry Quality Using Multi-Features of Hyperspectral Imaging and Multivariate Methods

Soluble solid content (SSC), pH, and vitamin C (VC) are considered as key parameters for strawberry quality. Spectral, color, and textural features from hyperspectral reflectance imaging of 400-1000 nm was to develop the non-destructive detection approaches for SSC, pH, and VC of strawberries by integrating various multivariate methods as partial least-squares regression (PLSR), support vector regression, and locally weighted regression (LWR). SSC, pH, and VC of 120 strawberries were statistically analyzed to facilitate the partitioning of data sets, which helped optimize the model. PLSR, with spectral and color features, obtained the optimal prediction of SSC with determination coefficient of prediction (Rp2) of 0.9370 and the root mean square error of prediction (RMSEP) of 0.1145. Through spectral features, the best prediction for pH was obtained by LWR with Rp2 = 0.8493 and RMSEP = 0.0501. Combination of spectral and textural features with PLSR provided the best results of VC with Rp2 = 0.8769 and RMSEP = 0.0279. Competitive adaptive reweighted sampling and uninformative variable elimination (UVE) were used to select important variables from the above features. Based on the important variables, the accuracy of SSC, pH, and VC prediction both gain the promotion. Finally, the distribution maps of SSC, pH, and VC over time were generated, and the change trend of three quality parameters was observed. Thus, the proposed method can nondestructively and accurately determine SSC, pH, and VC of strawberries and is expected to design and construct the simple sensors for the above quality parameters of strawberries.

Prediction of Congou Black Tea Fermentation Quality Indices from Color Features Using Non-Linear Regression Methods

Fermentation is the key process to produce the special color of congou black tea. The machine vision technology is applied to detect the color space changes of black tea's color in RGB, Lab and HSV, and to find out its relevance to black tea's fermentation quality. And then the color feature parameter is used as input to establish physicochemical indexes (TFs, TRs, and TBs) and sensory features' linear and non-linear quantitative evaluation model. Results reveal that color features are significantly correlated to quality indices. Compared with the other two color models (RGB and HSV), CIE Lab model can better reflect the dynamic variation features of quality indices and foliage color information of black tea. The predictability of non-linear models (RF and SVM) is superior to PLS linear model, while RF model presents a slight advantage over the classic SVM model since RF model can better represent the quantitative analytical relationship between image information and quality indices. This research has proved that computer image color features and non-linear method can be used to quantitatively evaluate the changes of quality indices (e.g. sensory quality) and the pigment during black tea's fermentation. Besides, the test is simple, fast, and nondestructive.

Prediction of Moisture Content for Congou Black Tea Withering Leaves Using Image Features and Nonlinear Method

Withering is the first step in the processing of congou black tea. With respect to the deficiency of traditional water content detection methods, a machine vision based NDT (Non Destructive Testing) method was established to detect the moisture content of withered leaves. First, according to the time sequences using computer visual system collected visible light images of tea leaf surfaces, and color and texture characteristics are extracted through the spatial changes of colors. Then quantitative prediction models for moisture content detection of withered tea leaves was established through linear PLS (Partial Least Squares) and non-linear SVM (Support Vector Machine). The results showed correlation coefficients higher than 0.8 between the water contents and green component mean value (G), lightness component mean value (L^*) and uniformity (U), which means that the extracted characteristics have great potential to predict the water contents. The performance parameters as correlation coefficient of prediction set (Rp), root-mean-square error of prediction (RMSEP), and relative standard deviation (RPD) of the SVM prediction model are 0.9314, 0.0411 and 1.8004, respectively. The non-linear modeling method can better describe the quantitative analytical relations between the image and water content. With superior generalization and robustness, the method would provide a new train of thought and theoretical basis for the online water content monitoring technology of automated production of black tea.

Application of variable selection in the origin discrimination of Wolfiporia cocos (F.A. Wolf) Ryvarden & Gilb. based on near infrared spectroscopy

Dried sclerotium of Wolfiporia cocos (F.A. Wolf) Ryvarden & Gilb. is a traditional Chinese medicine. Its chemical components showed difference among geographical origins, which made it difficult to keep therapeutic potency consistent. The identification of the geographical origin of W. cocos is the fundamental prerequisite for its worldwide recognition and acceptance. Four variable selection methods were employed for near infrared spectroscopy (NIR) variable selection and the characteristic variables were screened for the establishment of Fisher function models in further identification of the origin of W. cocos from Yunnan, China. For the obvious differences between poriae cutis (fu-ling-pi in Chinese, or FLP) and the inner part (bai-fu-ling in Chinese, or BFL) of the sclerotia of W. cocos in the pattern space of principal component analysis (PCA), we established discriminant models for FLP and BFL separately. Through variable selection, the models were significant improved and also the models were simplified by using only a small part of the variables. The characteristic variables were screened (13 for BFL and 10 for FLP) to build Fisher discriminant function models and the validation results showed the models were reliable and effective. Additionally, the characteristic variables were interpreted.

Uninformative Biological Variability Elimination in Apple Soluble Solids Content Inspection by Using Fourier Transform Near-Infrared Spectroscopy Combined with Multivariate Analysis and Wavelength Selection Algorithm

Uninformative biological variability elimination methods were studied in the near-infrared calibration model for predicting the soluble solids content of apples. Four different preprocessing methods, namely, Savitzky-Golay smoothing, multiplicative scatter correction, standard normal variate, and mean normalization, as well as their combinations were conducted on raw Fourier transform near-infrared spectra to eliminate the uninformative biological variability. Subsequently, robust calibration models were established by using partial least squares regression analysis and wavelength selection algorithms. Results indicated that the partial least squares calibration models with characteristic variables selected by CARS method coupled with preprocessing of Savitzky-Golay smoothing and multiplicative scatter correction had a considerable potential for predicting apple soluble solids content regardless of the biological variability.

A bi-layer model for nondestructive prediction of soluble solids content in apple based on reflectance spectra and peel pigments

Hyperspectral imaging technology was used to investigate the effect of various peel colors on soluble solids content (SSC) prediction model and build a SSC model insensitive to the color distribution of apple peel. The SSC and peel pigments were measured, effective wavelengths (EWs) of SSC and pigments were selected from the acquired hyperspectral images of the intact and peeled apple samples, respectively. The effect of pigments on the SSC prediction was studied and optimal SSC EWs were selected from the peel-flesh layers spectra after removing the chlorophyll and anthocyanin EWs. Then, the optimal bi-layer model for SSC prediction was built based on the finally selected optimal SSC EWs. Results showed that the correlation coefficient of prediction, root mean square error of prediction and selected bands of the bi-layer model were 0.9560, 0.2528 and 41, respectively, which will be more acceptable for future online SSC prediction of various colors of apple.

High-throughput analysis of chemical components and theoretical ethanol yield of dedicated bioenergy sorghum using dual-optimized partial least squares calibration models

BACKGROUND

Due to its chemical composition and abundance, lignocellulosic biomass is an attractive feedstock source for global bioenergy production. However, chemical composition variations interfere with the success of any single methodology for efficient bioenergy extraction from diverse lignocellulosic biomass sources. Although chemical component distributions could guide process design, they are difficult to obtain and vary widely among lignocellulosic biomass types. Therefore, expensive and laborious "one-size-fits-all" processes are still widely used. Here, a non-destructive and rapid analytical technology, near-infrared spectroscopy (NIRS) coupled with multivariate calibration, shows promise for addressing these challenges. Recent advances in molecular spectroscopy analysis have led to methodologies for dual-optimized NIRS using sample subset partitioning and variable selection, which could significantly enhance the robustness and accuracy of partial least squares (PLS) calibration models. Using this methodology, chemical components and theoretical ethanol yield (TEY) values were determined for 70 sweet and 77 biomass sorghum samples from six sweet and six biomass sorghum varieties grown in 2013 and 2014 at two study sites in northern China.

RESULTS

Chemical components and TEY of the 147 bioenergy sorghum samples were initially analyzed and compared using wet chemistry methods. Based on linear discriminant analysis, a correct classification assignment rate (either sweet or biomass type) of 99.3% was obtained using 20 principal components. Next, detailed statistical analysis demonstrated that partial optimization using sample set partitioning based on joint X-Y distances (SPXY) for sample subset partitioning enhanced the robustness and accuracy of PLS calibration models. Finally, comparisons between five dual-optimized strategies indicated that competitive adaptive reweighted sampling coupled with the SPXY (CARS-SPXY) was the most efficient and effective method for improving predictive performance of PLS multivariate calibrations.

CONCLUSIONS

As a dual-optimized methodology, sample subset partitioning combined with variable selection is an efficient and straightforward strategy to enhance the accuracy and robustness of NIRS models. This knowledge should facilitate generation of improved lignocellulosic biomass feedstocks for bioethanol production. Moreover, methods described here should have wider applicability for use with feedstocks incorporating multispecies biomass resource streams.

Hyperspectral Imaging Coupled with Random Frog and Calibration Models for Assessment of Total Soluble Solids in Mulberries

Chemometrics methods coupled with hyperspectral imaging technology in visible and near infrared (Vis/NIR) region (380-1030 nm) were introduced to assess total soluble solids (TSS) in mulberries. Hyperspectral images of 310 mulberries were acquired by hyperspectral reflectance imaging system (512 bands) and their corresponding TSS contents were measured by a Brix meter. Random frog (RF) method was used to select important wavelengths from the full wavelengths. TSS values in mulberry fruits were predicted by partial least squares regression (PLSR) and least-square support vector machine (LS-SVM) models based on full wavelengths and the selected important wavelengths. The optimal PLSR model with 23 important wavelengths was employed to visualise the spatial distribution of TSS in tested samples, and TSS concentrations in mulberries were revealed through the TSS spatial distribution. The results declared that hyperspectral imaging is promising for determining the spatial distribution of TSS content in mulberry fruits, which provides a reference for detecting the internal quality of fruits.