Recent Advances in the Application of Hyperspectral Imaging for Evaluating Fruit Quality

Learning to Recover Spectral Reflectance From RGB Images

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL.

Computer vision based deep learning approach for toxic and harmful substances detection in fruits

Formaldehyde (CH₂O) is one of the significant chemicals mixed with different perishable fruits in Bangladesh. The fruits are artificially preserved for extended periods by dishonest vendors using this dangerous chemical. Such substances are complicated to detect in appearance. Hence, a reliable and robust detection technique is required. To overcome this challenge and address the issue, we introduce comprehensive deep learning-based techniques for detecting toxic substances. Four different types of fruits, both in fresh and chemically mixed conditions, are used in this experiment. We have applied diverse data augmentation techniques to enlarge the dataset. The performance of four different pre-trained deep learning models was then assessed, and a brand-new model named "DurbeenNet," created especially for this task, was presented. The primary objective was to gauge the efficacy of our proposed model compared to well-established deep learning architectures. Our assessment centered on the models' accuracy in detecting toxic substances. According to our research, GoogleNet detected toxic substances with an accuracy rate of 85.53 %, VGG-16 with an accuracy rate of 87.44 %, DenseNet with an impressive accuracy rate of 90.37 %, and ResNet50 with an accuracy rate of 91.66 %. Notably, the proposed model, DurbeenNet, outshone all other models, boasting an impressive accuracy rate of 96.71 % in detecting toxic substances among the sample fruits.

Gray mold and anthracnose disease detection on strawberry leaves using hyperspectral imaging

BACKGROUND

Gray mold and anthracnose are the main factors affecting strawberry quality and yield. Accurate and rapid early disease identification is of great significance to achieve precise targeted spraying to avoid large-scale spread of diseases and improve strawberry yield and quality. However, the characteristics between early disease infected and healthy leaves are very similar, making the early identification of strawberry gray mold and anthracnose still a challenge.

RESULTS

Based on hyperspectral imaging technology, this study explored the potential of combining spectral fingerprint features and vegetation indices (VIs) for early detection (24-h infected) of strawberry leaves diseases. The competitive adaptive reweighted sampling (CARS) algorithm and ReliefF algorithm were used for the extraction of spectral fingerprint features and VIs, respectively. Three machine learning models, Backpropagation Neural Network (BPNN), Support Vector Machine (SVM) and Random Forest (RF), were developed for the early identification of strawberry gray mold and anthracnose, using spectral fingerprint, VIs and their combined features as inputs respectively. The results showed that the combination of spectral fingerprint features and VIs had better recognition accuracy compared with individual features as inputs, and the accuracies of the three classifiers (BPNN, SVM and RF) were 97.78%, 94.44%, and 93.33%, respectively, which indicate that the fusion features approach proposed in this study can effectively improve the early detection performance of strawberry leaves diseases.

CONCLUSIONS

This study provided an accurate, rapid, and nondestructive recognition of strawberry gray mold and anthracnose disease in early stage.

Non-invasive assessment of cultivar and sex of Cannabis sativa L. by means of hyperspectral measurement

Cannabis sativa L. is a versatile crop attracting increasing attention for food, fiber, and medical uses. As a dioecious species, males and females are visually indistinguishable during early growth. For seed or cannabinoid production, a higher number of female plants is economically advantageous. Currently, sex determination is labor-intensive and costly. Instead, we used rapid and non-destructive hyperspectral measurement, an emerging means of assessing plant physiological status, to reliably differentiate males and females. One industrial hemp (low tetrahydrocannabinol [THC]) cultivar was pre-grown in trays before transfer to the field in control soil. Reflectance spectra were acquired from leaves during flowering and machine learning algorithms applied allowed sex classification, which was best using a radial basis function (RBF) network. Eight industrial hemp (low THC) cultivars were field grown on fertilized and control soil. Reflectance spectra were acquired from leaves at early development when the plants of all cultivars had developed between four and six leaf pairs and in three cases only flower buds were visible (start of flowering). Machine learning algorithms were applied, allowing sex classification, differentiation of cultivars and fertilizer regime, again with best results for RBF networks. Differentiating nutrient status and varietal identity is feasible with high prediction accuracy. Sex classification was error-free at flowering but less accurate (between 60% and 87%) when using spectra from leaves at early growth stages. This was influenced by both cultivar and soil conditions, reflecting developmental differences between cultivars related to nutritional status. Hyperspectral measurement combined with machine learning algorithms is valuable for non-invasive assessment of C. sativa cultivar and sex. This approach can potentially improve regulatory security and productivity of cannabis farming.

Towards a Multispectral Imaging System for Spatial Mapping of Chemical Composition in Fresh-Cut Pineapple (Ananas comosus)

With increasing public demand for ready-to-eat fresh-cut fruit, the postharvest industry requires the development and adaptation of monitoring technologies to provide customers with a product of consistent quality. The fresh-cut trade of pineapples (Ananas comosus) is on the rise, favored by the sensory quality of the product and mechanization of the cutting process. In this paper, a multispectral imaging-based approach is introduced to provide distribution maps of moisture content, soluble solids content, and carotenoids content in fresh-cut pineapple. A dataset containing hyperspectral images (380-1690 nm) and reference measurements in 10 regions of interest of 60 fruit (n = 600) was prepared. Ranking and uncorrelatedness (based on ReliefF algorithm) and subset selection (based on CfsSubset algorithm) approaches were applied to find the most informative wavelengths in which bandpass optical filters or light sources are commercially available. The correlation coefficient and error metrics obtained by cross-validated multilayer perceptron neural network models indicated that the superior selected wavelengths (495, 500, 505, 1215, 1240, and 1425 nm) resulted in prediction of moisture content with R = 0.56, MAPE = 1.92%, soluble solids content with R = 0.52, MAPE = 14.72%, and carotenoids content with R = 0.63, MAPE = 43.99%. Prediction of chemical composition in each pixel of the multispectral images using the calibration models yielded spatially distributed quantification of the fruit slice, spatially varying according to the maturation of single fruitlets in the whole pineapple. Calibration models provided reliable responses spatially throughout the surface of fresh-cut pineapple slices with a constant error. According to the approach to use commercially relevant wavelengths, calibration models could be applied in classifying fruit segments in the mechanized preparation of fresh-cut produce.

Field Application of a Vis/NIR Hyperspectral Imaging System for Nondestructive Evaluation of Physicochemical Properties in ‘Madoka’ Peaches

Extensive research has been performed on the in-field nondestructive evaluation (NDE) of the physicochemical properties of ‘Madoka’ peaches, such as chromaticity (a*), soluble solids content (SSC), firmness, and titratable acidity (TA) content. To accomplish this, a snapshot-based hyperspectral imaging (HSI) approach for filed application was conducted in the visible and near-infrared (Vis/NIR) region. The hyperspectral images of ‘Madoka’ samples were captured and combined with commercial HSI analysis software, and then the physicochemical properties of the ‘Madoka’ samples were predicted. To verify the performance of the field-based HSI application, a lab-based HSI application was also conducted, and their coefficient of determination values (R²) were compared. Finally, pixel-based chemical images were produced to interpret the dynamic changes of the physicochemical properties in ‘Madoka’ peach. Consequently, the a* values and SSC content shows statistically significant R² values (0.84). On the other hand, the firmness and TA content shows relatively lower accuracy (R² = 0.6 to 0.7). Then, the resultant chemical images of the a* values and SSC content were created and could represent their different levels using grey scale gradation. This indicates that the HSI system with integrated HSI software used in this work has promising potential as an in-field NDE for analyzing the physicochemical properties in ‘Madoka’ peaches.

Nondestructive measurement of pectin polysaccharides using hyperspectral imaging in mulberry fruit

Pectin polysaccharide is an important phytochemical with potential biomedical applications. It is commonly measured by time-consuming destructive chemical methods. This work demonstrates the feasibility of using visible and near-infrared hyperspectral imaging (HSI) techniques to rapidly measure pectin polysaccharides in intact mulberry fruits. Based on spatial information provided by HSI images, the representative spectrum of each whole mulberry was accurately extracted without background. The effects of storage temperature on two varieties of mulberries for model establishment were studied. The performances of two spectral ranges obtained by Si and InGaAs CCD detectors for pectin prediction were compared. The best predictions were obtained from dilute alkali soluble pectin and total soluble pectin in Dashi mulberry fruit stored at room temperature, with residual predictive deviation values of 2.317 and 1.935, respectively. Our results show that HSI is a promising alternative to the chemical method to rapidly and nondestructively measure the pectin content.

Evaluation of image processing technique and discriminant analysis methods in postharvest processing of carrot fruit

The most important process before packaging and preserving agricultural products is sorting operation. Sort of carrot by human labor is involved in many problems such as high cost and product waste. Image processing is a modern method, which has different applications in agriculture including classification and sorting. The aim of this study was to classify carrot based on shape using image processing technique. For this, 135 samples with different regular and irregular shapes were selected. After image acquisition and preprocessing, some features such as length, width, breadth, perimeter, elongation, compactness, roundness, area, eccentricity, centroid, centroid nonhomogeneity, and width nonhomogeneity were extracted. After feature selection, linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) methods were used to classify the features. The classification accuracies of the methods were 92.59 and 96.30, respectively. It can be stated that image processing is an effective way in improving the traditional carrot sorting techniques.

Hyperspectral Imaging System with Rotation Platform for Investigation of Jujube Skin Defects

A novel object rotation hyperspectral imaging system with the wavelength range of 468–950 nm for investigating round-shaped fruits was developed. This system was used to obtain the reflection spectra of jujubes for the application of surface defect detection. Compared to the traditional linear scan system, which can scan about 49% of jujube surface in one scan pass, this novel object rotation scan system can scan 95% of jujube surface in one scan pass. Six types of jujube skin condition, including rusty spots, decay, white fungus, black fungus, cracks, and glare, were classified by using hyperspectral data. Support vector machine (SVM) and artificial neural network (ANN) models were used to differentiate the six jujube skin conditions. Classification effectiveness of models was evaluated based on confusion matrices. The percentage of classification accuracy of SVM and ANN models were 97.3% and 97.4%, respectively. The object rotation scan method developed for this study could be used for other round-shaped fruits and integrated into online hyperspectral investigation systems.

Hyperspectral Imaging for Evaluating Impact Damage to Mango According to Changes in Quality Attributes

Evaluation of impact damage to mango (Mangifera indica Linn) as a result of dropping from three different heights, namely, 0.5, 1.0 and 1.5 m, was conducted by hyperspectral imaging (HSI). Reflectance spectra in the 900–1700 nm region were used to develop prediction models for pulp firmness (PF), total soluble solids (TSS), titratable acidity (TA) and chroma (∆b*) by a partial least squares (PLS) regression algorithm. The results showed that the changes in the mangoes’ quality attributes, which were also reflected in the spectra, had a strong relationship with dropping height. The best predictive performance measured by coefficient of determination (R²) and root mean square errors of prediction (RMSEP) values were: 0.84 and 31.6 g for PF, 0.9 and 0.49 ^oBrix for TSS, 0.65 and 0.1% for TA, 0.94 and 0.96 for chroma, respectively. Classification of the degree of impact damage to mango achieved an accuracy of more than 77.8% according to ripening index (RPI). The results show the potential of HSI to evaluate impact damage to mango by combining with changes in quality attributes.

Categorical Nature of Consumer Price Estimations of Postharvest Bruised Apples

Postharvest fruits are susceptible to damage which eventually results in large product and financial losses. While abundant studies have been conducted to objectively index the severity of such damage, how consumers subjectively assess the severity of damaged apples has been understudied. Previous studies have indicated that consumers’ aesthetic devaluation of product quality is reflected in estimated price. Thus, the current online questionnaire study was conducted to examine the effect of objectively indexed severity of damage on consumers’ subjective price estimations. Four hundred thirty-nine consumers of apples were asked to estimate the market price for apples in photographic images of 1 or 3 “Orin” (“Golden Delicious” × “Indo”) apples at 9 levels of severity of damage. A 2 (1- and 3-piece) × 9 (severity of damage) within ANCOVA with reference price as a covariate indicated significant two-way interaction between the number of apples and severity of damage on estimated price. Consequently, the 1- and 3-piece conditions were examined separately. The results of both analyses indicate a categorical rather than quantitative, continuous reduction in estimated price.

Hyperspectral Imaging for Predicting the Internal Quality of Kiwifruits Based on Variable Selection Algorithms and Chemometric Models

We investigated the feasibility and potentiality of determining firmness, soluble solids content (SSC), and pH in kiwifruits using hyperspectral imaging, combined with variable selection methods and calibration models. The images were acquired by a push-broom hyperspectral reflectance imaging system covering two spectral ranges. Weighted regression coefficients (BW), successive projections algorithm (SPA) and genetic algorithm-partial least square (GAPLS) were compared and evaluated for the selection of effective wavelengths. Moreover, multiple linear regression (MLR), partial least squares regression and least squares support vector machine (LS-SVM) were developed to predict quality attributes quantitatively using effective wavelengths. The established models, particularly SPA-MLR, SPA-LS-SVM and GAPLS-LS-SVM, performed well. The SPA-MLR models for firmness (R _pre  = 0.9812, RPD = 5.17) and SSC (R _pre  = 0.9523, RPD = 3.26) at 380-1023 nm showed excellent performance, whereas GAPLS-LS-SVM was the optimal model at 874-1734 nm for predicting pH (R _pre  = 0.9070, RPD = 2.60). Image processing algorithms were developed to transfer the predictive model in every pixel to generate prediction maps that visualize the spatial distribution of firmness and SSC. Hence, the results clearly demonstrated that hyperspectral imaging has the potential as a fast and non-invasive method to predict the quality attributes of kiwifruits.