Non-Destructive Detection of Strawberry Quality Using Multi-Features of Hyperspectral Imaging and Multivariate Methods

Application, challenges and future prospects of recent nondestructive techniques based on the electromagnetic spectrum in food quality and safety

Safety and quality aspects of food products have always been critical issues for the food production and processing industries. Since conventional quality measurements are laborious, time-consuming, and expensive, it is vital to develop new, fast, non-invasive, cost-effective, and direct techniques to eliminate those challenges. Recently, non-destructive techniques have been applied in the food sector to improve the quality and safety of foodstuffs. The aim of this review is an effort to list non-destructive techniques (X-ray, computer tomography, ultraviolet-visible spectroscopy, hyperspectral imaging, infrared, Raman, terahertz, nuclear magnetic resonance, magnetic resonance imaging, and ultrasound imaging) based on the electromagnetic spectrum and discuss their principle and application in the food sector. This review provides an in-depth assessment of the different non-destructive techniques used for the quality and safety analysis of foodstuffs. We also discussed comprehensively about advantages, disadvantages, challenges, and opportunities for the application of each technique and recommended some solutions and developments for future trends.

Simulation of Automatically Annotated Visible and Multi-/Hyperspectral Images Using the Helios 3D Plant and Radiative Transfer Modeling Framework

Deep learning and multimodal remote and proximal sensing are widely used for analyzing plant and crop traits, but many of these deep learning models are supervised and necessitate reference datasets with image annotations. Acquiring these datasets often demands experiments that are both labor-intensive and time-consuming. Furthermore, extracting traits from remote sensing data beyond simple geometric features remains a challenge. To address these challenges, we proposed a radiative transfer modeling framework based on the Helios 3-dimensional (3D) plant modeling software designed for plant remote and proximal sensing image simulation. The framework has the capability to simulate RGB, multi-/hyperspectral, thermal, and depth cameras, and produce associated plant images with fully resolved reference labels such as plant physical traits, leaf chemical concentrations, and leaf physiological traits. Helios offers a simulated environment that enables generation of 3D geometric models of plants and soil with random variation, and specification or simulation of their properties and function. This approach differs from traditional computer graphics rendering by explicitly modeling radiation transfer physics, which provides a critical link to underlying plant biophysical processes. Results indicate that the framework is capable of generating high-quality, labeled synthetic plant images under given lighting scenarios, which can lessen or remove the need for manually collected and annotated data. Two example applications are presented that demonstrate the feasibility of using the model to enable unsupervised learning by training deep learning models exclusively with simulated images and performing prediction tasks using real images.

UV reflectance in crop remote sensing: Assessing the current state of knowledge and extending research with strawberry cultivars

Remote sensing of spectral reflectance is a crucial parameter in precision agriculture. In particular, the visual color produced from reflected light can be used to determine plant health (VIS-IR) or attract pollinators (Near-UV). However, the UV spectral reflectance studies largely focus on non-crop plants, even though they provide essential information for plant-pollinator interactions. This literature review presents an overview of UV-reflectance in crops, identifies gaps in the literature, and contributes new data based on strawberry cultivars. The study found that most crop spectral reflectance studies relied on lab-based methodologies and examined a wide spectral range (Near UV to IR). Moreover, the plant family distribution largely mirrored global food market trends. Through a spectral comparison of white flowering strawberry cultivars, this study discovered visual differences for pollinators in the Near UV and Blue ranges. The variation in pollinator visibility within strawberry cultivars underscores the importance of considering UV spectral reflectance when developing new crop breeding lines and managing pollinator preferences in agricultural fields.

Towards the synthesis of spectral imaging and machine learning-based approaches for non-invasive phenotyping of plants

High-throughput phenotyping is now central to the progress of plant sciences, accelerated breeding, and precision farming. The power of phenotyping comes from the automated, rapid, non-invasive collection of large datasets describing plant objects. In this context, the goal of extracting relevant information from different kinds of images is of paramount importance. We review both the spectral and machine learning-based approaches to imaging of plants for the purpose of their phenotyping. The advantages and drawbacks of both approaches will be discussed with a focus on the monitoring of plants. We argue that an emerging approach combining the strengths of the spectral and the machine learning-based approaches will remain a promising direction in plant phenotyping in the nearest future.

Statistic and Network Features of RGB and Hyperspectral Imaging for Determination of Black Root Mold Infection in Apples

Apples damaged by black root mold (BRM) lose moisture, vitamins, and minerals as well as carry dangerous toxins. Determination of the infection degree can allow for customized use of apples, reduce financial losses, and ensure food safety. In this study, red-green-blue (RGB) imaging and hyperspectral imaging (HSI) are combined to detect the infection degree of BRM in apple fruits. First, RGB and HSI images of healthy, mildly, moderately, and severely infected fruits are measured, and those with effective wavelengths (EWs) are screened from HSI by random frog. Second, the statistic and network features of images are extracted by using color moment and convolutional neural network. Meanwhile, random forest (RF), K-nearest neighbor, and support vector machine are used to construct classification models with the above two features of RGB and HSI images of EWs. Optimal results with the 100% accuracy of training set and 96% accuracy of prediction set are obtained by RF with the statistic and network features of the two images, outperforming the other cases. The proposed method furnishes an accurate and effective solution for determining the BRM infection degree in apples.

"How sweet are your strawberries?": Predicting sugariness using non-destructive and affordable hardware

Global soft fruit supply chains rely on trustworthy descriptions of product quality. However, crucial criteria such as sweetness and firmness cannot be accurately established without destroying the fruit. Since traditional alternatives are subjective assessments by human experts, it is desirable to obtain quality estimations in a consistent and non-destructive manner. The majority of research on fruit quality measurements analyzed fruits in the lab with uniform data collection. However, it is laborious and expensive to scale up to the level of the whole yield. The "harvest-first, analysis-second" method also comes too late to decide to adjust harvesting schedules. In this research, we validated our hypothesis of using in-field data acquirable via commodity hardware to obtain acceptable accuracies. The primary instance that the research concerns is the sugariness of strawberries, described by the juice's total soluble solid (TSS) content (unit: °Brix or Brix). We benchmarked the accuracy of strawberry Brix prediction using convolutional neural networks (CNN), variational autoencoders (VAE), principal component analysis (PCA), kernelized ridge regression (KRR), support vector regression (SVR), and multilayer perceptron (MLP), based on fusions of image data, environmental records, and plant load information, etc. Our results suggest that: (i) models trained by environment and plant load data can perform reliable prediction of aggregated Brix values, with the lowest RMSE at 0.59; (ii) using image data can further supplement the Brix predictions of individual fruits from (i), from 1.27 to as low up to 1.10, but they by themselves are not sufficiently reliable.

Visualization of Sugar Content Distribution of White Strawberry by Near-Infrared Hyperspectral Imaging

In this study, an approach to visualize the spatial distribution of sugar content in white strawberry fruit flesh using near-infrared hyperspectral imaging (NIR-HSI; 913-2166 nm) is developed. NIR-HSI data collected from 180 samples of "Tochigi iW1 go" white strawberries are investigated. In order to recognize the pixels corresponding to the flesh and achene on the surface of the strawberries, principal component analysis (PCA) and image processing are conducted after smoothing and standard normal variate (SNV) pretreatment of the data. Explanatory partial least squares regression (PLSR) analysis is performed to develop an appropriate model to predict Brix reference values. The PLSR model constructed from the raw spectra extracted from the flesh region of interest yields high prediction accuracy with an RMSEP and R2p values of 0.576 and 0.841, respectively, and with a relatively low number of PLS factors. The Brix heatmap images and violin plots for each sample exhibit characteristics feature of sugar content distribution in the flesh of the strawberries. These findings offer insights into the feasibility of designing a noncontact system to monitor the quality of white strawberries.

Low temperature response index for monitoring freezing injury of tea plant

Freezing damage has been a common natural disaster for tea plantations. Quantitative detection of low temperature stress is significant for evaluating the degree of freezing injury to tea plants. Traditionally, the determination of physicochemical parameters of tea leaves and the investigation of freezing damage phenotype are the main approaches to detect the low temperature stress. However, these methods are time-consuming and laborious. In this study, different low temperature treatments were carried out on tea plants. The low temperature response index (LTRI) was established by measuring seven low temperature-induced components of tea leaves. The hyperspectral data of tea leaves was obtained by hyperspectral imaging and the feature bands were screened by successive projections algorithm (SPA), competitive adaptive reweighted sampling (CARS) and uninformative variable elimination (UVE). The LTRI and seven indexes of tea plant were modeled by partial least squares (PLS), support vector machine (SVM), random forests (RF), back propagation (BP) machine learning methods and convolutional neural networks (CNN), long short-term memory (LSTM) deep learning methods. The results indicated that: (1) the best prediction model for the seven indicators was LTRI-UVE-CNN (R^{2 =} 0.890, RMSEP=0.325, RPD=2.904); (2) the feature bands screened by UVE algorithm were more abundant, and the later modeling effect was better than CARS and SPA algorithm; (3) comparing the effects of the six modeling algorithms, the overall modeling effect of the CNN model was better than other models. It can be concluded that out of all the combined models in this paper, the LTRI-UVE-CNN was a promising model for predicting the degree of low temperature stress in tea plants.

Advances in Machine Learning and Hyperspectral Imaging in the Food Supply Chain

Food quality and safety are the essential hot issues of social concern. In recent years, there has been a growing demand for real-time food information, and non-destructive testing is gradually replacing traditional manual sensory testing and chemical analysis methods with lagging and destructive effects and has strong potential for application in the food supply chain. With the maturity and development of computer science and spectroscopic techniques, machine learning and hyperspectral imaging (HSI) have been widely demonstrated as efficient detection techniques that can be applied to rapidly evaluate sensory characteristics and quality attributes of food products nondestructively and efficiently. This paper first briefly described the basic concepts of hyperspectral imaging and machine learning, including the imaging process of HSI, the type of algorithms contained in machine learning, and the data processing flow. Secondly, this paper provided an objective and comprehensive overview of the current applications of machine learning and HSI in the food supply chain for sorting, packaging, transportation, storage, and sales, based on the state-of-art literature from 2017 to 2022. Finally, the potential of the technology is further discussed to provide optimized ideas for practical application.

Prediction of Total Soluble Solids and pH of Strawberry Fruits Using RGB, HSV and HSL Colour Spaces and Machine Learning Models

Determination of internal qualities such as total soluble solids (TSS) and pH is a paramount concern in strawberry cultivation. Therefore, the main objective of the current study was to develop a non-destructive approach with machine learning algorithms for predicting TSS and pH of strawberries. Six hundred samples (100 samples in each ripening stage) in six ripening stages were collected randomly for measuring the biometrical characteristics, i.e., length, diameters, weight and TSS and pH values. An image of each strawberry fruit was captured for colour feature extraction using an image processing technique. Channels of each colour space (RGB, HSV and HSL) were used as input variables for developing multiple linear regression (MLR) and support vector machine regression (SVM-R) models. The result of the study indicated that SVM-R model with HSV colour space performed slightly better than MLR model for TSS and pH prediction. The HSV based SVM-R model could explain a maximum of 84.1% and 79.2% for TSS and 78.8% and 72.6% for pH of the variations in measured and predicted data in training and testing stages, respectively. Further experiments need to be conducted with different strawberry cultivars for the prediction of more internal qualities along with the improvement of model performance.

Rapid and Non-Destructive Techniques for the Discrimination of Ripening Stages in Candonga Strawberries

Electronic nose (e-nose), attenuated total reflection-Fourier transform infrared (ATR-FTIR) spectroscopy and image analysis (IA) were used to discriminate the ripening stage (half-red or red) of strawberries (cv Sabrosa, commercially named Candonga), harvested at three different times (H1, H2 and H3). Principal component analysis (PCA) performed on the e-nose, ATR-FTIR and IA data allowed us to clearly discriminate samples based on the ripening stage, as in the score space they clustered in distinct regions of the plot. Moreover, a correlation analysis between the e-nose sensor and 57 volatile organic compounds (VOCs), which were overall detected in all the investigated fruit samples by headspace solid-phase microextraction coupled to gas chromatography-mass spectrometry (HS-SPME/GC-MS), allowed us to distinguish half-red and red strawberries, as the e-nose sensors gave distinct responses to samples with different flavours. Three suitable broad bands were individuated by PCA in the ATR-FTIR spectra to discriminate half-red and red samples: the band centred at 3295 cm⁻¹ is generated by compounds that decline, whereas those at 1717 cm⁻¹ and at 1026 cm⁻¹ stem from compounds that accumulate during ripening. Among the chemical parameters (titratable acidity, total phenols, antioxidant activity and total soluble solid) assayed in this study, only titratable acidity was somehow correlated to ATR-FTIR and IA patterns. Thus, ATR-FTIR spectroscopy and IA might be exploited to rapidly assess titratable acidity, which is an objective indicator of the ripening stage.

Rapid Foreign Object Detection System on Seaweed Using VNIR Hyperspectral Imaging

The consumption of seaweed is increasing year by year worldwide. Therefore, the foreign object inspection of seaweed is becoming increasingly important. Seaweed is mixed with various materials such as laver and sargassum fusiforme. So it has various colors even in the same seaweed. In addition, the surface is uneven and greasy, causing diffuse reflections frequently. For these reasons, it is difficult to detect foreign objects in seaweed, so the accuracy of conventional foreign object detectors used in real manufacturing sites is less than 80%. Supporting real-time inspection should also be considered when inspecting foreign objects. Since seaweed requires mass production, rapid inspection is essential. However, hyperspectral imaging techniques are generally not suitable for high-speed inspection. In this study, we overcome this limitation by using dimensionality reduction and using simplified operations. For accuracy improvement, the proposed algorithm is carried out in 2 stages. Firstly, the subtraction method is used to clearly distinguish seaweed and conveyor belts, and also detect some relatively easy to detect foreign objects. Secondly, a standardization inspection is performed based on the result of the subtraction method. During this process, the proposed scheme adopts simplified and burdenless calculations such as subtraction, division, and one-by-one matching, which achieves both accuracy and low latency performance. In the experiment to evaluate the performance, 60 normal seaweeds and 60 seaweeds containing foreign objects were used, and the accuracy of the proposed algorithm is 95%. Finally, by implementing the proposed algorithm as a foreign object detection platform, it was confirmed that real-time operation in rapid inspection was possible, and the possibility of deployment in real manufacturing sites was confirmed.

Remote Sensing and Machine Learning in Crop Phenotyping and Management, with an Emphasis on Applications in Strawberry Farming

Measurement of plant characteristics is still the primary bottleneck in both plant breeding and crop management. Rapid and accurate acquisition of information about large plant populations is critical for monitoring plant health and dissecting the underlying genetic traits. In recent years, high-throughput phenotyping technology has benefitted immensely from both remote sensing and machine learning. Simultaneous use of multiple sensors (e.g., high-resolution RGB, multispectral, hyperspectral, chlorophyll fluorescence, and light detection and ranging (LiDAR)) allows a range of spatial and spectral resolutions depending on the trait in question. Meanwhile, computer vision and machine learning methodology have emerged as powerful tools for extracting useful biological information from image data. Together, these tools allow the evaluation of various morphological, structural, biophysical, and biochemical traits. In this review, we focus on the recent development of phenomics approaches in strawberry farming, particularly those utilizing remote sensing and machine learning, with an eye toward future prospects for strawberries in precision agriculture. The research discussed is broadly categorized according to strawberry traits related to (1) fruit/flower detection, fruit maturity, fruit quality, internal fruit attributes, fruit shape, and yield prediction; (2) leaf and canopy attributes; (3) water stress; and (4) pest and disease detection. Finally, we present a synthesis of the potential research opportunities and directions that could further promote the use of remote sensing and machine learning in strawberry farming.

Application of Hyperspectral Imaging for Maturity and Soluble Solids Content Determination of Strawberry With Deep Learning Approaches

Maturity degree and quality evaluation are important for strawberry harvest, trade, and consumption. Deep learning has been an efficient artificial intelligence tool for food and agro-products. Hyperspectral imaging coupled with deep learning was applied to determine the maturity degree and soluble solids content (SSC) of strawberries with four maturity degrees. Hyperspectral image of each strawberry was obtained and preprocessed, and the spectra were extracted from the images. One-dimension residual neural network (1D ResNet) and three-dimension (3D) ResNet were built using 1D spectra and 3D hyperspectral image as inputs for maturity degree evaluation. Good performances were obtained for maturity identification, with the classification accuracy over 84% for both 1D ResNet and 3D ResNet. The corresponding saliency maps showed that the pigments related wavelengths and image regions contributed more to the maturity identification. For SSC determination, 1D ResNet model was also built, with the determination of coefficient (R ²) over 0.55 of the training, validation, and testing sets. The saliency maps of 1D ResNet for the SSC determination were also explored. The overall results showed that deep learning could be used to identify strawberry maturity degree and determine SSC. More efforts were needed to explore the use of 3D deep learning methods for the SSC determination. The close results of 1D ResNet and 3D ResNet for classification indicated that more samples might be used to improve the performances of 3D ResNet. The results in this study would help to develop 1D and 3D deep learning models for fruit quality inspection and other researches using hyperspectral imaging, providing efficient analysis approaches of fruit quality inspection using hyperspectral imaging.