Comparison of 2D and 3D convolutional neural networks in hyperspectral image analysis of fruits applied to orange bruise detection

Recent advances in hyperspectral imaging (HSI) have demonstrated its ability to detect defects in fruit that may not be visible in RGB images. HSIs can be considered 3D images containing two spatial dimensions and one spectral dimension. Therefore, the first question that arises is how to process this type of information, either using 2D or 3D models. In this study, HSI in the 550-900 nm spectral range was used to detect bruising in oranges. Sixty samples of Thompson oranges were subjected to a mechanical bruising process, and HSIs were taken at different time intervals: before bruising, and 8 and 16 h after bruising. The samples were then classified using two convolutional neural network (CNN) models, a shallow 7-layer network (CNN-7) and a deep 18-layer network (CNN-18). In addition, two different input processing approaches are used: using 2D information from each band, and using the full 3D data from each HSI. The 3D models were the most accurate, with 94% correct classification for 3D-CNN-18, compared to 90% for 3D-CNN-7, and less than 83% for the 2D models. Our study suggests that 3D HSI may be a more effective technique for detecting fruit bruising, allowing the development of a fast, accurate, and nondestructive method for fruit sorting. PRACTICAL APPLICATION: Orange bruises can reduce the market value of food, which is why the food processing industry needs to carry out quality inspections. An effective way to perform this inspection is by using hyperspectral images that can be processed with 2D or 3D models, either with deep or shallow neural networks. The results of the comparison performed in this work can be useful for the development of more accurate and efficient bruise detection methods for fruit inspection.

Low-Cost Sensor for Lycopene Content Measurement in Tomato Based on Raspberry Pi 4

Measuring lycopene in tomatoes is fundamental to the agrifood industry because of its health benefits. It is one of the leading quality criteria for consuming this fruit. Traditionally, the amount determination of this carotenoid is performed using the high-performance liquid chromatography (HPLC) technique. This is a very reliable and accurate method, but it has several disadvantages, such as long analysis time, high cost, and destruction of the sample. In this sense, this work proposes a low-cost sensor that correlates the lycopene content in tomato with the color present in its epicarp. A Raspberry Pi 4 programmed with Python language was used to develop the lycopene prediction model. Various regression models were evaluated using neural networks, fuzzy logic, and linear regression. The best model was the fuzzy nonlinear regression as the RGB input, with a correlation of R² = 0.99 and a mean error of 1.9 × 10^-5. This work was able to demonstrate that it is possible to determine the lycopene content using a digital camera and a low-cost integrated system in a non-invasive way.

Recognition and Counting of Apples in a Dynamic State Using a 3D Camera and Deep Learning Algorithms for Robotic Harvesting Systems

Recognition and 3D positional estimation of apples during harvesting from a robotic platform in a moving vehicle are still challenging. Fruit clusters, branches, foliage, low resolution, and different illuminations are unavoidable and cause errors in different environmental conditions. Therefore, this research aimed to develop a recognition system based on training datasets from an augmented, complex apple orchard. The recognition system was evaluated using deep learning algorithms established from a convolutional neural network (CNN). The dynamic accuracy of the modern artificial neural networks involving 3D coordinates for deploying robotic arms at different forward-moving speeds from an experimental vehicle was investigated to compare the recognition and tracking localization accuracy. In this study, a Realsense D455 RGB-D camera was selected to acquire 3D coordinates of each detected and counted apple attached to artificial trees placed in the field to propose a specially designed structure for ease of robotic harvesting. A 3D camera, YOLO (You Only Look Once), YOLOv4, YOLOv5, YOLOv7, and EfficienDet state-of-the-art models were utilized for object detection. The Deep SORT algorithm was employed for tracking and counting detected apples using perpendicular, 15°, and 30° orientations. The 3D coordinates were obtained for each tracked apple when the on-board camera in the vehicle passed the reference line and was set in the middle of the image frame. To optimize harvesting at three different speeds (0.052 ms^-1, 0.069 ms^-1, and 0.098 ms^-1), the accuracy of 3D coordinates was compared for three forward-moving speeds and three camera angles (15°, 30°, and 90°). The mean average precision (mAP@0.5) values of YOLOv4, YOLOv5, YOLOv7, and EfficientDet were 0.84, 0.86, 0.905, and 0.775, respectively. The lowest root mean square error (RMSE) was 1.54 cm for the apples detected by EfficientDet at a 15° orientation and a speed of 0.098 ms^-1. In terms of counting apples, YOLOv5 and YOLOv7 showed a higher number of detections in outdoor dynamic conditions, achieving a counting accuracy of 86.6%. We concluded that the EfficientDet deep learning algorithm at a 15° orientation in 3D coordinates can be employed for further robotic arm development while harvesting apples in a specially designed orchard.

Spatial Frequency Domain Imaging System Calibration, Correction and Application for Pear Surface Damage Detection

Spatial frequency domain imaging (SFDI) is a non-contact wide-field optical imaging technique for optical property detection. This study aimed to establish an SFDI system and investigate the effects of system calibration, error analysis and correction on the measurement of optical properties. Optical parameter characteristic measurements of normal pears with three different damage types were performed using the calibrated system. The obtained absorption coefficient μ_a and the reduced scattering coefficient μ'_s were used for discriminating pears with different surface damage using a linear discriminant analysis model. The results showed that at 527 nm and 675 nm, the pears' quadruple classification (normal, bruised, scratched and abraded) accuracy using the SFDI technique was 92.5% and 83.8%, respectively, which has an advantage compared with the conventional planar light classification results of 82.5% and 77.5%. The three-way classification (normal, minor damage and serious damage) SFDI technique was as high as 100% and 98.8% at 527 nm and 675 nm, respectively, while the classification accuracy of conventional planar light was 93.8% and 93.8%, respectively. The results of this study indicated that SFDI has the potential to detect different damage types in fruit and that the SFDI technique has a promising future in agricultural product quality inspection in further research.

Fast 3D Rotation Estimation of Fruits Using Spheroid Models

Automated fruit inspection using cameras involves the analysis of a collection of views of the same fruit obtained by rotating a fruit while it is transported. Conventionally, each view is analyzed independently. However, in order to get a global score of the fruit quality, it is necessary to match the defects between adjacent views to prevent counting them more than once and assert that the whole surface has been examined. To accomplish this goal, this paper estimates the 3D rotation undergone by the fruit using a single camera. A 3D model of the fruit geometry is needed to estimate the rotation. This paper proposes to model the fruit shape as a 3D spheroid. The spheroid size and pose in each view is estimated from the silhouettes of all views. Once the geometric model has been fitted, a single 3D rotation for each view transition is estimated. Once all rotations have been estimated, it is possible to use them to propagate defects to neighbor views or to even build a topographic map of the whole fruit surface, thus opening the possibility to analyze a single image (the map) instead of a collection of individual views. A large effort was made to make this method as fast as possible. Execution times are under 0.5 ms to estimate each 3D rotation on a standard I7 CPU using a single core.

Integration of Partial Least Squares Regression and Hyperspectral Data Processing for the Nondestructive Detection of the Scaling Rate of Carp (Cyprinus carpio)

The scaling rate of carp is one of the most important factors restricting the automation and intelligence level of carp processing. In order to solve the shortcomings of the commonly-used manual detection, this paper aimed to study the potential of hyperspectral technology (400-1024.7 nm) in detecting the scaling rate of carp. The whole fish body was divided into three regions (belly, back, and tail) for analysis because spectral responses are different for different regions. Different preprocessing methods, including Savitzky-Golay (SG), first derivative (FD), multivariate scattering correction (MSC), and standard normal variate (SNV) were applied for spectrum pretreatment. Then, the successive projections algorithm (SPA), regression coefficient (RC), and two-dimensional correlation spectroscopy (2D-COS) were applied for selecting characteristic wavelengths (CWs), respectively. The partial least square regression (PLSR) models for scaling rate detection using full wavelengths (FWs) and CWs were established. According to the modeling results, FD-RC-PLSR, SNV-SPA-PLSR, and SNV-RC-PLSR were determined to be the optimal models for predicting the scaling rate in the back (the coefficient of determination in calibration set (R_C²) = 96.23%, the coefficient of determination in prediction set (R_P²) = 95.55%, root mean square error by calibration (RMSEC) = 6.20%, the root mean square error by prediction (RMSEP)= 7.54%, and the relative percent deviation (RPD) = 3.98), belly (R_C² = 93.44%, R_P² = 90.81%, RMSEC = 8.05%, RMSEP = 9.13%, and RPD = 3.07) and tail (R_C² = 95.34%, R_P² = 93.71%, RMSEC = 6.66%, RMSEP = 8.37%, and RPD = 3.42) regions, respectively. It can be seen that PLSR integrated with specific pretreatment and dimension reduction methods had great potential for scaling rate detection in different carp regions. These results confirmed the possibility of using hyperspectral technology in nondestructive and convenient detection of the scaling rate of carp.