On line detection of defective apples using computer vision system combined with deep learning methods

Smartphone video imaging: A versatile, low-cost technology for food authentication

This study presents a low-cost smartphone-based imaging technique called smartphone video imaging (SVI) to capture short videos of samples that are illuminated by a colour-changing screen. Assisted by artificial intelligence, the study develops new capabilities to make SVI a versatile imaging technique such as the hyperspectral imaging (HSI). SVI enables classification of samples with heterogeneous contents, spatial representation of analyte contents and reconstruction of hyperspectral images from videos. When integrated with a residual neural network, SVI outperforms traditional computer vision methods for ginseng classification. Moreover, the technique effectively maps the spatial distribution of saffron purity in powder mixtures with predictive performance that is comparable to that of HSI. In addition, SVI combined with the U-Net deep learning module can produce high-quality images that closely resemble the target images acquired by HSI. These results suggest that SVI can serve as a consumer-oriented solution for food authentication.

Predicting the properties of wheat flour from grains during debranning: A machine learning approach

In the food industry, meeting food quality demands is challenging. The quality of wheat flour, one of the most commonly used ingredients, depends on the extent of debranning done to remove the aleurone layer before milling. Therefore, the end product management can be simplified by predicting the properties of wheat flour during the debranning stage. Therefore, the chemical and rheological properties of grains were analyzed at different debranning durations (0, 30, 60 s). Then the images of wheat grain were taken to develop a regression model for predicting the chemical quality (i.e., ash, starch, fat, and protein contents) of the wheat flour. The resulting regression model comprises a convolutional neural network and is evaluated using the coefficient of determination (R 2), root-mean-square error, and mean absolute error as metrics. The results demonstrated that wheat flour contained more fat and protein and less ash with increasing debranning time. The model proved reliable in terms of root-mean-square error, mean absolute error, and R 2 for predicting ash content but not starch, fat, or protein contents, which can be attributed to the lack of features in the collected images of wheat kernels during debranning. In addition, the selected method, debranning, was beneficial to the rheological characteristics of wheat flour. The proportion of fine particles increased with the debranning time. The study experimentally revealed that the end product diversity for wheat flour can be controlled to provide selectable ingredients to customers.

The grading detection model for fingered citron slices (citrus medica 'fingered') based on YOLOv8-FCS

Introduction

Fingered citron slices possess significant nutritional value and economic advantages as herbal products that are experiencing increasing demand. The grading of fingered citron slices plays a crucial role in the marketing strategy to maximize profits. However, due to the limited adoption of standardization practices and the decentralized structure of producers and distributors, the grading process of fingered citron slices requires substantial manpower and lead to a reduction in profitability. In order to provide authoritative, rapid and accurate grading standards for the market of fingered citron slices, this paper proposes a grading detection model for fingered citron slices based on improved YOLOv8n.

Methods

Firstly, we obtained the raw materials of fingered citron slices from a dealer of Sichuan fingered citron origin in Shimian County, Ya'an City, Sichuan Province, China. Subsequently, high-resolution fingered citron slices images were taken using an experimental bench, and the dataset for grading detection of fingered citron slices was formed after manual screening and labelling. Based on this dataset, we chose YOLOv8n as the base model, and then replaced the YOLOv8n backbone structure with the Fasternet main module to improve the computational efficiency in the feature extraction process. Then we redesigned the PAN-FPN structure used in the original model with BiFPN structure to make full use of the high-resolution features to extend the sensory field of the model while balancing the computation amount and model volume, and finally we get the improved target detection algorithm YOLOv8-FCS.

Results

The findings from the experiments indicated that this approach surpassed the conventional RT-DETR, Faster R-CNN, SSD300 and YOLOv8n models in most evaluation indicators. The experimental results show that the grading accuracy of the YOLOv8-FCS model reaches 98.1%, and the model size is only 6.4 M, and the FPS is 130.3.

Discussion

The results suggest that our model offers both rapid and precise grading for fingered citron slices, holding significant practical value for promoting the advancement of automated grading systems tailored to fingered citron slices.

Robust deep learning method for fruit decay detection and plant identification: enhancing food security and quality control

This paper presents a robust deep learning method for fruit decay detection and plant identification. By addressing the limitations of previous studies that primarily focused on model accuracy, our approach aims to provide a more comprehensive solution that considers the challenges of robustness and limited data scenarios. The proposed method achieves exceptional accuracy of 99.93%, surpassing established models. In addition to its exceptional accuracy, the proposed method highlights the significance of robustness and adaptability in limited data scenarios. The proposed model exhibits strong performance even under the challenging conditions, such as intense lighting variations and partial image obstructions. Extensive evaluations demonstrate its robust performance, generalization ability, and minimal misclassifications. The inclusion of Class Activation Maps enhances the model's capability to identify distinguishing features between fresh and rotten fruits. This research has significant implications for fruit quality control, economic loss reduction, and applications in agriculture, transportation, and scientific research. The proposed method serves as a valuable resource for fruit and plant-related industries. It offers precise adaptation to specific data, customization of the network architecture, and effective training even with limited data. Overall, this research contributes to fruit quality control, economic loss reduction, and waste minimization.

Technological tools for the measurement of sensory characteristics in food: A review

The use of technological tools, in the food industry, has allowed a quick and reliable identification and measurement of the sensory characteristics of food matrices is of great importance, since they emulate the functioning of the five senses (smell, taste, sight, touch, and hearing). Therefore, industry and academia have been conducting research focused on developing and using these instruments which is evidenced in various studies that have been reported in the scientific literature. In this review, several of these technological tools are documented, such as the e-nose, e-tongue, colorimeter, artificial vision systems, and instruments that allow texture measurement (texture analyzer, electromyography, others). These allow us to carry out processes of analysis, review, and evaluation of food to determine essential characteristics such as quality, composition, maturity, authenticity, and origin. The determination of these characteristics allows the standardization of food matrices, achieving the improvement of existing foods and encouraging the development of new products that satisfy the sensory experiences of the consumer, driving growth in the food sector. However, the tools discussed have some limitations such as acquisition cost, calibration and maintenance cost, and in some cases, they are designed to work with a specific food matrix.

Recent developments of e-sensing devices coupled to data processing techniques in food quality evaluation: a critical review

A greater demand for high-quality food is being driven by the growth of economic and technological advancements. In this context, consumers are currently paying special attention to organoleptic characteristics such as smell, taste, and appearance. Motivated to mimic human senses, scientists developed electronic devices such as e-noses, e-tongues, and e-eyes, to spot signals relative to different chemical substances prevalent in food systems. To interpret the information provided by the sensors' responses, multiple chemometric approaches are used depending on the aim of the study. This review based on the Web of Science database, endeavored to scrutinize three e-sensing systems coupled to chemometric approaches for food quality evaluation. A total of 122 eligible articles pertaining to the e-nose, e-tongue and e-eye devices were selected to conduct this review. Most of the performed studies used exploratory analysis based on linear factorial methods, while classification and regression techniques came in the second position. Although their applications have been less common in food science, it is to be noted that nonlinear approaches based on artificial intelligence and machine learning deployed in a big-data context have generally yielded better results for classification and regression purposes, providing new perspectives for future studies.

Quasi Real-Time Apple Defect Segmentation Using Deep Learning

Defect segmentation of apples is an important task in the agriculture industry for quality control and food safety. In this paper, we propose a deep learning approach for the automated segmentation of apple defects using convolutional neural networks (CNNs) based on a U-shaped architecture with skip-connections only within the noise reduction block. An ad-hoc data synthesis technique has been designed to increase the number of samples and at the same time to reduce neural network overfitting. We evaluate our model on a dataset of multi-spectral apple images with pixel-wise annotations for several types of defects. In this paper, we show that our proposal outperforms in terms of segmentation accuracy general-purpose deep learning architectures commonly used for segmentation tasks. From the application point of view, we improve the previous methods for apple defect segmentation. A measure of the computational cost shows that our proposal can be employed in real-time (about 100 frame-per-second on GPU) and in quasi-real-time (about 7/8 frame-per-second on CPU) visual-based apple inspection. To further improve the applicability of the method, we investigate the potential of using only RGB images instead of multi-spectral images as input images. The results prove that the accuracy in this case is almost comparable with the multi-spectral case.

Development of effective model for non-destructive detection of defective kiwifruit based on graded lines

The accurate detection of external defects in kiwifruit is an important part of postharvest quality assessment. Previous studies have not considered the problems posed by the actual grading environment. In this study, we designed a novel approach based on improved Yolov5 to achieve real-time and efficient non-destructive detection of multiple defect categories in kiwifruit. First, a kiwifruit image acquisition device based on grading lines was developed to enhance the image acquisition. Subsequently, a kiwifruit dataset was constructed based on the external defect characteristics and a new data enhancement method was proposed to augment the kiwifruit samples. Thereafter, the SPD-Conv and DW-Conv modules were combined to improve Yolov5s, with EIOU as the loss calculation function. The results demonstrated that the improved model training loss value was 0.013 lower, the convergence was accelerated, the number of parameters was reduced, and the computational effort was increased. The detection accuracies of the samples in the test set, which included healthy, leaf-rubbing damaged, healed cuts or scarred, and sunburned samples, were 98.8%, 98.7%, 97.6%, and 95.9%, respectively, with an overall detection accuracy of 97.7%. The detection time was 8.0 ms, thereby meeting real-time sorting demands. The average detection accuracy and model size of SSD, Yolov5s, Yolov7, and Yolov5-Ours were compared. When the confidence threshold was 0.5, the detection accuracy of Yolov5-Ours was 10% and 6.4% higher than that of SSD and Yolov5s, respectively. In terms of the model size, Yolov5-Ours was approximately 6.5- and 4-fold smaller than SSD and Yolov7, respectively. Thus, Yolov5-Ours achieved the highest accuracy, adaptability, and robustness for the detection of all kiwifruit categories as well as a small volume and portability. These results can provide technical support for the non-destructive detection and grading of agricultural products in the future.

An efficient convolutional neural network-based diagnosis system for citrus fruit diseases

Introduction: Fruit diseases have a serious impact on fruit production, causing a significant drop in economic returns from agricultural products. Due to its excellent performance, deep learning is widely used for disease identification and severity diagnosis of crops. This paper focuses on leveraging the high-latitude feature extraction capability of deep convolutional neural networks to improve classification performance. Methods: The proposed neural network is formed by combining the Inception module with the current state-of-the-art EfficientNetV2 for better multi-scale feature extraction and disease identification of citrus fruits. The VGG is used to replace the U-Net backbone to enhance the segmentation performance of the network. Results: Compared to existing networks, the proposed method achieved recognition accuracy of over 95%. In addition, the accuracies of the segmentation models were compared. VGG-U-Net, a network generated by replacing the backbone of U-Net with VGG, is found to have the best segmentation performance with an accuracy of 87.66%. This method is most suitable for diagnosing the severity level of citrus fruit diseases. In the meantime, transfer learning is applied to improve the training cycle of the network model, both in the detection and severity diagnosis phases of the disease. Discussion: The results of the comparison experiments reveal that the proposed method is effective in identifying and diagnosing the severity of citrus fruit diseases identification.

Research on Apple Recognition Algorithm in Complex Orchard Environment Based on Deep Learning

In the complex environment of orchards, in view of low fruit recognition accuracy, poor real-time and robustness of traditional recognition algorithms, this paper propose an improved fruit recognition algorithm based on deep learning. Firstly, the residual module was assembled with the cross stage parity network (CSP Net) to optimize recognition performance and reduce the computing burden of the network. Secondly, the spatial pyramid pool (SPP) module is integrated into the recognition network of the YOLOv5 to blend the local and global features of the fruit, thus improving the recall rate of the minimum fruit target. Meanwhile, the NMS algorithm was replaced by the Soft NMS algorithm to enhance the ability of identifying overlapped fruits. Finally, a joint loss function was constructed based on focal and CIoU loss to optimize the algorithm, and the recognition accuracy was significantly improved. The test results show that the MAP value of the improved model after dataset training reaches 96.3% in the test set, which is 3.8% higher than the original model. F1 value reaches 91.8%, which is 3.8% higher than the original model. The average detection speed under GPU reaches 27.8 frames/s, which is 5.6 frames/s higher than the original model. Compared with current advanced detection methods such as Faster RCNN and RetinaNet, among others, the test results show that this method has excellent detection accuracy, good robustness and real-time performance, and has important reference value for solving the problem of accurate recognition of fruit in complex environment.

Apple Grading Based on Multi-Dimensional View Processing and Deep Learning

This research proposes an apple quality grading approach based on multi-dimensional view information processing using YOLOv5s network as the framework to rapidly and accurately perform the apple quality grading task. The Retinex algorithm is employed initially to finish picture improvement. Then, the YOLOv5s model, which is improved by adding ODConv dynamic convolution and GSConv convolution and VoVGSCSP lightweight backbone, is used to simultaneously complete the detection of apple surface defects and the identification and screening of fruit stem information, retaining only the side information of the apple multi-view. After that, the YOLOv5s network model-based approach for assessing apple quality is then developed. The introduction of the Swin Transformer module to the Resnet18 backbone increases the grading accuracy and brings the judgment closer to the global optimal solution. In this study, datasets were made using a total of 1244 apple images, each containing 8 to 10 apples. Training sets and test sets were randomly created and divided into 3:1. The experimental results demonstrated that in the multi-dimensional view information processing, the recognition accuracy of the designed fruit stem and surface defect recognition model reached 96.56% after 150 iteration training, the loss function value decreased to 0.03, the model parameter was only 6.78 M, and the detection rate was 32 frames/s. After 150 iteration training, the average grading accuracy of the quality grading model reached 94.46%, the loss function value decreased to 0.05, and the model parameter was only 3.78 M. The test findings indicate that the proposed strategy has a good application prospect in the apple grading task.

Analyzing the Quality Parameters of Apples by Spectroscopy from Vis/NIR to NIR Region: A Comprehensive Review

Spectroscopic methods deliver a valuable non-destructive analytical tool that provides simultaneous qualitative and quantitative characterization of various samples. Apples belong to the world's most consumed crops and with the current challenges of climate change and human impacts on the environment, maintaining high-quality apple production has become critical. This review comprehensively analyzes the application of spectroscopy in near-infrared (NIR) and visible (Vis) regions, which not only show particular potential in evaluating the quality parameters of apples but also in optimizing their production and supply routines. This includes the assessment of the external and internal characteristics such as color, size, shape, surface defects, soluble solids content (SSC), total titratable acidity (TA), firmness, starch pattern index (SPI), total dry matter concentration (DM), and nutritional value. The review also summarizes various techniques and approaches used in Vis/NIR studies of apples, such as authenticity, origin, identification, adulteration, and quality control. Optical sensors and associated methods offer a wide suite of solutions readily addressing the main needs of the industry in practical routines as well, e.g., efficient sorting and grading of apples based on sweetness and other quality parameters, facilitating quality control throughout the production and supply chain. This review also evaluates ongoing development trends in the application of handheld and portable instruments operating in the Vis/NIR and NIR spectral regions for apple quality control. The use of these technologies can enhance apple crop quality, maintain competitiveness, and meet the demands of consumers, making them a crucial topic in the apple industry. The focal point of this review is placed on the literature published in the last five years, with the exceptions of seminal works that have played a critical role in shaping the field or representative studies that highlight the progress made in specific areas.

Deep Learning Network of Amomum villosum Quality Classification and Origin Identification Based on X-ray Technology

A machine vision system based on a convolutional neural network (CNN) was proposed to sort Amomum villosum using X-ray non-destructive testing technology in this study. The Amomum villosum fruit network (AFNet) algorithm was developed to identify the internal structure for quality classification and origin identification in this manuscript. This network model is composed of experimental features of Amomum villosum. In this study, we adopted a binary classification method twice consecutive to identify the origin and quality of Amomum villosum. The results show that the accuracy, precision, and specificity of the AFNet for quality classification were 96.33%, 96.27%, and 100.0%, respectively, achieving higher accuracy than traditional CNN under the condition of faster operation speed. In addition, the model can also achieve an accuracy of 90.60% for the identification of places of origin. The accuracy of multi-category classification performed later with the consistent network structure is lower than that of the cascaded CNNs solution. With this intelligent feature recognition model, the internal structure information of Amomum villosum can be determined based on X-ray technology. Its application will play a positive role to improve industrial production efficiency.

Multi-Camera-Based Sorting System for Surface Defects of Apples

In this paper, we addressed the challenges in sorting high-yield apple cultivars that traditionally relied on manual labor or system-based defect detection. Existing single-camera methods failed to uniformly capture the entire surface of apples, potentially leading to misclassification due to defects in unscanned areas. Various methods were proposed where apples were rotated using rollers on a conveyor. However, since the rotation was highly random, it was difficult to scan the apples uniformly for accurate classification. To overcome these limitations, we proposed a multi-camera-based apple sorting system with a rotation mechanism that ensured uniform and accurate surface imaging. The proposed system applied a rotation mechanism to individual apples while simultaneously utilizing three cameras to capture the entire surface of the apples. This method offered the advantage of quickly and uniformly acquiring the entire surface compared to single-camera and random rotation conveyor setups. The images captured by the system were analyzed using a CNN classifier deployed on embedded hardware. To maintain excellent CNN classifier performance while reducing its size and inference time, we employed knowledge distillation techniques. The CNN classifier demonstrated an inference speed of 0.069 s and an accuracy of 93.83% based on 300 apple samples. The integrated system, which included the proposed rotation mechanism and multi-camera setup, took a total of 2.84 s to sort one apple. Our proposed system provided an efficient and precise solution for detecting defects on the entire surface of apples, improving the sorting process with high reliability.

An online automatic sorting system for defective et using deep learning

Objective

To establish a deep-learning architecture based on faster region-based convolutional neural networks (Faster R-CNN) algorithm for detection and sorting of red ginseng (Ginseng Radix et Rhizoma Rubra) with internal defects automatically on an online X-ray machine vision system.

Methods

A Faster R-CNN based classifier was trained with around 20 000 samples with mean average precision value (mAP) of 0.95. A traditional image processing method based on feedforward neural network (FNN) obtained a bad performance with the accuracy, recall and specificity of 69.0%, 68.0%, and 70.0%, respectively. Therefore, the Faster R-CNN model was saved to evaluate the model performance on the defective red ginseng online sorting system.

Results

An independent set of 2 000 red ginsengs were used to validate the performance of the Faster R-CNN based online sorting system in three parallel tests, achieving accuracy of 95.8%, 95.2% and 96.2%, respectively.

Conclusion

The overall results indicated that the proposed Faster R-CNN based classification model has great potential for non-destructive detection of red ginseng with internal defects.

Deep Learning-Based Intelligent Apple Variety Classification System and Model Interpretability Analysis

In this study, series networks (AlexNet and VGG-19) and directed acyclic graph (DAG) networks (ResNet-18, ResNet-50, and ResNet-101) with transfer learning were employed to identify and classify 13 classes of apples from 7439 images. Two training datasets, model evaluation metrics, and three visualization methods were used to objectively assess, compare, and interpret five Convolutional Neural Network (CNN)-based models. The results show that the dataset configuration had a significant impact on the classification results, as all models achieved over 96.1% accuracy on dataset A (training-to-testing = 2.4:1.0) compared to 89.4-93.9% accuracy on dataset B (training-to-testing = 1.0:3.7). VGG-19 achieved the highest accuracy of 100.0% on dataset A and 93.9% on dataset B. Moreover, for networks of the same framework, the model size, accuracy, and training and testing times increased as the model depth (number of layers) increased. Furthermore, feature visualization, strongest activations, and local interpretable model-agnostic explanations techniques were used to show the understanding of apple images by different trained models, as well as to reveal how and why the models make classification decisions. These results improve the interpretability and credibility of CNN-based models, which provides guidance for future applications of deep learning methods in agriculture.

Evaluation of vegetable sauerkraut quality during storage based on convolution neural network

Vegetable sauerkraut is a traditional fermented food. Due to oxidation reactions that occur during storage, the quality and flavor in different periods will change. In this study, the quality evaluation and flavor characteristics of 13 groups of vegetable sauerkraut samples with different storage time were analyzed by using physical and chemical parameters combined with electronic nose. Photographs of samples of various periods were collected, and a convolutional neural network (CNN) framework was established. The relationship between total phenol oxidative decomposition and flavor compounds was linearly negatively correlated. The vegetable sauerkraut during storage can be divided into three categories (full acceptance period, acceptance period and unacceptance period) by principal component analysis and Fisher discriminant analysis. The CNN parameters were fine-tuned based on the classification results, and its output results can reflect the quality changes and flavor characteristics of the samples, and have better fitting, prediction capabilities. After 50 epochs of the model, the accuracy of three sets of data namely training set, validation set and test set recorded 94%, 85% and 93%, respectively. In addition, the accuracy of CNN in identifying different quality sauerkraut was 95.30%. It is proved that the convolutional neural network has excellent performance in predicting the quality of Szechuan Sauerkraut with high reliability.

A cross-domain fruit classification method based on lightweight attention networks and unsupervised domain adaptation

Image-based fruit classification offers many useful applications in industrial production and daily life, such as self-checkout in the supermarket, automatic fruit sorting and dietary guidance. However, fruit classification task will have different data distributions due to different application scenarios. One feasible solution to solve this problem is to use domain adaptation that adapts knowledge from the original training data (source domain) to the new testing data (target domain). In this paper, we propose a novel deep learning-based unsupervised domain adaptation method for cross-domain fruit classification. A hybrid attention module is proposed and added to MobileNet V3 to construct the HAM-MobileNet that can suppress the impact of complex backgrounds and extract more discriminative features. A hybrid loss function combining subdomain alignment and implicit distribution metrics is used to reduce domain discrepancy during model training and improve model classification performance. Two fruit classification datasets covering several domains are established to simulate common industrial and daily life application scenarios. We validate the proposed method on our constructed grape classification dataset and general fruit classification dataset. The experimental results show that the proposed method achieves an average accuracy of 95.0% and 93.2% on the two datasets, respectively. The classification model after domain adaptation can well overcome the domain discrepancy brought by different fruit classification scenarios. Meanwhile, the proposed datasets and method can serve as a benchmark for future cross-domain fruit classification research.

Citrus disease detection using convolution neural network generated features and Softmax classifier on hyperspectral image data

Identification and segregation of citrus fruit with diseases and peel blemishes are required to preserve market value. Previously developed machine vision approaches could only distinguish cankerous from non-cankerous citrus, while this research focused on detecting eight different peel conditions on citrus fruit using hyperspectral (HSI) imagery and an AI-based classification algorithm. The objectives of this paper were: (i) selecting the five most discriminating bands among 92 using PCA, (ii) training and testing a custom convolution neural network (CNN) model for classification with the selected bands, and (iii) comparing the CNN's performance using 5 PCA bands compared to five randomly selected bands. A hyperspectral imaging system from earlier work was used to acquire reflectance images in the spectral region from 450 to 930 nm (92 spectral bands). Ruby Red grapefruits with normal, cankerous, and 5 other common peel diseases including greasy spot, insect damage, melanose, scab, and wind scar were tested. A novel CNN based on the VGG-16 architecture was developed for feature extraction, and SoftMax for classification. The PCA-based bands were found to be 666.15, 697.54, 702.77, 849.24 and 917.25 nm, which resulted in an average accuracy, sensitivity, and specificity of 99.84%, 99.84% and 99.98% respectively. However, 10 trials of five randomly selected bands resulted in only a slightly lower performance, with accuracy, sensitivity, and specificity of 98.87%, 98.43% and 99.88%, respectively. These results demonstrate that an AI-based algorithm can successfully classify eight different peel conditions. The findings reported herein can be used as a precursor to develop a machine vision-based, real-time peel condition classification system for citrus processing.

Emerging Postharvest Technologies to Enhance the Shelf-Life of Fruit and Vegetables: An Overview

Quality losses in fresh produce throughout the postharvest phase are often due to the inappropriate use of preservation technologies. In the last few decades, besides the traditional approaches, advanced postharvest physical and chemical treatments (active packaging, dipping, vacuum impregnation, conventional heating, pulsed electric field, high hydrostatic pressure, and cold plasma) and biocontrol techniques have been implemented to preserve the nutritional value and safety of fresh produce. The application of these methodologies after harvesting is useful when addressing quality loss due to the long duration when transporting products to distant markets. Among the emerging technologies and contactless and non-destructive techniques for quality monitoring (image analysis, electronic noses, and near-infrared spectroscopy) present numerous advantages over the traditional, destructive methods. The present review paper has grouped original studies within the topic of advanced postharvest technologies, to preserve quality and reduce losses and waste in fresh produce. Moreover, the effectiveness and advantages of some contactless and non-destructive methodologies for monitoring the quality of fruit and vegetables will also be discussed and compared to the traditional methods.

Study on the Identification and Detection of Walnut Quality Based on Terahertz Imaging

Objective: Walnuts have rich nutritional value and are favored by the majority of consumers. As walnuts are shelled nuts, they are prone to suffer from defects such as mildew during storage. The fullness and mildew of the fruit impose effects on the quality of the walnuts. Therefore, it is of great economic significance to carry out a study on the rapid, non-destructive detection of walnut quality. Methods: Terahertz spectroscopy, with wavelengths between infrared and electromagnetic waves, has unique detection advantages. In this paper, the rapid and nondestructive detection of walnut mildew and fullness based on terahertz spectroscopy is carried out using the emerging terahertz transmission spectroscopy imaging technology. First, the normal walnuts and mildewed walnuts are identified and analyzed. At the same time, the image processing is carried out on the physical samples with different kernel sizes to calculate the fullness of the walnut kernels. The THz image of the walnuts is collected to extract the spectral information in different regions of interest. Four kinds of time domain signals in different regions of interest are extracted, and three qualitative discrimination models are established, including the support vector machine (SVM), random forest (RF), and k-nearest neighbor (KNN) algorithms. In addition, in order to realize the visual expression of walnut fullness, the terahertz images of the walnut are segmented with a binarization threshold, and the walnut fullness is calculated by the proportion of the shell and kernel pixels. Results: In the frequency domain signal, the amplitude intensity from high to low is the mildew sample, walnut kernel, and walnut shell, and the distinction between walnut kernel, shell samples, and mildew samples is high. The overall identification accuracy of the aforementioned three models is 90.83%, 97.38%, and 97.87%, respectively. Among them, KNN has the best qualitative discrimination effect. In a single category, the recognition accuracy of the model for the walnut kernel, walnut shell, mildew sample, and reference group (background) reaches 94%, 100%, 97.43%, and 100%, respectively. The terahertz transmission images of the five categories of walnut samples with different kernel sizes are processed to visualize the detection of kernel fullness inside walnuts, and the errors are less than 5% compared to the actual fullness of walnuts. Conclusion: This study illustrates that terahertz spectroscopy detection can achieve the detection of walnut mildew, and terahertz imaging technology can realize the visual expression and fullness calculation of walnut kernels. Terahertz spectroscopy and imaging provides a non-destructive detection method for walnut quality, which can provide a reference for the quality detection of other dried nuts with shells, thus having significant practical value.

High Speed Railway Fastener Defect Detection by Using Improved YoLoX-Nano Model

Rails play a vital role in the bearing and guidance of high-speed trains, and the normal condition of rail components is the guarantee of the operation and maintenance safety. Fasteners are critical components for fixing the rails, so it is particularly important to detect whether they are in a normal state or not. The current rail-fastener detection models have some drawbacks, including poor generalization ability, large model volume and low detection efficiency. In view of this, an improved YoLoX-Nano rail-fastener-defect-detection method is proposed in this paper. The CA attention mechanism is added to the three output feature maps of CSPDarknet and the enhanced feature extraction part of the Path Aggregation Feature Pyramid Network (PAFPN); the Adaptively Spatial Feature Fusion (ASFF) is added after the PAFPN output feature map, which enables the semantic information of the high-level features and the fine-grained features of the bottom layer to be further enhanced. The improved YoLoX-Nano model has improved the AP value by 27.42% on fractured fasteners, 15.88% on displacement fasteners and 12.96% on normal fasteners. Moreover, the mAP value is improved by 18.75%, and it is 14.75% higher than the two-stage model Faster-RCNN on mAP. In addition, compared with YoLov7-tiny, the improved YoLoX-Nano model achieves 13.56% improvement on mAP. Although the improved model increases a certain amount of calculation, the detection speed of the improved model has been increased by 30.54 fps and by 32.33 fps when compared with that of the Single-Shot Multi-Box Detector (SSD) model and the You Only Look Once v3 (YoLov3) model, reaching 54.35 fps. The improved YoLoX-Nano model enables accurate and rapid identification of the defects of rail fasteners, which can meet the needs of real-time detection. Furthermore, it has advantages in lightweight deployment of terminals for rail-fastener detection, thus providing some reference for image recognition and detection in other fields.

Improved Classification Approach for Fruits and Vegetables Freshness Based on Deep Learning

Classification of fruit and vegetable freshness plays an essential role in the food industry. Freshness is a fundamental measure of fruit and vegetable quality that directly affects the physical health and purchasing motivation of consumers. In addition, it is a significant determinant of market price; thus, it is imperative to study the freshness of fruits and vegetables. Owing to similarities in color, texture, and external environmental changes, such as shadows, lighting, and complex backgrounds, the automatic recognition and classification of fruits and vegetables using machine vision is challenging. This study presents a deep-learning system for multiclass fruit and vegetable categorization based on an improved YOLOv4 model that first recognizes the object type in an image before classifying it into one of two categories: fresh or rotten. The proposed system involves the development of an optimized YOLOv4 model, creating an image dataset of fruits and vegetables, data argumentation, and performance evaluation. Furthermore, the backbone of the proposed model was enhanced using the Mish activation function for more precise and rapid detection. Compared with the previous YOLO series, a complete experimental evaluation of the proposed method can obtain a higher average precision than the original YOLOv4 and YOLOv3 with 50.4%, 49.3%, and 41.7%, respectively. The proposed system has outstanding prospects for the construction of an autonomous and real-time fruit and vegetable classification system for the food industry and marketplaces and can also help visually impaired people to choose fresh food and avoid food poisoning.

Convolutional neural network for apple bruise detection based on hyperspectral

The timely detection of apple bruises caused by collision and squeeze is of great significance to reduce the economic losses of the apple industry. This study proposed a spectral analysis model (SpectralCNN) based on a one-dimensional convolutional neural network to detect apple bruises. The influences of six spectral preprocessing methods on the SpectralCNN model were firstly analyzed in this paper. Compared with traditional chemometric models, the SpectralCNN model had a better accuracy, which was demonstrated not depend on the spectral preprocessing method by experiment results. Then, 20 characteristic wavelengths could be extracted by successive projection algorithm. The SpectralCNN model could achieve an accuracy of 95.79% on the test set of characteristic wavelengths, indicating that the extracted characteristic wavelengths contain most of the features of bruised and healthy pixels.

Applications of machine learning techniques for enhancing nondestructive food quality and safety detection

In considering the need of people all over the world for high-quality food, there has been a recent increase in interest in the role of nondestructive and rapid detection technologies in the food industry. Moreover, the analysis of data acquired by most nondestructive technologies is complex, time-consuming, and requires highly skilled operators. Meanwhile, the general applicability of various chemometric or statistical methods is affected by noise, sample, variability, and data complexity that vary under various testing conditions. Nowadays, machine learning (ML) techniques have a wide range of applications in the food industry, especially in nondestructive technology and equipment intelligence, due to their powerful ability in handling irrelevant information, extracting feature variables, and building calibration models. The review provides an introduction and comparison of machine learning techniques, and summarizes these algorithms as traditional machine learning (TML), and deep learning (DL). Moreover, several novel nondestructive technologies, namely acoustic analysis, machine vision (MV), electronic nose (E-nose), and spectral imaging, combined with different advanced ML techniques and their applications in food quality assessment such as variety identification and classification, safety inspection and processing control, are presented. In addition to this, the existing challenges and prospects are discussed. The result of this review indicates that nondestructive testing technologies combined with state-of-the-art machine learning techniques show great potential for monitoring the quality and safety of food products and different machine learning algorithms have their characteristics and applicability scenarios. Due to the nature of feature learning, DL is one of the most promising and powerful techniques for real-time applications, which needs further research for full and wide applications in the food industry.

Real-Time Grading of Defect Apples Using Semantic Segmentation Combination with a Pruned YOLO V4 Network

At present, the apple grading system usually conveys apples by a belt or rollers. This usually leads to low hardness or expensive fruits being bruised, resulting in economic losses. In order to realize real-time detection and classification of high-quality apples, separate fruit trays were designed to convey apples and used to prevent apples from being bruised during image acquisition. A semantic segmentation method based on the BiSeNet V2 deep learning network was proposed to segment the defective parts of defective apples. BiSeNet V2 for apple defect detection obtained a slightly better result in MPA with a value of 99.66%, which was 0.14 and 0.19 percentage points higher than DAnet and Unet, respectively. A model pruning method was used to optimize the structure of the YOLO V4 network. The detection accuracy of defect regions in apple images was further improved by the pruned YOLO V4 network. Then, a surface mapping method between the defect area in apple images and the actual defect area was proposed to accurately calculate the defect area. Finally, apples on separate fruit trays were sorted according to the number and area of defects in the apple images. The experimental results showed that the average accuracy of apple classification was 92.42%, and the F1 score was 94.31. In commercial separate fruit tray grading and sorting machines, it has great application potential.

Surface Defect Detection of Fresh-Cut Cauliflowers Based on Convolutional Neural Network with Transfer Learning

A fresh-cut cauliflower surface defect detection and classification model based on a convolutional neural network with transfer learning is proposed to address the low efficiency of the traditional manual detection of fresh-cut cauliflower surface defects. Four thousand, seven hundred and ninety images of fresh-cut cauliflower were collected in four categories including healthy, diseased, browning, and mildewed. In this study, the pre-trained MobileNet model was fine-tuned to improve training speed and accuracy. The model optimization was achieved by selecting the optimal combination of training hyper-parameters and adjusting the different number of frozen layers; the parameters downloaded from ImageNet were optimally integrated with the parameters trained on our own model. A comparison of test results was presented by combining VGG19, InceptionV3, and NASNetMobile. Experimental results showed that the MobileNet model's loss value was 0.033, its accuracy was 99.27%, and the F1 score was 99.24% on the test set when the learning rate was set as 0.001, dropout was set as 0.5, and the frozen layer was set as 80. This model had better capability and stronger robustness and was more suitable for the surface defect detection of fresh-cut cauliflower when compared with other models, and the experiment's results demonstrated the method's feasibility.

Slight crack identification of cottonseed using air-coupled ultrasound with sound to image encoding

Slight crack of cottonseed is a critical factor influencing the germination rate of cotton due to foamed acid or water entering cottonseed through testa. However, it is very difficult to detect cottonseed with slight crack using common non-destructive detection methods, such as machine vision, optical spectroscopy, and thermal imaging, because slight crack has little effect on morphology, chemical substances or temperature. By contrast, the acoustic method shows a sensitivity to fine structure defects and demonstrates potential application in seed detection. This paper presents a novel method to detect slightly cracked cottonseed using air-coupled ultrasound with a light-weight vision transformer (ViT) and a sound-to-image encoding method. The echo signal of air-coupled ultrasound from cottonseed is obtained by non-contact and non-destructive methods. The intrinsic mode functions (IMFs) of ultrasound signal are obtained as the sound features using variational mode decomposition (VMD) approach. Then the sound features are converted into colorful images by a color encoding method. This method uses different colored lines to represent the changes of different values of IMFs according to the specified encoding period. A light-weight MobileViT method is utilized to identify the slightly cracked cottonseeds using encoding colorful images corresponding to cottonseeds. The experimental results show an average overall recognition accuracy of 90.7% for slightly cracked cottonseed from normal cottonseed, which indicates that the proposed method is reliable to applications in detection task of cottonseed with slight crack.

Detecting Starch-Head and Mildewed Fruit in Dried Hami Jujubes Using Visible/Near-Infrared Spectroscopy Combined with MRSA-SVM and Oversampling

Dried Hami jujube has great commercial and nutritional value. Starch-head and mildewed fruit are defective jujubes that pose a threat to consumer health. A novel method for detecting starch-head and mildewed fruit in dried Hami jujubes with visible/near-infrared spectroscopy was proposed. For this, the diffuse reflectance spectra in the range of 400–1100 nm of dried Hami jujubes were obtained. Borderline synthetic minority oversampling technology (BL-SMOTE) was applied to solve the problem of imbalanced sample distribution, and its effectiveness was demonstrated compared to other methods. Then, the feature variables selected by competitive adaptive reweighted sampling (CARS) were used as the input to establish the support vector machine (SVM) classification model. The parameters of SVM were optimized by the modified reptile search algorithm (MRSA). In MRSA, Tent chaotic mapping and the Gaussian random walk strategy were used to improve the optimization ability of the original reptile search algorithm (RSA). The final results showed that the MRSA-SVM method combined with BL-SMOTE had the best classification performance, and the detection accuracy reached 97.22%. In addition, the recall, precision, F₁ and kappa coefficient outperform other models. Furthermore, this study provided a valuable reference for the detection of defective fruit in other fruits.

Detection of early decayed oranges by structured-illumination reflectance imaging coupling with texture feature classification models

Citrus fruits are susceptible to fungal infection after harvest. To reduce the economic loss, it is necessary to reject the infected citrus fruit before storage and transportation. However, the infected area in the early stage of decay is almost invisible on the fruit surface, so the detection of early decayed citrus is very challenging. In this study, a structured-illumination reflectance imaging (SIRI) system combined with a visible light-emitting diode (LED) lamp and a monochrome camera was developed to detect early fungal infection in oranges. Under sinusoidal modulation illumination with spatial frequencies of 0.05, 0.15, and 0.25 cycles mm^-1, three-phase-shifted images with phase offsets of - 2π/3, 0, and 2π/3 were acquired for each spatial frequency. The direct component (DC) and alternating component (AC) images were then recovered by image demodulation using a three-phase-shifting approach. Compared with the DC image, the decayed area can be clearly identified in the AC image and RT image (AC/DC). The optimal spatial frequency was determined by analyzing the AC image and pixel intensity distribution. Based on the texture features extracted from DC, AC, and RT images, four kinds of classification models including partial least square discriminant analysis (PLS-DA), support vector machine (SVM), least squares-support vector machine (LS-SVM), and k-nearest neighbor (KNN) were established to detect the infected oranges, respectively. Model optimization was also performed by extracting important texture features. Compared to all models, the PLS-DA model developed based on eight texture features of RT images achieved the optimal classification accuracy of 96.4%. This study showed for the first time that the proposed SIRI system combined with appropriate texture features and classification model can realize the early detection of decayed oranges.

Selective transplantation method of leafy vegetable seedlings based on ResNet 18 network

To solve the problem of low survival rate caused by unscreened transplanting of seedlings. This study proposed a selective transplanting method of leafy vegetable seedlings based on the ResNet 18 network. Lettuce seedlings were selected as the research object, and a total of 3,388 images were obtained in the dataset. The images were randomly divided into the training set, validation set, and test set in the ratio of 6:2:2. The ResNet 18 network was used to perform transfer learning after tuning, identifying, and classifying leafy vegetable seedlings, and then establishing a model to screen leafy vegetable seedlings. The results showed that the optimal detection accuracy of the presence and health of seedlings in the training data set was above 100%, and the model loss remained at around 0.005. Nine hundred seedlings were selected for the validation test, and the screening accuracy rate was 97.44%, the precision rate of healthy seedlings was 97.56%, the recall rate was 97.34%, the precision rate of unhealthy seedlings was 92%, and the recall rate was 92.62%, which was better than the screening model based on the physical characteristics of seedlings. If they were identified as unhealthy seedlings, the manipulator would remove them during the transplanting process and perform the seedling replenishment operation to increase the survival rate of the transplanted seedlings. Moreover, the seedling image is extracted by background removal technology, so the model processing time for a single image is only 0.0129 s. This research will provide technical support for the selective transplantation of leafy vegetable seedlings.

Detection storage time of mild bruise’s yellow peaches using the combined hyperspectral imaging and machine learning method

To deduce the process of bruise and reduce the number of bruised fruits from the source, the storage time of yellow peaches after bruise should be identified. In order to distinguish the different storage times of mild bruise’s yellow peaches more effectively than current detection methods, the combined hyperspectral imaging and machine learning method was proposed. Firstly, the sample bruise region spectrum was extracted as spectral features, and then, the hyperspectral images were processed by Principal Component Analysis (PCA), and eight single-wavelength images were selected according to the weight coefficient curve of PC1 images, and the gray values of the selected images were calculated as image features. Finally, in order to find the optimal discriminative model, random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGBoost) models were built based on spectral features, image features, and spectral features combined with image features, respectively. The results show that the XGBoost models based on spectral features, image features, and spectral features combined with image features are the optimal models with the overall accuracy of 77.50%, 87.50% and 90.00%, respectively. To simplify the model, Competitive Adaptive Reweighted Sampling (CARS) algorithm was used to screen the wavelength of the normalized spectral data, and then they were fused with the image feature data again and the XGBoost model with an overall model accuracy of 95.00% was built. To sum up, the combined hyperspectral imaging and machine learning method can be used to distinguish the different storage times (2 h, 8 h, 24 h and 48 h) of mild bruise’s yellow peaches effectively. It provides a certain theoretical basis for hyperspectral imaging technology in fruit bruise detection.

Application of Convolutional Neural Network-Based Detection Methods in Fresh Fruit Production: A Comprehensive Review

As one of the representative algorithms of deep learning, a convolutional neural network (CNN) with the advantage of local perception and parameter sharing has been rapidly developed. CNN-based detection technology has been widely used in computer vision, natural language processing, and other fields. Fresh fruit production is an important socioeconomic activity, where CNN-based deep learning detection technology has been successfully applied to its important links. To the best of our knowledge, this review is the first on the whole production process of fresh fruit. We first introduced the network architecture and implementation principle of CNN and described the training process of a CNN-based deep learning model in detail. A large number of articles were investigated, which have made breakthroughs in response to challenges using CNN-based deep learning detection technology in important links of fresh fruit production including fruit flower detection, fruit detection, fruit harvesting, and fruit grading. Object detection based on CNN deep learning was elaborated from data acquisition to model training, and different detection methods based on CNN deep learning were compared in each link of the fresh fruit production. The investigation results of this review show that improved CNN deep learning models can give full play to detection potential by combining with the characteristics of each link of fruit production. The investigation results also imply that CNN-based detection may penetrate the challenges created by environmental issues, new area exploration, and multiple task execution of fresh fruit production in the future.

Application of Machine Vision System in Food Detection

Food processing technology is an important part of modern life globally and will undoubtedly play an increasingly significant role in future development of industry. Food quality and safety are societal concerns, and food health is one of the most important aspects of food processing. However, ensuring food quality and safety is a complex process that necessitates huge investments in labor. Currently, machine vision system based image analysis is widely used in the food industry to monitor food quality, greatly assisting researchers and industry in improving food inspection efficiency. Meanwhile, the use of deep learning in machine vision has significantly improved food identification intelligence. This paper reviews the application of machine vision in food detection from the hardware and software of machine vision systems, introduces the current state of research on various forms of machine vision, and provides an outlook on the challenges that machine vision system faces.

Fruit classification using attention-based MobileNetV2 for industrial applications

Recent deep learning methods for fruits classification resulted in promising performance. However, these methods are with heavy-weight architectures in nature, and hence require a higher storage and expensive training operations due to feeding a large number of training parameters. There is a necessity to explore lightweight deep learning models without compromising the classification accuracy. In this paper, we propose a lightweight deep learning model using the pre-trained MobileNetV2 model and attention module. First, the convolution features are extracted to capture the high-level object-based information. Second, an attention module is used to capture the interesting semantic information. The convolution and attention modules are then combined together to fuse both the high-level object-based information and the interesting semantic information, which is followed by the fully connected layers and the softmax layer. Evaluation of our proposed method, which leverages transfer learning approach, on three public fruit-related benchmark datasets shows that our proposed method outperforms the four latest deep learning methods with a smaller number of trainable parameters and a superior classification accuracy. Our model has a great potential to be adopted by industries closely related to the fruit growing and retailing or processing chain for automatic fruit identification and classifications in the future.

Nondestructive Detection of Codling Moth Infestation in Apples Using Pixel-Based NIR Hyperspectral Imaging with Machine Learning and Feature Selection

Codling moth (CM) (Cydia pomonella L.), a devastating pest, creates a serious issue for apple production and marketing in apple-producing countries. Therefore, effective nondestructive early detection of external and internal defects in CM-infested apples could remarkably prevent postharvest losses and improve the quality of the final product. In this study, near-infrared (NIR) hyperspectral reflectance imaging in the wavelength range of 900–1700 nm was applied to detect CM infestation at the pixel level for three organic apple cultivars, namely Gala, Fuji and Granny Smith. An effective region of interest (ROI) acquisition procedure along with different machine learning and data processing methods were used to build robust and high accuracy classification models. Optimal wavelength selection was implemented using sequential stepwise selection methods to build multispectral imaging models for fast and effective classification purposes. The results showed that the infested and healthy samples were classified at pixel level with up to 97.4% total accuracy for validation dataset using a gradient tree boosting (GTB) ensemble classifier, among others. The feature selection algorithm obtained a maximum accuracy of 91.6% with only 22 selected wavelengths. These findings indicate the high potential of NIR hyperspectral imaging (HSI) in detecting and classifying latent CM infestation in apples of different cultivars.

A feasibility research on the application of machine vision technology in appearance quality inspection of Xuesaitong dropping pills

Defect detection is a critical issue for the quality control of dropping pills, which is a special dosage form of traditional Chinese Medicine. Machine vision is a non-destructing testing technology and cost-effective with high accuracy that can be used to predict the detects of both interior and exterior of the sample by employing the camera. In this research, a machine vision system for inspecting quality of the Xuesaitong dropping pills (XDPs) that include non-spherical, abnormal sizes and colors was developed to evaluate the appearance quality of XDPs rapidly and accurately. Firstly, 270 images of XDPs containing qualified and three different types of defects were collected. Subsequently, the processing of the XDPs images were carried out. Finally, Three defecting categories classification models were developed and compared based on contour and color features. The experimental results showed that the Random Forest outperformed all the explored models and the classification accuracy for non-spherical, abnormal sizes and colors reached 98.52%, 100.00% and 100.00%, respectively. In summary, the method established in this research is scientific, reliable, fast and accurate, which has great application potential and can provide technical support for the automatic defect detection of dropping pills.