Real Time Pear Fruit Detection and Counting Using YOLOv4 Models and Deep SORT

Development and validation of machine-learning models for monitoring individual behaviors in group-housed broiler chickens

Animals' individual behavior is commonly monitored by live or video observation by a person. This can be labor intensive and inconsistent. An alternative is the use of machine learning-based computer vision systems. The objectives of this study were to 1) develop and optimize machine learning model frameworks for detecting, tracking and classifying individual behaviors of group-housed broiler chickens in continuous video recordings; and 2) use an independent dataset to evaluate the performance of the developed machine leaning model framework for individual poultry behaviors differentiation. Forty-two video recordings from 4 different pens (total video duration = 1,620 min) were used to develop and train multiple models for detecting and tracking individual birds and classifying 4 behaviors: feeding, drinking, active, and inactive. The optimal model framework was used to continuously analyze an external set of 11 videos (duration = 326 min), and the second-by-second behavior of each individual broiler was extracted for the comparison of human observation. After comparison of model performance, the YOLOv5l, out of 5 detection models, was selected for detecting individual broilers in a pen; the osnet_x0_25_msmt17, out of 4 tracking algorithms, was selected to track each detected bird in continuous frames; and the Gradient Boosting Classifier, out of 12 machine learning classifiers, was selected to classify the 4 behaviors. Most of the models were able to keep previously assigned individual identifications of the chickens in limited amounts of time, but lost the identities throughout an examination period (≥4 min). The final framework was able to accurately predict feeding (accuracy = 0.895) and drinking time (accuracy = 0.9) but subpar for active (accuracy = 0.545) and inactive time (accuracy = 0.505). The algorithms employed by the machine learning models were able to accurately detect feeding and drinking behavior but still need to be improved for maintaining individual identities of the chickens and identifying active and inactive behaviors.

An Evaluation of Multi-Channel Sensors and Density Estimation Learning for Detecting Fire Blight Disease in Pear Orchards

Fire blight is an infectious disease found in apple and pear orchards. While managing the disease is critical to maintaining orchard health, identifying symptoms early is a challenging task which requires trained expert personnel. This paper presents an inspection technique that targets individual symptoms via deep learning and density estimation. We evaluate the effects of including multi-spectral sensors in the model's pipeline. Results show that adding near infrared (NIR) channels can help improve prediction performance and that density estimation can detect possible symptoms when severity is in the mid-high range.

T-YOLO: a lightweight and efficient detection model for nutrient buds in complex tea-plantation environments

BACKGROUND

Quick and accurate detection of nutrient buds is essential for yield prediction and field management in tea plantations. However, the complexity of tea plantation environments and the similarity in color between nutrient buds and older leaves make the location of tea nutrient buds challenging.

RESULTS

This research presents a lightweight and efficient detection model, T-YOLO, for the accurate detection of tea nutrient buds in unstructured environments. First, a lightweight module, C2fG2, and an efficient feature extraction module, DBS, are introduced into the backbone and neck of the YOLOv5 baseline model. Second, the head network of the model is pruned to achieve further lightweighting. Finally, the dynamic detection head is integrated to mitigate the feature loss caused by lightweighting. The experimental data show that T-YOLO achieves a mean average precision (mAP) of 84.1%, the total number of parameters for model training (Params) is 11.26 million (M), and the number of floating-point operations (FLOPs) is 17.2 Giga (G). Compared with the baseline YOLOv5 model, T-YOLO reduces Params by 47% and lowers FLOPs by 65%. T-YOLO also outperforms the existing optimal detection YOLOv8 model by 7.5% in terms of mAP.

CONCLUSION

The T-YOLO model proposed in this study performs well in detecting small tea nutrient buds. It provides a decision-making basis for tea farmers to manage smart tea gardens. The T-YOLO model outperforms mainstream detection models on the public dataset, Global Wheat Head Detection (GWHD), which offers a reference for the construction of lightweight and efficient detection models for other small target crops. © 2024 Society of Chemical Industry.

Improved DeepSORT-Based Object Tracking in Foggy Weather for AVs Using Sematic Labels and Fused Appearance Feature Network

The presence of fog in the background can prevent small and distant objects from being detected, let alone tracked. Under safety-critical conditions, multi-object tracking models require faster tracking speed while maintaining high object-tracking accuracy. The original DeepSORT algorithm used YOLOv4 for the detection phase and a simple neural network for the deep appearance descriptor. Consequently, the feature map generated loses relevant details about the track being matched with a given detection in fog. Targets with a high degree of appearance similarity on the detection frame are more likely to be mismatched, resulting in identity switches or track failures in heavy fog. We propose an improved multi-object tracking model based on the DeepSORT algorithm to improve tracking accuracy and speed under foggy weather conditions. First, we employed our camera-radar fusion network (CR-YOLOnet) in the detection phase for faster and more accurate object detection. We proposed an appearance feature network to replace the basic convolutional neural network. We incorporated GhostNet to take the place of the traditional convolutional layers to generate more features and reduce computational complexities and costs. We adopted a segmentation module and fed the semantic labels of the corresponding input frame to add rich semantic information to the low-level appearance feature maps. Our proposed method outperformed YOLOv5 + DeepSORT with a 35.15% increase in multi-object tracking accuracy, a 32.65% increase in multi-object tracking precision, a speed increase by 37.56%, and identity switches decreased by 46.81%.

An improved deep learning network for image detection and its application in Dendrobii caulis decoction piece

In recent years, with the increasing demand for high-quality Dendrobii caulis decoction piece, the identification of D. caulis decoction piece species has become an urgent issue. However, the current methods are primarily designed for professional quality control and supervision. Therefore, ordinary consumers should not rely on these methods to assess the quality of products when making purchases. This research proposes a deep learning network called improved YOLOv5 for detecting different types of D. caulis decoction piece from images. In the main architecture of improved YOLOv5, we have designed the C2S module to replace the C3 module in YOLOv5, thereby enhancing the network's feature extraction capability for dense and small targets. Additionally, we have introduced the Reparameterized Generalized Feature Pyramid Network (RepGFPN) module and Optimal Transport Assignment (OTA) operator to more effectively integrate the high-dimensional and low-dimensional features of the network. Furthermore, a new large-scale dataset of Dendrobium images has been established. Compared to other models with similar computational complexity, improved YOLOv5 achieves the highest detection accuracy, with an average mAP@.05 of 96.5%. It is computationally equivalent to YOLOv5 but surpasses YOLOv5 by 2 percentage points in terms of accuracy.

Research on Dynamic Pig Counting Method Based on Improved YOLOv7 Combined with DeepSORT

A pig inventory is a crucial component of achieving precise and large-scale farming. In complex pigsty environments, due to pigs' stress reactions and frequent obstructions, it is challenging to count them accurately and automatically. This difficulty contrasts with most current deep learning studies, which rely on overhead views or static images for counting. This research proposes a video-based dynamic counting method, combining YOLOv7 with DeepSORT. By utilizing the YOLOv7 network structure and optimizing the second and third 3 × 3 convolution operations in the head network ELAN-W with PConv, the model reduces the computational demand and improves the inference speed without sacrificing accuracy. To ensure that the network acquires accurate position perception information at oblique angles and extracts rich semantic information, we introduce the coordinate attention (CA) mechanism before the three re-referentialization paths (REPConv) in the head network, enhancing robustness in complex scenarios. Experimental results show that, compared to the original model, the improved model increases the mAP by 3.24, 0.05, and 1.00 percentage points for oblique, overhead, and all pig counting datasets, respectively, while reducing the computational cost by 3.6 GFLOPS. The enhanced YOLOv7 outperforms YOLOv5, YOLOv4, YOLOv3, Faster RCNN, and SSD in target detection with mAP improvements of 2.07, 5.20, 2.16, 7.05, and 19.73 percentage points, respectively. In dynamic counting experiments, the improved YOLOv7 combined with DeepSORT was tested on videos with total pig counts of 144, 201, 285, and 295, yielding errors of -3, -3, -4, and -26, respectively, with an average accuracy of 96.58% and an FPS of 22. This demonstrates the model's capability of performing the real-time counting of pigs in various scenes, providing valuable data and references for automated pig counting research.

Customized Tracking Algorithm for Robust Cattle Detection and Tracking in Occlusion Environments

Ensuring precise calving time prediction necessitates the adoption of an automatic and precisely accurate cattle tracking system. Nowadays, cattle tracking can be challenging due to the complexity of their environment and the potential for missed or false detections. Most existing deep-learning tracking algorithms face challenges when dealing with track-ID switch cases caused by cattle occlusion. To address these concerns, the proposed research endeavors to create an automatic cattle detection and tracking system by leveraging the remarkable capabilities of Detectron2 while embedding tailored modifications to make it even more effective and efficient for a variety of applications. Additionally, the study conducts a comprehensive comparison of eight distinct deep-learning tracking algorithms, with the objective of identifying the most optimal algorithm for achieving precise and efficient individual cattle tracking. This research focuses on tackling occlusion conditions and track-ID increment cases for miss detection. Through a comparison of various tracking algorithms, we discovered that Detectron2, coupled with our customized tracking algorithm (CTA), achieves 99% in detecting and tracking individual cows for handling occlusion challenges. Our algorithm stands out by successfully overcoming the challenges of miss detection and occlusion problems, making it highly reliable even during extended periods in a crowded calving pen.

DomAda-FruitDet: Domain-Adaptive Anchor-Free Fruit Detection Model for Auto Labeling

Recently, deep learning-based fruit detection applications have been widely used in the modern fruit industry; however, the training data labeling process remains a time-consuming and labor-intensive process. Auto labeling can provide a convenient and efficient data source for constructing smart orchards based on deep-learning technology. In our previous study, based on a labeled source domain fruit dataset, we used a generative adversarial network and a fruit detection model to achieve auto labeling of unlabeled target domain fruit images. However, since the current method uses one species source domain fruit to label multiple species target domain fruits, there is a problem of the domain gap in both the foreground and the background between the training data (retaining the source domain fruit label information) and the application data (target domain fruit images) of the fruit detection model. Therefore, we propose a domain-adaptive anchor-free fruit detection model, DomAda-FruitDet, and apply it to the previously proposed fruit labeling method to further improve the accuracy. It consists of 2 design aspects: (a) With a foreground domain-adaptive structure based on double prediction layers, an anchor-free method with multiscale detection capability is constructed to generate adaptive bounding boxes that overcome the foreground domain gap; (b) with a background domain-adaptive strategy based on sample allocation, we enhance the ability of the model to extract foreground object features to overcome the background domain gap. As a result, the proposed method can label actual apple, tomato, pitaya, and mango datasets, with an average precision of 90.9%, 90.8%, 88.3%, and 94.0%, respectively. In conclusion, the proposed DomAda-FruitDet effectively addressed the problem of the domain gap and improved effective auto labeling for fruit detection tasks.

A simplified network topology for fruit detection, counting and mobile-phone deployment

The complex network topology, deployment unfriendliness, computation cost, and large parameters, including the natural changeable environment are challenges faced by fruit detection. Thus, a Simplified network topology for fruit detection, tracking and counting was designed to solve these problems. The network used common networks of Conv, Maxpool, feature concatenation and SPPF as new backbone and a modified decoupled head of YOLOv8 as head network. At the same time, it was validated on a dataset of images encompassing strawberry, jujube, and cherry fruits. Having compared to YOLO-mainstream variants, the params of Simplified network is 32.6%, 127%, and 50.0% lower than YOLOv5n, YOLOv7-tiny, and YOLOv8n, respectively. The results of mAP@50% tested using test-set show that the 82.4% of Simplified network is 0.4%, -0.2%, and 0.2% respectively more accurate than 82.0% of YOLOv5n, 82.6% of YOLOv7-tiny, and 82.2% of YOLOv8n. Furthermore, the Simplified network is 12.8%, 17.8%, and 11.8% respectively faster than YOLOv5n, YOLOv7-tiny, and YOLOv8n, including outperforming in tracking, counting, and mobile-phone deployment process. Hence, the Simplified network is robust, fast, accurate, easy-to-understand, fewer in parameters and deployable friendly.

Broiler Mobility Assessment via a Semi-Supervised Deep Learning Model and Neo-Deep Sort Algorithm

Mobility is a vital welfare indicator that may influence broilers' daily activities. Classical broiler mobility assessment methods are laborious and cannot provide timely insights into their conditions. Here, we proposed a semi-supervised Deep Learning (DL) model, YOLOv5 (You Only Look Once version 5), combined with a deep sort algorithm conjoined with our newly proposed algorithm, neo-deep sort, for individual broiler mobility tracking. Initially, 1650 labeled images from five days were employed to train the YOLOv5 model. Through semi-supervised learning (SSL), this narrowly trained model was then used for pseudo-labeling 2160 images, of which 2153 were successfully labeled. Thereafter, the YOLOv5 model was fine-tuned on the newly labeled images. Lastly, the trained YOLOv5 and the neo-deep sort algorithm were applied to detect and track 28 broilers in two pens and categorize them in terms of hourly and daily travel distances and speeds. SSL helped in increasing the YOLOv5 model's mean average precision (mAP) in detecting birds from 81% to 98%. Compared with the manually measured covered distances of broilers, the combined model provided individual broilers' hourly moved distances with a validation accuracy of about 80%. Eventually, individual and flock-level mobilities were quantified while overcoming the occlusion, false, and miss-detection issues.

An improved YOLOv5s model using feature concatenation with attention mechanism for real-time fruit detection and counting

An improved YOLOv5s model was proposed and validated on a new fruit dataset to solve the real-time detection task in a complex environment. With the incorporation of feature concatenation and an attention mechanism into the original YOLOv5s network, the improved YOLOv5s recorded 122 layers, 4.4 × 10⁶ params, 12.8 GFLOPs, and 8.8 MB weight size, which are 45.5%, 30.2%, 14.1%, and 31.3% smaller than the original YOLOv5s, respectively. Meanwhile, the obtained 93.4% of mAP tested on the valid set, 96.0% of mAP tested on the test set, and 74 fps of speed tested on videos using improved YOLOv5s is 0.6%, 0.5%, and 10.4% higher than the original YOLOv5s model, respectively. Using videos, the fruit tracking and counting tested on the improved YOLOv5s observed less missed and incorrect detections compared to the original YOLOv5s. Furthermore, the aggregated detection performance of improved YOLOv5s outperformed the network of GhostYOLOv5s, YOLOv4-tiny, and YOLOv7-tiny, including other mainstream YOLO variants. Therefore, the improved YOLOv5s is lightweight with reduced computation costs, can better generalize against complex conditions, and is applicable for real-time detection in fruit picking robots and low-power devices.

Deep learning supported machine vision system to precisely automate the wild blueberry harvester header

An operator of a wild blueberry harvester faces the fatigue of manually adjusting the height of the harvester's head, considering spatial variations in plant height, fruit zone, and field topography affecting fruit yield. For stress-free harvesting of wild blueberries, a deep learning-supported machine vision control system has been developed to detect the fruit height and precisely auto-adjust the header picking teeth rake position. The OpenCV AI Kit (OAK-D) was used with YOLOv4-tiny deep learning model with code developed in Python to solve the challenge of matching fruit heights with the harvester's head position. The system accuracy was statistically evaluated with R2 (coefficient of determination) and σ (standard deviation) measured on the difference in distances between the berries picking teeth and average fruit heights, which were 72, 43% and 2.1, 2.3 cm for the auto and manual head adjustment systems, respectively. This innovative system performed well in weed-free areas but requires further work to operate in weedy sections of the fields. Benefits of using this system include automated control of the harvester's head to match the header picking rake height to the level of the fruit height while reducing the operator's stress by creating safer working environments.

Parking Time Violation Tracking Using YOLOv8 and Tracking Algorithms

The major problem in Thailand related to parking is time violation. Vehicles are not allowed to park for more than a specified amount of time. Implementation of closed-circuit television (CCTV) surveillance cameras along with human labor is the present remedy. However, this paper presents an approach that can introduce a low-cost time violation tracking system using CCTV, Deep Learning models, and object tracking algorithms. This approach is fairly new because of its appliance of the SOTA detection technique, object tracking approach, and time boundary implementations. YOLOv8, along with the DeepSORT/OC-SORT algorithm, is utilized for the detection and tracking that allows us to set a timer and track the time violation. Using the same apparatus along with Deep Learning models and algorithms has produced a better system with better performance. The performance of both tracking algorithms was well depicted in the results, obtaining MOTA scores of (1.0, 1.0, 0.96, 0.90) and (1, 0.76, 0.90, 0.83) in four different surveillance data for DeepSORT and OC-SORT, respectively.

Detection of Defective Lettuce Seedlings Grown in an Indoor Environment under Different Lighting Conditions Using Deep Learning Algorithms

Sorting seedlings is laborious and requires attention to identify damage. Separating healthy seedlings from damaged or defective seedlings is a critical task in indoor farming systems. However, sorting seedlings manually can be challenging and time-consuming, particularly under complex lighting conditions. Different indoor lighting conditions can affect the visual appearance of the seedlings, making it difficult for human operators to accurately identify and sort the seedlings consistently. Therefore, the objective of this study was to develop a defective-lettuce-seedling-detection system under different indoor cultivation lighting systems using deep learning algorithms to automate the seedling sorting process. The seedling images were captured under different indoor lighting conditions, including white, blue, and red. The detection approach utilized and compared several deep learning algorithms, specifically CenterNet, YOLOv5, YOLOv7, and faster R-CNN to detect defective seedlings in indoor farming environments. The results demonstrated that the mean average precision (mAP) of YOLOv7 (97.2%) was the highest and could accurately detect defective lettuce seedlings compared to CenterNet (82.8%), YOLOv5 (96.5%), and faster R-CNN (88.6%). In terms of detection under different light variables, YOLOv7 also showed the highest detection rate under white and red/blue/white lighting. Overall, the detection of defective lettuce seedlings by YOLOv7 shows great potential for introducing automated seedling-sorting systems and classification under actual indoor farming conditions. Defective-seedling-detection can improve the efficiency of seedling-management operations in indoor farming.

Annotated Datasets of Oil Palm Fruit Bunch Piles for Ripeness Grading Using Deep Learning

The quality of palm oil is strongly influenced by the maturity level of the fruit to be processed into palm oil. Many studies have been carried out for detecting and classifying the maturity level of oil palm fruit to improve the quality with the use of computer vision. However, most of these studies use datasets in the form of images of oil palm fresh fruit bunches (FFB) with incomplete categorization according to real conditions in palm oil mills. Therefore, this study introduces a new complete dataset obtained directly from palm oil mills in the form of videos and images with different categories in accordance with the real conditions faced by the grading section of the palm oil mill. The video dataset consists of 45 videos with a single category of FFB videos and 56 videos with a collection of FFB with multiple categories for each video. Videos are collected using a smart phone with a size of 1280 × 720 pixels with .mp4 format. In addition, this dataset has also been annotated and labelled based on the maturity level of oil palm fruit with 6 categories, which are unripe, under-ripe, ripe, overripe, empty bunches and abnormal fruit.

Video based oil palm ripeness detection model using deep learning

Research on oil palm detection has been carried out for years, but there are only a few research that have conducted research using video datasets and only focus on development using non-sequential image. The use of the video dataset aims to adjust to the detection conditions carried out in real time so that it can automatically harvest directly from oil palm trees to increase efficiency in harvesting. To solve this problem, in this research, we develop an object detection model using a video dataset in training and testing. We used the 3 series YOLOv4 architecture to develop the model using video. Model development is done by means of hyperparameter tuning and frozen layer with data augmentation consisting of photometric and geometric augmentation experiment. To validate the outcomes of the YOLOv4 model development, a comparison of SSD-MobileNetV2 FPN and EfficientDet-D0 was performed. The results obtained show that YOLOv4-Tiny 3L is the most suitable architecture for use in real time object detection conditions with an mAP of 90.56% for single class category detection and 70.21% for multi class category detection with a detection speed of almost 4× faster than YOLOv4-CSPDarknet53, 5× faster than SSD-MobileNetV2 FPN, and 9× faster than EfficientDet-D0.

YOLO-P: An efficient method for pear fast detection in complex orchard picking environment

INTRODUCTION

Fruit detection is one of the key functions of an automatic picking robot, but fruit detection accuracy is seriously decreased when fruits are against a disordered background and in the shade of other objects, as is commmon in a complex orchard environment.

METHODS

Here, an effective mode based on YOLOv5, namely YOLO-P, was proposed to detect pears quickly and accurately. Shuffle block was used to replace the Conv, Batch Norm, SiLU (CBS) structure of the second and third stages in the YOLOv5 backbone, while the inverted shuffle block was designed to replace the fourth stage's CBS structure. The new backbone could extract features of pears from a long distance more efficiently. A convolutional block attention module (CBAM) was inserted into the reconstructed backbone to improve the robot's ability to capture pears' key features. Hard-Swish was used to replace the activation functions in other CBS structures in the whole YOLOv5 network. A weighted confidence loss function was designed to enhance the detection effect of small targets.

RESULT

At last, model comparison experiments, ablation experiments, and daytime and nighttime pear detection experiments were carried out. In the model comparison experiments, the detection effect of YOLO-P was better than other lightweight networks. The results showed that the module's average precision (AP) was 97.6%, which was 1.8% higher than the precision of the original YOLOv5s. The model volume had been compressed by 39.4%, from 13.7MB to only 8.3MB. Ablation experiments verified the effectiveness of the proposed method. In the daytime and nighttime pear detection experiments, an embedded industrial computer was used to test the performance of YOLO-P against backgrounds of different complexities and when fruits are in different degrees of shade.

DISCUSSION

The results showed that YOLO-P achieved the highest F1 score (96.1%) and frames per second (FPS) (32 FPS). It was sufficient for the picking robot to quickly and accurately detect pears in orchards. The proposed method can quickly and accurately detect pears in unstructured environments. YOLO-P provides support for automated pear picking and can be a reference for other types of fruit detection in similar environments.

Achieving Adaptive Visual Multi-Object Tracking with Unscented Kalman Filter

As an essential part of intelligent monitoring, behavior recognition, automatic driving, and others, the challenge of multi-object tracking is still to ensure tracking accuracy and robustness, especially in complex occlusion environments. Aiming at the issues of the occlusion, background noise, and motion state violent change for multi-object in a complex scene, an improved DeepSORT algorithm based on YOLOv5 is proposed for multi-object tracking to enhance the speed and accuracy of tracking. Firstly, a general object motion model is devised, which is similar to the variable acceleration motion model, and a multi-object tracking framework with the general motion model is established. Then, the latest YOLOv5 algorithm, which has satisfactory detection accuracy, is utilized to obtain the object information as the input of multi-object tracking. An unscented Kalman filter (UKF) is proposed to estimate the motion state of multi-object to solve nonlinear errors. In addition, the adaptive factor is introduced to evaluate observation noise and detect abnormal observations so as to adaptively adjust the innovation covariance matrix. Finally, an improved DeepSORT algorithm for multi-object tracking is formed to promote robustness and accuracy. Extensive experiments are carried out on the MOT16 data set, and we compare the proposed algorithm with the DeepSORT algorithm. The results indicate that the speed and precision of the improved DeepSORT are increased by 4.75% and 2.30%, respectively. Especially in the MOT16 of the dynamic camera, the improved DeepSORT shows better performance.

Monitoring and Identification of Road Construction Safety Factors via UAV

The safety of road construction is one of the most important concerns of construction managers for the following reasons: long-span construction operation, no fixed monitoring cameras, and huge impacts on existing traffic, while the managers still rely on manual inspection and a lack of image records. With the fast development of Unmanned Aerial Vehicle (UAV) and Artificial Intelligence (AI), monitoring safety concerns of road construction sites becomes easily accessible. This research aims to integrate UAVs and AI to establish a UAV-based road construction safety monitoring platform. In this study, road construction safety factors including constructors, construction vehicles, safety signs, and guardrails are defined and monitored to make up for the lack of image data at the road construction site. The main findings of this study include three aspects. First, the flight and photography schemes are proposed based on the UAV platform for information collection for road construction. Second, deep learning algorithms including YOLOv4 and DeepSORT are utilized to automatically detect and track safety factors. Third, a road construction dataset is established with 3594 images. The results show that the UAV-based monitoring platform can help managers with security inspection and recording images.

Sugarcane-Seed-Cutting System Based on Machine Vision in Pre-Seed Mode

China is the world's third-largest producer of sugarcane, slightly behind Brazil and India. As an important cash crop in China, sugarcane has always been the main source of sugar, the basic strategic material. The planting method of sugarcane used in China is mainly the pre-cutting planting mode. However, there are many problems with this technology, which has a great impact on the planting quality of sugarcane. Aiming at a series of problems, such as low cutting efficiency and poor quality in the pre-cutting planting mode of sugarcane, a sugarcane-seed-cutting device was proposed, and a sugarcane-seed-cutting system based on automatic identification technology was designed. The system consists of a sugarcane-cutting platform, a seed-cutting device, a visual inspection system, and a control system. Among them, the visual inspection system adopts the YOLO V5 network model to identify and detect the eustipes of sugarcane, and the seed-cutting device is composed of a self-tensioning conveying mechanism, a reciprocating crank slider transmission mechanism, and a high-speed rotary cutting mechanism so that the cutting device can complete the cutting of sugarcane seeds of different diameters. The test shows that the recognition rate of sugarcane seed cutting is no less than 94.3%, the accuracy rate is between 94.3% and 100%, and the average accuracy is 98.2%. The bud injury rate is no higher than 3.8%, while the average cutting time of a single seed is about 0.7 s, which proves that the cutting system has a high cutting rate, recognition rate, and low injury rate. The findings of this paper have important application values for promoting the development of sugarcane pre-cutting planting mode and sugarcane planting technology.

Experimental Study of Garlic Root Cutting Based on Deep Learning Application in Food Primary Processing

Garlic root cutting is generally performed manually; it is easy for the workers to sustain hand injuries, and the labor efficiency is low. However, the significant differences between individual garlic bulbs limit the development of an automatic root cutting system. To address this problem, a deep learning model based on transfer learning and a low-cost computer vision module was used to automatically detect garlic bulb position, adjust the root cutter, and cut garlic roots on a garlic root cutting test bed. The proposed object detection model achieved good performance and high detection accuracy, running speed, and detection reliability. The visual image of the output layer channel of the backbone network showed the high-level features extracted by the network vividly, and the differences in learning of different networks clearly. The position differences of the cutting lines predicted by different backbone networks were analyzed through data visualization. The excellent and stable performance indicated that the proposed model had learned the correct features in the data of different brightness. Finally, the root cutting system was verified experimentally. The results of three experiments with 100 garlic bulbs each indicated that the mean qualified value of the system was 96%. Therefore, the proposed deep learning system can be applied in garlic root cutting which belongs to food primary processing.

Fast and Non-Destructive Quail Egg Freshness Assessment Using a Thermal Camera and Deep Learning-Based Air Cell Detection Algorithms for the Revalidation of the Expiration Date of Eggs

Freshness is one of the most important parameters for assessing the quality of avian eggs. Available techniques to estimate the degradation of albumen and enlargement of the air cell are either destructive or not suitable for high-throughput applications. The aim of this research was to introduce a new approach to evaluate the air cell of quail eggs for freshness assessment as a fast, noninvasive, and nondestructive method. A new methodology was proposed by using a thermal microcamera and deep learning object detection algorithms. To evaluate the new method, we stored 174 quail eggs and collected thermal images 30, 50, and 60 days after the labeled expiration date. These data, 522 in total, were expanded to 3610 by image augmentation techniques and then split into training and validation samples to produce models of the deep learning algorithms, referred to as "You Only Look Once" version 4 and 5 (YOLOv4 and YOLOv5) and EfficientDet. We tested the models in a new dataset composed of 60 eggs that were kept for 15 days after the labeled expiration label date. The validation of our methodology was performed by measuring the air cell area highlighted in the thermal images at the pixel level; thus, we compared the difference in the weight of eggs between the first day of storage and after 10 days under accelerated aging conditions. The statistical significance showed that the two variables (air cell and weight) were negatively correlated (R² = 0.676). The deep learning models could predict freshness with F1 scores of 0.69, 0.89, and 0.86 for the YOLOv4, YOLOv5, and EfficientDet models, respectively. The new methodology for freshness assessment demonstrated that the best model reclassified 48.33% of our testing dataset. Therefore, those expired eggs could have their expiration date extended for another 2 weeks from the original label date.

Detection of Tip-Burn Stress on Lettuce Grown in an Indoor Environment Using Deep Learning Algorithms

Lettuce grown in indoor farms under fully artificial light is susceptible to a physiological disorder known as tip-burn. A vital factor that controls plant growth in indoor farms is the ability to adjust the growing environment to promote faster crop growth. However, this rapid growth process exacerbates the tip-burn problem, especially for lettuce. This paper presents an automated detection of tip-burn lettuce grown indoors using a deep-learning algorithm based on a one-stage object detector. The tip-burn lettuce images were captured under various light and indoor background conditions (under white, red, and blue LEDs). After augmentation, a total of 2333 images were generated and used for training using three different one-stage detectors, namely, CenterNet, YOLOv4, and YOLOv5. In the training dataset, all the models exhibited a mean average precision (mAP) greater than 80% except for YOLOv4. The most accurate model for detecting tip-burns was YOLOv5, which had the highest mAP of 82.8%. The performance of the trained models was also evaluated on the images taken under different indoor farm light settings, including white, red, and blue LEDs. Again, YOLOv5 was significantly better than CenterNet and YOLOv4. Therefore, detecting tip-burn on lettuce grown in indoor farms under different lighting conditions can be recognized by using deep-learning algorithms with a reliable overall accuracy. Early detection of tip-burn can help growers readjust the lighting and controlled environment parameters to increase the freshness of lettuce grown in plant factories.

An Improved Crucible Spatial Bubble Detection Based on YOLOv5 Fusion Target Tracking

A three-dimensional spatial bubble counting method is proposed to solve the problem of the existing crucible bubble detection only being able to perform two-dimensional statistics. First, spatial video images of the transparent layer of the crucible are acquired by a digital microscope, and a quartz crucible bubble dataset is constructed independently. Secondly, to address the problems of poor real-time and the insufficient small-target detection capability of existing methods for quartz crucible bubble detection, rich detailed feature information is retained by reducing the depth of down-sampling in the YOLOv5 network structure. In the neck, the dilated convolution algorithm is used to increase the feature map perceptual field to achieve the extraction of global semantic features; in front of the detection layer, an effective channel attention network (ECA-Net) mechanism is added to improve the capability of expressing significant channel characteristics. Furthermore, a tracking algorithm based on Kalman filtering and Hungarian matching is presented for bubble counting in crucible space. The experimental results demonstrate that the detector algorithm presented in this paper can effectively reduce the missed detection rate of tiny bubbles and increase the average detection precision from 96.27% to 98.76% while reducing weight by half and reaching a speed of 82 FPS. The excellent detector performance improves the tracker's accuracy significantly, allowing for real-time and high-precision counting of bubbles in quartz crucibles. It is an effective method for detecting crucible spatial bubbles.

Pear Recognition in an Orchard from 3D Stereo Camera Datasets to Develop a Fruit Picking Mechanism Using Mask R-CNN

In orchard fruit picking systems for pears, the challenge is to identify the full shape of the soft fruit to avoid injuries while using robotic or automatic picking systems. Advancements in computer vision have brought the potential to train for different shapes and sizes of fruit using deep learning algorithms. In this research, a fruit recognition method for robotic systems was developed to identify pears in a complex orchard environment using a 3D stereo camera combined with Mask Region-Convolutional Neural Networks (Mask R-CNN) deep learning technology to obtain targets. This experiment used 9054 RGBA original images (3018 original images and 6036 augmented images) to create a dataset divided into a training, validation, and testing sets. Furthermore, we collected the dataset under different lighting conditions at different times which were high-light (9–10 am) and low-light (6–7 pm) conditions at JST, Tokyo Time, August 2021 (summertime) to prepare training, validation, and test datasets at a ratio of 6:3:1. All the images were taken by a 3D stereo camera which included PERFORMANCE, QUALITY, and ULTRA models. We used the PERFORMANCE model to capture images to make the datasets; the camera on the left generated depth images and the camera on the right generated the original images. In this research, we also compared the performance of different types with the R-CNN model (Mask R-CNN and Faster R-CNN); the mean Average Precisions (mAP) of Mask R-CNN and Faster R-CNN were compared in the same datasets with the same ratio. Each epoch in Mask R-CNN was set at 500 steps with total 80 epochs. And Faster R-CNN was set at 40,000 steps for training. For the recognition of pears, the Mask R-CNN, had the mAPs of 95.22% for validation set and 99.45% was observed for the testing set. On the other hand, mAPs were observed 87.9% in the validation set and 87.52% in the testing set using Faster R-CNN. The different models using the same dataset had differences in performance in gathering clustered pears and individual pear situations. Mask R-CNN outperformed Faster R-CNN when the pears are densely clustered at the complex orchard. Therefore, the 3D stereo camera-based dataset combined with the Mask R-CNN vision algorithm had high accuracy in detecting the individual pears from gathered pears in a complex orchard environment.

Tree Trunk Recognition in Orchard Autonomous Operations under Different Light Conditions Using a Thermal Camera and Faster R-CNN

In an orchard automation process, a current challenge is to recognize natural landmarks and tree trunks to localize intelligent robots. To overcome low-light conditions and global navigation satellite system (GNSS) signal interruptions under a dense canopy, a thermal camera may be used to recognize tree trunks using a deep learning system. Therefore, the objective of this study was to use a thermal camera to detect tree trunks at different times of the day under low-light conditions using deep learning to allow robots to navigate. Thermal images were collected from the dense canopies of two types of orchards (conventional and joint training systems) under high-light (12-2 PM), low-light (5-6 PM), and no-light (7-8 PM) conditions in August and September 2021 (summertime) in Japan. The detection accuracy for a tree trunk was confirmed by the thermal camera, which observed an average error of 0.16 m for 5 m, 0.24 m for 15 m, and 0.3 m for 20 m distances under high-, low-, and no-light conditions, respectively, in different orientations of the thermal camera. Thermal imagery datasets were augmented to train, validate, and test using the Faster R-CNN deep learning model to detect tree trunks. A total of 12,876 images were used to train the model, 2318 images were used to validate the training process, and 1288 images were used to test the model. The mAP of the model was 0.8529 for validation and 0.8378 for the testing process. The average object detection time was 83 ms for images and 90 ms for videos with the thermal camera set at 11 FPS. The model was compared with the YOLO v3 with same number of datasets and training conditions. In the comparisons, Faster R-CNN achieved a higher accuracy than YOLO v3 in tree truck detection using the thermal camera. Therefore, the results showed that Faster R-CNN can be used to recognize objects using thermal images to enable robot navigation in orchards under different lighting conditions.

Classification of Fruit Flies by Gender in Images Using Smartphones and the YOLOv4-Tiny Neural Network

The fruit fly Drosophila melanogaster is a classic research object in genetics and systems biology. In the genetic analysis of flies, a routine task is to determine the offspring size and gender ratio in their populations. Currently, these estimates are made manually, which is a very time-consuming process. The counting and gender determination of flies can be automated by using image analysis with deep learning neural networks on mobile devices. We proposed an algorithm based on the YOLOv4-tiny network to identify Drosophila flies and determine their gender based on the protocol of taking pictures of insects on a white sheet of paper with a cell phone camera. Three strategies with different types of augmentation were used to train the network. The best performance (F1 = 0.838) was achieved using synthetic images with mosaic generation. Females gender determination is worse than that one of males. Among the factors that most strongly influencing the accuracy of fly gender recognition, the fly’s position on the paper was the most important. Increased light intensity and higher quality of the device cameras have a positive effect on the recognition accuracy. We implement our method in the FlyCounter Android app for mobile devices, which performs all the image processing steps using the device processors only. The time that the YOLOv4-tiny algorithm takes to process one image is less than 4 s.

Green Citrus Detection and Counting in Orchards Based on YOLOv5-CS and AI Edge System

Green citrus detection in citrus orchards provides reliable support for production management chains, such as fruit thinning, sunburn prevention and yield estimation. In this paper, we proposed a lightweight object detection YOLOv5-CS (Citrus Sort) model to realize object detection and the accurate counting of green citrus in the natural environment. First, we employ image rotation codes to improve the generalization ability of the model. Second, in the backbone, a convolutional layer is replaced by a convolutional block attention module, and a detection layer is embedded to improve the detection accuracy of the little citrus. Third, both the loss function CIoU (Complete Intersection over Union) and cosine annealing algorithm are used to get the better training effect of the model. Finally, our model is migrated and deployed to the AI (Artificial Intelligence) edge system. Furthermore, we apply the scene segmentation method using the “virtual region” to achieve accurate counting of the green citrus, thereby forming an embedded system of green citrus counting by edge computing. The results show that the mAP@.5 of the YOLOv5-CS model for green citrus was 98.23%, and the recall is 97.66%. The inference speed of YOLOv5-CS detecting a picture on the server is 0.017 s, and the inference speed on Nvidia Jetson Xavier NX is 0.037 s. The detection and counting frame rate of the AI edge system-side counting system is 28 FPS, which meets the counting requirements of green citrus.

OATCR: Outdoor Autonomous Trash-Collecting Robot Design Using YOLOv4-Tiny

This paper proposed an innovative mechanical design using the Rocker-bogie mechanism for resilient Trash-Collecting Robots. Mask-RCNN, YOLOV4, and YOLOv4-tiny were experimented on and analyzed for trash detection. The Trash-Collecting Robot was developed to be completely autonomous as it was able to detect trash, move towards it, and pick it up while avoiding any obstacles along the way. Sensors including a camera, ultrasonic sensor, and GPS module played an imperative role in automation. The brain of the Robot, namely, Raspberry Pi and Arduino, processed the data from the sensors and performed path-planning and consequent motion of the robot through actuation of motors. Three models for object detection were tested for potential use in the robot: Mask-RCNN, YOLOv4, and YOLOv4-tiny. Mask-RCNN achieved an average precision (mAP) of over 83% and detection time (DT) of 3973.29 ms, YOLOv4 achieved 97.1% (mAP) and 32.76 DT, and YOLOv4-tiny achieved 95.2% and 5.21 ms DT. The YOLOv4-tiny was selected as it offered a very similar mAP to YOLOv4, but with a much lower DT. The design was simulated on different terrains and behaved as expected.