Detection of citrus fruit and tree trunks in natural environments using a multi-elliptical boundary model

Object-oriented multi-scale segmentation and multi-feature fusion-based method for identifying typical fruit trees in arid regions using Sentinel-1/2 satellite images

Fruit tree identification that is quick and precise lays the groundwork for scientifically evaluating orchard yields and dynamically monitoring planting areas. This study aims to evaluate the applicability of time series Sentinel-1/2 satellite data for fruit tree classification and to provide a new method for accurately extracting fruit tree species. Therefore, the study area selected is the Tarim Basin, the most important fruit-growing region in northwest China. The main focus is on identifying several major fruit tree species in this region. Time series Sentinel-1/2 satellite images acquired from the Google Earth Engine (GEE) platform are used for the study. A multi-scale segmentation approach is applied, and six categories of features including spectral, phenological, texture, polarization, vegetation index, and red edge index features are constructed. A total of forth-four features are extracted and optimized using the Vi feature importance index to determine the best time phase. Based on this, an object-oriented (OO) segmentation combined with the Random Forest (RF) method is used to identify fruit tree species. To find the best method for fruit tree identification, the results are compared with three other widely used traditional machine learning algorithms: Support Vector Machine (SVM), Gradient Boosting Decision Tree (GBDT), and Classification and Regression Tree (CART). The results show that: (1) the object-oriented segmentation method helps to improve the accuracy of fruit tree identification features, and September satellite images provide the best time window for fruit tree identification, with spectral, phenological, and texture features contributing the most to fruit tree species identification. (2) The RF model has higher accuracy in identifying fruit tree species than other machine learning models, with an overall accuracy (OA) and a kappa coefficient (KC) of 94.60% and 93.74% respectively, indicating that the combination of object-oriented segmentation and RF algorithm has great value and potential for fruit tree identification and classification. This method can be applied to large-scale fruit tree remote sensing classification and provides an effective technical means for monitoring fruit tree planting areas using medium-to-high-resolution remote sensing images.

YOLOC-tiny: a generalized lightweight real-time detection model for multiripeness fruits of large non-green-ripe citrus in unstructured environments

This study addresses the challenges of low detection precision and limited generalization across various ripeness levels and varieties for large non-green-ripe citrus fruits in complex scenarios. We present a high-precision and lightweight model, YOLOC-tiny, built upon YOLOv7, which utilizes EfficientNet-B0 as the feature extraction backbone network. To augment sensing capabilities and improve detection accuracy, we embed a spatial and channel composite attention mechanism, the convolutional block attention module (CBAM), into the head's efficient aggregation network. Additionally, we introduce an adaptive and complete intersection over union regression loss function, designed by integrating the phenotypic features of large non-green-ripe citrus, to mitigate the impact of data noise and efficiently calculate detection loss. Finally, a layer-based adaptive magnitude pruning strategy is employed to further eliminate redundant connections and parameters in the model. Targeting three types of citrus widely planted in Sichuan Province-navel orange, Ehime Jelly orange, and Harumi tangerine-YOLOC-tiny achieves an impressive mean average precision (mAP) of 83.0%, surpassing most other state-of-the-art (SOTA) detectors in the same class. Compared with YOLOv7 and YOLOv8x, its mAP improved by 1.7% and 1.9%, respectively, with a parameter count of only 4.2M. In picking robot deployment applications, YOLOC-tiny attains an accuracy of 92.8% at a rate of 59 frames per second. This study provides a theoretical foundation and technical reference for upgrading and optimizing low-computing-power ground-based robots, such as those used for fruit picking and orchard inspection.

Predicting and Visualizing Citrus Color Transformation Using a Deep Mask-Guided Generative Network

Citrus rind color is a good indicator of fruit development, and methods to monitor and predict color transformation therefore help the decisions of crop management practices and harvest schedules. This work presents the complete workflow to predict and visualize citrus color transformation in the orchard featuring high accuracy and fidelity. A total of 107 sample Navel oranges were observed during the color transformation period, resulting in a dataset containing 7,535 citrus images. A framework is proposed that integrates visual saliency into deep learning, and it consists of a segmentation network, a deep mask-guided generative network, and a loss network with manually designed loss functions. Moreover, the fusion of image features and temporal information enables one single model to predict the rind color at different time intervals, thus effectively shrinking the number of model parameters. The semantic segmentation network of the framework achieves the mean intersection over a union score of 0.9694, and the generative network obtains a peak signal-to-noise ratio of 30.01 and a mean local style loss score of 2.710, which indicate both high quality and similarity of the generated images and are also consistent with human perception. To ease the applications in the real world, the model is ported to an Android-based application for mobile devices. The methods can be readily expanded to other fruit crops with a color transformation period. The dataset and the source code are publicly available at GitHub.

Detection of Dense Citrus Fruits by Combining Coordinated Attention and Cross-Scale Connection with Weighted Feature Fusion

The accuracy detection of individual citrus fruits in a citrus orchard environments is one of the key steps in realizing precision agriculture applications such as yield estimation, fruit thinning, and mechanical harvesting. This study proposes an improved object detection YOLOv5 model to achieve accurate the identification and counting of citrus fruits in an orchard environment. First, the latest visual attention mechanism coordinated attention module (CA) was inserted into an improved backbone network to focus on fruit-dense regions to recognize small target fruits. Second, an efficient two-way cross-scale connection and weighted feature fusion BiFPN in the neck network were used to replace the PANet multiscale feature fusion network, giving effective feature corresponding weights to fully fuse the high-level and bottom-level features. Finally, the varifocal loss function was used to calculate the model loss for better model training results. The results of the experiments on four varieties of citrus trees showed that our improved model proposed to this study could effectively identify dense small citrus fruits. Specifically, the recognized AP (average precision) reached 98.4%, and the average recognition time was 0.019 s per image. Compared with the original YOLOv5 (including deferent variants of n, s, m, l, and x), the increase in the average accuracy precision of the improved YOLOv5 ranged from 7.5% to 0.8% while maintaining similar average inference time. Four different citrus varieties were also tested to evaluate the generalization performance of the improved model. The method can be further used as a part in a vision system to provide technical support for the real-time and accurate detection of multiple fruit targets during mechanical picking in citrus orchards.

Pine Cone Detection Using Boundary Equilibrium Generative Adversarial Networks and Improved YOLOv3 Model

The real-time detection of pine cones in Korean pine forests is not only the data basis for the mechanized picking of pine cones, but also one of the important methods for evaluating the yield of Korean pine forests. In recent years, there has been a certain number of detection accuracy for image processing of fruits in trees using deep-learning methods, but the overall performance of these methods has not been satisfactory, and they have never been used in the detection of pine cones. In this paper, a pine cone detection method based on Boundary Equilibrium Generative Adversarial Networks (BEGAN) and You Only Look Once (YOLO) v3 mode is proposed to solve the problems of insufficient data set, inaccurate detection result and slow detection speed. First, we use traditional image augmentation technology and generative adversarial network BEGAN to implement data augmentation. Second, we introduced a densely connected network (DenseNet) structure in the backbone network of YOLOv3. Third, we expanded the detection scale of YOLOv3, and optimized the loss function of YOLOv3 using the Distance-IoU (DIoU) algorithm. Finally, we conducted a comparative experiment. The experimental results show that the performance of the model can be effectively improved by using BEGAN for data augmentation. Under same conditions, the improved YOLOv3 model is better than the Single Shot MultiBox Detector (SSD), the faster-regions with convolutional neural network (Faster R-CNN) and the original YOLOv3 model. The detection accuracy reaches 95.3%, and the detection efficiency is 37.8% higher than that of the original YOLOv3.

Optimization and Experimental Study of an Intelligent Bamboo-Splitting Machine Charging Manipulator

A nonautomatic bamboo-splitting machine must charge with material and change tools manually. However, manual charging is very dangerous. An intelligent bamboo-splitting machine can feed automatically and change tools intelligently and has broad application prospects. A charging manipulator is an important part of an intelligent bamboo-splitting machine. The size of the manipulator was optimized here using a genetic algorithm. The capture rate, centering rate, and dynamic characteristics of an intelligent bamboo-splitting machine charging manipulator, in which key factors were considered, were experimentally studied. First, three different manipulators, with arm lengths at 210, 220, and 230 mm, were developed. Then, the bamboo materials were divided into three gradients (60–85, 85–110, and 110–135 mm) according to diameter ranges. Accelerators were used to measure the manipulator arm dynamic characteristics, and a high-speed charge-coupled device was used to record the grasping process. Experimental results showed that the manipulator capture rate with an arm length of = 220 mm was as high as 100%, but that of manipulators with arm lengths of = 210 and 230 mm was 96 and 98.67%, respectively. Thus, the manipulator with a 220 mm arm length showed better performance than the other two manipulators. Trend curves of the influence of material diameter on capture time were similar to an exponential function.

Lychee Fruit Detection Based on Monocular Machine Vision in Orchard Environment

Due to the change of illumination environment and overlapping conditions caused by the neighboring fruits and other background objects, the simple application of the traditional machine vision method limits the detection accuracy of lychee fruits in natural orchard environments. Therefore, this research presented a detection method based on monocular machine vision to detect lychee fruits growing in overlapped conditions. Specifically, a combination of contrast limited adaptive histogram equalization (CLAHE), red/blue chromatic mapping, Otsu thresholding and morphology operations were adopted to segment the foreground regions of the lychees. A stepwise method was proposed for extracting individual lychee fruit from the lychee foreground region. The first step in this process was based on the relative position relation of the Hough circle and an equivalent area circle (equal to the area of the potential lychee foreground region) and was designed to distinguish lychee fruits growing in isolated or overlapped states. Then, a process based on the three-point definite circle theorem was performed to extract individual lychee fruits from the foreground regions of overlapped lychee fruit clusters. Finally, to enhance the robustness of the detection method, a local binary pattern support vector machine (LBP-SVM) was adopted to filter out the false positive detections generated by background chaff interferences. The performance of the presented method was evaluated using 485 images captured in a natural lychee orchard in Conghua (Area), Guangzhou. The detection results showed that the recall rate was 86.66%, the precision rate was greater than 87% and the F₁-score was 87.07%.

Individual Rubber Tree Segmentation Based on Ground-Based LiDAR Data and Faster R-CNN of Deep Learning

Rubber trees in southern China are often impacted by natural disturbances that can result in a tilted tree body. Accurate crown segmentation for individual rubber trees from scanned point clouds is an essential prerequisite for accurate tree parameter retrieval. In this paper, three plots of different rubber tree clones, PR107, CATAS 7-20-59, and CATAS 8-7-9, were taken as the study subjects. Through data collection using ground-based mobile light detection and ranging (LiDAR), a voxelisation method based on the scanned tree trunk data was proposed, and deep images (i.e., images normally used for deep learning) were generated through frontal and lateral projection transform of point clouds in each voxel with a length of 8 m and a width of 3 m. These images provided the training and testing samples for the faster region-based convolutional neural network (Faster R-CNN) of deep learning. Consequently, the Faster R-CNN combined with the generated training samples comprising 802 deep images with pre-marked trunk locations was trained to automatically recognize the trunk locations in the testing samples, which comprised 359 deep images. Finally, the point clouds for the lower parts of each trunk were extracted through back-projection transform from the recognized trunk locations in the testing samples and used as the seed points for the region’s growing algorithm to accomplish individual rubber tree crown segmentation. Compared with the visual inspection results, the recognition rate of our method reached 100% for the deep images of the testing samples when the images contained one or two trunks or the trunk information was slightly occluded by leaves. For the complicated cases, i.e., multiple trunks or overlapping trunks in one deep image or a trunk appearing in two adjacent deep images, the recognition accuracy of our method was greater than 90%. Our work represents a new method that combines a deep learning framework with point cloud processing for individual rubber tree crown segmentation based on ground-based mobile LiDAR scanned data.