GBCNet: In-Field Grape Berries Counting for Yield Estimation by Dilated CNNs

Semi-supervised Counting of Grape Berries in the Field Based on Density Mutual Exclusion

Automated counting of grape berries has become one of the most important tasks in grape yield prediction. However, dense distribution of berries and the severe occlusion between berries bring great challenges to counting algorithm based on deep learning. The collection of data required for model training is also a tedious and expensive work. To address these issues and cost-effectively count grape berries, a semi-supervised counting of grape berries in the field based on density mutual exclusion (CDMENet) is proposed. The algorithm uses VGG16 as the backbone to extract image features. Auxiliary tasks based on density mutual exclusion are introduced. The tasks exploit the spatial distribution pattern of grape berries in density levels to make full use of unlabeled data. In addition, a density difference loss is designed. The feature representation is enhanced by amplifying the difference of features between different density levels. The experimental results on the field grape berry dataset show that CDMENet achieves less counting errors. Compared with the state of the arts, coefficient of determination (R2) is improved by 6.10%, and mean absolute error and root mean square error are reduced by 49.36% and 54.08%, respectively. The code is available at https://github.com/youth-tang/CDMENet-main.

Instance Segmentation and Berry Counting of Table Grape before Thinning Based on AS-SwinT

Berry thinning is one of the most important tasks in the management of high-quality table grapes. Farmers often thin the berries per cluster to a standard number by counting. With an aging population, it is hard to find adequate skilled farmers to work during thinning season. It is urgent to design an intelligent berry-thinning machine to avoid exhaustive repetitive labor. A machine vision system that can determine the number of berries removed and locate the berries removed is a challenge for the thinning machine. A method for instance segmentation of berries and berry counting in a single bunch is proposed based on AS-SwinT. In AS-SwinT, Swin Transformer is performed as the backbone to extract the rich characteristics of grape berries. An adaptive feature fusion is introduced to the neck network to sufficiently preserve the underlying features and enhance the detection of small berries. The size of berries in the dataset is statistically analyzed to optimize the anchor scale, and Soft-NMS is used to filter the candidate frames to reduce the missed detection of densely shaded berries. Finally, the proposed method could achieve 65.7 APbox, 95.0 A P 0.5 box , 57 A P s box , 62.8 APmask, 94.3 A P 0.5 mask , 48 A P s mask , which is markedly superior to Mask R-CNN, Mask Scoring R-CNN, and Cascade Mask R-CNN. Linear regressions between predicted numbers and actual numbers are also developed to verify the precision of the proposed model. RMSE and R2 values are 7.13 and 0.95, respectively, which are substantially higher than other models, showing the advantage of the AS-SwinT model in berry counting estimation.

Designing a Proximal Sensing Camera Acquisition System for Vineyard Applications: Results and Feedback on 8 Years of Experiments

The potential of image proximal sensing for agricultural applications has been a prolific scientific subject in the recent literature. Its main appeal lies in the sensing of precise information about plant status, which is either harder or impossible to extract from lower-resolution downward-looking image sensors such as satellite or drone imagery. Yet, many theoretical and practical problems arise when dealing with proximal sensing, especially on perennial crops such as vineyards. Indeed, vineyards exhibit challenging physical obstacles and many degrees of variability in their layout. In this paper, we present the design of a mobile camera suited to vineyards and harsh experimental conditions, as well as the results and assessments of 8 years' worth of studies using that camera. These projects ranged from in-field yield estimation (berry counting) to disease detection, providing new insights on typical viticulture problems that could also be generalized to orchard crops. Different recommendations are then provided using small case studies, such as the difficulties related to framing plots with different structures or the mounting of the sensor on a moving vehicle. While results stress the obvious importance and strong benefits of a thorough experimental design, they also indicate some inescapable pitfalls, illustrating the need for more robust image analysis algorithms and better databases. We believe sharing that experience with the scientific community can only benefit the future development of these innovative approaches.

Behind the Leaves: Estimation of Occluded Grapevine Berries With Conditional Generative Adversarial Networks

The need for accurate yield estimates for viticulture is becoming more important due to increasing competition in the wine market worldwide. One of the most promising methods to estimate the harvest is berry counting, as it can be approached non-destructively, and its process can be automated. In this article, we present a method that addresses the challenge of occluded berries with leaves to obtain a more accurate estimate of the number of berries that will enable a better estimate of the harvest. We use generative adversarial networks, a deep learning-based approach that generates a highly probable scenario behind the leaves exploiting learned patterns from images with non-occluded berries. Our experiments show that the estimate of the number of berries after applying our method is closer to the manually counted reference. In contrast to applying a factor to the berry count, our approach better adapts to local conditions by directly involving the appearance of the visible berries. Furthermore, we show that our approach can identify which areas in the image should be changed by adding new berries without explicitly requiring information about hidden areas.

Grape Berry Detection and Size Measurement Based on Edge Image Processing and Geometric Morphology

Counting grape berries and measuring their size can provide accurate data for robot picking behavior decision-making, yield estimation, and quality evaluation. When grapes are picked, there is a strong uncertainty in the external environment and the shape of the grapes. Counting grape berries and measuring berry size are challenging tasks. Computer vision has made a huge breakthrough in this field. Although the detection method of grape berries based on 3D point cloud information relies on scanning equipment to estimate the number and yield of grape berries, the detection method is difficult to generalize. Grape berry detection based on 2D images is an effective method to solve this problem. However, it is difficult for traditional algorithms to accurately measure the berry size and other parameters, and there is still the problem of the low robustness of berry counting. In response to the above problems, we propose a grape berry detection method based on edge image processing and geometric morphology. The edge contour search and the corner detection algorithm are introduced to detect the concave point position of the berry edge contour extracted by the Canny algorithm to obtain the best contour segment. To correctly obtain the edge contour information of each berry and reduce the error grouping of contour segments, this paper proposes an algorithm for combining contour segments based on clustering search strategy and rotation direction determination, which realizes the correct reorganization of the segmented contour segments, to achieve an accurate calculation of the number of berries and an accurate measurement of their size. The experimental results prove that our proposed method has an average accuracy of 87.76% for the detection of the concave points of the edge contours of different types of grapes, which can achieve a good edge contour segmentation. The average accuracy of the detection of the number of grapes berries in this paper is 91.42%, which is 4.75% higher than that of the Hough transform. The average error between the measured berry size and the actual berry size is 2.30 mm, and the maximum error is 5.62 mm, which is within a reasonable range. The results prove that the method proposed in this paper is robust enough to detect different types of grape berries.

In-Field Automatic Detection of Grape Bunches under a Totally Uncontrolled Environment

An early estimation of the exact number of fruits, flowers, and trees helps farmers to make better decisions on cultivation practices, plant disease prevention, and the size of harvest labor force. The current practice of yield estimation based on manual counting of fruits or flowers by workers is a time consuming and expensive process and it is not feasible for large fields. Automatic yield estimation based on robotic agriculture provides a viable solution in this regard. In a typical image classification process, the task is not only to specify the presence or absence of a given object on a specific location, while counting how many objects are present in the scene. The success of these tasks largely depends on the availability of a large amount of training samples. This paper presents a detector of bunches of one fruit, grape, based on a deep convolutional neural network trained to detect vine bunches directly on the field. Experimental results show a 91% mean Average Precision.