Robotic Technologies for High-Throughput Plant Phenotyping: Contemporary Reviews and Future Perspectives

Advances in the tea plants phenotyping using hyperspectral imaging technology

Rapid detection of plant phenotypic traits is crucial for plant breeding and cultivation. Traditional measurement methods are carried out by rich-experienced agronomists, which are time-consuming and labor-intensive. However, with the increasing demand for rapid and high-throughput testing in tea plants traits, digital breeding and smart cultivation of tea plants rely heavily on precise plant phenotypic trait measurement techniques, among which hyperspectral imaging (HSI) technology stands out for its ability to provide real-time and rich-information. In this paper, we provide a comprehensive overview of the principles of hyperspectral imaging technology, the processing methods of cubic data, and relevant algorithms in tea plant phenomics, reviewing the progress of applying hyperspectral imaging technology to obtain information on tea plant phenotypes, growth conditions, and quality indicators under environmental stress. Lastly, we discuss the challenges faced by HSI technology in the detection of tea plant phenotypic traits from different perspectives, propose possible solutions, and envision the potential development prospects of HSI technology in the digital breeding and smart cultivation of tea plants. This review aims to provide theoretical and technical support for the application of HSI technology in detecting tea plant phenotypic information, further promoting the trend of developing high quality and high yield tea leaves.

All-organic transparent plant e-skin for noninvasive phenotyping

Real-time in situ monitoring of plant physiology is essential for establishing a phenotyping platform for precision agriculture. A key enabler for this monitoring is a device that can be noninvasively attached to plants and transduce their physiological status into digital data. Here, we report an all-organic transparent plant e-skin by micropatterning poly(3,4-ethylenedioxythiophene) polystyrene sulfonate (PEDOT:PSS) on polydimethylsiloxane (PDMS) substrate. This plant e-skin is optically and mechanically invisible to plants with no observable adverse effects to plant health. We demonstrate the capabilities of our plant e-skins as strain and temperature sensors, with the application to Brassica rapa leaves for collecting corresponding parameters under normal and abiotic stress conditions. Strains imposed on the leaf surface during growth as well as diurnal fluctuation of surface temperature were captured. We further present a digital-twin interface to visualize real-time plant surface environment, providing an intuitive and vivid platform for plant phenotyping.

Cyber-agricultural systems for crop breeding and sustainable production

The cyber-agricultural system (CAS) represents an overarching framework of agriculture that leverages recent advances in ubiquitous sensing, artificial intelligence, smart actuators, and scalable cyberinfrastructure (CI) in both breeding and production agriculture. We discuss the recent progress and perspective of the three fundamental components of CAS - sensing, modeling, and actuation - and the emerging concept of agricultural digital twins (DTs). We also discuss how scalable CI is becoming a key enabler of smart agriculture. In this review we shed light on the significance of CAS in revolutionizing crop breeding and production by enhancing efficiency, productivity, sustainability, and resilience to changing climate. Finally, we identify underexplored and promising future directions for CAS research and development.

Root Development Monitoring under Different Water Supply Levels in Processing Tomato Plants

Managing crop yields and optimizing water use is a global challenge, as fresh water supply decreases rapidly and demand remains high. Therefore, understanding how plants react to varying water levels is crucial for efficient water usage. This study evaluates how tomato plants adapt to varying water levels (100%, 50% of crop evapotranspiration, and non-irrigated control) over two growing seasons in 2020 and 2021. Root images were captured weekly during an 8-week monitoring period in 2020 and 6 weeks in 2021 using a non-destructive CI-600 in-situ root imager at depths between 10 and 70 cm. Under water stress, plants developed deeper, more extensive root systems to maximize water uptake, consistent with prior research. Root depth and architecture varied with soil depth and the severity of water stress. Year-to-year variations were also found, likely due to changes in irrigation levels and environmental conditions such as temperature. SPAD values were higher under control conditions, especially in the 2021 growing season, suggesting reduced chlorophyll degradation, while no significant differences were observed in chlorophyll fluorescence (Fv/Fm) between treatments, suggesting stable photosynthetic efficiency under varied water stress conditions. These findings contribute to our understanding of root zone optimization and drought-resilient cultivar breeding, contributing to more sustainable agricultural practices.

Adapting crop production to climate change and air pollution at different scales

Air pollution and climate change are tightly interconnected and jointly affect field crop production and agroecosystem health. Although our understanding of the individual and combined impacts of air pollution and climate change factors is improving, the adaptation of crop production to concurrent air pollution and climate change remains challenging to resolve. Here we evaluate recent advances in the adaptation of crop production to climate change and air pollution at the plant, field and ecosystem scales. The main approaches at the plant level include the integration of genetic variation, molecular breeding and phenotyping. Field-level techniques include optimizing cultivation practices, promoting mixed cropping and diversification, and applying technologies such as antiozonants, nanotechnology and robot-assisted farming. Plant- and field-level techniques would be further facilitated by enhancing soil resilience, incorporating precision agriculture and modifying the hydrology and microclimate of agricultural landscapes at the ecosystem level. Strategies and opportunities for crop production under climate change and air pollution are discussed.

The Era of Plant Breeding: Conventional Breeding to Genomics-assisted Breeding for Crop Improvement

Plant breeding has made a significant contribution to increasing agricultural production. Conventional breeding based on phenotypic selection is not effective for crop improvement. Because phenotype is considerably influenced by environmental factors, which will affect the selection of breeding materials for crop improvement. The past two decades have seen tremendous progress in plant breeding research. Especially the availability of high-throughput molecular markers followed by genomic-assisted approaches significantly contributed to advancing plant breeding. Integration of speed breeding with genomic and phenomic facilities allowed rapid quantitative trait loci (QTL)/gene identifications and ultimately accelerated crop improvement programs. The advances in sequencing technology helps to understand the genome organization of many crops and helped with genomic selection in crop breeding. Plant breeding has gradually changed from phenotype-to-genotype-based to genotype-to-phenotype-based selection. High-throughput phenomic platforms have played a significant role in the modern breeding program and are considered an essential part of precision breeding. In this review, we discuss the rapid advance in plant breeding technology for efficient crop improvements and provide details on various approaches/platforms that are helpful for crop improvement. This review will help researchers understand the recent developments in crop breeding and improvements.

Efficacy of plant breeding using genomic information

Genomic selection (GS) proposed by Meuwissen et al. more than 20 years ago, is revolutionizing plant and animal breeding. Although GS has been widely accepted and applied to plant and animal breeding, there are many factors affecting its efficacy. We studied 14 real datasets to respond to the practical question of whether the accuracy of genomic prediction increases when considering genomic as compared with not using genomic. We found across traits, environments, datasets, and metrics, that the average gain in prediction accuracy when genomic information is considered was 26.31%, while only in terms of Pearson's correlation the gain was of 46.1%, while only in terms of normalized root mean squared error the gain was of 6.6%. If the quality of the makers and relatedness of the individuals increase, major gains in prediction accuracy can be obtained, but if these two factors decrease, a lower increase is possible. Finally, our findings reinforce genomic is vital for improving the prediction accuracy and, therefore, the realized genetic gain in genomic assisted plant breeding programs.

A Systematic Review of Effective Hardware and Software Factors Affecting High-Throughput Plant Phenotyping

Plant phenotyping studies the complex characteristics of plants, with the aim of evaluating and assessing their condition and finding better exemplars. Recently, a new branch emerged in the phenotyping field, namely, high-throughput phenotyping (HTP). Specifically, HTP exploits modern data sampling techniques to gather a high amount of data that can be used to improve the effectiveness of phenotyping. Hence, HTP combines the knowledge derived from the phenotyping domain with computer science, engineering, and data analysis techniques. In this scenario, machine learning (ML) and deep learning (DL) algorithms have been successfully integrated with noninvasive imaging techniques, playing a key role in automation, standardization, and quantitative data analysis. This study aims to systematically review two main areas of interest for HTP: hardware and software. For each of these areas, two influential factors were identified: for hardware, platforms and sensing equipment were analyzed; for software, the focus was on algorithms and new trends. The study was conducted following the PRISMA protocol, which allowed the refinement of the research on a wide selection of papers by extracting a meaningful dataset of 32 articles of interest. The analysis highlighted the diffusion of ground platforms, which were used in about 47% of reviewed methods, and RGB sensors, mainly due to their competitive costs, high compatibility, and versatility. Furthermore, DL-based algorithms accounted for the larger share (about 69%) of reviewed approaches, mainly due to their effectiveness and the focus posed by the scientific community over the last few years. Future research will focus on improving DL models to better handle hardware-generated data. The final aim is to create integrated, user-friendly, and scalable tools that can be directly deployed and used on the field to improve the overall crop yield.

Genomics combined with UAS data enhances prediction of grain yield in winter wheat

With the human population continuing to increase worldwide, there is pressure to employ novel technologies to increase genetic gain in plant breeding programs that contribute to nutrition and food security. Genomic selection (GS) has the potential to increase genetic gain because it can accelerate the breeding cycle, increase the accuracy of estimated breeding values, and improve selection accuracy. However, with recent advances in high throughput phenotyping in plant breeding programs, the opportunity to integrate genomic and phenotypic data to increase prediction accuracy is present. In this paper, we applied GS to winter wheat data integrating two types of inputs: genomic and phenotypic. We observed the best accuracy of grain yield when combining both genomic and phenotypic inputs, while only using genomic information fared poorly. In general, the predictions with only phenotypic information were very competitive to using both sources of information, and in many cases using only phenotypic information provided the best accuracy. Our results are encouraging because it is clear we can enhance the prediction accuracy of GS by integrating high quality phenotypic inputs in the models.

Machine Learning-Assisted Approaches in Modernized Plant Breeding Programs

In the face of a growing global population, plant breeding is being used as a sustainable tool for increasing food security. A wide range of high-throughput omics technologies have been developed and used in plant breeding to accelerate crop improvement and develop new varieties with higher yield performance and greater resilience to climate changes, pests, and diseases. With the use of these new advanced technologies, large amounts of data have been generated on the genetic architecture of plants, which can be exploited for manipulating the key characteristics of plants that are important for crop improvement. Therefore, plant breeders have relied on high-performance computing, bioinformatics tools, and artificial intelligence (AI), such as machine-learning (ML) methods, to efficiently analyze this vast amount of complex data. The use of bigdata coupled with ML in plant breeding has the potential to revolutionize the field and increase food security. In this review, some of the challenges of this method along with some of the opportunities it can create will be discussed. In particular, we provide information about the basis of bigdata, AI, ML, and their related sub-groups. In addition, the bases and functions of some learning algorithms that are commonly used in plant breeding, three common data integration strategies for the better integration of different breeding datasets using appropriate learning algorithms, and future prospects for the application of novel algorithms in plant breeding will be discussed. The use of ML algorithms in plant breeding will equip breeders with efficient and effective tools to accelerate the development of new plant varieties and improve the efficiency of the breeding process, which are important for tackling some of the challenges facing agriculture in the era of climate change.

Evaluation of convolutional neural networks for herbicide susceptibility-based weed detection in turf

Deep learning methods for weed detection typically focus on distinguishing weed species, but a variety of weed species with comparable plant morphological characteristics may be found in turfgrass. Thus, it is difficult for deep learning models to detect and distinguish every weed species with high accuracy. Training convolutional neural networks for detecting weeds susceptible to herbicides can offer a new strategy for implementing site-specific weed detection in turf. DenseNet, EfficientNet-v2, and ResNet showed high F₁ scores (≥0.986) and MCC values (≥0.984) to detect and distinguish the sub-images containing dollarweed, goosegrass, old world diamond-flower, purple nutsedge, or Virginia buttonweed growing in bermudagrass turf. However, they failed to reliably detect crabgrass and tropical signalgrass due to the similarity in plant morphology. When training the convolutional neural networks for detecting and distinguishing the sub-images containing weeds susceptible to ACCase-inhibitors, weeds susceptible to ALS-inhibitors, or weeds susceptible to synthetic auxin herbicides, all neural networks evaluated in this study achieved excellent F₁ scores (≥0.995) and MCC values (≥0.994) in the validation and testing datasets. ResNet demonstrated the fastest inference rate and outperformed the other convolutional neural networks on detection efficiency, while the slow inference of EfficientNet-v2 may limit its potential applications. Grouping different weed species growing in turf according to their susceptibility to herbicides and detecting and distinguishing weeds by herbicide categories enables the implementation of herbicide susceptibility-based precision herbicide application. We conclude that the proposed method is an effective strategy for site-specific weed detection in turf, which can be employed in a smart sprayer to achieve precision herbicide spraying.

Dynamic detection of three-dimensional crop phenotypes based on a consumer-grade RGB-D camera

INTRODUCTION

Nondestructive detection of crop phenotypic traits in the field is very important for crop breeding. Ground-based mobile platforms equipped with sensors can efficiently and accurately obtain crop phenotypic traits. In this study, we propose a dynamic 3D data acquisition method in the field suitable for various crops by using a consumer-grade RGB-D camera installed on a ground-based movable platform, which can collect RGB images as well as depth images of crop canopy sequences dynamically.

METHODS

A scale-invariant feature transform (SIFT) operator was used to detect adjacent date frames acquired by the RGB-D camera to calculate the point cloud alignment coarse matching matrix and the displacement distance of adjacent images. The data frames used for point cloud matching were selected according to the calculated displacement distance. Then, the colored ICP (iterative closest point) algorithm was used to determine the fine matching matrix and generate point clouds of the crop row. The clustering method was applied to segment the point cloud of each plant from the crop row point cloud, and 3D phenotypic traits, including plant height, leaf area and projected area of individual plants, were measured.

RESULTS AND DISCUSSION

We compared the effects of LIDAR and image-based 3D reconstruction methods, and experiments were carried out on corn, tobacco, cottons and Bletilla striata in the seedling stage. The results show that the measurements of the plant height (R²= 0.9~0.96, RSME = 0.015~0.023 m), leaf area (R²= 0.8~0.86, RSME = 0.0011~0.0041 m ² ) and projected area (R² = 0.96~0.99) have strong correlations with the manual measurement results. Additionally, 3D reconstruction results with different moving speeds and times throughout the day and in different scenes were also verified. The results show that the method can be applied to dynamic detection with a moving speed up to 0.6 m/s and can achieve acceptable detection results in the daytime, as well as at night. Thus, the proposed method can improve the efficiency of individual crop 3D point cloud data extraction with acceptable accuracy, which is a feasible solution for crop seedling 3D phenotyping outdoors.

Analysis of the multilayer organization of a sunflower leaf during dehydration with terahertz time-domain spectroscopy

We apply reverse engineering techniques (RET) to analyze the dehydration process of a sunflower leaf with terahertz time-domain spectroscopy. The multilayer structure of the leaf is extracted with accuracy during the entire process. Time variations of thickness and the complex index are emphasized for all leaf layers (2 cuticules, 2 epiderms, and 2 mesophylls). The global thickness of the sunflower leaf is reduced by up to 40% of its initial value.

A Review of High-Throughput Field Phenotyping Systems: Focusing on Ground Robots

Manual assessments of plant phenotypes in the field can be labor-intensive and inefficient. The high-throughput field phenotyping systems and in particular robotic systems play an important role to automate data collection and to measure novel and fine-scale phenotypic traits that were previously unattainable by humans. The main goal of this paper is to review the state-of-the-art of high-throughput field phenotyping systems with a focus on autonomous ground robotic systems. This paper first provides a brief review of nonautonomous ground phenotyping systems including tractors, manually pushed or motorized carts, gantries, and cable-driven systems. Then, a detailed review of autonomous ground phenotyping robots is provided with regard to the robot's main components, including mobile platforms, sensors, manipulators, computing units, and software. It also reviews the navigation algorithms and simulation tools developed for phenotyping robots and the applications of phenotyping robots in measuring plant phenotypic traits and collecting phenotyping datasets. At the end of the review, this paper discusses current major challenges and future research directions.

Introducing Three-Dimensional Scanning for Phenotyping of Olive Fruits Based on an Extensive Germplasm Survey

Morphological characterization of olive (Olea europaea L.) varieties to detect desirable traits has been based on the training of expert panels and implementation of laborious multiyear measurements with limitations in accuracy and throughput of measurements. The present study compares two- and three-dimensional imaging systems for phenotyping a large dataset of 50 olive varieties maintained in the National Germplasm Depository of Greece, employing this technology for the first time in olive fruit and endocarps. The olive varieties employed for the present study exhibited high phenotypic variation, particularly for the endocarp shadow area, which ranged from 0.17−3.34 cm2 as evaluated via 2D and 0.32−2.59 cm2 as determined by 3D scanning. We found significant positive correlations (p < 0.001) between the two methods for eight quantitative morphological traits using the Pearson correlation coefficient. The highest correlation between the two methods was detected for the endocarp length (r = 1) and width (r = 1) followed by the fruit length (r = 0.9865), mucro length (r = 0.9631), fruit shadow area (r = 0.9573), fruit width (r = 0.9480), nipple length (r = 0.9441), and endocarp area (r = 0.9184). The present study unraveled novel morphological indicators of olive fruits and endocarps such as volume, total area, up- and down-skin area, and center of gravity using 3D scanning. The highest volume and area regarding both endocarp and fruit were observed for ‘Gaidourelia’. This methodology could be integrated into existing olive breeding programs, especially when the speed of scanning increases. Another potential future application could be assessing olive fruit quality on the trees or in the processing facilities.

A Loosely Coupled Extended Kalman Filter Algorithm for Agricultural Scene-Based Multi-Sensor Fusion

With the arrival of aging society and the development of modern agriculture, the use of agricultural robots for large-scale agricultural production activities will become a major trend in the future. Therefore, it is necessary to develop suitable robots and autonomous navigation technology for agricultural production. However, there is still a problem of external noise and other factors causing the failure of the navigation system. To solve this problem, we propose an agricultural scene-based multi-sensor fusion method via a loosely coupled extended Kalman filter algorithm to reduce interference from external environment. Specifically, the proposed method fuses inertial measurement unit (IMU), robot odometer (ODOM), global navigation and positioning system (GPS), and visual inertial odometry (VIO), and uses visualization tools to simulate and analyze the robot trajectory and error. In experiments, we verify the high accuracy and the robustness of the proposed algorithm when sensors fail. The experimental results show that the proposed algorithm has better accuracy and robustness on the agricultural dataset than other algorithms.

Deep Learning-Based Occlusion Handling of Overlapped Plants for Robotic Grasping

Instance segmentation of overlapping plants to detect their grasps for possible robotic grasping presents a challenging task due to the need to address the problem of occlusion. We addressed the problem of occlusion using a powerful convolutional neural network for segmenting objects with complex forms and occlusions. The network was trained with a novel dataset named the “occluded plants” dataset, containing real and synthetic images of plant cuttings on flat surfaces with differing degrees of occlusion. The synthetic images were created using the novel framework for synthesizing 2D images by using all plant cutting instances of available real images. In addition to the method for occlusion handling for overlapped plants, we present a novel method for determining the grasps of segmented plant cuttings that is based on conventional image processing. The result of the employed instance segmentation network on our plant dataset shows that it can accurately segment the overlapped plants, and it has a robust performance for different levels of occlusions. The presented plants’ grasp detection method achieved 94% on the rectangle metric which had an angular deviation of 30 degrees and an IoU of 0.50. The achieved results show the viability of our approach on plant species with an irregular shape and provide confidence that the presented method can provide a basis for various applications in the food and agricultural industries.

Pathways to de novo domestication of crop wild relatives

Growing knowledge about crop domestication, combined with increasingly powerful gene-editing toolkits, sets the stage for the continual domestication of crop wild relatives and other lesser-known plant species.

In Situ Measuring Stem Diameters of Maize Crops with a High-Throughput Phenotyping Robot

Robotic High-Throughput Phenotyping (HTP) technology has been a powerful tool for selecting high-quality crop varieties among large quantities of traits. Due to the advantages of multi-view observation and high accuracy, ground HTP robots have been widely studied in recent years. In this paper, we study an ultra-narrow wheeled robot equipped with RGB-D cameras for inter-row maize HTP. The challenges of the narrow operating space, intensive light changes, and messy cross-leaf interference in rows of maize crops are considered. An in situ and inter-row stem diameter measurement method for HTP robots is proposed. To this end, we first introduce the stem diameter measurement pipeline, in which a convolutional neural network is employed to detect stems, and the point cloud is analyzed to estimate the stem diameters. Second, we present a clustering strategy based on DBSCAN for extracting stem point clouds under the condition that the stem is shaded by dense leaves. Third, we present a point cloud filling strategy to fill the stem region with missing depth values due to the occlusion by other organs. Finally, we employ convex hull and plane projection of the point cloud to estimate the stem diameters. The results show that the R2 and RMSE of stem diameter measurement are up to 0.72 and 2.95 mm, demonstrating its effectiveness.

Design Considerations for In-Field Measurement of Plant Architecture Traits Using Ground-Based Platforms

This work provides a high-level overview of system design considerations for measuring plant architecture traits in row crops using ground-based, mobile platforms. High-throughput phenotyping technologies are commonly deployed in isolated growth chambers or greenhouses; however, there is a need for field-based systems to measure large quantities of plants exposed to natural climates throughout a growing season. High-throughput methods using ground-based mobile systems collect valuable phenotypic information at higher temporal resolutions compared to manual methods (e.g., handheld calipers and measuring sticks). Additionally, the close proximity to plants when using ground-based systems compared to aerial platforms enables plant phenotyping at the organ level. While there is no single best platform for obtaining ground-based plant measurements across crop varieties with different planting configurations, there are a wide range of off-the-shelf systems and sensors that can be integrated to accommodate varying row widths, plant spacing, plant heights, and plot sizes, in addition to emerging commercially available platforms. This chapter will provide an overview of sensor types suitable for phenotyping plant size and shape, as well as provide guidance for deployment with ground-based systems, including push carts or buggies, modified tractors, and robotic platforms.