Plant Phenotyping: An Active Vision Cell for Three-Dimensional Plant Shoot Reconstruction

A fast phenotype approach of 3D point clouds of Pinus massoniana seedlings

The phenotyping of Pinus massoniana seedlings is essential for breeding, vegetation protection, resource investigation, and so on. Few reports regarding estimating phenotypic parameters accurately in the seeding stage of Pinus massoniana plants using 3D point clouds exist. In this study, seedlings with heights of approximately 15-30 cm were taken as the research object, and an improved approach was proposed to automatically calculate five key parameters. The key procedure of our proposed method includes point cloud preprocessing, stem and leaf segmentation, and morphological trait extraction steps. In the skeletonization step, the cloud points were sliced in vertical and horizontal directions, gray value clustering was performed, the centroid of the slice was regarded as the skeleton point, and the alternative skeleton point of the main stem was determined by the DAG single source shortest path algorithm. Then, the skeleton points of the canopy in the alternative skeleton point were removed, and the skeleton point of the main stem was obtained. Last, the main stem skeleton point after linear interpolation was restored, while stem and leaf segmentation was achieved. Because of the leaf morphological characteristics of Pinus massoniana, its leaves are large and dense. Even using a high-precision industrial digital readout, it is impossible to obtain a 3D model of Pinus massoniana leaves. In this study, an improved algorithm based on density and projection is proposed to estimate the relevant parameters of Pinus massoniana leaves. Finally, five important phenotypic parameters, namely plant height, stem diameter, main stem length, regional leaf length, and total leaf number, are obtained from the skeleton and the point cloud after separation and reconstruction. The experimental results showed that there was a high correlation between the actual value from manual measurement and the predicted value from the algorithm output. The accuracies of the main stem diameter, main stem length, and leaf length were 93.5%, 95.7%, and 83.8%, respectively, which meet the requirements of real applications.

How to make sense of 3D representations for plant phenotyping: a compendium of processing and analysis techniques

Computer vision technology is moving more and more towards a three-dimensional approach, and plant phenotyping is following this trend. However, despite its potential, the complexity of the analysis of 3D representations has been the main bottleneck hindering the wider deployment of 3D plant phenotyping. In this review we provide an overview of typical steps for the processing and analysis of 3D representations of plants, to offer potential users of 3D phenotyping a first gateway into its application, and to stimulate its further development. We focus on plant phenotyping applications where the goal is to measure characteristics of single plants or crop canopies on a small scale in research settings, as opposed to large scale crop monitoring in the field.

Quantification of the three-dimensional root system architecture using an automated rotating imaging system

BACKGROUND

Crop breeding based on root system architecture (RSA) optimization is an essential factor for improving crop production in developing countries. Identification, evaluation, and selection of root traits of soil-grown crops require innovations that enable high-throughput and accurate quantification of three-dimensional (3D) RSA of crops over developmental time.

RESULTS

We proposed an automated imaging system and 3D imaging data processing pipeline to quantify the 3D RSA of soil-grown individual plants across seedlings to the mature stage. A multi-view automated imaging system composed of a rotary table and an imaging arm with 12 cameras mounted with a combination of fan-shaped and vertical distribution was developed to obtain 3D image data of roots grown on a customized root support mesh. A 3D imaging data processing pipeline was developed to quantify the 3D RSA based on the point cloud generated from multi-view images. The global architecture of root systems can be quantified automatically. Detailed analysis of the reconstructed 3D root model also allowed us to investigate the Spatio-temporal distribution of roots. A method combining horizontal slicing and iterative erosion and dilation was developed to automatically segment different root types, and identify local root traits (e.g., length, diameter of the main root, and length, diameter, initial angle, and the number of nodal roots or lateral roots). One maize (Zea mays L.) cultivar and two rapeseed (Brassica napus L.) cultivars at different growth stages were selected to test the performance of the automated imaging system and 3D imaging data processing pipeline.

CONCLUSIONS

The results demonstrated the capabilities of the proposed imaging and analytical system for high-throughput phenotyping of root traits for both monocotyledons and dicotyledons across growth stages. The proposed system offers a potential tool to further explore the 3D RSA for improving root traits and agronomic qualities of crops.

A Review of High-Throughput Field Phenotyping Systems: Focusing on Ground Robots

Manual assessments of plant phenotypes in the field can be labor-intensive and inefficient. The high-throughput field phenotyping systems and in particular robotic systems play an important role to automate data collection and to measure novel and fine-scale phenotypic traits that were previously unattainable by humans. The main goal of this paper is to review the state-of-the-art of high-throughput field phenotyping systems with a focus on autonomous ground robotic systems. This paper first provides a brief review of nonautonomous ground phenotyping systems including tractors, manually pushed or motorized carts, gantries, and cable-driven systems. Then, a detailed review of autonomous ground phenotyping robots is provided with regard to the robot's main components, including mobile platforms, sensors, manipulators, computing units, and software. It also reviews the navigation algorithms and simulation tools developed for phenotyping robots and the applications of phenotyping robots in measuring plant phenotypic traits and collecting phenotyping datasets. At the end of the review, this paper discusses current major challenges and future research directions.

Phenotyping Complex Plant Structures with a Large Format Industrial Scale High-Resolution X-Ray Tomography Instrument

Phenotyping specific plant traits is difficult when the samples to be measured are architecturally complex. Inflorescence and root system traits are of great biological interest, but these structures present unique phenotyping challenges due to their often complicated and three-dimensional (3D) forms. We describe how a large industrial scale X-ray tomography (XRT) instrument can be used to scan architecturally complex plant structures for the goal of rapid and accurate measurement of traits that are otherwise cumbersome or not possible to capture by other means. The combination of a large imaging cabinet that can accommodate a wide range of sample size geometries and a variable microfocus reflection X-ray source allows noninvasive X-ray imaging and 3D volume generation of diverse sample types. Specific sample fixturing (mounting) and scanning conditions are presented. These techniques can be moderate to high throughput and still provide unprecedented levels of accuracy and information content in the 3D volume data they generate.

A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction

Plant phenotyping is essential in plant breeding and management. High-throughput data acquisition and automatic phenotypes extraction are common concerns in plant phenotyping. Despite the development of phenotyping platforms and the realization of high-throughput three-dimensional (3D) data acquisition in tall plants, such as maize, handling small-size plants with complex structural features remains a challenge. This study developed a miniaturized shoot phenotyping platform MVS-Pheno V2 focusing on low plant shoots. The platform is an improvement of MVS-Pheno V1 and was developed based on multi-view stereo 3D reconstruction. It has the following four components: Hardware, wireless communication and control, data acquisition system, and data processing system. The hardware sets the rotation on top of the platform, separating plants to be static while rotating. A novel local network was established to realize wireless communication and control; thus, preventing cable twining. The data processing system was developed to calibrate point clouds and extract phenotypes, including plant height, leaf area, projected area, shoot volume, and compactness. This study used three cultivars of wheat shoots at four growth stages to test the performance of the platform. The mean absolute percentage error of point cloud calibration was 0.585%. The squared correlation coefficient R ² was 0.9991, 0.9949, and 0.9693 for plant height, leaf length, and leaf width, respectively. The root mean squared error (RMSE) was 0.6996, 0.4531, and 0.1174 cm for plant height, leaf length, and leaf width. The MVS-Pheno V2 platform provides an alternative solution for high-throughput phenotyping of low individual plants and is especially suitable for shoot architecture-related plant breeding and management studies.

Robotic Assay for Drought (RoAD): an automated phenotyping system for brassinosteroid and drought responses

Brassinosteroids (BRs) are a group of plant steroid hormones involved in regulating growth, development, and stress responses. Many components of the BR pathway have previously been identified and characterized. However, BR phenotyping experiments are typically performed in a low-throughput manner, such as on Petri plates. Additionally, the BR pathway affects drought responses, but drought experiments are time consuming and difficult to control. To mitigate these issues and increase throughput, we developed the Robotic Assay for Drought (RoAD) system to perform BR and drought response experiments in soil-grown Arabidopsis plants. RoAD is equipped with a robotic arm, a rover, a bench scale, a precisely controlled watering system, an RGB camera, and a laser profilometer. It performs daily weighing, watering, and imaging tasks and is capable of administering BR response assays by watering plants with Propiconazole (PCZ), a BR biosynthesis inhibitor. We developed image processing algorithms for both plant segmentation and phenotypic trait extraction to accurately measure traits including plant area, plant volume, leaf length, and leaf width. We then applied machine learning algorithms that utilize the extracted phenotypic parameters to identify image-derived traits that can distinguish control, drought-treated, and PCZ-treated plants. We carried out PCZ and drought experiments on a set of BR mutants and Arabidopsis accessions with altered BR responses. Finally, we extended the RoAD assays to perform BR response assays using PCZ in Zea mays (maize) plants. This study establishes an automated and non-invasive robotic imaging system as a tool to accurately measure morphological and growth-related traits of Arabidopsis and maize plants in 3D, providing insights into the BR-mediated control of plant growth and stress responses.

Deep Segmentation of Point Clouds of Wheat

The 3D analysis of plants has become increasingly effective in modeling the relative structure of organs and other traits of interest. In this paper, we introduce a novel pattern-based deep neural network, Pattern-Net, for segmentation of point clouds of wheat. This study is the first to segment the point clouds of wheat into defined organs and to analyse their traits directly in 3D space. Point clouds have no regular grid and thus their segmentation is challenging. Pattern-Net creates a dynamic link among neighbors to seek stable patterns from a 3D point set across several levels of abstraction using the K-nearest neighbor algorithm. To this end, different layers are connected to each other to create complex patterns from the simple ones, strengthen dynamic link propagation, alleviate the vanishing-gradient problem, encourage link reuse and substantially reduce the number of parameters. The proposed deep network is capable of analysing and decomposing unstructured complex point clouds into semantically meaningful parts. Experiments on a wheat dataset verify the effectiveness of our approach for segmentation of wheat in 3D space.

Active Vision and Surface Reconstruction for 3D Plant Shoot Modelling

Plant phenotyping is the quantitative description of a plant's physiological, biochemical, and anatomical status which can be used in trait selection and helps to provide mechanisms to link underlying genetics with yield. Here, an active vision- based pipeline is presented which aims to contribute to reducing the bottleneck associated with phenotyping of architectural traits. The pipeline provides a fully automated response to photometric data acquisition and the recovery of three-dimensional (3D) models of plants without the dependency of botanical expertise, whilst ensuring a non-intrusive and non-destructive approach. Access to complete and accurate 3D models of plants supports computation of a wide variety of structural measurements. An Active Vision Cell (AVC) consisting of a camera-mounted robot arm plus combined software interface and a novel surface reconstruction algorithm is proposed. This pipeline provides a robust, flexible, and accurate method for automating the 3D reconstruction of plants. The reconstruction algorithm can reduce noise and provides a promising and extendable framework for high throughput phenotyping, improving current state-of-the-art methods. Furthermore, the pipeline can be applied to any plant species or form due to the application of an active vision framework combined with the automatic selection of key parameters for surface reconstruction.

Image-Based Dynamic Quantification of Aboveground Structure of Sugar Beet in Field

Sugar beet is one of the main crops for sugar production in the world. With the increasing demand for sugar, more desirable sugar beet genotypes need to be cultivated through plant breeding programs. Precise plant phenotyping in the field still remains challenge. In this study, structure from motion (SFM) approach was used to reconstruct a three-dimensional (3D) model for sugar beets from 20 genotypes at three growth stages in the field. An automatic data processing pipeline was developed to process point clouds of sugar beet including preprocessing, coordinates correction, filtering and segmentation of point cloud of individual plant. Phenotypic traits were also automatically extracted regarding plant height, maximum canopy area, convex hull volume, total leaf area and individual leaf length. Total leaf area and convex hull volume were adopted to explore the relationship with biomass. The results showed that high correlations between measured and estimated values with R2 > 0.8. Statistical analyses between biomass and extracted traits proved that both convex hull volume and total leaf area can predict biomass well. The proposed pipeline can estimate sugar beet traits precisely in the field and provide a basis for sugar beet breeding.

Machine Learning Approaches to Improve Three Basic Plant Phenotyping Tasks Using Three-Dimensional Point Clouds

Developing automated methods to efficiently process large volumes of point cloud data remains a challenge for three-dimensional (3D) plant phenotyping applications. Here, we describe the development of machine learning methods to tackle three primary challenges in plant phenotyping: lamina/stem classification, lamina counting, and stem skeletonization. For classification, we assessed and validated the accuracy of our methods on a dataset of 54 3D shoot architectures, representing multiple growth conditions and developmental time points for two Solanaceous species, tomato (Solanum lycopersicum cv 75 m82D) and Nicotiana benthamiana Using deep learning, we classified lamina versus stems with 97.8% accuracy. Critically, we also demonstrated the robustness of our method to growth conditions and species that have not been trained on, which is important in practical applications but is often untested. For lamina counting, we developed an enhanced region-growing algorithm to reduce oversegmentation; this method achieved 86.6% accuracy, outperforming prior methods developed for this problem. Finally, for stem skeletonization, we developed an enhanced tip detection technique, which ran an order of magnitude faster and generated more precise skeleton architectures than prior methods. Overall, our improvements enable higher throughput and accurate extraction of phenotypic properties from 3D point cloud data.

Relationship between irradiance and levels of Calvin-Benson cycle and other intermediates in the model eudicot Arabidopsis and the model monocot rice

Metabolite profiles provide a top-down overview of the balance between the reactions in a pathway. We compared Calvin-Benson cycle (CBC) intermediate profiles in different conditions in Arabidopsis (Arabidopsis thaliana) and rice (Oryza sativa) to learn which features of CBC regulation differ and which are shared between these model eudicot and monocot C3 species. Principal component analysis revealed that CBC intermediate profiles follow different trajectories in Arabidopsis and rice as irradiance increases. The balance between subprocesses or reactions differed, with 3-phosphoglycerate reduction being favoured in Arabidopsis and ribulose 1,5-bisphosphate regeneration in rice, and sedoheptulose-1,7-bisphosphatase being favoured in Arabidopsis compared with fructose-1,6-bisphosphatase in rice. Photosynthesis rates rose in parallel with ribulose 1,5-bisphosphate levels in Arabidopsis, but not in rice. Nevertheless, some responses were shared between Arabidopsis and rice. Fructose 1,6-bisphosphate and sedoheptulose-1,7-bisphosphate were high or peaked at very low irradiance in both species. Incomplete activation of fructose-1,6-bisphosphatase and sedoheptulose-1,7-bisphosphatase may prevent wasteful futile cycles in low irradiance. End-product synthesis is inhibited and high levels of CBC intermediates are maintained in low light or in low CO2 in both species. This may improve photosynthetic efficiency in fluctuating irradiance, and facilitate rapid CBC flux to support photorespiration and energy dissipation in low CO2.

Network trade-offs and homeostasis in Arabidopsis shoot architectures

Understanding the optimization objectives that shape shoot architectures remains a critical problem in plant biology. Here, we performed 3D scanning of 152 Arabidopsis shoot architectures, including wildtype and 10 mutant strains, and we uncovered a design principle that describes how architectures make trade-offs between competing objectives. First, we used graph-theoretic analysis to show that Arabidopsis shoot architectures strike a Pareto optimal that can be captured as maximizing performance in transporting nutrients and minimizing costs in building the architecture. Second, we identify small sets of genes that can be mutated to shift the weight prioritizing one objective over the other. Third, we show that this prioritization weight feature is significantly less variable across replicates of the same genotype compared to other common plant traits (e.g., number of rosette leaves, total volume occupied). This suggests that this feature is a robust descriptor of a genotype, and that local variability in structure may be compensated for globally in a homeostatic manner. Overall, our work provides a framework to understand optimization trade-offs made by shoot architectures and provides evidence that these trade-offs can be modified genetically, which may aid plant breeding and selection efforts.

In both engineered and biological systems, there is often no single structure that performs optimally on all tasks. For example, a transport system that can very quickly shuttle people to and from work will often not be very cheap to build, and vice-versa. Thus, trade-offs are born, and it is natural to ask how well evolution has resolved trade-offs between competing tasks. Here, we use 3D laser scanning and network analysis to show that Arabidopsis plant architectures make Pareto optimal trade-offs, which means that improving upon one task requires a sacrifice in the other task. In other words, an architecture that performs better on both tasks cannot be built. We also identify a small set of genes that can change how the architecture prioritizes one task versus the other, which may allow for better crop design in the future. Finally, we show that two replicate architectures that look visually diverse (e.g., variation in size, number of leaves, number of branches, etc.) often prioritize each task similarly. This suggests that despite local variability in the architecture, there may be a homeostatic drive to maintain globally balanced trade-offs.

Crop Phenomics: Current Status and Perspectives

Reliable, automatic, multifunctional, and high-throughput phenotypic technologies are increasingly considered important tools for rapid advancement of genetic gain in breeding programs. With the rapid development in high-throughput phenotyping technologies, research in this area is entering a new era called 'phenomics.' The crop phenotyping community not only needs to build a multi-domain, multi-level, and multi-scale crop phenotyping big database, but also to research technical systems for phenotypic traits identification and develop bioinformatics technologies for information extraction from the overwhelming amounts of omics data. Here, we provide an overview of crop phenomics research, focusing on two parts, from phenotypic data collection through various sensors to phenomics analysis. Finally, we discussed the challenges and prospective of crop phenomics in order to provide suggestions to develop new methods of mining genes associated with important agronomic traits, and propose new intelligent solutions for precision breeding.

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth

BACKGROUND

Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS).

RESULTS

We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set.

CONCLUSIONS

PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small- and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

A canopy conundrum: can wind-induced movement help to increase crop productivity by relieving photosynthetic limitations?

Wind-induced movement is a ubiquitous occurrence for all plants grown in natural or agricultural settings, and in the context of high, damaging wind speeds it has been well studied. However, the impact of lower wind speeds (which do not cause any damage) on mode of movement, light transmission, and photosynthetic properties has, surprisingly, not been fully explored. This impact is likely to be influenced by biomechanical properties and architectural features of the plant and canopy. A limited number of eco-physiological studies have indicated that movement in wind has the potential to alter light distribution within canopies, improving canopy productivity by relieving photosynthetic limitations. Given the current interest in canopy photosynthesis, it is timely to consider such movement in terms of crop yield progress. This opinion article sets out the background to wind-induced crop movement and argues that plant biomechanical properties may have a role in the optimization of whole-canopy photosynthesis via established physiological processes. We discuss how this could be achieved using canopy models.

An Accurate Skeleton Extraction Approach From 3D Point Clouds of Maize Plants

Accurate and high-throughput determination of plant morphological traits is essential for phenotyping studies. Nowadays, there are many approaches to acquire high-quality three-dimensional (3D) point clouds of plants. However, it is difficult to estimate phenotyping parameters accurately of the whole growth stages of maize plants using these 3D point clouds. In this paper, an accurate skeleton extraction approach was proposed to bridge the gap between 3D point cloud and phenotyping traits estimation of maize plants. The algorithm first uses point cloud clustering and color difference denoising to reduce the noise of the input point clouds. Next, the Laplacian contraction algorithm is applied to shrink the points. Then the key points representing the skeleton of the plant are selected through adaptive sampling, and neighboring points are connected to form a plant skeleton composed of semantic organs. Finally, deviation skeleton points to the input point cloud are calibrated by building a step forward local coordinate along the tangent direction of the original points. The proposed approach successfully generates accurately extracted skeleton from 3D point cloud and helps to estimate phenotyping parameters with high precision of maize plants. Experimental verification of the skeleton extraction process, tested using three cultivars and different growth stages maize, demonstrates that the extracted matches the input point cloud well. Compared with 3D digitizing data-derived morphological parameters, the NRMSE of leaf length, leaf inclination angle, leaf top length, leaf azimuthal angle, leaf growth height, and plant height, estimated using the extracted plant skeleton, are 5.27, 8.37, 5.12, 4.42, 1.53, and 0.83%, respectively, which could meet the needs of phenotyping analysis. The time required to process a single maize plant is below 100 s. The proposed approach may play an important role in further maize research and applications, such as genotype-to-phenotype study, geometric reconstruction, functional structural maize modeling, and dynamic growth animation.

Salinity stress response and 'omics' approaches for improving salinity stress tolerance in major grain legumes

Sustaining yield gains of grain legume crops under growing salt-stressed conditions demands a thorough understanding of plant salinity response and more efficient breeding techniques that effectively integrate modern omics knowledge. Grain legume crops are important to global food security being an affordable source of dietary protein and essential mineral nutrients to human population, especially in the developing countries. The global productivity of grain legume crops is severely challenged by the salinity stress particularly in the face of changing climates coupled with injudicious use of irrigation water and improper agricultural land management. Plants adapt to sustain under salinity-challenged conditions through evoking complex molecular mechanisms. Elucidating the underlying complex mechanisms remains pivotal to our knowledge about plant salinity response. Improving salinity tolerance of plants demand enriching cultivated gene pool of grain legume crops through capitalizing on 'adaptive traits' that contribute to salinity stress tolerance. Here, we review the current progress in understanding the genetic makeup of salinity tolerance and highlight the role of germplasm resources and omics advances in improving salt tolerance of grain legumes. In parallel, scope of next generation phenotyping platforms that efficiently bridge the phenotyping-genotyping gap and latest research advances including epigenetics is also discussed in context to salt stress tolerance. Breeding salt-tolerant cultivars of grain legumes will require an integrated "omics-assisted" approach enabling accelerated improvement of salt-tolerance traits in crop breeding programs.