A novel mesh processing based technique for 3D plant analysis

An integrated method for phenotypic analysis of wheat based on multi-view image sequences: from seedling to grain filling stages

Wheat exhibits complex characteristics during its growth, such as extensive tillering, slender and soft leaves, and severe organ cross-obscuration, posing a considerable challenge in full-cycle phenotypic monitoring. To address this, this study presents a synthesized method based on SFM-MVS (Structure-from-Motion, Multi-View Stereo) processing for handling and segmenting wheat point clouds, covering the entire growth cycle from seedling to grain filling stages. First, a multi-view image acquisition platform was constructed to capture image sequences of wheat plants, and dense point clouds were generated using SFM-MVS technology. High-quality dense point clouds were produced by implementing improved Euclidean clustering combined with centroids, color filtering, and statistical filtering methods. Subsequently, the segmentation of wheat plant stems and leaves was performed using the region growth segmentation algorithm. Although segmentation performance was suboptimal during the tillering, jointing, and booting stages due to the glut leaves and severe overlap, there was a salient improvement in wheat leaf segmentation efficiency over the entire growth cycle. Finally, phenotypic parameters were analyzed across different growth stages, comparing automated measurements of plant height, leaf length, and leaf width with actual measurements. The results demonstrated coefficients of determination (R 2 ) of 0.9979, 0.9977, and 0.995; root mean square errors (RMSE) of 1.0773 cm, 0.2612 cm, and 0.0335 cm; and relative root mean square errors (RRMSE) of 2.1858%, 1.7483%, and 2.8462%, respectively. These results validate the reliability and accuracy of our proposed workflow in processing wheat point clouds and automatically extracting plant height, leaf length, and leaf width, indicating that our 3D reconstructed wheat model achieves high precision and can quickly, accurately, and non-destructively extract phenotypic parameters. Additionally, plant height, convex hull volume, plant surface area, and Crown area were extracted, providing a detailed analysis of dynamic changes in wheat throughout its growth cycle. ANOVA was conducted across different cultivars, accurately revealing significant differences at various growth stages. This study proposes a convenient, rapid, and quantitative analysis method, offering crucial technical support for wheat plant phenotypic analysis and growth dynamics monitoring, applicable for precise full-cycle phenotypic monitoring of wheat.

Cotton morphological traits tracking through spatiotemporal registration of terrestrial laser scanning time-series data

Understanding the complex interactions between genotype-environment dynamics is fundamental for optimizing crop improvement. However, traditional phenotyping methods limit assessments to the end of the growing season, restricting continuous crop monitoring. To address this limitation, we developed a methodology for spatiotemporal registration of time-series 3D point cloud data, enabling field phenotyping over time for accurate crop growth tracking. Leveraging multi-scan terrestrial laser scanning (TLS), we captured high-resolution 3D LiDAR data in a cotton breeding field across various stages of the growing season to generate four-dimensional (4D) crop models, seamlessly integrating spatial and temporal dimensions. Our registration procedure involved an initial pairwise terrain-based matching for rough alignment, followed by a bird's-eye view adjustment for fine registration. Point clouds collected throughout nine sessions across the growing season were successfully registered both spatially and temporally, with average registration errors of approximately 3 cm. We used the generated 4D models to monitor canopy height (CH) and volume (CV) for eleven cotton genotypes over two months. The consistent height reference established via our spatiotemporal registration process enabled precise estimations of CH (R 2 = 0.95, RMSE = 7.6 cm). Additionally, we analyzed the relationship between CV and the interception of photosynthetically active radiation (IPAR f ), finding that it followed a curve with exponential saturation, consistent with theoretical models, with a standard error of regression (SER) of 11%. In addition, we compared mathematical models from the Richards family of sigmoid curves for crop growth modeling, finding that the logistic model effectively captured CH and CV evolution, aiding in identifying significant genotype differences. Our novel TLS-based digital phenotyping methodology enhances precision and efficiency in field phenotyping over time, advancing plant phenomics and empowering efficient decision-making for crop improvement efforts.

A Point-Cloud Segmentation Network Based on SqueezeNet and Time Series for Plants

The phenotyping of plant growth enriches our understanding of intricate genetic characteristics, paving the way for advancements in modern breeding and precision agriculture. Within the domain of phenotyping, segmenting 3D point clouds of plant organs is the basis of extracting plant phenotypic parameters. In this study, we introduce a novel method for point-cloud downsampling that adeptly mitigates the challenges posed by sample imbalances. In subsequent developments, we architect a deep learning framework founded on the principles of SqueezeNet for the segmentation of plant point clouds. In addition, we also use the time series as input variables, which effectively improves the segmentation accuracy of the network. Based on semantic segmentation, the MeanShift algorithm is employed to execute instance segmentation on the point-cloud data of crops. In semantic segmentation, the average Precision, Recall, F1-score, and IoU of maize reached 99.35%, 99.26%, 99.30%, and 98.61%, and the average Precision, Recall, F1-score, and IoU of tomato reached 97.98%, 97.92%, 97.95%, and 95.98%. In instance segmentation, the accuracy of maize and tomato reached 98.45% and 96.12%. This research holds the potential to advance the fields of plant phenotypic extraction, ideotype selection, and precision agriculture.

Pitfalls and potential of high-throughput plant phenotyping platforms

Automated high-throughput plant phenotyping (HTPP) enables non-invasive, fast and standardized evaluations of a large number of plants for size, development, and certain physiological variables. Many research groups recognize the potential of HTPP and have made significant investments in HTPP infrastructure, or are considering doing so. To make optimal use of limited resources, it is important to plan and use these facilities prudently and to interpret the results carefully. Here we present a number of points that users should consider before purchasing, building or utilizing such equipment. They relate to (1) the financial and time investment for acquisition, operation, and maintenance, (2) the constraints associated with such machines in terms of flexibility and growth conditions, (3) the pros and cons of frequent non-destructive measurements, (4) the level of information provided by proxy traits, and (5) the utilization of calibration curves. Using data from an Arabidopsis experiment, we demonstrate how diurnal changes in leaf angle can impact plant size estimates from top-view cameras, causing deviations of more than 20% over the day. Growth analysis data from another rosette species showed that there was a curvilinear relationship between total and projected leaf area. Neglecting this curvilinearity resulted in linear calibration curves that, although having a high r2 (> 0.92), also exhibited large relative errors. Another important consideration we discussed is the frequency at which calibration curves need to be generated and whether different treatments, seasons, or genotypes require distinct calibration curves. In conclusion, HTPP systems have become a valuable addition to the toolbox of plant biologists, provided that these systems are tailored to the research questions of interest, and users are aware of both the possible pitfalls and potential involved.

How to make sense of 3D representations for plant phenotyping: a compendium of processing and analysis techniques

Computer vision technology is moving more and more towards a three-dimensional approach, and plant phenotyping is following this trend. However, despite its potential, the complexity of the analysis of 3D representations has been the main bottleneck hindering the wider deployment of 3D plant phenotyping. In this review we provide an overview of typical steps for the processing and analysis of 3D representations of plants, to offer potential users of 3D phenotyping a first gateway into its application, and to stimulate its further development. We focus on plant phenotyping applications where the goal is to measure characteristics of single plants or crop canopies on a small scale in research settings, as opposed to large scale crop monitoring in the field.

DFSP: A fast and automatic distance field-based stem-leaf segmentation pipeline for point cloud of maize shoot

The 3D point cloud data are used to analyze plant morphological structure. Organ segmentation of a single plant can be directly used to determine the accuracy and reliability of organ-level phenotypic estimation in a point-cloud study. However, it is difficult to achieve a high-precision, automatic, and fast plant point cloud segmentation. Besides, a few methods can easily integrate the global structural features and local morphological features of point clouds relatively at a reduced cost. In this paper, a distance field-based segmentation pipeline (DFSP) which could code the global spatial structure and local connection of a plant was developed to realize rapid organ location and segmentation. The terminal point clouds of different plant organs were first extracted via DFSP during the stem-leaf segmentation, followed by the identification of the low-end point cloud of maize stem based on the local geometric features. The regional growth was then combined to obtain a stem point cloud. Finally, the instance segmentation of the leaf point cloud was realized using DFSP. The segmentation method was tested on 420 maize and compared with the manually obtained ground truth. Notably, DFSP had an average processing time of 1.52 s for about 15,000 points of maize plant data. The mean precision, recall, and micro F1 score of the DFSP segmentation algorithm were 0.905, 0.899, and 0.902, respectively. These findings suggest that DFSP can accurately, rapidly, and automatically achieve maize stem-leaf segmentation tasks and could be effective in maize phenotype research. The source code can be found at https://github.com/syau-miao/DFSP.git.

Four-dimensional measurement of root system development using time-series three-dimensional volumetric data analysis by backward prediction

BACKGROUND

Root system architecture (RSA) is an essential characteristic for efficient water and nutrient absorption in terrestrial plants; its plasticity enables plants to respond to different soil environments. Better understanding of root plasticity is important in developing stress-tolerant crops. Non-invasive techniques that can measure roots in soils nondestructively, such as X-ray computed tomography (CT), are useful to evaluate RSA plasticity. However, although RSA plasticity can be measured by tracking individual root growth, only a few methods are available for tracking individual roots from time-series three-dimensional (3D) images.

RESULTS

We developed a semi-automatic workflow that tracks individual root growth by vectorizing RSA from time-series 3D images via two major steps. The first step involves 3D alignment of the time-series RSA images by iterative closest point registration with point clouds generated by high-intensity particles in potted soils. This alignment ensures that the time-series RSA images overlap. The second step consists of backward prediction of vectorization, which is based on the phenomenon that the root length of the RSA vector at the earlier time point is shorter than that at the last time point. In other words, when CT scanning is performed at time point A and again at time point B for the same pot, the CT data and RSA vectors at time points A and B will almost overlap, but not where the roots have grown. We assumed that given a manually created RSA vector at the last time point of the time series, all RSA vectors except those at the last time point could be automatically predicted by referring to the corresponding RSA images. Using 21 time-series CT volumes of a potted plant of upland rice (Oryza sativa), this workflow revealed that the root elongation speed increased with age. Compared with a workflow that does not use backward prediction, the workflow with backward prediction reduced the manual labor time by 95%.

CONCLUSIONS

We developed a workflow to efficiently generate time-series RSA vectors from time-series X-ray CT volumes. We named this workflow 'RSAtrace4D' and are confident that it can be applied to the time-series analysis of RSA development and plasticity.

PhenoTrack3D: an automatic high-throughput phenotyping pipeline to track maize organs over time

BACKGROUND

High-throughput phenotyping platforms allow the study of the form and function of a large number of genotypes subjected to different growing conditions (GxE). A number of image acquisition and processing pipelines have been developed to automate this process, for micro-plots in the field and for individual plants in controlled conditions. Capturing shoot development requires extracting from images both the evolution of the 3D plant architecture as a whole, and a temporal tracking of the growth of its organs.

RESULTS

We propose PhenoTrack3D, a new pipeline to extract a 3D + t reconstruction of maize. It allows the study of plant architecture and individual organ development over time during the entire growth cycle. The method tracks the development of each organ from a time-series of plants whose organs have already been segmented in 3D using existing methods, such as Phenomenal [Artzet et al. in BioRxiv 1:805739, 2019] which was chosen in this study. First, a novel stem detection method based on deep-learning is used to locate precisely the point of separation between ligulated and growing leaves. Second, a new and original multiple sequence alignment algorithm has been developed to perform the temporal tracking of ligulated leaves, which have a consistent geometry over time and an unambiguous topological position. Finally, growing leaves are back-tracked with a distance-based approach. This pipeline is validated on a challenging dataset of 60 maize hybrids imaged daily from emergence to maturity in the PhenoArch platform (ca. 250,000 images). Stem tip was precisely detected over time (RMSE < 2.1 cm). 97.7% and 85.3% of ligulated and growing leaves respectively were assigned to the correct rank after tracking, on 30 plants × 43 dates. The pipeline allowed to extract various development and architecture traits at organ level, with good correlation to manual observations overall, on random subsets of 10-355 plants.

CONCLUSIONS

We developed a novel phenotyping method based on sequence alignment and deep-learning. It allows to characterise the development of maize architecture at organ level, automatically and at a high-throughput. It has been validated on hundreds of plants during the entire development cycle, showing its applicability on GxE analyses of large maize datasets.

A graph-based approach for simultaneous semantic and instance segmentation of plant 3D point clouds

Accurate simultaneous semantic and instance segmentation of a plant 3D point cloud is critical for automatic plant phenotyping. Classically, each organ of the plant is detected based on the local geometry of the point cloud, but the consistency of the global structure of the plant is rarely assessed. We propose a two-level, graph-based approach for the automatic, fast and accurate segmentation of a plant into each of its organs with structural guarantees. We compute local geometric and spectral features on a neighbourhood graph of the points to distinguish between linear organs (main stem, branches, petioles) and two-dimensional ones (leaf blades) and even 3-dimensional ones (apices). Then a quotient graph connecting each detected macroscopic organ to its neighbors is used both to refine the labelling of the organs and to check the overall consistency of the segmentation. A refinement loop allows to correct segmentation defects. The method is assessed on both synthetic and real 3D point-cloud data sets of Chenopodium album (wild spinach) and Solanum lycopersicum (tomato plant).

ROS scavenging and ion homeostasis is required for the adaptation of halophyte Karelinia caspia to high salinity

The halophyte Karelinia caspia has not only fodder and medical value but also can remediate saline-alkali soils. Our previous study showed that salt-secreting by salt glands is one of main adaptive strategies of K. caspia under high salinity. However, ROS scavenging, ion homeostasis, and photosynthetic characteristics responses to high salinity remain unclear in K. caspia. Here, physio-biochemical responses and gene expression associated with ROS scavenging and ions transport were tested in K. caspia subjected to 100-400 mM NaCl for 7 days. Results showed that both antioxidant enzymes (SOD, APX) activities and non-enzymatic antioxidants (chlorogenic acid, α-tocopherol, flavonoids, polyamines) contents were significantly enhanced, accompanied by up-regulating the related enzyme and non-enzymatic antioxidant synthesis gene (KcCu/Zn-SOD, KcAPX6, KcHCT, KcHPT1, Kcγ-TMT, KcF3H, KcSAMS and KcSMS) expression with increasing concentrations of NaCl. These responses are beneficial for removing excess ROS to maintain a stable level of H2O2 and O2 - without lipid peroxidation in the K. caspia response to high salt. Meanwhile, up-regulating expression of KcSOS1/2/3, KcNHX1, and KcAVP was linked to Na+ compartmentalization into vacuoles or excretion through salt glands in K. caspia. Notably, salt can improve the function of PSII that facilitate net photosynthetic rates, which is helpful to growing normally in high saline. Overall, the findings suggested that ROS scavenging systems and Na+/K+ transport synergistically contributed to redox equilibrium, ion homeostasis, and the enhancement of PSII function, thereby conferring high salt tolerance.

The field phenotyping platform's next darling: Dicotyledons

The genetic information and functional properties of plants have been further identified with the completion of the whole-genome sequencing of numerous crop species and the rapid development of high-throughput phenotyping technologies, laying a suitable foundation for advanced precision agriculture and enhanced genetic gains. Collecting phenotypic data from dicotyledonous crops in the field has been identified as a key factor in the collection of large-scale phenotypic data of crops. On the one hand, dicotyledonous plants account for 4/5 of all angiosperm species and play a critical role in agriculture. However, their morphology is complex, and an abundance of dicot phenotypic information is available, which is critical for the analysis of high-throughput phenotypic data in the field. As a result, the focus of this paper is on the major advancements in ground-based, air-based, and space-based field phenotyping platforms over the last few decades and the research progress in the high-throughput phenotyping of dicotyledonous field crop plants in terms of morphological indicators, physiological and biochemical indicators, biotic/abiotic stress indicators, and yield indicators. Finally, the future development of dicots in the field is explored from the perspectives of identifying new unified phenotypic criteria, developing a high-performance infrastructure platform, creating a phenotypic big data knowledge map, and merging the data with those of multiomic techniques.

A Review of Integrative Omic Approaches for Understanding Rice Salt Response Mechanisms

Soil salinity is one of the most serious environmental challenges, posing a growing threat to agriculture across the world. Soil salinity has a significant impact on rice growth, development, and production. Hence, improving rice varieties’ resistance to salt stress is a viable solution for meeting global food demand. Adaptation to salt stress is a multifaceted process that involves interacting physiological traits, biochemical or metabolic pathways, and molecular mechanisms. The integration of multi-omics approaches contributes to a better understanding of molecular mechanisms as well as the improvement of salt-resistant and tolerant rice varieties. Firstly, we present a thorough review of current knowledge about salt stress effects on rice and mechanisms behind rice salt tolerance and salt stress signalling. This review focuses on the use of multi-omics approaches to improve next-generation rice breeding for salinity resistance and tolerance, including genomics, transcriptomics, proteomics, metabolomics and phenomics. Integrating multi-omics data effectively is critical to gaining a more comprehensive and in-depth understanding of the molecular pathways, enzyme activity and interacting networks of genes controlling salinity tolerance in rice. The key data mining strategies within the artificial intelligence to analyse big and complex data sets that will allow more accurate prediction of outcomes and modernise traditional breeding programmes and also expedite precision rice breeding such as genetic engineering and genome editing.

PSegNet: Simultaneous Semantic and Instance Segmentation for Point Clouds of Plants

Phenotyping of plant growth improves the understanding of complex genetic traits and eventually expedites the development of modern breeding and intelligent agriculture. In phenotyping, segmentation of 3D point clouds of plant organs such as leaves and stems contributes to automatic growth monitoring and reflects the extent of stress received by the plant. In this work, we first proposed the Voxelized Farthest Point Sampling (VFPS), a novel point cloud downsampling strategy, to prepare our plant dataset for training of deep neural networks. Then, a deep learning network-PSegNet, was specially designed for segmenting point clouds of several species of plants. The effectiveness of PSegNet originates from three new modules including the Double-Neighborhood Feature Extraction Block (DNFEB), the Double-Granularity Feature Fusion Module (DGFFM), and the Attention Module (AM). After training on the plant dataset prepared with VFPS, the network can simultaneously realize the semantic segmentation and the leaf instance segmentation for three plant species. Comparing to several mainstream networks such as PointNet++, ASIS, SGPN, and PlantNet, the PSegNet obtained the best segmentation results quantitatively and qualitatively. In semantic segmentation, PSegNet achieved 95.23%, 93.85%, 94.52%, and 89.90% for the mean Prec, Rec, F1, and IoU, respectively. In instance segmentation, PSegNet achieved 88.13%, 79.28%, 83.35%, and 89.54% for the mPrec, mRec, mCov, and mWCov, respectively.

Plant trait measurement in 3D for growth monitoring

BACKGROUND

There is a demand for non-destructive systems in plant phenotyping which could precisely measure plant traits for growth monitoring. In this study, the growth of chilli plants (Capsicum annum L.) was monitored in outdoor conditions. A non-destructive solution is proposed for growth monitoring in 3D using a single mobile phone camera based on a structure from motion algorithm. A method to measure leaf length and leaf width when the leaf is curled is also proposed. Various plant traits such as number of leaves, stem height, leaf length, and leaf width were measured from the reconstructed and segmented 3D models at different plant growth stages.

RESULTS

The accuracy of the proposed system is measured by comparing the values derived from the 3D plant model with manual measurements. The results demonstrate that the proposed system has potential to non-destructively monitor plant growth in outdoor conditions with high precision, when compared to the state-of-the-art systems.

CONCLUSIONS

In conclusion, this study demonstrated that the methods proposed to calculate plant traits can monitor plant growth in outdoor conditions.

Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods

BACKGROUND

Segmentation of structural parts of 3D models of plants is an important step for plant phenotyping, especially for monitoring architectural and morphological traits. Current state-of-the art approaches rely on hand-crafted 3D local features for modeling geometric variations in plant structures. While recent advancements in deep learning on point clouds have the potential of extracting relevant local and global characteristics, the scarcity of labeled 3D plant data impedes the exploration of this potential.

RESULTS

We adapted six recent point-based deep learning architectures (PointNet, PointNet++, DGCNN, PointCNN, ShellNet, RIConv) for segmentation of structural parts of rosebush models. We generated 3D synthetic rosebush models to provide adequate amount of labeled data for modification and pre-training of these architectures. To evaluate their performance on real rosebush plants, we used the ROSE-X data set of fully annotated point cloud models. We provided experiments with and without the incorporation of synthetic data to demonstrate the potential of point-based deep learning techniques even with limited labeled data of real plants.

CONCLUSION

The experimental results show that PointNet++ produces the highest segmentation accuracy among the six point-based deep learning methods. The advantage of PointNet++ is that it provides a flexibility in the scales of the hierarchical organization of the point cloud data. Pre-training with synthetic 3D models boosted the performance of all architectures, except for PointNet.

A Three-Dimensional Scanning System for Digital Archiving and Quantitative Evaluation of Arabidopsis Plant Architectures

A plant's architecture contributes to its ability to acquire resources and reduce mechanical load. Arabidopsis thaliana is the most common model plant in molecular biology, and there are several mutants and transgenic lines with modified plant architecture regulation, such as lazy1 mutants, which have reversed angles of lateral branches. Although some phenotyping methods have been used in larger agricultural plants, limited suitable methods are available for three-dimensional reconstruction of Arabidopsis, which is smaller and has more uniform surface textures and structures. An inexpensive, easily adopted three-dimensional reconstruction system that can be used for Arabidopsis is needed so that researchers can view and quantify morphological changes over time. We developed a three-dimensional reconstruction system for A. thaliana using the visual volume intersection method, which uses a fixed camera to capture plant images from multiple directions while the plant slowly rotates. We then developed a script to autogenerate stack images from the obtained input movie and visualized the plant architecture by rendering the output stack image using the general bioimage analysis software. We successfully three-dimensionally and time-sequentially scanned wild-type and lazy1 mutant A. thaliana plants and measured the angles of the lateral branches. This non-contact, non-destructive method requires no specialized equipment and is space efficient, inexpensive and easily adopted by Arabidopsis researchers. Consequently, this system will promote three- and four-dimensional phenotyping of this model plant, and it can be used in combination with molecular genetics to further elucidate the molecular mechanisms that regulate Arabidopsis architecture.

Flexible Wearables for Plants

The excellent stretchability and biocompatibility of flexible sensors have inspired an emerging field of plant wearables, which enable intimate contact with the plants to continuously monitor the growth status and localized microclimate in real-time. Plant flexible wearables provide a promising platform for the development of plant phenotype and the construction of intelligent agriculture via monitoring and regulating the critical physiological parameters and microclimate of plants. Here, the emerging applications of plant flexible wearables together with their pros and cons from four aspects, including physiological indicators, surrounding environment, crop quality, and active control of growth, are highlighted. Self-powered energy supply systems and signal transmission mechanisms are also elucidated. Furthermore, the future opportunities and challenges of plant wearables are discussed in detail.

A digital sensor to measure real-time leaf movements and detect abiotic stress in plants

Plant and plant organ movements are the result of a complex integration of endogenous growth and developmental responses, partially controlled by the circadian clock, and external environmental cues. Monitoring of plant motion is typically done by image-based phenotyping techniques with the aid of computer vision algorithms. Here we present a method to measure leaf movements using a digital inertial measurement unit (IMU) sensor. The lightweight sensor is easily attachable to a leaf or plant organ and records angular traits in real-time for two dimensions (pitch and roll) with high resolution (measured sensor oscillations of 0.36 ± 0.53° for pitch and 0.50 ± 0.65° for roll). We were able to record simple movements such as petiole bending, as well as complex lamina motions, in several crops, ranging from tomato to banana. We also assessed growth responses in terms of lettuce rosette expansion and maize seedling stem movements. The IMU sensors are capable of detecting small changes of nutations (i.e. bending movements) in leaves of different ages and in different plant species. In addition, the sensor system can also monitor stress-induced leaf movements. We observed that unfavorable environmental conditions evoke certain leaf movements, such as drastic epinastic responses, as well as subtle fading of the amplitude of nutations. In summary, the presented digital sensor system enables continuous detection of a variety of leaf motions with high precision, and is a low-cost tool in the field of plant phenotyping, with potential applications in early stress detection.

Visualization Methods of 3D Plant Models: A Systematic Mapping Study

Three-dimensional plant model visualization (3D-PMV) is based on plant biology and the plant structure construction model for virtual simulation three-dimensional display, reflecting plant structure characteristics and demonstrating the growth process. In this work, we used the method of systematic mapping study (SMS) to screen the relevant literature in order to explore and analyze the methods and goals of virtual plant visualization research and to provide assistance in future literature reviews. To this end, we conducted extensive searches to identify articles related to plant model visualization technology. We searched for papers from July 2010 to November 2020 from four mainstream databases, namely, ACM, IEEE Xplore, EI, and Web of Science, and found more than 2,900 papers on 3D-PMV. Finally, we selected 60 qualified papers. We mainly followed the SMS method to classify papers by answering seven questions, mainly extracting data for analysis on research types, research goals, model construction methods, visual model categories, number of publications, tools, and methods. The results show that solution proposals account for the largest proportion of research types, accounting for 75% of the total number of papers. This shows that the focus is on the improvement of original methods or the proposal of new methods in this field; research goals of research plant phenotypic characteristics account for the total 35%; 28% of the research objectives focus on showing the reflection of plant growth process, indicating that the research objective trends in this research field regard plant phenotypic characteristics and visualization of the growth process; static type model visualization also accounts for 83% of the total, indicating that the related research on visualization in this field is mostly static typing. Therefore, our work not only analyzes the challenges faced by 3D-PVM and the research directions that need more attention in the future but also provides some suggestions for researchers to find suitable research directions in this field. As the future direction of development, researchers need to pay more attention to the study of dynamic visualization methods of plant growth. We believe that for future research, it is indispensable to provide 3D-PMV with research methods, research goals, visualization model types, development, and other tools. This article analyzes the results of the extracted data based on the SMS method and makes an important contribution to the researcher's extensive understanding of the current status of 3D-PMV and the potential future research opportunities in the field.

Making Use of 3D Models for Plant Physiognomic Analysis: A Review

Use of 3D sensors in plant phenotyping has increased in the last few years. Various image acquisition, 3D representations, 3D model processing and analysis techniques exist to help the researchers. However, a review of approaches, algorithms, and techniques used for 3D plant physiognomic analysis is lacking. In this paper, we investigate the techniques and algorithms used at various stages of processing and analysing 3D models of plants, and identify their current limiting factors. This review will serve potential users as well as new researchers in this field. The focus is on exploring studies monitoring the plant growth of single plants or small scale canopies as opposed to large scale monitoring in the field.

Registration of spatio-temporal point clouds of plants for phenotyping

Plant phenotyping is a central task in crop science and plant breeding. It involves measuring plant traits to describe the anatomy and physiology of plants and is used for deriving traits and evaluating plant performance. Traditional methods for phenotyping are often time-consuming operations involving substantial manual labor. The availability of 3D sensor data of plants obtained from laser scanners or modern depth cameras offers the potential to automate several of these phenotyping tasks. This automation can scale up the phenotyping measurements and evaluations that have to be performed to a larger number of plant samples and at a finer spatial and temporal resolution. In this paper, we investigate the problem of registering 3D point clouds of the plants over time and space. This means that we determine correspondences between point clouds of plants taken at different points in time and register them using a new, non-rigid registration approach. This approach has the potential to form the backbone for phenotyping applications aimed at tracking the traits of plants over time. The registration task involves finding data associations between measurements taken at different times while the plants grow and change their appearance, allowing 3D models taken at different points in time to be compared with each other. Registering plants over time is challenging due to its anisotropic growth, changing topology, and non-rigid motion in between the time of the measurements. Thus, we propose a novel approach that first extracts a compact representation of the plant in the form of a skeleton that encodes both topology and semantic information, and then use this skeletal structure to determine correspondences over time and drive the registration process. Through this approach, we can tackle the data association problem for the time-series point cloud data of plants effectively. We tested our approach on different datasets acquired over time and successfully registered the 3D plant point clouds recorded with a laser scanner. We demonstrate that our method allows for developing systems for automated temporal plant-trait analysis by tracking plant traits at an organ level.

Parametric Surface Modelling for Tea Leaf Point Cloud Based on Non-Uniform Rational Basis Spline Technique

Plant leaf 3D architecture changes during growth and shows sensitive response to environmental stresses. In recent years, acquisition and segmentation methods of leaf point cloud developed rapidly, but 3D modelling leaf point clouds has not gained much attention. In this study, a parametric surface modelling method was proposed for accurately fitting tea leaf point cloud. Firstly, principal component analysis was utilized to adjust posture and position of the point cloud. Then, the point cloud was sliced into multiple sections, and some sections were selected to generate a point set to be fitted (PSF). Finally, the PSF was fitted into non-uniform rational B-spline (NURBS) surface. Two methods were developed to generate the ordered PSF and the unordered PSF, respectively. The PSF was firstly fitted as B-spline surface and then was transformed to NURBS form by minimizing fitting error, which was solved by particle swarm optimization (PSO). The fitting error was specified as weighted sum of the root-mean-square error (RMSE) and the maximum value (MV) of Euclidean distances between fitted surface and a subset of the point cloud. The results showed that the proposed modelling method could be used even if the point cloud is largely simplified (RMSE < 1 mm, MV < 2 mm, without performing PSO). Future studies will model wider range of leaves as well as incomplete point cloud.

A Fully Automated Three-Stage Procedure for Spatio-Temporal Leaf Segmentation with Regard to the B-Spline-Based Phenotyping of Cucumber Plants

Plant phenotyping deals with the metrological acquisition of plants in order to investigate the impact of environmental factors and a plant’s genotype on its appearance. Phenotyping methods that are used as standard in crop science are often invasive or even destructive. Due to the increase of automation within geodetic measurement systems and with the development of quasi-continuous measurement techniques, geodetic techniques are perfectly suitable for performing automated and non-invasive phenotyping and, hence, are an alternative to standard phenotyping methods. In this contribution, sequentially acquired point clouds of cucumber plants are used to determine the plants’ phenotypes in terms of their leaf areas. The focus of this contribution is on the spatio-temporal segmentation of the acquired point clouds, which automatically groups and tracks those sub point clouds that describe the same leaf. The application on example data sets reveals a successful segmentation of 93% of the leafs. Afterwards, the segmented leaves are approximated by means of B-spline surfaces, which provide the basis for the subsequent determination of the leaf areas. In order to validate the results, the determined leaf areas are compared to results obtained by means of standard methods used in crop science. The investigations reveal consistency of the results with maximal deviations in the determined leaf areas of up to 5%.

Linking Genes to Shape in Plants Using Morphometrics

A transition from qualitative to quantitative descriptors of morphology has been facilitated through the growing field of morphometrics, representing the conversion of shapes and patterns into numbers. The analysis of plant form at the macromorphological scale using morphometric approaches quantifies what is commonly referred to as a phenotype. Quantitative phenotypic analysis of individuals with contrasting genotypes in turn provides a means to establish links between genes and shapes. The path from a gene to a morphological phenotype is, however, not direct, with instructive information progressing both across multiple scales of biological complexity and through nonintuitive feedback, such as mechanical signals. In this review, we explore morphometric approaches used to perform whole-plant phenotyping and quantitative approaches in capture processes in the mesoscales, which bridge the gaps between genes and shapes in plants. Quantitative frameworks involving both the computational simulation and the discretization of data into networks provide a putative path to predicting emergent shape from underlying genetic programs.

A simulation framework for bio-inspired sonar sensing with Unmanned Aerial Vehicles

We introduce a unified simulation framework that generates natural sensing environments and produces biosonar echoes under various sensing scenarios. This framework produces rich sensory data with environmental information completely known, thus can be used for the training of robotic algorithms for biosonar-based Unmanned Aerial Vehicles. The simulated environment consists of random trees with full geometry of the tree foliage. To simulate a single tree, we adopt the Lindenmayer system to generate the initial branching pattern and integrate that with the available measurements of the 3D computer-aided design object files to create natural-looking branches, sub-branches, and leaves. A forest is formed by simulating trees at random locations generated by using an inhomogeneous Poisson process. While our simulated environments can be generally used for testing other sensors and training robotic algorithms, in this study we focus on testing bat-inspired Unmanned Aerial Vehicles that recreate bat's flying behavior through biosonar sensors. To this end, we also introduce an foliage echo simulator that produces biosonar echoes while mimicking bat's biosonar system. We demonstrate the application of the proposed simulation framework by generating real-world scenarios with multiple trees and computing the resulting impulse responses under static or dynamic motions of an Unmanned Aerial Vehicle.

A Low-Cost 3D Phenotype Measurement Method of Leafy Vegetables Using Video Recordings from Smartphones

Leafy vegetables are an essential source of the various nutrients that people need in their daily lives. The quantification of vegetable phenotypes and yield estimation are prerequisites for the selection of genetic varieties and for the improvement of planting methods. The traditional method is manual measurement, which is time-consuming and cumbersome. Therefore, there is a need for efficient and convenient in situ vegetable phenotype identification methods to provide data support for breeding research and for crop yield monitoring, thereby increasing vegetable yield. In this paper, a novel approach was developed for the in-situ determination of the three-dimensional (3D) phenotype of vegetables by recording video clips using smartphones. First, a smartphone was used to record the vegetable from different angles, and then the key frame containing the crop area in the video was obtained using an algorithm based on the vegetation index and scale-invariant feature transform algorithm (SIFT) matching. After obtaining the key frame, a dense point cloud of the vegetables was reconstructed using the Structure from Motion (SfM) method, and then the segmented point cloud and a point cloud skeleton were obtained using the clustering algorithm. Finally, the plant height, leaf number, leaf length, leaf angle, and other phenotypic parameters were obtained through the point cloud and point cloud skeleton. Comparing the obtained phenotypic parameters to the manual measurement results, the root-mean-square error (RMSE) of the plant height, leaf number, leaf length, and leaf angle were 1.82, 1.57, 2.43, and 4.7, respectively. The measurement accuracy of each indicators is greater than 80%. The results show that the proposed method provides a convenient, fast, and low-cost 3D phenotype measurement pipeline. Compared to other methods based on photogrammetry, this method does not need a labor-intensive image-capturing process and can reconstruct a high-quality point cloud model by directly recording videos of crops.

Quantification of light interception within image-based 3-D reconstruction of sole and intercropped canopies over the entire growth season

BACKGROUND AND AIMS

Light interception is closely related to canopy architecture. Few studies based on multi-view photography have been conducted in a field environment, particularly studies that link 3-D plant architecture with a radiation model to quantify the dynamic canopy light interception. In this study, we combined realistic 3-D plant architecture with a radiation model to quantify and evaluate the effect of differences in planting patterns and row orientations on canopy light interception.

METHODS

The 3-D architectures of maize and soybean plants were reconstructed for sole crops and intercrops based on multi-view images obtained at five growth dates in the field. We evaluated the accuracy of the calculated leaf length, maximum leaf width, plant height and leaf area according to the measured data. The light distribution within the 3-D plant canopy was calculated with a 3-D radiation model. Finally, we evaluated canopy light interception in different row orientations.

KEY RESULTS

There was good agreement between the measured and calculated phenotypic traits, with an R2 >0.97. The light distribution was more uniform for intercropped maize and more concentrated for sole maize. At the maize silking stage, 85 % of radiation was intercepted by approx. 55 % of the upper canopy region for maize and by approx. 33 % of the upper canopy region for soybean. There was no significant difference in daily light interception between the different row orientations for the entire intercropping and sole systems. However, for intercropped maize, near east-west orientations showed approx. 19 % higher daily light interception than near south-north orientations. For intercropped soybean, daily light interception showed the opposite trend. It was approx. 49 % higher for near south-north orientations than for near east-west orientations.

CONCLUSIONS

The accurate reconstruction of 3-D plants grown in the field based on multi-view images provides the possibility for high-throughput 3-D phenotyping in the field and allows a better understanding of the relationship between canopy architecture and the light environment.

Defining strawberry shape uniformity using 3D imaging and genetic mapping

Strawberry shape uniformity is a complex trait, influenced by multiple genetic and environmental components. To complicate matters further, the phenotypic assessment of strawberry uniformity is confounded by the difficulty of quantifying geometric parameters 'by eye' and variation between assessors. An in-depth genetic analysis of strawberry uniformity has not been undertaken to date, due to the lack of accurate and objective data. Nonetheless, uniformity remains one of the most important fruit quality selection criteria for the development of a new variety. In this study, a 3D-imaging approach was developed to characterise berry shape uniformity. We show that circularity of the maximum circumference had the closest predictive relationship with the manual uniformity score. Combining five or six automated metrics provided the best predictive model, indicating that human assessment of uniformity is highly complex. Furthermore, visual assessment of strawberry fruit quality in a multi-parental QTL mapping population has allowed the identification of genetic components controlling uniformity. A "regular shape" QTL was identified and found to be associated with three uniformity metrics. The QTL was present across a wide array of germplasm, indicating a potential candidate for marker-assisted breeding, while the potential to implement genomic selection is explored. A greater understanding of berry uniformity has been achieved through the study of the relative impact of automated metrics on human perceived uniformity. Furthermore, the comprehensive definition of strawberry shape uniformity using 3D imaging tools has allowed precision phenotyping, which has improved the accuracy of trait quantification and unlocked the ability to accurately select for uniform berries.

Performances Evaluation of a Low-Cost Platform for High-Resolution Plant Phenotyping

This study aims to test the performances of a low-cost and automatic phenotyping platform, consisting of a Red-Green-Blue (RGB) commercial camera scanning objects on rotating plates and the reconstruction of main plant phenotypic traits via the structure for motion approach (SfM). The precision of this platform was tested in relation to three-dimensional (3D) models generated from images of potted maize, tomato and olive tree, acquired at a different frequency (steps of 4°, 8° and 12°) and quality (4.88, 6.52 and 9.77 µm/pixel). Plant and organs heights, angles and areas were extracted from the 3D models generated for each combination of these factors. Coefficient of determination (R²), relative Root Mean Square Error (rRMSE) and Akaike Information Criterion (AIC) were used as goodness-of-fit indexes to compare the simulated to the observed data. The results indicated that while the best performances in reproducing plant traits were obtained using 90 images at 4.88 µm/pixel (R² = 0.81, rRMSE = 9.49% and AIC = 35.78), this corresponded to an unviable processing time (from 2.46 h to 28.25 h for herbaceous plants and olive trees, respectively). Conversely, 30 images at 4.88 µm/pixel resulted in a good compromise between a reliable reconstruction of considered traits (R² = 0.72, rRMSE = 11.92% and AIC = 42.59) and processing time (from 0.50 h to 2.05 h for herbaceous plants and olive trees, respectively). In any case, the results pointed out that this input combination may vary based on the trait under analysis, which can be more or less demanding in terms of input images and time according to the complexity of its shape (R² = 0.83, rRSME = 10.15% and AIC = 38.78). These findings highlight the reliability of the developed low-cost platform for plant phenotyping, further indicating the best combination of factors to speed up the acquisition and elaboration process, at the same time minimizing the bias between observed and simulated data.

An Efficient Processing Approach for Colored Point Cloud-Based High-Throughput Seedling Phenotyping

Plant height and leaf area are important morphological properties of leafy vegetable seedlings, and they can be particularly useful for plant growth and health research. The traditional measurement scheme is time-consuming and not suitable for continuously monitoring plant growth and health. Individual vegetable seedling quick segmentation is the prerequisite for high-throughput seedling phenotype data extraction at individual seedling level. This paper proposes an efficient learning- and model-free 3D point cloud data processing pipeline to measure the plant height and leaf area of every single seedling in a plug tray. The 3D point clouds are obtained by a low-cost red–green–blue (RGB)-Depth (RGB-D) camera. Firstly, noise reduction is performed on the original point clouds through the processing of useable-area filter, depth cut-off filter, and neighbor count filter. Secondly, the surface feature histograms-based approach is used to automatically remove the complicated natural background. Then, the Voxel Cloud Connectivity Segmentation (VCCS) and Locally Convex Connected Patches (LCCP) algorithms are employed for individual vegetable seedling partition. Finally, the height and projected leaf area of respective seedlings are calculated based on segmented point clouds and validation is carried out. Critically, we also demonstrate the robustness of our method for different growth conditions and species. The experimental results show that the proposed method could be used to quickly calculate the morphological parameters of each seedling and it is practical to use this approach for high-throughput seedling phenotyping.

Non-Destructive Measurement of Three-Dimensional Plants Based on Point Cloud

In agriculture, information about the spatial distribution of plant growth is valuable for applications. Quantitative study of the characteristics of plants plays an important role in the plants’ growth and development research, and non-destructive measurement of the height of plants based on machine vision technology is one of the difficulties. We propose a methodology for three-dimensional reconstruction under growing plants by Kinect v2.0 and explored the measure growth parameters based on three-dimensional (3D) point cloud in this paper. The strategy includes three steps—firstly, preprocessing 3D point cloud data, completing the 3D plant registration through point cloud outlier filtering and surface smooth method; secondly, using the locally convex connected patches method to segment the leaves and stem from the plant model; extracting the feature boundary points from the leaf point cloud, and using the contour extraction algorithm to get the feature boundary lines; finally, calculating the length, width of the leaf by Euclidean distance, and the area of the leaf by surface integral method, measuring the height of plant using the vertical distance technology. The results show that the automatic extraction scheme of plant information is effective and the measurement accuracy meets the need of measurement standard. The established 3D plant model is the key to study the whole plant information, which reduces the inaccuracy of occlusion to the description of leaf shape and conducive to the study of the real plant growth status.

Analysing the phenotype development of soybean plants using low-cost 3D reconstruction

With the development of digital agriculture, 3D reconstruction technology has been widely used to analyse crop phenotypes. To date, most research on 3D reconstruction of field crops has been limited to analysis of population characteristics. Therefore, in this study, we propose a method based on low-cost 3D reconstruction technology to analyse the phenotype development during the whole growth period. Based on the phenotypic parameters extracted from the 3D reconstruction model, we identified the "phenotypic fingerprint" of the relevant phenotypes throughout the whole growth period of soybean plants and completed analysis of the plant growth patterns using a logistic growth model. The phenotypic fingerprint showed that, before the R3 period, the growth of the five varieties was similar. After the R5 period, the differences among the five cultivars gradually increased. This result indicates that the phenotypic fingerprint can accurately reveal the patterns of phenotypic changes. The logistic growth model of soybean plants revealed the time points of maximum growth rate of the five soybean varieties, and this information can provide a basis for developing guidelines for water and fertiliser application to crops. These findings will provide effective guidance for breeding and field management of soybean and other crops.

ROSE-X: an annotated data set for evaluation of 3D plant organ segmentation methods

BACKGROUND

The production and availability of annotated data sets are indispensable for training and evaluation of automatic phenotyping methods. The need for complete 3D models of real plants with organ-level labeling is even more pronounced due to the advances in 3D vision-based phenotyping techniques and the difficulty of full annotation of the intricate 3D plant structure.

RESULTS

We introduce the ROSE-X data set of 11 annotated 3D models of real rosebush plants acquired through X-ray tomography and presented both in volumetric form and as point clouds. The annotation is performed manually to provide ground truth data in the form of organ labels for the voxels corresponding to the plant shoot. This data set is constructed to serve both as training data for supervised learning methods performing organ-level segmentation and as a benchmark to evaluate their performance. The rosebush models in the data set are of high quality and complex architecture with organs frequently touching each other posing a challenge for the current plant organ segmentation methods. We report leaf/stem segmentation results obtained using four baseline methods. The best performance is achieved by the volumetric approach where local features are trained with a random forest classifier, giving Intersection of Union (IoU) values of 97.93% and 86.23% for leaf and stem classes, respectively.

CONCLUSION

We provided an annotated 3D data set of 11 rosebush plants for training and evaluation of organ segmentation methods. We also reported leaf/stem segmentation results of baseline methods, which are open to improvement. The data set, together with the baseline results, has the potential of becoming a significant resource for future studies on automatic plant phenotyping.

A Computation Method Based on the Combination of Chlorophyll Fluorescence Parameters to Improve the Discrimination of Visually Similar Phenotypes Induced by Bacterial Virulence Factors

Phenotyping biotic stresses in plant-pathogen interactions studies is often hindered by phenotypes that can hardly be discriminated by visual assessment. Particularly, single gene mutants in virulence factors could lack visible phenotypes. Chlorophyll fluorescence (CF) imaging is a valuable tool to monitor plant-pathogen interactions. However, while numerous CF parameters can be measured, studies on plant-pathogen interactions often focus on a restricted number of parameters. It could result in limited abilities to discriminate visually similar phenotypes. In this study, we assess the ability of the combination of multiple CF parameters to improve the discrimination of such phenotypes. Such an approach could be of interest for screening and discriminating the impact of bacterial virulence factors without prior knowledge. A computation method was developed, based on the combination of multiple CF parameters, without any parameter selection. It involves histogram Bhattacharyya distance calculations and hierarchical clustering, with a normalization approach to take into account the inter-leaves and intra-phenotypes heterogeneities. To assess the efficiency of the method, two datasets were analyzed the same way. The first dataset featured single gene mutants of a Xanthomonas strain which differed only by their abilities to secrete bacterial virulence proteins. This dataset displayed expected phenotypes at 6 days post-inoculation and was used as ground truth dataset to setup the method. The efficiency of the computation method was demonstrated by the relevant discrimination of phenotypes at 3 days post-inoculation. A second dataset was composed of transient expression (agrotransformation) of Type 3 Effectors. This second dataset displayed phenotypes that cannot be discriminated by visual assessment and no prior knowledge can be made on the respective impact of each Type 3 Effectors on leaf tissues. Using the computation method resulted in clustering the leaf samples according to the Type 3 Effectors, thereby demonstrating an improvement of the discrimination of the visually similar phenotypes. The relevant discrimination of visually similar phenotypes induced by bacterial strains differing only by one virulence factor illustrated the importance of using a combination of CF parameters to monitor plant-pathogen interactions. It opens a perspective for the identification of specific signatures of biotic stresses.

Image-Based Dynamic Quantification of Aboveground Structure of Sugar Beet in Field

Sugar beet is one of the main crops for sugar production in the world. With the increasing demand for sugar, more desirable sugar beet genotypes need to be cultivated through plant breeding programs. Precise plant phenotyping in the field still remains challenge. In this study, structure from motion (SFM) approach was used to reconstruct a three-dimensional (3D) model for sugar beets from 20 genotypes at three growth stages in the field. An automatic data processing pipeline was developed to process point clouds of sugar beet including preprocessing, coordinates correction, filtering and segmentation of point cloud of individual plant. Phenotypic traits were also automatically extracted regarding plant height, maximum canopy area, convex hull volume, total leaf area and individual leaf length. Total leaf area and convex hull volume were adopted to explore the relationship with biomass. The results showed that high correlations between measured and estimated values with R2 > 0.8. Statistical analyses between biomass and extracted traits proved that both convex hull volume and total leaf area can predict biomass well. The proposed pipeline can estimate sugar beet traits precisely in the field and provide a basis for sugar beet breeding.

Automatic method for segmenting leaves by combining 2D and 3D image-processing techniques

In this study, a method to automatically segment plant leaves from three-dimensional (3D) images using structure from motion is proposed. First, leaves in the 3D images are roughly segmented using a region-growing method in which near points with distances less than 0.2 cm are assigned to the same group. By repeating this process, the leaves not touching each other can be segmented. Then, each segmented leaf is projected onto two-dimensional (2D) images, and the watershed algorithm is executed. This process successfully segments overlapping leaves.

Machine Learning Approaches to Improve Three Basic Plant Phenotyping Tasks Using Three-Dimensional Point Clouds

Developing automated methods to efficiently process large volumes of point cloud data remains a challenge for three-dimensional (3D) plant phenotyping applications. Here, we describe the development of machine learning methods to tackle three primary challenges in plant phenotyping: lamina/stem classification, lamina counting, and stem skeletonization. For classification, we assessed and validated the accuracy of our methods on a dataset of 54 3D shoot architectures, representing multiple growth conditions and developmental time points for two Solanaceous species, tomato (Solanum lycopersicum cv 75 m82D) and Nicotiana benthamiana Using deep learning, we classified lamina versus stems with 97.8% accuracy. Critically, we also demonstrated the robustness of our method to growth conditions and species that have not been trained on, which is important in practical applications but is often untested. For lamina counting, we developed an enhanced region-growing algorithm to reduce oversegmentation; this method achieved 86.6% accuracy, outperforming prior methods developed for this problem. Finally, for stem skeletonization, we developed an enhanced tip detection technique, which ran an order of magnitude faster and generated more precise skeleton architectures than prior methods. Overall, our improvements enable higher throughput and accurate extraction of phenotypic properties from 3D point cloud data.

Estimation of Plant and Canopy Architectural Traits Using the Digital Plant Phenotyping Platform

The extraction of desirable heritable traits for crop improvement from high-throughput phenotyping (HTP) observations remains challenging. We developed a modeling workflow named "Digital Plant Phenotyping Platform" (D3P), to access crop architectural traits from HTP observations. D3P couples the Architectural model of DEvelopment based on L-systems (ADEL) wheat (Triticum aestivum) model (ADEL-Wheat), which describes the time course of the three-dimensional architecture of wheat crops, with simulators of images acquired with HTP sensors. We demonstrated that a sequential assimilation of the green fraction derived from Red-Green-Blue images of the crop into D3P provides accurate estimates of five key parameters (phyllochron, lamina length of the first leaf, rate of elongation of leaf lamina, number of green leaves at the start of leaf senescence, and minimum number of green leaves) of the ADEL-Wheat model that drive the time course of green area index and the number of axes with more than three leaves at the end of the tillering period. However, leaf and tiller orientation and inclination characteristics were poorly estimated. D3P was also used to optimize the observational configuration. The results, obtained from in silico experiments conducted on wheat crops at several vegetative stages, showed that the accessible traits could be estimated accurately with observations made at 0° and 60° zenith view inclination with a temporal frequency of 100 °Cd (degree day). This illustrates the potential of the proposed holistic approach that integrates all the available information into a consistent system for interpretation. The potential benefits and limitations of the approach are further discussed.

Machine Vision System for 3D Plant Phenotyping

Machine vision for plant phenotyping is an emerging research area for producing high throughput in agriculture and crop science applications. Since 2D based approaches have their inherent limitations, 3D plant analysis is becoming state of the art for current phenotyping technologies. We present an automated system for analyzing plant growth in indoor conditions. A gantry robot system is used to perform scanning tasks in an automated manner throughout the lifetime of the plant. A 3D laser scanner mounted as the robot's payload captures the surface point cloud data of the plant from multiple views. The plant is monitored from the vegetative to reproductive stages in light/dark cycles inside a controllable growth chamber. An efficient 3D reconstruction algorithm is used, by which multiple scans are aligned together to obtain a 3D mesh of the plant, followed by surface area and volume computations. The whole system, including the programmable growth chamber, robot, scanner, data transfer, and analysis is fully automated in such a way that a naive user can, in theory, start the system with a mouse click and get back the growth analysis results at the end of the lifetime of the plant with no intermediate intervention. As evidence of its functionality, we show and analyze quantitative results of the rhythmic growth patterns of the dicot Arabidopsis thaliana (L.), and the monocot barley (Hordeum vulgare L.) plants under their diurnal light/dark cycles.

Measuring crops in 3D: using geometry for plant phenotyping

Using 3D sensing for plant phenotyping has risen within the last years. This review provides an overview on 3D traits for the demands of plant phenotyping considering different measuring techniques, derived traits and use-cases of biological applications. A comparison between a high resolution 3D measuring device and an established measuring tool, the leaf meter, is shown to categorize the possible measurement accuracy. Furthermore, different measuring techniques such as laser triangulation, structure from motion, time-of-flight, terrestrial laser scanning or structured light approaches enable the assessment of plant traits such as leaf width and length, plant size, volume and development on plant and organ level. The introduced traits were shown with respect to the measured plant types, the used measuring technique and the link to their biological use case. These were trait and growth analysis for measurements over time as well as more complex investigation on water budget, drought responses and QTL (quantitative trait loci) analysis. The used processing pipelines were generalized in a 3D point cloud processing workflow showing the single processing steps to derive plant parameters on plant level, on organ level using machine learning or over time using time series measurements. Finally the next step in plant sensing, the fusion of different sensor types namely 3D and spectral measurements is introduced by an example on sugar beet. This multi-dimensional plant model is the key to model the influence of geometry on radiometric measurements and to correct it. This publication depicts the state of the art for 3D measuring of plant traits as they were used in plant phenotyping regarding how the data is acquired, how this data is processed and what kind of traits is measured at the single plant, the miniplot, the experimental field and the open field scale. Future research will focus on highly resolved point clouds on the experimental and field scale as well as on the automated trait extraction of organ traits to track organ development at these scales.

Low-Cost Three-Dimensional Modeling of Crop Plants

Plant modeling can provide a more detailed overview regarding the basis of plant development throughout the life cycle. Three-dimensional processing algorithms are rapidly expanding in plant phenotyping programmes and in decision-making for agronomic management. Several methods have already been tested, but for practical implementations the trade-off between equipment cost, computational resources needed and the fidelity and accuracy in the reconstruction of the end-details needs to be assessed and quantified. This study examined the suitability of two low-cost systems for plant reconstruction. A low-cost Structure from Motion (SfM) technique was used to create 3D models for plant crop reconstruction. In the second method, an acquisition and reconstruction algorithm using an RGB-Depth Kinect v2 sensor was tested following a similar image acquisition procedure. The information was processed to create a dense point cloud, which allowed the creation of a 3D-polygon mesh representing every scanned plant. The selected crop plants corresponded to three different crops (maize, sugar beet and sunflower) that have structural and biological differences. The parameters measured from the model were validated with ground truth data of plant height, leaf area index and plant dry biomass using regression methods. The results showed strong consistency with good correlations between the calculated values in the models and the ground truth information. Although, the values obtained were always accurately estimated, differences between the methods and among the crops were found. The SfM method showed a slightly better result with regard to the reconstruction the end-details and the accuracy of the height estimation. Although the use of the processing algorithm is relatively fast, the use of RGB-D information is faster during the creation of the 3D models. Thus, both methods demonstrated robust results and provided great potential for use in both for indoor and outdoor scenarios. Consequently, these low-cost systems for 3D modeling are suitable for several situations where there is a need for model generation and also provide a favourable time-cost relationship.

High-throughput 3D modelling to dissect the genetic control of leaf elongation in barley (Hordeum vulgare)

To optimize shoot growth and structure of cereals, we need to understand the genetic components controlling initiation and elongation. While measuring total shoot growth at high throughput using 2D imaging has progressed, recovering the 3D shoot structure of small grain cereals at a large scale is still challenging. Here, we present a method for measuring defined individual leaves of cereals, such as wheat and barley, using few images. Plant shoot modelling over time was used to measure the initiation and elongation of leaves in a bi-parental barley mapping population under low and high soil salinity. We detected quantitative trait loci (QTL) related to shoot growth per se, using both simple 2D total shoot measurements and our approach of measuring individual leaves. In addition, we detected QTL specific to leaf elongation and not to total shoot size. Of particular importance was the detection of a QTL on chromosome 3H specific to the early responses of leaf elongation to salt stress, a locus that could not be detected without the computer vision tools developed in this study.

3-D Image-Driven Morphological Crop Analysis: A Novel Method for Detection of Sunflower Broomrape Initial Subsoil Parasitism

Effective control of the parasitic weed sunflower broomrape (Orobanche cumana Wallr.) can be achieved by herbicides application in early parasitism stages. However, the growing environmental concerns associated with herbicide treatments have motivated the adoption of precise chemical control approaches that detect and treat infested areas exclusively. The main challenge in developing such control practices for O. cumana lies in the fact that most of its life-cycle occurs in the soil sub-surface and by the time shoots emerge and become observable, the damage to the crop is irreversible. This paper approaches early O. cumana detection by hypothesizing that its parasitism already impacts the host plant morphology at the sub-soil surface developmental stage. To validate this hypothesis, O. cumana- infested sunflower and non-infested control plants were grown in pots and imaged weekly over 45-day period. Three-dimensional plant models were reconstructed using image-based multi-view stereo followed by derivation of their morphological parameters, down to the organ-level. Among the parameters estimated, height and first internode length were the earliest definitive indicators of infection. Furthermore, the detection timing of both parameters was early enough for herbicide post-emergence application. Considering the fact that 3-D morphological modeling is nondestructive, is based on commercially available RGB sensors and can be used under natural illumination; this approach holds potential contribution for site specific pre-emergence managements of parasitic weeds and as a phenotyping tool in O. cumana resistant sunflower breeding projects.

Evaluating maize phenotype dynamics under drought stress using terrestrial lidar

BACKGROUND

Maize (Zea mays L.) is the third most consumed grain in the world and improving maize yield is of great importance of the world food security, especially under global climate change and more frequent severe droughts. Due to the limitation of phenotyping methods, most current studies only focused on the responses of phenotypes on certain key growth stages. Although light detection and ranging (lidar) technology showed great potential in acquiring three-dimensional (3D) vegetation information, it has been rarely used in monitoring maize phenotype dynamics at an individual plant level.

RESULTS

In this study, we used a terrestrial laser scanner to collect lidar data at six growth stages for 20 maize varieties under drought stress. Three drought-related phenotypes, i.e., plant height, plant area index (PAI) and projected leaf area (PLA), were calculated from the lidar point clouds at the individual plant level. The results showed that terrestrial lidar data can be used to estimate plant height, PAI and PLA at an accuracy of 96%, 70% and 92%, respectively. All three phenotypes showed a pattern of first increasing and then decreasing during the growth period. The high drought tolerance group tended to keep lower plant height and PAI without losing PLA during the tasseling stage. Moreover, the high drought tolerance group inclined to have lower plant area density in the upper canopy than the low drought tolerance group.

CONCLUSION

The results demonstrate the feasibility of using terrestrial lidar to monitor 3D maize phenotypes under drought stress in the field and may provide new insights on identifying the key phenotypes and growth stages influenced by drought stress.

Dynamic Environmental Photosynthetic Imaging Reveals Emergent Phenotypes

Understanding and improving the productivity and robustness of plant photosynthesis requires high-throughput phenotyping under environmental conditions that are relevant to the field. Here we demonstrate the dynamic environmental photosynthesis imager (DEPI), an experimental platform for integrated, continuous, and high-throughput measurements of photosynthetic parameters during plant growth under reproducible yet dynamic environmental conditions. Using parallel imagers obviates the need to move plants or sensors, reducing artifacts and allowing simultaneous measurement on large numbers of plants. As a result, DEPI can reveal phenotypes that are not evident under standard laboratory conditions but emerge under progressively more dynamic illumination. We show examples in mutants of Arabidopsis of such "emergent phenotypes" that are highly transient and heterogeneous, appearing in different leaves under different conditions and depending in complex ways on both environmental conditions and plant developmental age. These emergent phenotypes appear to be caused by a range of phenomena, suggesting that such previously unseen processes are critical for plant responses to dynamic environments.