Calculation Method for Phenotypic Traits Based on the 3D Reconstruction of Maize Canopies

Review of applications of artificial intelligence (AI) methods in crop research

Sophisticated and modern crop improvement techniques can bridge the gap for feeding the ever-increasing population. Artificial intelligence (AI) refers to the simulation of human intelligence in machines, which refers to the application of computational algorithms, machine learning (ML) and deep learning (DL) techniques. This is aimed to generalise patterns and relationships from historical data, employing various mathematical optimisation techniques thus making prediction models for facilitating selection of superior genotypes. These techniques are less resource intensive and can solve the problem based on the analysis of large-scale phenotypic datasets. ML for genomic selection (GS) uses high-throughput genotyping technologies to gather genetic information on a large number of markers across the genome. The prediction of GS models is based on the mathematical relation between genotypic and phenotypic data from the training population. ML techniques have emerged as powerful tools for genome editing through analysing large-scale genomic data and facilitating the development of accurate prediction models. Precise phenotyping is a prerequisite to advance crop breeding for solving agricultural production-related issues. ML algorithms can solve this problem through generating predictive models, based on the analysis of large-scale phenotypic datasets. DL models also have the potential reliability of precise phenotyping. This review provides a comprehensive overview on various ML and DL models, their applications, potential to enhance the efficiency, specificity and safety towards advanced crop improvement protocols such as genomic selection, genome editing, along with phenotypic prediction to promote accelerated breeding.

Detection of maize stem diameter by using RGB-D cameras' depth information under selected field condition

Stem diameter is a critical phenotypic parameter for maize, integral to yield prediction and lodging resistance assessment. Traditionally, the quantification of this parameter through manual measurement has been the norm, notwithstanding its tedious and laborious nature. To address these challenges, this study introduces a non-invasive field-based system utilizing depth information from RGB-D cameras to measure maize stem diameter. This technology offers a practical solution for conducting rapid and non-destructive phenotyping. Firstly, RGB images, depth images, and 3D point clouds of maize stems were captured using an RGB-D camera, and precise alignment between the RGB and depth images was achieved. Subsequently, the contours of maize stems were delineated using 2D image processing techniques, followed by the extraction of the stem's skeletal structure employing a thinning-based skeletonization algorithm. Furthermore, within the areas of interest on the maize stems, horizontal lines were constructed using points on the skeletal structure, resulting in 2D pixel coordinates at the intersections of these horizontal lines with the maize stem contours. Subsequently, a back-projection transformation from 2D pixel coordinates to 3D world coordinates was achieved by combining the depth data with the camera's intrinsic parameters. The 3D world coordinates were then precisely mapped onto the 3D point cloud using rigid transformation techniques. Finally, the maize stem diameter was sensed and determined by calculating the Euclidean distance between pairs of 3D world coordinate points. The method demonstrated a Mean Absolute Percentage Error (MAPE) of 3.01%, a Mean Absolute Error (MAE) of 0.75 mm, a Root Mean Square Error (RMSE) of 1.07 mm, and a coefficient of determination (R²) of 0.96, ensuring accurate measurement of maize stem diameter. This research not only provides a new method of precise and efficient crop phenotypic analysis but also offers theoretical knowledge for the advancement of precision agriculture.

The improved stratified transformer for organ segmentation of Arabidopsis

Segmenting plant organs is a crucial step in extracting plant phenotypes. Despite the advancements in point-based neural networks, the field of plant point cloud segmentation suffers from a lack of adequate datasets. In this study, we addressed this issue by generating Arabidopsis models using L-system and proposing the surface-weighted sampling method. This approach enables automated point sampling and annotation, resulting in fully annotated point clouds. To create the Arabidopsis dataset, we employed Voxel Centroid Sampling and Random Sampling as point cloud downsampling methods, effectively reducing the number of points. To enhance the efficiency of semantic segmentation in plant point clouds, we introduced the Plant Stratified Transformer. This network is an improved version of the Stratified Transformer, incorporating the Fast Downsample Layer. Our improved network underwent training and testing on our dataset, and we compared its performance with PointNet++, PAConv, and the original Stratified Transformer network. For semantic segmentation, our improved network achieved mean Precision, Recall, F1-score and IoU of 84.20, 83.03, 83.61 and 73.11%, respectively. It outperformed PointNet++ and PAConv and performed similarly to the original network. Regarding efficiency, the training time and inference time were 714.3 and 597.9 ms, respectively, which were reduced by 320.9 and 271.8 ms, respectively, compared to the original network. The improved network significantly accelerated the speed of feeding point clouds into the network while maintaining segmentation performance. We demonstrated the potential of virtual plants and deep learning methods in rapidly extracting plant phenotypes, contributing to the advancement of plant phenotype research.

Using high-throughput phenotype platform MVS-Pheno to reconstruct the 3D morphological structure of wheat

It is of great significance to study the plant morphological structure for improving crop yield and achieving efficient use of resources. Three dimensional (3D) information can more accurately describe the morphological and structural characteristics of crop plants. Automatic acquisition of 3D information is one of the key steps in plant morphological structure research. Taking wheat as the research object, we propose a point cloud data-driven 3D reconstruction method that achieves 3D structure reconstruction and plant morphology parameterization at the phytomer scale. Specifically, we use the MVS-Pheno platform to reconstruct the point cloud of wheat plants and segment organs through the deep learning algorithm. On this basis, we automatically reconstructed the 3D structure of leaves and tillers and extracted the morphological parameters of wheat. The results show that the semantic segmentation accuracy of organs is 95.2%, and the instance segmentation accuracy AP50 is 0.665. The R2 values for extracted leaf length, leaf width, leaf attachment height, stem leaf angle, tiller length, and spike length were 0.97, 0.80, 1.00, 0.95, 0.99, and 0.95, respectively. This method can significantly improve the accuracy and efficiency of 3D morphological analysis of wheat plants, providing strong technical support for research in fields such as agricultural production optimization and genetic breeding.

Three-dimensional reconstruction and phenotype measurement of maize seedlings based on multi-view image sequences

As an important method for crop phenotype quantification, three-dimensional (3D) reconstruction is of critical importance for exploring the phenotypic characteristics of crops. In this study, maize seedlings were subjected to 3D reconstruction based on the imaging technology, and their phenotypic characters were analyzed. In the first stage, a multi-view image sequence was acquired via an RGB camera and video frame extraction method, followed by 3D reconstruction of maize based on structure from motion algorithm. Next, the original point cloud data of maize were preprocessed through Euclidean clustering algorithm, color filtering algorithm and point cloud voxel filtering algorithm to obtain a point cloud model of maize. In the second stage, the phenotypic parameters in the development process of maize seedlings were analyzed, and the maize plant height, leaf length, relative leaf area and leaf width measured through point cloud were compared with the corresponding manually measured values, and the two were highly correlated, with the coefficient of determination (R ²) of 0.991, 0.989, 0.926 and 0.963, respectively. In addition, the errors generated between the two were also analyzed, and results reflected that the proposed method was capable of rapid, accurate and nondestructive extraction. In the third stage, maize stem leaves were segmented and identified through the region growing segmentation algorithm, and the expected segmentation effect was achieved. In general, the proposed method could accurately construct the 3D morphology of maize plants, segment maize leaves, and nondestructively and accurately extract the phenotypic parameters of maize plants, thus providing a data support for the research on maize phenotypes.

Automatic Measurement of Morphological Traits of Typical Leaf Samples

It is still a challenging task to automatically measure plants. A novel method for automatic plant measurement based on a hand-held three-dimensional (3D) laser scanner is proposed. The objective of this method is to automatically select typical leaf samples and estimate their morphological traits from different occluded live plants. The method mainly includes data acquisition and processing. Data acquisition is to obtain the high-precision 3D mesh model of the plant that is reconstructed in real-time during data scanning by a hand-held 3D laser scanner (ZGScan 717, made in Zhongguan Automation Technology, Wuhan, China). Data processing mainly includes typical leaf sample extraction and morphological trait estimation based on a multi-level region growing segmentation method using two leaf shape models. Four scale-related traits and six corresponding scale-invariant traits can be automatically estimated. Experiments on four groups of different canopy-occluded plants are conducted. Experiment results show that for plants with different canopy occlusions, 94.02% of typical leaf samples can be scanned well and 87.61% of typical leaf samples can be automatically extracted. The automatically estimated morphological traits are correlated with the manually measured values EF (the modeling efficiency) above 0.8919 for scale-related traits and EF above 0.7434 for scale-invariant traits). It takes an average of 196.37 seconds (186.08 seconds for data scanning, 5.95 seconds for 3D plant model output, and 4.36 seconds for data processing) for a plant measurement. The robustness and low time cost of the proposed method for different canopy-occluded plants show potential applications for real-time plant measurement and high-throughput plant phenotype.

Phenotyping: New Windows into the Plant for Breeders

Plant phenotyping enables noninvasive quantification of plant structure and function and interactions with environments. High-capacity phenotyping reaches hitherto inaccessible phenotypic characteristics. Diverse, challenging, and valuable applications of phenotyping have originated among scientists, prebreeders, and breeders as they study the phenotypic diversity of genetic resources and apply increasingly complex traits to crop improvement. Noninvasive technologies are used to analyze experimental and breeding populations. We cover the most recent research in controlled-environment and field phenotyping for seed, shoot, and root traits. Select field phenotyping technologies have become state of the art and show promise for speeding up the breeding process in early generations. We highlight the technologies behind the rapid advances in proximal and remote sensing of plants in fields. We conclude by discussing the new disciplines working with the phenotyping community: data science, to address the challenge of generating FAIR (findable, accessible, interoperable, and reusable) data, and robotics, to apply phenotyping directly on farms.