High-throughput phenotyping analysis of maize at the seedling stage using end-to-end segmentation network

Detection of maize stem diameter by using RGB-D cameras' depth information under selected field condition

Stem diameter is a critical phenotypic parameter for maize, integral to yield prediction and lodging resistance assessment. Traditionally, the quantification of this parameter through manual measurement has been the norm, notwithstanding its tedious and laborious nature. To address these challenges, this study introduces a non-invasive field-based system utilizing depth information from RGB-D cameras to measure maize stem diameter. This technology offers a practical solution for conducting rapid and non-destructive phenotyping. Firstly, RGB images, depth images, and 3D point clouds of maize stems were captured using an RGB-D camera, and precise alignment between the RGB and depth images was achieved. Subsequently, the contours of maize stems were delineated using 2D image processing techniques, followed by the extraction of the stem's skeletal structure employing a thinning-based skeletonization algorithm. Furthermore, within the areas of interest on the maize stems, horizontal lines were constructed using points on the skeletal structure, resulting in 2D pixel coordinates at the intersections of these horizontal lines with the maize stem contours. Subsequently, a back-projection transformation from 2D pixel coordinates to 3D world coordinates was achieved by combining the depth data with the camera's intrinsic parameters. The 3D world coordinates were then precisely mapped onto the 3D point cloud using rigid transformation techniques. Finally, the maize stem diameter was sensed and determined by calculating the Euclidean distance between pairs of 3D world coordinate points. The method demonstrated a Mean Absolute Percentage Error (MAPE) of 3.01%, a Mean Absolute Error (MAE) of 0.75 mm, a Root Mean Square Error (RMSE) of 1.07 mm, and a coefficient of determination (R²) of 0.96, ensuring accurate measurement of maize stem diameter. This research not only provides a new method of precise and efficient crop phenotypic analysis but also offers theoretical knowledge for the advancement of precision agriculture.

Time-Series Field Phenotyping of Soybean Growth Analysis by Combining Multimodal Deep Learning and Dynamic Modeling

The rate of soybean canopy establishment largely determines photoperiodic sensitivity, subsequently influencing yield potential. However, assessing the rate of soybean canopy development in large-scale field breeding trials is both laborious and time-consuming. High-throughput phenotyping methods based on unmanned aerial vehicle (UAV) systems can be used to monitor and quantitatively describe the development of soybean canopies for different genotypes. In this study, high-resolution and time-series raw data from field soybean populations were collected using UAVs. The RGB (red, green, and blue) and infrared images are used as inputs to construct the multimodal image segmentation model-the RGB & Infrared Feature Fusion Segmentation Network (RIFSeg-Net). Subsequently, the segment anything model was employed to extract complete individual leaves from the segmentation results obtained from RIFSeg-Net. These leaf aspect ratios facilitated the accurate categorization of soybean populations into 2 distinct varieties: oval leaf type variety and lanceolate leaf type variety. Finally, dynamic modeling was conducted to identify 5 phenotypic traits associated with the canopy development rate that differed substantially among the classified soybean varieties. The results showed that the developed multimodal image segmentation model RIFSeg-Net for extracting soybean canopy cover from UAV images outperformed traditional deep learning image segmentation networks (precision = 0.94, recall = 0.93, F1-score = 0.93). The proposed method has high practical value in the field of germplasm resource identification. This approach could lead to the use of a practical tool for further genotypic differentiation analysis and the selection of target genes.

Three-Dimensional Modeling of Maize Canopies Based on Computational Intelligence

The 3-dimensional (3D) modeling of crop canopies is fundamental for studying functional-structural plant models. Existing studies often fail to capture the structural characteristics of crop canopies, such as organ overlapping and resource competition. To address this issue, we propose a 3D maize modeling method based on computational intelligence. An initial 3D maize canopy is created using the t-distribution method to reflect characteristics of the plant architecture. The subsequent model considers the 3D phytomers of maize as intelligent agents. The aim is to maximize the ratio of sunlit leaf area, and by iteratively modifying the azimuth angle of the 3D phytomers, a 3D maize canopy model that maximizes light resource interception can be constructed. Additionally, the method incorporates a reflective approach to optimize the canopy and utilizes a mesh deformation technique for detecting and responding to leaf collisions within the canopy. Six canopy models of 2 varieties plus 3 planting densities was constructed for validation. The average R2 of the difference in azimuth angle between adjacent leaves is 0.71, with a canopy coverage error range of 7% to 17%. Another 3D maize canopy model constructed using 12 distinct density gradients demonstrates the proportion of leaves perpendicular to the row direction increases along with the density. The proportion of these leaves steadily increased after 9 × 104 plants ha-1. This study presents a 3D modeling method for the maize canopy. It is a beneficial exploration of swarm intelligence on crops and generates a new way for exploring efficient resources utilization of crop canopies.

In Situ Measuring Stem Diameters of Maize Crops with a High-Throughput Phenotyping Robot

Robotic High-Throughput Phenotyping (HTP) technology has been a powerful tool for selecting high-quality crop varieties among large quantities of traits. Due to the advantages of multi-view observation and high accuracy, ground HTP robots have been widely studied in recent years. In this paper, we study an ultra-narrow wheeled robot equipped with RGB-D cameras for inter-row maize HTP. The challenges of the narrow operating space, intensive light changes, and messy cross-leaf interference in rows of maize crops are considered. An in situ and inter-row stem diameter measurement method for HTP robots is proposed. To this end, we first introduce the stem diameter measurement pipeline, in which a convolutional neural network is employed to detect stems, and the point cloud is analyzed to estimate the stem diameters. Second, we present a clustering strategy based on DBSCAN for extracting stem point clouds under the condition that the stem is shaded by dense leaves. Third, we present a point cloud filling strategy to fill the stem region with missing depth values due to the occlusion by other organs. Finally, we employ convex hull and plane projection of the point cloud to estimate the stem diameters. The results show that the R2 and RMSE of stem diameter measurement are up to 0.72 and 2.95 mm, demonstrating its effectiveness.

Genetic architecture of variation in Arabidopsis thaliana rosettes

Rosette morphology across Arabidopsis accessions exhibits considerable variation. Here we report a high-throughput phenotyping approach based on automatic image analysis to quantify rosette shape and dissect the underlying genetic architecture. Shape measurements of the rosettes in a core set of Recombinant Inbred Lines from an advanced mapping population (Multiparent Advanced Generation Inter-Cross or MAGIC) derived from inter-crossing 19 natural accessions. Image acquisition and analysis was scaled to extract geometric descriptors from time stamped images of growing rosettes. Shape analyses revealed heritable morphological variation at early juvenile stages and QTL mapping resulted in over 116 chromosomal regions associated with trait variation within the population. Many QTL linked to variation in shape were located near genes related to hormonal signalling and signal transduction pathways while others are involved in shade avoidance and transition to flowering. Our results suggest rosette shape arises from modular integration of sub-organ morphologies and can be considered a functional trait subjected to selective pressures of subsequent morphological traits. On an applied aspect, QTLs found will be candidates for further research on plant architecture.

UAV Oblique Imagery with an Adaptive Micro-Terrain Model for Estimation of Leaf Area Index and Height of Maize Canopy from 3D Point Clouds

Leaf area index (LAI) and height are two critical measures of maize crops that are used in ecophysiological and morphological studies for growth evaluation, health assessment, and yield prediction. However, mapping spatial and temporal variability of LAI in fields using handheld tools and traditional techniques is a tedious and costly pointwise operation that provides information only within limited areas. The objective of this study was to evaluate the reliability of mapping LAI and height of maize canopy from 3D point clouds generated from UAV oblique imagery with the adaptive micro-terrain model. The experiment was carried out in a field planted with three cultivars having different canopy shapes and four replicates covering a total area of 48 × 36 m. RGB images in nadir and oblique view were acquired from the maize field at six different time slots during the growing season. Images were processed by Agisoft Metashape to generate 3D point clouds using the structure from motion method and were later processed by MATLAB to obtain clean canopy structure, including height and density. The LAI was estimated by a multivariate linear regression model using crop canopy descriptors derived from the 3D point cloud, which account for height and leaf density distribution along the canopy height. A simulation analysis based on the Sine function effectively demonstrated the micro-terrain model from point clouds. For the ground truth data, a randomized block design with 24 sample areas was used to manually measure LAI, height, N-pen data, and yield during the growing season. It was found that canopy height data from the 3D point clouds has a relatively strong correlation (R2 = 0.89, 0.86, 0.78) with the manual measurement for three cultivars with CH90. The proposed methodology allows a cost-effective high-resolution mapping of in-field LAI index extraction through UAV 3D data to be used as an alternative to the conventional LAI assessments even in inaccessible regions.

3D phytomer-based geometric modelling method for plants-the case of maize

Geometric plant modelling is crucial in in silico plants. Existing geometric modelling methods have focused on the topological structure and basic organ profiles, simplifying the morphological features. However, the models cannot effectively differentiate cultivars, limiting FSPM application in crop breeding and management. This study proposes a 3D phytomer-based geometric modelling method with maize (Zea Mays) as the representative plant. Specifically, conversion methods between skeleton and mesh models of 3D phytomer are specified. This study describes the geometric modelling of maize shoots and populations by assembling 3D phytomers. Results show that the method can quickly and efficiently construct 3D models of maize plants and populations, with the ability to show morphological, structural and functional differences among four representative cultivars. The method takes into account both the geometric modelling efficiency and 3D detail features to achieve automatic operation of geometric modelling through the standardized description of 3D phytomers. Therefore, this study provides a theoretical and technical basis for the research and application of in silico plants.