Leaf Segmentation on Dense Plant Point Clouds with Facet Region Growing

An integrated method for phenotypic analysis of wheat based on multi-view image sequences: from seedling to grain filling stages

Wheat exhibits complex characteristics during its growth, such as extensive tillering, slender and soft leaves, and severe organ cross-obscuration, posing a considerable challenge in full-cycle phenotypic monitoring. To address this, this study presents a synthesized method based on SFM-MVS (Structure-from-Motion, Multi-View Stereo) processing for handling and segmenting wheat point clouds, covering the entire growth cycle from seedling to grain filling stages. First, a multi-view image acquisition platform was constructed to capture image sequences of wheat plants, and dense point clouds were generated using SFM-MVS technology. High-quality dense point clouds were produced by implementing improved Euclidean clustering combined with centroids, color filtering, and statistical filtering methods. Subsequently, the segmentation of wheat plant stems and leaves was performed using the region growth segmentation algorithm. Although segmentation performance was suboptimal during the tillering, jointing, and booting stages due to the glut leaves and severe overlap, there was a salient improvement in wheat leaf segmentation efficiency over the entire growth cycle. Finally, phenotypic parameters were analyzed across different growth stages, comparing automated measurements of plant height, leaf length, and leaf width with actual measurements. The results demonstrated coefficients of determination (R 2 ) of 0.9979, 0.9977, and 0.995; root mean square errors (RMSE) of 1.0773 cm, 0.2612 cm, and 0.0335 cm; and relative root mean square errors (RRMSE) of 2.1858%, 1.7483%, and 2.8462%, respectively. These results validate the reliability and accuracy of our proposed workflow in processing wheat point clouds and automatically extracting plant height, leaf length, and leaf width, indicating that our 3D reconstructed wheat model achieves high precision and can quickly, accurately, and non-destructively extract phenotypic parameters. Additionally, plant height, convex hull volume, plant surface area, and Crown area were extracted, providing a detailed analysis of dynamic changes in wheat throughout its growth cycle. ANOVA was conducted across different cultivars, accurately revealing significant differences at various growth stages. This study proposes a convenient, rapid, and quantitative analysis method, offering crucial technical support for wheat plant phenotypic analysis and growth dynamics monitoring, applicable for precise full-cycle phenotypic monitoring of wheat.

Deep Segmentation of Point Clouds of Wheat

The 3D analysis of plants has become increasingly effective in modeling the relative structure of organs and other traits of interest. In this paper, we introduce a novel pattern-based deep neural network, Pattern-Net, for segmentation of point clouds of wheat. This study is the first to segment the point clouds of wheat into defined organs and to analyse their traits directly in 3D space. Point clouds have no regular grid and thus their segmentation is challenging. Pattern-Net creates a dynamic link among neighbors to seek stable patterns from a 3D point set across several levels of abstraction using the K-nearest neighbor algorithm. To this end, different layers are connected to each other to create complex patterns from the simple ones, strengthen dynamic link propagation, alleviate the vanishing-gradient problem, encourage link reuse and substantially reduce the number of parameters. The proposed deep network is capable of analysing and decomposing unstructured complex point clouds into semantically meaningful parts. Experiments on a wheat dataset verify the effectiveness of our approach for segmentation of wheat in 3D space.

Automatic Measurement of Morphological Traits of Typical Leaf Samples

It is still a challenging task to automatically measure plants. A novel method for automatic plant measurement based on a hand-held three-dimensional (3D) laser scanner is proposed. The objective of this method is to automatically select typical leaf samples and estimate their morphological traits from different occluded live plants. The method mainly includes data acquisition and processing. Data acquisition is to obtain the high-precision 3D mesh model of the plant that is reconstructed in real-time during data scanning by a hand-held 3D laser scanner (ZGScan 717, made in Zhongguan Automation Technology, Wuhan, China). Data processing mainly includes typical leaf sample extraction and morphological trait estimation based on a multi-level region growing segmentation method using two leaf shape models. Four scale-related traits and six corresponding scale-invariant traits can be automatically estimated. Experiments on four groups of different canopy-occluded plants are conducted. Experiment results show that for plants with different canopy occlusions, 94.02% of typical leaf samples can be scanned well and 87.61% of typical leaf samples can be automatically extracted. The automatically estimated morphological traits are correlated with the manually measured values EF (the modeling efficiency) above 0.8919 for scale-related traits and EF above 0.7434 for scale-invariant traits). It takes an average of 196.37 seconds (186.08 seconds for data scanning, 5.95 seconds for 3D plant model output, and 4.36 seconds for data processing) for a plant measurement. The robustness and low time cost of the proposed method for different canopy-occluded plants show potential applications for real-time plant measurement and high-throughput plant phenotype.

Automatic method for segmenting leaves by combining 2D and 3D image-processing techniques

In this study, a method to automatically segment plant leaves from three-dimensional (3D) images using structure from motion is proposed. First, leaves in the 3D images are roughly segmented using a region-growing method in which near points with distances less than 0.2 cm are assigned to the same group. By repeating this process, the leaves not touching each other can be segmented. Then, each segmented leaf is projected onto two-dimensional (2D) images, and the watershed algorithm is executed. This process successfully segments overlapping leaves.

Three-Dimensional Monitoring of Plant Structural Parameters and Chlorophyll Distribution

Image analysis is widely used for accurate and efficient plant monitoring. Plants have complex three-dimensional (3D) structures; hence, 3D image acquisition and analysis is useful for determining the status of plants. Here, 3D images of plants were reconstructed using a photogrammetric approach, called "structure from motion". Chlorophyll content is an important parameter that determines the status of plants. Chlorophyll content was estimated from 3D images of plants with color information. To observe changes in the chlorophyll content and plant structure, a potted plant was kept for five days under a water stress condition and its 3D images were taken once a day. As a result, the normalized Red value and the chlorophyll content were correlated; a high R² value (0.81) was obtained. The absolute error of the chlorophyll content estimation in cross-validation studies was 4.0 × 10-2 μg/mm². At the same time, the structural parameters (i.e., the leaf inclination angle and the azimuthal angle) were calculated by simultaneously monitoring the changes in the plant's status in terms of its chlorophyll content and structural parameters. By combining these parameters related to plant information in plant image analysis, early detection of plant stressors, such as water stress, becomes possible.