Nondestructive 3D Image Analysis Pipeline to Extract Rice Grain Traits Using X-Ray Computed Tomography

A High-Throughput Method for Accurate Extraction of Intact Rice Panicle Traits

Rice panicle traits serve as critical indicators of both yield potential and germplasm resource quality. However, traditional manual measurements of these traits, which typically involve threshing, are not only laborious and time-consuming but also prone to introducing measurement errors. This study introduces a high-throughput and nondestructive method, termed extraction of panicle traits (EOPT), along with the software Panicle Analyzer, which is designed to assess unshaped intact rice panicle traits, including the panicle grain number, grain length, grain width, and panicle length. To address the challenge of grain occlusion within an intact panicle, we define a panicle morphology index to quantify the occlusion levels among the rice grains within the panicle. By calibrating the grain number obtained directly from rice panicle images based on the panicle morphology index, we substantially improve the grain number detection accuracy. For measuring grain length and width, the EOPT selects rice grains using an intersection over union threshold of 0.8 and a confidence threshold of 0.7 during the grain detection process. The mean values of these grains were calculated to represent all the panicle grain lengths and widths. In addition, EOPT extracted the main path of the skeleton of the rice panicle using the Astar algorithm to determine panicle lengths. Validation on a dataset of 1,554 panicle images demonstrated the effectiveness of the proposed method, achieving 93.57% accuracy in panicle grain counting with a mean absolute percentage error of 6.62%. High accuracy rates were also recorded for grain length (96.83%) and panicle length (97.13%). Moreover, the utility of EOPT was confirmed across different years and scenes, both indoors and outdoors. A genome-wide association study was conducted, leveraging the phenotypic traits obtained via EOPT and genotypic data. This study identified single-nucleotide polymorphisms associated with grain length, width, number per panicle, and panicle length, further emphasizing the utility and potential of this method in advancing rice breeding.

Toward robust and high-throughput detection of seed defects in X-ray images via deep learning

BACKGROUND

The detection of internal defects in seeds via non-destructive imaging techniques is a topic of high interest to optimize the quality of seed lots. In this context, X-ray imaging is especially suited. Recent studies have shown the feasibility of defect detection via deep learning models in 3D tomography images. We demonstrate the possibility of performing such deep learning-based analysis on 2D X-ray radiography for a faster yet robust method via the X-Robustifier pipeline proposed in this article.

RESULTS

2D X-ray images of both defective and defect-free seeds were acquired. A deep learning model based on state-of-the-art object detection neural networks is proposed. Specific data augmentation techniques are introduced to compensate for the low ratio of defects and increase the robustness to variation of the physical parameters of the X-ray imaging systems. The seed defects were accurately detected (F1-score >90%), surpassing human performance in computation time and error rates. The robustness of these models against the principal distortions commonly found in actual agro-industrial conditions is demonstrated, in particular, the robustness to physical noise, dimensionality reduction and the presence of seed coating.

CONCLUSION

This work provides a full pipeline to automatically detect common defects in seeds via 2D X-ray imaging. The method is illustrated on sugar beet and faba bean and could be efficiently extended to other species via the proposed generic X-ray data processing approach (X-Robustifier). Beyond a simple proof of feasibility, this constitutes important results toward the effective use in the routine of deep learning-based automatic detection of seed defects.

Research on the evolutionary history of the morphological structure of cotton seeds: a new perspective based on high-resolution micro-CT technology

Cotton (Gossypium hirsutum L.) seed morphological structure has a significant impact on the germination, growth and quality formation. However, the wide variation of cotton seed morphology makes it difficult to achieve quantitative analysis using traditional phenotype acquisition methods. In recent years, the application of micro-CT technology has made it possible to analyze the three-dimensional morphological structure of seeds, and has shown technical advantages in accurate identification of seed phenotypes. In this study, we reconstructed the seed morphological structure based on micro-CT technology, deep neural network Unet-3D model, and threshold segmentation methods, extracted 11 basics phenotypes traits, and constructed three new phenotype traits of seed coat specific surface area, seed coat thickness ratio and seed density ratio, using 102 cotton germplasm resources with clear year characteristics. Our results show that there is a significant positive correlation (P< 0.001) between the cotton seed size and that of the seed kernel and seed coat volume, with correlation coefficients ranging from 0.51 to 0.92, while the cavity volume has a lower correlation with other phenotype indicators (r<0.37, P< 0.001). Comparison of changes in Chinese self-bred varieties showed that seed volume, seed surface area, seed coat volume, cavity volume and seed coat thickness increased by 11.39%, 10.10%, 18.67%, 115.76% and 7.95%, respectively, while seed kernel volume, seed kernel surface area and seed fullness decreased by 7.01%, 0.72% and 16.25%. Combining with the results of cluster analysis, during the hundred-year cultivation history of cotton in China, it showed that the specific surface area of seed structure decreased by 1.27%, the relative thickness of seed coat increased by 8.70%, and the compactness of seed structure increased by 50.17%. Furthermore, the new indicators developed based on micro-CT technology can fully consider the three-dimensional morphological structure and cross-sectional characteristics among the indicators and reflect technical advantages. In this study, we constructed a microscopic phenotype research system for cotton seeds, revealing the morphological changes of cotton seeds with the year in China and providing a theoretical basis for the quantitative analysis and evaluation of seed morphology.

High-throughput and separating-free phenotyping method for on-panicle rice grains based on deep learning

Rice is a vital food crop that feeds most of the global population. Cultivating high-yielding and superior-quality rice varieties has always been a critical research direction. Rice grain-related traits can be used as crucial phenotypic evidence to assess yield potential and quality. However, the analysis of rice grain traits is still mainly based on manual counting or various seed evaluation devices, which incur high costs in time and money. This study proposed a high-precision phenotyping method for rice panicles based on visible light scanning imaging and deep learning technology, which can achieve high-throughput extraction of critical traits of rice panicles without separating and threshing rice panicles. The imaging of rice panicles was realized through visible light scanning. The grains were detected and segmented using the Faster R-CNN-based model, and an improved Pix2Pix model cascaded with it was used to compensate for the information loss caused by the natural occlusion between the rice grains. An image processing pipeline was designed to calculate fifteen phenotypic traits of the on-panicle rice grains. Eight varieties of rice were used to verify the reliability of this method. The R2 values between the extraction by the method and manual measurements of the grain number, grain length, grain width, grain length/width ratio and grain perimeter were 0.99, 0.96, 0.83, 0.90 and 0.84, respectively. Their mean absolute percentage error (MAPE) values were 1.65%, 7.15%, 5.76%, 9.13% and 6.51%. The average imaging time of each rice panicle was about 60 seconds, and the total time of data processing and phenotyping traits extraction was less than 10 seconds. By randomly selecting one thousand grains from each of the eight varieties and analyzing traits, it was found that there were certain differences between varieties in the number distribution of thousand-grain length, thousand-grain width, and thousand-grain length/width ratio. The results show that this method is suitable for high-throughput, non-destructive, and high-precision extraction of on-panicle grains traits without separating. Low cost and robust performance make it easy to popularize. The research results will provide new ideas and methods for extracting panicle traits of rice and other crops.

Automated extraction of pod phenotype data from micro-computed tomography

INTRODUCTION

Plant image datasets have the potential to greatly improve our understanding of the phenotypic response of plants to environmental and genetic factors. However, manual data extraction from such datasets are known to be time-consuming and resource intensive. Therefore, the development of efficient and reliable machine learning methods for extracting phenotype data from plant imagery is crucial.

METHODS

In this paper, a current gold standard computed vision method for detecting and segmenting objects in three-dimensional imagery (StartDist-3D) is applied to X-ray micro-computed tomography scans of oilseed rape (Brassica napus) mature pods.

RESULTS

With a relatively minimal training effort, this fine-tuned StarDist-3D model accurately detected (Validation F1-score = 96.3%,Testing F1-score = 99.3%) and predicted the shape (mean matched score = 90%) of seeds.

DISCUSSION

This method then allowed rapid extraction of data on the number, size, shape, seed spacing and seed location in specific valves that can be integrated into models of plant development or crop yield. Additionally, the fine-tuned StarDist-3D provides an efficient way to create a dataset of segmented images of individual seeds that could be used to further explore the factors affecting seed development, abortion and maturation synchrony within the pod. There is also potential for the fine-tuned Stardist-3D method to be applied to imagery of seeds from other plant species, as well as imagery of similarly shaped plant structures such as beans or wheat grains, provided the structures targeted for detection and segmentation can be described as star-convex polygons.

Nondestructive 3D phenotyping method of passion fruit based on X-ray micro-computed tomography and deep learning

Passion fruit is a tropical liana of the Passiflora family that is commonly planted throughout the world due to its abundance of nutrients and industrial value. Researchers are committed to exploring the relationship between phenotype and genotype to promote the improvement of passion fruit varieties. However, the traditional manual phenotyping methods have shortcomings in accuracy, objectivity, and measurement efficiency when obtaining large quantities of personal data on passion fruit, especially internal organization data. This study selected samples of passion fruit from three widely grown cultivars, which differed significantly in fruit shape, size, and other morphological traits. A Micro-CT system was developed to perform fully automated nondestructive imaging of the samples to obtain 3D models of passion fruit. A designed label generation method and segmentation method based on U-Net model were used to distinguish different tissues in the samples. Finally, fourteen traits, including fruit volume, surface area, length and width, sarcocarp volume, pericarp thickness, and traits of fruit type, were automatically calculated. The experimental results show that the segmentation accuracy of the deep learning model reaches more than 0.95. Compared with the manual measurements, the mean absolute percentage error of the fruit width and length measurements by the Micro-CT system was 1.94% and 2.89%, respectively, and the squares of the correlation coefficients were 0.96 and 0.93. It shows that the measurement accuracy of external traits of passion fruit is comparable to manual operations, and the measurement of internal traits is more reliable because of the nondestructive characteristics of our method. According to the statistical data of the whole samples, the Pearson analysis method was used, and the results indicated specific correlations among fourteen phenotypic traits of passion fruit. At the same time, the results of the principal component analysis illustrated that the comprehensive quality of passion fruit could be scored using this method, which will help to screen for high-quality passion fruit samples with large sizes and high sarcocarp content. The results of this study will firstly provide a nondestructive method for more accurate and efficient automatic acquisition of comprehensive phenotypic traits of passion fruit and have the potential to be extended to more fruit crops. The preliminary study of the correlation between the characteristics of passion fruit can also provide a particular reference value for molecular breeding and comprehensive quality evaluation.

An Intelligent Rice Yield Trait Evaluation System Based on Threshed Panicle Compensation

High-throughput phenotyping of yield-related traits is meaningful and necessary for rice breeding and genetic study. The conventional method for rice yield-related trait evaluation faces the problems of rice threshing difficulties, measurement process complexity, and low efficiency. To solve these problems, a novel intelligent system, which includes an integrated threshing unit, grain conveyor-imaging units, threshed panicle conveyor-imaging unit, and specialized image analysis software has been proposed to achieve rice yield trait evaluation with high throughput and high accuracy. To improve the threshed panicle detection accuracy, the Region of Interest Align, Convolution Batch normalization activation with Leaky Relu module, Squeeze-and-Excitation unit, and optimal anchor size have been adopted to optimize the Faster-RCNN architecture, termed 'TPanicle-RCNN,' and the new model achieved F1 score 0.929 with an increase of 0.044, which was robust to indica and japonica varieties. Additionally, AI cloud computing was adopted, which dramatically reduced the system cost and improved flexibility. To evaluate the system accuracy and efficiency, 504 panicle samples were tested, and the total spikelet measurement error decreased from 11.44 to 2.99% with threshed panicle compensation. The average measuring efficiency was approximately 40 s per sample, which was approximately twenty times more efficient than manual measurement. In this study, an automatic and intelligent system for rice yield-related trait evaluation was developed, which would provide an efficient and reliable tool for rice breeding and genetic research.

The influence of grain shape and size on the relationship between porosity and permeability in sandstone: a digital approach

An accurate and reliable description of the porosity–permeability relationship in geological materials is valuable in understanding subsurface fluid movement. This is important for reservoir characterisation, energy exploitation, geological carbon storage (GCS) and groundwater contamination and remediation. Whilst the relationship between pore characteristics and porosity and permeability are well examined, further investigation into the influence of grain characteristics on porosity and permeability would be beneficial due to the inherent relationship between grains and pores. This work aims to determine whether incorporation of grain characteristics into a porosity–permeability model is effective in constraining this relationship. Two fully digital approaches to individual 3D grain analysis based upon watershed segmentation are compared to determine the most effective, yet simple, workflow applicable to core plugs of significantly compacted grains. The identification of an effective segmentation workflow will facilitate future work on similarly complex materials, removing the need for traditional time-consuming and manual techniques. We use the most effective approach of measuring grain shape (sphericity) and size (Feret diameter) alongside an established fully digital workflow to measure porosity and permeability to investigate the impact of grain characteristics on porosity and permeability. We show that grain sphericity and porosity exhibit a positive relationship whereas no such relationship exists with grain size. Measurements of grain sphericity are applied to calculate a Kozeny–Carman (K–C) type porosity–permeability fit which was found to be unsatisfactory, compared to a simpler fit excluding any grain parameters. This is possibly due to the lower sphericity of the studied grains, deviating significantly from the K–C assumption that grains are entirely spherical. The simpler fit is most suitable for the studied materials, showing that inclusion of grain characteristics is not effective for better defining the porosity–permeability relationship in a K–C paradigm for these samples. This highlights the need for a model capable of considering a range of grain sphericities to further constrain the porosity–permeability relationship.

3D modeling and reconstruction of plants and trees: A cross-cutting review across computer graphics, vision, and plant phenotyping

This paper reviews the past and current trends of three-dimensional (3D) modeling and reconstruction of plants and trees. These topics have been studied in multiple research fields, including computer vision, graphics, plant phenotyping, and forestry. This paper, therefore, provides a cross-cutting review. Representations of plant shape and structure are first summarized, where every method for plant modeling and reconstruction is based on a shape/structure representation. The methods were then categorized into 1) creating non-existent plants (modeling) and 2) creating models from real-world plants (reconstruction). This paper also discusses the limitations of current methods and possible future directions.

High-Throughput Legume Seed Phenotyping Using a Handheld 3D Laser Scanner

High-throughput phenotyping involves many samples and diverse trait types. For the goal of automatic measurement and batch data processing, a novel method for high-throughput legume seed phenotyping is proposed. A pipeline of automatic data acquisition and processing, including point cloud acquisition, single-seed extraction, pose normalization, three-dimensional (3D) reconstruction, and trait estimation, is proposed. First, a handheld laser scanner is used to obtain the legume seed point clouds in batches. Second, a combined segmentation method using the RANSAC method, the Euclidean segmentation method, and the dimensionality of the features is proposed to conduct single-seed extraction. Third, a coordinate rotation method based on PCA and the table normal is proposed to conduct pose normalization. Fourth, a fast symmetry-based 3D reconstruction method is built to reconstruct a 3D model of the single seed, and the Poisson surface reconstruction method is used for surface reconstruction. Finally, 34 traits, including 11 morphological traits, 11 scale factors, and 12 shape factors, are automatically calculated. A total of 2500 samples of five kinds of legume seeds are measured. Experimental results show that the average accuracies of scanning and segmentation are 99.52% and 100%, respectively. The overall average reconstruction error is 0.014 mm. The average morphological trait measurement accuracy is submillimeter, and the average relative percentage error is within 3%. The proposed method provides a feasible method of batch data acquisition and processing, which will facilitate the automation in high-throughput legume seed phenotyping.

Mask R-CNN-based feature extraction and three-dimensional recognition of rice panicle CT images

The rice panicle seed setting rate is extremely important for calculating rice yield and performing genetic analysis. Unlike machine vision, X-ray computed tomography (CT) imaging is a nondestructive technique that provides direct information on the internal and external structure of rice panicles. However, occlusion and adhesion of panicles and grains in a CT image sequence make these objects difficult to identify, which in turn hinders accurate determination of the seed setting rate of rice panicles. Therefore, this paper proposes a method based on a mask region convolutional neural network (Mask R-CNN) for feature extraction and three-dimensional (3-D) recognition of CT images of rice panicles. X-ray CT feature characterization was combined with the Mask R-CNN algorithm to perform feature extraction and classification of a panicle and grains in each layer of the CT sequence. The Euclidean distance between adjacent layers was minimized to extract the features of a 3-D panicle and grains. The results were used to calculate the rice panicle seed setting rate. The proposed method was experimentally verified using eight sets of different rice panicles. The results showed that the proposed method can efficiently identify and count plump grains and blighted grains to achieve an accuracy above 99% for the seed setting rate.

A high-throughput and low-cost maize ear traits scorer

In this study, based on automatic control and image processing, a high-throughput and low-cost maize ear traits scorer (METS) was developed for the automatic measurement of 34 maize ear traits. In total, 813 maize ears were measured using METS, and the results showed that the square of the correlation coefficient (R2) of the manual measurements versus the automatic measurements for ear length, ear diameter, and kernel thickness were 0.96, 0.79, and 0.85, respectively. These maize ear traits could be used to classify the type, and the results showed that the classification accuracy of the support vector machine (SVM) model for the test set was better than that of the random forest (RF) model. In addition, the general applicability of the image analysis pipeline was also demonstrated on other independent maize ear phenotyping platforms. In conclusion, equipped with image processing and automatic control technologies, we have developed a high-throughput method for maize ear scoring, which could be popularized in maize functional genetics, genomics, and breeding applications.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11032-021-01205-4.

High-Throughput Phenotyping of Morphological Seed and Fruit Characteristics Using X-Ray Computed Tomography

Traditional seed and fruit phenotyping are mainly accomplished by manual measurement or extraction of morphological properties from two-dimensional images. These methods are not only in low-throughput but also unable to collect their three-dimensional (3D) characteristics and internal morphology. X-ray computed tomography (CT) scanning, which provides a convenient means of non-destructively recording the external and internal 3D structures of seeds and fruits, offers a potential to overcome these limitations. However, the current CT equipment cannot be adopted to scan seeds and fruits with high throughput. And there is no specialized software for automatic extraction of phenotypes from CT images. Here, we introduced a high-throughput image acquisition approach by mounting a specially designed seed-fruit container onto the scanning bed. The corresponding 3D image analysis software, 3DPheno-Seed&Fruit, was created for automatic segmentation and rapid quantification of eight morphological phenotypes of internal and external compartments of seeds and fruits. 3DPheno-Seed&Fruit is a graphical user interface design and user-friendly software with an excellent phenotype result visualization function. We described the software in detail and benchmarked it based upon CT image analyses in seeds of soybean, wheat, peanut, pine nut, pistachio nut and dwarf Russian almond fruit. R 2 values between the extracted and manual measurements of seed length, width, thickness, and radius ranged from 0.80 to 0.96 for soybean and wheat. High correlations were found between the 2D (length, width, thickness, and radius) and 3D (volume and surface area) phenotypes for soybean. Overall, our methods provide robust and novel tools for phenotyping the morphological seed and fruit traits of various plant species, which could benefit crop breeding and functional genomics.

3D Visualization and Volume-Based Quantification of Rice Chalkiness In Vivo by Using High Resolution Micro-CT

BACKGROUND

Rice quality research attracts attention worldwide. Rice chalkiness is one of the key indexes determining rice kernel quality. The traditional rice chalkiness measurement methods only use milled rice as materials and are mainly based on naked-eye observation or area-based two-dimensional (2D) image analysis and the results could not represent the three-dimensional (3D) characteristics of chalkiness in the rice kernel. These methods are neither in vivo thus are unable to analyze living rice seeds for high throughput screening of rice chalkiness phenotype.

RESULTS

Here, we introduced a novel method for 3D visualization and accurate volume-based quantification of rice chalkiness in vivo by using X-ray microcomputed tomography (micro-CT). This approach not only develops a novel volume-based method to measure the 3D rice chalkiness index, but also provides a high throughput solution for rice chalkiness phenotype analysis by using living rice seeds.

CONCLUSIONS

Our method could be a new powerful tool for rice chalkiness measurement, especially for high throughput chalkiness phenotype screening using living rice seeds. This method could be used in chalkiness phenotype identification and screening, and would greatly promote the basic research in rice chalkiness regulation as well as the quality evaluation in rice production practice.