1
|
Cen Y, Huang X, Liu J, Qin Y, Wu X, Ye S, Du S, Liao W. Application of three-dimensional reconstruction technology in dentistry: a narrative review. BMC Oral Health 2023; 23:630. [PMID: 37667286 PMCID: PMC10476426 DOI: 10.1186/s12903-023-03142-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2023] [Accepted: 06/16/2023] [Indexed: 09/06/2023] Open
Abstract
BACKGROUND Three-dimensional(3D) reconstruction technology is a method of transforming real goals into mathematical models consistent with computer logic expressions and has been widely used in dentistry, but the lack of review and summary leads to confusion and misinterpretation of information. The purpose of this review is to provide the first comprehensive link and scientific analysis of 3D reconstruction technology and dentistry to bridge the information bias between these two disciplines. METHODS The IEEE Xplore and PubMed databases were used for rigorous searches based on specific inclusion and exclusion criteria, supplemented by Google Academic as a complementary tool to retrieve all literature up to February 2023. We conducted a narrative review focusing on the empirical findings of the application of 3D reconstruction technology to dentistry. RESULTS We classify the technologies applied to dentistry according to their principles and summarize the different characteristics of each category, as well as the different application scenarios determined by these characteristics of each technique. In addition, we indicate their development prospects and worthy research directions in the field of dentistry, from individual techniques to the overall discipline of 3D reconstruction technology, respectively. CONCLUSIONS Researchers and clinicians should make different decisions on the choice of 3D reconstruction technology based on different objectives. The main trend in the future development of 3D reconstruction technology is the joint application of technology.
Collapse
Affiliation(s)
- Yueyan Cen
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Xinyue Huang
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Jialing Liu
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Yichun Qin
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Xinrui Wu
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Shiyang Ye
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China
| | - Shufang Du
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China.
| | - Wen Liao
- State Key Laboratory of Oral Diseases & National Clinical Research Center for Oral Diseases, Department of Orthodontics, West China Hospital of Stomatology, Sichuan University, No.14, 3Rd Section of Ren Min Nan Rd. Chengdu, Sichuan, 610041, China.
| |
Collapse
|
2
|
Chen MK, Liu X, Wu Y, Zhang J, Yuan J, Zhang Z, Tsai DP. A Meta-Device for Intelligent Depth Perception. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2107465. [PMID: 35986633 DOI: 10.1002/adma.202107465] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 08/09/2022] [Indexed: 06/15/2023]
Abstract
The optical illusion affects depth-sensing due to the limited and specific light-field information acquired by single-lens imaging. The incomplete depth information or visual deception would cause cognitive errors. To resolve this problem, an intelligent and compact depth-sensing meta-device that is miniaturized, integrated, and applicable for diverse scenes in all light levels is demonstrated. The compact and multifunction stereo vision system adopts an array with 3600 achromatic meta-lenses and a size of 1.2 × 1.2 mm2 to measure the depth over a 30 cm range with deep-learning support. The meta-lens array can act as multiple imaging lenses to collect light field information. It can also work with a light source as an active optical device to project a structured light. The meta-lens array can serve as the core functional component of a light-field imaging system under bright conditions or a structured-light projection system in the dark. The depth information in both ways can be analyzed and extracted by the convolutional neural network. This work provides a new avenue for the applications such as autonomous driving, machine vision, human-computer interaction, augmented reality, biometric identification, etc.
Collapse
Affiliation(s)
- Mu Ku Chen
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
- Centre for Biosystems, Neuroscience and Nanotechnology, City University of Hong Kong, Kowloon, Hong Kong, 999077, P. R. China
- The State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Kowloon, Hong Kong, 999077, P. R. China
| | - Xiaoyuan Liu
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
| | - Yongfeng Wu
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
| | - Jingcheng Zhang
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
| | - Jiaqi Yuan
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
| | - Zhengnan Zhang
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
| | - Din Ping Tsai
- Department of Electrical Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong, 999077, P. R. China
- Centre for Biosystems, Neuroscience and Nanotechnology, City University of Hong Kong, Kowloon, Hong Kong, 999077, P. R. China
- The State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Kowloon, Hong Kong, 999077, P. R. China
| |
Collapse
|
3
|
Li H, Wu G, Tao S, Yin H, Qi K, Zhang S, Guo W, Ninomiya S, Mu Y. Automatic Branch-Leaf Segmentation and Leaf Phenotypic Parameter Estimation of Pear Trees Based on Three-Dimensional Point Clouds. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23094572. [PMID: 37177776 PMCID: PMC10181666 DOI: 10.3390/s23094572] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Revised: 04/30/2023] [Accepted: 05/05/2023] [Indexed: 05/15/2023]
Abstract
The leaf phenotypic traits of plants have a significant impact on the efficiency of canopy photosynthesis. However, traditional methods such as destructive sampling will hinder the continuous monitoring of plant growth, while manual measurements in the field are both time-consuming and laborious. Nondestructive and accurate measurements of leaf phenotypic parameters can be achieved through the use of 3D canopy models and object segmentation techniques. This paper proposed an automatic branch-leaf segmentation pipeline based on lidar point cloud and conducted the automatic measurement of leaf inclination angle, length, width, and area, using pear canopy as an example. Firstly, a three-dimensional model using a lidar point cloud was established using SCENE software. Next, 305 pear tree branches were manually divided into branch points and leaf points, and 45 branch samples were selected as test data. Leaf points were further marked as 572 leaf instances on these test data. The PointNet++ model was used, with 260 point clouds as training input to carry out semantic segmentation of branches and leaves. Using the leaf point clouds in the test dataset as input, a single leaf instance was extracted by means of a mean shift clustering algorithm. Finally, based on the single leaf point cloud, the leaf inclination angle was calculated by plane fitting, while the leaf length, width, and area were calculated by midrib fitting and triangulation. The semantic segmentation model was tested on 45 branches, with a mean Precisionsem, mean Recallsem, mean F1-score, and mean Intersection over Union (IoU) of branches and leaves of 0.93, 0.94, 0.93, and 0.88, respectively. For single leaf extraction, the Precisionins, Recallins, and mean coverage (mCoV) were 0.89, 0.92, and 0.87, respectively. Using the proposed method, the estimated leaf inclination, length, width, and area of pear leaves showed a high correlation with manual measurements, with correlation coefficients of 0.94 (root mean squared error: 4.44°), 0.94 (root mean squared error: 0.43 cm), 0.91 (root mean squared error: 0.39 cm), and 0.93 (root mean squared error: 5.21 cm2), respectively. These results demonstrate that the method can automatically and accurately measure the phenotypic parameters of pear leaves. This has great significance for monitoring pear tree growth, simulating canopy photosynthesis, and optimizing orchard management.
Collapse
Affiliation(s)
- Haitao Li
- Academy for Advanced Interdisciplinary Studies, Collaborative Innovation Center for Modern Crop Production Co-Sponsored by Province and Ministry, Nanjing Agricultural University, Nanjing 210095, China
- College of Artificial Intelligence, Nanjing Agricultural University, Nanjing 210095, China
| | - Gengchen Wu
- Academy for Advanced Interdisciplinary Studies, Collaborative Innovation Center for Modern Crop Production Co-Sponsored by Province and Ministry, Nanjing Agricultural University, Nanjing 210095, China
| | - Shutian Tao
- Centre of Pear Engineering Technology Research, State Key Laboratory of Crop Genetics & Germplasm Enhancement and Utilization, Nanjing Agricultural University, Nanjing 210095, China
| | - Hao Yin
- Centre of Pear Engineering Technology Research, State Key Laboratory of Crop Genetics & Germplasm Enhancement and Utilization, Nanjing Agricultural University, Nanjing 210095, China
| | - Kaijie Qi
- Centre of Pear Engineering Technology Research, State Key Laboratory of Crop Genetics & Germplasm Enhancement and Utilization, Nanjing Agricultural University, Nanjing 210095, China
| | - Shaoling Zhang
- Centre of Pear Engineering Technology Research, State Key Laboratory of Crop Genetics & Germplasm Enhancement and Utilization, Nanjing Agricultural University, Nanjing 210095, China
| | - Wei Guo
- Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Tokyo 188-0002, Japan
| | - Seishi Ninomiya
- Academy for Advanced Interdisciplinary Studies, Collaborative Innovation Center for Modern Crop Production Co-Sponsored by Province and Ministry, Nanjing Agricultural University, Nanjing 210095, China
- Graduate School of Agricultural and Life Sciences, The University of Tokyo, 1-1-1 Midori-cho, Tokyo 188-0002, Japan
| | - Yue Mu
- Academy for Advanced Interdisciplinary Studies, Collaborative Innovation Center for Modern Crop Production Co-Sponsored by Province and Ministry, Nanjing Agricultural University, Nanjing 210095, China
| |
Collapse
|
4
|
Dharni JS, Dhatt BK, Paul P, Gao T, Awada T, Bacher H, Peleg Z, Staswick P, Hupp J, Yu H, Walia H. A non-destructive approach for measuring rice panicle-level photosynthetic responses using 3D-image reconstruction. PLANT METHODS 2022; 18:126. [PMID: 36443862 PMCID: PMC9703705 DOI: 10.1186/s13007-022-00959-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 11/14/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND Our understanding of the physiological responses of rice inflorescence (panicle) to environmental stresses is limited by the challenge of accurately determining panicle photosynthetic parameters and their impact on grain yield. This is primarily due to the lack of a suitable gas exchange methodology for panicles and non-destructive methods to accurately determine panicle surface area. RESULTS To address these challenges, we have developed a custom panicle gas exchange cylinder compatible with the LiCor 6800 Infra-red Gas Analyzer. Accurate surface area measurements were determined using 3D panicle imaging to normalize the panicle-level photosynthetic measurements. We observed differential responses in both panicle and flag leaf for two temperate Japonica rice genotypes (accessions TEJ-1 and TEJ-2) exposed to heat stress during early grain filling. There was a notable divergence in the relative photosynthetic contribution of flag leaf and panicles for the heat-tolerant genotype (TEJ-2) compared to the sensitive genotype (TEJ-1). CONCLUSION The novelty of this method is the non-destructive and accurate determination of panicle area and photosynthetic parameters, enabling researchers to monitor temporal changes in panicle physiology during the reproductive development. The method is useful for panicle-level measurements under diverse environmental stresses and is sensitive enough to evaluate genotypic variation for panicle physiology and architecture in cereals with compact inflorescences.
Collapse
Affiliation(s)
- Jaspinder Singh Dharni
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA
| | - Balpreet Kaur Dhatt
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA
| | - Puneet Paul
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA
| | - Tian Gao
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, NE, 68588, USA
| | - Tala Awada
- School of Natural Resources, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA
| | - Harel Bacher
- The Robert H. Smith Institute of Plant Sciences and Genetics in Agriculture, The Hebrew University of Jerusalem, Rehovot, Israel
| | - Zvi Peleg
- The Robert H. Smith Institute of Plant Sciences and Genetics in Agriculture, The Hebrew University of Jerusalem, Rehovot, Israel
| | - Paul Staswick
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA
| | - Jason Hupp
- LI-COR Inc., 4647 Superior Street, Lincoln, NE, 68505, USA
| | - Hongfeng Yu
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, NE, 68588, USA
| | - Harkamal Walia
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, NE, 68583, USA.
| |
Collapse
|
5
|
Li Z, Yuan X, Wang C. A review on structural development and recognition–localization methods for end-effector of fruit–vegetable picking robots. INT J ADV ROBOT SYST 2022. [DOI: 10.1177/17298806221104906] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
The excellent performance of fruit and vegetable picking robots is usually contributed by the reasonable structure of end-effector and recognition–localization methods with high accuracy. As a result, efforts are focused on two aspects, and diverse structures of end-effector, target recognition methods as well as their combinations are yielded continuously. A good understanding for the working principle, advantages, limitations, and the adaptability in respective fields is helpful to design picking robots. Therefore, depending on different grasping ways, separating methods, structures, materials, and driving modes, main characteristics existing in traditional schemes will be depicted firstly. According to technical routes, advantages, potential applications, and challenges, underactuated manipulators and soft manipulators representing future development are then summarized systematically. Secondly, partial recognition and localization methods are also demonstrated. Specifically, current recognition manners adopting the single-feature, multi-feature fusion and deep learning are explained in view of their advantages, limitations, and successful instances. In the field of 3D localization, active vision based on the structured light, laser scanning, time of flight, and radar is reflected through the respective applications. Also, another 3D localization method called passive vision is also evaluated by advantages, limitations, the degree of automation, reconstruction effects, and the application scenario, such as monocular vision, binocular vision, and multiocular vision. Finally portrayed from structural development, recognition, and localization methods, it is possible to develop future end-effectors for fruit and vegetable picking robots with superior characteristics containing the less driving element, rigid–flexible–bionic coupling soft manipulators, simple control program, high efficiency, low damage, low cost, high versatility, and high recognition accuracy in all-season picking tasks.
Collapse
Affiliation(s)
- Ziyue Li
- School of Automotive Engineering, Hubei University of Automotive Technology, Shiyan, PR China
| | - Xianju Yuan
- School of Automotive Engineering, Hubei University of Automotive Technology, Shiyan, PR China
- Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada
| | - Chuyan Wang
- School of Automotive Engineering, Hubei University of Automotive Technology, Shiyan, PR China
| |
Collapse
|
6
|
Zhang K, Chen H, Wu H, Zhao X, Zhou C. Point cloud registration method for maize plants based on conical surface fitting-ICP. Sci Rep 2022; 12:6852. [PMID: 35478217 PMCID: PMC9046160 DOI: 10.1038/s41598-022-10921-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 03/17/2022] [Indexed: 11/09/2022] Open
Abstract
Reconstructing three-dimensional (3D) point cloud model of maize plants can provide reliable data for its growth observation and agricultural machinery research. The existing data collection systems and registration methods have low collection efficiency and poor registration accuracy. A point cloud registration method for maize plants based on conical surface fitting-iterative closest point (ICP) with automatic point cloud collection platform was proposed in this paper. Firstly, a Kinect V2 was selected to cooperate with an automatic point cloud collection platform to collect multi-angle point clouds. Then, the conical surface fitting algorithm was employed to fit the point clouds of the flowerpot wall to acquire the fitted rotation axis for coarse registration. Finally, the interval ICP registration algorithm was used for precise registration, and the Delaunay triangle meshing algorithm was chosen to triangulate the point clouds of maize plants. The maize plant at the flowering and kernel stage was selected for reconstruction experiments, the results show that: the full-angle registration takes 57.32 s, and the registration mean distance error is 1.98 mm. The measured value's relative errors between the reconstructed model and the material object of maize plant are controlled within 5%, the reconstructed model can replace maize plants for research.
Collapse
Affiliation(s)
- Kai'xing Zhang
- College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian, 271018, China
| | - He Chen
- College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian, 271018, China
| | - Hao Wu
- College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian, 271018, China
| | - Xiu'yan Zhao
- College of Information Science and Engineering, Shandong Agricultural University, Taian, 271018, China
| | - Chang'an Zhou
- College of Mechanical and Electronic Engineering, Shandong Agricultural University, Taian, 271018, China.
| |
Collapse
|
7
|
Development of Stereo NIR-II Fluorescence Imaging System for 3D Tumor Vasculature in Small Animals. BIOSENSORS 2022; 12:bios12020085. [PMID: 35200345 PMCID: PMC8869613 DOI: 10.3390/bios12020085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 01/28/2022] [Accepted: 01/28/2022] [Indexed: 11/16/2022]
Abstract
Near-infrared-II (NIR-II, 1000–1700 nm) fluorescence imaging boasts high spatial resolution and deep tissue penetration due to low light scattering, reduced photon absorption, and low tissue autofluorescence. NIR-II biological imaging is applied mainly in the noninvasive visualization of blood vessels and tumors in deep tissue. In the study, a stereo NIR-II fluorescence imaging system was developed for acquiring three-dimension (3D) images on tumor vasculature in real-time, on top of the development of fluorescent semiconducting polymer dots (IR-TPE Pdots) with ultra-bright NIR-II fluorescence (1000–1400 nm) and high stability to perform long-term fluorescence imaging. The NIR-II imaging system only consists of one InGaAs camera and a moving stage to simulate left-eye view and right-eye view for the construction of 3D in-depth blood vessel images. The system was validated with blood vessel phantom of tumor-bearing mice and was applied successfully in obtaining 3D blood vessel images with 0.6 mm- and 5 mm-depth resolution and 0.15 mm spatial resolution. The NIR-II stereo vision provides precise 3D information on the tumor microenvironment and blood vessel path.
Collapse
|
8
|
Automatic leaf segmentation and overlapping leaf separation using stereo vision. ARRAY 2021. [DOI: 10.1016/j.array.2021.100099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
|
9
|
Dandrifosse S, Bouvry A, Leemans V, Dumont B, Mercatoris B. Imaging Wheat Canopy Through Stereo Vision: Overcoming the Challenges of the Laboratory to Field Transition for Morphological Features Extraction. FRONTIERS IN PLANT SCIENCE 2020; 11:96. [PMID: 32133023 PMCID: PMC7040167 DOI: 10.3389/fpls.2020.00096] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2018] [Accepted: 01/22/2020] [Indexed: 05/22/2023]
Abstract
Stereo vision is a 3D imaging method that allows quick measurement of plant architecture. Historically, the method has mainly been developed in controlled conditions. This study identified several challenges to adapt the method to natural field conditions and propose solutions. The plant traits studied were leaf area, mean leaf angle, leaf angle distribution, and canopy height. The experiment took place in a winter wheat, Triticum aestivum L., field dedicated to fertilization trials at Gembloux (Belgium). Images were acquired thanks to two nadir cameras. A machine learning algorithm using RGB and HSV color spaces is proposed to perform soil-plant segmentation robust to light conditions. The matching between images of the two cameras and the leaf area computation was improved if the number of pixels in the image of a scene was binned from 2560 × 2048 to 1280 × 1024 pixels, for a distance of 1 m between the cameras and the canopy. Height descriptors such as median or 95th percentile of plant heights were useful to precisely compare the development of different canopies. Mean spike top height was measured with an accuracy of 97.1 %. The measurement of leaf area was affected by overlaps between leaves so that a calibration curve was necessary. The leaf area estimation presented a root mean square error (RMSE) of 0.37. The impact of wind on the variability of leaf area measurement was inferior to 3% except at the stem elongation stage. Mean leaf angles ranging from 53° to 62° were computed for the whole growing season. For each acquisition date during the vegetative stages, the variability of mean angle measurement was inferior to 1.5% which underpins that the method is precise.
Collapse
Affiliation(s)
- Sébastien Dandrifosse
- Biosystems Dynamics and Exchanges, TERRA Teaching and Research Centre, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
| | - Arnaud Bouvry
- Biosystems Dynamics and Exchanges, TERRA Teaching and Research Centre, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
| | - Vincent Leemans
- Biosystems Dynamics and Exchanges, TERRA Teaching and Research Centre, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
| | - Benjamin Dumont
- Plant Sciences, TERRA Teaching and Research Centre, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
| | - Benoît Mercatoris
- Biosystems Dynamics and Exchanges, TERRA Teaching and Research Centre, Gembloux Agro-Bio Tech, University of Liège, Gembloux, Belgium
| |
Collapse
|
10
|
Sandhu J, Zhu F, Paul P, Gao T, Dhatt BK, Ge Y, Staswick P, Yu H, Walia H. PI-Plat: a high-resolution image-based 3D reconstruction method to estimate growth dynamics of rice inflorescence traits. PLANT METHODS 2019; 15:162. [PMID: 31889986 PMCID: PMC6933716 DOI: 10.1186/s13007-019-0545-2] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/02/2019] [Accepted: 12/09/2019] [Indexed: 05/03/2023]
Abstract
BACKGROUND Recent advances in image-based plant phenotyping have improved our capability to study vegetative stage growth dynamics. However, more complex agronomic traits such as inflorescence architecture (IA), which predominantly contributes to grain crop yield are more challenging to quantify and hence are relatively less explored. Previous efforts to estimate inflorescence-related traits using image-based phenotyping have been limited to destructive end-point measurements. Development of non-destructive inflorescence phenotyping platforms could accelerate the discovery of the phenotypic variation with respect to inflorescence dynamics and mapping of the underlying genes regulating critical yield components. RESULTS The major objective of this study is to evaluate post-fertilization development and growth dynamics of inflorescence at high spatial and temporal resolution in rice. For this, we developed the Panicle Imaging Platform (PI-Plat) to comprehend multi-dimensional features of IA in a non-destructive manner. We used 11 rice genotypes to capture multi-view images of primary panicle on weekly basis after the fertilization. These images were used to reconstruct a 3D point cloud of the panicle, which enabled us to extract digital traits such as voxel count and color intensity. We found that the voxel count of developing panicles is positively correlated with seed number and weight at maturity. The voxel count from developing panicles projected overall volumes that increased during the grain filling phase, wherein quantification of color intensity estimated the rate of panicle maturation. Our 3D based phenotyping solution showed superior performance compared to conventional 2D based approaches. CONCLUSIONS For harnessing the potential of the existing genetic resources, we need a comprehensive understanding of the genotype-to-phenotype relationship. Relatively low-cost sequencing platforms have facilitated high-throughput genotyping, while phenotyping, especially for complex traits, has posed major challenges for crop improvement. PI-Plat offers a low cost and high-resolution platform to phenotype inflorescence-related traits using 3D reconstruction-based approach. Further, the non-destructive nature of the platform facilitates analyses of the same panicle at multiple developmental time points, which can be utilized to explore the genetic variation for dynamic inflorescence traits in cereals.
Collapse
Affiliation(s)
- Jaspreet Sandhu
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Feiyu Zhu
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Puneet Paul
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Tian Gao
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Balpreet K. Dhatt
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Yufeng Ge
- Biological Systems Engineering Department, University of Nebraska-Lincoln, Lincoln, USA
| | - Paul Staswick
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Hongfeng Yu
- Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Harkamal Walia
- Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| |
Collapse
|
11
|
Shaaf S, Bretani G, Biswas A, Fontana IM, Rossini L. Genetics of barley tiller and leaf development. JOURNAL OF INTEGRATIVE PLANT BIOLOGY 2019; 61:226-256. [PMID: 30548413 DOI: 10.1111/jipb.12757] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/14/2018] [Accepted: 12/10/2018] [Indexed: 06/09/2023]
Abstract
In cereals, tillering and leaf development are key factors in the concept of crop ideotype, introduced in the 1960s to enhance crop yield, via manipulation of plant architecture. In the present review, we discuss advances in genetic analysis of barley shoot architecture, focusing on tillering, leaf size and angle. We also discuss novel phenotyping techniques, such as 2D and 3D imaging, that have been introduced in the era of phenomics, facilitating reliable trait measurement. We discuss the identification of genes and pathways that are involved in barley tillering and leaf development, highlighting key hormones involved in the control of plant architecture in barley and rice. Knowledge on genetic control of traits related to plant architecture provides useful resources for designing ideotypes for enhanced barley yield and performance.
Collapse
Affiliation(s)
- Salar Shaaf
- University of Milan, DiSAA, Via Celoria 2, 20133 Milan, Italy
| | | | - Abhisek Biswas
- University of Milan, DiSAA, Via Celoria 2, 20133 Milan, Italy
| | | | - Laura Rossini
- University of Milan, DiSAA, Via Celoria 2, 20133 Milan, Italy
| |
Collapse
|
12
|
Sandhu J, Zhu F, Paul P, Gao T, Dhatt BK, Ge Y, Staswick P, Yu H, Walia H. PI-Plat: a high-resolution image-based 3D reconstruction method to estimate growth dynamics of rice inflorescence traits. PLANT METHODS 2019. [PMID: 31889986 DOI: 10.1186/s13007-019-0545-542] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
BACKGROUND Recent advances in image-based plant phenotyping have improved our capability to study vegetative stage growth dynamics. However, more complex agronomic traits such as inflorescence architecture (IA), which predominantly contributes to grain crop yield are more challenging to quantify and hence are relatively less explored. Previous efforts to estimate inflorescence-related traits using image-based phenotyping have been limited to destructive end-point measurements. Development of non-destructive inflorescence phenotyping platforms could accelerate the discovery of the phenotypic variation with respect to inflorescence dynamics and mapping of the underlying genes regulating critical yield components. RESULTS The major objective of this study is to evaluate post-fertilization development and growth dynamics of inflorescence at high spatial and temporal resolution in rice. For this, we developed the Panicle Imaging Platform (PI-Plat) to comprehend multi-dimensional features of IA in a non-destructive manner. We used 11 rice genotypes to capture multi-view images of primary panicle on weekly basis after the fertilization. These images were used to reconstruct a 3D point cloud of the panicle, which enabled us to extract digital traits such as voxel count and color intensity. We found that the voxel count of developing panicles is positively correlated with seed number and weight at maturity. The voxel count from developing panicles projected overall volumes that increased during the grain filling phase, wherein quantification of color intensity estimated the rate of panicle maturation. Our 3D based phenotyping solution showed superior performance compared to conventional 2D based approaches. CONCLUSIONS For harnessing the potential of the existing genetic resources, we need a comprehensive understanding of the genotype-to-phenotype relationship. Relatively low-cost sequencing platforms have facilitated high-throughput genotyping, while phenotyping, especially for complex traits, has posed major challenges for crop improvement. PI-Plat offers a low cost and high-resolution platform to phenotype inflorescence-related traits using 3D reconstruction-based approach. Further, the non-destructive nature of the platform facilitates analyses of the same panicle at multiple developmental time points, which can be utilized to explore the genetic variation for dynamic inflorescence traits in cereals.
Collapse
Affiliation(s)
- Jaspreet Sandhu
- 1Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Feiyu Zhu
- 2Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Puneet Paul
- 1Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Tian Gao
- 2Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Balpreet K Dhatt
- 1Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Yufeng Ge
- 3Biological Systems Engineering Department, University of Nebraska-Lincoln, Lincoln, USA
| | - Paul Staswick
- 1Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| | - Hongfeng Yu
- 2Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA
| | - Harkamal Walia
- 1Department of Agronomy and Horticulture, University of Nebraska-Lincoln, Lincoln, USA
| |
Collapse
|
13
|
Li D, Cao Y, Tang XS, Yan S, Cai X. Leaf Segmentation on Dense Plant Point Clouds with Facet Region Growing. SENSORS 2018; 18:s18113625. [PMID: 30366434 PMCID: PMC6263610 DOI: 10.3390/s18113625] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/15/2018] [Revised: 10/07/2018] [Accepted: 10/22/2018] [Indexed: 11/22/2022]
Abstract
Leaves account for the largest proportion of all organ areas for most kinds of plants, and are comprise the main part of the photosynthetically active material in a plant. Observation of individual leaves can help to recognize their growth status and measure complex phenotypic traits. Current image-based leaf segmentation methods have problems due to highly restricted species and vulnerability toward canopy occlusion. In this work, we propose an individual leaf segmentation approach for dense plant point clouds using facet over-segmentation and facet region growing. The approach can be divided into three steps: (1) point cloud pre-processing, (2) facet over-segmentation, and (3) facet region growing for individual leaf segmentation. The experimental results show that the proposed method is effective and efficient in segmenting individual leaves from 3D point clouds of greenhouse ornamentals such as Epipremnum aureum, Monstera deliciosa, and Calathea makoyana, and the average precision and recall are both above 90%. The results also reveal the wide applicability of the proposed methodology for point clouds scanned from different kinds of 3D imaging systems, such as stereo vision and Kinect v2. Moreover, our method is potentially applicable in a broad range of applications that aim at segmenting regular surfaces and objects from a point cloud.
Collapse
Affiliation(s)
- Dawei Li
- College of Information Science and Technology, Donghua University, Shanghai 201620, China.
- Engineering Research Center of Digitized Textile & Fashion Technology, Ministry of Education, Donghua University, Shanghai 201620, China.
| | - Yan Cao
- College of Information Science and Technology, Donghua University, Shanghai 201620, China.
| | - Xue-Song Tang
- College of Information Science and Technology, Donghua University, Shanghai 201620, China.
- Engineering Research Center of Digitized Textile & Fashion Technology, Ministry of Education, Donghua University, Shanghai 201620, China.
| | - Siyuan Yan
- College of Information Science and Technology, Donghua University, Shanghai 201620, China.
| | - Xin Cai
- College of Information Science and Technology, Donghua University, Shanghai 201620, China.
- Engineering Research Center of Digitized Textile & Fashion Technology, Ministry of Education, Donghua University, Shanghai 201620, China.
| |
Collapse
|
14
|
Evaluating Geometric Measurement Accuracy Based on 3D Reconstruction of Automated Imagery in a Greenhouse. SENSORS 2018; 18:s18072270. [PMID: 30011865 PMCID: PMC6068631 DOI: 10.3390/s18072270] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/10/2018] [Revised: 07/08/2018] [Accepted: 07/12/2018] [Indexed: 11/17/2022]
Abstract
Geometric dimensions of plants are significant parameters for showing plant dynamic responses to environmental variations. An image-based high-throughput phenotyping platform was developed to automatically measure geometric dimensions of plants in a greenhouse. The goal of this paper was to evaluate the accuracy in geometric measurement using the Structure from Motion (SfM) method from images acquired using the automated image-based platform. Images of nine artificial objects of different shapes were taken under 17 combinations of three different overlaps in x and y directions, respectively, and two different spatial resolutions (SRs) with three replicates. Dimensions in x, y and z of these objects were measured from 3D models reconstructed using the SfM method to evaluate the geometric accuracy. A metric power of unit (POU) was proposed to combine the effects of image overlap and SR. Results showed that measurement error of dimension in z is the least affected by overlap and SR among the three dimensions and measurement error of dimensions in x and y increased following a power function with the decrease of POU (R2 = 0.78 and 0.88 for x and y respectively). POUs from 150 to 300 are a preferred range to obtain reasonable accuracy and efficiency for the developed image-based high-throughput phenotyping system. As a study case, the developed system was used to measure the height of 44 plants using an optimal POU in greenhouse environment. The results showed a good agreement (R2 = 92% and Root Mean Square Error = 9.4 mm) between the manual and automated method.
Collapse
|
15
|
Modeling and Testing of Growth Status for Chinese Cabbage and White Radish with UAV-Based RGB Imagery. REMOTE SENSING 2018. [DOI: 10.3390/rs10040563] [Citation(s) in RCA: 40] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
16
|
A High Throughput Integrated Hyperspectral Imaging and 3D Measurement System. SENSORS 2018; 18:s18041068. [PMID: 29614839 PMCID: PMC5948655 DOI: 10.3390/s18041068] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/30/2018] [Revised: 03/12/2018] [Accepted: 03/26/2018] [Indexed: 11/16/2022]
Abstract
Hyperspectral and three-dimensional measurements can obtain the intrinsic physicochemical properties and external geometrical characteristics of objects, respectively. The combination of these two kinds of data can provide new insights into objects, which has gained attention in the fields of agricultural management, plant phenotyping, cultural heritage conservation, and food production. Currently, a variety of sensors are integrated into a system to collect spectral and morphological information in agriculture. However, previous experiments were usually performed with several commercial devices on a single platform. Inadequate registration and synchronization among instruments often resulted in mismatch between spectral and 3D information of the same target. In addition, using slit-based spectrometers and point-based 3D sensors extends the working hours in farms due to the narrow field of view (FOV). Therefore, we propose a high throughput prototype that combines stereo vision and grating dispersion to simultaneously acquire hyperspectral and 3D information. Furthermore, fiber-reformatting imaging spectrometry (FRIS) is adopted to acquire the hyperspectral images. Test experiments are conducted for the verification of the system accuracy, and vegetation measurements are carried out to demonstrate its feasibility. The proposed system is an improvement in multiple data acquisition and has the potential to improve plant phenotyping.
Collapse
|
17
|
|