Individual Tree Crown Segmentation Directly from UAV-Borne LiDAR Data Using the PointNet of Deep Learning

Deep Ordinal Classification in Forest Areas Using Light Detection and Ranging Point Clouds

Recent advances in Deep Learning and aerial Light Detection And Ranging (LiDAR) have offered the possibility of refining the classification and segmentation of 3D point clouds to contribute to the monitoring of complex environments. In this context, the present study focuses on developing an ordinal classification model in forest areas where LiDAR point clouds can be classified into four distinct ordinal classes: ground, low vegetation, medium vegetation, and high vegetation. To do so, an effective soft labeling technique based on a novel proposed generalized exponential function (CE-GE) is applied to the PointNet network architecture. Statistical analyses based on Kolmogorov-Smirnov and Student's t-test reveal that the CE-GE method achieves the best results for all the evaluation metrics compared to other methodologies. Regarding the confusion matrices of the best alternative conceived and the standard categorical cross-entropy method, the smoothed ordinal classification obtains a more consistent classification compared to the nominal approach. Thus, the proposed methodology significantly improves the point-by-point classification of PointNet, reducing the errors in distinguishing between the middle classes (low vegetation and medium vegetation).

Three-Dimensional Point Cloud Segmentation Algorithm Based on Depth Camera for Large Size Model Point Cloud Unsupervised Class Segmentation

This paper proposes a 3D point cloud segmentation algorithm based on a depth camera for large-scale model point cloud unsupervised class segmentation. The algorithm utilizes depth information obtained from a depth camera and a voxelization technique to reduce the size of the point cloud, and then uses clustering methods to segment the voxels based on their density and distance to the camera. Experimental results show that the proposed algorithm achieves high segmentation accuracy and fast segmentation speed on various large-scale model point clouds. Compared with recent similar works, the algorithm demonstrates superior performance in terms of accuracy metrics, with an average Intersection over Union (IoU) of 90.2% on our own benchmark dataset.

Development of a multispectral fluorescence LiDAR for point cloud segmentation of plants

The accelerating development of high-throughput plant phenotyping demands a LiDAR system to achieve spectral point cloud, which will significantly improve the accuracy and efficiency of segmentation based on its intrinsic fusion of spectral and spatial data. Meanwhile, a relatively longer detection range is required for platforms e.g., unmanned aerial vehicles (UAV) and poles. Towards the aims above, what we believe to be, a novel multispectral fluorescence LiDAR, featuring compact volume, light weight, and low cost, has been proposed and designed. A 405 nm laser diode was employed to excite the fluorescence of plants, and the point cloud attached with both the elastic and inelastic signal intensities that was obtained through the R-, G-, B-channels of a color image sensor. A new position retrieval method has been developed to evaluate far field echo signals, from which the spectral point cloud can be obtained. Experiments were designed to validate the spectral/spatial accuracy and the segmentation performance. It has been found out that the values obtained through the R-, G-, B-channels are consistent with the emission spectrum measured by a spectrometer, achieving a maximum R² of 0.97. The theoretical spatial resolution can reach up to 47 mm and 0.7 mm in the x- and y-direction at a distance of around 30 m, respectively. The values of recall, precision, and F score for the segmentation of the fluorescence point cloud were all beyond 0.97. Besides, a field test has been carried out on plants at a distance of about 26 m, which further demonstrated that the multispectral fluorescence data can significantly facilitate the segmentation process in a complex scene. These promising results prove that the proposed multispectral fluorescence LiDAR has great potential in applications of digital forestry inventory and intelligent agriculture.

Individual Tree Crown Segmentation and Crown Width Extraction From a Heightmap Derived From Aerial Laser Scanning Data Using a Deep Learning Framework

Deriving individual tree crown (ITC) information from light detection and ranging (LiDAR) data is of great significance to forest resource assessment and smart management. After proof-of-concept studies, advanced deep learning methods have been shown to have high efficiency and accuracy in remote sensing data analysis and geoscience problem solving. This study proposes a novel concept for synergetic use of the YOLO-v4 deep learning network based on heightmaps directly generated from airborne LiDAR data for ITC segmentation and a computer graphics algorithm for refinement of the segmentation results involving overlapping tree crowns. This concept overcomes the limitations experienced by existing ITC segmentation methods that use aerial photographs to obtain texture and crown appearance information and commonly encounter interference due to heterogeneous solar illumination intensities or interlacing branches and leaves. Three generative adversarial networks (WGAN, CycleGAN, and SinGAN) were employed to generate synthetic images. These images were coupled with manually labeled training samples to train the network. Three forest plots, namely, a tree nursery, forest landscape and mixed tree plantation, were used to verify the effectiveness of our approach. The results showed that the overall recall of our method for detecting ITCs in the three forest plot types reached 83.6%, with an overall precision of 81.4%. Compared with reference field measurement data, the coefficient of determination R 2 was ≥ 79.93% for tree crown width estimation, and the accuracy of our deep learning method was not influenced by the values of key parameters, yielding 3.9% greater accuracy than the traditional watershed method. The results demonstrate an enhancement of tree crown segmentation in the form of a heightmap for different forest plot types using the concept of deep learning, and our method bypasses the visual complications arising from aerial images featuring diverse textures and unordered scanned points with irregular geometrical properties.

Restoration of Individual Tree Missing Point Cloud Based on Local Features of Point Cloud

LiDAR (Light Detection And Ranging) technology is an important means to obtain three-dimensional information of trees and vegetation. However, due to the influence of scanning mode, environmental occlusion and mutual occlusion between tree canopies and other factors, a tree point cloud often has different degrees of data loss, which affects the high-precision quantitative extraction of vegetation parameters. Aiming at the problem of a tree laser point cloud being missing, an individual tree incomplete point cloud restoration method based on local features of the point cloud is proposed. The L1-Median algorithm is used to extract key points of the tree skeleton, then the dominant direction of skeleton key points and local point cloud density are calculated, and the point cloud near the missing area is moved based on these features to gradually complete the incomplete point cloud compensation. The experimental results show that the above repair method can effectively repair the incomplete point cloud with good robustness and can adapt to the individual tree point cloud with different geometric structures and correct the branch topological connection errors.

Design and Testing of a Novel Unoccupied Aircraft System for the Collection of Forest Canopy Samples

Unoccupied Aircraft Systems (UAS) are beginning to replace conventional forest plot mensuration through their use as low-cost and powerful remote sensing tools for monitoring growth, estimating biomass, evaluating carbon stocks and detecting weeds; however, physical samples remain mostly collected through time-consuming, expensive and potentially dangerous conventional techniques. Such conventional techniques include the use of arborists to climb the trees to retrieve samples, shooting branches with firearms from the ground, canopy cranes or the use of pole-mounted saws to access lower branches. UAS hold much potential to improve the safety, efficiency, and reduce the cost of acquiring canopy samples. In this work, we describe and demonstrate four iterations of 3D printed canopy sampling UAS. This work includes detailed explanations of designs and how each iteration informed the design decisions in the subsequent iteration. The fourth iteration of the aircraft was tested for the collection of 30 canopy samples from three tree species: eucalyptus pulchella, eucalyptus globulus and acacia dealbata trees. The collection times ranged from 1 min and 23 s, up to 3 min and 41 s for more distant and challenging to capture samples. A vision for the next iteration of this design is also provided. Future work may explore the integration of advanced remote sensing techniques with UAS-based canopy sampling to progress towards a fully-automated and holistic forest information capture system.

Evidence That Supertriangles Exist in Nature from the Vertical Projections of Koelreuteria paniculata Fruit

Many natural radial symmetrical shapes (e.g., sea stars) follow the Gielis equation (GE) or its twin equation (TGE). A supertriangle (three triangles arranged around a central polygon) represents such a shape, but no study has tested whether natural shapes can be represented as/are supertriangles or whether the GE or TGE can describe their shape. We collected 100 pieces of Koelreuteria paniculata fruit, which have a supertriangular shape, extracted the boundary coordinates for their vertical projections, and then fitted them with the GE and TGE. The adjusted root mean square errors (RMSEadj) of the two equations were always less than 0.08, and >70% were less than 0.05. For 57/100 fruit projections, the GE had a lower RMSEadj than the TGE, although overall differences in the goodness of fit were non-significant. However, the TGE produces more symmetrical shapes than the GE as the two parameters controlling the extent of symmetry in it are approximately equal. This work demonstrates that natural supertriangles exist, validates the use of the GE and TGE to model their shapes, and suggests that different complex radially symmetrical shapes can be generated by the same equation, implying that different types of biological symmetry may result from the same biophysical mechanisms.

Unimodal and Multimodal Perception for Forest Management: Review and Dataset

Robotics navigation and perception for forest management are challenging due to the existence of many obstacles to detect and avoid and the sharp illumination changes. Advanced perception systems are needed because they can enable the development of robotic and machinery solutions to accomplish a smarter, more precise, and sustainable forestry. This article presents a state-of-the-art review about unimodal and multimodal perception in forests, detailing the current developed work about perception using a single type of sensors (unimodal) and by combining data from different kinds of sensors (multimodal). This work also makes a comparison between existing perception datasets in the literature and presents a new multimodal dataset, composed by images and laser scanning data, as a contribution for this research field. Lastly, a critical analysis of the works collected is conducted by identifying strengths and research trends in this domain.

GIS-Based Forest Fire Risk Model: A Case Study in Laoshan National Forest Park, Nanjing

Fire risk prediction is significant for fire prevention and fire resource allocation. Fire risk maps are effective methods for quantifying regional fire risk. Laoshan National Forest Park has many precious natural resources and tourist attractions, but there is no fire risk assessment model. This paper aims to construct the forest fire risk map for Nanjing Laoshan National Forest Park. The forest fire risk model is constructed by factors (altitude, aspect, topographic wetness index, slope, distance to roads and populated areas, normalized difference vegetation index, and temperature) which have a great influence on the probability of inducing fire in Laoshan. Since the importance of factors in different study areas is inconsistent, it is necessary to calculate the significance of each factor of Laoshan. After the significance calculation is completed, the fire risk model of Laoshan can be obtained. Then, the fire risk map can be plotted based on the model. This fire risk map can clarify the fire risk level of each part of the study area, with 16.97% extremely low risk, 48.32% low risk, 17.35% moderate risk, 12.74% high risk and 4.62% extremely high risk, and it is compared with the data of MODIS fire anomaly point. The result shows that the accuracy of the risk map is 76.65%.

Estimation of Northern Hardwood Forest Inventory Attributes Using UAV Laser Scanning (ULS): Transferability of Laser Scanning Methods and Comparison of Automated Approaches at the Tree- and Stand-Level

UAV laser scanning (ULS) has the potential to support forest operations since it provides high-density data with flexible operational conditions. This study examined the use of ULS systems to estimate several tree attributes from an uneven-aged northern hardwood stand. We investigated: (1) the transferability of raster-based and bottom-up point cloud-based individual tree detection (ITD) algorithms to ULS data; and (2) automated approaches to the retrieval of tree-level (i.e., height, crown diameter (CD), DBH) and stand-level (i.e., tree count, basal area (BA), DBH-distribution) forest inventory attributes. These objectives were studied under leaf-on and leaf-off canopy conditions. Results achieved from ULS data were cross-compared with ALS and TLS to better understand the potential and challenges faced by different laser scanning systems and methodological approaches in hardwood forest environments. The best results that characterized individual trees from ULS data were achieved under leaf-off conditions using a point cloud-based bottom-up ITD. The latter outperformed the raster-based ITD, improving the accuracy of tree detection (from 50% to 71%), crown delineation (from R2 = 0.29 to R2 = 0.61), and prediction of tree DBH (from R2 = 0.36 to R2 = 0.67), when compared with values that were estimated from reference TLS data. Major improvements were observed for the detection of trees in the lower canopy layer (from 9% with raster-based ITD to 51% with point cloud-based ITD) and in the intermediate canopy layer (from 24% with raster-based ITD to 59% with point cloud-based ITD). Under leaf-on conditions, LiDAR data from aerial systems include substantial signal occlusion incurred by the upper canopy. Under these conditions, the raster-based ITD was unable to detect low-level canopy trees (from 5% to 15% of trees detected from lower and intermediate canopy layers, respectively), resulting in a tree detection rate of about 40% for both ULS and ALS data. The cylinder-fitting method used to estimate tree DBH under leaf-off conditions did not meet inventory standards when compared to TLS DBH, resulting in RMSE = 7.4 cm, Bias = 3.1 cm, and R2 = 0.75. Yet, it yielded more accurate estimates of the BA (+3.5%) and DBH-distribution of the stand than did allometric models −12.9%), when compared with in situ field measurements. Results suggest that the use of bottom-up ITD on high-density ULS data from leaf-off hardwood forest leads to promising results when estimating trees and stand attributes, which opens up new possibilities for supporting forest inventories and operations.

Combining Multiple Geospatial Data for Estimating Aboveground Biomass in North Carolina Forests

Mapping and quantifying forest inventories are critical for the management and development of forests for natural resource conservation and for the evaluation of the aboveground forest biomass (AGFB) technically available for bioenergy production. The AGFB estimation procedures that rely on traditional, spatially sparse field inventory samples constitute a problem for geographically diverse regions such as the state of North Carolina in the southeastern U.S. We propose an alternative AGFB estimation procedure that combines multiple geospatial data. The procedure uses land cover maps to allocate forested land areas to alternative forest types; uses the light detection and ranging (LiDAR) data to evaluate tree heights; calculates the area-total AGFB using region- and tree-type-specific functions that relate the tree heights to the AGFB. We demonstrate the procedure for a selected North Carolina region, a 2.3 km2 area randomly chosen in Duplin County. The tree diameter functions are statistically estimated based on the Forest Inventory Analysis (FIA) data, and two publicly available, open source land cover maps, Crop Data Layer (CDL) and National Land Cover Database (NLCD), are compared and contrasted as a source of information on the location and typology of forests in the study area. The assessment of the consistency of forestland mapping derived from the CDL and the NLCD data lets us estimate how the disagreement between the two alternative, widely used maps affects the AGFB estimation. The methodology and the results we present are expected to complement and inform large-scale assessments of woody biomass in the region.

Benchmarking Anchor-Based and Anchor-Free State-of-the-Art Deep Learning Methods for Individual Tree Detection in RGB High-Resolution Images

Urban forests contribute to maintaining livability and increase the resilience of cities in the face of population growth and climate change. Information about the geographical distribution of individual trees is essential for the proper management of these systems. RGB high-resolution aerial images have emerged as a cheap and efficient source of data, although detecting and mapping single trees in an urban environment is a challenging task. Thus, we propose the evaluation of novel methods for single tree crown detection, as most of these methods have not been investigated in remote sensing applications. A total of 21 methods were investigated, including anchor-based (one and two-stage) and anchor-free state-of-the-art deep-learning methods. We used two orthoimages divided into 220 non-overlapping patches of 512 × 512 pixels with a ground sample distance (GSD) of 10 cm. The orthoimages were manually annotated, and 3382 single tree crowns were identified as the ground-truth. Our findings show that the anchor-free detectors achieved the best average performance with an AP50 of 0.686. We observed that the two-stage anchor-based and anchor-free methods showed better performance for this task, emphasizing the FSAF, Double Heads, CARAFE, ATSS, and FoveaBox models. RetinaNet, which is currently commonly applied in remote sensing, did not show satisfactory performance, and Faster R-CNN had lower results than the best methods but with no statistically significant difference. Our findings contribute to a better understanding of the performance of novel deep-learning methods in remote sensing applications and could be used as an indicator of the most suitable methods in such applications.

LiDAR Applications to Estimate Forest Biomass at Individual Tree Scale: Opportunities, Challenges and Future Perspectives

Accurate forest biomass estimation at the individual tree scale is the foundation of timber industry and forest management. It plays an important role in explaining ecological issues and small-scale processes. Remotely sensed images, across a range of spatial and temporal resolutions, with their advantages of non-destructive monitoring, are widely applied in forest biomass monitoring at global, ecoregion or community scales. However, the development of remote sensing applications for forest biomass at the individual tree scale has been relatively slow due to the constraints of spatial resolution and evaluation accuracy of remotely sensed data. With the improvements in platforms and spatial resolutions, as well as the development of remote sensing techniques, the potential for forest biomass estimation at the single tree level has been demonstrated. However, a comprehensive review of remote sensing of forest biomass scaled at individual trees has not been done. This review highlights the theoretical bases, challenges and future perspectives for Light Detection and Ranging (LiDAR) applications of individual trees scaled to whole forests. We summarize research on estimating individual tree volume and aboveground biomass (AGB) using Terrestrial Laser Scanning (TLS), Airborne Laser Scanning (ALS), Unmanned Aerial Vehicle Laser Scanning (UAV-LS) and Mobile Laser Scanning (MLS, including Vehicle-borne Laser Scanning (VLS) and Backpack Laser Scanning (BLS)) data.

A Review: Point Cloud-Based 3D Human Joints Estimation

Joint estimation of the human body is suitable for many fields such as human-computer interaction, autonomous driving, video analysis and virtual reality. Although many depth-based researches have been classified and generalized in previous review or survey papers, the point cloud-based pose estimation of human body is still difficult due to the disorder and rotation invariance of the point cloud. In this review, we summarize the recent development on the point cloud-based pose estimation of the human body. The existing works are divided into three categories based on their working principles, including template-based method, feature-based method and machine learning-based method. Especially, the significant works are highlighted with a detailed introduction to analyze their characteristics and limitations. The widely used datasets in the field are summarized, and quantitative comparisons are provided for the representative methods. Moreover, this review helps further understand the pertinent applications in many frontier research directions. Finally, we conclude the challenges involved and problems to be solved in future researches.

Plant Age Has a Minor Effect on Non-Destructive Leaf Area Calculations in Moso Bamboo (Phyllostachys edulis)

Leaf area is among the most important leaf functional traits, and it determines leaf temperature and alters light harvesting. The calculation of individual leaf area is the basis of calculating the leaf area index (i.e., the total leaf area per unit ground area) that is directly associated with the ability of plants to intercept light for photosynthesis. It is valuable to provide a fast and reliable approach to measuring leaf area. Here, we examined the validity and calculation accuracy of the Montgomery equation (ME), which describes the area of a leaf as a product of leaf length, width and a specific coefficient referred to as the Montgomery parameter, MP. Using ME, we calculated leaf areas of different age groups of bamboo culms. For most broad-leaved plants, leaf area is proportional to the product of leaf length and width, and MP falls within a range of 1/2 to π/4, depending on leaf shape. However, it is unknown whether there is an intra-specific variation in MP resulting from age structure and whether such a variation can significantly reduce the predictability of ME in calculating leaf area. This is relevant as a population of perennial plants usually composes of different age groups. We used Moso bamboos as model as this species is of ecological and economic importance in southern China, and pure stands can cover six to seven plant age groups. We used five age groups of moso bamboo and sampled 260–380 leaves for each group to test whether ME holds true for each group and all groups combined, whether there are significant differences in MP among different age groups, and whether the differences in MP can lead to large prediction errors for leaf area. We observed that for each age group and all groups combined, there were significant proportional relationships between leaf area and the product of leaf length and width. There were small but significant differences in MP among the five age groups (MP values ranged from 0.6738 to 0.7116 for individual plant ages; MP = 0.6936 for all age groups combined), which can be accounted for by the minor intergroup variation of leaf shape (reflected by the leaf width/length ratio). For all age classes, MP estimated for the pooled data resulted in <4% mean absolute percentage error, indicating that the effect of variation in MP among different age groups was small. We conclude that ME can serve as a useful tool for accurate calculations of leaf area in moso bamboo independent of culm age, which is valuable for estimation of leaf area index as well as evaluating the productivity and carbon sequestration capacity of bamboo forests.