Non-destructive estimation of field maize biomass using terrestrial lidar: an evaluation from plot level to individual leaf level

Field-measured canopy height may not be as accurate and heritable as believed: evidence from advanced 3D sensing

Canopy height (CH) is an important trait for crop breeding and production. The rapid development of 3D sensing technologies shed new light on high-throughput height measurement. However, a systematic comparison of the accuracy and heritability of different 3D sensing technologies is seriously lacking. Moreover, it is questionable whether the field-measured height is as reliable as believed. This study uncovered these issues by comparing traditional height measurement with four advanced 3D sensing technologies, including terrestrial laser scanning (TLS), backpack laser scanning (BLS), gantry laser scanning (GLS), and digital aerial photogrammetry (DAP). A total of 1920 plots covering 120 varieties were selected for comparison. Cross-comparisons of different data sources were performed to evaluate their performances in CH estimation concerning different CH, leaf area index (LAI), and growth stage (GS) groups. Results showed that 1) All 3D sensing data sources had high correlations with field measurement (r > 0.82), while the correlations between different 3D sensing data sources were even better (r > 0.87). 2) The prediction accuracy between different data sources decreased in subgroups of CH, LAI, and GS. 3) Canopy height showed high heritability from all datasets, and 3D sensing datasets had even higher heritability (H² = 0.79-0.89) than FM (field measurement) (H² = 0.77). Finally, outliers of different datasets are analyzed. The results provide novel insights into different methods for canopy height measurement that may ensure the high-quality application of this important trait.

Applications of LiDAR in Agriculture and Future Research Directions

Light detection and ranging (LiDAR) sensors have accrued an ever-increasing presence in the agricultural sector due to their non-destructive mode of capturing data. LiDAR sensors emit pulsed light waves that return to the sensor upon bouncing off surrounding objects. The distances that the pulses travel are calculated by measuring the time for all pulses to return to the source. There are many reported applications of the data obtained from LiDAR in agricultural sectors. LiDAR sensors are widely used to measure agricultural landscaping and topography and the structural characteristics of trees such as leaf area index and canopy volume; they are also used for crop biomass estimation, phenotype characterisation, crop growth, etc. A LiDAR-based system and LiDAR data can also be used to measure spray drift and detect soil properties. It has also been proposed in the literature that crop damage detection and yield prediction can also be obtained with LiDAR data. This review focuses on different LiDAR-based system applications and data obtained from LiDAR in agricultural sectors. Comparisons of aspects of LiDAR data in different agricultural applications are also provided. Furthermore, future research directions based on this emerging technology are also presented in this review.

The 30-year impact of post-windthrow management on the forest regeneration process in northern Japan

The frequency and intensity of typhoons are expected to increase over time due to climate change. These changes may expose forests to more windthrow in the future, and increasing the resilience of hemiboreal forests through forest management after windthrow is important. Here, we quantified forest structure recovery using aerial photos and light detection and ranging (LiDAR) data after catastrophic windthrow events. Our aims are to test the following three hypotheses: (1) forest structure will not recover within 30 years after windthrow, (2) forest recovery will be affected not only by salvaging but also pre-windthrow attributes and geographical features, and (3) various post-windthrow management including salvaging will drastically alter tree species composition and delay forest recovery. Our results revealed that hypothesis (1) and (2) were supported and (3) was partially supported. The ordination results suggested that more than 30 years were needed to recover canopy tree height after windthrow in hemiboreal forests in Hokkaido, Japan. Salvage logging did not delay natural succession, but it significantly decreased the cover ratio of conifer species sites (0.107 ± 0.023) compared with natural succession sites (0.310 ± 0.091). The higher the elevation, the steeper the site, and the higher the average canopy height before windthrow, the slower the recovery of forest stands after windthrow and salvaging. Scarification and planting after salvage logging significantly increased the number of canopy trees, but those sites differed completely in species composition from the old growth forests. Our study thus determined that the choice and intensity of post-disturbance management in hemiboreal forests should be carefully considered based on the management purpose and local characteristics.

Enhancing crop yields through improvements in the efficiency of photosynthesis and respiration

The rate with which crop yields per hectare increase each year is plateauing at the same time that human population growth and other factors increase food demand. Increasing yield potential (Y p ) of crops is vital to address these challenges. In this review, we explore a component ofY p that has yet to be optimised - that being improvements in the efficiency with which light energy is converted into biomass (ε c ) via modifications to CO2 fixed per unit quantum of light (α), efficiency of respiratory ATP production (ε prod ) and efficiency of ATP use (ε use ). For α, targets include changes in photoprotective machinery, ribulose bisphosphate carboxylase/oxygenase kinetics and photorespiratory pathways. There is also potential forε prod to be increased via targeted changes to the expression of the alternative oxidase and mitochondrial uncoupling pathways. Similarly, there are possibilities to improveε use via changes to the ATP costs of phloem loading, nutrient uptake, futile cycles and/or protein/membrane turnover. Recently developed high-throughput measurements of respiration can serve as a proxy for the cumulative energy cost of these processes. There are thus exciting opportunities to use our growing knowledge of factors influencing the efficiency of photosynthesis and respiration to create a step-change in yield potential of globally important crops.

Proximal and remote sensing in plant phenomics: 20 years of progress, challenges, and perspectives

Plant phenomics (PP) has been recognized as a bottleneck in studying the interactions of genomics and environment on plants, limiting the progress of smart breeding and precise cultivation. High-throughput plant phenotyping is challenging owing to the spatio-temporal dynamics of traits. Proximal and remote sensing (PRS) techniques are increasingly used for plant phenotyping because of their advantages in multi-dimensional data acquisition and analysis. Substantial progress of PRS applications in PP has been observed over the last two decades and is analyzed here from an interdisciplinary perspective based on 2972 publications. This progress covers most aspects of PRS application in PP, including patterns of global spatial distribution and temporal dynamics, specific PRS technologies, phenotypic research fields, working environments, species, and traits. Subsequently, we demonstrate how to link PRS to multi-omics studies, including how to achieve multi-dimensional PRS data acquisition and processing, how to systematically integrate all kinds of phenotypic information and derive phenotypic knowledge with biological significance, and how to link PP to multi-omics association analysis. Finally, we identify three future perspectives for PRS-based PP: (1) strengthening the spatial and temporal consistency of PRS data, (2) exploring novel phenotypic traits, and (3) facilitating multi-omics communication.

Estimation of Greenhouse Lettuce Growth Indices Based on a Two-Stage CNN Using RGB-D Images

Growth indices can quantify crop productivity and establish optimal environmental, nutritional, and irrigation control strategies. A convolutional neural network (CNN)-based model is presented for estimating various growth indices (i.e., fresh weight, dry weight, height, leaf area, and diameter) of four varieties of greenhouse lettuce using red, green, blue, and depth (RGB-D) data obtained using a stereo camera. Data from an online autonomous greenhouse challenge (Wageningen University, June 2021) were employed in this study. The data were collected using an Intel RealSense D415 camera. The developed model has a two-stage CNN architecture based on ResNet50V2 layers. The developed model provided coefficients of determination from 0.88 to 0.95, with normalized root mean square errors of 6.09%, 6.30%, 7.65%, 7.92%, and 5.62% for fresh weight, dry weight, height, diameter, and leaf area, respectively, on unknown lettuce images. Using red, green, blue (RGB) and depth data employed in the CNN improved the determination accuracy for all five lettuce growth indices due to the ability of the stereo camera to extract height information on lettuce. The average time for processing each lettuce image using the developed CNN model run on a Jetson SUB mini-PC with a Jetson Xavier NX was 0.83 s, indicating the potential for the model in fast real-time sensing of lettuce growth indices.

Non-destructive Plant Biomass Monitoring With High Spatio-Temporal Resolution via Proximal RGB-D Imagery and End-to-End Deep Learning

Plant breeders, scientists, and commercial producers commonly use growth rate as an integrated signal of crop productivity and stress. Plant growth monitoring is often done destructively via growth rate estimation by harvesting plants at different growth stages and simply weighing each individual plant. Within plant breeding and research applications, and more recently in commercial applications, non-destructive growth monitoring is done using computer vision to segment plants in images from the background, either in 2D or 3D, and relating these image-based features to destructive biomass measurements. Recent advancements in machine learning have improved image-based localization and detection of plants, but such techniques are not well suited to make biomass predictions when there is significant self-occlusion or occlusion from neighboring plants, such as those encountered under leafy green production in controlled environment agriculture. To enable prediction of plant biomass under occluded growing conditions, we develop an end-to-end deep learning approach that directly predicts lettuce plant biomass from color and depth image data as provided by a low cost and commercially available sensor. We test the performance of the proposed deep neural network for lettuce production, observing a mean prediction error of 7.3% on a comprehensive test dataset of 864 individuals and substantially outperforming previous work on plant biomass estimation. The modeling approach is robust to the busy and occluded scenes often found in commercial leafy green production and requires only measured mass values for training. We then demonstrate that this level of prediction accuracy allows for rapid, non-destructive detection of changes in biomass accumulation due to experimentally induced stress induction in as little as 2 days. Using this method growers may observe and react to changes in plant-environment interactions in near real time. Moreover, we expect that such a sensitive technique for non-destructive biomass estimation will enable novel research and breeding of improved productivity and yield in response to stress.

Simultaneous Prediction of Wheat Yield and Grain Protein Content Using Multitask Deep Learning from Time-Series Proximal Sensing

Wheat yield and grain protein content (GPC) are two main optimization targets for breeding and cultivation. Remote sensing provides nondestructive and early predictions of yield and GPC, respectively. However, whether it is possible to simultaneously predict yield and GPC in one model and the accuracy and influencing factors are still unclear. In this study, we made a systematic comparison of different deep learning models in terms of data fusion, time-series feature extraction, and multitask learning. The results showed that time-series data fusion significantly improved yield and GPC prediction accuracy with R² values of 0.817 and 0.809. Multitask learning achieved simultaneous prediction of yield and GPC with comparable accuracy to the single-task model. We further proposed a two-to-two model that combines data fusion (two kinds of data sources for input) and multitask learning (two outputs) and compared different feature extraction layers, including RNN (recurrent neural network), LSTM (long short-term memory), CNN (convolutional neural network), and attention module. The two-to-two model with the attention module achieved the best prediction accuracy for yield (R² = 0.833) and GPC (R² = 0.846). The temporal distribution of feature importance was visualized based on the attention feature values. Although the temporal patterns of structural traits and spectral traits were inconsistent, the overall importance of both structural traits and spectral traits at the postanthesis stage was more important than that at the preanthesis stage. This study provides new insights into the simultaneous prediction of yield and GPC using deep learning from time-series proximal sensing, which may contribute to the accurate and efficient predictions of agricultural production.

Effects of UAV-LiDAR and Photogrammetric Point Density on Tea Plucking Area Identification

High-cost data collection and processing are challenges for UAV LiDAR (light detection and ranging) mounted on unmanned aerial vehicles in crop monitoring. Reducing the point density can lower data collection costs and increase efficiency but may lead to a loss in mapping accuracy. It is necessary to determine the appropriate point cloud density for tea plucking area identification to maximize the cost–benefits. This study evaluated the performance of different LiDAR and photogrammetric point density data when mapping the tea plucking area in the Huashan Tea Garden, Wuhan City, China. The object-based metrics derived from UAV point clouds were used to classify tea plantations with the extreme learning machine (ELM) and random forest (RF) algorithms. The results indicated that the performance of different LiDAR point density data, from 0.25 (1%) to 25.44 pts/m2 (100%), changed obviously (overall classification accuracies: 90.65–94.39% for RF and 89.78–93.44% for ELM). For photogrammetric data, the point density was found to have little effect on the classification accuracy, with 10% of the initial point density (2.46 pts/m2), a similar accuracy level was obtained (difference of approximately 1%). LiDAR point cloud density had a significant influence on the DTM accuracy, with the RMSE for DTMs ranging from 0.060 to 2.253 m, while the photogrammetric point cloud density had a limited effect on the DTM accuracy, with the RMSE ranging from 0.256 to 0.477 m due to the high proportion of ground points in the photogrammetric point clouds. Moreover, important features for identifying the tea plucking area were summarized for the first time using a recursive feature elimination method and a novel hierarchical clustering-correlation method. The resultant architecture diagram can indicate the specific role of each feature/group in identifying the tea plucking area and could be used in other studies to prepare candidate features. This study demonstrates that low UAV point density data, such as 2.55 pts/m2 (10%), as used in this study, might be suitable for conducting finer-scale tea plucking area mapping without compromising the accuracy.

Above-Ground Biomass Estimation of Plantation with Complex Forest Stand Structure Using Multiple Features from Airborne Laser Scanning Point Cloud Data

Accurate forest above-ground biomass (AGB) estimation is important for dynamic monitoring of forest resources and evaluation of forest carbon sequestration capacity. However, it is difficult to depict the forest’s vertical structure and its heterogeneity using optical remote sensing when estimating forest AGB, for the reason that electromagnetic waves cannot penetrate the canopy’s surface to obtain low vegetation information, especially in subtropical and tropical forests with complex layer structure and tree species composition. As an active remote sensing technology, an airborne laser scanner (ALS) can penetrate the canopy surface to obtain three-dimensional structure information related to AGB. This paper takes the Jiepai sub-forest farm and the Gaofeng state-owned forest farm in southern China as the experimental area and explores the optimal features from the ALS point cloud data and AGB inversion model in the subtropical forest with complex tree species composition and structure. Firstly, considering tree canopy structure, terrain features, point cloud structure and density features, 63 point cloud features were extracted. In view of the biomass distribution differences of different tree species, the random forest (RF) method was used to select the optimal features of each tree species. Secondly, four modeling methods were used to establish the AGB estimation models of each tree species and verify their accuracy. The results showed that the features related to tree height had a great impact on forest AGB. The top features of Cunninghamia Lanceolata (Chinese fir) and Eucalyptus are all related to height, Pinus (pine tree) is also related to terrain features and other broadleaved trees are also related to point cloud density features. The accuracy of the stepwise regression model is best with the AGB estimation accuracy of 0.19, 0.76, 0.71 and 0.40, respectively, for the Chinese fir, pine tree, eucalyptus and other broadleaved trees. In conclusion, the proposed linear regression AGB estimation model of each tree species combining different features derived from ALS point cloud data has high applicability, which can provide effective support for more accurate forest AGB and carbon stock inventory and monitoring.

Exploring Seasonal and Circadian Rhythms in Structural Traits of Field Maize from LiDAR Time Series

Plant growth rhythm in structural traits is important for better understanding plant response to the ever-changing environment. Terrestrial laser scanning (TLS) is a well-suited tool to study structural rhythm under field conditions. Recent studies have used TLS to describe the structural rhythm of trees, but no consistent patterns have been drawn. Meanwhile, whether TLS can capture structural rhythm in crops is unclear. Here, we aim to explore the seasonal and circadian rhythms in maize structural traits at both the plant and leaf levels from time-series TLS. The seasonal rhythm was studied using TLS data collected at four key growth periods, including jointing, bell-mouthed, heading, and maturity periods. Circadian rhythms were explored by using TLS data acquired around every 2 hours in a whole day under standard and cold stress conditions. Results showed that TLS can quantify the seasonal and circadian rhythm in structural traits at both plant and leaf levels. (1) Leaf inclination angle decreased significantly between the jointing stage and bell-mouthed stage. Leaf azimuth was stable after the jointing stage. (2) Some individual-level structural rhythms (e.g., azimuth and projected leaf area/PLA) were consistent with leaf-level structural rhythms. (3) The circadian rhythms of some traits (e.g., PLA) were not consistent under standard and cold stress conditions. (4) Environmental factors showed better correlations with leaf traits under cold stress than standard conditions. Temperature was the most important factor that significantly correlated with all leaf traits except leaf azimuth. This study highlights the potential of time-series TLS in studying outdoor agricultural chronobiology.

Mapping Potato Plant Density Variation Using Aerial Imagery and Deep Learning Techniques for Precision Agriculture

In potato (Solanum tuberosum) production, the number of tubers harvested and their sizes are related to the plant population. Field maps of the spatial variation in plant density can therefore provide a decision support tool for spatially variable harvest timing to optimize tuber sizes by allowing densely populated management zones more tuber-bulking time. Computer vision has been proposed to enumerate plant numbers using images from unmanned aerial vehicles (UAV) but inaccurate predictions in images of merged canopies remains a challenge. Some research has been done on individual potato plant bounding box prediction but there is currently no information on the spatial structure of plant density that these models may reveal and its relationship with potato yield quality attributes. In this study, the Faster Region-based Convolutional Neural Network (FRCNN) framework was used to produce a plant detection model and estimate plant densities across a UAV orthomosaic. Using aerial images of 2 mm ground sampling distance (GSD) collected from potatoes at 40 days after planting, the FRCNN model was trained to an average precision (aP) of 0.78 on unseen testing data. The model was then used to generate predictions on quadrants imposed on orthorectified rasters captured at 14 and 18 days after emergence. After spatially interpolating the plant densities, the resultant surfaces were highly correlated to manually-determined plant density (R2 = 0.80). Further correlations were observed with tuber number (r = 0.54 at Butter Hill; r = 0.53 at Horse Foxhole), marketable tuber weight per plant (r = −0.57 at Buttery Hill; r = −0.56 at Horse Foxhole) and the normalized difference vegetation index (r = 0.61). These results show that accurate two-dimensional maps of plant density can be constructed from UAV imagery with high correlation to important yield components, despite the loss of accuracy of FRCNN models in partially merged canopies.

EasyIDP: A Python Package for Intermediate Data Processing in UAV-Based Plant Phenotyping

Unmanned aerial vehicle (UAV) and structure from motion (SfM) photogrammetry techniques are widely used for field-based, high-throughput plant phenotyping nowadays, but some of the intermediate processes throughout the workflow remain manual. For example, geographic information system (GIS) software is used to manually assess the 2D/3D field reconstruction quality and cropping region of interests (ROIs) from the whole field. In addition, extracting phenotypic traits from raw UAV images is more competitive than directly from the digital orthomosaic (DOM). Currently, no easy-to-use tools are available to implement previous tasks for commonly used commercial SfM software, such as Pix4D and Agisoft Metashape. Hence, an open source software package called easy intermediate data processor (EasyIDP; MIT license) was developed to decrease the workload in intermediate data processing mentioned above. The functions of the proposed package include (1) an ROI cropping module, assisting in reconstruction quality assessment and cropping ROIs from the whole field, and (2) an ROI reversing module, projecting ROIs to relative raw images. The result showed that both cropping and reversing modules work as expected. Moreover, the effects of ROI height selection and reversed ROI position on raw images to reverse calculation were discussed. This tool shows great potential for decreasing workload in data annotation for machine learning applications.

Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots

Background

The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking.

Results

We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation.

Conclusion

Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.