Understanding Growth Dynamics and Yield Prediction of Sorghum Using High Temporal Resolution UAV Imagery Time Series and Machine Learning

Improving the estimation of rice above-ground biomass based on spatio-temporal UAV imagery and phenological stages

Introduction

Unmanned aerial vehicles (UAVs) equipped with visible and multispectral cameras provide reliable and efficient methods for remote crop monitoring and above-ground biomass (AGB) estimation in rice fields. However, existing research predominantly focuses on AGB estimation based on canopy spectral features or by incorporating plant height (PH) as a parameter. Insufficient consideration has been given to the spatial structure and the phenological stages of rice in these studies. In this study, a novel method was introduced by fully considering the three-dimensional growth dynamics of rice, integrating both horizontal (canopy cover, CC) and vertical (PH) aspects of canopy development, and accounting for the growing days of rice.

Methods

To investigate the synergistic effects of combining spectral, spatial and temporal parameters, both small-scale plot experiments and large-scale field testing were conducted in Jiangsu Province, China from 2021 to 2022. Twenty vegetation indices (VIs) were used as spectral features, PH and CC as spatial parameters, and days after transplanting (DAT) as a temporal parameter. AGB estimation models were built with five regression methods (MSR, ENet, PLSR, RF and SVR), using the derived data from six feature combinations (VIs, PH+CC, PH+CC+DAT, VIs+PH +CC, VIs+DAT, VIs+PH+CC+DAT).

Results

The results showed a strong correlation between extracted and ground-measured PH (R2 = 0.89, RMSE=5.08 cm). Furthermore, VIs, PH and CC exhibit strong correlations with AGB during the mid-tillering to flowering stages. The optimal AGB estimation results during the mid-tillering to flowering stages on plot data were from the PLSR model with VIs and DAT as inputs (R 2 = 0.88, RMSE=1111kg/ha, NRMSE=9.76%), and with VIs, PH, CC, and DAT all as inputs (R 2 = 0.88, RMSE=1131 kg/ha, NRMSE=9.94%). For the field sampling data, the ENet model combined with different feature inputs had the best estimation results (%error=0.6%-13.5%), demonstrating excellent practical applicability.

Discussion

Model evaluation and feature importance ranking demonstrated that augmenting VIs with temporal and spatial parameters significantly enhanced the AGB estimation accuracy. In summary, the fusion of spectral and spatio-temporal features enhanced the actual physical significance of the AGB estimation models and showed great potential for accurate rice AGB estimation during the main phenological stages.

Yield prediction in a peanut breeding program using remote sensing data and machine learning algorithms

Peanut is a critical food crop worldwide, and the development of high-throughput phenotyping techniques is essential for enhancing the crop's genetic gain rate. Given the obvious challenges of directly estimating peanut yields through remote sensing, an approach that utilizes above-ground phenotypes to estimate underground yield is necessary. To that end, this study leveraged unmanned aerial vehicles (UAVs) for high-throughput phenotyping of surface traits in peanut. Using a diverse set of peanut germplasm planted in 2021 and 2022, UAV flight missions were repeatedly conducted to capture image data that were used to construct high-resolution multitemporal sigmoidal growth curves based on apparent characteristics, such as canopy cover and canopy height. Latent phenotypes extracted from these growth curves and their first derivatives informed the development of advanced machine learning models, specifically random forest and eXtreme Gradient Boosting (XGBoost), to estimate yield in the peanut plots. The random forest model exhibited exceptional predictive accuracy (R2 = 0.93), while XGBoost was also reasonably effective (R2 = 0.88). When using confusion matrices to evaluate the classification abilities of each model, the two models proved valuable in a breeding pipeline, particularly for filtering out underperforming genotypes. In addition, the random forest model excelled in identifying top-performing material while minimizing Type I and Type II errors. Overall, these findings underscore the potential of machine learning models, especially random forests and XGBoost, in predicting peanut yield and improving the efficiency of peanut breeding programs.

A transformer-based approach for early prediction of soybean yield using time-series images

Crop yield prediction which provides critical information for management decision-making is of significant importance in precision agriculture. Traditional manual inspection and calculation are often laborious and time-consuming. For yield prediction using high-resolution images, existing methods, e.g., convolutional neural network, are challenging to model long range multi-level dependencies across image regions. This paper proposes a transformer-based approach for yield prediction using early-stage images and seed information. First, each original image is segmented into plant and soil categories. Two vision transformer (ViT) modules are designed to extract features from each category. Then a transformer module is established to deal with the time-series features. Finally, the image features and seed features are combined to estimate the yield. A case study has been conducted using a dataset that was collected during the 2020 soybean-growing seasons in Canadian fields. Compared with other baseline models, the proposed method can reduce the prediction error by more than 40%. The impact of seed information on predictions is studied both between models and within a single model. The results show that the influence of seed information varies among different plots but it is particularly important for the prediction of low yields.

Machine learning classification of plant genotypes grown under different light conditions through the integration of multi-scale time-series data

In order to mitigate the effects of a changing climate, agriculture requires more effective evaluation, selection, and production of crop cultivars in order to accelerate genotype-to-phenotype connections and the selection of beneficial traits. Critically, plant growth and development are highly dependent on sunlight, with light energy providing plants with the energy required to photosynthesize as well as a means to directly intersect with the environment in order to develop. In plant analyses, machine learning and deep learning techniques have a proven ability to learn plant growth patterns, including detection of disease, plant stress, and growth using a variety of image data. To date, however, studies have not assessed machine learning and deep learning algorithms for their ability to differentiate a large cohort of genotypes grown under several growth conditions using time-series data automatically acquired across multiple scales (daily and developmentally). Here, we extensively evaluate a wide range of machine learning and deep learning algorithms for their ability to differentiate 17 well-characterized photoreceptor deficient genotypes differing in their light detection capabilities grown under several different light conditions. Using algorithm performance measurements of precision, recall, F1-Score, and accuracy, we find that Suport Vector Machine (SVM) maintains the greatest classification accuracy, while a combined ConvLSTM2D deep learning model produces the best genotype classification results across the different growth conditions. Our successful integration of time-series growth data across multiple scales, genotypes and growth conditions sets a new foundational baseline from which more complex plant science traits can be assessed for genotype-to-phenotype connections.

Advancing designer crops for climate resilience through an integrated genomics approach

Climate change and exponential population growth are exposing an immediate need for developing future crops that are highly resilient and adaptable to changing environments to maintain global food security in the next decade. Rigorous selection from long domestication history has rendered cultivated crops genetically disadvantaged, raising concerns in their ability to adapt to these new challenges and limiting their usefulness in breeding programmes. As a result, future crop improvement efforts must rely on integrating various genomic strategies ranging from high-throughput sequencing to machine learning, in order to exploit germplasm diversity and overcome bottlenecks created by domestication, expansive multi-dimensional phenotypes, arduous breeding processes, complex traits and big data.

Installation and imaging of thousands of minirhizotrons to phenotype root systems of field-grown plants

BACKGROUND

Roots are vital to plant performance because they acquire resources from the soil and provide anchorage. However, it remains difficult to assess root system size and distribution because roots are inaccessible in the soil. Existing methods to phenotype entire root systems range from slow, often destructive, methods applied to relatively small numbers of plants in the field to rapid methods that can be applied to large numbers of plants in controlled environment conditions. Much has been learned recently by extensive sampling of the root crown portion of field-grown plants. But, information on large-scale genetic and environmental variation in the size and distribution of root systems in the field remains a key knowledge gap. Minirhizotrons are the only established, non-destructive technology that can address this need in a standard field trial. Prior experiments have used only modest numbers of minirhizotrons, which has limited testing to small numbers of genotypes or environmental conditions. This study addressed the need for methods to install and collect images from thousands of minirhizotrons and thereby help break the phenotyping bottleneck in the field.

RESULTS

Over three growing seasons, methods were developed and refined to install and collect images from up to 3038 minirhizotrons per experiment. Modifications were made to four tractors and hydraulic soil corers mounted to them. High quality installation was achieved at an average rate of up to 84.4 minirhizotron tubes per tractor per day. A set of four commercially available minirhizotron camera systems were each transported by wheelbarrow to allow collection of images of mature maize root systems at an average rate of up to 65.3 tubes per day per camera. This resulted in over 300,000 images being collected in as little as 11 days for a single experiment.

CONCLUSION

The scale of minirhizotron installation was increased by two orders of magnitude by simultaneously using four tractor-mounted, hydraulic soil corers with modifications to ensure high quality, rapid operation. Image collection can be achieved at the corresponding scale using commercially available minirhizotron camera systems. Along with recent advances in image analysis, these advances will allow use of minirhizotrons at unprecedented scale to address key knowledge gaps regarding genetic and environmental effects on root system size and distribution in the field.

Estimation of Above-Ground Biomass of Winter Wheat Based on Consumer-Grade Multi-Spectral UAV

One of the problems of optical remote sensing of crop above-ground biomass (AGB) is that vegetation indices (VIs) often saturate from the middle to late growth stages. This study focuses on combining VIs acquired by a consumer-grade multiple-spectral UAV and machine learning regression techniques to (i) determine the optimal time window for AGB estimation of winter wheat and to (ii) determine the optimal combination of multi-spectral VIs and regression algorithms. UAV-based multi-spectral data and manually measured AGB of winter wheat, under five nitrogen rates, were obtained from the jointing stage until 25 days after flowering in the growing season 2020/2021. Forty-four multi-spectral VIs were used in the linear regression (LR), partial least squares regression (PLSR), and random forest (RF) models in this study. Results of LR models showed that the heading stage was the most suitable stage for AGB prediction, with R2 values varying from 0.48 to 0.93. Three PLSR models based on different datasets performed differently in estimating AGB in the training dataset (R2 = 0.74~0.92, RMSE = 0.95~2.87 t/ha, MAE = 0.75~2.18 t/ha, and RPD = 2.00~3.67) and validation dataset (R2 = 0.50~0.75, RMSE = 1.56~2.57 t/ha, MAE = 1.44~2.05 t/ha, RPD = 1.45~1.89). Compared with PLSR models, the performance of the RF models was more stable in the prediction of AGB in the training dataset (R2 = 0.95~0.97, RMSE = 0.58~1.08 t/ha, MAE = 0.46~0.89 t/ha, and RPD = 3.95~6.35) and validation dataset (R2 = 0.83~0.93, RMSE = 0.93~2.34 t/ha, MAE = 0.72~2.01 t/ha, RPD = 1.36~3.79). Monitoring AGB prior to flowering was found to be more effective than post-flowering. Moreover, this study demonstrates that it is feasible to estimate AGB for multiple growth stages of winter wheat by combining the optimal VIs and PLSR and RF models, which overcomes the saturation problem of using individual VI-based linear regression models.

Implementing Spatio-Temporal 3D-Convolution Neural Networks and UAV Time Series Imagery to Better Predict Lodging Damage in Sorghum

Unmanned aerial vehicle (UAV)-based remote sensing is gaining momentum in a variety of agricultural and environmental applications. Very-high-resolution remote sensing image sets collected repeatedly throughout a crop growing season are becoming increasingly common. Analytical methods able to learn from both spatial and time dimensions of the data may allow for an improved estimation of crop traits, as well as the effects of genetics and the environment on these traits. Multispectral and geometric time series imagery was collected by UAV on 11 dates, along with ground-truth data, in a field trial of 866 genetically diverse biomass sorghum accessions. We compared the performance of Convolution Neural Network (CNN) architectures that used image data from single dates (two spatial dimensions, 2D) versus multiple dates (two spatial dimensions + temporal dimension, 3D) to estimate lodging detection and severity. Lodging was detected with 3D-CNN analysis of time series imagery with 0.88 accuracy, 0.92 precision, and 0.83 recall. This outperformed the best 2D-CNN on a single date with 0.85 accuracy, 0.84 precision, and 0.76 recall. The variation in lodging severity was estimated by the best 3D-CNN analysis with 9.4% mean absolute error (MAE), 11.9% root mean square error (RMSE), and goodness-of-fit (R2) of 0.76. This was a significant improvement over the best 2D-CNN analysis with 11.84% MAE, 14.91% RMSE, and 0.63 R2. The success of the improved 3D-CNN analysis approach depended on the inclusion of “before and after” data, i.e., images collected on dates before and after the lodging event. The integration of geometric and spectral features with 3D-CNN architecture was also key to the improved assessment of lodging severity, which is an important and difficult-to-assess phenomenon in bioenergy feedstocks such as biomass sorghum. This demonstrates that spatio-temporal CNN architectures based on UAV time series imagery have significant potential to enhance plant phenotyping capabilities in crop breeding and precision agriculture applications.

Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

Assessing crop production in the field often requires breeders to wait until the end of the season to collect yield-related measurements, limiting the pace of the breeding cycle. Early prediction of crop performance can reduce this constraint by allowing breeders more time to focus on the highest-performing varieties. Here, we present a multimodal deep learning model for predicting the performance of maize (Zea mays) at an early developmental stage, offering the potential to accelerate crop breeding. We employed multispectral images and eight vegetation indices, collected by an uncrewed aerial vehicle approximately 60 days after sowing, over three consecutive growing cycles (2017, 2018 and 2019). The multimodal deep learning approach was used to integrate field management and genotype information with the multispectral data, providing context to the conditions that the plants experienced during the trial. Model performance was assessed using holdout data, in which the model accurately predicted the yield (RMSE 1.07 t/ha, a relative RMSE of 7.60% of 16 t/ha, and R2 score 0.73) and identified the majority of high-yielding varieties, outperforming previously published models for early yield prediction. The inclusion of vegetation indices was important for model performance, with a normalized difference vegetation index and green with normalized difference vegetation index contributing the most to model performance. The model provides a decision support tool, identifying promising lines early in the field trial.

Comparison of UAS-Based Structure-from-Motion and LiDAR for Structural Characterization of Short Broadacre Crops

The use of small unmanned aerial system (UAS)-based structure-from-motion (SfM; photogrammetry) and LiDAR point clouds has been widely discussed in the remote sensing community. Here, we compared multiple aspects of the SfM and the LiDAR point clouds, collected concurrently in five UAS flights experimental fields of a short crop (snap bean), in order to explore how well the SfM approach performs compared with LiDAR for crop phenotyping. The main methods include calculating the cloud-to-mesh distance (C2M) maps between the preprocessed point clouds, as well as computing a multiscale model-to-model cloud comparison (M3C2) distance maps between the derived digital elevation models (DEMs) and crop height models (CHMs). We also evaluated the crop height and the row width from the CHMs and compared them with field measurements for one of the data sets. Both SfM and LiDAR point clouds achieved an average RMSE of ~0.02 m for crop height and an average RMSE of ~0.05 m for row width. The qualitative and quantitative analyses provided proof that the SfM approach is comparable to LiDAR under the same UAS flight settings. However, its altimetric accuracy largely relied on the number and distribution of the ground control points.

Remote and Proximal Assessment of Plant Traits

The inference of functional vegetation traits from remotely sensed signals is key to providing efficient information for multiple plant-based applications and to solve related problems [...]