EasyIDP: A Python Package for Intermediate Data Processing in UAV-Based Plant Phenotyping

High-Throughput Yield Prediction of Diallele Crossed Sugar Beet in a Breeding Field Using UAV-Derived Growth Dynamics

Data-driven techniques could be used to enhance decision-making capacity of breeders and farmers. We used an RGB camera on an unmanned aerial vehicle (UAV) to collect time series data on sugar beet canopy coverage (CC) and canopy height (CH) from small-plot breeding fields including 20 genotypes per season over 3 seasons. Digital orthomosaic and digital surface models were created from each flight and were converted to individual plot-level data. Plot-level data including CC and CH were calculated on a per-plot basis. A multiple regression model was fitted, which predicts root weight (RW) (r = 0.89, 0.89, and 0.92 in the 3 seasons, respectively) and sugar content (SC) (r = 0.79, 0.83, and 0.77 in the 3 seasons, respectively) using individual time point CC and CH data. Individual CC and CH values in late June tended to be strong predictors of RW and SC, suggesting that early season growth is critical for obtaining high RW and SC. Coefficient of parentage was not a strong factor influencing SC. Integrals of CC and CH time series data were calculated for genetic analysis purposes since they are more stable over multiple growing seasons. Calculations of general combining ability and specific combining ability in F1 offspring demonstrate how growth curve quantification can be used in diallel cross analysis and yield prediction. Our simple yet robust solution demonstrates how state-of-the-art remote sensing tools and basic analysis methods can be applied to small-plot breeder fields for selection purpose.

Simulation of Automatically Annotated Visible and Multi-/Hyperspectral Images Using the Helios 3D Plant and Radiative Transfer Modeling Framework

Deep learning and multimodal remote and proximal sensing are widely used for analyzing plant and crop traits, but many of these deep learning models are supervised and necessitate reference datasets with image annotations. Acquiring these datasets often demands experiments that are both labor-intensive and time-consuming. Furthermore, extracting traits from remote sensing data beyond simple geometric features remains a challenge. To address these challenges, we proposed a radiative transfer modeling framework based on the Helios 3-dimensional (3D) plant modeling software designed for plant remote and proximal sensing image simulation. The framework has the capability to simulate RGB, multi-/hyperspectral, thermal, and depth cameras, and produce associated plant images with fully resolved reference labels such as plant physical traits, leaf chemical concentrations, and leaf physiological traits. Helios offers a simulated environment that enables generation of 3D geometric models of plants and soil with random variation, and specification or simulation of their properties and function. This approach differs from traditional computer graphics rendering by explicitly modeling radiation transfer physics, which provides a critical link to underlying plant biophysical processes. Results indicate that the framework is capable of generating high-quality, labeled synthetic plant images under given lighting scenarios, which can lessen or remove the need for manually collected and annotated data. Two example applications are presented that demonstrate the feasibility of using the model to enable unsupervised learning by training deep learning models exclusively with simulated images and performing prediction tasks using real images.

UAV-based individual Chinese cabbage weight prediction using multi-temporal data

The use of unmanned aerial vehicles (UAVs) has facilitated crop canopy monitoring, enabling yield prediction by integrating regression models. However, the application of UAV-based data to individual-level harvest weight prediction is limited by the effectiveness of obtaining individual features. In this study, we propose a method that automatically detects and extracts multitemporal individual plant features derived from UAV-based data to predict harvest weight. We acquired data from an experimental field sown with 1196 Chinese cabbage plants, using two cameras (RGB and multi-spectral) mounted on UAVs. First, we used three RGB orthomosaic images and an object detection algorithm to detect more than 95% of the individual plants. Next, we used feature selection methods and five different multi-temporal resolutions to predict individual plant weights, achieving a coefficient of determination (R2) of 0.86 and a root mean square error (RMSE) of 436 g/plant. Furthermore, we achieved predictions with an R2 greater than 0.72 and an RMSE less than 560 g/plant up to 53 days prior to harvest. These results demonstrate the feasibility of accurately predicting individual Chinese cabbage harvest weight using UAV-based data and the efficacy of utilizing multi-temporal features to predict plant weight more than one month prior to harvest.

Drone-Based Harvest Data Prediction Can Reduce On-Farm Food Loss and Improve Farmer Income

On-farm food loss (i.e., grade-out vegetables) is a difficult challenge in sustainable agricultural systems. The simplest method to reduce the number of grade-out vegetables is to monitor and predict the size of all individuals in the vegetable field and determine the optimal harvest date with the smallest grade-out number and highest profit, which is not cost-effective by conventional methods. Here, we developed a full pipeline to accurately estimate and predict every broccoli head size (n > 3,000) automatically and nondestructively using drone remote sensing and image analysis. The individual sizes were fed to the temperature-based growth model and predicted the optimal harvesting date. Two years of field experiments revealed that our pipeline successfully estimated and predicted the head size of all broccolis with high accuracy. We also found that a deviation of only 1 to 2 days from the optimal date can considerably increase grade-out and reduce farmer's profits. This is an unequivocal demonstration of the utility of these approaches to economic crop optimization and minimization of food losses.

Performance and Accuracy Comparisons of Classification Methods and Perspective Solutions for UAV-Based Near-Real-Time "Out of the Lab" Data Processing

Today, integration into automated systems has become a priority in the development of remote sensing sensors carried on drones. For this purpose, the primary task is to achieve real-time data processing. Increasing sensor resolution, fast data capture and the simultaneous use of multiple sensors is one direction of development. However, this poses challenges on the data processing side due to the increasing amount of data. Our study intends to investigate how the running time and accuracy of commonly used image classification algorithms evolve using Altum Micasense multispectral and thermal acquisition data with GSD = 2 cm spatial resolution. The running times were examined for two PC configurations, with a 4 GB and 8 GB DRAM capacity, respectively, as these parameters are closer to the memory of NRT microcomputers and laptops, which can be applied "out of the lab". During the accuracy assessment, we compared the accuracy %, the Kappa index value and the area ratio of correct pixels. According to our results, in the case of plant cover, the Spectral Angles Mapper (SAM) method achieved the best accuracy among the validated classification solutions. In contrast, the Minimum Distance (MD) method achieved the best accuracy on water surface. In terms of temporality, the best results were obtained with the individually constructed decision tree classification. Thus, it is worth developing these two directions into real-time data processing solutions.

Development of a high-throughput field phenotyping rover optimized for size-limited breeding fields as open-source hardware

Phenotyping is a critical process in plant breeding, especially when there is an increasing demand for streamlining a selection process in a breeding program. Since manual phenotyping has limited efficiency, high-throughput phenotyping methods are recently popularized owing to progress in sensor and image processing technologies. However, in a size-limited breeding field, which is common in Japan and other Asian countries, it is challenging to introduce large machinery in the field or fly unmanned aerial vehicles over the field. In this study, we developed a ground-based high-throughput field phenotyping rover that could be easily introduced to a field regardless of the scale and location of the field even without special facilities. We also made the field rover open-source hardware, making its system available to public for easy modification, so that anyone can build one for their own use at a low cost. The trial run of the field rover revealed that it allowed the collection of detailed remote-sensing images of plants and quantitative analyses based on the images. The results suggest that the field rover developed in this study could allow efficient phenotyping of plants especially in a small breeding field.

High-throughput field crop phenotyping: current status and challenges

In contrast to the rapid advances made in plant genotyping, plant phenotyping is considered a bottleneck in plant science. This has promoted high-throughput plant phenotyping (HTP) studies, resulting in an exponential increase in phenotyping-related publications. The development of HTP was originally intended for use as indoor HTP technologies for model plant species under controlled environments. However, this subsequently shifted to HTP for use in crops in fields. Although HTP in fields is much more difficult to conduct due to unstable environmental conditions compared to HTP in controlled environments, recent advances in HTP technology have allowed these difficulties to be overcome, allowing for rapid, efficient, non-destructive, non-invasive, quantitative, repeatable, and objective phenotyping. Recent HTP developments have been accelerated by the advances in data analysis, sensors, and robot technologies, including machine learning, image analysis, three dimensional (3D) reconstruction, image sensors, laser sensors, environmental sensors, and drones, along with high-speed computational resources. This article provides an overview of recent HTP technologies, focusing mainly on canopy-based phenotypes of major crops, such as canopy height, canopy coverage, canopy biomass, and canopy stressed appearance, in addition to crop organ detection and counting in the fields. Current topics in field HTP are also presented, followed by a discussion on the low rates of adoption of HTP in practical breeding programs.

A miniaturized phenotyping platform for individual plants using multi-view stereo 3D reconstruction

Plant phenotyping is essential in plant breeding and management. High-throughput data acquisition and automatic phenotypes extraction are common concerns in plant phenotyping. Despite the development of phenotyping platforms and the realization of high-throughput three-dimensional (3D) data acquisition in tall plants, such as maize, handling small-size plants with complex structural features remains a challenge. This study developed a miniaturized shoot phenotyping platform MVS-Pheno V2 focusing on low plant shoots. The platform is an improvement of MVS-Pheno V1 and was developed based on multi-view stereo 3D reconstruction. It has the following four components: Hardware, wireless communication and control, data acquisition system, and data processing system. The hardware sets the rotation on top of the platform, separating plants to be static while rotating. A novel local network was established to realize wireless communication and control; thus, preventing cable twining. The data processing system was developed to calibrate point clouds and extract phenotypes, including plant height, leaf area, projected area, shoot volume, and compactness. This study used three cultivars of wheat shoots at four growth stages to test the performance of the platform. The mean absolute percentage error of point cloud calibration was 0.585%. The squared correlation coefficient R ² was 0.9991, 0.9949, and 0.9693 for plant height, leaf length, and leaf width, respectively. The root mean squared error (RMSE) was 0.6996, 0.4531, and 0.1174 cm for plant height, leaf length, and leaf width. The MVS-Pheno V2 platform provides an alternative solution for high-throughput phenotyping of low individual plants and is especially suitable for shoot architecture-related plant breeding and management studies.

An Efficient Method for Estimating Wheat Heading Dates Using UAV Images

Convenient, efficient, and high-throughput estimation of wheat heading dates is of great significance in plant sciences and agricultural research. However, documenting heading dates is time-consuming, labor-intensive, and subjective on a large-scale field. To overcome these challenges, model- and image-based approaches are used to estimate heading dates. Phenology models usually require complicated parameters calibrations, making it difficult to model other varieties and different locations, while in situ field-image recognition usually requires the deployment of a large amount of observational equipment, which is expensive. Therefore, in this study, we proposed a growth curve-based method for estimating wheat heading dates. The method first generates a height-based continuous growth curve based on five time-series unmanned aerial vehicle (UAV) images captured over the entire wheat growth cycle (>200 d). Then estimate the heading date by generated growth curve. As a result, the proposed method had a mean absolute error of 2.81 d and a root mean square error of 3.49 d for 72 wheat plots composed of different varieties and densities sown on different dates. Thus, the proposed method is straightforward, efficient, and affordable and meets the high-throughput estimation requirements of large-scale fields and underdeveloped areas.

Global Wheat Head Detection 2021: An Improved Dataset for Benchmarking Wheat Head Detection Methods

The Global Wheat Head Detection (GWHD) dataset was created in 2020 and has assembled 193,634 labelled wheat heads from 4700 RGB images acquired from various acquisition platforms and 7 countries/institutions. With an associated competition hosted in Kaggle, GWHD_2020 has successfully attracted attention from both the computer vision and agricultural science communities. From this first experience, a few avenues for improvements have been identified regarding data size, head diversity, and label reliability. To address these issues, the 2020 dataset has been reexamined, relabeled, and complemented by adding 1722 images from 5 additional countries, allowing for 81,553 additional wheat heads. We now release in 2021 a new version of the Global Wheat Head Detection dataset, which is bigger, more diverse, and less noisy than the GWHD_2020 version.