UAV-Based High Resolution Thermal Imaging for Vegetation Monitoring, and Plant Phenotyping Using ICI 8640 P, FLIR Vue Pro R 640, and thermoMap Cameras

Spatio-temporal modeling of high-throughput multispectral aerial images improves agronomic trait genomic prediction in hybrid maize

Design randomizations and spatial corrections have increased understanding of genotypic, spatial, and residual effects in field experiments, but precisely measuring spatial heterogeneity in the field remains a challenge. To this end, our study evaluated approaches to improve spatial modeling using high-throughput phenotypes (HTP) via unoccupied aerial vehicle (UAV) imagery. The normalized difference vegetation index was measured by a multispectral MicaSense camera and processed using ImageBreed. Contrasting to baseline agronomic trait spatial correction and a baseline multitrait model, a two-stage approach was proposed. Using longitudinal normalized difference vegetation index data, plot level permanent environment effects estimated spatial patterns in the field throughout the growing season. Normalized difference vegetation index permanent environment were separated from additive genetic effects using 2D spline, separable autoregressive models, or random regression models. The Permanent environment were leveraged within agronomic trait genomic best linear unbiased prediction either modeling an empirical covariance for random effects, or by modeling fixed effects as an average of permanent environment across time or split among three growth phases. Modeling approaches were tested using simulation data and Genomes-to-Fields hybrid maize (Zea mays L.) field experiments in 2015, 2017, 2019, and 2020 for grain yield, grain moisture, and ear height. The two-stage approach improved heritability, model fit, and genotypic effect estimation compared to baseline models. Electrical conductance and elevation from a 2019 soil survey significantly improved model fit, while 2D spline permanent environment were most strongly correlated with the soil parameters. Simulation of field effects demonstrated improved specificity for random regression models. In summary, the use of longitudinal normalized difference vegetation index measurements increased experimental accuracy and understanding of field spatio-temporal heterogeneity.

Using Infrared Thermography for High-Throughput Plant Phenotyping

Infrared thermography offers a rapid, noninvasive method for measuring plant temperature, which provides a proxy for stomatal conductance and plant water status and can therefore be used as an index for plant stress. Thermal imaging can provide an efficient method for high-throughput screening of large numbers of plants. This chapter provides guidelines for using thermal imaging equipment and illustrative methodologies, coupled with essential considerations, to access plant physiological processes.

Thermal imaging: The digital eye facilitates high-throughput phenotyping traits of plant growth and stress responses

Plant phenotyping is important for plants to cope with environmental changes and ensure plant health. Imaging techniques are perceived as the most critical and reliable tools for studying plant phenotypes. Thermal imaging has opened up new opportunities for nondestructive imaging of plant phenotyping. However, a comprehensive summary of thermal imaging in plant phenotyping is still lacking. Here we discuss the progress and future prospects of thermal imaging for assessing plant growth and stress responses. First, we classify thermal imaging into ground-based and aerial platforms based on their adaptability to different experimental environments (including laboratory, greenhouse, and field). It is convenient to collect phenotypic information of different dimensions. Second, in order to enhance the efficiency of thermal image processing, automatic algorithms based on deep learning are employed instead of traditional manual methods, greatly reducing the time cost of experiments. Considering its ease of implementation, handling and instant response, thermal imaging has been widely used in research on environmental stress, crop yield, and seed vigor. We have found that thermal imaging can detect thermal energy dissipation caused by living organisms (e.g., pests, viruses, bacteria, fungi, and oomycetes), enabling early disease diagnosis. It also recognizes changes leaf surface temperatures resulting from reduced transpiration rates caused by nutrient deficiency, drought, salinity, or freezing. Furthermore, thermal imaging predicts crop yield under different water states and forecasts the viability of dormant seeds after water absorption by monitoring temperature changes in the seeds. This work will assist biologists and agronomists in studying plant phenotypes and serve a guide for breeders to develop high-yielding, stress-tolerant, and superior crops.

Field phenotyping for African crops: overview and perspectives

Improvements in crop productivity are required to meet the dietary demands of the rapidly-increasing African population. The development of key staple crop cultivars that are high-yielding and resilient to biotic and abiotic stresses is essential. To contribute to this objective, high-throughput plant phenotyping approaches are important enablers for the African plant science community to measure complex quantitative phenotypes and to establish the genetic basis of agriculturally relevant traits. These advances will facilitate the screening of germplasm for optimum performance and adaptation to low-input agriculture and resource-constrained environments. Increasing the capacity to investigate plant function and structure through non-invasive technologies is an effective strategy to aid plant breeding and additionally may contribute to precision agriculture. However, despite the significant global advances in basic knowledge and sensor technology for plant phenotyping, Africa still lags behind in the development and implementation of these systems due to several practical, financial, geographical and political barriers. Currently, field phenotyping is mostly carried out by manual methods that are prone to error, costly, labor-intensive and may come with adverse economic implications. Therefore, improvements in advanced field phenotyping capabilities and appropriate implementation are key factors for success in modern breeding and agricultural monitoring. In this review, we provide an overview of the current state of field phenotyping and the challenges limiting its implementation in some African countries. We suggest that the lack of appropriate field phenotyping infrastructures is impeding the development of improved crop cultivars and will have a detrimental impact on the agricultural sector and on food security. We highlight the prospects for integrating emerging and advanced low-cost phenotyping technologies into breeding protocols and characterizing crop responses to environmental challenges in field experimentation. Finally, we explore strategies for overcoming the barriers and maximizing the full potential of emerging field phenotyping technologies in African agriculture. This review paper will open new windows and provide new perspectives for breeders and the entire plant science community in Africa.

Remotely Sensed Phenotypic Traits for Heritability Estimates and Grain Yield Prediction of Barley Using Multispectral Imaging from UAVs

This study tested the potential of parametric and nonparametric regression modeling utilizing multispectral data from two different unoccupied aerial vehicles (UAVs) as a tool for the prediction of and indirect selection of grain yield (GY) in barley breeding experiments. The coefficient of determination (R²) of the nonparametric models for GY prediction ranged between 0.33 and 0.61 depending on the UAV and flight date, where the highest value was achieved with the DJI Phantom 4 Multispectral (P4M) image from 26 May (milk ripening). The parametric models performed worse than the nonparametric ones for GY prediction. Independent of the retrieval method and UAV, GY retrieval was more accurate in milk ripening than dough ripening. The leaf area index (LAI), fraction of absorbed photosynthetically active radiation (fAPAR), fraction vegetation cover (fCover), and leaf chlorophyll content (LCC) were modeled at milk ripening using nonparametric models with the P4M images. A significant effect of the genotype was found for the estimated biophysical variables, which was referred to as remotely sensed phenotypic traits (RSPTs). Measured GY heritability was lower, with a few exceptions, compared to the RSPTs, indicating that GY was more environmentally influenced than the RSPTs. The moderate to strong genetic correlation of the RSPTs to GY in the present study indicated their potential utility as an indirect selection approach to identify high-yield genotypes of winter barley.

UAV-based RGB and TIR imaging for geothermal monitoring: a case study at Kestanbol geothermal field, Northwestern Turkey

Kestanbol is one of the most important geothermal fields in NW Turkey. This study conducted the first-ever surveys over a 10 ha reach of the Kestanbol geothermal field using an unmanned aerial vehicle (UAV) equipped with visible (RGB) and thermal infrared (TIR) cameras. Low-altitude flights below 40 m above the ground were operated above the Kestanbol geothermal field. Approximately 3500 RGB and TIR images were captured using the UAV. We recorded high-resolution RGB and TIR data of the Kestanbol geothermal field and applied the structure from motion (SfM) algorithm to identify the distribution of geothermal springs and seeps. The Kestanbol geothermal field was monitored to create a georeferenced RGB orthophoto, RGB 3D surface model, thermal anomaly map, and digital surface model (DSM) of the area with centimeter-level accuracy. In the TIR orthophoto, the surface temperature in the geothermal field was found to be between 15 and 75 °C. All the thermal anomalies revealed by the survey were verified by field observations. The geothermal springs and seeps were parallel to the NE-SW regional tectonic trends. The results of this study demonstrate an effective technique for monitoring and assessing geothermal water using UAV-based RGB and TIR imaging and provide an accurate basis for geothermal development projects. RGB and TIR imaging using UAVs are considered promising methods for improving the assessment of the effects of geothermal water on the environment.

Spatiotemporal Winter Wheat Water Status Assessment Improvement Using a Water Deficit Index Derived from an Unmanned Aerial System in the North China Plain

Agricultural droughts cause a great reduction in winter wheat productivity; therefore, timely and precise irrigation recommendations are needed to alleviate the impact. This study aims to assess drought stress in winter wheat with the use of an unmanned aerial system (UAS) with multispectral and thermal sensors. High-resolution Water Deficit Index (WDI) maps were derived to assess crop drought stress and evaluate winter wheat actual evapotranspiration rate (ET_a). However, the estimation of WDI needs to be improved by using more appropriate vegetation indices as a proximate of the fraction of vegetation cover. The experiments involved six irrigation levels of winter wheat in the harvest years 2019 and 2020 at Luancheng, North China Plain on seasonal and diurnal timescales. Additionally, WDI derived from several vegetation indices (VIs) were compared: near-infrared-, red edge-, and RGB-based. The WDIs derived from different VIs were highly correlated with each other and had similar performances. The WDI had a consistently high correlation to stomatal conductance during the whole season (R² between 0.63-0.99) and the correlation was the highest in the middle of the growing season. On the contrary, the correlation between WDI and leaf water potential increased as the season progressed with R² up to 0.99. Additionally, WDI and ET_a had a strong connection to soil water status with R² up to 0.93 to the fraction of transpirable soil water and 0.94 to the soil water change at 2 m depth at the hourly rate. The results indicated that WDI derived from multispectral and thermal sensors was a reliable factor in assessing the water status of the crop for irrigation scheduling.

UAV Multisensory Data Fusion and Multi-Task Deep Learning for High-Throughput Maize Phenotyping

Recent advances in unmanned aerial vehicles (UAV), mini and mobile sensors, and GeoAI (a blend of geospatial and artificial intelligence (AI) research) are the main highlights among agricultural innovations to improve crop productivity and thus secure vulnerable food systems. This study investigated the versatility of UAV-borne multisensory data fusion within a framework of multi-task deep learning for high-throughput phenotyping in maize. UAVs equipped with a set of miniaturized sensors including hyperspectral, thermal, and LiDAR were collected in an experimental corn field in Urbana, IL, USA during the growing season. A full suite of eight phenotypes was in situ measured at the end of the season for ground truth data, specifically, dry stalk biomass, cob biomass, dry grain yield, harvest index, grain nitrogen utilization efficiency (Grain NutE), grain nitrogen content, total plant nitrogen content, and grain density. After being funneled through a series of radiometric calibrations and geo-corrections, the aerial data were analytically processed in three primary approaches. First, an extended version normalized difference spectral index (NDSI) served as a simple arithmetic combination of different data modalities to explore the correlation degree with maize phenotypes. The extended NDSI analysis revealed the NIR spectra (750-1000 nm) alone in a strong relation with all of eight maize traits. Second, a fusion of vegetation indices, structural indices, and thermal index selectively handcrafted from each data modality was fed to classical machine learning regressors, Support Vector Machine (SVM) and Random Forest (RF). The prediction performance varied from phenotype to phenotype, ranging from R² = 0.34 for grain density up to R² = 0.85 for both grain nitrogen content and total plant nitrogen content. Further, a fusion of hyperspectral and LiDAR data completely exceeded limitations of single data modality, especially addressing the vegetation saturation effect occurring in optical remote sensing. Third, a multi-task deep convolutional neural network (CNN) was customized to take a raw imagery data fusion of hyperspectral, thermal, and LiDAR for multi-predictions of maize traits at a time. The multi-task deep learning performed predictions comparably, if not better in some traits, with the mono-task deep learning and machine learning regressors. Data augmentation used for the deep learning models boosted the prediction accuracy, which helps to alleviate the intrinsic limitation of a small sample size and unbalanced sample classes in remote sensing research. Theoretical and practical implications to plant breeders and crop growers were also made explicit during discussions in the studies.

Estimation of tomato water status with photochemical reflectance index and machine learning: Assessment from proximal sensors and UAV imagery

Tracking plant water status is a critical step towards the adaptive precision irrigation management of processing tomatoes, one of the most important specialty crops in California. The photochemical reflectance index (PRI) from proximal sensors and the high-resolution unmanned aerial vehicle (UAV) imagery provide an opportunity to monitor the crop water status efficiently. Based on data from an experimental tomato field with intensive aerial and plant-based measurements, we developed random forest machine learning regression models to estimate tomato stem water potential (ψ _stem), (using observations from proximal sensors and 12-band UAV imagery, respectively, along with weather data. The proximal sensor-based model estimation agreed well with the plant ψ _stem with R ² of 0.74 and mean absolute error (MAE) of 0.63 bars. The model included PRI, normalized difference vegetation index, vapor pressure deficit, and air temperature and tracked well with the seasonal dynamics of ψ _stem across different plots. A separate model, built with multiple vegetation indices (VIs) from UAV imagery and weather variables, had an R ² of 0.81 and MAE of 0.67 bars. The plant-level ψ _stem maps generated from UAV imagery closely represented the water status differences of plots under different irrigation treatments and also tracked well the temporal change among flights. PRI was found to be the most important VI in both the proximal sensor- and the UAV-based models, providing critical information on tomato plant water status. This study demonstrated that machine learning models can accurately estimate the water status by integrating PRI, other VIs, and weather data, and thus facilitate data-driven irrigation management for processing tomatoes.

Use of thermal imaging and the photochemical reflectance index (PRI) to detect wheat response to elevated CO2 and drought

The spectral-based photochemical reflectance index (PRI) and leaf surface temperature (Tleaf ) derived from thermal imaging are two indicative metrics of plant functioning. The relationship of PRI with radiation-use efficiency (RUE) and Tleaf with leaf transpiration could be leveraged to monitor crop photosynthesis and water use from space. Yet, it is unclear how such relationships will change under future high carbon dioxide concentrations ([CO2 ]) and drought. Here we established an [CO2 ] enrichment experiment in which three wheat genotypes were grown at ambient (400 ppm) and elevated (550 ppm) [CO2 ] and exposed to well-watered and drought conditions in two glasshouse rooms in two replicates. Leaf transpiration (Tr ) and latent heat flux (LE) were derived to assess evaporative cooling, and RUE was calculated from assimilation and radiation measurements on several dates along the season. Simultaneous hyperspectral and thermal images were taken at~ $\unicode{x0007E}$ 1.5 m from the plants to derive PRI and the temperature difference between the leaf and its surrounding air (∆ $\unicode{x02206}$ Tleaf-air ). We found significant PRI and RUE and∆ $\unicode{x02206}$ Tleaf-air and Tr correlations, with no significant differences among the genotypes. A PRI-RUE decoupling was observed under drought at ambient [CO2 ] but not at elevated [CO2 ], likely due to changes in photorespiration. For a LE range of 350 W m-2 , the ΔTleaf-air range was~ $\unicode{x0007E}$ 10°C at ambient [CO2 ] and only~ $\unicode{x0007E}$ 4°C at elevated [CO2 ]. Thicker leaves in plants grown at elevated [CO2 ] suggest higher leaf water content and consequently more efficient thermoregulation at high [CO2 ] conditions. In general, Tleaf was maintained closer to the ambient temperature at elevated [CO2 ], even under drought. PRI, RUE, ΔTleaf -air , and Tr decreased linearly with canopy depth, displaying a single PRI-RUE and ΔTleaf -air  Tr model through the canopy layers. Our study shows the utility of these sensing metrics in detecting wheat responses to future environmental changes.

IoT-Based Systems for Soil Nutrients Assessment in Horticulture

Soil nutrients assessment has great importance in horticulture. Implementation of an information system for horticulture faces many challenges: (i) great spatial variability within farms (e.g., hilly topography); (ii) different soil properties (e.g., different water holding capacity, different content in sand, sit, clay, and soil organic matter, different pH, and different permeability) for different cultivated plants; (iii) different soil nutrient uptake by different cultivated plants; (iv) small size of monoculture; and (v) great variety of farm components, agroecological zone, and socio-economic factors. Advances in information and communication technologies enable creation of low cost, efficient information systems that would improve resources management and increase productivity and sustainability of horticultural farms. We present an information system based on different sensing capability, Internet of Things, and mobile application for horticultural farms. An overview on different techniques and technologies for soil fertility evaluation is also presented. The results obtained in a botanical garden that simulates the diversity of environment and plant diversity of a horticultural farm are discussed considering the challenges identified in the literature and field research. The study provides a theoretical basis and technical support for the development of technologies that enable horticultural farmers to improve resources management.

Proximal and remote sensing in plant phenomics: 20 years of progress, challenges, and perspectives

Plant phenomics (PP) has been recognized as a bottleneck in studying the interactions of genomics and environment on plants, limiting the progress of smart breeding and precise cultivation. High-throughput plant phenotyping is challenging owing to the spatio-temporal dynamics of traits. Proximal and remote sensing (PRS) techniques are increasingly used for plant phenotyping because of their advantages in multi-dimensional data acquisition and analysis. Substantial progress of PRS applications in PP has been observed over the last two decades and is analyzed here from an interdisciplinary perspective based on 2972 publications. This progress covers most aspects of PRS application in PP, including patterns of global spatial distribution and temporal dynamics, specific PRS technologies, phenotypic research fields, working environments, species, and traits. Subsequently, we demonstrate how to link PRS to multi-omics studies, including how to achieve multi-dimensional PRS data acquisition and processing, how to systematically integrate all kinds of phenotypic information and derive phenotypic knowledge with biological significance, and how to link PP to multi-omics association analysis. Finally, we identify three future perspectives for PRS-based PP: (1) strengthening the spatial and temporal consistency of PRS data, (2) exploring novel phenotypic traits, and (3) facilitating multi-omics communication.

Near-surface wind profile test based on accuracy verification of UAV anemometer lifting height in an urban fringe built-up area

Multirotor UAVs (unmanned aerial vehicles) have been widely used in urban vertical wind environment testing, whereas less attention has been given to the accuracy of wind speed captured by anemometers as drones fly. This paper aims to identify the ideal location of the anemometer on the UAV to obtain more accurate wind speeds and to assess the variation characteristics of wind speed in different spatial types in urban fringe areas. Accuracy verification of the lifting height of the anemometer in the UAV and wind profile test was carried out at three locations (a tennis court, a residential area, and a green park) on the iHarbour campus of Xi'an Jiaotong University. The following results were obtained: (1) the background wind speed was captured more accurately (R = 0.727, P = 0.001) when the lifting height of the anemometer was 0.00 m (as the height of the anemometer was the same as the rotors) and when the multirotor UAV was hovering in the air. However, this optimal lifting height lost 29.6% of the accuracy for capturing the background wind speed. Interestingly, when the lifting height was 0.75 m, the anemometer captured by the anemometer on the drone showed a significant negative correlation (R =  - 0.682, P = 0.005) with the background wind speed. (2) The wind speed at an altitude of 1.5 m in the residential area was significantly lower than that noted at other heights, and the wind speed at 24 m was significantly lower than that at 100 m. (3) In addition, a sudden increase in wind speeds was always observed near the surface of 12 m inside the campus, which may be due to the interaction of hot surface air in this newly built-up area with the cool rural winds around it. The study presents methods and quantitative references for the application of multirotor UAVs in urban vertical wind environment testing and the evaluation of ventilation performance at different heights inside high-rise houses in urban fringe areas.

Biotic Interactions as Mediators of Context-Dependent Biodiversity-Ecosystem Functioning Relationships

Biodiversity drives the maintenance and stability of ecosystem functioning as well as many of nature’s benefits to people, yet people cause substantial biodiversity change. Despite broad consensus about a positive relationship between biodiversity and ecosystem functioning (BEF), the underlying mechanisms and their context-dependencies are not well understood. This proposal, submitted to the European Research Council (ERC), aims at filling this knowledge gap by providing a novel conceptual framework for integrating biotic interactions across guilds of organisms, i.e. plants and mycorrhizal fungi, to explain the ecosystem consequences of biodiversity change. The overarching hypothesis is that EF increases when more tree species associate with functionally dissimilar mycorrhizal fungi. Taking a whole-ecosystem perspective, we propose to explore the role of tree-mycorrhiza interactions in driving BEF across environmental contexts and how this relates to nutrient dynamics. Given the significant role that mycorrhizae play in soil nutrient and water uptake, BEF relationships will be investigated under normal and drought conditions. Resulting ecosystem consequences will be explored by studying main energy channels and ecosystem multifunctionality using food web energy fluxes and by assessing carbon storage. Synthesising drivers of biotic interactions will allow us to understand context-dependent BEF relationships. This interdisciplinary and integrative project spans the whole gradient from local-scale process assessments to global relationships by building on unique experimental infrastructures like the MyDiv Experiment, iDiv Ecotron and the global network TreeDivNet, to link ecological mechanisms to reforestation initiatives. This innovative combination of basic scientific research with real-world interventions links trait-based community ecology, global change research and ecosystem ecology, pioneering a new generation of BEF research and represents a significant step towards implementing BEF theory for human needs.

Thinning increases forest resiliency during unprecedented drought

Regional droughts are now widespread and are projected to further increase. Semi-arid ponderosa pine forests across the western USA, which occupy > 56 million ha, are experiencing unprecedented levels of drought due to the currently ongoing North American megadrought. Using unpiloted aerial vehicle (UAV) thermal images and ground-based hyperspectral data, here we show that ponderosa pine forest canopy temperatures increased during the 2021 summer drought up to 34.6 °C, far above a typical canopy temperature when ponderosa pine trees no longer uptake carbon. We infer that much of the western US ponderosa pine forests likely served as a net carbon source rather than a sink during the 2021 summer drought period. We also demonstrate that regional forest restoration thinning significantly reduced the drought impacts. Thinned ponderosa pine forests had significantly lower increase in canopy temperature and canopy water stress during the drought period compared to the non-thinned forest stands. Furthermore, our extensive soil moisture network data indicate that available soil moisture in the thinned forest was significantly greater at all soil depths of 25 cm, 50 cm, and 100 cm compared to the non-thinned forest, where soil moisture dry-down in the spring started significantly earlier and stayed dry for one month longer causing critical water stress for trees. Forest restoration thinning benefits that are otherwise unappreciated during average precipitation years are significantly amplified during unprecedented drought periods.

Using near–surface temperature data to vicariously calibrate high-resolution thermal infrared imagery and estimate physical surface properties

Thermal response of the surface to solar insolation is a function of the topography and the thermal physical characteristics of the landscape, which include bulk density, heat capacity, thermal conductivity and surface albedo and emissivity. Thermal imaging is routinely used to constrain thermal physical properties by characterizing or modeling changes in the diurnal temperature profiles. Images need to be acquired throughout the diurnal cycle – typically this is done twice during a diurnal cycle, but we suggest multiple times. Comparison of images acquired over 24 hours requires that either the data be calibrated to surface temperature, or the response of the thermal camera is linear and stable over the image acquisition period. Depending on the type and age of the thermal instrument, imagery may be self-calibrated in radiance, corrected for atmospheric effects, and pixels converted to surface temperature. We used an experimental instrumentation where the calibration should be stable, but calibration coefficients are unknown. Cases may occur where one wishes to validate the camera's calibration. We present a method to validate and calibrate the instrument and characterize the thermal physical properties for areas of interest. Finally, in situ high-temporal-resolution oblique thermal imaging can be invaluable in preparation for conducting overflight missions. We present the following:•

The use of oblique thermal high temporal resolution thermal imaging over diurnal or multiday periods for the characterization of landscapes has not been widespread but poses great potential.

•

A method of collecting and analyzing thermal data that can be used to either determine or validate thermal camera calibration coefficients.

•

An approach to characterize thermophysical properties of the landscape using oblique temporally high-resolution thermal imaging, combined with in situ ground measurements.

Remote Sensing in Studies of the Growing Season: A Bibliometric Analysis

Analyses of climate change based on point observations indicate an extension of the plant growing season, which may have an impact on plant production and functioning of natural ecosystems. Analyses involving remote sensing methods, which have added more detail to results obtained in the traditional way, have been carried out only since the 1980s. The paper presents the results of a bibliometric analysis of papers related to the growing season published from 2000–2021 included in the Web of Science database. Through filtering, 285 publications were selected and subjected to statistical processing and analysis of their content. This resulted in the identification of author teams that mostly focused their research on vegetation growth and in the selection of the most common keywords describing the beginning, end, and duration of the growing season. It was found that most studies on the growing season were reported from Asia, Europe, and North America (i.e., 32%, 28%, and 28%, respectively). The analyzed articles show the advantage of satellite data over low-altitude and ground-based data in providing information on plant vegetation. Over three quarters of the analyzed publications focused on natural plant communities. In the case of crops, wheat and rice were the most frequently studied plants (i.e., they were analyzed in over 30% and over 20% of publications, respectively).

High-throughput field crop phenotyping: current status and challenges

In contrast to the rapid advances made in plant genotyping, plant phenotyping is considered a bottleneck in plant science. This has promoted high-throughput plant phenotyping (HTP) studies, resulting in an exponential increase in phenotyping-related publications. The development of HTP was originally intended for use as indoor HTP technologies for model plant species under controlled environments. However, this subsequently shifted to HTP for use in crops in fields. Although HTP in fields is much more difficult to conduct due to unstable environmental conditions compared to HTP in controlled environments, recent advances in HTP technology have allowed these difficulties to be overcome, allowing for rapid, efficient, non-destructive, non-invasive, quantitative, repeatable, and objective phenotyping. Recent HTP developments have been accelerated by the advances in data analysis, sensors, and robot technologies, including machine learning, image analysis, three dimensional (3D) reconstruction, image sensors, laser sensors, environmental sensors, and drones, along with high-speed computational resources. This article provides an overview of recent HTP technologies, focusing mainly on canopy-based phenotypes of major crops, such as canopy height, canopy coverage, canopy biomass, and canopy stressed appearance, in addition to crop organ detection and counting in the fields. Current topics in field HTP are also presented, followed by a discussion on the low rates of adoption of HTP in practical breeding programs.

Factors Influencing Temperature Measurements from Miniaturized Thermal Infrared (TIR) Cameras: A Laboratory-Based Approach

The workflow for estimating the temperature in agricultural fields from multiple sensors needs to be optimized upon testing each type of sensor’s actual user performance. In this sense, readily available miniaturized UAV-based thermal infrared (TIR) cameras can be combined with proximal sensors in measuring the surface temperature. Before the two types of cameras can be operationally used in the field, laboratory experiments are needed to fully understand their capabilities and all the influencing factors. We present the measurement results of laboratory experiments of UAV-borne WIRIS 2nd GEN and handheld FLIR E8-XT cameras. For these uncooled sensors, it took 30 to 60 min for the measured signal to stabilize and the sensor temperature drifted continuously. The drifting sensor temperature was strongly correlated to the measured signal. Specifically for WIRIS, the automated non-uniformity correction (NUC) contributed to extra uncertainty in measurements. Another problem was the temperature measurement dependency on various ambient environmental parameters. An increase in the measuring distance resulted in the underestimation of surface temperature, though the degree of change may also come from reflected radiation from neighboring objects, water vapor absorption, and the object size in the field of view (FOV). Wind and radiation tests suggested that these factors can contribute to the uncertainty of several Celsius degrees in measured results. Based on these indoor experiment results, we provide a list of suggestions on the potential practices for deriving accurate temperature data from radiometric miniaturized TIR cameras in actual field practices for (agro-)environmental research.

Phenotyping stomatal closure by thermal imaging for GWAS and TWAS of water use efficiency-related genes

Stomata allow CO2 uptake by leaves for photosynthetic assimilation at the cost of water vapor loss to the atmosphere. The opening and closing of stomata in response to fluctuations in light intensity regulate CO2 and water fluxes and are essential for maintaining water-use efficiency (WUE). However, a little is known about the genetic basis for natural variation in stomatal movement, especially in C4 crops. This is partly because the stomatal response to a change in light intensity is difficult to measure at the scale required for association studies. Here, we used high-throughput thermal imaging to bypass the phenotyping bottleneck and assess 10 traits describing stomatal conductance (gs) before, during and after a stepwise decrease in light intensity for a diversity panel of 659 sorghum (Sorghum bicolor) accessions. Results from thermal imaging significantly correlated with photosynthetic gas exchange measurements. gs traits varied substantially across the population and were moderately heritable (h2 up to 0.72). An integrated genome-wide and transcriptome-wide association study identified candidate genes putatively driving variation in stomatal conductance traits. Of the 239 unique candidate genes identified with the greatest confidence, 77 were putative orthologs of Arabidopsis (Arabidopsis thaliana) genes related to functions implicated in WUE, including stomatal opening/closing (24 genes), stomatal/epidermal cell development (35 genes), leaf/vasculature development (12 genes), or chlorophyll metabolism/photosynthesis (8 genes). These findings demonstrate an approach to finding genotype-to-phenotype relationships for a challenging trait as well as candidate genes for further investigation of the genetic basis of WUE in a model C4 grass for bioenergy, food, and forage production.

UAV-Assisted Thermal Infrared and Multispectral Imaging of Weed Canopies for Glyphosate Resistance Detection

The foundation of contemporary weed management practices in many parts of the world is glyphosate. However, dependency on the effectiveness of herbicide practices has led to overuse through continuous growth of crops resistant to a single mode of action. In order to provide a cost-effective weed management strategy that does not promote glyphosate-resistant weed biotypes, differences between resistant and susceptible biotypes have to be identified accurately in the field conditions. Unmanned Aerial Vehicle (UAV)-assisted thermal and multispectral remote sensing has potential for detecting biophysical characteristics of weed biotypes during the growing season, which includes distinguishing glyphosate-susceptible and glyphosate-resistant weed populations based on canopy temperature and deep learning driven weed identification algorithms. The objective of this study was to identify herbicide resistance after glyphosate application in true field conditions by analyzing the UAV-acquired thermal and multispectral response of kochia, waterhemp, redroot pigweed, and common ragweed. The data were processed in ArcGIS for raster classification as well as spectral comparison of glyphosate-resistant and glyphosate-susceptible weeds. The classification accuracy between the sensors and classification methods of maximum likelihood, random trees, and Support Vector Machine (SVM) were compared. The random trees classifier performed the best at 4 days after application (DAA) for kochia with 62.9% accuracy. The maximum likelihood classifier provided the highest performing result out of all classification methods with an accuracy of 75.2%. A commendable classification was made at 8 DAA where the random trees classifier attained an accuracy of 87.2%. However, thermal reflectance measurements as a predictor for glyphosate resistance within weed populations in field condition was unreliable due to its susceptibility to environmental conditions. Normalized Difference Vegetation Index (NDVI) and a composite reflectance of 842 nm, 705 nm, and 740 nm wavelength managed to provide better classification results than thermal in most cases.

A Practical Validation of Uncooled Thermal Imagers for Small RPAS

Uncooled thermal imaging sensors in the LWIR (7.5 μm to 14 μm) have recently been developed for use with small RPAS. This study derives a new thermal imaging validation methodology via the use of a blackbody source (indoors) and real-world field conditions (outdoors). We have demonstrated this method with three popular LWIR cameras by DJI (Zenmuse XT-R, Zenmuse XT2 and, the M2EA) operated by three different popular DJI RPAS platforms (Matrice 600 Pro, M300 RTK and, the Mavic 2 Enterprise Advanced). Results from the blackbody work show that each camera has a highly linearized response (R2 > 0.99) in the temperature range 5–40 °C as well as a small (<2 °C) temperature bias that is less than the stated accuracy of the cameras. Field validation was accomplished by imaging vegetation and concrete targets (outdoors and at night), that were instrumented with surface temperature sensors. Environmental parameters (air temperature, humidity, pressure and, wind and gusting) were measured for several hours prior to imaging data collection and found to either not be a factor, or were constant, during the ~30 min data collection period. In-field results from imagery at five heights between 10 m and 50 m show absolute temperature retrievals of the concrete and two vegetation sites were within the specifications of the cameras. The methodology has been developed with consideration of active RPAS operational requirements.

Integrating Drone Technology into an Innovative Agrometeorological Methodology for the Precise and Real-Time Estimation of Crop Water Requirements

Precision agriculture has been at the cutting edge of research during the recent decade, aiming to reduce water consumption and ensure sustainability in agriculture. The proposed methodology was based on the crop water stress index (CWSI) and was applied in Greece within the ongoing research project GreenWaterDrone. The innovative approach combines real spatial data, such as infrared canopy temperature, air temperature, air relative humidity, and thermal infrared image data, taken above the crop field using an aerial micrometeorological station (AMMS) and a thermal (IR) camera installed on an unmanned aerial vehicle (UAV). Following an initial calibration phase, where the ground micrometeorological station (GMMS) was installed in the crop, no equipment needed to be maintained in the field. Aerial and ground measurements were transferred in real time to sophisticated databases and applications over existing mobile networks for further processing and estimation of the actual water requirements of a specific crop at the field level, dynamically alerting/informing local farmers/agronomists of the irrigation necessity and additionally for potential risks concerning their fields. The supported services address farmers’, agricultural scientists’, and local stakeholders’ needs to conform to regional water management and sustainable agriculture policies. As preliminary results of this study, we present indicative original illustrations and data from applying the methodology to assess UAV functionality while aiming to evaluate and standardize all system processes.

Diurnal and Seasonal Mapping of Water Deficit Index and Evapotranspiration by an Unmanned Aerial System: A Case Study for Winter Wheat in Denmark

Precision irrigation is a promising method to mitigate the impacts of drought stress on crop production with the optimal use of water resources. However, the reliable assessment of plant water status has not been adequately demonstrated, and unmanned aerial systems (UAS) offer great potential for spatiotemporal improvements. This study utilized UAS equipped with multispectral and thermal sensors to detect and quantify drought stress in winter wheat (Triticum aestivum L.) using the Water Deficit Index (WDI). Biennial field experiments were conducted on coarse sand soil in Denmark and analyses were performed at both diurnal and seasonal timescales. The WDI was significantly correlated with leaf stomatal conductance (R2 = 0.61–0.73), and the correlation was weaker with leaf water potential (R2 = 0.39–0.56) and topsoil water status (the highest R2 of 0.68). A semi-physical model depicting the relationship between WDI and fraction of transpirable soil water (FTSW) in the root zone was derived with R2 = 0.74. Moreover, WDI estimates were improved using an energy balance model with an iterative scheme to estimate the net radiation and land surface temperature, as well as the dual crop coefficient. The diurnal variation in WDI revealed a pattern of the ratio of actual to potential evapotranspiration, being higher in the morning, decreasing at noon hours and ‘recovering’ in the afternoon. Future work should investigate the temporal upscaling of evapotranspiration, which may be used to develop methods for site-specific irrigation recommendations.

Cloud Cover throughout All the Paddy Rice Fields in Guangdong, China: Impacts on Sentinel 2 MSI and Landsat 8 OLI Optical Observations

Cloud cover hinders the effective use of vegetation indices from optical satellite-acquired imagery in cloudy agricultural production areas, such as Guangdong, a subtropical province in southern China which supports two-season rice production. The number of cloud-free observations for the earth-orbiting optical satellite sensors must be determined to verify how much their observations are affected by clouds. This study determines the quantified wide-ranging impact of clouds on optical satellite observations by mapping the annual total observations (ATOs), annual cloud-free observations (ACFOs), monthly cloud-free observations (MCFOs) maps, and acquisition probability (AP) of ACFOs for the Sentinel 2 (2017–2019) and Landsat 8 (2014–2019) for all the paddy rice fields in Guangdong province (APRFG), China. The ATOs of Landsat 8 showed relatively stable observations compared to the Sentinel 2, and the per-field ACFOs of Sentinel 2 and Landsat 8 were unevenly distributed. The MCFOs varied on a monthly basis, but in general, the MCFOs were greater between August and December than between January and July. Additionally, the AP of usable ACFOs with 52.1% (Landsat 8) and 47.7% (Sentinel 2) indicated that these two satellite sensors provided markedly restricted observation capability for rice in the study area. Our findings are particularly important and useful in the tropics and subtropics, and the analysis has described cloud cover frequency and pervasiveness throughout different portions of the rice growing season, providing insight into how rice monitoring activities by using Sentinel 2 and Landsat 8 imagery in Guangdong would be impacted by cloud cover.

High night temperature effects on wheat and rice: Current status and way forward

Rapid increases in minimum night temperature than in maximum day temperature is predicted to continue, posing significant challenges to crop productivity. Rice and wheat are two major staples that are sensitive to high night-temperature (HNT) stress. This review aims to (i) systematically compare the grain yield responses of rice and wheat exposed to HNT stress across scales, and (ii) understand the physiological and biochemical responses that affect grain yield and quality. To achieve this, we combined a synthesis of current literature on HNT effects on rice and wheat with information from a series of independent experiments we conducted across scales, using a common set of genetic materials to avoid confounding our findings with differences in genetic background. In addition, we explored HNT-induced alterations in physiological mechanisms including carbon balance, source-sink metabolite changes and reactive oxygen species. Impacts of HNT on grain developmental dynamics focused on grain-filling duration, post-flowering senescence, changes in grain starch and protein composition, starch metabolism enzymes and chalk formation in rice grains are summarized. Finally, we highlight the need for high-throughput field-based phenotyping facilities for improved assessment of large-diversity panels and mapping populations to aid breeding for increased resilience to HNT in crops.

Correlation and co-localization of QTL for stomatal density, canopy temperature, and productivity with and without drought stress in Setaria

Mechanistic modeling indicates that stomatal conductance could be reduced to improve water use efficiency (WUE) in C4 crops. Genetic variation in stomatal density and canopy temperature was evaluated in the model C4 genus, Setaria. Recombinant inbred lines (RILs) derived from a Setaria italica×Setaria viridis cross were grown with ample or limiting water supply under field conditions in Illinois. An optical profilometer was used to rapidly assess stomatal patterning, and canopy temperature was measured using infrared imaging. Stomatal density and canopy temperature were positively correlated but both were negatively correlated with total above-ground biomass. These trait relationships suggest a likely interaction between stomatal density and the other drivers of water use such as stomatal size and aperture. Multiple quantitative trait loci (QTL) were identified for stomatal density and canopy temperature, including co-located QTL on chromosomes 5 and 9. The direction of the additive effect of these QTL on chromosome 5 and 9 was in accordance with the positive phenotypic relationship between these two traits. This, along with prior experiments, suggests a common genetic architecture between stomatal patterning and WUE in controlled environments with canopy transpiration and productivity in the field, while highlighting the potential of Setaria as a model to understand the physiology and genetics of WUE in C4 species.

Assessment of Ensemble Learning to Predict Wheat Grain Yield Based on UAV-Multispectral Reflectance

Grain yield is increasingly affected by climate factors such as drought and heat. To develop resilient and high-yielding cultivars, high-throughput phenotyping (HTP) techniques are essential for precise decisions in wheat breeding. The ability of unmanned aerial vehicle (UAV)-based multispectral imaging and ensemble learning methods to increase the accuracy of grain yield prediction in practical breeding work is evaluated in this study. For this, 211 winter wheat genotypes were planted under full and limited irrigation treatments, and multispectral data were collected at heading, flowering, early grain filling (EGF), and mid-grain filling (MGF) stages. Twenty multispectral vegetation indices (VIs) were estimated, and VIs with heritability greater than 0.5 were selected to evaluate the models across the growth stages under both irrigation treatments. A framework for ensemble learning was developed by combining multiple base models such as random forest (RF), support vector machine (SVM), Gaussian process (GP), and ridge regression (RR). The R2 values between VIs and grain yield for individual base models were ranged from 0.468 to 0.580 and 0.537 to 0.598 for grain yield prediction in full and limited irrigation treatments across growth stages, respectively. The prediction results of ensemble models were ranged from 0.491 to 0.616 and 0.560 to 0.616 under full and limited irrigation treatments respectively, and were higher than that of the corresponding base learners. Moreover, the grain yield prediction results were observed high at mid grain filling stage under both full (R2 = 0.625) and limited (R2 = 0.628) irrigation treatments through ensemble learning based stacking of four base learners. Further improvements in ensemble learning models can accelerate the use of UAV-based multispectral data for accurate predictions of complex traits like grain yield in wheat.

UAS-Based Plant Phenotyping for Research and Breeding Applications

Unmanned aircraft system (UAS) is a particularly powerful tool for plant phenotyping, due to reasonable cost of procurement and deployment, ease and flexibility for control and operation, ability to reconfigure sensor payloads to diversify sensing, and the ability to seamlessly fit into a larger connected phenotyping network. These advantages have expanded the use of UAS-based plant phenotyping approach in research and breeding applications. This paper reviews the state of the art in the deployment, collection, curation, storage, and analysis of data from UAS-based phenotyping platforms. We discuss pressing technical challenges, identify future trends in UAS-based phenotyping that the plant research community should be aware of, and pinpoint key plant science and agronomic questions that can be resolved with the next generation of UAS-based imaging modalities and associated data analysis pipelines. This review provides a broad account of the state of the art in UAS-based phenotyping to reduce the barrier to entry to plant science practitioners interested in deploying this imaging modality for phenotyping in plant breeding and research areas.

Simultaneous corn and soybean yield prediction from remote sensing data using deep transfer learning

Large-scale crop yield estimation is, in part, made possible due to the availability of remote sensing data allowing for the continuous monitoring of crops throughout their growth cycle. Having this information allows stakeholders the ability to make real-time decisions to maximize yield potential. Although various models exist that predict yield from remote sensing data, there currently does not exist an approach that can estimate yield for multiple crops simultaneously, and thus leads to more accurate predictions. A model that predicts the yield of multiple crops and concurrently considers the interaction between multiple crop yields. We propose a new convolutional neural network model called YieldNet which utilizes a novel deep learning framework that uses transfer learning between corn and soybean yield predictions by sharing the weights of the backbone feature extractor. Additionally, to consider the multi-target response variable, we propose a new loss function. We conduct our experiment using data from 1132 counties for corn and 1076 counties for soybean across the United States. Numerical results demonstrate that our proposed method accurately predicts corn and soybean yield from one to four months before the harvest with an MAE being 8.74% and 8.70% of the average yield, respectively, and is competitive to other state-of-the-art approaches.

Recent developments and potential of robotics in plant eco-phenotyping

Automated acquisition of plant eco-phenotypic information can serve as a decision-making basis for precision agricultural management and can also provide detailed insights into plant growth status, pest management, water and fertilizer management for plant breeders and plant physiologists. Because the microscopic components and macroscopic morphology of plants will be affected by the ecological environment, research on plant eco-phenotyping is more meaningful than the study of single-plant phenotyping. To achieve high-throughput acquisition of phenotyping information, the combination of high-precision sensors and intelligent robotic platforms have become an emerging research focus. Robotic platforms and automated systems are the important carriers of phenotyping monitoring sensors that enable large-scale screening. Through the diverse design and flexible systems, an efficient operation can be achieved across a range of experimental and field platforms. The combination of robot technology and plant phenotyping monitoring tools provides the data to inform novel artificial intelligence (AI) approaches that will provide steppingstones for new research breakthroughs. Therefore, this article introduces robotics and eco-phenotyping and examines research significant to this novel domain of plant eco-phenotyping. Given the monitoring scenarios of phenotyping information at different scales, the used intelligent robot technology, efficient automation platform, and advanced sensor equipment are summarized in detail. We further discuss the challenges posed to current research as well as the future developmental trends in the application of robot technology and plant eco-phenotyping. These include the use of collected data for AI applications and high-bandwidth data transfer, and large well-structured (meta) data storage approaches in plant sciences and agriculture.

Thermal Analysis of Stomatal Response under Salinity and High Light

A non-destructive thermal imaging method was used to study the stomatal response of salt-treated Arabidopsis thaliana plants to excessive light. The plants were exposed to different levels of salt concentrations (0, 75, 150, and 220 mM NaCl). Time-dependent thermograms showed the changes in the temperature distribution over the lamina and provided new insights into the acute light-induced temporary response of Arabidopsis under short-term salinity. The initial response of plants, which was associated with stomatal aperture, revealed an exponential growth in temperature kinetics. Using a single-exponential function, we estimated the time constants of thermal courses of plants exposed to acute high light. The saline-induced impairment in stomatal movement caused the reduced stomatal conductance and transpiration rate. Limited transpiration of NaCl-treated plants resulted in an increased rosette temperature and decreased thermal time constants as compared to the controls. The net CO₂ assimilation rate decreased for plants exposed to 220 mM NaCl; in the case of 75 mM NaCl treatment, an increase was observed. A significant decline in the maximal quantum yield of photosystem II under excessive light was noticeable for the control and NaCl-treated plants. This study provides evidence that thermal imaging as a highly sensitive technique may be useful for analyzing the stomatal aperture and movement under dynamic environmental conditions.

Investigating the Potential of a Newly Developed UAV-Mounted VNIR/SWIR Imaging System for Monitoring Crop Traits—A Case Study for Winter Wheat

UAV-based multispectral multi-camera systems are widely used in scientific research for non-destructive crop traits estimation to optimize agricultural management decisions. These systems typically provide data from the visible and near-infrared (VNIR) domain. However, several key absorption features related to biomass and nitrogen (N) are located in the short-wave infrared (SWIR) domain. Therefore, this study investigates a novel multi-camera system prototype that addresses this spectral gap with a sensitivity from 600 to 1700 nm by implementing dedicated bandpass filter combinations to derive application-specific vegetation indices (VIs). In this study, two VIs, GnyLi and NRI, were applied using data obtained on a single observation date at a winter wheat field experiment located in Germany. Ground truth data were destructively sampled for the entire growing season. Likewise, crop heights were derived from UAV-based RGB image data using an improved approach developed within this study. Based on these variables, regression models were derived to estimate fresh and dry biomass, crop moisture, N concentration, and N uptake. The relationships between the NIR/SWIR-based VIs and the estimated crop traits were successfully evaluated (R2: 0.57 to 0.66). Both VIs were further validated against the sampled ground truth data (R2: 0.75 to 0.84). These results indicate the imaging system’s potential for monitoring crop traits in agricultural applications, but further multitemporal validations are needed.

Estimating Agricultural Soil Moisture Content through UAV-Based Hyperspectral Images in the Arid Region

Unmanned aerial vehicle (UAV)-based hyperspectral remote sensing is an important monitoring technology for the soil moisture content (SMC) of agroecological systems in arid regions. This technology develops precision farming and agricultural informatization. However, hyperspectral data are generally used in data mining. In this study, UAV-based hyperspectral imaging data with a resolution o 4 cm and totaling 70 soil samples (0–10 cm) were collected from farmland (2.5 × 104 m2) near Fukang City, Xinjiang Uygur Autonomous Region, China. Four estimation strategies were tested: the original image (strategy I), first- and second-order derivative methods (strategy II), the fractional-order derivative (FOD) technique (strategy III), and the optimal fractional order combined with the optimal multiband indices (strategy IV). These strategies were based on the eXtreme Gradient Boost (XGBoost) algorithm, with the aim of building the best estimation model for agricultural SMC in arid regions. The results demonstrated that FOD technology could effectively mine information (with an absolute maximum correlation coefficient of 0.768). By comparison, strategy IV yielded the best estimates out of the methods tested (R2val = 0.921, RMSEP = 1.943, and RPD = 2.736) for the SMC. The model derived from the order of 0.4 within strategy IV worked relatively well among the different derivative methods (strategy I, II, and III). In conclusion, the combination of FOD technology and the optimal multiband indices generated a highly accurate model within the XGBoost algorithm for SMC estimation. This research provided a promising data mining approach for UAV-based hyperspectral imaging data.

Water-Stressed Plants Do Not Cool: Leaf Surface Temperature of Living Wall Plants under Drought Stress

Urban green infrastructures offer thermal regulation to mitigate urban heat island effects. To gain a better understanding of the cooling ability of transpiring plants at the leaf level, we developed a method to measure the time series of thermal data with a miniaturized, uncalibrated thermal infrared camera. We examined the canopy temperature of four characteristic living wall plants (Heuchera x cultorum, Bergenia cordifolia, Geranium sanguineum, and Brunnera macrophylla) under increasing drought stress and compared them with a well-watered control group. The method proved suitable to evaluate differences in canopy temperature between the different treatments. Leaf temperatures of water-stressed plants were 6 to 8 °C higher than those well-watered, with differences among species. In order to cool through transpiration, vegetation in green infrastructures must be sufficiently supplied with water. Thermal cameras were found to be useful to monitor vertical greening because leaf surface temperature is closely related to drought stress. The usage of thermal cameras mounted on unmanned aerial vehicles could be a rapid and easy monitoring system to cover large façades.

UAV-Based Thermal, RGB Imaging and Gene Expression Analysis Allowed Detection of Fusarium Head Blight and Gave New Insights Into the Physiological Responses to the Disease in Durum Wheat

Wheat is one of the world's most economically important cereal crop, grown on 220 million hectares. Fusarium head blight (FHB) disease is considered a major threat to durum (Triticum turgidum subsp. durum (Desfontaines) Husnache) and bread wheat (T. aestivum L.) cultivars and is mainly managed by the application of fungicides at anthesis. However, fungicides are applied when FHB symptoms are clearly visible and the spikes are almost entirely bleached (% of diseased spikelets > 80%), by when it is too late to control FHB disease. For this reason, farmers often react by performing repeated fungicide treatments that, however, due to the advanced state of the infection, cause a waste of money and pose significant risks to the environment and non-target organisms. In the present study, we used unmanned aerial vehicle (UAV)-based thermal infrared (TIR) and red-green-blue (RGB) imaging for FHB detection in T. turgidum (cv. Marco Aurelio) under natural field conditions. TIR and RGB data coupled with ground-based measurements such as spike's temperature, photosynthetic efficiency and molecular identification of FHB pathogens, detected FHB at anthesis half-way (Zadoks stage 65, ZS 65), when the percentage (%) of diseased spikelets ranged between 20% and 60%. Moreover, in greenhouse experiments the transcripts of the key genes involved in stomatal closure were mostly up-regulated in F. graminearum-inoculated plants, demonstrating that the physiological mechanism behind the spike's temperature increase and photosynthetic efficiency decrease could be attributed to the closure of the guard cells in response to F. graminearum. In addition, preliminary analysis revealed that there is differential regulation of genes between drought-stressed and F. graminearum-inoculated plants, suggesting that there might be a possibility to discriminate between water stress and FHB infection. This study shows the potential of UAV-based TIR and RGB imaging for field phenotyping of wheat and other cereal crop species in response to environmental stresses. This is anticipated to have enormous promise for the detection of FHB disease and tremendous implications for optimizing the application of fungicides, since global food crop demand is to be met with minimal environmental impacts.

High-Throughput Phenotyping: A Platform to Accelerate Crop Improvement

Development of high-throughput phenotyping technologies has progressed considerably in the last 10 years. These technologies provide precise measurements of desired traits among thousands of field-grown plants under diversified environments; this is a critical step towards selection of better performing lines as to yield, disease resistance, and stress tolerance to accelerate crop improvement programs. High-throughput phenotyping techniques and platforms help unraveling the genetic basis of complex traits associated with plant growth and development and targeted traits. This review focuses on the advancements in technologies involved in high-throughput, field-based, aerial, and unmanned platforms. Development of user-friendly data management tools and softwares to better understand phenotyping will increase the use of field-based high-throughput techniques, which have potential to revolutionize breeding strategies and meet the future needs of stakeholders.

High Spatial and Temporal Resolution Energy Flux Mapping of Different Land Covers Using an Off-the-Shelf Unmanned Aerial System

With the development of low-cost, lightweight, integrated thermal infrared-multispectral cameras, unmanned aerial systems (UAS) have recently become a flexible complement to eddy covariance (EC) station methods for mapping surface energy fluxes of vegetated areas. These sensors facilitate the measurement of several site characteristics in one flight (e.g., radiometric temperature, vegetation indices, vegetation structure), which can be used alongside in-situ meteorology data to provide spatially-distributed estimates of energy fluxes at very high resolution. Here we test one such system (MicaSense Altum) integrated into an off-the-shelf long-range vertical take-off and landing (VTOL) unmanned aerial vehicle, and apply and evaluate our method by comparing flux estimates with EC-derived data, with specific and novel focus on heterogeneous vegetation communities at three different sites in Germany. Firstly, we present an empirical method for calibrating airborne radiometric temperature in standard units (K) using the Altum multispectral and thermal infrared instrument. Then we provide detailed methods using the two-source energy balance model (TSEB) for mapping net radiation (Rn), sensible (H), latent (LE) and ground (G) heat fluxes at <0.82 m resolution, with root mean square errors (RMSE) less than 45, 37, 39, 52 W m−2 respectively. Converting to radiometric temperature using our empirical method resulted in a 19% reduction in RMSE across all fluxes compared to the standard conversion equation provided by the manufacturer. Our results show the potential of this UAS for mapping energy fluxes at high resolution over large areas in different conditions, but also highlight the need for further surveys of different vegetation types and land uses.

Assimilation of LAI Derived from UAV Multispectral Data into the SAFY Model to Estimate Maize Yield

In this study, we develop a method to estimate corn yield based on remote sensing data and ground monitoring data under different water treatments. Spatially explicit information on crop yields is essential for farmers and agricultural agencies to make well-informed decisions. One approach to estimate crop yield with remote sensing is data assimilation, which integrates sequential observations of canopy development from remote sensing into model simulations of crop growth processes. We found that leaf area index (LAI) inversion based on unmanned aerial vehicle (UAV) vegetation index has a high accuracy, with R2 and root mean square error (RMSE) values of 0.877 and 0.609, respectively. Maize yield estimation based on UAV remote sensing data and simple algorithm for yield (SAFY) crop model data assimilation has different yield estimation accuracy under different water treatments. This method can be used to estimate corn yield, where R2 is 0.855 and RMSE is 692.8kg/ha. Generally, the higher the water stress, the lower the estimation accuracy. Furthermore, we perform the yield estimate mapping at 2 m spatial resolution, which has a higher spatial resolution and accuracy than satellite remote sensing. The great potential of incorporating UAV observations with crop data to monitor crop yield, and improve agricultural management is therefore indicated.

High-throughput image segmentation and machine learning approaches in the plant sciences across multiple scales

Agriculture has benefited greatly from the rise of big data and high-performance computing. The acquisition and analysis of data across biological scales have resulted in strategies modeling inter- actions between plant genotype and environment, models of root architecture that provide insight into resource utilization, and the elucidation of cell-to-cell communication mechanisms that are instrumental in plant development. Image segmentation and machine learning approaches for interpreting plant image data are among many of the computational methodologies that have evolved to address challenging agricultural and biological problems. These approaches have led to contributions such as the accelerated identification of gene that modulate stress responses in plants and automated high-throughput phenotyping for early detection of plant diseases. The continued acquisition of high throughput imaging across multiple biological scales provides opportunities to further push the boundaries of our understandings quicker than ever before. In this review, we explore the current state of the art methodologies in plant image segmentation and machine learning at the agricultural, organ, and cellular scales in plants. We show how the methodologies for segmentation and classification differ due to the diversity of physical characteristics found at these different scales. We also discuss the hardware technologies most commonly used at these different scales, the types of quantitative metrics that can be extracted from these images, and how the biological mechanisms by which plants respond to abiotic/biotic stresses or genotypic modifications can be extracted from these approaches.

Early Detection of Plant Viral Disease Using Hyperspectral Imaging and Deep Learning

Early detection of grapevine viral diseases is critical for early interventions in order to prevent the disease from spreading to the entire vineyard. Hyperspectral remote sensing can potentially detect and quantify viral diseases in a nondestructive manner. This study utilized hyperspectral imagery at the plant level to identify and classify grapevines inoculated with the newly discovered DNA virus grapevine vein-clearing virus (GVCV) at the early asymptomatic stages. An experiment was set up at a test site at South Farm Research Center, Columbia, MO, USA (38.92 N, −92.28 W), with two grapevine groups, namely healthy and GVCV-infected, while other conditions were controlled. Images of each vine were captured by a SPECIM IQ 400–1000 nm hyperspectral sensor (Oulu, Finland). Hyperspectral images were calibrated and preprocessed to retain only grapevine pixels. A statistical approach was employed to discriminate two reflectance spectra patterns between healthy and GVCV vines. Disease-centric vegetation indices (VIs) were established and explored in terms of their importance to the classification power. Pixel-wise (spectral features) classification was performed in parallel with image-wise (joint spatial–spectral features) classification within a framework involving deep learning architectures and traditional machine learning. The results showed that: (1) the discriminative wavelength regions included the 900–940 nm range in the near-infrared (NIR) region in vines 30 days after sowing (DAS) and the entire visual (VIS) region of 400–700 nm in vines 90 DAS; (2) the normalized pheophytization index (NPQI), fluorescence ratio index 1 (FRI1), plant senescence reflectance index (PSRI), anthocyanin index (AntGitelson), and water stress and canopy temperature (WSCT) measures were the most discriminative indices; (3) the support vector machine (SVM) was effective in VI-wise classification with smaller feature spaces, while the RF classifier performed better in pixel-wise and image-wise classification with larger feature spaces; and (4) the automated 3D convolutional neural network (3D-CNN) feature extractor provided promising results over the 2D convolutional neural network (2D-CNN) in learning features from hyperspectral data cubes with a limited number of samples.

A geostatistical fusion approach using UAV data for probabilistic estimation of Xylella fastidiosa subsp. pauca infection in olive trees

Xylella fastidiosa is one of the most destructive plant pathogenic bacteria worldwide, affecting more than 500 plant species. In Apulia region (southeastern Italy), X. fastidiosa subsp. pauca (Xfp) is responsible for a severe disease, the olive quick decline syndrome (OQDS), spreading epidemically and with dramatic impact on the agriculture, the landscape, the tourism, and the cultural heritage of this region. An early detection of the infected plants would hinder the rapid spread of the disease. The main objective of this paper was to define a geostatistical approach of data fusion, which combines remote (radiometric), and proximal (geophysical) sensor data and visual inspections with plant diagnostic tests, to provide probabilistic maps of Xfp infection risk. The study site was an olive grove located at Oria (province of Brindisi, Italy), where at the time of monitoring (September 2017) only few plants showed initial symptoms of the disease. The measurements included: 1) acquisitions of reflected electromagnetic radiation with UAV (Unmanned Aerial Vehicle) equipped with a multi-spectral camera; 2) geophysical surveys on the trunks of 49 plants with Ground Penetrating Radar (GPR); 3) disease severity rating, by visual inspection of the proportion of canopy with symptoms; 4) qPCR (real time-quantitative Polymerase Chain Reaction) data from tests on 61 plants. The data were submitted to a set of processing techniques to define a "data fusion" procedure, based on non-parametric multivariate geostatistics. The approach allowed marking those areas where the risk of infection was higher, and identifying the possible infection entry routes into the field. The probability map of infection risk could be used as an effective tool for a preventive action and for a better organization of the monitoring plans.

UAV-Based Heating Requirement Determination for Frost Management in Apple Orchard

Frost is a natural disaster that can cause catastrophic damages in agriculture, while traditional temperature monitoring in orchards has disadvantages such as being imprecise and laborious, which can lead to inadequate or wasteful frost protection treatments. In this article, we presented a heating requirement assessment methodology for frost protection in an apple orchard utilizing unmanned aerial vehicle (UAV)-based thermal and RGB cameras. A thermal image stitching algorithm using the BRISK feature was developed for creating georeferenced orchard temperature maps, which attained a sub-centimeter map resolution and a stitching speed of 100 thermal images within 30 s. YOLOv4 classifiers for six apple flower bud growth stages in various network sizes were trained based on 5040 RGB images, and the best model achieved a 71.57% mAP for a test dataset consisted of 360 images. A flower bud mapping algorithm was developed to map classifier detection results into dense growth stage maps utilizing RGB image geoinformation. Heating requirement maps were created using artificial flower bud critical temperatures to simulate orchard heating demands during frost events. The results demonstrated the feasibility of the proposed orchard heating requirement determination methodology, which has the potential to be a critical component of an autonomous, precise frost management system in future studies.

High-Throughput Plant Phenotyping Platform (HT3P) as a Novel Tool for Estimating Agronomic Traits From the Lab to the Field

Food scarcity, population growth, and global climate change have propelled crop yield growth driven by high-throughput phenotyping into the era of big data. However, access to large-scale phenotypic data has now become a critical barrier that phenomics urgently must overcome. Fortunately, the high-throughput plant phenotyping platform (HT3P), employing advanced sensors and data collection systems, can take full advantage of non-destructive and high-throughput methods to monitor, quantify, and evaluate specific phenotypes for large-scale agricultural experiments, and it can effectively perform phenotypic tasks that traditional phenotyping could not do. In this way, HT3Ps are novel and powerful tools, for which various commercial, customized, and even self-developed ones have been recently introduced in rising numbers. Here, we review these HT3Ps in nearly 7 years from greenhouses and growth chambers to the field, and from ground-based proximal phenotyping to aerial large-scale remote sensing. Platform configurations, novelties, operating modes, current developments, as well the strengths and weaknesses of diverse types of HT3Ps are thoroughly and clearly described. Then, miscellaneous combinations of HT3Ps for comparative validation and comprehensive analysis are systematically present, for the first time. Finally, we consider current phenotypic challenges and provide fresh perspectives on future development trends of HT3Ps. This review aims to provide ideas, thoughts, and insights for the optimal selection, exploitation, and utilization of HT3Ps, and thereby pave the way to break through current phenotyping bottlenecks in botany.

Qualification of Soybean Responses to Flooding Stress Using UAV-Based Imagery and Deep Learning

Soybean is sensitive to flooding stress that may result in poor seed quality and significant yield reduction. Soybean production under flooding could be sustained by developing flood-tolerant cultivars through breeding programs. Conventionally, soybean tolerance to flooding in field conditions is evaluated by visually rating the shoot injury/damage due to flooding stress, which is labor-intensive and subjective to human error. Recent developments of field high-throughput phenotyping technology have shown great potential in measuring crop traits and detecting crop responses to abiotic and biotic stresses. The goal of this study was to investigate the potential in estimating flood-induced soybean injuries using UAV-based image features collected at different flight heights. The flooding injury score (FIS) of 724 soybean breeding plots was taken visually by breeders when soybean showed obvious injury symptoms. Aerial images were taken on the same day using a five-band multispectral and an infrared (IR) thermal camera at 20, 50, and 80 m above ground. Five image features, i.e., canopy temperature, normalized difference vegetation index, canopy area, width, and length, were extracted from the images at three flight heights. A deep learning model was used to classify the soybean breeding plots to five FIS ratings based on the extracted image features. Results show that the image features were significantly different at three flight heights. The best classification performance was obtained by the model developed using image features at 20 m with 0.9 for the five-level FIS. The results indicate that the proposed method is very promising in estimating FIS for soybean breeding.

Entropy Weight Ensemble Framework for Yield Prediction of Winter Wheat Under Different Water Stress Treatments Using Unmanned Aerial Vehicle-Based Multispectral and Thermal Data

Crop breeding programs generally perform early field assessments of candidate selection based on primary traits such as grain yield (GY). The traditional methods of yield assessment are costly, inefficient, and considered a bottleneck in modern precision agriculture. Recent advances in an unmanned aerial vehicle (UAV) and development of sensors have opened a new avenue for data acquisition cost-effectively and rapidly. We evaluated UAV-based multispectral and thermal images for in-season GY prediction using 30 winter wheat genotypes under 3 water treatments. For this, multispectral vegetation indices (VIs) and normalized relative canopy temperature (NRCT) were calculated and selected by the gray relational analysis (GRA) at each growth stage, i.e., jointing, booting, heading, flowering, grain filling, and maturity to reduce the data dimension. The elastic net regression (ENR) was developed by using selected features as input variables for yield prediction, whereas the entropy weight fusion (EWF) method was used to combine the predicted GY values from multiple growth stages. In our results, the fusion of dual-sensor data showed high yield prediction accuracy [coefficient of determination (R ²) = 0.527-0.667] compared to using a single multispectral sensor (R ² = 0.130-0.461). Results showed that the grain filling stage was the optimal stage to predict GY with R ² = 0.667, root mean square error (RMSE) = 0.881 t ha^-1, relative root-mean-square error (RRMSE) = 15.2%, and mean absolute error (MAE) = 0.721 t ha^-1. The EWF model outperformed at all the individual growth stages with R ² varying from 0.677 to 0.729. The best prediction result (R ² = 0.729, RMSE = 0.831 t ha^-1, RRMSE = 14.3%, and MAE = 0.684 t ha^-1) was achieved through combining the predicted values of all growth stages. This study suggests that the fusion of UAV-based multispectral and thermal IR data within an ENR-EWF framework can provide a precise and robust prediction of wheat yield.

Performance of the Two-Source Energy Balance (TSEB) Model as a Tool for Monitoring the Response of Durum Wheat to Drought by High-Throughput Field Phenotyping

The current lack of efficient methods for high throughput field phenotyping is a constraint on the goal of increasing durum wheat yields. This study illustrates a comprehensive methodology for phenotyping this crop's water use through the use of the two-source energy balance (TSEB) model employing very high resolution imagery. An unmanned aerial vehicle (UAV) equipped with multispectral and thermal cameras was used to phenotype 19 durum wheat cultivars grown under three contrasting irrigation treatments matching crop evapotranspiration levels (ETc): 100%ETc treatment meeting all crop water requirements (450 mm), 50%ETc treatment meeting half of them (285 mm), and a rainfed treatment (122 mm). Yield reductions of 18.3 and 48.0% were recorded in the 50%ETc and rainfed treatments, respectively, in comparison with the 100%ETc treatment. UAV flights were carried out during jointing (April 4th), anthesis (April 30th), and grain-filling (May 22nd). Remotely-sensed data were used to estimate: (1) plant height from a digital surface model (H, R ² = 0.95, RMSE = 0.18m), (2) leaf area index from multispectral vegetation indices (LAI, R ² = 0.78, RMSE = 0.63), and (3) actual evapotranspiration (ETa) and transpiration (T) through the TSEB model (R ² = 0.50, RMSE = 0.24 mm/h). Compared with ground measurements, the four traits estimated at grain-filling provided a good prediction of days from sowing to heading (DH, r = 0.58-0.86), to anthesis (DA, r = 0.59-0.85) and to maturity (r = 0.67-0.95), grain-filling duration (GFD, r = 0.54-0.74), plant height (r = 0.62-0.69), number of grains per spike (NGS, r = 0.41-0.64), and thousand kernel weight (TKW, r = 0.37-0.42). The best trait to estimate yield, DH, DA, and GFD was ETa at anthesis or during grain filling. Better forecasts for yield-related traits were recorded in the irrigated treatments than in the rainfed one. These results show a promising perspective in the use of energy balance models for the phenotyping of large numbers of durum wheat genotypes under Mediterranean conditions.

Thermal Imaging for Plant Stress Detection and Phenotyping

In the last few years, large efforts have been made to develop new methods to optimize stress detection in crop fields. Thus, plant phenotyping based on imaging techniques has become an essential tool in agriculture. In particular, leaf temperature is a valuable indicator of the physiological status of plants, responding to both biotic and abiotic stressors. Often combined with other imaging sensors and data-mining techniques, thermography is crucial in the implementation of a more automatized, precise and sustainable agriculture. However, thermal data need some corrections related to the environmental and measuring conditions in order to achieve a correct interpretation of the data. This review focuses on the state of the art of thermography applied to the detection of biotic stress. The work will also revise the most important abiotic stress factors affecting the measurements as well as practical issues that need to be considered in order to implement this technique, particularly at the field scale.