Detecting Plant Stress Using Thermal and Optical Imagery From an Unoccupied Aerial Vehicle

Reviewing the essential roles of remote phenotyping, GWAS and explainable AI in practical marker-assisted selection for drought-tolerant winter wheat breeding

Marker-assisted selection (MAS) plays a crucial role in crop breeding improving the speed and precision of conventional breeding programmes by quickly and reliably identifying and selecting plants with desired traits. However, the efficacy of MAS depends on several prerequisites, with precise phenotyping being a key aspect of any plant breeding programme. Recent advancements in high-throughput remote phenotyping, facilitated by unmanned aerial vehicles coupled to machine learning, offer a non-destructive and efficient alternative to traditional, time-consuming, and labour-intensive methods. Furthermore, MAS relies on knowledge of marker-trait associations, commonly obtained through genome-wide association studies (GWAS), to understand complex traits such as drought tolerance, including yield components and phenology. However, GWAS has limitations that artificial intelligence (AI) has been shown to partially overcome. Additionally, AI and its explainable variants, which ensure transparency and interpretability, are increasingly being used as recognised problem-solving tools throughout the breeding process. Given these rapid technological advancements, this review provides an overview of state-of-the-art methods and processes underlying each MAS, from phenotyping, genotyping and association analyses to the integration of explainable AI along the entire workflow. In this context, we specifically address the challenges and importance of breeding winter wheat for greater drought tolerance with stable yields, as regional droughts during critical developmental stages pose a threat to winter wheat production. Finally, we explore the transition from scientific progress to practical implementation and discuss ways to bridge the gap between cutting-edge developments and breeders, expediting MAS-based winter wheat breeding for drought tolerance.

AI-assisted image analysis and physiological validation for progressive drought detection in a diverse panel of Gossypium hirsutum L

Introduction

Drought detection, spanning from early stress to severe conditions, plays a crucial role in maintaining productivity, facilitating recovery, and preventing plant mortality. While handheld thermal cameras have been widely employed to track changes in leaf water content and stomatal conductance, research on thermal image classification remains limited due mainly to low resolution and blurry images produced by handheld cameras.

Methods

In this study, we introduce a computer vision pipeline to enhance the significance of leaf-level thermal images across 27 distinct cotton genotypes cultivated in a greenhouse under progressive drought conditions. Our approach involved employing a customized software pipeline to process raw thermal images, generating leaf masks, and extracting a range of statistically relevant thermal features (e.g., min and max temperature, median value, quartiles, etc.). These features were then utilized to develop machine learning algorithms capable of assessing leaf hydration status and distinguishing between well-watered (WW) and dry-down (DD) conditions.

Results

Two different classifiers were trained to predict the plant treatment-random forest and multilayer perceptron neural networks-finding 75% and 78% accuracy in the treatment prediction, respectively. Furthermore, we evaluated the predicted versus true labels based on classic physiological indicators of drought in plants, including volumetric soil water content, leaf water potential, and chlorophyll a fluorescence, to provide more insights and possible explanations about the classification outputs.

Discussion

Interestingly, mislabeled leaves mostly exhibited notable responses in fluorescence, water uptake from the soil, and/or leaf hydration status. Our findings emphasize the potential of AI-assisted thermal image analysis in enhancing the informative value of common heterogeneous datasets for drought detection. This application suggests widening the experimental settings to be used with deep learning models, designing future investigations into the genotypic variation in plant drought response and potential optimization of water management in agricultural settings.

Thermal imaging: The digital eye facilitates high-throughput phenotyping traits of plant growth and stress responses

Plant phenotyping is important for plants to cope with environmental changes and ensure plant health. Imaging techniques are perceived as the most critical and reliable tools for studying plant phenotypes. Thermal imaging has opened up new opportunities for nondestructive imaging of plant phenotyping. However, a comprehensive summary of thermal imaging in plant phenotyping is still lacking. Here we discuss the progress and future prospects of thermal imaging for assessing plant growth and stress responses. First, we classify thermal imaging into ground-based and aerial platforms based on their adaptability to different experimental environments (including laboratory, greenhouse, and field). It is convenient to collect phenotypic information of different dimensions. Second, in order to enhance the efficiency of thermal image processing, automatic algorithms based on deep learning are employed instead of traditional manual methods, greatly reducing the time cost of experiments. Considering its ease of implementation, handling and instant response, thermal imaging has been widely used in research on environmental stress, crop yield, and seed vigor. We have found that thermal imaging can detect thermal energy dissipation caused by living organisms (e.g., pests, viruses, bacteria, fungi, and oomycetes), enabling early disease diagnosis. It also recognizes changes leaf surface temperatures resulting from reduced transpiration rates caused by nutrient deficiency, drought, salinity, or freezing. Furthermore, thermal imaging predicts crop yield under different water states and forecasts the viability of dormant seeds after water absorption by monitoring temperature changes in the seeds. This work will assist biologists and agronomists in studying plant phenotypes and serve a guide for breeders to develop high-yielding, stress-tolerant, and superior crops.

Field Plant Monitoring from Macro to Micro Scale: Feasibility and Validation of Combined Field Monitoring Approaches from Remote to in Vivo to Cope with Drought Stress in Tomato

Monitoring plant growth and development during cultivation to optimize resource use efficiency is crucial to achieve an increased sustainability of agriculture systems and ensure food security. In this study, we compared field monitoring approaches from the macro to micro scale with the aim of developing novel in vivo tools for field phenotyping and advancing the efficiency of drought stress detection at the field level. To this end, we tested different methodologies in the monitoring of tomato growth under different water regimes: (i) micro-scale (inserted in the plant stem) real-time monitoring with an organic electrochemical transistor (OECT)-based sensor, namely a bioristor, that enables continuous monitoring of the plant; (ii) medium-scale (<1 m from the canopy) monitoring through red-green-blue (RGB) low-cost imaging; (iii) macro-scale multispectral and thermal monitoring using an unmanned aerial vehicle (UAV). High correlations between aerial and proximal remote sensing were found with chlorophyll-related indices, although at specific time points (NDVI and NDRE with GGA and SPAD). The ion concentration and allocation monitored by the index R of the bioristor during the drought defense response were highly correlated with the water use indices (Crop Water Stress Index (CSWI), relative water content (RWC), vapor pressure deficit (VPD)). A high negative correlation was observed with the CWSI and, in turn, with the RWC. Although proximal remote sensing measurements correlated well with water stress indices, vegetation indices provide information about the crop's status at a specific moment. Meanwhile, the bioristor continuously monitors the ion movements and the correlated water use during plant growth and development, making this tool a promising device for field monitoring.

An overview of image-based phenotyping as an adaptive 4.0 technology for studying plant abiotic stress: A bibliometric and literature review

Improving the tolerance of crop species to abiotic stresses that limit plant growth and productivity is essential for mitigating the emerging problems of global warming. In this context, imaged data analysis represents an effective method in the 4.0 technology era, where this method has the non-destructive and recursive characterization of plant phenotypic traits as selection criteria. So, the plant breeders are helped in the development of adapted and climate-resilient crop varieties. Although image-based phenotyping has recently resulted in remarkable improvements for identifying the crop status under a range of growing conditions, the topic of its application for assessing the plant behavioral responses to abiotic stressors has not yet been extensively reviewed. For such a purpose, bibliometric analysis is an ideal analytical concept to analyze the evolution and interplay of image-based phenotyping to abiotic stresses by objectively reviewing the literature in light of existing database. Bibliometricy, a bibliometric analysis was applied using a systematic methodology which involved data mining, mining data improvement and analysis, and manuscript construction. The obtained results indicate that there are 554 documents related to image-based phenotyping to abiotic stress until 5 January 2023. All document showed the future development trends of image-based phenotyping will be mainly centered in the United States, European continent and China. The keywords analysis major focus to the application of 4.0 technology and machine learning in plant breeding, especially to create the tolerant variety under abiotic stresses. Drought and saline become an abiotic stress often using image-based phenotyping. Besides that, the rice, wheat and maize as the main commodities in this topic. In conclusion, the present work provides information on resolutive interactions in developing image-based phenotyping to abiotic stress, especially optimizing high-throughput sensors in image-based phenotyping for the future development.

An Open-Source Package for Thermal and Multispectral Image Analysis for Plants in Glasshouse

Advanced plant phenotyping techniques to measure biophysical traits of crops are helping to deliver improved crop varieties faster. Phenotyping of plants using different sensors for image acquisition and its analysis with novel computational algorithms are increasingly being adapted to measure plant traits. Thermal and multispectral imagery provides novel opportunities to reliably phenotype crop genotypes tested for biotic and abiotic stresses under glasshouse conditions. However, optimization for image acquisition, pre-processing, and analysis is required to correct for optical distortion, image co-registration, radiometric rescaling, and illumination correction. This study provides a computational pipeline that optimizes these issues and synchronizes image acquisition from thermal and multispectral sensors. The image processing pipeline provides a processed stacked image comprising RGB, green, red, NIR, red edge, and thermal, containing only the pixels present in the object of interest, e.g., plant canopy. These multimodal outputs in thermal and multispectral imageries of the plants can be compared and analysed mutually to provide complementary insights and develop vegetative indices effectively. This study offers digital platform and analytics to monitor early symptoms of biotic and abiotic stresses and to screen a large number of genotypes for improved growth and productivity. The pipeline is packaged as open source and is hosted online so that it can be utilized by researchers working with similar sensors for crop phenotyping.

Machine Learning-Based Crop Stress Detection in Greenhouses

Greenhouse climate control systems are usually based on greenhouse microclimate settings to exert any control. However, to save energy, water and nutrients, additional parameters related to crop performance and physiology will have to be considered. In addition, detecting crop stress before it is clearly visible by naked eye is an advantage that could aid in microclimate control. In this study, a Machine Learning (ML) model which takes into account microclimate and crop physiological data to detect different types of crop stress was developed and tested. For this purpose, a multi-sensor platform was used to record tomato plant physiological characteristics under different fertigation and air temperature conditions. The innovation of the current model lies in the integration of photosynthesis rate (Ps) values estimated by means of remote sensing using a photochemical reflectance index (PRI). Through this process, the time-series Ps data were combined with crop leaf temperature and microclimate data by means of the ML model. Two different algorithms were evaluated: Gradient Boosting (GB) and MultiLayer perceptron (MLP). Two runs with different structures took place for each algorithm. In RUN 1, there were more feature inputs than the outputs to build a model with high predictive accuracy. However, in order to simplify the process and develop a user-friendly approach, a second, different run was carried out. Thus, in RUN 2, the inputs were fewer than the outputs, and that is why the performance of the model in this case was lower than in the case of RUN 1. Particularly, MLP showed 91% and 83% accuracy in the training sample, and 89% and 82% in testing sample, for RUNs 1 and 2, respectively. GB showed 100% accuracy in the training sample for both runs, and 91% and 83% in testing sample in RUN 1 and RUN 2, respectively. To improve the accuracy of RUN 2, a larger database is required. Both models, however, could easily be incorporated into existing greenhouse climate monitoring and control systems, replacing human experience in detecting greenhouse crop stress conditions.

Real-Time Assessment of Mandarin Crop Water Stress Index

The use of plant-based indicators and other conventional means to detect the level of water stress in crops may be challenging, due to their difficulties in automation, their arduousness, and their time-consuming nature. Non-contact and non-destructive sensing methods can be used to detect the level of water stress in plants continuously and to provide automatic sensing and controls. This research aimed at determining the viability, efficiency, and swiftness in employing the commercial Workswell WIRIS Agro R infrared camera (WWARIC) in monitoring water stress and scheduling appropriate irrigation regimes in mandarin plants. The experiment used a four-by-three randomized complete block design with 80−100% FC water treatment as full field capacity and three deficit irrigation treatments at 70−75% FC, 60−65% FC, and 50−55% FC. Air temperature, canopy temperature, and vapor pressure deficits were measured and employed to deduce the empirical crop water stress index, using the Idso approach (CWSI(Idso)) as well as baseline equations to calculate non-water stress and water stressed conditions. The relative leaf water content (RLWC) of mandarin plants was also determined for the growing season. From the experiment, CWSI(Idso) and CWSI were estimated using the Workswell Wiris Agro R infrared camera (CWSIW) and showed a high correlation (R2 = 0.75 at p < 0.05) in assessing the extent of water stress in mandarin plants. The results also showed that at an altitude of 12 m above the mandarin canopy, the WWARIC was able to identify water stress using three modes (empirical, differential, and theoretical). The WWARIC’s color map feature, presented in real time, makes the camera a suitable device, as there is no need for complex computations or expert advice before determining the extent of the stress the crops are subjected to. The results prove that this novel use of the WWARIC demonstrated sufficient precision, swiftness, and intelligibility in the real-time detection of the mandarin water stress index and, accordingly, assisted in scheduling irrigation.