Maize Yield Prediction at an Early Developmental Stage Using Multispectral Images and Genotype Data for Preliminary Hybrid Selection

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

Epistasis and pleiotropy-induced variation for plant breeding

Epistasis refers to nonallelic interaction between genes that cause bias in estimates of genetic parameters for a phenotype with interactions of two or more genes affecting the same trait. Partitioning of epistatic effects allows true estimation of the genetic parameters affecting phenotypes. Multigenic variation plays a central role in the evolution of complex characteristics, among which pleiotropy, where a single gene affects several phenotypic characters, has a large influence. While pleiotropic interactions provide functional specificity, they increase the challenge of gene discovery and functional analysis. Overcoming pleiotropy-based phenotypic trade-offs offers potential for assisting breeding for complex traits. Modelling higher order nonallelic epistatic interaction, pleiotropy and non-pleiotropy-induced variation, and genotype × environment interaction in genomic selection may provide new paths to increase the productivity and stress tolerance for next generation of crop cultivars. Advances in statistical models, software and algorithm developments, and genomic research have facilitated dissecting the nature and extent of pleiotropy and epistasis. We overview emerging approaches to exploit positive (and avoid negative) epistatic and pleiotropic interactions in a plant breeding context, including developing avenues of artificial intelligence, novel exploitation of large-scale genomics and phenomics data, and involvement of genes with minor effects to analyse epistatic interactions and pleiotropic quantitative trait loci, including missing heritability.

Deep learning methods improve genomic prediction of wheat breeding

In the field of plant breeding, various machine learning models have been developed and studied to evaluate the genomic prediction (GP) accuracy of unseen phenotypes. Deep learning has shown promise. However, most studies on deep learning in plant breeding have been limited to small datasets, and only a few have explored its application in moderate-sized datasets. In this study, we aimed to address this limitation by utilizing a moderately large dataset. We examined the performance of a deep learning (DL) model and compared it with the widely used and powerful best linear unbiased prediction (GBLUP) model. The goal was to assess the GP accuracy in the context of a five-fold cross-validation strategy and when predicting complete environments using the DL model. The results revealed the DL model outperformed the GBLUP model in terms of GP accuracy for two out of the five included traits in the five-fold cross-validation strategy, with similar results in the other traits. This indicates the superiority of the DL model in predicting these specific traits. Furthermore, when predicting complete environments using the leave-one-environment-out (LOEO) approach, the DL model demonstrated competitive performance. It is worth noting that the DL model employed in this study extends a previously proposed multi-modal DL model, which had been primarily applied to image data but with small datasets. By utilizing a moderately large dataset, we were able to evaluate the performance and potential of the DL model in a context with more information and challenging scenario in plant breeding.

Unravelling inversions: Technological advances, challenges, and potential impact on crop breeding

Inversions, a type of chromosomal structural variation, significantly influence plant adaptation and gene functions by impacting gene expression and recombination rates. However, compared with other structural variations, their roles in functional biology and crop improvement remain largely unexplored. In this review, we highlight technological and methodological advancements that have allowed a comprehensive understanding of inversion variants through the pangenome framework and machine learning algorithms. Genome editing is an efficient method for inducing or reversing inversion mutations in plants, providing an effective mechanism to modify local recombination rates. Given the potential of inversions in crop breeding, we anticipate increasing attention on inversions from the scientific community in future research and breeding applications.

Plant genome resequencing and population genomics: Current status and future prospects

Advances in DNA sequencing technology have sparked a genomics revolution, driving breakthroughs in plant genetics and crop breeding. Recently, the focus has shifted from cataloging genetic diversity in plants to exploring their functional significance and delivering beneficial alleles for crop improvement. This transformation has been facilitated by the increasing adoption of whole-genome resequencing. In this review, we summarize the current progress of population-based genome resequencing studies and how these studies affect crop breeding. A total of 187 land plants from 163 countries have been resequenced, comprising 54 413 accessions. As part of resequencing efforts 367 traits have been surveyed and 86 genome-wide association studies have been conducted. Economically important crops, particularly cereals, vegetables, and legumes, have dominated the resequencing efforts, leaving a gap in 49 orders, including Lycopodiales, Liliales, Acorales, Austrobaileyales, and Commelinales. The resequenced germplasm is distributed across diverse geographic locations, providing a global perspective on plant genomics. We highlight genes that have been selected during domestication, or associated with agronomic traits, and form a repository of candidate genes for future research and application. Despite the opportunities for cross-species comparative genomics, many population genomic datasets are not accessible, impeding secondary analyses. We call for a more open and collaborative approach to population genomics that promotes data sharing and encourages contribution-based credit policy. The number of plant genome resequencing studies will continue to rise with the decreasing DNA sequencing costs, coupled with advances in analysis and computational technologies. This expansion, in terms of both scale and quality, holds promise for deeper insights into plant trait genetics and breeding design.

Multimodal deep learning methods enhance genomic prediction of wheat breeding

While several statistical machine learning methods have been developed and studied for assessing the genomic prediction (GP) accuracy of unobserved phenotypes in plant breeding research, few methods have linked genomics and phenomics (imaging). Deep learning (DL) neural networks have been developed to increase the GP accuracy of unobserved phenotypes while simultaneously accounting for the complexity of genotype-environment interaction (GE); however, unlike conventional GP models, DL has not been investigated for when genomics is linked with phenomics. In this study we used 2 wheat data sets (DS1 and DS2) to compare a novel DL method with conventional GP models. Models fitted for DS1 were GBLUP, gradient boosting machine (GBM), support vector regression (SVR) and the DL method. Results indicated that for 1 year, DL provided better GP accuracy than results obtained by the other models. However, GP accuracy obtained for other years indicated that the GBLUP model was slightly superior to the DL. DS2 is comprised only of genomic data from wheat lines tested for 3 years, 2 environments (drought and irrigated) and 2-4 traits. DS2 results showed that when predicting the irrigated environment with the drought environment, DL had higher accuracy than the GBLUP model in all analyzed traits and years. When predicting drought environment with information on the irrigated environment, the DL model and GBLUP model had similar accuracy. The DL method used in this study is novel and presents a strong degree of generalization as several modules can potentially be incorporated and concatenated to produce an output for a multi-input data structure.

Diversifying agrifood systems to ensure global food security following the Russia–Ukraine crisis

The recent Russia–Ukraine conflict has raised significant concerns about global food security, leaving many countries with restricted access to imported staple food crops, particularly wheat and sunflower oil, sending food prices soaring with other adverse consequences in the food supply chain. This detrimental effect is particularly prominent for low-income countries relying on grain imports, with record-high food prices and inflation affecting their livelihoods. This review discusses the role of Russia and Ukraine in the global food system and the impact of the Russia–Ukraine conflict on food security. It also highlights how diversifying four areas of agrifood systems—markets, production, crops, and technology can contribute to achieving food supply chain resilience for future food security and sustainability.

Monitoring of Paddy and Maize Fields Using Sentinel-1 SAR Data and NGB Images: A Case Study in Papua, Indonesia

This study addresses the question of how to evaluate the growth stage of food crops, for instance, paddy (Oryza sativa) and maize (Zea mays), from two different sensors in selected developed areas of Papua Province of Indonesia. Level-1 Ground Range Detected (L1 GRD) images from Sentinel-1 Synthetic Aperture Radar (SAR) data were used to investigate the growth of paddy and maize crops. An NGB camera was then used to obtain the Green Normalized Difference Vegetation Index (GNDVI), and the Enhanced Normalized Difference Vegetation Index (ENDVI) as in situ measurement. Afterwards, the results were analyzed based on the Radar Vegetation Index (RVI) and the Vertical-Vertical (VV) and Vertical Horizontal (VH) band backscatters at incidence angles of 30.55°–45.88°, and 30.59°–46.16° in 2021 and 2022, respectively. The findings showed that Sigma0_VV_db and sigma0_VH_db had a strong correlation (R2 above 0.900); however, polarization modification is required, specifically in the maize field. The RVI calculated and backscatter changes in this study were comparable to the in situ measurements, specifically those of paddy fields, in 2022. Even though the results of this study were not able to prove the RVI values from the two relative orbits (orbit31 and orbit155) due to the different angle incidences and the availability of the Sentinel-1 SAR data set over the study area, the division of SAR image data based on each relative orbit adequately represents the development of crops in our study areas. The significance of this study is expected to support food crop security and the implementation of development plans that contribute to the local government’s goals and settings.

Innovative Advances in Plant Genotyping

Over the past decade, advances in plant genotyping have been critical in enabling the identification of genetic diversity, in understanding evolution, and in dissecting important traits in both crops and native plants. The widespread popularity of single-nucleotide polymorphisms (SNPs) has prompted significant improvements to SNP-based genotyping, including SNP arrays, genotyping by sequencing, and whole-genome resequencing. More recent approaches, including genotyping structural variants, utilizing pangenomes to capture species-wide genetic diversity and exploiting machine learning to analyze genotypic data sets, are pushing the boundaries of what plant genotyping can offer. In this chapter, we highlight these innovations and discuss how they will accelerate and advance future genotyping efforts.

Analysis of Corn Yield Prediction Potential at Various Growth Phases Using a Process-Based Model and Deep Learning

Early and accurate prediction of grain yield is of great significance for ensuring food security and formulating food policy. The exploration of key growth phases and features is beneficial to improving the efficiency and accuracy of yield prediction. In this study, a hybrid approach using the WOFOST model and deep learning was developed to forecast corn yield, which analysed yield prediction potential at different growth phases and features. The World Food Studies (WOFOST) model was used to build a comprehensive simulated dataset by inputting meteorological, soil, crop and management data. Different feature combinations at various growth phases were designed to forecast yield using machine learning and deep learning methods. The results show that the key features of corn's vegetative growth stage and reproductive growth stage were growth state features and water-related features, respectively. With the continuous advancement of the crop growth stage, the ability to predict yield continued to improve. Especially after entering the reproductive growth stage, corn kernels begin to form, and the yield prediction performance is significantly improved. The performance of the optimal yield prediction model in flowering (R² = 0.53, RMSE = 554.84 kg/ha, MRE = 8.27%), in milk maturity (R² = 0.89, RMSE = 268.76 kg/ha, MRE = 4.01%), and in maturity (R² = 0.98, RMSE = 102.65 kg/ha, MRE = 1.53%) were given. Thus, our method improves the accuracy of yield prediction, and provides reliable analysis results for predicting yield at various growth phases, which is helpful for farmers and governments in agricultural decision making. This can also be applied to yield prediction for other crops, which is of great value to guide agricultural production.

Cumulative temporal vegetation indices from unoccupied aerial systems allow maize (Zea mays L.) hybrid yield to be estimated across environments with fewer flights

Unoccupied aerial systems (UAS) based high throughput phenotyping studies require further investigation to combine different environments and planting times into one model. Here 100 elite breeding hybrids of maize (Zea mays L.) were evaluated in two environment trials-one with optimal planting and irrigation (IHOT), and one dryland with delayed planting (DHOT). RGB (Red-Green-Blue) based canopy height measurement (CHM) and vegetation indices (VIs) were estimated from a UAS platform. Time series and cumulative VIs, by both summation (ΣVI-SUMs) and area under the curve (ΣVI-AUCs), were fit via machine learning regression modeling (random forest, linear, ridge, lasso, elastic net regressions) to estimate grain yield. VIs were more valuable predictors of yield to combine different environments than CHM. Time series VIs and CHM produced high accuracies (~68-72%), but inconsistent models. A little sacrifice in accuracy (~60-65%) produced consistent models using ΣVI-SUMs and CHM during pre-reproductive vegetative growth. Absence of VIs produced poorer accuracies (by about ~5-10%). Normalized difference type VIs produced maximum accuracies, and flowering times were the best times for UAS data acquisition. This study suggests that the best yielding varieties can be accurately predicted in new environments at or before flowering when combining multiple temporal flights and predictors.

Application of crop wild relatives in modern breeding: An overview of resources, experimental and computational methodologies

Global agricultural industries are under pressure to meet the future food demand; however, the existing crop genetic diversity might not be sufficient to meet this expectation. Advances in genome sequencing technologies and availability of reference genomes for over 300 plant species reveals the hidden genetic diversity in crop wild relatives (CWRs), which could have significant impacts in crop improvement. There are many ex-situ and in-situ resources around the world holding rare and valuable wild species, of which many carry agronomically important traits and it is crucial for users to be aware of their availability. Here we aim to explore the available ex-/in- situ resources such as genebanks, botanical gardens, national parks, conservation hotspots and inventories holding CWR accessions. In addition we highlight the advances in availability and use of CWR genomic resources, such as their contribution in pangenome construction and introducing novel genes into crops. We also discuss the potential and challenges of modern breeding experimental approaches (e.g. de novo domestication, genome editing and speed breeding) used in CWRs and the use of computational (e.g. machine learning) approaches that could speed up utilization of CWR species in breeding programs towards crop adaptability and yield improvement.

The Global Assessment of Oilseed Brassica Crop Species Yield, Yield Stability and the Underlying Genetics

The global demand for oilseeds is increasing along with the human population. The family of Brassicaceae crops are no exception, typically harvested as a valuable source of oil, rich in beneficial molecules important for human health. The global capacity for improving Brassica yield has steadily risen over the last 50 years, with the major crop Brassica napus (rapeseed, canola) production increasing to ~72 Gt in 2020. In contrast, the production of Brassica mustard crops has fluctuated, rarely improving in farming efficiency. The drastic increase in global yield of B. napus is largely due to the demand for a stable source of cooking oil. Furthermore, with the adoption of highly efficient farming techniques, yield enhancement programs, breeding programs, the integration of high-throughput phenotyping technology and establishing the underlying genetics, B. napus yields have increased by >450 fold since 1978. Yield stability has been improved with new management strategies targeting diseases and pests, as well as by understanding the complex interaction of environment, phenotype and genotype. This review assesses the global yield and yield stability of agriculturally important oilseed Brassica species and discusses how contemporary farming and genetic techniques have driven improvements.

Estimation of Maize Yield and Flowering Time Using Multi-Temporal UAV-Based Hyperspectral Data

Maize (Zea mays L.) is one of the most consumed grains in the world. Within the context of continuous climate change and the reduced availability of arable land, it is urgent to breed new maize varieties and screen for the desired traits, e.g., high yield and strong stress tolerance. Traditional phenotyping methods relying on manual assessment are time-consuming and prone to human errors. Recently, the application of uncrewed aerial vehicles (UAVs) has gained increasing attention in plant phenotyping due to their efficiency in data collection. Moreover, hyperspectral sensors integrated with UAVs can offer data streams with high spectral and spatial resolutions, which are valuable for estimating plant traits. In this study, we collected UAV hyperspectral imagery over a maize breeding field biweekly across the growing season, resulting in 11 data collections in total. Multiple machine learning models were developed to estimate the grain yield and flowering time of the maize breeding lines using the hyperspectral imagery. The performance of the machine learning models and the efficacy of different hyperspectral features were evaluated. The results showed that the models with the multi-temporal imagery outperformed those with imagery from single data collections, and the ridge regression using the full band reflectance achieved the best estimation accuracies, with the correlation coefficients (r) between the estimates and ground truth of 0.54 for grain yield, 0.91 for days to silking, and 0.92 for days to anthesis. In addition, we assessed the estimation performance with data acquired at different growth stages to identify the good periods for the UAV survey. The best estimation results were achieved using the data collected around the tasseling stage (VT) for the grain yield estimation and around the reproductive stages (R1 or R4) for the flowering time estimation. Our results showed that the robust phenotyping framework proposed in this study has great potential to help breeders efficiently estimate key agronomic traits at early growth stages.

Advancing designer crops for climate resilience through an integrated genomics approach

Climate change and exponential population growth are exposing an immediate need for developing future crops that are highly resilient and adaptable to changing environments to maintain global food security in the next decade. Rigorous selection from long domestication history has rendered cultivated crops genetically disadvantaged, raising concerns in their ability to adapt to these new challenges and limiting their usefulness in breeding programmes. As a result, future crop improvement efforts must rely on integrating various genomic strategies ranging from high-throughput sequencing to machine learning, in order to exploit germplasm diversity and overcome bottlenecks created by domestication, expansive multi-dimensional phenotypes, arduous breeding processes, complex traits and big data.

Phenomic data-facilitated rust and senescence prediction in maize using machine learning algorithms

Current methods in measuring maize (Zea mays L.) southern rust (Puccinia polyspora Underw.) and subsequent crop senescence require expert observation and are resource-intensive and prone to subjectivity. In this study, unoccupied aerial system (UAS) field-based high-throughput phenotyping (HTP) was employed to collect high-resolution aerial imagery of elite maize hybrids planted in the 2020 and 2021 growing seasons, with 13 UAS flights obtained from 2020 and 17 from 2021. In total, 36 vegetation indices (VIs) were extracted from mosaicked aerial images that served as temporal phenomic predictors for southern rust scored in the field and senescence as scored using UAS-acquired mosaic images. Temporal best linear unbiased predictors (TBLUPs) were calculated using a nested model that treated hybrid performance as nested within flights in terms of rust and senescence. All eight machine learning regressions tested (ridge, lasso, elastic net, random forest, support vector machine with radial and linear kernels, partial least squares, and k-nearest neighbors) outperformed a general linear model with both higher prediction accuracies (92-98%) and lower root mean squared error (RMSE) for rust and senescence scores (linear model RMSE ranged from 65.8 to 2396.5 across all traits, machine learning regressions RMSE ranged from 0.3 to 17.0). UAS-acquired VIs enabled the discovery of novel early quantitative phenotypic indicators of maize senescence and southern rust before being detectable by expert annotation and revealed positive correlations between grain filling time and yield (0.22 and 0.44 in 2020 and 2021), with practical implications for precision agricultural practices.

A Systematic Literature Review on Crop Yield Prediction with Deep Learning and Remote Sensing

Deep learning has emerged as a potential tool for crop yield prediction, allowing the model to automatically extract features and learn from the datasets. Meanwhile, smart farming technology enables the farmers to achieve maximum crop yield by extracting essential parameters of crop growth. This systematic literature review highlights the existing research gaps in a particular area of deep learning methodologies and guides us in analyzing the impact of vegetation indices and environmental factors on crop yield. To achieve the aims of this study, prior studies from 2012 to 2022 from various databases are collected and analyzed. The study focuses on the advantages of using deep learning in crop yield prediction, the suitable remote sensing technology based on the data acquisition requirements, and the various features that influence crop yield prediction. This study finds that Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN) are the most widely used deep learning approaches for crop yield prediction. The commonly used remote sensing technology is satellite remote sensing technology—in particular, the use of the Moderate-Resolution Imaging Spectroradiometer (MODIS). Findings show that vegetation indices are the most used feature for crop yield prediction. However, it is also observed that the most used features in the literature do not always work for all the approaches. The main challenges of using deep learning approaches and remote sensing for crop yield prediction are how to improve the working model for better accuracy, the practical implication of the model for providing accurate information about crop yield to agriculturalists, growers, and policymakers, and the issue with the black box property.

Artificial intelligence framework for modeling and predicting crop yield to enhance food security in Saudi Arabia

Predicting crop yields is a critical issue in agricultural production optimization and intensification research. Accurate foresights of natural circumstances a year in advance can have a considerable impact on management decisions regarding crop selection, rotational location in crop rotations, agrotechnical methods employed, and long-term land use planning. One of the most important aspects of precision farming is sustainability. The novelty of this study is to evidence the effective of the temperature, pesticides, and rainfall environment parameters in the influence sustainable agriculture and economic efficiency at the farm level in Saudi Arabia. Furthermore, predicting the future values of main crop yield in Saudi Arabia. The use of artificial intelligence (AI) to estimate the impact of environment factors and agrotechnical parameters on agricultural crop yields and to anticipate yields is examined in this study. Using artificial neural networks (ANNs), a highly effective multilayer perceptron (MLP) model was built to accurately predict the crop yield, temperature, insecticides, and rainfall based on environmental data. The dataset is collected from different Saudi Arabia regions from 1994 to 2016, including the temperature, insecticides, rainfall, and crop yields for potatoes, rice, sorghum, and wheat. For this study, we relied on five different statistical evaluation metrics: the mean square error (MSE), the root-mean-square error (RMSE), normalized root mean square error (NRMSE), Pearson's correlation coefficient (R%), and the determination coefficient (R ²). Analyses of datasets for crop yields, temperature, and insecticides led to the development of the MLP models. The datasets are randomly divided into separate samples, 70% for training and 30% for testing. The best-performing MLP model is characterized by values of (R = 100%) and (R ² = 96.33) for predicting insecticides in the testing process. The temperature, insecticides, and rainfall were examined with different crop yields to confirm the effectiveness of these parameters for increasing product crop yields in Saudi Arabia; we found that these items had highest relationships. The average values are R = 98.20%, 96.50, and 99.14% with for the temperature, insecticides, and rainfall, respectively. Based on these findings, it appeared that each of the parameter categories that are considered (temperature, pesticides, and rainfall) had a similar contribution to the accuracy of anticipated yield projection.