Greenotyper: Image-Based Plant Phenotyping Using Distributed Computing and Deep Learning

Cyber-agricultural systems for crop breeding and sustainable production

The cyber-agricultural system (CAS) represents an overarching framework of agriculture that leverages recent advances in ubiquitous sensing, artificial intelligence, smart actuators, and scalable cyberinfrastructure (CI) in both breeding and production agriculture. We discuss the recent progress and perspective of the three fundamental components of CAS - sensing, modeling, and actuation - and the emerging concept of agricultural digital twins (DTs). We also discuss how scalable CI is becoming a key enabler of smart agriculture. In this review we shed light on the significance of CAS in revolutionizing crop breeding and production by enhancing efficiency, productivity, sustainability, and resilience to changing climate. Finally, we identify underexplored and promising future directions for CAS research and development.

Artificial intelligence-driven systems engineering for next-generation plant-derived biopharmaceuticals

Recombinant biopharmaceuticals including antigens, antibodies, hormones, cytokines, single-chain variable fragments, and peptides have been used as vaccines, diagnostics and therapeutics. Plant molecular pharming is a robust platform that uses plants as an expression system to produce simple and complex recombinant biopharmaceuticals on a large scale. Plant system has several advantages over other host systems such as humanized expression, glycosylation, scalability, reduced risk of human or animal pathogenic contaminants, rapid and cost-effective production. Despite many advantages, the expression of recombinant proteins in plant system is hindered by some factors such as non-human post-translational modifications, protein misfolding, conformation changes and instability. Artificial intelligence (AI) plays a vital role in various fields of biotechnology and in the aspect of plant molecular pharming, a significant increase in yield and stability can be achieved with the intervention of AI-based multi-approach to overcome the hindrance factors. Current limitations of plant-based recombinant biopharmaceutical production can be circumvented with the aid of synthetic biology tools and AI algorithms in plant-based glycan engineering for protein folding, stability, viability, catalytic activity and organelle targeting. The AI models, including but not limited to, neural network, support vector machines, linear regression, Gaussian process and regressor ensemble, work by predicting the training and experimental data sets to design and validate the protein structures thereby optimizing properties such as thermostability, catalytic activity, antibody affinity, and protein folding. This review focuses on, integrating systems engineering approaches and AI-based machine learning and deep learning algorithms in protein engineering and host engineering to augment protein production in plant systems to meet the ever-expanding therapeutics market.

Image-based phenotyping of seed architectural traits and prediction of seed weight using machine learning models in soybean

Among seed attributes, weight is one of the main factors determining the soybean harvest index. Recently, the focus of soybean breeding has shifted to improving seed size and weight for crop optimization in terms of seed and oil yield. With recent technological advancements, there is an increasing application of imaging sensors that provide simple, real-time, non-destructive, and inexpensive image data for rapid image-based prediction of seed traits in plant breeding programs. The present work is related to digital image analysis of seed traits for the prediction of hundred-seed weight (HSW) in soybean. The image-based seed architectural traits (i-traits) measured were area size (AS), perimeter length (PL), length (L), width (W), length-to-width ratio (LWR), intersection of length and width (IS), seed circularity (CS), and distance between IS and CG (DS). The phenotypic investigation revealed significant genetic variability among 164 soybean genotypes for both i-traits and manually measured seed weight. Seven popular machine learning (ML) algorithms, namely Simple Linear Regression (SLR), Multiple Linear Regression (MLR), Random Forest (RF), Support Vector Regression (SVR), LASSO Regression (LR), Ridge Regression (RR), and Elastic Net Regression (EN), were used to create models that can predict the weight of soybean seeds based on the image-based novel features derived from the Red-Green-Blue (RGB)/visual image. Among the models, random forest and multiple linear regression models that use multiple explanatory variables related to seed size traits (AS, L, W, and DS) were identified as the best models for predicting seed weight with the highest prediction accuracy (coefficient of determination, R2=0.98 and 0.94, respectively) and the lowest prediction error, i.e., root mean square error (RMSE) and mean absolute error (MAE). Finally, principal components analysis (PCA) and a hierarchical clustering approach were used to identify IC538070 as a superior genotype with a larger seed size and weight. The identified donors/traits can potentially be used in soybean improvement programs.

A Synthetic Review of Various Dimensions of Non-Destructive Plant Stress Phenotyping

Non-destructive plant stress phenotyping begins with traditional one-dimensional (1D) spectroscopy, followed by two-dimensional (2D) imaging, three-dimensional (3D) or even temporal-three-dimensional (T-3D), spectral-three-dimensional (S-3D), and temporal-spectral-three-dimensional (TS-3D) phenotyping, all of which are aimed at observing subtle changes in plants under stress. However, a comprehensive review that covers all these dimensional types of phenotyping, ordered in a spatial arrangement from 1D to 3D, as well as temporal and spectral dimensions, is lacking. In this review, we look back to the development of data-acquiring techniques for various dimensions of plant stress phenotyping (1D spectroscopy, 2D imaging, 3D phenotyping), as well as their corresponding data-analyzing pipelines (mathematical analysis, machine learning, or deep learning), and look forward to the trends and challenges of high-performance multi-dimension (integrated spatial, temporal, and spectral) phenotyping demands. We hope this article can serve as a reference for implementing various dimensions of non-destructive plant stress phenotyping.

PhytoOracle: Scalable, modular phenomics data processing pipelines

As phenomics data volume and dimensionality increase due to advancements in sensor technology, there is an urgent need to develop and implement scalable data processing pipelines. Current phenomics data processing pipelines lack modularity, extensibility, and processing distribution across sensor modalities and phenotyping platforms. To address these challenges, we developed PhytoOracle (PO), a suite of modular, scalable pipelines for processing large volumes of field phenomics RGB, thermal, PSII chlorophyll fluorescence 2D images, and 3D point clouds. PhytoOracle aims to (i) improve data processing efficiency; (ii) provide an extensible, reproducible computing framework; and (iii) enable data fusion of multi-modal phenomics data. PhytoOracle integrates open-source distributed computing frameworks for parallel processing on high-performance computing, cloud, and local computing environments. Each pipeline component is available as a standalone container, providing transferability, extensibility, and reproducibility. The PO pipeline extracts and associates individual plant traits across sensor modalities and collection time points, representing a unique multi-system approach to addressing the genotype-phenotype gap. To date, PO supports lettuce and sorghum phenotypic trait extraction, with a goal of widening the range of supported species in the future. At the maximum number of cores tested in this study (1,024 cores), PO processing times were: 235 minutes for 9,270 RGB images (140.7 GB), 235 minutes for 9,270 thermal images (5.4 GB), and 13 minutes for 39,678 PSII images (86.2 GB). These processing times represent end-to-end processing, from raw data to fully processed numerical phenotypic trait data. Repeatability values of 0.39-0.95 (bounding area), 0.81-0.95 (axis-aligned bounding volume), 0.79-0.94 (oriented bounding volume), 0.83-0.95 (plant height), and 0.81-0.95 (number of points) were observed in Field Scanalyzer data. We also show the ability of PO to process drone data with a repeatability of 0.55-0.95 (bounding area).

A method of cotton root segmentation based on edge devices

The root is an important organ for plants to absorb water and nutrients. In situ root research method is an intuitive method to explore root phenotype and its change dynamics. At present, in situ root research, roots can be accurately extracted from in situ root images, but there are still problems such as low analysis efficiency, high acquisition cost, and difficult deployment of image acquisition devices outdoors. Therefore, this study designed a precise extraction method of in situ roots based on semantic segmentation model and edge device deployment. It initially proposes two data expansion methods, pixel by pixel and equal proportion, expand 100 original images to 1600 and 53193 respectively. It then presents an improved DeeplabV3+ root segmentation model based on CBAM and ASPP in series is designed, and the segmentation accuracy is 93.01%. The root phenotype parameters were verified through the Rhizo Vision Explorers platform, and the root length error was 0.669%, and the root diameter error was 1.003%. It afterwards designs a time-saving Fast prediction strategy. Compared with the Normal prediction strategy, the time consumption is reduced by 22.71% on GPU and 36.85% in raspberry pie. It ultimately deploys the model to Raspberry Pie, realizing the low-cost and portable root image acquisition and segmentation, which is conducive to outdoor deployment. In addition, the cost accounting is only $247. It takes 8 hours to perform image acquisition and segmentation tasks, and the power consumption is as low as 0.051kWh. In conclusion, the method proposed in this study has good performance in model accuracy, economic cost, energy consumption, etc. This paper realizes low-cost and high-precision segmentation of in-situ root based on edge equipment, which provides new insights for high-throughput field research and application of in-situ root.

Open Hardware in Science: The Benefits of Open Electronics

Openly shared low-cost electronic hardware applications, known as open electronics, have sparked a new open-source movement, with much untapped potential to advance scientific research. Initially designed to appeal to electronic hobbyists, open electronics have formed a global “maker” community and are increasingly used in science and industry. In this perspective article, we review the current costs and benefits of open electronics for use in scientific research ranging from the experimental to the theoretical sciences. We discuss how user-made electronic applications can help (I) individual researchers, by increasing the customization, efficiency, and scalability of experiments, while improving data quantity and quality; (II) scientific institutions, by improving access to customizable high-end technologies, sustainability, visibility, and interdisciplinary collaboration potential; and (III) the scientific community, by improving transparency and reproducibility, helping decouple research capacity from funding, increasing innovation, and improving collaboration potential among researchers and the public. We further discuss how current barriers like poor awareness, knowledge access, and time investments can be resolved by increased documentation and collaboration, and provide guidelines for academics to enter this emerging field. We highlight that open electronics are a promising and powerful tool to help scientific research to become more innovative and reproducible and offer a key practical solution to improve democratic access to science.

Machine Learning for Plant Stress Modeling: A Perspective towards Hormesis Management

Plant stress is one of the most significant factors affecting plant fitness and, consequently, food production. However, plant stress may also be profitable since it behaves hormetically; at low doses, it stimulates positive traits in crops, such as the synthesis of specialized metabolites and additional stress tolerance. The controlled exposure of crops to low doses of stressors is therefore called hormesis management, and it is a promising method to increase crop productivity and quality. Nevertheless, hormesis management has severe limitations derived from the complexity of plant physiological responses to stress. Many technological advances assist plant stress science in overcoming such limitations, which results in extensive datasets originating from the multiple layers of the plant defensive response. For that reason, artificial intelligence tools, particularly Machine Learning (ML) and Deep Learning (DL), have become crucial for processing and interpreting data to accurately model plant stress responses such as genomic variation, gene and protein expression, and metabolite biosynthesis. In this review, we discuss the most recent ML and DL applications in plant stress science, focusing on their potential for improving the development of hormesis management protocols.

Development of a high-throughput field phenotyping rover optimized for size-limited breeding fields as open-source hardware

Phenotyping is a critical process in plant breeding, especially when there is an increasing demand for streamlining a selection process in a breeding program. Since manual phenotyping has limited efficiency, high-throughput phenotyping methods are recently popularized owing to progress in sensor and image processing technologies. However, in a size-limited breeding field, which is common in Japan and other Asian countries, it is challenging to introduce large machinery in the field or fly unmanned aerial vehicles over the field. In this study, we developed a ground-based high-throughput field phenotyping rover that could be easily introduced to a field regardless of the scale and location of the field even without special facilities. We also made the field rover open-source hardware, making its system available to public for easy modification, so that anyone can build one for their own use at a low cost. The trial run of the field rover revealed that it allowed the collection of detailed remote-sensing images of plants and quantitative analyses based on the images. The results suggest that the field rover developed in this study could allow efficient phenotyping of plants especially in a small breeding field.

Major effect loci for plant size before onset of nitrogen fixation allow accurate prediction of yield in white clover

Accurate genomic prediction of yield within and across generations was achieved by estimating the genetic merit of individual white clover genotypes based on extensive genetic replication using cloned material. White clover is an agriculturally important forage legume grown throughout temperate regions as a mixed clover-grass crop. It is typically cultivated with low nitrogen input, making yield dependent on nitrogen fixation by rhizobia in root nodules. Here, we investigate the effects of clover and rhizobium genetic variation by monitoring plant growth and quantifying dry matter yield of 704 combinations of 145 clover genotypes and 170 rhizobium inocula. We find no significant effect of rhizobium variation. In contrast, we can predict yield based on a few white clover markers strongly associated with plant size prior to nitrogen fixation, and the prediction accuracy for polycross offspring yield is remarkably high. Several of the markers are located near a homolog of Arabidopsis thaliana GIGANTUS 1, which regulates growth rate and biomass accumulation. Our work provides fundamental insight into the genetics of white clover yield and identifies specific candidate genes as breeding targets.

CBM: An IoT Enabled LiDAR Sensor for In-Field Crop Height and Biomass Measurements

The phenotypic characterization of crop genotypes is an essential, yet challenging, aspect of crop management and agriculture research. Digital sensing technologies are rapidly advancing plant phenotyping and speeding-up crop breeding outcomes. However, off-the-shelf sensors might not be fully applicable and suitable for agricultural research due to the diversity in crop species and specific needs during plant breeding selections. Customized sensing systems with specialized sensor hardware and software architecture provide a powerful and low-cost solution. This study designed and developed a fully integrated Raspberry Pi-based LiDAR sensor named CropBioMass (CBM), enabled by internet of things to provide a complete end-to-end pipeline. The CBM is a low-cost sensor, provides high-throughput seamless data collection in field, small data footprint, injection of data onto the remote server, and automated data processing. The phenotypic traits of crop fresh biomass, dry biomass, and plant height that were estimated by CBM data had high correlation with ground truth manual measurements in a wheat field trial. The CBM is readily applicable for high-throughput plant phenotyping, crop monitoring, and management for precision agricultural applications.

Competition, Nodule Occupancy, and Persistence of Inoculant Strains: Key Factors in the Rhizobium-Legume Symbioses

Biological nitrogen fixation by Rhizobium-legume symbioses represents an environmentally friendly and inexpensive alternative to the use of chemical nitrogen fertilizers in legume crops. Rhizobial inoculants, applied frequently as biofertilizers, play an important role in sustainable agriculture. However, inoculants often fail to compete for nodule occupancy against native rhizobia with inferior nitrogen-fixing abilities, resulting in low yields. Strains with excellent performance under controlled conditions are typically selected as inoculants, but the rates of nodule occupancy compared to native strains are rarely investigated. Lack of persistence in the field after agricultural cycles, usually due to the transfer of symbiotic genes from the inoculant strain to naturalized populations, also limits the suitability of commercial inoculants. When rhizobial inoculants are based on native strains with a high nitrogen fixation ability, they often have superior performance in the field due to their genetic adaptations to the local environment. Therefore, knowledge from laboratory studies assessing competition and understanding how diverse strains of rhizobia behave, together with assays done under field conditions, may allow us to exploit the effectiveness of native populations selected as elite strains and to breed specific host cultivar-rhizobial strain combinations. Here, we review current knowledge at the molecular level on competition for nodulation and the advances in molecular tools for assessing competitiveness. We then describe ongoing approaches for inoculant development based on native strains and emphasize future perspectives and applications using a multidisciplinary approach to ensure optimal performance of both symbiotic partners.

Development of a Raspberry Pi-Based Sensor System for Automated In-Field Monitoring to Support Crop Breeding Programs

Sensor applications for plant phenotyping can advance and strengthen crop breeding programs. One of the powerful sensing options is the automated sensor system, which can be customized and applied for plant science research. The system can provide high spatial and temporal resolution data to delineate crop interaction with weather changes in a diverse environment. Such a system can be integrated with the internet to enable the internet of things (IoT)-based sensor system development for real-time crop monitoring and management. In this study, the Raspberry Pi-based sensor (imaging) system was fabricated and integrated with a microclimate sensor to evaluate crop growth in a spring wheat breeding trial for automated phenotyping applications. Such an in-field sensor system will increase the reproducibility of measurements and improve the selection efficiency by investigating dynamic crop responses as well as identifying key growth stages (e.g., heading), assisting in the development of high-performing crop varieties. In the low-cost system developed here-in, a Raspberry Pi computer and multiple cameras (RGB and multispectral) were the main components. The system was programmed to automatically capture and manage the crop image data at user-defined time points throughout the season. The acquired images were suitable for extracting quantifiable plant traits, and the images were automatically processed through a Python script (an open-source programming language) to extract vegetation indices, representing crop growth and overall health. Ongoing efforts are conducted towards integrating the sensor system for real-time data monitoring via the internet that will allow plant breeders to monitor multiple trials for timely crop management and decision making.