Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry

Development of a deep-learning phenotyping tool for analyzing image-based strawberry phenotypes

Introduction

In strawberry farming, phenotypic traits (such as crown diameter, petiole length, plant height, flower, leaf, and fruit size) measurement is essential as it serves as a decision-making tool for plant monitoring and management. To date, strawberry plant phenotyping has relied on traditional approaches. In this study, an image-based Strawberry Phenotyping Tool (SPT) was developed using two deep-learning (DL) architectures, namely "YOLOv4" and "U-net" integrated into a single system. We aimed to create the most suitable DL-based tool with enhanced robustness to facilitate digital strawberry plant phenotyping directly at the natural scene or indirectly using captured and stored images.

Methods

Our SPT was developed primarily through two steps (subsequently called versions) using image data with different backgrounds captured with simple smartphone cameras. The two versions (V1 and V2) were developed using the same DL networks but differed by the amount of image data and annotation method used during their development. For V1, 7,116 images were annotated using the single-target non-labeling method, whereas for V2, 7,850 images were annotated using the multitarget labeling method.

Results

The results of the held-out dataset revealed that the developed SPT facilitates strawberry phenotype measurements. By increasing the dataset size combined with multitarget labeling annotation, the detection accuracy of our system changed from 60.24% in V1 to 82.28% in V2. During the validation process, the system was evaluated using 70 images per phenotype and their corresponding actual values. The correlation coefficients and detection frequencies were higher for V2 than for V1, confirming the superiority of V2. Furthermore, an image-based regression model was developed to predict the fresh weight of strawberries based on the fruit size (R2 = 0.92).

Discussion

The results demonstrate the efficiency of our system in recognizing the aforementioned six strawberry phenotypic traits regardless of the complex scenario of the environment of the strawberry plant. This tool could help farmers and researchers make accurate and efficient decisions related to strawberry plant management, possibly causing increased productivity and yield potential.

A novel method for vegetable and fruit classification based on using diffusion maps and machine learning

Vegetable and fruit classification can help all links of agricultural product circulation to better carry out inventory management, logistics planning and supply chain coordination, and improve the efficiency and response speed of the supply chain. However, the current classification of vegetables and fruits mainly relies on manual classification, which inevitably introduces the influence of human subjective factors, resulting in errors and misjudgments in the classification of vegetables and fruits. In response to this serious problem, this research proposes an efficient and reproducible novel model to classify multiple vegetables and fruits using handcrafted features. In the proposed model, preprocessing operations such as Gaussian filtering, grayscale and binarization are performed on the pictures of vegetables and fruits to improve the quality of the pictures; statistical texture features representing vegetable and fruit categories, wavelet transform features and shape features are extracted from the preprocessed images; the feature dimension reduction method of diffusion maps is used to reduce the redundant information of the combined features composed of statistical texture features, wavelet transform features and shape features; five effective machine learning methods were used to classify the types of vegetables and fruits. In this research, the proposed method was rigorously verified experimentally and the results show that the SVM classifier achieves 96.25% classification accuracy of vegetables and fruits, which proves that the proposed method is helpful to improve the quality and management level of vegetables and fruits, and provide strong support for agricultural production and supply chain.

Molecular bases of strawberry fruit quality traits: Advances, challenges, and opportunities

The strawberry is one of the world's most popular fruits, providing humans with vitamins, fibers, and antioxidants. Cultivated strawberry (Fragaria × ananassa) is an allo-octoploid and highly heterozygous, making it a challenge for breeding, quantitative trait locus (QTL) mapping, and gene discovery. Some wild strawberry relatives, such as Fragaria vesca, have diploid genomes and are becoming laboratory models for the cultivated strawberry. Recent advances in genome sequencing and CRISPR-mediated genome editing have greatly improved the understanding of various aspects of strawberry growth and development in both cultivated and wild strawberries. This review focuses on fruit quality traits that are most relevant to the consumers, including fruit aroma, sweetness, color, firmness, and shape. Recently available phased-haplotype genomes, single nucleotide polymorphism (SNP) arrays, extensive fruit transcriptomes, and other big data have made it possible to locate key genomic regions or pinpoint specific genes that underlie volatile synthesis, anthocyanin accumulation for fruit color, and sweetness intensity or perception. These new advances will greatly facilitate marker-assisted breeding, the introgression of missing genes into modern varieties, and precise genome editing of selected genes and pathways. Strawberries are poised to benefit from these recent advances, providing consumers with fruit that is tastier, longer-lasting, healthier, and more beautiful.

Image-Based High-Throughput Phenotyping in Horticultural Crops

Plant phenotyping is the primary task of any plant breeding program, and accurate measurement of plant traits is essential to select genotypes with better quality, high yield, and climate resilience. The majority of currently used phenotyping techniques are destructive and time-consuming. Recently, the development of various sensors and imaging platforms for rapid and efficient quantitative measurement of plant traits has become the mainstream approach in plant phenotyping studies. Here, we reviewed the trends of image-based high-throughput phenotyping methods applied to horticultural crops. High-throughput phenotyping is carried out using various types of imaging platforms developed for indoor or field conditions. We highlighted the applications of different imaging platforms in the horticulture sector with their advantages and limitations. Furthermore, the principles and applications of commonly used imaging techniques, visible light (RGB) imaging, thermal imaging, chlorophyll fluorescence, hyperspectral imaging, and tomographic imaging for high-throughput plant phenotyping, are discussed. High-throughput phenotyping has been widely used for phenotyping various horticultural traits, which can be morphological, physiological, biochemical, yield, biotic, and abiotic stress responses. Moreover, the ability of high-throughput phenotyping with the help of various optical sensors will lead to the discovery of new phenotypic traits which need to be explored in the future. We summarized the applications of image analysis for the quantitative evaluation of various traits with several examples of horticultural crops in the literature. Finally, we summarized the current trend of high-throughput phenotyping in horticultural crops and highlighted future perspectives.

Using phenomics to identify and integrate traits of interest for better-performing common beans: A validation study on an interspecific hybrid and its Acutifolii parents

INTRODUCTION

Evaluations of interspecific hybrids are limited, as classical genebank accession descriptors are semi-subjective, have qualitative traits and show complications when evaluating intermediate accessions. However, descriptors can be quantified using recognized phenomic traits. This digitalization can identify phenomic traits which correspond to the percentage of parental descriptors remaining expressed/visible/measurable in the particular interspecific hybrid. In this study, a line of P. vulgaris, P. acutifolius and P. parvifolius accessions and their crosses were sown in the mesh house according to CIAT seed regeneration procedures.

METHODOLOGY

Three accessions and one derived breeding line originating from their interspecific crosses were characterized and classified by selected phenomic descriptors using multivariate and machine learning techniques. The phenomic proportions of the interspecific hybrid (line INB 47) with respect to its three parent accessions were determined using a random forest and a respective confusion matrix.

RESULTS

The seed and pod morphometric traits, physiological behavior and yield performance were evaluated. In the classification of the accession, the phenomic descriptors with highest prediction force were Fm', Fo', Fs', LTD, Chl, seed area, seed height, seed Major, seed MinFeret, seed Minor, pod AR, pod Feret, pod round, pod solidity, pod area, pod major, pod seed weight and pod weight. Physiological traits measured in the interspecific hybrid present 2.2% similarity with the P. acutifolius and 1% with the P. parvifolius accessions. In addition, in seed morphometric characteristics, the hybrid showed 4.5% similarity with the P. acutifolius accession.

CONCLUSIONS

Here we were able to determine the phenomic proportions of individual parents in their interspecific hybrid accession. After some careful generalization the methodology can be used to: i) verify trait-of-interest transfer from P. acutifolius and P. parvifolius accessions into their hybrids; ii) confirm selected traits as "phenomic markers" which would allow conserving desired physiological traits of exotic parental accessions, without losing key seed characteristics from elite common bean accessions; and iii) propose a quantitative tool that helps genebank curators and breeders to make better-informed decisions based on quantitative analysis.

Towards smart and sustainable development of modern berry cultivars in Europe

Fresh berries are a popular and important component of the human diet. The demand for high-quality berries and sustainable production methods is increasing globally, challenging breeders to develop modern berry cultivars that fulfill all desired characteristics. Since 1994, research projects have characterized genetic resources, developed modern tools for high-throughput screening, and published data in publicly available repositories. However, the key findings of different disciplines are rarely linked together, and only a limited range of traits and genotypes has been investigated. The Horizon2020 project BreedingValue will address these challenges by studying a broader panel of strawberry, raspberry and blueberry genotypes in detail, in order to recover the lost genetic diversity that has limited the aroma and flavor intensity of recent cultivars. We will combine metabolic analysis with sensory panel tests and surveys to identify the key components of taste, flavor and aroma in berries across Europe, leading to a high-resolution map of quality requirements for future berry cultivars. Traits linked to berry yields and the effect of environmental stress will be investigated using modern image analysis methods and modeling. We will also use genetic analysis to determine the genetic basis of complex traits for the development and optimization of modern breeding technologies, such as molecular marker arrays, genomic selection and genome-wide association studies. Finally, the results, raw data and metadata will be made publicly available on the open platform Germinate in order to meet FAIR data principles and provide the basis for sustainable research in the future.

Symptom recognition of disease and insect damage based on Mask R-CNN, wavelet transform, and F-RNet

Brown blight, target spot, and tea coal diseases are three major leaf diseases of tea plants, and Apolygus lucorum is a major pest in tea plantations. The traditional symptom recognition of tea leaf diseases and insect pests is mainly through manual identification, which has some problems, such as low accuracy, low efficiency, strong subjectivity, and so on. Therefore, it is very necessary to find a method that could effectively identify tea plants diseases and pests. In this study, we proposed a recognition framework of tea leaf disease and insect pest symptoms based on Mask R-CNN, wavelet transform and F-RNet. First, Mask R-CNN model was used to segment disease spots and insect spots from tea leaves. Second, the two-dimensional discrete wavelet transform was used to enhance the features of the disease spots and insect spots images, so as to obtain the images with four frequencies. Finally, the images of four frequencies were simultaneously input into the four-channeled residual network (F-RNet) to identify symptoms of tea leaf diseases and insect pests. The results showed that Mask R-CNN model could detect 98.7% of DSIS, which ensure that almost disease spots and insect spots can be extracted from leaves. The accuracy of F-RNet model is 88%, which is higher than that of the other models (like SVM, AlexNet, VGG16 and ResNet18). Therefore, this experimental framework can accurately segment and identify diseases and insect spots of tea leaves, which not only of great significance for the accurate identification of tea plant diseases and insect pests, but also of great value for further using artificial intelligence to carry out the comprehensive control of tea plant diseases and insect pests.

Genetic and phenotypic analyses reveal major quantitative loci associated to fruit size and shape traits in a non-flat peach collection (P. persica L. Batsch)

Fruit size and shape are critical agronomical and pomological attributes and prime targets in peach breeding programs. Apart from the flat peach type, a Mendelian trait well-characterized at the genetic level, ample diversity of fruit size and shapes is present across peach germplasms. Nevertheless, knowledge of the underlying genomic loci remains limited. In this work, fruit size and shape were assessed in a collection of non-flat peach accessions and selections, under controlled fruit load conditions. The architecture of these traits was then dissected by combining association and linkage mapping, revealing a major locus on the proximal end of chromosome 6 (qSHL/Fs6.1) explaining a large proportion of phenotypic variability for longitudinal shape and also affecting fruit size. A second major locus for fruit longitudinal shape (qSHL5.1), probably also affecting fruit size, was found co-localizing at locus G, suggesting pleiotropic effects of peach/nectarine traits. An additional QTL for fruit longitudinal shape (qSHL6.2) was identified in the distal end of chromosome 6 in a cross with an ornamental double-flower peach and co-localized with the Di2 locus, controlling flower morphology. Besides assisting breeding activities, knowledge of loci controlling fruit size and shape paves the way for more in-depth studies aimed at the identification of underlying genetic variant(s).

Multivariate Analysis Approaches for Dimension and Shape Discrimination of Vitis vinifera Varieties

In this study, berry dimensions and shape traits, which are important for the design of the grape processing system and the classification of 10 different grape varieties grown in same ecological conditions (‘Ata Sarısı’, ‘Barış’, ‘Dımışkı’, ‘Hatun Parmağı’, ‘Helvani’, ‘Horoz Karası’, ‘Hönüsü’, ‘İtalia’, ‘Mevlana Sarısı’, and ‘Red Globe’) were determined; differences between the varieties were identified with the use of discriminant analysis. The largest grape varieties were identified as ‘Ata Sarısı’ and ‘Red Globe’. The ‘Red Globe’ and ‘Helvani’ varieties had geometrically sphere-like shape. The ‘Barış’ variety had the lowest size averages. According to elliptic Fourier analysis, the primary source of shape variation was ellipse and sphere-looking varieties. However, shape variation was seen due to the existence of a small number of drop-like varieties. According to discriminant analysis, shape differences of the varieties were defined by two discriminant functions. Based on these discriminant functions, the greatest classification performance was achieved for ‘Mevlana Sarısı’ and ‘Dımışkı’. In scatter plots, three shape definitions (sphere, ellipse, and drop) were made for grape varieties. Cluster analysis revealed 4 sub-groups. The first sub-group included the ‘Mevlana Sarısı’ variety; the second sub-group included the ‘Hönüsü’, ‘Hatun Parmağı’, ‘Dımışkı’, and ‘Horoz Karası’ varieties; the third sub-group included the ‘Ata Sarısı’ variety; the fourth sub-group included the ‘Barış’, ‘Helvani’, ‘İtalia’, and ‘Red Globe’ varieties. The variety in the first group had a geometrically ellipse-like shape, the largest length, and the smallest width. The size data were the smallest for the second sub-group. The third sub-group, with the ellipse-like shape, had the large size data. The grape varieties the closest to the sphere were classified in the fourth group, and these varieties had the large sizes.

Deep learning-based high-throughput phenotyping can drive future discoveries in plant reproductive biology

Advances in deep learning are providing a powerful set of image analysis tools that are readily accessible for high-throughput phenotyping applications in plant reproductive biology. High-throughput phenotyping systems are becoming critical for answering biological questions on a large scale. These systems have historically relied on traditional computer vision techniques. However, neural networks and specifically deep learning are rapidly becoming more powerful and easier to implement. Here, we examine how deep learning can drive phenotyping systems and be used to answer fundamental questions in reproductive biology. We describe previous applications of deep learning in the plant sciences, provide general recommendations for applying these methods to the study of plant reproduction, and present a case study in maize ear phenotyping. Finally, we highlight several examples where deep learning has enabled research that was previously out of reach and discuss the future outlook of these methods.

Automatic Fruit Morphology Phenome and Genetic Analysis: An Application in the Octoploid Strawberry

Automatizing phenotype measurement will decisively contribute to increase plant breeding efficiency. Among phenotypes, morphological traits are relevant in many fruit breeding programs, as appearance influences consumer preference. Often, these traits are manually or semiautomatically obtained. Yet, fruit morphology evaluation can be enhanced using fully automatized procedures and digital images provide a cost-effective opportunity for this purpose. Here, we present an automatized pipeline for comprehensive phenomic and genetic analysis of morphology traits extracted from internal and external strawberry (Fragaria x ananassa) images. The pipeline segments, classifies, and labels the images and extracts conformation features, including linear (area, perimeter, height, width, circularity, shape descriptor, ratio between height and width) and multivariate (Fourier elliptical components and Generalized Procrustes) statistics. Internal color patterns are obtained using an autoencoder to smooth out the image. In addition, we develop a variational autoencoder to automatically detect the most likely number of underlying shapes. Bayesian modeling is employed to estimate both additive and dominance effects for all traits. As expected, conformational traits are clearly heritable. Interestingly, dominance variance is higher than the additive component for most of the traits. Overall, we show that fruit shape and color can be quickly and automatically evaluated and are moderately heritable. Although we study strawberry images, the algorithm can be applied to other fruits, as shown in the GitHub repository.

Remote Sensing and Machine Learning in Crop Phenotyping and Management, with an Emphasis on Applications in Strawberry Farming

Measurement of plant characteristics is still the primary bottleneck in both plant breeding and crop management. Rapid and accurate acquisition of information about large plant populations is critical for monitoring plant health and dissecting the underlying genetic traits. In recent years, high-throughput phenotyping technology has benefitted immensely from both remote sensing and machine learning. Simultaneous use of multiple sensors (e.g., high-resolution RGB, multispectral, hyperspectral, chlorophyll fluorescence, and light detection and ranging (LiDAR)) allows a range of spatial and spectral resolutions depending on the trait in question. Meanwhile, computer vision and machine learning methodology have emerged as powerful tools for extracting useful biological information from image data. Together, these tools allow the evaluation of various morphological, structural, biophysical, and biochemical traits. In this review, we focus on the recent development of phenomics approaches in strawberry farming, particularly those utilizing remote sensing and machine learning, with an eye toward future prospects for strawberries in precision agriculture. The research discussed is broadly categorized according to strawberry traits related to (1) fruit/flower detection, fruit maturity, fruit quality, internal fruit attributes, fruit shape, and yield prediction; (2) leaf and canopy attributes; (3) water stress; and (4) pest and disease detection. Finally, we present a synthesis of the potential research opportunities and directions that could further promote the use of remote sensing and machine learning in strawberry farming.

Genomic Informed Breeding Strategies for Strawberry Yield and Fruit Quality Traits

Over the last two centuries, breeders have drastically modified the fruit quality of strawberries through artificial selection. However, there remains significant variation in quality across germplasm with scope for further improvements to be made. We reported extensive phenotyping of fruit quality and yield traits in a multi-parental strawberry population to allow genomic prediction and quantitative trait nucleotide (QTN) identification, thereby enabling the description of genetic architecture to inform the efficacy of implementing advanced breeding strategies. A negative relationship (r = -0.21) between total soluble sugar content and class one yield was identified, indicating a trade-off between these two essential traits. This result highlighted an established dilemma for strawberry breeders and a need to uncouple the relationship, particularly under June-bearing, protected production systems comparable to this study. A large effect of quantitative trait nucleotide was associated with perceived acidity and pH whereas multiple loci were associated with firmness. Therefore, we recommended the implementation of both marker assisted selection (MAS) and genomic prediction to capture the observed variation respectively. Furthermore, we identified a large effect locus associated with a 10% increase in the number of class one fruit and a further 10 QTN which, when combined, are associated with a 27% increase in the number of marketable strawberries. Ultimately, our results suggested that the best method to improve strawberry yield is through selecting parental lines based upon the number of marketable fruits produced per plant. Not only were strawberry number metrics less influenced by environmental fluctuations, but they had a larger additive genetic component when compared with mass traits. As such, selecting using "number" traits should lead to faster genetic gain.

Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry

BACKGROUND

Shape is a critical element of the visual appeal of strawberry fruit and is influenced by both genetic and non-genetic determinants. Current fruit phenotyping approaches for external characteristics in strawberry often rely on the human eye to make categorical assessments. However, fruit shape is an inherently multi-dimensional, continuously variable trait and not adequately described by a single categorical or quantitative feature. Morphometric approaches enable the study of complex, multi-dimensional forms but are often abstract and difficult to interpret. In this study, we developed a mathematical approach for transforming fruit shape classifications from digital images onto an ordinal scale called the Principal Progression of k Clusters (PPKC). We use these human-recognizable shape categories to select quantitative features extracted from multiple morphometric analyses that are best fit for genetic dissection and analysis.

RESULTS

We transformed images of strawberry fruit into human-recognizable categories using unsupervised machine learning, discovered 4 principal shape categories, and inferred progression using PPKC. We extracted 68 quantitative features from digital images of strawberries using a suite of morphometric analyses and multivariate statistical approaches. These analyses defined informative feature sets that effectively captured quantitative differences between shape classes. Classification accuracy ranged from 68% to 99% for the newly created phenotypic variables for describing a shape.

CONCLUSIONS

Our results demonstrated that strawberry fruit shapes could be robustly quantified, accurately classified, and empirically ordered using image analyses, machine learning, and PPKC. We generated a dictionary of quantitative traits for studying and predicting shape classes and identifying genetic factors underlying phenotypic variability for fruit shape in strawberry. The methods and approaches that we applied in strawberry should apply to other fruits, vegetables, and specialty crops.