Landmark-free statistical analysis of the shape of plant leaves

Deep learning to capture leaf shape in plant images: Validation by geometric morphometrics

Plant leaves play a pivotal role in automated species identification using deep learning (DL). However, achieving reproducible capture of leaf variation remains challenging due to the inherent "black box" problem of DL models. To evaluate the effectiveness of DL in capturing leaf shape, we used geometric morphometrics (GM), an emerging component of eXplainable Artificial Intelligence (XAI) toolkits. We photographed Ranunculus auricomus leaves directly in situ and after herbarization. From these corresponding leaf images, we automatically extracted DL features using a neural network and digitized leaf shapes using GM. The association between the extracted DL features and GM shapes was then evaluated using dimension reduction and covariation models. DL features facilitated the clustering of leaf images by source populations in both in situ and herbarized leaf image datasets, and certain DL features were significantly associated with biological leaf shape variation as inferred by GM. DL features also enabled leaf classification into morpho-phylogenomic groups within the intricate R. auricomus species complex. We demonstrated that simple in situ leaf imaging and DL reproducibly captured leaf shape variation at the population level, while combining this approach with GM provided key insights into the shape information extracted from images by computer vision, a necessary prerequisite for reliable automated plant phenotyping.

Classifying organisms and artefacts by their outline shapes

We often wish to classify objects by their shapes. Indeed, the study of shapes is an important part of many scientific fields, such as evolutionary biology, structural biology, image processing and archaeology. However, mathematical shape spaces are rather complicated and nonlinear. The most widely used methods of shape analysis, geometric morphometrics, treat the shapes as sets of points. Diffeomorphic methods consider the underlying curve rather than points, but have rarely been applied to real-world problems. Using a machine classifier, we tested the ability of several of these methods to describe and classify the shapes of a variety of organic and man-made objects. We find that one method, based on square-root velocity functions (SRVFs), outperforms all others, including a standard geometric morphometric method (eigenshapes), and that it is also superior to human experts using shape alone. When the SRVF approach is constrained to take account of homologous landmarks it can accurately classify objects of very different shapes. The SRVF method identifies a shortest path between shapes, and we show that this can be used to estimate the shapes of intermediate steps in evolutionary series. Diffeomorphic shape analysis methods, we conclude, now provide practical and effective solutions to many shape description and classification problems in the natural and human sciences.

μMatch: 3D Shape Correspondence for Biological Image Data

Modern microscopy technologies allow imaging biological objects in 3D over a wide range of spatial and temporal scales, opening the way for a quantitative assessment of morphology. However, establishing a correspondence between objects to be compared, a first necessary step of most shape analysis workflows, remains challenging for soft-tissue objects without striking features allowing them to be landmarked. To address this issue, we introduce the μMatch 3D shape correspondence pipeline. μMatch implements a state-of-the-art correspondence algorithm initially developed for computer graphics and packages it in a streamlined pipeline including tools to carry out all steps from input data pre-processing to classical shape analysis routines. Importantly, μMatch does not require any landmarks on the object surface and establishes correspondence in a fully automated manner. Our open-source method is implemented in Python and can be used to process collections of objects described as triangular meshes. We quantitatively assess the validity of μMatch relying on a well-known benchmark dataset and further demonstrate its reliability by reproducing published results previously obtained through manual landmarking.

Evidence That Supertriangles Exist in Nature from the Vertical Projections of Koelreuteria paniculata Fruit

Many natural radial symmetrical shapes (e.g., sea stars) follow the Gielis equation (GE) or its twin equation (TGE). A supertriangle (three triangles arranged around a central polygon) represents such a shape, but no study has tested whether natural shapes can be represented as/are supertriangles or whether the GE or TGE can describe their shape. We collected 100 pieces of Koelreuteria paniculata fruit, which have a supertriangular shape, extracted the boundary coordinates for their vertical projections, and then fitted them with the GE and TGE. The adjusted root mean square errors (RMSEadj) of the two equations were always less than 0.08, and >70% were less than 0.05. For 57/100 fruit projections, the GE had a lower RMSEadj than the TGE, although overall differences in the goodness of fit were non-significant. However, the TGE produces more symmetrical shapes than the GE as the two parameters controlling the extent of symmetry in it are approximately equal. This work demonstrates that natural supertriangles exist, validates the use of the GE and TGE to model their shapes, and suggests that different complex radially symmetrical shapes can be generated by the same equation, implying that different types of biological symmetry may result from the same biophysical mechanisms.

Geometric and topological approaches to shape variation in Ginkgo leaves

Leaf shape is a key plant trait that varies enormously. The range of applications for data on this trait requires frequent methodological development so that researchers have an up-to-date toolkit with which to quantify leaf shape. We generated a dataset of 468 leaves produced by Ginkgo biloba, and 24 fossil leaves produced by evolutionary relatives of extant Ginkgo. We quantified the shape of each leaf by developing a geometric method based on elastic curves and a topological method based on persistent homology. Our geometric method indicates that shape variation in modern leaves is dominated by leaf size, furrow depth and the angle of the two lobes at the leaf base that is also related to leaf width. Our topological method indicates that shape variation in modern leaves is dominated by leaf size and furrow depth. We have applied both methods to modern and fossil material: the methods are complementary, identifying similar primary patterns of variation, but also revealing different aspects of morphological variation. Our topological approach distinguishes long-shoot leaves from short-shoot leaves, both methods indicate that leaf shape influences or is at least related to leaf area, and both could be applied in palaeoclimatic and evolutionary studies of leaf shape.

Plant Species Identification from Occluded Leaf Images

We present an approach to identify the plant species from the contour information from occluded leaf image using a database of full plant leaves. Although contour based 2D shape matching has been studied extensively in the last couple of decades, matching occluded leaves with full leaf databases is an open and little worked on problem. Classifying occluded plant leaves is even more challenging than full leaf matching because of large variations and complexity of leaf structures. Matching an occluded contour with all the full contours in a database is an NP-hard problem, so our algorithm is necessarily suboptimal. First, we represent the 2D contour points as a β-Spline curve. Then, we extract interest points on these curves via the Discrete Contour Evolution (DCE) algorithm. We use subgraph matching using the DCE points as graph nodes, which produces a number of open curves for each closed leaf contour. Next, we compute the similarity transformation parameters (translation, rotation, and uniform scaling) for each open curve. We then "overlay" each open curve with the inverse similarity transformed occluded curve and use the Fréchet distance metric to measure the quality of the match, retaining the best η matched curves. Since the Fréchet metric is cheap to compute but not perfectly correlated with the quality of the match, we formulate an energy functional that is well correlated with the quality of the match, but is considerably more expensive to compute. The functional uses local and global curvature, Shape Context descriptors, and String Cut features. We minimize this energy functional using a convex-concave relaxation framework. The curve among these best η curves, that has the minimum energy, is considered to be the best overall match with the occluded leaf. Experiments on three publicly available leaf image database shows that our method is both effective and efficient, outperforming other current state-of-the-art methods. Occlusion is measured as the percentage of the overall contour (and not leaf area) that is missing. We show that our algorithm can, even for leaves with a high amounts of occlusion (say 50 percent occlusion), still identify the best full leaf match from the databases.

Analysis of shape data: From landmarks to elastic curves

Proliferation of high-resolution imaging data in recent years has led to sub-stantial improvements in the two popular approaches for analyzing shapes of data objects based on landmarks and/or continuous curves. We provide an expository account of elastic shape analysis of parametric planar curves representing shapes of two-dimensional (2D) objects by discussing its differences, and its commonalities, to the landmark-based approach. Particular attention is accorded to the role of reparameterization of a curve, which in addition to rotation, scaling and translation, represents an important shape-preserving transformation of a curve. The transition to the curve-based approach moves the mathematical setting of shape analysis from finite-dimensional non-Euclidean spaces to infinite-dimensional ones. We discuss some of the challenges associated with the infinite-dimensionality of the shape space, and illustrate the use of geometry-based methods in the computation of intrinsic statistical summaries and in the definition of statistical models on a 2D imaging dataset consisting of mouse vertebrae. We conclude with an overview of the current state-of-the-art in the field.

Statistical analysis and modeling of the geometry and topology of plant roots

The root is an important organ of a plant since it is responsible for water and nutrient uptake. Analyzing and modeling variabilities in the geometry and topology of roots can help in assessing the plant's health, understanding its growth patterns, and modeling relations between plant species and between plants and their environment. In this article, we develop a framework for the statistical analysis and modeling of the geometry and topology of plant roots. We represent root structures as points in a tree-shape space equipped with a metric that quantifies geometric and topological differences between pairs of roots. We then use these building blocks to compute geodesics, i.e., optimal deformations under the metric between root structures, and to perform statistical analysis on root populations. We demonstrate the utility of the proposed framework through an application to a dataset of wheat roots grown in different environmental conditions. We also show that the framework can be used in various applications including classification and regression.

Elastic Statistical Shape Analysis of Biological Structures with Case Studies: A Tutorial

We describe a recent framework for statistical shape analysis of curves and show its applicability to various biological datasets. The presented methods are based on a functional representation of shape called the square-root velocity function and a closely related elastic metric. The main benefit of this approach is its invariance to reparameterization (in addition to the standard shape-preserving transformations of translation, rotation and scale), and ability to compute optimal registrations (point correspondences) across objects. Building upon the defined distance between shapes, we additionally describe tools for computing sample statistics including the mean and covariance. Based on the covariance structure, one can also explore variability in shape samples via principal component analysis. Finally, the estimated mean and covariance can be used to define Wrapped Gaussian models on the shape space, which are easy to sample from. We present multiple case studies on various biological datasets including (1) leaf outlines, (2) internal carotid arteries, (3) Diffusion Tensor Magnetic Resonance Imaging fiber tracts, (4) Glioblastoma Multiforme tumors, and (5) vertebrae in mice. We additionally provide a MATLAB package that can be used to produce the results given in this manuscript.

Deformation Based Curved Shape Representation

In this paper, we introduce a deformation based representation space for curved shapes in . Given an ordered set of points sampled from a curved shape, the proposed method represents the set as an element of a finite dimensional matrix Lie group. Variation due to scale and location are filtered in a preprocessing stage, while shapes that vary only in rotation are identified by an equivalence relationship. The use of a finite dimensional matrix Lie group leads to a similarity metric with an explicit geodesic solution. Subsequently, we discuss some of the properties of the metric and its relationship with a deformation by least action. Furthermore, invariance to reparametrization or estimation of point correspondence between shapes is formulated as an estimation of sampling function. Thereafter, two possible approaches are presented to solve the point correspondence estimation problem. Finally, we propose an adaptation of k-means clustering for shape analysis in the proposed representation space. Experimental results show that the proposed representation is robust to uninformative cues, e.g., local shape perturbation and displacement. In comparison to state of the art methods, it achieves a high precision on the Swedish and the Flavia leaf datasets and a comparable result on MPEG-7, Kimia99 and Kimia216 datasets.

Morphological Plant Modeling: Unleashing Geometric and Topological Potential within the Plant Sciences

The geometries and topologies of leaves, flowers, roots, shoots, and their arrangements have fascinated plant biologists and mathematicians alike. As such, plant morphology is inherently mathematical in that it describes plant form and architecture with geometrical and topological techniques. Gaining an understanding of how to modify plant morphology, through molecular biology and breeding, aided by a mathematical perspective, is critical to improving agriculture, and the monitoring of ecosystems is vital to modeling a future with fewer natural resources. In this white paper, we begin with an overview in quantifying the form of plants and mathematical models of patterning in plants. We then explore the fundamental challenges that remain unanswered concerning plant morphology, from the barriers preventing the prediction of phenotype from genotype to modeling the movement of leaves in air streams. We end with a discussion concerning the education of plant morphology synthesizing biological and mathematical approaches and ways to facilitate research advances through outreach, cross-disciplinary training, and open science. Unleashing the potential of geometric and topological approaches in the plant sciences promises to transform our understanding of both plants and mathematics.

Minimum action principle and shape dynamics

In this paper, we propose a new method for computing a distance between two shapes embedded in three-dimensional space. Instead of comparing directly the geometric properties of the two shapes, we measure the cost of deforming one of the two shapes into the other. The deformation is computed as the geodesic between the two shapes in the space of shapes. The geodesic is found as a minimizer of the Onsager-Machlup action, based on an elastic energy for shapes that we define. Its length is set to be the integral of the action along that path; it defines an intrinsic quasi-metric on the space of shapes. We illustrate applications of our method to geometric morphometrics using three datasets representing bones and teeth of primates. Experiments on these datasets show that the variational quasi-metric we have introduced performs remarkably well both in shape recognition and in identifying evolutionary patterns, with success rates similar to, and in some cases better than, those obtained by expert observers.

Phenotyping of plants in competitive but controlled environments: a study of drought response in transgenic wheat

In recent years, the interest in new technologies for wheat improvement has increased greatly. To screen genetically modified germplasm in conditions more realistic for a field situation we developed a phenotyping platform where transgenic wheat and barley are grown in competition. In this study, we used the platform to (1) test selected promoter and gene combinations for their capacity to increase drought tolerance, (2) test the function and power of our platform to screen the performance of transgenic plants growing in competition, and (3) develop and test an imaging and analysis process as a means of obtaining additional, non-destructive data on plant growth throughout the whole growth cycle instead of relying solely on destructive sampling at the end of the season. The results showed that several transgenic lines under well watered conditions had higher biomass and/or grain weight than the wild-type control but the advantage was significant in one case only. None of the transgenics seemed to show any grain weight advantage under drought stress and only two lines had a substantially but not significantly higher biomass weight than the wild type. However, their evaluation under drought stress was disadvantaged by their delayed flowering date, which increased the drought stress they experienced in comparison to the wild type. Continuous imaging during the season provided additional and non-destructive phenotyping information on the canopy development of mini-plots in our phenotyping platform. A correlation analysis of daily canopy coverage data with harvest metrics showed that the best predictive value from canopy coverage data for harvest metrics was achieved with observations from around heading/flowering to early ripening whereas early season observations had only a limited diagnostic value. The result that the biomass/leaf development in the early growth phase has little correlation with biomass or grain yield data questions imaging approaches concentrating only on the early development stage.

Morphological Plant Modeling: Unleashing Geometric and Topological Potential within the Plant Sciences

The geometries and topologies of leaves, flowers, roots, shoots, and their arrangements have fascinated plant biologists and mathematicians alike. As such, plant morphology is inherently mathematical in that it describes plant form and architecture with geometrical and topological techniques. Gaining an understanding of how to modify plant morphology, through molecular biology and breeding, aided by a mathematical perspective, is critical to improving agriculture, and the monitoring of ecosystems is vital to modeling a future with fewer natural resources. In this white paper, we begin with an overview in quantifying the form of plants and mathematical models of patterning in plants. We then explore the fundamental challenges that remain unanswered concerning plant morphology, from the barriers preventing the prediction of phenotype from genotype to modeling the movement of leaves in air streams. We end with a discussion concerning the education of plant morphology synthesizing biological and mathematical approaches and ways to facilitate research advances through outreach, cross-disciplinary training, and open science. Unleashing the potential of geometric and topological approaches in the plant sciences promises to transform our understanding of both plants and mathematics.

A Hybrid Approach for Improving Image Segmentation: Application to Phenotyping of Wheat Leaves

In this article we propose a novel tool that takes an initial segmented image and returns a more accurate segmentation that accurately captures sharp features such as leaf tips, twists and axils. Our algorithm utilizes basic a-priori information about the shape of plant leaves and local image orientations to fit active contour models to important plant features that have been missed during the initial segmentation. We compare the performance of our approach with three state-of-the-art segmentation techniques, using three error metrics. The results show that leaf tips are detected with roughly one half of the original error, segmentation accuracy is almost always improved and more than half of the leaf breakages are corrected.

Landmark-free geometric methods in biological shape analysis

In this paper, we propose a new approach for computing a distance between two shapes embedded in three-dimensional space. We take as input a pair of triangulated genus zero surfaces that are topologically equivalent to spheres with no holes or handles, and construct a discrete conformal map f between the surfaces. The conformal map is chosen to minimize a symmetric deformation energy Esd(f) which we introduce. This measures the distance of f from an isometry, i.e. a non-distorting correspondence. We show that the energy of the minimizing map gives a well-behaved metric on the space of genus zero surfaces. In contrast to most methods in this field, our approach does not rely on any assignment of landmarks on the two surfaces. We illustrate applications of our approach to geometric morphometrics using three datasets representing the bones and teeth of primates. Experiments on these datasets show that our approach performs remarkably well both in shape recognition and in identifying evolutionary patterns, with success rates similar to, and in some cases better than, those obtained by expert observers.

RootAnalyzer: A Cross-Section Image Analysis Tool for Automated Characterization of Root Cells and Tissues

The morphology of plant root anatomical features is a key factor in effective water and nutrient uptake. Existing techniques for phenotyping root anatomical traits are often based on manual or semi-automatic segmentation and annotation of microscopic images of root cross sections. In this article, we propose a fully automated tool, hereinafter referred to as RootAnalyzer, for efficiently extracting and analyzing anatomical traits from root-cross section images. Using a range of image processing techniques such as local thresholding and nearest neighbor identification, RootAnalyzer segments the plant root from the image's background, classifies and characterizes the cortex, stele, endodermis and epidermis, and subsequently produces statistics about the morphological properties of the root cells and tissues. We use RootAnalyzer to analyze 15 images of wheat plants and one maize plant image and evaluate its performance against manually-obtained ground truth data. The comparison shows that RootAnalyzer can fully characterize most root tissue regions with over 90% accuracy.