Flowers, leaves or both? How to obtain suitable images for automated plant identification

Photographs as an essential biodiversity resource: drivers of gaps in the vascular plant photographic record

The photographic record is increasingly becoming an important biodiversity resource for primary research and conservation monitoring. However, globally, there are important gaps in this record even in relatively well-researched floras. To quantify the gaps in the Australian native vascular plant photographic record, we systematically surveyed 33 sources of well-curated species photographs, assembling a list of species with accessible and verifiable photographs, as well as the species for which this search failed. Of 21 077 Australian native species, 3715 lack a verifiable photograph across our 33 surveyed resources. There are three major geographic hotspots of unphotographed species in Australia, all far from current population centres. Many unphotographed species are small in stature or uncharismatic, and many are also recently described. The large number of recently described species without accessible photographs was surprising. There are longstanding efforts in Australia to organise the plant photographic record, but in the absence of a global consensus to treat photographs as an essential biodiversity resource, this has not become common practice. Many recently described species are small-range endemics and some have special conservation status. Completing the botanical photographic record across the globe will facilitate a virtuous feedback loop of more efficient identification, monitoring and conservation.

Geometric Morphometric Versus Genomic Patterns in a Large Polyploid Plant Species Complex

Plant species complexes represent a particularly interesting example of taxonomically complex groups (TCGs), linking hybridization, apomixis, and polyploidy with complex morphological patterns. In such TCGs, mosaic-like character combinations and conflicts of morphological data with molecular phylogenies present a major problem for species classification. Here, we used the large polyploid apomictic European Ranunculus auricomus complex to study relationships among five diploid sexual progenitor species and 75 polyploid apomictic derivate taxa, based on geometric morphometrics using 11,690 landmarked objects (basal and stem leaves, receptacles), genomic data (97,312 RAD-Seq loci, 48 phased target enrichment genes, 71 plastid regions) from 220 populations. We showed that (1) observed genomic clusters correspond to morphological groupings based on basal leaves and concatenated traits, and morphological groups were best resolved with RAD-Seq data; (2) described apomictic taxa usually overlap within trait morphospace except for those taxa at the space edges; (3) apomictic phenotypes are highly influenced by parental subgenome composition and to a lesser extent by climatic factors; and (4) allopolyploid apomictic taxa, compared to their sexual progenitor, resemble a mosaic of ecological and morphological intermediate to transgressive biotypes. The joint evaluation of phylogenomic, phenotypic, reproductive, and ecological data supports a revision of purely descriptive, subjective traditional morphological classifications.

Small Leaves, Big Diversity: Citizen Science and Taxonomic Revision Triples Species Number in the Carnivorous Drosera microphylla Complex (D. Section Ergaleium, Droseraceae)

The carnivorous Drosera microphylla complex from southwest Western Australia comprises a group of rare, narrowly endemic species that are potentially threatened by habitat destruction and illegal collection, thus highlighting a need for accurate taxonomic classification to facilitate conservation efforts. Following extensive fieldwork over two decades, detailed studies of both Australian and European herbaria and consideration of both crucial contributions by citizen scientists and social media observations, nine species of the D. microphylla complex are here described and illustrated, including four new species: D. atrata, D. hortiorum, D. koikyennuruff, and D. reflexa. The identities of the previously described infraspecific taxa D. calycina var. minor and D. microphylla var. macropetala are clarified. Both are here lectotypified, reinstated, and elevated to species rank. A replacement name, D. rubricalyx, is provided for the former taxon. Key morphological characters distinguishing the species of this complex include the presence or absence of axillary leaves, lamina shape, petal colour, filament shape, and style length. A detailed identification key, comparison figures, and a distribution map are provided. Six of the nine species are recommended for inclusion on the Priority Flora List under the Conservation Codes for Western Australian Flora and Fauna.

A Performance Comparison of Classification Algorithms for Rose Plants

One of the key roles of Botanists is to be able to recognize flowers. This role has become highly challenging given that the number of discovered flower types are nearing half a million. To support Botanists, Information Technology offers promising solutions. Specifically, machine learning techniques are intrinsically appealing due to being precise enough as required. To this aim, two observations on flower leaves are relevant and leverage flower identification: one, flower plants exhibit unique features in their leaves thus allow distinction of their co-located flowers; two, leaves have a much longer life than flowers thus preserve identity properties longer. This paper proposes the use of machine learning-based identification of rose types by leveraging the features from their leaves. For this purpose, the performance of Naive Bayes, Generalized Linear Model, Multilayer Perceptron, Decision Tree, Random Forest, Gradient Boosted Trees, and Support Vector Machine has been analyzed. This study optimizes the RF model by investigating and tuning its various parameters such as the number of trees, the depth of trees, and splitting criteria. The best results are achieved with gain ratio because it takes more distinct values to avoid the problems associated with Information Gain. Optimizing the number of trees and the depth of trees of RF yield better accuracy than other models. Extensive experiments are performed to analyze the results of ensemble algorithms by using the voting method for each instance. Results suggest that the performance of ensemble classifiers is superior to that of individual models.

Synaptic Scaling-An Artificial Neural Network Regularization Inspired by Nature

Nature has always inspired the human spirit and scientists frequently developed new methods based on observations from nature. Recent advances in imaging and sensing technology allow fascinating insights into biological neural processes. With the objective of finding new strategies to enhance the learning capabilities of neural networks, we focus on a phenomenon that is closely related to learning tasks and neural stability in biological neural networks, called homeostatic plasticity. Among the theories that have been developed to describe homeostatic plasticity, synaptic scaling has been found to be the most mature and applicable. We systematically discuss previous studies on the synaptic scaling theory and how they could be applied to artificial neural networks. Therefore, we utilize information theory to analytically evaluate how mutual information is affected by synaptic scaling. Based on these analytic findings, we propose two flavors in which synaptic scaling can be applied in the training process of simple and complex, feedforward, and recurrent neural networks. We compare our approach with state-of-the-art regularization techniques on standard benchmarks. We found that the proposed method yields the lowest error in both regression and classification tasks compared to previous regularization approaches in our experiments across a wide range of network feedforward and recurrent topologies and data sets.

Decoding family-level features for modern and fossil leaves from computer-vision heat maps

PREMISE

Angiosperm leaves present a classic identification problem due to their morphological complexity. Computer-vision algorithms can identify diagnostic regions in images, and heat map outputs illustrate those regions for identification, providing novel insights through visual feedback. We investigate the potential of analyzing leaf heat maps to reveal novel, human-friendly botanical information with applications for extant- and fossil-leaf identification.

METHODS

We developed a manual scoring system for hotspot locations on published computer-vision heat maps of cleared leaves that showed diagnostic regions for family identification. Heat maps of 3114 cleared leaves of 930 genera in 14 angiosperm families were analyzed. The top-5 and top-1 hotspot regions of highest diagnostic value were scored for 21 leaf locations. The resulting data were viewed using box plots and analyzed using cluster and principal component analyses. We manually identified similar features in fossil leaves to informally demonstrate potential fossil applications.

RESULTS

The method successfully mapped machine strategy using standard botanical language, and distinctive patterns emerged for each family. Hotspots were concentrated on secondary veins (Salicaceae, Myrtaceae, Anacardiaceae), tooth apices (Betulaceae, Rosaceae), and on the little-studied margins of untoothed leaves (Rubiaceae, Annonaceae, Ericaceae). Similar features drove the results from multivariate analyses. The results echo many traditional observations, while also showing that most diagnostic leaf features remain undescribed.

CONCLUSIONS

Machine-derived heat maps that initially appear to be dominated by noise can be translated into human-interpretable knowledge, highlighting paths forward for botanists and paleobotanists to discover new diagnostic botanical characters.

Identifying and extracting bark key features of 42 tree species using convolutional neural networks and class activation mapping

The significance of automatic plant identification has already been recognized by academia and industry. There were several attempts to utilize leaves and flowers for identification; however, bark also could be beneficial, especially for trees, due to its consistency throughout the seasons and its easy accessibility, even in high crown conditions. Previous studies regarding bark identification have mostly contributed quantitatively to increasing classification accuracy. However, ever since computer vision algorithms surpassed the identification ability of humans, an open question arises as to how machines successfully interpret and unravel the complicated patterns of barks. Here, we trained two convolutional neural networks (CNNs) with distinct architectures using a large-scale bark image dataset and applied class activation mapping (CAM) aggregation to investigate diagnostic keys for identifying each species. CNNs could identify the barks of 42 species with > 90% accuracy, and the overall accuracies showed a small difference between the two models. Diagnostic keys matched with salient shapes, which were also easily recognized by human eyes, and were typified as blisters, horizontal and vertical stripes, lenticels of various shapes, and vertical crevices and clefts. The two models exhibited disparate quality in the diagnostic features: the old and less complex model showed more general and well-matching patterns, while the better-performing model with much deeper layers indicated local patterns less relevant to barks. CNNs were also capable of predicting untrained species by 41.98% and 48.67% within the correct genus and family, respectively. Our methodologies and findings are potentially applicable to identify and visualize crucial traits of other plant organs.

Deep Learning in Plant Phenological Research: A Systematic Literature Review

Climate change represents one of the most critical threats to biodiversity with far-reaching consequences for species interactions, the functioning of ecosystems, or the assembly of biotic communities. Plant phenology research has gained increasing attention as the timing of periodic events in plants is strongly affected by seasonal and interannual climate variation. Recent technological development allowed us to gather invaluable data at a variety of spatial and ecological scales. The feasibility of phenological monitoring today and in the future depends heavily on developing tools capable of efficiently analyzing these enormous amounts of data. Deep Neural Networks learn representations from data with impressive accuracy and lead to significant breakthroughs in, e.g., image processing. This article is the first systematic literature review aiming to thoroughly analyze all primary studies on deep learning approaches in plant phenology research. In a multi-stage process, we selected 24 peer-reviewed studies published in the last five years (2016-2021). After carefully analyzing these studies, we describe the applied methods categorized according to the studied phenological stages, vegetation type, spatial scale, data acquisition- and deep learning methods. Furthermore, we identify and discuss research trends and highlight promising future directions. We present a systematic overview of previously applied methods on different tasks that can guide this emerging complex research field.

Automatic detection of alien plant species in action camera images using the chopped picture method and the potential of citizen science

Monitoring and detection of invasive alien plant species are necessary for effective management and control measures. Although efforts have been made to detect alien trees using satellite images, the detection of alien herbaceous species has been difficult. In this study, we examined the possibility of detecting non-native plants using deep learning on images captured by two action cameras. We created a model for each camera using the chopped picture method. The models were able to detect the alien plant Solidago altissima (tall goldenrod) and obtained an average accuracy of 89%. This study proved that it is possible to automatically detect exotic plants using inexpensive action cameras through deep learning. This advancement suggests that, in the future, citizen science may be useful for conducting distribution surveys of alien plants in a wide area at a low cost.

Sustainable Ecotourism through Cutting-Edge Technologies

Tourism is a phenomenon that dates back to ancient times. Ancient Greek philosophers recognised, adopted, and promoted the concept of rest-based tourism. Ecotourism is a particular type of tourism that connects with activities that take place in nature, without harming it, along with the herbal and animal wealth. According to estimates, the global ecotourism industry is currently booming due to various reasons, and it is becoming an important factor of sustainable regional development. This article presents the vision, work, and outcomes of project AdVENt, a project focusing natively in sustainable ecotourism through natural science and technological innovation. AdVENt’s study area includes the National Parks of Oiti (or Oeta) and Parnassus in Central Greece, where there is a remarkable native flora with a high endemism rate integrated with areas of cultural value and national and European hiking routes and paths of varying difficulty.

GRASP-125: A Dataset for Greek Vascular Plant Recognition in Natural Environment

Plant identification from images has become a rapidly developing research field in computer vision and is particularly challenging due to the morphological complexity of plants. The availability of large databases of plant images, and the research advancements in image processing, pattern recognition and machine learning, have resulted in a number of remarkably accurate and reliable image-based plant identification techniques, overcoming the time and expertise required for conventional plant identification, which is feasible only for expert botanists. In this paper, we introduce the GReek vAScular Plants (GRASP) dataset, a set of images composed of 125 classes of different species, for the automatic identification of vascular plants of Greece. In this context, we describe the methodology of data acquisition and dataset organization, along with the statistical features of the dataset. Furthermore, we present results of the application of popular deep learning architectures to the classification of the images in the dataset. Using transfer learning, we report 91% top-1 and 98% top-5 accuracy.

Plant image identification application demonstrates high accuracy in Northern Europe

Automated image-based plant identification has experienced rapid development and has been already used in research and nature management. However, there is a need for extensive studies on how accurately automatic plant identification works and which characteristics of observations and study species influence the results. We investigated the accuracy of the Flora Incognita application, a research-based tool for automated plant image identification. Our study was conducted in Estonia, Northern Europe. Photos originated from the Estonian national curated biodiversity observations database, originally without the intention to use them for automated identification (1496 photos, 542 species) were examined. Flora Incognita was also directly tested in field conditions in various habitats, taking images of plant organs as guided by the application (998 observations, 1703 photos, 280 species). Identification accuracy was compared among species characteristics: plant family, growth forms and life forms, habitat type and regional frequency. We also analysed image characteristics (plant organs, background, number of species in focus), and the number of training images that were available for particular species to develop the automated identification algorithm. From database images 79.6 % of species were correctly identified by Flora Incognita; in the field conditions species identification accuracy reached 85.3 %. Overall, the correct genus was found for 89 % and the correct plant family for 95 % of the species. Accuracy varied among different plant families, life forms and growth forms. Rare and common species and species from different habitats were identified with equal accuracy. Images with reproductive organs or with only the target species in focus were identified with greater success. The number of training images per species was positively correlated with the identification success. Even though a high accuracy has been already achieved for Flora Incognita, allowing its usage for research and practices, our results can guide further improvements of this application and automated plant identification in general.

Citizen science and online data: Opportunities and challenges for snake ecology and action against snakebite

The secretive behavior and life history of snakes makes studying their biology, distribution, and the epidemiology of venomous snakebite challenging. One of the most useful, most versatile, and easiest to collect types of biological data are photographs, particularly those that are connected with geographic location and date-time metadata. Photos verify occurrence records, provide data on phenotypes and ecology, and are often used to illustrate new species descriptions, field guides and identification keys, as well as in training humans and computer vision algorithms to identify snakes. We scoured eleven online and two offline sources of snake photos in an attempt to collect as many photos of as many snake species as possible, and attempt to explain some of the inter-species variation in photograph quantity among global regions and taxonomic groups, and with regard to medical importance, human population density, and range size. We collected a total of 725,565 photos-between 1 and 48,696 photos of 3098 of the world's 3879 snake species (79.9%), leaving 781 "most wanted" species with no photos (20.1% of all currently-described species as of the December 2020 release of The Reptile Database). We provide a list of most wanted species sortable by family, continent, authority, and medical importance, and encourage snake photographers worldwide to submit photos and associated metadata, particularly of "missing" species, to the most permanent and useful online archives: The Reptile Database, iNaturalist, and HerpMapper.

Supervised Learning Computer Vision Benchmark for Snake Species Identification From Photographs: Implications for Herpetology and Global Health

We trained a computer vision algorithm to identify 45 species of snakes from photos and compared its performance to that of humans. Both human and algorithm performance is substantially better than randomly guessing (null probability of guessing correctly given 45 classes = 2.2%). Some species (e.g., Boa constrictor) are routinely identified with ease by both algorithm and humans, whereas other groups of species (e.g., uniform green snakes, blotched brown snakes) are routinely confused. A species complex with largely molecular species delimitation (North American ratsnakes) was the most challenging for computer vision. Humans had an edge at identifying images of poor quality or with visual artifacts. With future improvement, computer vision could play a larger role in snakebite epidemiology, particularly when combined with information about geographic location and input from human experts.

Natural Products, the Fourth Industrial Revolution, and the Quintuple Helix

The profound interconnectedness of the sciences and technologies embodied in the Fourth Industrial Revolution is discussed in terms of the global role of natural products, and how that interplays with the development of sustainable and climate-conscious practices of cyberecoethnopharmacolomics within the Quintuple Helix for the promotion of a healthier planet and society.

Crowdsourcing snake identification with online communities of professional herpetologists and avocational snake enthusiasts

Species identification can be challenging for biologists, healthcare practitioners and members of the general public. Snakes are no exception, and the potential medical consequences of venomous snake misidentification can be significant. Here, we collected data on identification of 100 snake species by building a week-long online citizen science challenge which attracted more than 1000 participants from around the world. We show that a large community including both professional herpetologists and skilled avocational snake enthusiasts with the potential to quickly (less than 2 min) and accurately (69-90%; see text) identify snakes is active online around the clock, but that only a small fraction of community members are proficient at identifying snakes to the species level, even when provided with the snake's geographical origin. Nevertheless, participants showed great enthusiasm and engagement, and our study provides evidence that innovative citizen science/crowdsourcing approaches can play significant roles in training and building capacity. Although identification by an expert familiar with the local snake fauna will always be the gold standard, we suggest that healthcare workers, clinicians, epidemiologists and other parties interested in snakebite could become more connected to these communities, and that professional herpetologists and skilled avocational snake enthusiasts could organize ways to help connect medical professionals to crowdsourcing platforms. Involving skilled avocational snake enthusiasts in decision making could build the capacity of healthcare workers to identify snakes more quickly, specifically and accurately, and ultimately improve snakebite treatment data and outcomes.

Image-Based Automated Recognition of 31 Poaceae Species: The Most Relevant Perspectives

Poaceae represent one of the largest plant families in the world. Many species are of great economic importance as food and forage plants while others represent important weeds in agriculture. Although a large number of studies currently address the question of how plants can be best recognized on images, there is a lack of studies evaluating specific approaches for uniform species groups considered difficult to identify because they lack obvious visual characteristics. Poaceae represent an example of such a species group, especially when they are non-flowering. Here we present the results from an experiment to automatically identify Poaceae species based on images depicting six well-defined perspectives. One perspective shows the inflorescence while the others show vegetative parts of the plant such as the collar region with the ligule, adaxial and abaxial side of the leaf and culm nodes. For each species we collected 80 observations, each representing a series of six images taken with a smartphone camera. We extract feature representations from the images using five different convolutional neural networks (CNN) trained on objects from different domains and classify them using four state-of-the art classification algorithms. We combine these perspectives via score level fusion. In order to evaluate the potential of identifying non-flowering Poaceae we separately compared perspective combinations either comprising inflorescences or not. We find that for a fusion of all six perspectives, using the best combination of feature extraction CNN and classifier, an accuracy of 96.1% can be achieved. Without the inflorescence, the overall accuracy is still as high as 90.3%. In all but one case the perspective conveying the most information about the species (excluding inflorescence) is the ligule in frontal view. Our results show that even species considered very difficult to identify can achieve high accuracies in automatic identification as long as images depicting suitable perspectives are available. We suggest that our approach could be transferred to other difficult-to-distinguish species groups in order to identify the most relevant perspectives.

Flora Capture: a citizen science application for collecting structured plant observations

Background

Digital plant images are becoming increasingly important. First, given a large number of images deep learning algorithms can be trained to automatically identify plants. Second, structured image-based observations provide information about plant morphological characteristics. Finally in the course of digitalization, digital plant collections receive more and more interest in schools and universities.

Results

We developed a freely available mobile application called Flora Capture allowing users to collect series of plant images from predefined perspectives. These images, together with accompanying metadata, are transferred to a central project server where each observation is reviewed and validated by a team of botanical experts. Currently, more than 4800 plant species, naturally occurring in the Central European region, are covered by the application. More than 200,000 images, depicting more than 1700 plant species, have been collected by thousands of users since the initial app release in 2016.

Conclusion

Flora Capture allows experts, laymen and citizen scientists to collect a digital herbarium and share structured multi-modal observations of plants. Collected images contribute, e.g., to the training of plant identification algorithms, but also suit educational purposes. Additionally, presence records collected with each observation allow contribute to verifiable records of plant occurrences across the world.

What plant is that? Tests of automated image recognition apps for plant identification on plants from the British flora

There has been a recent explosion in development of image recognition technology and its application to automated plant identification, so it is timely to consider its potential for field botany. Nine free apps or websites for automated plant identification and suitable for use on mobile phones or tablet computers in the field were tested on a disparate set of 38 images of plants or parts of plants chosen from the higher plant flora of Britain and Ireland. There were large differences in performance with the best apps identifying >50 % of samples tested to genus or better. Although the accuracy is good for some of the top-rated apps, for any quantitative biodiversity study or for ecological surveys, there remains a need for validation by experts or against conventional floras. Nevertheless, the better-performing apps should be of great value to beginners and amateurs and may usefully stimulate interest in plant identification and nature. Potential uses of automated image recognition plant identification apps are discussed and recommendations made for their future use.

Tree species identification based on the fusion of bark and leaves

For trees, leaves are often used for identification, but the shape of leaves changes greatly, bark will be another identifying feature. However, it is difficult to recognize by a single organ when there are intra class differences and inter class similarities between leaves or bark. So we fuse features of leaf and bark. Firstly, we collected 17 species of leaves and bark of trees through field shooting and web crawling. Then propose a method of combining convolution neural network (CNN) with cascade fusion, additive fusion algorithm, bilinear fusion and score level fusion. Finally, the features extracted from the leaves and bark are fused in the ReLu layer and Fully connected layer. The method was compared with single organ recognition, Support Vector Machines (SVM), and existing fusion methods, results show that the two organ fusion method proposed are better than the other recognition methods, and recognition accuracy is 87.86%. For similar trees, when it is impossible to accurately determine its species by a single organ, the fusion of two organs can effectively improve this situation.

Convolutional Neural Networks enable efficient, accurate and fine-grained segmentation of plant species and communities from high-resolution UAV imagery

Recent technological advances in remote sensing sensors and platforms, such as high-resolution satellite imagers or unmanned aerial vehicles (UAV), facilitate the availability of fine-grained earth observation data. Such data reveal vegetation canopies in high spatial detail. Efficient methods are needed to fully harness this unpreceded source of information for vegetation mapping. Deep learning algorithms such as Convolutional Neural Networks (CNN) are currently paving new avenues in the field of image analysis and computer vision. Using multiple datasets, we test a CNN-based segmentation approach (U-net) in combination with training data directly derived from visual interpretation of UAV-based high-resolution RGB imagery for fine-grained mapping of vegetation species and communities. We demonstrate that this approach indeed accurately segments and maps vegetation species and communities (at least 84% accuracy). The fact that we only used RGB imagery suggests that plant identification at very high spatial resolutions is facilitated through spatial patterns rather than spectral information. Accordingly, the presented approach is compatible with low-cost UAV systems that are easy to operate and thus applicable to a wide range of users.