A new fine-grained method for automated visual analysis of herbarium specimens: A case study for phenological data extraction

PREMISE

Herbarium specimens represent an outstanding source of material with which to study plant phenological changes in response to climate change. The fine-scale phenological annotation of such specimens is nevertheless highly time consuming and requires substantial human investment and expertise, which are difficult to rapidly mobilize.

METHODS

We trained and evaluated new deep learning models to automate the detection, segmentation, and classification of four reproductive structures of Streptanthus tortuosus (flower buds, flowers, immature fruits, and mature fruits). We used a training data set of 21 digitized herbarium sheets for which the position and outlines of 1036 reproductive structures were annotated manually. We adjusted the hyperparameters of a mask R-CNN (regional convolutional neural network) to this specific task and evaluated the resulting trained models for their ability to count reproductive structures and estimate their size.

RESULTS

The main outcome of our study is that the performance of detection and segmentation can vary significantly with: (i) the type of annotations used for training, (ii) the type of reproductive structures, and (iii) the size of the reproductive structures. In the case of Streptanthus tortuosus, the method can provide quite accurate estimates (77.9% of cases) of the number of reproductive structures, which is better estimated for flowers than for immature fruits and buds. The size estimation results are also encouraging, showing a difference of only a few millimeters between the predicted and actual sizes of buds and flowers.

DISCUSSION

This method has great potential for automating the analysis of reproductive structures in high-resolution images of herbarium sheets. Deeper investigations regarding the taxonomic scalability of this approach and its potential improvement will be conducted in future work.

A new phenological metric for use in pheno-climatic models: A case study using herbarium specimens of Streptanthus tortuosus

PREMISE

Herbarium specimens have been used to detect climate-induced shifts in flowering time by using the day of year of collection (DOY) as a proxy for first or peak flowering date. Variation among herbarium sheets in their phenological status, however, undermines the assumption that DOY accurately represents any particular phenophase. Ignoring this variation can reduce the explanatory power of pheno-climatic models (PCMs) designed to predict the effects of climate on flowering date.

METHODS

Here we present a protocol for the phenological scoring of imaged herbarium specimens using an ImageJ plugin, and we introduce a quantitative metric of a specimen's phenological status, the phenological index (PI), which we use in PCMs to control for phenological variation among specimens of Streptanthus tortuosus (Brassicaceeae) when testing for the effects of climate on DOY. We demonstrate that including PI as an independent variable improves model fit.

RESULTS

Including PI in PCMs increased the model R 2 relative to PCMs that excluded PI; regression coefficients for climatic parameters, however, remained constant.

DISCUSSION

Our protocol provides a simple, quantitative phenological metric for any observed plant. Including PI in PCMs increases R 2 and enables predictions of the DOY of any phenophase under any specified climatic conditions.