Measuring agreement among experts in classifying camera images of similar species

Linking camera-trap data to taxonomy: Identifying photographs of morphologically similar chipmunks

Remote cameras are a common method for surveying wildlife and recently have been promoted for implementing large-scale regional biodiversity monitoring programs. The use of camera-trap data depends on the correct identification of animals captured in the photographs, yet misidentification rates can be high, especially when morphologically similar species co-occur, and this can lead to faulty inferences and hinder conservation efforts. Correct identification is dependent on diagnosable taxonomic characters, photograph quality, and the experience and training of the observer. However, keys rooted in taxonomy are rarely used for the identification of camera-trap images and error rates are rarely assessed, even when morphologically similar species are present in the study area. We tested a method for ensuring high identification accuracy using two sympatric and morphologically similar chipmunk (Neotamias) species as a case study. We hypothesized that the identification accuracy would improve with use of the identification key and with observer training, resulting in higher levels of observer confidence and higher levels of agreement among observers. We developed an identification key and tested identification accuracy based on photographs of verified museum specimens. Our results supported predictions for each of these hypotheses. In addition, we validated the method in the field by comparing remote-camera data with live-trapping data. We recommend use of these methods to evaluate error rates and to exclude ambiguous records in camera-trap datasets. We urge that ensuring correct and scientifically defensible species identifications is incumbent on researchers and should be incorporated into the camera-trap workflow.

Dependent double-observer method reduces false-positive errors in auditory avian survey data

Bias introduced by detection errors is a well-documented issue for abundance and occupancy estimates of wildlife. Detection errors bias estimates of detection and abundance or occupancy in positive and negative directions, which can produce misleading results. There have been considerable design- and model-based methods to address false-negative errors, or missed detections. However, false-positive errors, or detections of individuals that are absent but counted as present because of misidentifications or double counts, are often assumed to not occur in ecological studies. The dependent double-observer survey method is a design-based approach speculated to reduce false positives because observations have the ability to be confirmed by two observers. However, whether this method reduces false positives compared to single-observer methods has not been empirically tested. We used prairie songbirds as a model system to test if a dependent double-observer method reduced false positives compared to a single-observer method. We used vocalizations of ten species to create auditory simulations and used naive and expert observers to survey these simulations using single-observer and dependent double-observer methods. False-positive rates were significantly lower using the dependent double-observer survey method in both observer groups. Expert observers reported a 3.2% false-positive rate in dependent double-observer surveys and a 9.5% false-positive rate in single-observer surveys, while naive observers reported a 39.1% false-positive rate in dependent double-observer surveys and a 49.1% false-positive rate in single-observer surveys. Misidentification errors arose in all survey scenarios and almost all species combinations. However, expert observers using the dependent double-observer method performed significantly better than other survey scenarios. Given the use of double-observer methods and the accumulating evidence that false positives occur in many survey methods across different taxa, this study is an important step forward in acknowledging and addressing false positives.

Error rates in wildlife image classification

We address the comments made by Thornton et al. (Ecology and Evolution, 2019) in response to our recent article on measuring the agreement among experts in classifying camera images of bobcats and Canada lynx.

Reassessing the success of experts and nonexperts at correctly differentiating between closely related species from camera trap images: A reply to Gooliaff and Hodges

We present a reply to a recent article in Ecology and Evolution ("Measuring agreement among experts in classifying camera images of similar species" by Gooliaff and Hodges) that demonstrated a lack of consistency in expert-based classification of images of similar-looking species. We disagree with several conclusions from the study, and show that with some training, and use of multiple images that is becoming standard practice in camera-trapping studies, even nonexperts can identify similar sympatric species with high consistency.