An Automated Light Trap to Monitor Moths (Lepidoptera) Using Computer Vision-Based Tracking and Deep Learning

Flower Visitation through the Lens: Exploring the Foraging Behaviour of Bombus terrestris with a Computer Vision-Based Application

To understand the processes behind pollinator declines and for the conservation of pollination services, we need to understand fundamental drivers influencing pollinator behaviour. Here, we aimed to elucidate how wild bumblebees interact with three plant species and investigated their foraging behaviour with varying flower densities. We video-recorded Bombus terrestris in 60 × 60 cm quadrats of Lotus creticus, Persicaria capitata, and Trifolium pratense in urban areas of Terceira (Azores, Portugal). For the automated bumblebee detection and counting, we created deep learning-based computer vision models with custom datasets. We achieved high model accuracy of 0.88 for Lotus and Persicaria and 0.95 for Trifolium, indicating accurate bumblebee detection. In our study, flower cover was the only factor that influenced the attractiveness of flower patches, and plant species did not have an effect. We detected a significant positive effect of flower cover on the attractiveness of flower patches for flower-visiting bumblebees. The time spent per unit of inflorescence surface area was longer on the Trifolium than those on the Lotus and Persicaria. However, our result did not indicate significant differences in the time bumblebees spent on inflorescences among the three plant species. Here, we also justify computer vision-based analysis as a reliable tool for studying pollinator behavioural ecology.

Towards a standardized framework for AI-assisted, image-based monitoring of nocturnal insects

Automated sensors have potential to standardize and expand the monitoring of insects across the globe. As one of the most scalable and fastest developing sensor technologies, we describe a framework for automated, image-based monitoring of nocturnal insects-from sensor development and field deployment to workflows for data processing and publishing. Sensors comprise a light to attract insects, a camera for collecting images and a computer for scheduling, data storage and processing. Metadata is important to describe sampling schedules that balance the capture of relevant ecological information against power and data storage limitations. Large data volumes of images from automated systems necessitate scalable and effective data processing. We describe computer vision approaches for the detection, tracking and classification of insects, including models built from existing aggregations of labelled insect images. Data from automated camera systems necessitate approaches that account for inherent biases. We advocate models that explicitly correct for bias in species occurrence or abundance estimates resulting from the imperfect detection of species or individuals present during sampling occasions. We propose ten priorities towards a step-change in automated monitoring of nocturnal insects, a vital task in the face of rapid biodiversity loss from global threats. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Towards a toolkit for global insect biodiversity monitoring

Insects are the most diverse group of animals on Earth, yet our knowledge of their diversity, ecology and population trends remains abysmally poor. Four major technological approaches are coming to fruition for use in insect monitoring and ecological research-molecular methods, computer vision, autonomous acoustic monitoring and radar-based remote sensing-each of which has seen major advances over the past years. Together, they have the potential to revolutionize insect ecology, and to make all-taxa, fine-grained insect monitoring feasible across the globe. So far, advances within and among technologies have largely taken place in isolation, and parallel efforts among projects have led to redundancy and a methodological sprawl; yet, given the commonalities in their goals and approaches, increased collaboration among projects and integration across technologies could provide unprecedented improvements in taxonomic and spatio-temporal resolution and coverage. This theme issue showcases recent developments and state-of-the-art applications of these technologies, and outlines the way forward regarding data processing, cost-effectiveness, meaningful trend analysis, technological integration and open data requirements. Together, these papers set the stage for the future of automated insect monitoring. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Towards global insect biomonitoring with frugal methods

None of the global targets for protecting nature are currently met, although humanity is critically dependent on biodiversity. A significant issue is the lack of data for most biodiverse regions of the planet where the use of frugal methods for biomonitoring would be particularly important because the available funding for monitoring is insufficient, especially in low-income countries. We here discuss how three approaches to insect biomonitoring (computer vision, lidar, DNA sequences) could be made more frugal and urge that all biomonitoring techniques should be evaluated for global suitability before becoming the default in high-income countries. This requires that techniques popular in high-income countries should undergo a phase of 'innovation through simplification' before they are implemented more broadly. We predict that techniques that acquire raw data at low cost and are suitable for analysis with AI (e.g. images, lidar-signals) will be particularly suitable for global biomonitoring, while techniques that rely heavily on patented technologies may be less promising (e.g. DNA sequences). We conclude the opinion piece by pointing out that the widespread use of AI for data analysis will require a global strategy for providing the necessary computational resources and training. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Insect detect: An open-source DIY camera trap for automated insect monitoring

Insect monitoring is essential to design effective conservation strategies, which are indispensable to mitigate worldwide declines and biodiversity loss. For this purpose, traditional monitoring methods are widely established and can provide data with a high taxonomic resolution. However, processing of captured insect samples is often time-consuming and expensive, which limits the number of potential replicates. Automated monitoring methods can facilitate data collection at a higher spatiotemporal resolution with a comparatively lower effort and cost. Here, we present the Insect Detect DIY (do-it-yourself) camera trap for non-invasive automated monitoring of flower-visiting insects, which is based on low-cost off-the-shelf hardware components combined with open-source software. Custom trained deep learning models detect and track insects landing on an artificial flower platform in real time on-device and subsequently classify the cropped detections on a local computer. Field deployment of the solar-powered camera trap confirmed its resistance to high temperatures and humidity, which enables autonomous deployment during a whole season. On-device detection and tracking can estimate insect activity/abundance after metadata post-processing. Our insect classification model achieved a high top-1 accuracy on the test dataset and generalized well on a real-world dataset with captured insect images. The camera trap design and open-source software are highly customizable and can be adapted to different use cases. With custom trained detection and classification models, as well as accessible software programming, many possible applications surpassing our proposed deployment method can be realized.

The Use and Prospects of Nonlethal Methods in Entomology

Arthropods are declining globally, and entomologists ought to be in the forefront of protecting them. However, entomological study methods are typically lethal, and we argue that this makes the ethical status of the profession precarious. Lethal methods are used in most studies, even those that aim to support arthropod conservation. Additionally, almost all collecting methods result in bycatch, and a first step toward less destructive research practices is to minimize bycatch and/or ensure its proper storage and use. In this review, we describe the available suite of nonlethal methods with the aim of promoting their use. We classify nonlethal methods into (a) reuse of already collected material, (b) methods that are damaging but not lethal, (c) methods that modify behavior, and (d) true nonlethal methods. Artificial intelligence and miniaturization will help to extend the nonlethal methodological toolkit, but the need for further method development and testing remains.

Nature 4.0: A networked sensor system for integrated biodiversity monitoring

Ecosystem functions and services are severely threatened by unprecedented global loss in biodiversity. To counteract these trends, it is essential to develop systems to monitor changes in biodiversity for planning, evaluating, and implementing conservation and mitigation actions. However, the implementation of monitoring systems suffers from a trade-off between grain (i.e., the level of detail), extent (i.e., the number of study sites), and temporal repetition. Here, we present an applied and realized networked sensor system for integrated biodiversity monitoring in the Nature 4.0 project as a solution to these challenges, which considers plants and animals not only as targets of investigation, but also as parts of the modular sensor network by carrying sensors. Our networked sensor system consists of three main closely interlinked components with a modular structure: sensors, data transmission, and data storage, which are integrated into pipelines for automated biodiversity monitoring. We present our own real-world examples of applications, share our experiences in operating them, and provide our collected open data. Our flexible, low-cost, and open-source solutions can be applied for monitoring individual and multiple terrestrial plants and animals as well as their interactions. Ultimately, our system can also be applied to area-wide ecosystem mapping tasks, thereby providing an exemplary cost-efficient and powerful solution for biodiversity monitoring. Building upon our experiences in the Nature 4.0 project, we identified ten key challenges that need to be addressed to better understand and counteract the ongoing loss of biodiversity using networked sensor systems. To tackle these challenges, interdisciplinary collaboration, additional research, and practical solutions are necessary to enhance the capability and applicability of networked sensor systems for researchers and practitioners, ultimately further helping to ensure the sustainable management of ecosystems and the provision of ecosystem services.

MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation

This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come with intensive restrictions, such as the need for sufficient lighting, high contrast between the animal and background, absence of occlusion, significant user input, etc. Tracking flight paths of bumblebees in a social group is challenging. They suddenly adjust movements and change their appearance during different wing beat states while exhibiting significant similarities in their individual appearance. The MBT3D tracker, developed in this research, is an adaptation of an existing ant tracking algorithm for bumblebee tracking. It incorporates an offline trained appearance descriptor along with a Kalman Filter for appearance and motion matching. Different detector architectures for upstream detections (You Only Look Once (YOLOv5), Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and RetinaNet) are investigated in a comparative study to optimize performance. The detection models were trained on a dataset containing 11359 labeled bumblebee images. YOLOv5 reaches an Average Precision of AP = 53, 8%, Faster R-CNN achieves AP = 45, 3% and RetinaNet AP = 38, 4% on the bumblebee validation dataset, which consists of 1323 labeled bumblebee images. The tracker's appearance model is trained on 144 samples. The tracker (with Faster R-CNN detections) reaches a Multiple Object Tracking Accuracy MOTA = 93, 5% and a Multiple Object Tracking Precision MOTP = 75, 6% on a validation dataset containing 2000 images, competing with state-of-the-art computer vision methods. The framework allows reliable tracking of different bumblebees in the same video stream with rarely occurring identity switches (IDS). MBT3D has much lower IDS than other commonly used algorithms, with one of the lowest false positive rates, competing with state-of-the-art animal tracking algorithms. The developed framework reconstructs the 3-dimensional (3D) flight paths of the bumblebees by triangulation. It also handles and compares two alternative stereo camera pairs if desired.

Real-time biodiversity analysis using deep-learning algorithms on mobile robotic platforms

Ecological biodiversity is declining at an unprecedented rate. To combat such irreversible changes in natural ecosystems, biodiversity conservation initiatives are being conducted globally. However, the lack of a feasible methodology to quantify biodiversity in real-time and investigate population dynamics in spatiotemporal scales prevents the use of ecological data in environmental planning. Traditionally, ecological studies rely on the census of an animal population by the "capture, mark and recapture" technique. In this technique, human field workers manually count, tag and observe tagged individuals, making it time-consuming, expensive, and cumbersome to patrol the entire area. Recent research has also demonstrated the potential for inexpensive and accessible sensors for ecological data monitoring. However, stationary sensors collect localised data which is highly specific on the placement of the setup. In this research, we propose the methodology for biodiversity monitoring utilising state-of-the-art deep learning (DL) methods operating in real-time on sample payloads of mobile robots. Such trained DL algorithms demonstrate a mean average precision (mAP) of 90.51% in an average inference time of 67.62 milliseconds within 6,000 training epochs. We claim that the use of such mobile platform setups inferring real-time ecological data can help us achieve our goal of quick and effective biodiversity surveys. An experimental test payload is fabricated, and online as well as offline field surveys are conducted, validating the proposed methodology for species identification that can be further extended to geo-localisation of flora and fauna in any ecosystem.

Object Detection of Small Insects in Time-Lapse Camera Recordings

As pollinators, insects play a crucial role in ecosystem management and world food production. However, insect populations are declining, necessitating efficient insect monitoring methods. Existing methods analyze video or time-lapse images of insects in nature, but analysis is challenging as insects are small objects in complex and dynamic natural vegetation scenes. In this work, we provide a dataset of primarily honeybees visiting three different plant species during two months of the summer. The dataset consists of 107,387 annotated time-lapse images from multiple cameras, including 9423 annotated insects. We present a method for detecting insects in time-lapse RGB images, which consists of a two-step process. Firstly, the time-lapse RGB images are preprocessed to enhance insects in the images. This motion-informed enhancement technique uses motion and colors to enhance insects in images. Secondly, the enhanced images are subsequently fed into a convolutional neural network (CNN) object detector. The method improves on the deep learning object detectors You Only Look Once (YOLO) and faster region-based CNN (Faster R-CNN). Using motion-informed enhancement, the YOLO detector improves the average micro F1-score from 0.49 to 0.71, and the Faster R-CNN detector improves the average micro F1-score from 0.32 to 0.56. Our dataset and proposed method provide a step forward for automating the time-lapse camera monitoring of flying insects.

Real-Time Recognition and Detection of Bactrocera minax (Diptera: Trypetidae) Grooming Behavior Using Body Region Localization and Improved C3D Network

Pest management has long been a critical aspect of crop protection. Insect behavior is of great research value as an important indicator for assessing insect characteristics. Currently, insect behavior research is increasingly based on the quantification of behavior. Traditional manual observation and analysis methods can no longer meet the requirements of data volume and observation time. In this paper, we propose a method based on region localization combined with an improved 3D convolutional neural network for six grooming behaviors of Bactrocera minax: head grooming, foreleg grooming, fore-mid leg grooming, mid-hind leg grooming, hind leg grooming, and wing grooming. The overall recognition accuracy reached 93.46%. We compared the results obtained from the detection model with manual observations; the average difference was about 12%. This shows that the model reached a level close to manual observation. Additionally, recognition time using this method is only one-third of that required for manual observation, making it suitable for real-time detection needs. Experimental data demonstrate that this method effectively eliminates the interference caused by the walking behavior of Bactrocera minax, enabling efficient and automated detection of grooming behavior. Consequently, it offers a convenient means of studying pest characteristics in the field of crop protection.

Frame Structure Fault Diagnosis Based on a High-Precision Convolution Neural Network

Structural health monitoring and fault diagnosis are important scientific issues in mechanical engineering, civil engineering, and other disciplines. The basic premise of structural health work is to be able to accurately diagnose the fault in the structure. Therefore, the accurate fault diagnosis of structure can not only ensure the safe operation of mechanical equipment and the safe use of civil construction, but also ensure the safety of people's lives and property. In order to improve the accuracy fault diagnosis of frame structure under noise conditions, the existing Convolutional Neural Network with Training Interference (TICNN) model is improved, and a new convolutional neural network model with strong noise resistance is proposed. In order to verify THE superiority of the proposed improved TICNN in anti-noise, comparative experiments are carried out by using TICNN, One Dimensional Convolution Neural Network (1DCNN) and First Layer Wide Convolution Kernel Deep Convolution Neural Network (WDCNN). The experimental results show that the improved TICNN has the best anti-noise ability. Based on the improved TICNN, the fault diagnosis experiment of a four-story steel structure model is carried out. The experimental results show that the improved TICNN can obtain high diagnostic accuracy under strong noise conditions, which verifies the advantages of the improved TICNN.

Weather Sensitivity of Sugar Bait Trapping of Nocturnal Moths: A Case Study from Northern Europe

Assemblages of insects need to be quantitatively sampled in the context of various research questions. Light trapping is the most widely used method for sampling nocturnal Lepidoptera. Attracting moths to sugar baits offers a viable alternative. However, this method is rarely used in professional research despite its popularity among amateur lepidopterists. As the activity of insects is strongly dependent on ambient conditions, the sensitivity of any trapping method to weather parameters needs to be known for the quantitative interpretation of trapping results. In the present paper, we report data on the weather dependence of moth catches obtained by automatic bait traps. The study was performed in Estonia, representing the European hemiboreal forest zone. Portable weather stations set up next to each of the traps were used for collecting weather data. Both abundance and diversity of the moths in the catches depended strongly positively on temperature and negatively on air humidity. Diversity was also negatively correlated with air pressure and positively with the change in pressure during the night. The results show that in situ recording of weather parameters in connection to insect trapping provides useful insights for the study of insect behaviour and the interpretation of the results of monitoring projects.

An overview of remote monitoring methods in biodiversity conservation

Conservation of biodiversity is critical for the coexistence of humans and the sustenance of other living organisms within the ecosystem. Identification and prioritization of specific regions to be conserved are impossible without proper information about the sites. Advanced monitoring agencies like the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) had accredited that the sum total of species that are now threatened with extinction is higher than ever before in the past and are progressing toward extinct at an alarming rate. Besides this, the conceptualized global responses to these crises are still inadequate and entail drastic changes. Therefore, more sophisticated monitoring and conservation techniques are required which can simultaneously cover a larger surface area within a stipulated time frame and gather a large pool of data. Hence, this study is an overview of remote monitoring methods in biodiversity conservation via a survey of evidence-based reviews and related studies, wherein the description of the application of some technology for biodiversity conservation and monitoring is highlighted. Finally, the paper also describes various transformative smart technologies like artificial intelligence (AI) and/or machine learning algorithms for enhanced working efficiency of currently available techniques that will aid remote monitoring methods in biodiversity conservation.

A Review of Automated Bioacoustics and General Acoustics Classification Research

Automated bioacoustics classification has received increasing attention from the research community in recent years due its cross-disciplinary nature and its diverse application. Applications in bioacoustics classification range from smart acoustic sensor networks that investigate the effects of acoustic vocalizations on species to context-aware edge devices that anticipate changes in their environment adapt their sensing and processing accordingly. The research described here is an in-depth survey of the current state of bioacoustics classification and monitoring. The survey examines bioacoustics classification alongside general acoustics to provide a representative picture of the research landscape. The survey reviewed 124 studies spanning eight years of research. The survey identifies the key application areas in bioacoustics research and the techniques used in audio transformation and feature extraction. The survey also examines the classification algorithms used in bioacoustics systems. Lastly, the survey examines current challenges, possible opportunities, and future directions in bioacoustics.

Accurate image-based identification of macroinvertebrate specimens using deep learning-How much training data is needed?

Image-based methods for species identification offer cost-efficient solutions for biomonitoring. This is particularly relevant for invertebrate studies, where bulk samples often represent insurmountable workloads for sorting, identifying, and counting individual specimens. On the other hand, image-based classification using deep learning tools have strict requirements for the amount of training data, which is often a limiting factor. Here, we examine how classification accuracy increases with the amount of training data using the BIODISCOVER imaging system constructed for image-based classification and biomass estimation of invertebrate specimens. We use a balanced dataset of 60 specimens of each of 16 taxa of freshwater macroinvertebrates to systematically quantify how classification performance of a convolutional neural network (CNN) increases for individual taxa and the overall community as the number of specimens used for training is increased. We show a striking 99.2% classification accuracy when the CNN (EfficientNet-B6) is trained on 50 specimens of each taxon, and also how the lower classification accuracy of models trained on less data is particularly evident for morphologically similar species placed within the same taxonomic order. Even with as little as 15 specimens used for training, classification accuracy reached 97%. Our results add to a recent body of literature showing the huge potential of image-based methods and deep learning for specimen-based research, and furthermore offers a perspective to future automatized approaches for deriving ecological data from bulk arthropod samples.

A Multimodal Sensing Platform for Interdisciplinary Research in Agrarian Environments

Agricultural and environmental monitoring programs often require labor-intensive inputs and substantial costs to manually gather data from remote field locations. Recent advances in the Internet of Things enable the construction of wireless sensor systems to automate these remote monitoring efforts. This paper presents the design of a modular system to serve as a research platform for outdoor sensor development and deployment. The advantages of this system include low power consumption (enabling solar charging), the use of commercially available electronic parts for lower-cost and scaled up deployments, and the flexibility to include internal electronics and external sensors, allowing novel applications. In addition to tracking environmental parameters, the modularity of this system brings the capability to measure other non-traditional elements. This capability is demonstrated with two different agri- and aquacultural field applications: tracking moth phenology and monitoring bivalve gaping. Collection of these signals in conjunction with environmental parameters could provide a holistic and context-aware data analysis. Preliminary experiments generated promising results, demonstrating the reliability of the system. Idle power consumption of 27.2 mW and 16.6 mW for the moth- and bivalve-tracking systems, respectively, coupled with 2.5 W solar cells allows for indefinite deployment in remote locations.

Intelligent Correction Method of Shooting Action Based on Computer Vision

Aiming at the problem that the students' long-term use of nonstandard shooting action leads to poor basketball teaching effect, an intelligent correction method of shooting action based on computer vision is proposed. Combined with the principle of computer vision, the image acquisition model of basketball shooting action is constructed. The edge contour and adaptive feature segmentation of basketball images are detected, and abnormal shooting movements are recognized. The intelligent correction model of shooting action is constructed, and the intelligent correction of shooting action is realized. Finally, through experiments, it is proved that the visual analysis and intelligent correction effect of basketball shooting action are obviously better, and it can correct shooting action in real time and accurately.

Sticky Pi is a high-frequency smart trap that enables the study of insect circadian activity under natural conditions

In the face of severe environmental crises that threaten insect biodiversity, new technologies are imperative to monitor both the identity and ecology of insect species. Traditionally, insect surveys rely on manual collection of traps, which provide abundance data but mask the large intra- and interday variations in insect activity, an important facet of their ecology. Although laboratory studies have shown that circadian processes are central to insects' biological functions, from feeding to reproduction, we lack the high-frequency monitoring tools to study insect circadian biology in the field. To address these issues, we developed the Sticky Pi, a novel, autonomous, open-source, insect trap that acquires images of sticky cards every 20 minutes. Using custom deep learning algorithms, we automatically and accurately scored where, when, and which insects were captured. First, we validated our device in controlled laboratory conditions with a classic chronobiological model organism, Drosophila melanogaster. Then, we deployed an array of Sticky Pis to the field to characterise the daily activity of an agricultural pest, Drosophila suzukii, and its parasitoid wasps. Finally, we demonstrate the wide scope of our smart trap by describing the sympatric arrangement of insect temporal niches in a community, without targeting particular taxa a priori. Together, the automatic identification and high sampling rate of our tool provide biologists with unique data that impacts research far beyond chronobiology, with applications to biodiversity monitoring and pest control as well as fundamental implications for phenology, behavioural ecology, and ecophysiology. We released the Sticky Pi project as an open community resource on https://doc.sticky-pi.com.

Edge Computing for Vision-Based, Urban-Insects Traps in the Context of Smart Cities

Our aim is to promote the widespread use of electronic insect traps that report captured pests to a human-controlled agency. This work reports on edge-computing as applied to camera-based insect traps. We present a low-cost device with high power autonomy and an adequate picture quality that reports an internal image of the trap to a server and counts the insects it contains based on quantized and embedded deep-learning models. The paper compares different aspects of performance of three different edge devices, namely ESP32, Raspberry Pi Model 4 (RPi), and Google Coral, running a deep learning framework (TensorFlow Lite). All edge devices were able to process images and report accuracy in counting exceeding 95%, but at different rates and power consumption. Our findings suggest that ESP32 appears to be the best choice in the context of this application according to our policy for low-cost devices.

An Open-Source Low-Cost Imaging System Plug-In for Pheromone Traps Aiding Remote Insect Pest Population Monitoring in Fruit Crops

This note describes the development of a plug-in imaging system for pheromone delta traps used in pest population monitoring. The plug-in comprises an RGB imaging sensor integrated with a microcontroller unit and associated hardware for optimized power usage and data capture. The plug-in can be attached to the top of a modified delta trap to realize periodic image capture of the trap liner (17.8 cm × 17.8 cm). As configured, the captured images are stored on a microSD card with ~0.01 cm2 pixel−1 spatial resolution. The plug-in hardware is configured to conserve power, as it enters in sleep mode during idle operation. Twenty traps with plug-in units were constructed and evaluated in the 2020 field season for codling moth (Cydia pomonella) population monitoring in a research study. The units reliably captured images at daily interval over the course of two weeks with a 350 mAh DC power source. The captured images provided the temporal population dynamics of codling moths, which would otherwise be achieved through daily manual trap monitoring. The system’s build cost is about $33 per unit, and it has potential for scaling to commercial applications through Internet of Things-enabled technologies integration.

Confronting Deep-Learning and Biodiversity Challenges for Automatic Video-Monitoring of Marine Ecosystems

With the availability of low-cost and efficient digital cameras, ecologists can now survey the world's biodiversity through image sensors, especially in the previously rather inaccessible marine realm. However, the data rapidly accumulates, and ecologists face a data processing bottleneck. While computer vision has long been used as a tool to speed up image processing, it is only since the breakthrough of deep learning (DL) algorithms that the revolution in the automatic assessment of biodiversity by video recording can be considered. However, current applications of DL models to biodiversity monitoring do not consider some universal rules of biodiversity, especially rules on the distribution of species abundance, species rarity and ecosystem openness. Yet, these rules imply three issues for deep learning applications: the imbalance of long-tail datasets biases the training of DL models; scarce data greatly lessens the performances of DL models for classes with few data. Finally, the open-world issue implies that objects that are absent from the training dataset are incorrectly classified in the application dataset. Promising solutions to these issues are discussed, including data augmentation, data generation, cross-entropy modification, few-shot learning and open set recognition. At a time when biodiversity faces the immense challenges of climate change and the Anthropocene defaunation, stronger collaboration between computer scientists and ecologists is urgently needed to unlock the automatic monitoring of biodiversity.

DiversityScanner: Robotic handling of small invertebrates with machine learning methods

Invertebrate biodiversity remains poorly understood although it comprises much of the terrestrial animal biomass, most species and supplies many ecosystem services. The main obstacle is specimen-rich samples obtained with quantitative sampling techniques (e.g., Malaise trapping). Traditional sorting requires manual handling, while molecular techniques based on metabarcoding lose the association between individual specimens and sequences and thus struggle with obtaining precise abundance information. Here we present a sorting robot that prepares specimens from bulk samples for barcoding. It detects, images and measures individual specimens from a sample and then moves them into the wells of a 96-well microplate. We show that the images can be used to train convolutional neural networks (CNNs) that are capable of assigning the specimens to 14 insect taxa (usually families) that are particularly common in Malaise trap samples. The average assignment precision for all taxa is 91.4% (75%-100%). This ability of the robot to identify common taxa then allows for taxon-specific subsampling, because the robot can be instructed to only pick a prespecified number of specimens for abundant taxa. To obtain biomass information, the images are also used to measure specimen length and estimate body volume. We outline how the DiversityScanner can be a key component for tackling and monitoring invertebrate diversity by combining molecular and morphological tools: the images generated by the robot become training images for machine learning once they are labelled with taxonomic information from DNA barcodes. We suggest that a combination of automation, machine learning and DNA barcoding has the potential to tackle invertebrate diversity at an unprecedented scale.

Remote monitoring of Cydia pomonella adults among an assemblage of nontargets in sex pheromone-kairomone-baited smart traps

BACKGROUND

Captures of codling moth, Cydia pomonella (L.), in traps are used to establish action thresholds and time insecticide sprays. The need for frequent trap inspections in often remote orchards has created a niche for remote sensing smart traps. A smart trap baited with a five-component pheromone-kairomone blend was evaluated for codling moth monitoring among an assemblage of other nontargets in apple and pear orchards.

RESULTS

Codling moth captures did not differ between the smart trap and a standard trap when both were checked manually. However, the correlation between automatic and manual counts of codling moth in the smart traps was low, R²  = 0.66 ÷ 0.87. False-negative identifications by the smart trap were infrequent <5%, but false-positive identifications accounted for up to 67% of the count. These errors were primarily due to the misidentification of three moth species of fairly similar-size to codling moth: apple clearwing moth Synanthedon myopaeformis (Borkhausen), oriental fruit moth Grapholita molesta (Busck), and carnation tortrix Cacoecimorpha pronubana (Hübner). Other false-positive counts were less frequent and included the misidentifications of dipterans, other arthropods, patches of moth scales, and the double counting of some moths.

CONCLUSION

Codling moth was successfully monitored remotely with a smart trap baited with a nonselective sex pheromone-kairomone lure, but automatic counts were inflated in some orchards due to mischaracterizations of primarily similar-sized nontarget moths. Improved image-identification algorithms are needed for smart traps baited with less-selective lures and with lure sets targeting multiple species.

Deep learning and computer vision will transform entomology

Most animal species on Earth are insects, and recent reports suggest that their abundance is in drastic decline. Although these reports come from a wide range of insect taxa and regions, the evidence to assess the extent of the phenomenon is sparse. Insect populations are challenging to study, and most monitoring methods are labor intensive and inefficient. Advances in computer vision and deep learning provide potential new solutions to this global challenge. Cameras and other sensors can effectively, continuously, and noninvasively perform entomological observations throughout diurnal and seasonal cycles. The physical appearance of specimens can also be captured by automated imaging in the laboratory. When trained on these data, deep learning models can provide estimates of insect abundance, biomass, and diversity. Further, deep learning models can quantify variation in phenotypic traits, behavior, and interactions. Here, we connect recent developments in deep learning and computer vision to the urgent demand for more cost-efficient monitoring of insects and other invertebrates. We present examples of sensor-based monitoring of insects. We show how deep learning tools can be applied to exceptionally large datasets to derive ecological information and discuss the challenges that lie ahead for the implementation of such solutions in entomology. We identify four focal areas, which will facilitate this transformation: 1) validation of image-based taxonomic identification; 2) generation of sufficient training data; 3) development of public, curated reference databases; and 4) solutions to integrate deep learning and molecular tools.