Assessing the potential for deep learning and computer vision to identify bumble bee species from images

Trap colour strongly affects the ability of deep learning models to recognize insect species in images of sticky traps

BACKGROUND

The use of computer vision and deep learning models to automatically classify insect species on sticky traps has proven to be a cost- and time-efficient approach to pest monitoring. As different species are attracted to different colours, the variety of sticky trap colours poses a challenge to the performance of the models. However, the effectiveness of deep learning in classifying pests on different coloured sticky traps has not yet been sufficiently explored. In this study, we aim to investigate the influence of sticky trap colour and imaging devices on the performance of deep learning models in classifying pests on sticky traps.

RESULTS

Our results show that using the MobileNetV2 architecture with transparent sticky traps as training data, the model predicted the pest species on transparent sticky traps with an accuracy of at least 0.95 and on other sticky trap colours with at least 0.85 of the F1 score. Using a generalised linear model (GLM) and a Boruta feature selection algorithm, we also showed that the colour and architecture of the sticky traps significantly influenced the performance of the model.

CONCLUSION

Our results support the development of an automatic classification of pests on a sticky trap, which should focus on colour and deep learning architecture to achieve good results. Future studies could aim to incorporate the trap system into pest monitoring, providing more accurate and cost-effective results in a pest management programme. © 2024 The Author(s). Pest Management Science published by John Wiley & Sons Ltd on behalf of Society of Chemical Industry.

Hierarchical image classification using transfer learning to improve deep learning model performance for amazon parrots

Numerous studies have proven the potential of deep learning models for classifying wildlife. Such models can reduce the workload of experts by automating species classification to monitor wild populations and global trade. Although deep learning models typically perform better with more input data, the available wildlife data are ordinarily limited, specifically for rare or endangered species. Recently, citizen science programs have helped accumulate valuable wildlife data, but such data is still not enough to achieve the best performance of deep learning models compared to benchmark datasets. Recent studies have applied the hierarchical classification of a given wildlife dataset to improve model performance and classification accuracy. This study applied hierarchical classification by transfer learning for classifying Amazon parrot species. Specifically, a hierarchy was built based on diagnostic morphological features. Upon evaluating model performance, the hierarchical model outperformed the non-hierarchical model in detecting and classifying Amazon parrots. Notably, the hierarchical model achieved the mean Average Precision (mAP) of 0.944, surpassing the mAP of 0.908 achieved by the non-hierarchical model. Moreover, the hierarchical model improved classification accuracy between morphologically similar species. The outcomes of this study may facilitate the monitoring of wild populations and the global trade of Amazon parrots for conservation purposes.

InsectNet: Real-time identification of insects using an end-to-end machine learning pipeline

Insect pests significantly impact global agricultural productivity and crop quality. Effective integrated pest management strategies require the identification of insects, including beneficial and harmful insects. Automated identification of insects under real-world conditions presents several challenges, including the need to handle intraspecies dissimilarity and interspecies similarity, life-cycle stages, camouflage, diverse imaging conditions, and variability in insect orientation. An end-to-end approach for training deep-learning models, InsectNet, is proposed to address these challenges. Our approach has the following key features: (i) uses a large dataset of insect images collected through citizen science along with label-free self-supervised learning to train a global model, (ii) fine-tuning this global model using smaller, expert-verified regional datasets to create a local insect identification model, (iii) which provides high prediction accuracy even for species with small sample sizes, (iv) is designed to enhance model trustworthiness, and (v) democratizes access through streamlined machine learning operations. This global-to-local model strategy offers a more scalable and economically viable solution for implementing advanced insect identification systems across diverse agricultural ecosystems. We report accurate identification (>96% accuracy) of numerous agriculturally and ecologically relevant insect species, including pollinators, parasitoids, predators, and harmful insects. InsectNet provides fine-grained insect species identification, works effectively in challenging backgrounds, and avoids making predictions when uncertain, increasing its utility and trustworthiness. The model and associated workflows are available through a web-based portal accessible through a computer or mobile device. We envision InsectNet to complement existing approaches, and be part of a growing suite of AI technologies for addressing agricultural challenges.

Machine Learning Approach to Support Taxonomic Discrimination of Mayflies Species Based on Morphologic Data

Artificial intelligence (AI) and machine learning (ML) offer objective solutions in the elaboration of taxonomic keys, such as the processing of large numbers of samples, aiding in the species identification, and optimizing the time required for this process. We utilized ML to study the morphological data of eight species of Americabaetis Kluge 1992, a diverse genus in South American freshwater environments. Decision trees were employed, examining specimens from the Museu de Entomologia da Universidade Federal de Viçosa (UFVB/Brazil) and literature data. Eleven morphological traits of taxonomic importance from the literature, including frontal keel, shape of the mouthparts, and abdominal color pattern, were analyzed. The decision tree obtained with the Gini algorithm effectively differentiates eight species (40% of the known species), using only eight morphological characters. Our analysis revealed distinct groups within Americabaetis alphus Lugo-Ortiz and McCafferty 1996a, based on variations in abdominal tracheae pigmentation. This study introduces a novel approach, integrating AI techniques, biological collections, and literature data for aid in the Americabaetis species identification. It provides a valuable tool for taxonomic research on contemporary and extinct mayflies.

Inferring Taxonomic Affinities and Genetic Distances Using Morphological Features Extracted from Specimen Images: A Case Study with a Bivalve Data Set

Reconstructing the tree of life and understanding the relationships of taxa are core questions in evolutionary and systematic biology. The main advances in this field in the last decades were derived from molecular phylogenetics; however, for most species, molecular data are not available. Here, we explore the applicability of 2 deep learning methods-supervised classification approaches and unsupervised similarity learning-to infer organism relationships from specimen images. As a basis, we assembled an image data set covering 4144 bivalve species belonging to 74 families across all orders and subclasses of the extant Bivalvia, with molecular phylogenetic data being available for all families and a complete taxonomic hierarchy for all species. The suitability of this data set for deep learning experiments was evidenced by an ablation study resulting in almost 80% accuracy for identifications on the species level. Three sets of experiments were performed using our data set. First, we included taxonomic hierarchy and genetic distances in a supervised learning approach to obtain predictions on several taxonomic levels simultaneously. Here, we stimulated the model to consider features shared between closely related taxa to be more critical for their classification than features shared with distantly related taxa, imprinting phylogenetic and taxonomic affinities into the architecture and training procedure. Second, we used transfer learning and similarity learning approaches for zero-shot experiments to identify the higher-level taxonomic affinities of test species that the models had not been trained on. The models assigned the unknown species to their respective genera with approximately 48% and 67% accuracy. Lastly, we used unsupervised similarity learning to infer the relatedness of the images without prior knowledge of their taxonomic or phylogenetic affinities. The results clearly showed similarities between visual appearance and genetic relationships at the higher taxonomic levels. The correlation was 0.6 for the most species-rich subclass (Imparidentia), ranging from 0.5 to 0.7 for the orders with the most images. Overall, the correlation between visual similarity and genetic distances at the family level was 0.78. However, fine-grained reconstructions based on these observed correlations, such as sister-taxa relationships, require further work. Overall, our results broaden the applicability of automated taxon identification systems and provide a new avenue for estimating phylogenetic relationships from specimen images.

Deep learning-based image classification of sea turtles using object detection and instance segmentation models

Sea turtles exhibit high migratory rates and occupy a broad range of habitats, which in turn makes monitoring these taxa challenging. Applying deep learning (DL) models to vast image datasets collected from citizen science programs can offer promising solutions to overcome the challenge of monitoring the wide habitats of wildlife, particularly sea turtles. Among DL models, object detection models, such as the You Only Look Once (YOLO) series, have been extensively employed for wildlife classification. Despite their successful application in this domain, detecting objects in images with complex backgrounds, including underwater environments, remains a significant challenge. Recently, instance segmentation models have been developed to address this issue by providing more accurate classification of complex images compared to traditional object detection models. This study compared the performance of two state-of-the-art DL methods namely; the object detection model (YOLOv5) and instance segmentation model (YOLOv5-seg), to detect and classify sea turtles. The images were collected from iNaturalist and Google and then divided into 64% for training, 16% for validation, and 20% for test sets. Model performance during and after finishing training was evaluated by loss functions and various indexes, respectively. Based on loss functions, YOLOv5-seg demonstrated a lower error rate in detecting rather than classifying sea turtles than the YOLOv5. According to mean Average Precision (mAP) values, which reflect precision and recall, the YOLOv5-seg model showed superior performance than YOLOv5. The mAP0.5 and mAP0.5:0.95 for the YOLOv5 model were 0.885 and 0.795, respectively, whereas for the YOLOv5-seg, these values were 0.918 and 0.831, respectively. In particular, based on the loss functions and classification results, the YOLOv5-seg showed improved performance for detecting rather than classifying sea turtles compared to the YOLOv5. The results of this study may help improve sea turtle monitoring in the future.

Flower Visitation through the Lens: Exploring the Foraging Behaviour of Bombus terrestris with a Computer Vision-Based Application

To understand the processes behind pollinator declines and for the conservation of pollination services, we need to understand fundamental drivers influencing pollinator behaviour. Here, we aimed to elucidate how wild bumblebees interact with three plant species and investigated their foraging behaviour with varying flower densities. We video-recorded Bombus terrestris in 60 × 60 cm quadrats of Lotus creticus, Persicaria capitata, and Trifolium pratense in urban areas of Terceira (Azores, Portugal). For the automated bumblebee detection and counting, we created deep learning-based computer vision models with custom datasets. We achieved high model accuracy of 0.88 for Lotus and Persicaria and 0.95 for Trifolium, indicating accurate bumblebee detection. In our study, flower cover was the only factor that influenced the attractiveness of flower patches, and plant species did not have an effect. We detected a significant positive effect of flower cover on the attractiveness of flower patches for flower-visiting bumblebees. The time spent per unit of inflorescence surface area was longer on the Trifolium than those on the Lotus and Persicaria. However, our result did not indicate significant differences in the time bumblebees spent on inflorescences among the three plant species. Here, we also justify computer vision-based analysis as a reliable tool for studying pollinator behavioural ecology.

Exploring the landscape of automated species identification apps: Development, promise, and user appraisal

Two decades ago, Gaston and O'Neill (2004) deliberated on why automated species identification had not become widely employed. We no longer have to wonder: This AI-based technology is here, embedded in numerous web and mobile apps used by large audiences interested in nature. Now that automated species identification tools are available, popular, and efficient, it is time to look at how the apps are developed, what they promise, and how users appraise them. Delving into the automated species identification apps landscape, we found that free and paid apps differ fundamentally in presentation, experience, and the use of biodiversity and personal data. However, these two business models are deeply intertwined. Going forward, although big tech companies will eventually take over the landscape, citizen science programs will likely continue to have their own identification tools because of their specific purpose and their ability to create a strong sense of belonging among naturalist communities.

Deep learning for identifying bee species from images of wings and pinned specimens

One of the most challenging aspects of bee ecology and conservation is species-level identification, which is costly, time consuming, and requires taxonomic expertise. Recent advances in the application of deep learning and computer vision have shown promise for identifying large bumble bee (Bombus) species. However, most bees, such as sweat bees in the genus Lasioglossum, are much smaller and can be difficult, even for trained taxonomists, to identify. For this reason, the great majority of bees are poorly represented in the crowdsourced image datasets often used to train computer vision models. But even larger bees, such as bumble bees from the B. vagans complex, can be difficult to separate morphologically. Using images of specimens from our research collections, we assessed how deep learning classification models perform on these more challenging taxa, qualitatively comparing models trained on images of whole pinned specimens or on images of bee forewings. The pinned specimen and wing image datasets represent 20 and 18 species from 6 and 4 genera, respectively, and were used to train the EfficientNetV2L convolutional neural network. Mean test precision was 94.9% and 98.1% for pinned and wing images respectively. Results show that computer vision holds great promise for classifying smaller, more difficult to identify bees that are poorly represented in crowdsourced datasets. Images from research and museum collections will be valuable for expanding classification models to include additional species, which will be essential for large scale conservation monitoring efforts.

Comparing modern identification methods for wild bees: Metabarcoding and image-based morphological taxonomic assignment

With the decline of bee populations worldwide, studies determining current wild bee distributions and diversity are increasingly important. Wild bee identification is often completed by experienced taxonomists or by genetic analysis. The current study was designed to compare two methods of identification including: (1) morphological identification by experienced taxonomists using images of field-collected wild bees and (2) genetic analysis of composite bee legs (multiple taxa) using metabarcoding. Bees were collected from conservation grasslands in eastern Iowa in summer 2019 and identified to the lowest taxonomic unit using both methods. Sanger sequencing of individual wild bee legs was used as a positive control for metabarcoding. Morphological identification of bees using images resulted in 36 unique taxa among 22 genera, and >80% of Bombus specimens were identified to species. Metabarcoding was limited to genus-level assignments among 18 genera but resolved some morphologically similar genera. Metabarcoding did not consistently detect all genera in the composite samples, including kleptoparasitic bees. Sanger sequencing showed similar presence or absence detection results as metabarcoding but provided species-level identifications for cryptic species (i.e., Lasioglossum). Genus-specific detections were more frequent with morphological identification than metabarcoding, but certain genera such as Ceratina and Halictus were identified equally well with metabarcoding and morphology. Genera with proportionately less tissue in a composite sample were less likely to be detected using metabarcoding. Image-based methods were limited by image quality and visible morphological features, while genetic methods were limited by databases, primers, and amplification at target loci. This study shows how an image-based identification method compares with genetic techniques, and how in combination, the methods provide valuable genus- and species-level information for wild bees while preserving tissue for other analyses. These methods could be improved and transferred to a field setting to advance our understanding of wild bee distributions and to expedite conservation research.

A convolutional neural network to identify mosquito species (Diptera: Culicidae) of the genus Aedes by wing images

Accurate species identification is crucial to assess the medical relevance of a mosquito specimen, but requires intensive experience of the observers and well-equipped laboratories. In this proof-of-concept study, we developed a convolutional neural network (CNN) to identify seven Aedes species by wing images, only. While previous studies used images of the whole mosquito body, the nearly two-dimensional wings may facilitate standardized image capture and reduce the complexity of the CNN implementation. Mosquitoes were sampled from different sites in Germany. Their wings were mounted and photographed with a professional stereomicroscope. The data set consisted of 1155 wing images from seven Aedes species as well as 554 wings from different non-Aedes mosquitoes. A CNN was trained to differentiate between Aedes and non-Aedes mosquitoes and to classify the seven Aedes species based on grayscale and RGB images. Image processing, data augmentation, training, validation and testing were conducted in python using deep-learning framework PyTorch. Our best-performing CNN configuration achieved a macro F1 score of 99% to discriminate Aedes from non-Aedes mosquito species. The mean macro F1 score to predict the Aedes species was 90% for grayscale images and 91% for RGB images. In conclusion, wing images are sufficient to identify mosquito species by CNNs.

YOLO object detection models can locate and classify broad groups of flower-visiting arthropods in images

Develoment of image recognition AI algorithms for flower-visiting arthropods has the potential to revolutionize the way we monitor pollinators. Ecologists need light-weight models that can be deployed in a field setting and can classify with high accuracy. We tested the performance of three deep learning light-weight models, YOLOv5nano, YOLOv5small, and YOLOv7tiny, at object recognition and classification in real time on eight groups of flower-visiting arthropods using open-source image data. These eight groups contained four orders of insects that are known to perform the majority of pollination services in Europe (Hymenoptera, Diptera, Coleoptera, Lepidoptera) as well as other arthropod groups that can be seen on flowers but are not typically considered pollinators (e.g., spiders-Araneae). All three models had high accuracy, ranging from 93 to 97%. Intersection over union (IoU) depended on the relative area of the bounding box, and the models performed best when a single arthropod comprised a large portion of the image and worst when multiple small arthropods were together in a single image. The model could accurately distinguish flies in the family Syrphidae from the Hymenoptera that they are known to mimic. These results reveal the capability of existing YOLO models to contribute to pollination monitoring.

Aphid cluster recognition and detection in the wild using deep learning models

Aphid infestation poses a significant threat to crop production, rural communities, and global food security. While chemical pest control is crucial for maximizing yields, applying chemicals across entire fields is both environmentally unsustainable and costly. Hence, precise localization and management of aphids are essential for targeted pesticide application. The paper primarily focuses on using deep learning models for detecting aphid clusters. We propose a novel approach for estimating infection levels by detecting aphid clusters. To facilitate this research, we have captured a large-scale dataset from sorghum fields, manually selected 5447 images containing aphids, and annotated each individual aphid cluster within these images. To facilitate the use of machine learning models, we further process the images by cropping them into patches, resulting in a labeled dataset comprising 151,380 image patches. Then, we implemented and compared the performance of four state-of-the-art object detection models (VFNet, GFLV2, PAA, and ATSS) on the aphid dataset. Extensive experimental results show that all models yield stable similar performance in terms of average precision and recall. We then propose to merge close neighboring clusters and remove tiny clusters caused by cropping, and the performance is further boosted by around 17%. The study demonstrates the feasibility of automatically detecting and managing insects using machine learning models. The labeled dataset will be made openly available to the research community.

Accuracy vs. Energy: An Assessment of Bee Object Inference in Videos from On-Hive Video Loggers with YOLOv3, YOLOv4-Tiny, and YOLOv7-Tiny

A continuing trend in precision apiculture is to use computer vision methods to quantify characteristics of bee traffic in managed colonies at the hive's entrance. Since traffic at the hive's entrance is a contributing factor to the hive's productivity and health, we assessed the potential of three open-source convolutional network models, YOLOv3, YOLOv4-tiny, and YOLOv7-tiny, to quantify omnidirectional traffic in videos from on-hive video loggers on regular, unmodified one- and two-super Langstroth hives and compared their accuracies, energy efficacies, and operational energy footprints. We trained and tested the models with a 70/30 split on a dataset of 23,173 flying bees manually labeled in 5819 images from 10 randomly selected videos and manually evaluated the trained models on 3600 images from 120 randomly selected videos from different apiaries, years, and queen races. We designed a new energy efficacy metric as a ratio of performance units per energy unit required to make a model operational in a continuous hive monitoring data pipeline. In terms of accuracy, YOLOv3 was first, YOLOv7-tiny-second, and YOLOv4-tiny-third. All models underestimated the true amount of traffic due to false negatives. YOLOv3 was the only model with no false positives, but had the lowest energy efficacy and highest operational energy footprint in a deployed hive monitoring data pipeline. YOLOv7-tiny had the highest energy efficacy and the lowest operational energy footprint in the same pipeline. Consequently, YOLOv7-tiny is a model worth considering for training on larger bee datasets if a primary objective is the discovery of non-invasive computer vision models of traffic quantification with higher energy efficacies and lower operational energy footprints.

Detecting common coccinellids found in sorghum using deep learning models

Increased global production of sorghum has the potential to meet many of the demands of a growing human population. Developing automation technologies for field scouting is crucial for long-term and low-cost production. Since 2013, sugarcane aphid (SCA) Melanaphis sacchari (Zehntner) has become an important economic pest causing significant yield loss across the sorghum production region in the United States. Adequate management of SCA depends on costly field scouting to determine pest presence and economic threshold levels to spray insecticides. However, with the impact of insecticides on natural enemies, there is an urgent need to develop automated-detection technologies for their conservation. Natural enemies play a crucial role in the management of SCA populations. These insects, primary coccinellids, prey on SCA and help to reduce unnecessary insecticide applications. Although these insects help regulate SCA populations, the detection and classification of these insects is time-consuming and inefficient in lower value crops like sorghum during field scouting. Advanced deep learning software provides a means to perform laborious automatic agricultural tasks, including detection and classification of insects. However, deep learning models for coccinellids in sorghum have not been developed. Therefore, our objective was to develop and train machine learning models to detect coccinellids commonly found in sorghum and classify them according to their genera, species, and subfamily level. We trained a two-stage object detection model, specifically, Faster Region-based Convolutional Neural Network (Faster R-CNN) with the Feature Pyramid Network (FPN) and also one-stage detection models in the YOLO (You Only Look Once) family (YOLOv5 and YOLOv7) to detect and classify seven coccinellids commonly found in sorghum (i.e., Coccinella septempunctata, Coleomegilla maculata, Cycloneda sanguinea, Harmonia axyridis, Hippodamia convergens, Olla v-nigrum, Scymninae). We used images extracted from the iNaturalist project to perform training and evaluation of the Faster R-CNN-FPN and YOLOv5 and YOLOv7 models. iNaturalist is an imagery web server used to publish citizen's observations of images pertaining to living organisms. Experimental evaluation using standard object detection metrics, such as average precision (AP), AP@0.50, etc., has shown that the YOLOv7 model performs the best on the coccinellid images with an AP@0.50 as high as 97.3, and AP as high as 74.6. Our research contributes automated deep learning software to the area of integrated pest management, making it easier to detect natural enemies in sorghum.

Addressing Detection Uncertainty in Bombus affinis (Hymenoptera: Apidae) Surveys Can Improve Inferences Made From Monitoring

The U.S. Fish and Wildlife Service developed national guidelines to track species recovery of the endangered rusty patched bumble bee [Bombus affinis Cresson (Hymenoptera: Apidae)] and to investigate changes in species occupancy across space and time. As with other native bee monitoring efforts, managers have specifically acknowledged the need to address species detection uncertainty and determine the sampling effort required to infer species absence within sites. We used single-season, single-species occupancy models fit to field data collected in four states to estimate imperfect detection of B. affinis and to determine the survey effort required to achieve high confidence of species detection. Our analysis revealed a precipitous, seasonal, decline in B. affinis detection probability throughout the July through September sampling window in 2021. We estimated that six, 30-min surveys conducted in early July are required to achieve a 95% cumulative detection probability, whereas >10 surveys would be required in early August to achieve the same level of confidence. Our analysis also showed B. affinis was less likely to be detected during hot and humid days and at patches of reduced habitat quality. Bombus affinis was frequently observed on Monarda fistulosa (Lamiales: Lamiaceae), followed by [Pycnanthemum virginianum Rob. and Fernald (Lamiales: Lamiaceae)], Eutrochium maculatum Lamont (Asterales: Asteraceae), and Veronicastrum virginicum Farw. (Lamiales: Plantaginaceae). Although our research is focused on B. affinis, it is relevant for monitoring other bumble bees of conservation concern, such as B. occidentalis Greene (Hymenoptera: Apidae) and B. terricola Kirby (Hymenoptera: Apidae) for which monitoring efforts have been recently initiated and occupancy is a variable of conservation interest.

The Detection of Yarn Roll's Margin in Complex Background

Online detection of yarn roll's margin is one of the key issues in textile automation, which is related to the speed and scheduling of bobbin (empty yarn roll) replacement. The actual industrial site is characterized by uneven lighting, restricted shooting angles, diverse yarn colors and cylinder yarn types, and complex backgrounds. Due to the above characteristics, the neural network detection error is large, and the contour detection extraction edge accuracy is low. In this paper, an improved neural network algorithm is proposed, and the improved Yolo algorithm and the contour detection algorithm are integrated. First, the image is entered in the Yolo model to detect each yarn roll and its dimensions; second, the contour and dimensions of each yarn roll are accurately detected based on Yolo; third, the diameter of the yarn rolls detected by Yolo and the contour detection algorithm are fused, and then the length of the yarn rolls and the edges of the yarn rolls are calculated as measurements; finally, in order to completely eliminate the error detection, the yarn consumption speed is used to estimate the residual yarn volume and the measured and estimated values are fused using a Kalman filter. This method overcomes the effects of complex backgrounds and illumination while being applicable to different types of yarn rolls. It is experimentally verified that the average measurement error of the cylinder yarn diameter is less than 8.6 mm, and the measurement error of the cylinder yarn length does not exceed 3 cm.

Transfer Learning Approach and Nucleus Segmentation with MedCLNet Colon Cancer Database

Machine learning has been recently used especially in the medical field. In the diagnosis of serious diseases such as cancer, deep learning techniques can be used to reduce the workload of experts and to produce quick solutions. The nuclei found in the histopathology dataset are an essential parameter in disease detection. The nucleus segmentation was performed using the colorectal histology MNIST dataset for nucleus detection in this study. The graph theory, PSO, watershed, and random walker algorithms were used for the segmentation process. In addition, we present the 10-class MedCLNet visual dataset consisting of the NCT-CRC-HE-100 K dataset, LC25000 dataset, and GlaS dataset that can be used in transfer learning studies from deep learning techniques. The study proposes a transfer learning technique using the MedCLNet database. Deep neural networks pre-trained with the proposed transfer learning method were used in the classification with the colorectal histology MNIST dataset in the experimental process. DenseNet201, DenseNet169, InceptionResNetV2, InceptionV3, ResNet152V2, ResNet101V2, and Xception deep learning algorithms were used in transfer learning and the classification studies. The proposed approach was analyzed before and after transfer learning with different methods (DenseNet169 + SVM, DenseNet169 + GRU). In the performance measurement, using the colorectal histology MNIST dataset, 94.29% accuracy was obtained in the DenseNet169 model, which was initiated with random weights in the multi-classification study, and 95.00% accuracy after transfer learning was applied. In comparison with the results obtained from empirical studies, it was demonstrated that the proposed method produced satisfactory outcomes. The application is expected to provide a secondary evaluation for physicians in colon cancer detection and the segmentation.

Artificial intelligence versus natural selection: Using computer vision techniques to classify bees and bee mimics

Many groups of stingless insects have independently evolved mimicry of bees to fool would-be predators. To investigate this mimicry, we trained artificial intelligence (AI) algorithms—specifically, computer vision—to classify citizen scientist images of bees, bumble bees, and diverse bee mimics. For detecting bees and bumble bees, our models achieved accuracies of 91.71% and 88.86%, respectively. As a proxy for a natural predator, our models were poorest in detecting bee mimics that exhibit both aggressive and defensive mimicry. Using the explainable AI method of class activation maps, we validated that our models learn from appropriate components within the image, which in turn provided anatomical insights. Our t-SNE plot yielded perfect within-group clustering, as well as between-group clustering that grossly replicated the phylogeny. Ultimately, the transdisciplinary approaches herein can enhance global citizen science efforts as well as investigations of mimicry and morphology of bees and other insects.

•

AI models for classifying bees and bumble bees achieved 92% and 89% accuracy

•

AI models were fooled most by bee mimics exhibiting both aggressive and defensive mimicry

•

Class activation maps explained the anatomical reasoning of AI model classifications

•

t-SNE plot exhibited perfect phylogenetic clustering within and between groups

Image Classification of Amazon Parrots by Deep Learning: A Potentially Useful Tool for Wildlife Conservation

Simple Summary

Most parrot species are threatened with extinction because of habitat loss and commercial trade. Parrot conservation is vital because parrots play an important role in the ecosystem. The Amazon parrots are one of the most endangered parrot species. Monitoring their wild population and global trade is essential for their conservation. However, this is becoming more challenging because it requires manual analysis of large-scale image data. Furthermore, the morphological identification of the Amazon parrots can be difficult because they have similar morphological features. Deep learning-based object detection models are useful tools for monitoring wild populations and global trade. In this study, 26 Amazon parrot species were classified using eight object detection models. The object detection model, which showed the highest accuracy, classified the 26 Amazon parrot species at 90.7% on average. The continuous development of deep learning models for classifying Amazon parrots might help to improve the ability to monitor their wild populations and global trade.

Abstract

Parrots play a crucial role in the ecosystem by performing various roles, such as consuming the reproductive structures of plants and dispersing plant seeds. However, most are threatened because of habitat loss and commercial trade. Amazon parrots are one of the most traded and illegally traded parrots. Therefore, monitoring their wild populations and global trade is crucial for their conservation. However, monitoring wild populations is becoming more challenging because the manual analysis of large-scale datasets of images obtained from camera trap methods is labor-intensive and time consuming. Monitoring the wildlife trade is difficult because of the large quantities of wildlife trade. Amazon parrots can be difficult to identify because of their morphological similarity. Object detection models have been widely used for automatic and accurate species classification. In this study, to classify 26 Amazon parrot species, 8 Single Shot MultiBox Detector models were assessed. Among the eight models, the DenseNet121 model showed the highest mean average precision at 88.9%. This model classified the 26 Amazon parrot species at 90.7% on average. Continuous improvement of deep learning models classifying Amazon parrots may support monitoring wild populations and the global trade of these species.

Integrating Global Citizen Science Platforms to Enable Next-Generation Surveillance of Invasive and Vector Mosquitoes

Mosquito-borne diseases continue to ravage humankind with >700 million infections and nearly one million deaths every year. Yet only a small percentage of the >3500 mosquito species transmit diseases, necessitating both extensive surveillance and precise identification. Unfortunately, such efforts are costly, time-consuming, and require entomological expertise. As envisioned by the Global Mosquito Alert Consortium, citizen science can provide a scalable solution. However, disparate data standards across existing platforms have thus far precluded truly global integration. Here, utilizing Open Geospatial Consortium standards, we harmonized four data streams from three established mobile apps—Mosquito Alert, iNaturalist, and GLOBE Observer’s Mosquito Habitat Mapper and Land Cover—to facilitate interoperability and utility for researchers, mosquito control personnel, and policymakers. We also launched coordinated media campaigns that generated unprecedented numbers and types of observations, including successfully capturing the first images of targeted invasive and vector species. Additionally, we leveraged pooled image data to develop a toolset of artificial intelligence algorithms for future deployment in taxonomic and anatomical identification. Ultimately, by harnessing the combined powers of citizen science and artificial intelligence, we establish a next-generation surveillance framework to serve as a united front to combat the ongoing threat of mosquito-borne diseases worldwide.

Emerging technologies revolutionise insect ecology and monitoring

Insects are the most diverse group of animals on Earth, but their small size and high diversity have always made them challenging to study. Recent technological advances have the potential to revolutionise insect ecology and monitoring. We describe the state of the art of four technologies (computer vision, acoustic monitoring, radar, and molecular methods), and assess their advantages, current limitations, and future potential. We discuss how these technologies can adhere to modern standards of data curation and transparency, their implications for citizen science, and their potential for integration among different monitoring programmes and technologies. We argue that they provide unprecedented possibilities for insect ecology and monitoring, but it will be important to foster international standards via collaboration.

Multi-Information Model for Large-Flowered Chrysanthemum Cultivar Recognition and Classification

The traditional Chinese large-flowered chrysanthemum is one of the cultivar groups of chrysanthemum (Chrysanthemum × morifolium Ramat.) with great morphological variation based on many cultivars. Some experts have established several large-flowered chrysanthemum classification systems by using the method of comparative morphology. However, for many cultivars, accurate recognition and classification are still a problem. Combined with the comparative morphological traits of selected samples, we proposed a multi-information model based on deep learning to recognize and classify large-flowered chrysanthemum. In this study, we collected the images of 213 large-flowered chrysanthemum cultivars in two consecutive years, 2018 and 2019. Based on the 2018 dataset, we constructed a multi-information classification model using non-pre-trained ResNet18 as the backbone network. The model achieves 70.62% top-5 test accuracy for the 2019 dataset. We explored the ability of image features to represent the characteristics of large-flowered chrysanthemum. The affinity propagation (AP) clustering shows that the features are sufficient to discriminate flower colors. The principal component analysis (PCA) shows the petal type has a better interpretation than the flower type. The training sample processing, model training scheme, and learning rate adjustment method affected the convergence and generalization of the model. The non-pre-trained model overcomes the problem of focusing on texture by ignoring colors with the ImageNet pre-trained model. These results lay a foundation for the automated recognition and classification of large-flowered chrysanthemum cultivars based on image classification.

Wild Bee Nutritional Ecology: Integrative Strategies to Assess Foraging Preferences and Nutritional Requirements

Bees depend on flowering plants for their nutrition, and reduced availability of floral resources is a major driver of declines in both managed and wild bee populations. Understanding the nutritional needs of different bee species, and how these needs are met by the varying nutritional resources provided by different flowering plant taxa, can greatly inform land management recommendations to support bee populations and their associated ecosystem services. However, most bee nutrition research has focused on the three most commonly managed and commercially reared bee taxa—honey bees, bumble bees, and mason bees—with fewer studies focused on wild bees and other managed species, such as leafcutting bees, stingless bees, and alkali bees. Thus, we have limited information about the nutritional requirements and foraging preferences of the vast majority of bee species. Here, we discuss the approaches traditionally used to understand bee nutritional ecology: identification of floral visitors of selected focal plant species, evaluation of the foraging preferences of adults in selected focal bee species, evaluation of the nutritional requirements of focal bee species (larvae or adults) in controlled settings, and examine how these methods may be adapted to study a wider range of bee species. We also highlight emerging technologies that have the potential to greatly facilitate studies of the nutritional ecology of wild bee species, as well as evaluate bee nutritional ecology at significantly larger spatio-temporal scales than were previously feasible. While the focus of this review is on bee species, many of these techniques can be applied to other pollinator taxa as well.

Assessment of deep convolutional neural network models for species identification of forensically-important fly maggots based on images of posterior spiracles

Forensic entomology is the branch of forensic science that is related to using arthropod specimens found in legal issues. Fly maggots are one of crucial pieces of evidence that can be used for estimating post-mortem intervals worldwide. However, the species-level identification of fly maggots is difficult, time consuming, and requires specialized taxonomic training. In this work, a novel method for the identification of different forensically-important fly species is proposed using convolutional neural networks (CNNs). The data used for the experiment were obtained from a digital camera connected to a compound microscope. We compared the performance of four widely used models that vary in complexity of architecture to evaluate tradeoffs in accuracy and speed for species classification including ResNet-101, Densenet161, Vgg19_bn, and AlexNet. In the validation step, all of the studied models provided 100% accuracy for identifying maggots of 4 species including Chrysomya megacephala (Diptera: Calliphoridae), Chrysomya (Achoetandrus) rufifacies (Diptera: Calliphoridae), Lucilia cuprina (Diptera: Calliphoridae), and Musca domestica (Diptera: Muscidae) based on images of posterior spiracles. However, AlexNet showed the fastest speed to process the identification model and presented a good balance between performance and speed. Therefore, the AlexNet model was selected for the testing step. The results of the confusion matrix of AlexNet showed that misclassification was found between C. megacephala and C. (Achoetandrus) rufifacies as well as between C. megacephala and L. cuprina. No misclassification was found for M. domestica. In addition, we created a web-application platform called thefly.ai to help users identify species of fly maggots in their own images using our classification model. The results from this study can be applied to identify further species by using other types of images. This model can also be used in the development of identification features in mobile applications. This study is a crucial step for integrating information from biology and AI-technology to develop a novel platform for use in forensic investigation.

A Computer Vision-Based Approach for Tick Identification Using Deep Learning Models

A wide range of pathogens, such as bacteria, viruses, and parasites can be transmitted by ticks and can cause diseases, such as Lyme disease, anaplasmosis, or Rocky Mountain spotted fever. Landscape and climate changes are driving the geographic range expansion of important tick species. The morphological identification of ticks is critical for the assessment of disease risk; however, this process is time-consuming, costly, and requires qualified taxonomic specialists. To address this issue, we constructed a tick identification tool that can differentiate the most encountered human-biting ticks, Amblyomma americanum, Dermacentor variabilis, and Ixodes scapularis, by implementing artificial intelligence methods with deep learning algorithms. Many convolutional neural network (CNN) models (such as VGG, ResNet, or Inception) have been used for image recognition purposes but it is still a very limited application in the use of tick identification. Here, we describe the modified CNN-based models which were trained using a large-scale molecularly verified dataset to identify tick species. The best CNN model achieved a 99.5% accuracy on the test set. These results demonstrate that a computer vision system is a potential alternative tool to help in prescreening ticks for identification, an earlier diagnosis of disease risk, and, as such, could be a valuable resource for health professionals.

BacDive in 2022: the knowledge base for standardized bacterial and archaeal data

The bacterial metadatabase BacDive (https://bacdive.dsmz.de) has developed into a leading database for standardized prokaryotic data on strain level. With its current release (07/2021) the database offers information for 82 892 bacterial and archaeal strains covering taxonomy, morphology, cultivation, metabolism, origin, and sequence information within 1048 data fields. By integrating high-quality data from additional culture collections as well as detailed information from species descriptions, the amount of data provided has increased by 30% over the past three years. A newly developed query builder tool in the advanced search now allows complex database queries. Thereby bacterial strains can be systematically searched based on combinations of their attributes, e.g. growth and metabolic features for biotechnological applications or to identify gaps in the present knowledge about bacteria. A new interactive dashboard provides a statistic overview over the most important data fields. Additional new features are improved genomic sequence data, integrated NCBI TaxIDs and links to BacMedia, the new sister database on cultivation media. To improve the findability and interpretation of data through search engines, data in BacDive are annotated with bioschemas.org terms.

Identification of Oil Tea (Camellia oleifera C.Abel) Cultivars Using EfficientNet-B4 CNN Model with Attention Mechanism

Cultivar identification is a basic task in oil tea (Camellia oleifera C.Abel) breeding, quality analysis, and an adjustment in the industrial structure. However, because the differences in texture, shape, and color under different cultivars of oil tea are usually inconspicuous and subtle, the identification of oil tea cultivars can be a significant challenge. The main goal of this study is to propose an automatic and accurate method for identifying oil tea cultivars. In this study, a new deep learning model is built, called EfficientNet-B4-CBAM, to identify oil tea cultivars. First, 4725 images containing four cultivars were collected to build an oil tea cultivar identification dataset. EfficientNet-B4 was selected as the basic model of oil tea cultivar identification, and the Convolutional Block Attention Module (CBAM) was integrated into EfficientNet-B4 to build EfficientNet-B4-CBAM, thereby improving the focusing ability of the fruit areas and the information expression capability of the fruit areas. Finally, the cultivar identification capability of EfficientNet-B4-CBAM was tested on the testing dataset and compared with InceptionV3, VGG16, ResNet50, EfficientNet-B4, and EfficientNet-B4-SE. The experiment results showed that the EfficientNet-B4-CBAM model achieves an overall accuracy of 97.02% and a kappa coefficient of 0.96, which is higher than that of other methods used in comparative experiments. In addition, gradient-weighted class activation mapping network visualization also showed that EfficientNet-B4-CBAM can pay more attention to the fruit areas that play a key role in cultivar identification. This study provides new effective strategies and a theoretical basis for the application of deep learning technology in the identification of oil tea cultivars and provides technical support for the automatic identification and non-destructive testing of oil tea cultivars.

The Detection of Thread Roll's Margin Based on Computer Vision

The automatic detection of the thread roll's margin is one of the kernel problems in the textile field. As the traditional detection method based on the thread's tension has the disadvantages of high cost and low reliability, this paper proposes a technology that installs a camera on a mobile robot and uses computer vision to detect the thread roll's margin. Before starting, we define a thread roll's margin as follows: The difference between the thread roll's radius and the bobbin's radius. Firstly, we capture images of the thread roll's end surface. Secondly, we obtain the bobbin's image coordinates by calculating the image's convolutions with a Circle Gradient Operator. Thirdly, we fit the thread roll and bobbin's contours into ellipses, and then delete false detections according to the bobbin's image coordinates. Finally, we restore every sub-image of the thread roll by a perspective transformation method, and establish the conversion relationship between the actual size and pixel size. The difference value of the two concentric circles' radii is the thread roll's margin. However, there are false detections and these errors may be more than 19.4 mm when the margin is small. In order to improve the precision and delete false detections, we use deep learning to detect thread roll and bobbin's radii and then can calculate the thread roll's margin. After that, we fuse the two results. However, the deep learning method also has some false detections. As such, in order to eliminate the false detections completely, we estimate the thread roll's margin according to thread consumption speed. Lastly, we use a Kalman Filter to fuse the measured value and estimated value; the average error is less than 5.7 mm.