1
|
Gunduz H, Gunal S. A lightweight convolutional neural network (CNN) model for diatom classification: DiatomNet. PeerJ Comput Sci 2024; 10:e1970. [PMID: 38660184 PMCID: PMC11042002 DOI: 10.7717/peerj-cs.1970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 03/11/2024] [Indexed: 04/26/2024]
Abstract
Diatoms are a type of algae with many species. Accurate and quick classification of diatom species is important in many fields, such as water quality analysis and weather change forecasting. Traditional methods for diatom classification, specifically morphological taxonomy and molecular detection, are time-consuming and may not provide satisfactory performance. However, in recent years, deep learning has demonstrated impressive performance in this task, just like other image classification problems. On the other hand, networks with more layers do not guarantee increased accuracy. While increasing depth can be useful in capturing complex features and patterns, it also introduces challenges such as vanishing gradients, overfitting, and optimization challenges. Therefore, in our work, we propose DiatomNet, a lightweight convolutional neural network (CNN) model that can classify diatom species accurately while requiring low computing resources. A recently introduced dataset consisting of 3,027 diatom images and 68 diatom species is used to train and evaluate the model. The model is compared with well-known and successful CNN models (i.e., AlexNet, GoogleNet, Inceptionv3, ResNet18, VGG16, and Xception) and their customized versions obtained with transfer learning. The comparison is based on several success metrics: accuracy, precision, recall, F-measure, number of learnable parameters, training, and prediction time. Eventually, the experimental results reveal that DiatomNet outperforms the other models regarding all metrics with just a few exceptions. Therefore, it is a lightweight but strong candidate for diatom classification tasks.
Collapse
Affiliation(s)
- Huseyin Gunduz
- Department of Computer Engineering, Eskisehir Technical University, Eskisehir, Turkiye
| | - Serkan Gunal
- Department of Computer Engineering, Eskisehir Technical University, Eskisehir, Turkiye
| |
Collapse
|
2
|
Pu S, Zhang F, Shu Y, Fu W. Microscopic image recognition of diatoms based on deep learning. JOURNAL OF PHYCOLOGY 2023; 59:1166-1178. [PMID: 37994558 DOI: 10.1111/jpy.13390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Revised: 09/03/2023] [Accepted: 09/05/2023] [Indexed: 11/24/2023]
Abstract
Diatoms are a crucial component in the study of aquatic ecosystems and ancient environmental records. However, traditional methods for identifying diatoms, such as morphological taxonomy and molecular detection, are costly, are time consuming, and have limitations. To address these issues, we developed an extensive collection of diatom images, consisting of 7983 images from 160 genera and 1042 species, which we expanded to 49,843 through preprocessing, segmentation, and data augmentation. Our study compared the performance of different algorithms, including backbones, batch sizes, dynamic data augmentation, and static data augmentation on experimental results. We determined that the ResNet152 network outperformed other networks, producing the most accurate results with top-1 and top-5 accuracies of 85.97% and 95.26%, respectively, in identifying 1042 diatom species. Additionally, we propose a method that combines model prediction and cosine similarity to enhance the model's performance in low-probability predictions, achieving an 86.07% accuracy rate in diatom identification. Our research contributes significantly to the recognition and classification of diatom images and has potential applications in water quality assessment, ecological monitoring, and detecting changes in aquatic biodiversity.
Collapse
Affiliation(s)
- Siyue Pu
- College of Computer and Information Engineering (College of Artificial Intelligence), Nanjing Tech University, Nanjing, China
| | - Fan Zhang
- Ocean College, Zhejiang University, Zhoushan, China
- Kavli Institute for Astrophysics and Space Research Center, Massachusettes Institute of Technology, Cambridge, Massachusetts, USA
| | - Yuexuan Shu
- Ocean College, Zhejiang University, Zhoushan, China
| | - Weiqi Fu
- Ocean College, Zhejiang University, Zhoushan, China
- Center for Systems Biology and Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, School of Engineering and Natural Sciences, University of Iceland, Reykjavik, Iceland
| |
Collapse
|
3
|
Mittal S, Srivastava S, Jayanth JP. A Survey of Deep Learning Techniques for Underwater Image Classification. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2023; 34:6968-6982. [PMID: 35104229 DOI: 10.1109/tnnls.2022.3143887] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In recent years, there has been an enormous interest in using deep learning to classify underwater images to identify various objects, such as fishes, plankton, coral reefs, seagrass, submarines, and gestures of sea divers. This classification is essential for measuring the water bodies' health and quality and protecting the endangered species. Furthermore, it has applications in oceanography, marine economy and defense, environment protection, underwater exploration, and human-robot collaborative tasks. This article presents a survey of deep learning techniques for performing underwater image classification. We underscore the similarities and differences of several methods. We believe that underwater image classification is one of the killer application that would test the ultimate success of deep learning techniques. Toward realizing that goal, this survey seeks to inform researchers about state-of-the-art on deep learning on underwater images and also motivate them to push its frontiers forward.
Collapse
|
4
|
Hellal J, Barthelmebs L, Bérard A, Cébron A, Cheloni G, Colas S, Cravo-Laureau C, De Clerck C, Gallois N, Hery M, Martin-Laurent F, Martins J, Morin S, Palacios C, Pesce S, Richaume A, Vuilleumier S. Unlocking secrets of microbial ecotoxicology: recent achievements and future challenges. FEMS Microbiol Ecol 2023; 99:fiad102. [PMID: 37669892 PMCID: PMC10516372 DOI: 10.1093/femsec/fiad102] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 07/21/2023] [Accepted: 09/04/2023] [Indexed: 09/07/2023] Open
Abstract
Environmental pollution is one of the main challenges faced by humanity. By their ubiquity and vast range of metabolic capabilities, microorganisms are affected by pollution with consequences on their host organisms and on the functioning of their environment. They also play key roles in the fate of pollutants through the degradation, transformation, and transfer of organic or inorganic compounds. Thus, they are crucial for the development of nature-based solutions to reduce pollution and of bio-based solutions for environmental risk assessment of chemicals. At the intersection between microbial ecology, toxicology, and biogeochemistry, microbial ecotoxicology is a fast-expanding research area aiming to decipher the interactions between pollutants and microorganisms. This perspective paper gives an overview of the main research challenges identified by the Ecotoxicomic network within the emerging One Health framework and in the light of ongoing interest in biological approaches to environmental remediation and of the current state of the art in microbial ecology. We highlight prevailing knowledge gaps and pitfalls in exploring complex interactions among microorganisms and their environment in the context of chemical pollution and pinpoint areas of research where future efforts are needed.
Collapse
Affiliation(s)
| | - Lise Barthelmebs
- Université de Perpignan Via Domitia, Biocapteurs – Analyse-Environnement, Perpignan, France
- Laboratoire de Biodiversité et Biotechnologies Microbiennes, USR 3579 Sorbonne Universités (UPMC) Paris 6 et CNRS Observatoire Océanologique, Banyuls-sur-Mer, France
| | - Annette Bérard
- UMR EMMAH INRAE/AU – équipe SWIFT, 228, route de l'Aérodrome, 84914 Avignon Cedex 9, France
| | | | - Giulia Cheloni
- MARBEC, Univ Montpellier, CNRS, Ifremer, IRD, Sète, France
| | - Simon Colas
- Universite de Pau et des Pays de l'Adour, E2S UPPA, CNRS, IPREM, Pau, France
| | | | - Caroline De Clerck
- AgricultureIsLife, Gembloux Agro-Bio Tech (Liege University), Passage des Déportés 2, 5030 Gembloux, Belgium
| | | | - Marina Hery
- HydroSciences Montpellier, Université de Montpellier, CNRS, IRD, Montpellier, France
| | - Fabrice Martin-Laurent
- Institut Agro Dijon, INRAE, Université de Bourgogne, Université de Bourgogne Franche-Comté, Agroécologie, 21065 Dijon, France
| | - Jean Martins
- IGE, UMR 5001, Université Grenoble Alpes, CNRS, G-INP, INRAE, IRD Grenoble, France
| | | | - Carmen Palacios
- Université de Perpignan Via Domitia, CEFREM, F-66860 Perpignan, France
- CNRS, CEFREM, UMR5110, F-66860 Perpignan, France
| | | | - Agnès Richaume
- Université de Lyon, Université Claude Bernard Lyon 1, CNRS, UMR 5557, Ecologie Microbienne, Villeurbanne, France
| | | |
Collapse
|
5
|
Sadaiappan B, Balakrishnan P, C.R. V, Vijayan NT, Subramanian M, Gauns MU. Applications of Machine Learning in Chemical and Biological Oceanography. ACS OMEGA 2023; 8:15831-15853. [PMID: 37179641 PMCID: PMC10173431 DOI: 10.1021/acsomega.2c06441] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 02/22/2023] [Indexed: 05/15/2023]
Abstract
Machine learning (ML) refers to computer algorithms that predict a meaningful output or categorize complex systems based on a large amount of data. ML is applied in various areas including natural science, engineering, space exploration, and even gaming development. This review focuses on the use of machine learning in the field of chemical and biological oceanography. In the prediction of global fixed nitrogen levels, partial carbon dioxide pressure, and other chemical properties, the application of ML is a promising tool. Machine learning is also utilized in the field of biological oceanography to detect planktonic forms from various images (i.e., microscopy, FlowCAM, and video recorders), spectrometers, and other signal processing techniques. Moreover, ML successfully classified the mammals using their acoustics, detecting endangered mammalian and fish species in a specific environment. Most importantly, using environmental data, the ML proved to be an effective method for predicting hypoxic conditions and harmful algal bloom events, an essential measurement in terms of environmental monitoring. Furthermore, machine learning was used to construct a number of databases for various species that will be useful to other researchers, and the creation of new algorithms will help the marine research community better comprehend the chemistry and biology of the ocean.
Collapse
Affiliation(s)
- Balamurugan Sadaiappan
- Department
of Biology, United Arab Emirates University, Al Ain 971, UAE
- Plankton
Laboratory, Biological Oceanography Division, CSIR-National Institute of Oceanography, Dona Paula, Goa 403004, India
| | - Preethiya Balakrishnan
- Faraday-Fleming
Laboratory, London W148TL, United Kingdom
- University
of London, London WC1E 7HU, United
Kingdom
| | - Vishal C.R.
- Plankton
Laboratory, Biological Oceanography Division, CSIR-National Institute of Oceanography, Dona Paula, Goa 403004, India
| | - Neethu T. Vijayan
- Plankton
Laboratory, Biological Oceanography Division, CSIR-National Institute of Oceanography, Dona Paula, Goa 403004, India
| | - Mahendran Subramanian
- Faraday-Fleming
Laboratory, London W148TL, United Kingdom
- Department
of Computing, Imperial College, London SW7 2AZ, United Kingdom
| | - Mangesh U. Gauns
- Plankton
Laboratory, Biological Oceanography Division, CSIR-National Institute of Oceanography, Dona Paula, Goa 403004, India
| |
Collapse
|
6
|
Zhang J, Vieira DN, Cheng Q, Zhu Y, Deng K, Zhang J, Qin Z, Sun Q, Zhang T, Ma K, Zhang X, Huang P. DiatomNet v1.0: A novel approach for automatic diatom testing for drowning diagnosis in forensically biomedical application. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107434. [PMID: 36871544 DOI: 10.1016/j.cmpb.2023.107434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 09/11/2022] [Accepted: 02/18/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Diatom testing is supportive for drowning diagnosis in forensic medicine. However, it is very time-consuming and labor-intensive for technicians to identify microscopically a handful of diatoms in sample smears, especially under complex observable backgrounds. Recently, we successfully developed a software, named DiatomNet v1.0 intended to automatically identify diatom frustules in a whole slide under a clear background. Here, we introduced this new software and performed a validation study to elucidate how DiatomNet v1.0 improved its performance with the influence of visible impurities. METHODS DiatomNet v1.0 has an intuitive, user-friendly and easy-to-learn graphical user interface (GUI) built in the Drupal and its core architecture for slide analysis including a convolutional neural network (CNN) is written in Python language. The build-in CNN model was evaluated for diatom identification under very complex observable backgrounds with mixtures of common impurities, including carbon pigments and sand sediments. Compared to the original model, the enhanced model following optimization with limited new datasets was evaluated systematically by independent testing and random control trials (RCTs). RESULTS In independent testing, the original DiatomNet v1.0 was moderately affected, especially when higher densities of impurities existed, and achieved a low recall of 0.817 and F1 score of 0.858 but good precision of 0.905. Following transfer learning with limited new datasets, the enhanced version had better results, with recall and F1 score values of 0.968. A comparative study on real slides showed that the upgraded DiatomNet v1.0 obtained F1 scores of 0.86 and 0.84 for carbon pigment and sand sediment, respectively, slightly worse than manual identification (carbon pigment: 0.91; sand sediment: 0.86), but much less time was needed. CONCLUSIONS The study verified that forensic diatom testing with aid of DiatomNet v1.0 is much more efficient than traditionally manual identification even under complex observable backgrounds. In terms of forensic diatom testing, we proposed a suggested standard on build-in model optimization and evaluation to strengthen the software's generalization in potentially complex conditions.
Collapse
Affiliation(s)
- Ji Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Duarte Nuno Vieira
- Department of Forensic Medicine, Ethics and Medical Law, Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | - Qi Cheng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Yongzheng Zhu
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Kaifei Deng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Jianhua Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Zhiqiang Qin
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Qiran Sun
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Tianye Zhang
- Shanghai Key Laboratory of Crime Scene Evidence, Institute of Criminal Science and Technology, Shanghai Municipal Public Security Bureau, Shanghai, P.R. China
| | - Kaijun Ma
- Shanghai Key Laboratory of Crime Scene Evidence, Institute of Criminal Science and Technology, Shanghai Municipal Public Security Bureau, Shanghai, P.R. China.
| | - Xiaofeng Zhang
- School of Medicine, Shanghai University, Shanghai, P.R. China.
| | - Ping Huang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China.
| |
Collapse
|
7
|
Kloster M, Burfeid-Castellanos AM, Langenkämper D, Nattkemper TW, Beszteri B. Improving deep learning-based segmentation of diatoms in gigapixel-sized virtual slides by object-based tile positioning and object integrity constraint. PLoS One 2023; 18:e0272103. [PMID: 36827378 PMCID: PMC9956069 DOI: 10.1371/journal.pone.0272103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 10/22/2022] [Indexed: 02/26/2023] Open
Abstract
Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12-17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains.
Collapse
Affiliation(s)
- Michael Kloster
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany
- * E-mail:
| | | | - Daniel Langenkämper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Tim W. Nattkemper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Bánk Beszteri
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany
| |
Collapse
|
8
|
Chong JWR, Khoo KS, Chew KW, Vo DVN, Balakrishnan D, Banat F, Munawaroh HSH, Iwamoto K, Show PL. Microalgae identification: Future of image processing and digital algorithm. BIORESOURCE TECHNOLOGY 2023; 369:128418. [PMID: 36470491 DOI: 10.1016/j.biortech.2022.128418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/02/2022] [Revised: 11/08/2022] [Accepted: 11/11/2022] [Indexed: 06/17/2023]
Abstract
The identification of microalgae species is an important tool in scientific research and commercial application to prevent harmful algae blooms (HABs) and recognizing potential microalgae strains for the bioaccumulation of valuable bioactive ingredients. The aim of this study is to incorporate rapid, high-accuracy, reliable, low-cost, simple, and state-of-the-art identification methods. Thus, increasing the possibility for the development of potential recognition applications, that could identify toxic-producing and valuable microalgae strains. Recently, deep learning (DL) has brought the study of microalgae species identification to a much higher depth of efficiency and accuracy. In doing so, this review paper emphasizes the significance of microalgae identification, and various forms of machine learning algorithms for image classification, followed by image pre-processing techniques, feature extraction, and selection for further classification accuracy. Future prospects over the challenges and improvements of potential DL classification model development, application in microalgae recognition, and image capturing technologies are discussed accordingly.
Collapse
Affiliation(s)
- Jun Wei Roy Chong
- Department of Chemical and Environmental Engineering, Faculty of Science and Engineering, University of Nottingham Malaysia Campus, Jalan Broga, Semenyih 43500, Selangor Darul Ehsan, Malaysia
| | - Kuan Shiong Khoo
- Department of Chemical Engineering and Materials Science, Yuan Ze University, Taoyuan, Taiwan; Centre for Research and Graduate Studies, University of Cyberjaya, Persiaran Bestari, 63000 Cyberjaya, Selangor, Malaysia
| | - Kit Wayne Chew
- School of Chemistry, Chemical Engineering and Biotechnology, Nanyang Technological University, 62 Nanyang Drive, Singapore, 637459, Singapore
| | - Dai-Viet N Vo
- Institute of Applied Technology and Sustainable Development, Nguyen Tat Thanh University, Ho Chi Minh City 755414, Vietnam
| | - Deepanraj Balakrishnan
- Department of Mechanical Engineering, College of Engineering, Prince Mohammad Bin Fahd University, Al Khobar, 31952, Saudi Arabia
| | - Fawzi Banat
- Department of Chemical Engineering, Khalifa University, P.O Box 127788, Abu Dhabi, United Arab Emirates
| | - Heli Siti Halimatul Munawaroh
- Study Program of Chemistry, Department of Chemistry Education, Universitas Pendidikan Indonesia, Bandung 40154, West Java, Indonesia
| | - Koji Iwamoto
- Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, 54100, Kuala Lumpur, Malaysia
| | - Pau Loke Show
- Department of Chemical and Environmental Engineering, Faculty of Science and Engineering, University of Nottingham Malaysia Campus, Jalan Broga, Semenyih 43500, Selangor Darul Ehsan, Malaysia; Zhejiang Provincial Key Laboratory for Subtropical Water Environment and Marine Biological Resources Protection, Wenzhou University, Wenzhou 325035, China; Department of Sustainable Engineering, Saveetha School of Engineering, SIMATS, Chennai 602105, India.
| |
Collapse
|
9
|
Image dataset for benchmarking automated fish detection and classification algorithms. Sci Data 2023; 10:5. [PMID: 36596792 PMCID: PMC9810604 DOI: 10.1038/s41597-022-01906-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 12/14/2022] [Indexed: 01/05/2023] Open
Abstract
Multiparametric video-cabled marine observatories are becoming strategic to monitor remotely and in real-time the marine ecosystem. Those platforms can achieve continuous, high-frequency and long-lasting image data sets that require automation in order to extract biological time series. The OBSEA, located at 4 km from Vilanova i la Geltrú at 20 m depth, was used to produce coastal fish time series continuously over the 24-h during 2013-2014. The image content of the photos was extracted via tagging, resulting in 69917 fish tags of 30 taxa identified. We also provided a meteorological and oceanographic dataset filtered by a quality control procedure to define real-world conditions affecting image quality. The tagged fish dataset can be of great importance to develop Artificial Intelligence routines for the automated identification and classification of fishes in extensive time-lapse image sets.
Collapse
|
10
|
Yu W, Xiang Q, Hu Y, Du Y, Kang X, Zheng D, Shi H, Xu Q, Li Z, Niu Y, Liu C, Zhao J. An improved automated diatom detection method based on YOLOv5 framework and its preliminary study for taxonomy recognition in the forensic diatom test. Front Microbiol 2022; 13:963059. [PMID: 36060761 PMCID: PMC9437702 DOI: 10.3389/fmicb.2022.963059] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 07/19/2022] [Indexed: 11/13/2022] Open
Abstract
The diatom test is a forensic technique that can provide supportive evidence in the diagnosis of drowning but requires the laborious observation and counting of diatoms using a microscopy with too much effort, and therefore it is promising to introduce artificial intelligence (AI) to make the test process automatic. In this article, we propose an artificial intelligence solution based on the YOLOv5 framework for the automatic detection and recognition of the diatom genera. To evaluate the performance of this AI solution in different scenarios, we collected five lab-grown diatom genera and samples of some organic tissues from drowning cases to investigate the potential upper/lower limits of the capability in detecting the diatoms and recognizing their genera. Based on the study of the article, a recall score of 0.95 together with the corresponding precision score of 0.9 were achieved on the samples of the five lab-grown diatom genera via cross-validation, and the accuracy of the evaluation in the cases of kidney and liver is above 0.85 based on the precision and recall scores, which demonstrate the effectiveness of the AI solution to be used in drowning forensic routine.
Collapse
Affiliation(s)
- Weimin Yu
- Jiangsu JITRI Sioux Technologies Co., Ltd., Suzhou, China
| | - Qingqing Xiang
- School of Forensic Medicine, Kunming Medical University, Kunming, China
| | - Yingchao Hu
- LabWorld (Suzhou) Intelligent Technology Co., Ltd., Suzhou, China
| | - Yukun Du
- School of Forensic Medicine, Southern Medical University, Guangzhou, China
| | - Xiaodong Kang
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
| | - Dongyun Zheng
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
| | - He Shi
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
| | - Quyi Xu
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
| | - Zhigang Li
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
| | - Yong Niu
- Section of Forensic Sciences, Department of Criminal Investigation, Ministry of Public Security, Beijing, China
- *Correspondence: Yong Niu
| | - Chao Liu
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
- Chao Liu
| | - Jian Zhao
- Key Laboratory of Forensic Pathology, Guangzhou Forensic Science Institute, Ministry of Public Security, Guangzhou, China
- Jian Zhao
| |
Collapse
|
11
|
Orenstein EC, Ayata S, Maps F, Becker ÉC, Benedetti F, Biard T, de Garidel‐Thoron T, Ellen JS, Ferrario F, Giering SLC, Guy‐Haim T, Hoebeke L, Iversen MH, Kiørboe T, Lalonde J, Lana A, Laviale M, Lombard F, Lorimer T, Martini S, Meyer A, Möller KO, Niehoff B, Ohman MD, Pradalier C, Romagnan J, Schröder S, Sonnet V, Sosik HM, Stemmann LS, Stock M, Terbiyik‐Kurt T, Valcárcel‐Pérez N, Vilgrain L, Wacquet G, Waite AM, Irisson J. Machine learning techniques to characterize functional traits of plankton from image data. LIMNOLOGY AND OCEANOGRAPHY 2022; 67:1647-1669. [PMID: 36247386 PMCID: PMC9543351 DOI: 10.1002/lno.12101] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/16/2021] [Revised: 04/21/2022] [Accepted: 04/27/2022] [Indexed: 06/16/2023]
Abstract
Plankton imaging systems supported by automated classification and analysis have improved ecologists' ability to observe aquatic ecosystems. Today, we are on the cusp of reliably tracking plankton populations with a suite of lab-based and in situ tools, collecting imaging data at unprecedentedly fine spatial and temporal scales. But these data have potential well beyond examining the abundances of different taxa; the individual images themselves contain a wealth of information on functional traits. Here, we outline traits that could be measured from image data, suggest machine learning and computer vision approaches to extract functional trait information from the images, and discuss promising avenues for novel studies. The approaches we discuss are data agnostic and are broadly applicable to imagery of other aquatic or terrestrial organisms.
Collapse
Affiliation(s)
- Eric C. Orenstein
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
| | - Sakina‐Dorothée Ayata
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
- Sorbonne Université, Laboratoire d'Océanographie et du Climat, Institut Pierre Simon Laplace (LOCEAN‐IPSL, SU/CNRS/IRD/MNHN)ParisFrance
| | - Frédéric Maps
- Département de BiologieUniversité LavalQuébecCanada
- Takuvik Joint International Laboratory Université Laval‐CNRS (UMI 3376), Québec‐Océan, Université LavalQuébecCanada
| | - Érica C. Becker
- Universidade Federal de Santa Catarina (UFSC)FlorianópolisSanta CatarinaBrazil
| | - Fabio Benedetti
- ETH ZürichInstitute of Biogeochemistry and Pollutant DynamicsZürichSwitzerland
| | - Tristan Biard
- Laboratoire d'Océanologie et de GéosciencesUniversité du Littoral Côte d'Opale, Université de Lille, CNRS, UMR 8187WimereuxFrance
| | | | - Jeffrey S. Ellen
- Scripps Institution of Oceanography, University of California San DiegoLa JollaCalifornia
| | - Filippo Ferrario
- Département de BiologieUniversité LavalQuébecCanada
- Takuvik Joint International Laboratory Université Laval‐CNRS (UMI 3376), Québec‐Océan, Université LavalQuébecCanada
- Department of Fisheries and OceansMaurice Lamontagne InstituteMont‐JoliQuébecCanada
| | | | - Tamar Guy‐Haim
- National Institute of Oceanography, Israel Oceanographic and Limnological ResearchHaifaIsrael
| | - Laura Hoebeke
- KERMIT, Department of Data Analysis and Mathematical ModellingGhent UniversityGhentBelgium
| | | | - Thomas Kiørboe
- Centre for Ocean Life, DTU‐AquaTechnical University of DenmarkKongens LyngbyDenmark
| | | | - Arancha Lana
- Institut Mediterrani d'Estudis Avançats (IMEDEA, UIB‐CSIC)Balearic IslandsSpain
| | | | - Fabien Lombard
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
| | | | - Séverine Martini
- Aix Marseille University, Université de Toulon, CNRS, IRD, MIO UMMarseilleFrance
| | - Albin Meyer
- Université de Lorraine, CNRS, LIECMetzFrance
| | - Klas Ove Möller
- Helmholtz‐Zentrum HereonInstitute of Carbon CycleGeesthachtGermany
| | - Barbara Niehoff
- Alfred Wegener Institute for Polar and Marine ResearchBremerhavenGermany
| | - Mark D. Ohman
- Scripps Institution of Oceanography, University of California San DiegoLa JollaCalifornia
| | | | - Jean‐Baptiste Romagnan
- IFREMER, Centre Atlantique, Laboratoire Ecologie et Modèles pour l'Halieutique (EMH)Unité HALGO, UMR DECODNantesFrance
| | | | - Virginie Sonnet
- Graduate School of OceanographyUniversity of Rhode IslandNarragansettRhode Island
| | - Heidi M. Sosik
- Woods Hole Oceanographic InstitutionWoods HoleMassachusetts
| | - Lars S. Stemmann
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
| | - Michiel Stock
- KERMIT, Department of Data Analysis and Mathematical ModellingGhent UniversityGhentBelgium
| | - Tuba Terbiyik‐Kurt
- Department of Basic SciencesCukurova University, Faculty of FisheriesAdanaTurkey
| | | | - Laure Vilgrain
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
| | | | - Anya M. Waite
- Ocean Frontier Institute, Dalhousie UniversityHalifaxNova ScotiaCanada
| | - Jean‐Olivier Irisson
- Sorbonne Université, CNRS, Laboratoire d'Océanographie de VillefrancheVillefranche‐sur‐MerFrance
| |
Collapse
|
12
|
Novel Approach to Freshwater Diatom Profiling and Identification Using Raman Spectroscopy and Chemometric Analysis. WATER 2022. [DOI: 10.3390/w14132116] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
(1) An approach with great potential for fast and cost-effective profiling and identification of diatoms in lake ecosystems is presented herein. This approach takes advantage of Raman spectroscopy. (2) The study was based on the analysis of 790 Raman spectra from 29 species, belonging to 15 genera, 12 families, 9 orders and 4 subclasses, which were analysed using chemometric methods. The Raman data were first analysed by a partial least squares regression discriminant analysis (PLS-DA) to characterise the diatom species. Furthermore, a method was developed to streamline the integrated interpretation of PLS-DA when a high number of significant components is extracted. Subsequently, an artificial neural network (ANN) was used for taxa identification from Raman data. (3) The PLS interpretation produced a Raman profile for each species reflecting its biochemical composition. The ANN models were useful to identify various taxa with high accuracy. (4) Compared to studies in the literature, involving huge datasets one to four orders of magnitude larger than ours, high sensitivity was found for the identification of Achnanthidium exiguum (67%), Fragilaria pararumpens (67%), Amphora pediculus (71%), Achnanthidium minutissimum (80%) and Melosira varians (82%).
Collapse
|
13
|
Spaulding SA, Potapova MG, Bishop IW, Lee SS, Gasperak TS, Jovanoska E, Furey PC, Edlund MB. Diatoms.org: supporting taxonomists, connecting communities. DIATOM RESEARCH : THE JOURNAL OF THE INTERNATIONAL SOCIETY FOR DIATOM RESEARCH 2022; 36:291-304. [PMID: 35958044 PMCID: PMC9359083 DOI: 10.1080/0269249x.2021.2006790] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2021] [Accepted: 09/22/2021] [Indexed: 05/23/2023]
Abstract
Consistent identification of diatoms is a prerequisite for studying their ecology, biogeography, and successful application as environmental indicators. However, taxonomic consistency among observers has been difficult to achieve, because taxonomic information is scattered across numerous literature sources, presenting challenges to the diatomist. First, literature is often inaccessible because of cost, or its location in journals that are not widely circulated. Second, taxonomic revisions of diatoms are taking place faster than floras can be updated. Finally, taxonomic information is often contradictory across literature sources. These issues can be addressed by developing a content creation community dedicated to making taxonomic, ecological, and image-based data freely available for diatom researchers. Diatoms.org represents such a content curation community, providing open, online access to a vast amount of recent and historical information on North American diatom taxonomy and ecology. The content curation community aggregates existing taxonomic information, creates new content, and provides feedback in the form of corrections and notice of literature with nomenclatural changes. The website not only addresses the needs of experienced diatom scientists for consistent identification, but is also designed to meet users at their level of expertise, including engaging the lay public in the importance of diatom science. The website now contains over 1000 species pages contributed by over 100 content contributors, from students to established scientists. The project began with the intent to provide accurate information on diatom identification, ecology, and distribution using an approach that incorporates engaging design, user feedback, and advanced data access technology. In retrospect, the project that began as an "extended electronic book" has emerged not only as a means to support taxonomists, but for practitioners to communicate and collaborate, expanding the size of and benefits to the content curation community. In this paper, we outline the development of diatoms.org, document key elements of the project, examine ongoing challenges, and consider the unexpected emergent properties, including the value of diatoms.org as a source of data. Ultimately, if the field of diatom taxonomy, ecology, and biodiversity is to be relevant, a new generation of taxonomists needs to be trained and employed using new tools. We propose that diatoms.org is in a key position to serve as a hub of training and continuity for the study of diatom biodiversity and aquatic conditions.
Collapse
Affiliation(s)
- Sarah A Spaulding
- U.S. Geological Survey/INSTAAR, 4001 Discovery Drive, Boulder, CO 80309
| | - Marina G Potapova
- The Academy of Natural Sciences of Drexel University, 1900 Benjamin Franklin Parkway, Philadelphia PA 19103
| | - Ian W Bishop
- Graduate School of Oceanography, University of Rhode Island, 215 S. Ferry Rd, Narragansett, RI 02882
| | - Sylvia S Lee
- U.S. Environmental Protection Agency, Office of Research and Development, Center for Public Health and Environmental Assessment, 1200 Pennsylvania Ave. NW, Mail code 8623-P, Washington, D.C. 20460
| | | | - Elena Jovanoska
- Department of Palaeoanthropology, Senckenberg Research Institute, Senckenberganlage 25, 60325, Frankfurt am Main, Germany
| | - Paula C Furey
- Department of Biology, St. Catherine University, 2004 Randolph Ave., St. Paul, MN 55105
| | - Mark B Edlund
- St. Croix Watershed Res. Station, Science Museum of Minnesota, Marine on St. Croix MN 55047
| |
Collapse
|
14
|
Memmolo P, Carcagnì P, Bianco V, Merola F, Goncalves da Silva Junior A, Garcia Goncalves LM, Ferraro P, Distante C. Learning Diatoms Classification from a Dry Test Slide by Holographic Microscopy. SENSORS 2020; 20:s20216353. [PMID: 33171757 PMCID: PMC7664373 DOI: 10.3390/s20216353] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 11/03/2020] [Accepted: 11/05/2020] [Indexed: 01/05/2023]
Abstract
Diatoms are among the dominant phytoplankters in marine and freshwater habitats, and important biomarkers of water quality, making their identification and classification one of the current challenges for environmental monitoring. To date, taxonomy of the species populating a water column is still conducted by marine biologists on the basis of their own experience. On the other hand, deep learning is recognized as the elective technique for solving image classification problems. However, a large amount of training data is usually needed, thus requiring the synthetic enlargement of the dataset through data augmentation. In the case of microalgae, the large variety of species that populate the marine environments makes it arduous to perform an exhaustive training that considers all the possible classes. However, commercial test slides containing one diatom element per class fixed in between two glasses are available on the market. These are usually prepared by expert diatomists for taxonomy purposes, thus constituting libraries of the populations that can be found in oceans. Here we show that such test slides are very useful for training accurate deep Convolutional Neural Networks (CNNs). We demonstrate the successful classification of diatoms based on a proper CNNs ensemble and a fully augmented dataset, i.e., creation starting from one single image per class available from a commercial glass slide containing 50 fixed species in a dry setting. This approach avoids the time-consuming steps of water sampling and labeling by skilled marine biologists. To accomplish this goal, we exploit the holographic imaging modality, which permits the accessing of a quantitative phase-contrast maps and a posteriori flexible refocusing due to its intrinsic 3D imaging capability. The network model is then validated by using holographic recordings of live diatoms imaged in water samples i.e., in their natural wet environmental condition.
Collapse
Affiliation(s)
- Pasquale Memmolo
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | - Pierluigi Carcagnì
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Monteorni snc University Campus, 73100 Lecce, Italy; (P.C.); (C.D.)
| | - Vittorio Bianco
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
- Correspondence: ; Tel.: +39-0818675201
| | - Francesco Merola
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | | | - Luis Marcos Garcia Goncalves
- Department of Computer Engineering and Automation, Federal University of Rio Grande do Norte, 59078 Natal, Brazil; (A.G.d.S.J.); (L.M.G.G.)
| | - Pietro Ferraro
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | - Cosimo Distante
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Monteorni snc University Campus, 73100 Lecce, Italy; (P.C.); (C.D.)
| |
Collapse
|