1
|
Ramos JDS, Vieira IHP, Rocha WS, Esquerdo RP, Watanabe CYV, Zanchi FB. A transfer learning approach to identify Plasmodium in microscopic images. PLoS Comput Biol 2024; 20:e1012327. [PMID: 39102445 PMCID: PMC11326699 DOI: 10.1371/journal.pcbi.1012327] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2024] [Revised: 08/15/2024] [Accepted: 07/15/2024] [Indexed: 08/07/2024] Open
Abstract
Plasmodium parasites cause Malaria disease, which remains a significant threat to global health, affecting 200 million people and causing 400,000 deaths yearly. Plasmodium falciparum and Plasmodium vivax remain the two main malaria species affecting humans. Identifying the malaria disease in blood smears requires years of expertise, even for highly trained specialists. Literature studies have been coping with the automatic identification and classification of malaria. However, several points must be addressed and investigated so these automatic methods can be used clinically in a Computer-aided Diagnosis (CAD) scenario. In this work, we assess the transfer learning approach by using well-known pre-trained deep learning architectures. We considered a database with 6222 Region of Interest (ROI), of which 6002 are from the Broad Bioimage Benchmark Collection (BBBC), and 220 were acquired locally by us at Fundação Oswaldo Cruz (FIOCRUZ) in Porto Velho Velho, Rondônia-Brazil, which is part of the legal Amazon. We exhaustively cross-validated the dataset using 100 distinct partitions with 80% train and 20% test for each considering circular ROIs (rough segmentation). Our experimental results show that DenseNet201 has a potential to identify Plasmodium parasites in ROIs (infected or uninfected) of microscopic images, achieving 99.41% AUC with a fast processing time. We further validated our results, showing that DenseNet201 was significantly better (99% confidence interval) than the other networks considered in the experiment. Our results support claiming that transfer learning with texture features potentially differentiates subjects with malaria, spotting those with Plasmodium even in Leukocytes images, which is a challenge. In Future work, we intend scale our approach by adding more data and developing a friendly user interface for CAD use. We aim at aiding the worldwide population and our local natives living nearby the legal Amazon's rivers.
Collapse
Affiliation(s)
- Jonathan da Silva Ramos
- Computer Science Department, Federal University of Rondônia (DACC/UNIR), Porto Velho, Rondônia, Brazil
| | - Ivo Henrique Provensi Vieira
- Laboratório de Bioinformática e Química Medicinal, Fundação Oswaldo Cruz Rondônia, Porto Velho, Rondônia, Brazil
- Instituto Nacional de Epidemiologia na Amazônia Ocidental (INCT-EPIAMO), Porto Velho, Rondônia, Brazil
| | - Wan Song Rocha
- Instituto Nacional de Epidemiologia na Amazônia Ocidental (INCT-EPIAMO), Porto Velho, Rondônia, Brazil
- Programa de Pós-Graduação em Biologia Experimental, Universidade Federal de Rondônia (PGBIOEXP/UNIR), Porto Velho, Rondônia, Brazil
| | - Rosimar Pires Esquerdo
- Laboratório de Bioinformática e Química Medicinal, Fundação Oswaldo Cruz Rondônia, Porto Velho, Rondônia, Brazil
| | | | - Fernando Berton Zanchi
- Laboratório de Bioinformática e Química Medicinal, Fundação Oswaldo Cruz Rondônia, Porto Velho, Rondônia, Brazil
- Instituto Nacional de Epidemiologia na Amazônia Ocidental (INCT-EPIAMO), Porto Velho, Rondônia, Brazil
- Programa de Pós-Graduação em Biologia Experimental, Universidade Federal de Rondônia (PGBIOEXP/UNIR), Porto Velho, Rondônia, Brazil
| |
Collapse
|
2
|
Sukumarran D, Hasikin K, Khairuddin ASM, Ngui R, Sulaiman WYW, Vythilingam I, Divis PCS. An optimised YOLOv4 deep learning model for efficient malarial cell detection in thin blood smear images. Parasit Vectors 2024; 17:188. [PMID: 38627870 PMCID: PMC11022477 DOI: 10.1186/s13071-024-06215-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 02/25/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Malaria is a serious public health concern worldwide. Early and accurate diagnosis is essential for controlling the disease's spread and avoiding severe health complications. Manual examination of blood smear samples by skilled technicians is a time-consuming aspect of the conventional malaria diagnosis toolbox. Malaria persists in many parts of the world, emphasising the urgent need for sophisticated and automated diagnostic instruments to expedite the identification of infected cells, thereby facilitating timely treatment and reducing the risk of disease transmission. This study aims to introduce a more lightweight and quicker model-but with improved accuracy-for diagnosing malaria using a YOLOv4 (You Only Look Once v. 4) deep learning object detector. METHODS The YOLOv4 model is modified using direct layer pruning and backbone replacement. The primary objective of layer pruning is the removal and individual analysis of residual blocks within the C3, C4 and C5 (C3-C5) Res-block bodies of the backbone architecture's C3-C5 Res-block bodies. The CSP-DarkNet53 backbone is simultaneously replaced for enhanced feature extraction with a shallower ResNet50 network. The performance metrics of the models are compared and analysed. RESULTS The modified models outperform the original YOLOv4 model. The YOLOv4-RC3_4 model with residual blocks pruned from the C3 and C4 Res-block body achieves the highest mean accuracy precision (mAP) of 90.70%. This mAP is > 9% higher than that of the original model, saving approximately 22% of the billion floating point operations (B-FLOPS) and 23 MB in size. The findings indicate that the YOLOv4-RC3_4 model also performs better, with an increase of 9.27% in detecting the infected cells upon pruning the redundant layers from the C3 Res-block bodies of the CSP-DarkeNet53 backbone. CONCLUSIONS The results of this study highlight the use of the YOLOv4 model for detecting infected red blood cells. Pruning the residual blocks from the Res-block bodies helps to determine which Res-block bodies contribute the most and least, respectively, to the model's performance. Our method has the potential to revolutionise malaria diagnosis and pave the way for novel deep learning-based bioinformatics solutions. Developing an effective and automated process for diagnosing malaria will considerably contribute to global efforts to combat this debilitating disease. We have shown that removing undesirable residual blocks can reduce the size of the model and its computational complexity without compromising its precision.
Collapse
Affiliation(s)
- Dhevisha Sukumarran
- Department of Biomedical Engineering, Faculty of Engineering, Universiti Malaya, Kuala Lumpur, Malaysia
| | - Khairunnisa Hasikin
- Department of Biomedical Engineering, Faculty of Engineering, Universiti Malaya, Kuala Lumpur, Malaysia.
- Center of Intelligent Systems for Emerging Technology (CISET), Faculty of Engineering, Universiti Malaya, 50603, Kuala Lumpur, Malaysia.
| | - Anis Salwa Mohd Khairuddin
- Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya, Kuala Lumpur, Malaysia
- Malaria Research Centre, Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, Kota Samarahan, Sarawak, Malaysia
| | - Romano Ngui
- Department of Para-Clinical Sciences, Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, Sarawak, Malaysia.
| | | | - Indra Vythilingam
- Department of Parasitology, Faculty of Medicine, Universiti Malaya, Kuala Lumpur, Malaysia
| | - Paul Cliff Simon Divis
- Malaria Research Centre, Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, Kota Samarahan, Sarawak, Malaysia
| |
Collapse
|
3
|
Zedda L, Loddo A, Di Ruberto C. YOLO-PAM: Parasite-Attention-Based Model for Efficient Malaria Detection. J Imaging 2023; 9:266. [PMID: 38132684 PMCID: PMC10744183 DOI: 10.3390/jimaging9120266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 11/14/2023] [Accepted: 11/24/2023] [Indexed: 12/23/2023] Open
Abstract
Malaria is a potentially fatal infectious disease caused by the Plasmodium parasite. The mortality rate can be significantly reduced if the condition is diagnosed and treated early. However, in many underdeveloped countries, the detection of malaria parasites from blood smears is still performed manually by experienced hematologists. This process is time-consuming and error-prone. In recent years, deep-learning-based object-detection methods have shown promising results in automating this task, which is critical to ensure diagnosis and treatment in the shortest possible time. In this paper, we propose a novel Transformer- and attention-based object-detection architecture designed to detect malaria parasites with high efficiency and precision, focusing on detecting several parasite sizes. The proposed method was tested on two public datasets, namely MP-IDB and IML. The evaluation results demonstrated a mean average precision exceeding 83.6% on distinct Plasmodium species within MP-IDB and reaching nearly 60% on IML. These findings underscore the effectiveness of our proposed architecture in automating malaria parasite detection, offering a potential breakthrough in expediting diagnosis and treatment processes.
Collapse
Affiliation(s)
- Luca Zedda
- Department of Mathematics and Computer Science, University of Cagliari, Via Ospedale 72, 09124 Cagliari, Italy;
| | - Andrea Loddo
- Department of Mathematics and Computer Science, University of Cagliari, Via Ospedale 72, 09124 Cagliari, Italy;
| | | |
Collapse
|
4
|
Dong J, Fuentes A, Yoon S, Kim H, Park DS. An iterative noisy annotation correction model for robust plant disease detection. FRONTIERS IN PLANT SCIENCE 2023; 14:1238722. [PMID: 37941667 PMCID: PMC10628849 DOI: 10.3389/fpls.2023.1238722] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Accepted: 09/22/2023] [Indexed: 11/10/2023]
Abstract
Previous work on plant disease detection demonstrated that object detectors generally suffer from degraded training data, and annotations with noise may cause the training task to fail. Well-annotated datasets are therefore crucial to build a robust detector. However, a good label set generally requires much expert knowledge and meticulous work, which is expensive and time-consuming. This paper aims to learn robust feature representations with inaccurate bounding boxes, thereby reducing the model requirements for annotation quality. Specifically, we analyze the distribution of noisy annotations in the real world. A teacher-student learning paradigm is proposed to correct inaccurate bounding boxes. The teacher model is used to rectify the degraded bounding boxes, and the student model extracts more robust feature representations from the corrected bounding boxes. Furthermore, the method can be easily generalized to semi-supervised learning paradigms and auto-labeling techniques. Experimental results show that applying our method to the Faster-RCNN detector achieves a 26% performance improvement on the noisy dataset. Besides, our method achieves approximately 75% of the performance of a fully supervised object detector when 1% of the labels are available. Overall, this work provides a robust solution to real-world location noise. It alleviates the challenges posed by noisy data to precision agriculture, optimizes data labeling technology, and encourages practitioners to further investigate plant disease detection and intelligent agriculture at a lower cost. The code will be released at https://github.com/JiuqingDong/TS_OAMIL-for-Plant-disease-detection.
Collapse
Affiliation(s)
- Jiuqing Dong
- Department of Electronic Engineering, Jeonbuk National University, Jeonju, Republic of Korea
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, Republic of Korea
| | - Alvaro Fuentes
- Department of Electronic Engineering, Jeonbuk National University, Jeonju, Republic of Korea
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, Republic of Korea
| | - Sook Yoon
- Department of Computer Engineering, Mokpo National University, Muan, Republic of Korea
| | - Hyongsuk Kim
- Department of Electronic Engineering, Jeonbuk National University, Jeonju, Republic of Korea
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, Republic of Korea
| | - Dong Sun Park
- Department of Electronic Engineering, Jeonbuk National University, Jeonju, Republic of Korea
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, Republic of Korea
| |
Collapse
|
5
|
Fu M, Wu K, Li Y, Luo L, Huang W, Zhang Q. An intelligent detection method for plasmodium based on self-supervised learning and attention mechanism. Front Med (Lausanne) 2023; 10:1117192. [PMID: 37457573 PMCID: PMC10345499 DOI: 10.3389/fmed.2023.1117192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 05/23/2023] [Indexed: 07/18/2023] Open
Abstract
Background Malaria remains a severe life-threatening disease caused by plasmodium parasites. Microscopy is widely used for malaria diagnosis. However, it relies heavily on the skills and experience of inspectors. Due to low-level medical services and the lack of skilled inspectors, misdiagnoses are frequently made in some areas. Methods In recent years, many successful applications of CNN models have been reported. Unlike images in the ImageNet, the image of plasmodium only has a tiny defect area with a large amount of information. In addition, the dataset is extremely unbalanced: the number of positive samples is much less than that of negative samples. This paper proposes a classification network by combining attention mechanism and ResNeSt for plasmodium detection and using self-supervised learning to pre-train the network. First, the positive samples were adopted to pre-train the network. Then, attention modules were taken to highlight the feature area. To support current and future research, we also constructed a plasmodium dataset with Plasmodium falciparum, Plasmodium vivax, Plasmodium ovale, and Plasmodium malaria and non-Plasmodium. Through self-supervised learning, a large amount of unlabeled data is used to mine the representational features, thus improving the feature extraction capability of the model and achieving higher accuracy, while saving the physician's labeling time and improving the classification accuracy. Results The experiments show that our model exhibits an excellent performance and that the test accuracy, sensitivity, and specificity attain 97.8%, 96.5%, and 98.9%, respectively. Conclusion The AI classification method proposed in this paper can effectively assist clinicians in the diagnosis and provide a basis for the automatic detection of malaria parasites in the future.
Collapse
Affiliation(s)
- Min Fu
- School of Aerospace Engineering, Xiamen University, Xiamen, China
| | - Kai Wu
- Department of Schistosomiasis and Endemic Diseases, Wuhan Center for Disease Control and Prevention, Wuhan, China
| | - Yuxuan Li
- School of Art and Design, Wuhan University of Technology, Wuhan, China
| | - Linkai Luo
- School of Aerospace Engineering, Xiamen University, Xiamen, China
| | - Wei Huang
- Department of Orthopaedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
| | - Qi Zhang
- Wuhan Jinyintan Hospital, Tongji Medical College of Huazhong University of Science and Technology, Wuhan, China
| |
Collapse
|
6
|
Li X, Liu R, Wang Z, Zheng G, Lv J, Fan L, Guo Y, Gao Y. Automatic Penaeus Monodon Larvae Counting via Equal Keypoint Regression with Smartphones. Animals (Basel) 2023; 13:2036. [PMID: 37370546 DOI: 10.3390/ani13122036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 06/06/2023] [Accepted: 06/07/2023] [Indexed: 06/29/2023] Open
Abstract
Today, large-scale Penaeus monodon farms no longer incubate eggs but instead purchase larvae from large-scale hatcheries for rearing. The accurate counting of tens of thousands of larvae in these transactions is a challenging task due to the small size of the larvae and the highly congested scenes. To address this issue, we present the Penaeus Larvae Counting Strategy (PLCS), a simple and efficient method for counting Penaeus monodon larvae that only requires a smartphone to capture images without the need for any additional equipment. Our approach treats two different types of keypoints as equip keypoints based on keypoint regression to determine the number of shrimp larvae in the image. We constructed a high-resolution image dataset named Penaeus_1k using images captured by five smartphones. This dataset contains 1420 images of Penaeus monodon larvae and includes general annotations for three keypoints, making it suitable for density map counting, keypoint regression, and other methods. The effectiveness of the proposed method was evaluated on a real Penaeus monodon larvae dataset. The average accuracy of 720 images with seven different density groups in the test dataset was 93.79%, outperforming the classical density map algorithm and demonstrating the efficacy of the PLCS.
Collapse
Affiliation(s)
- Ximing Li
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| | - Ruixiang Liu
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| | - Zhe Wang
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| | - Guotai Zheng
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| | - Junlin Lv
- South China Sea Fisheries Research Institute (CAFS), Guangzhou 510300, China
| | - Lanfen Fan
- College of Marine Sciences, South China Agricultural University, Guangzhou 510642, China
| | - Yubin Guo
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| | - Yuefang Gao
- College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
| |
Collapse
|
7
|
Lewis JE, Pozdnyakova O. Digital assessment of peripheral blood and bone marrow aspirate smears. Int J Lab Hematol 2023. [PMID: 37211430 DOI: 10.1111/ijlh.14082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 04/20/2023] [Indexed: 05/23/2023]
Abstract
The diagnosis of benign and neoplastic hematologic disorders relies on analysis of peripheral blood and bone marrow aspirate smears. As demonstrated by the widespread laboratory adoption of hematology analyzers for automated assessment of peripheral blood, digital analysis of these samples provides many significant benefits compared to relying solely on manual review. Nonetheless, analogous instruments for digital bone marrow aspirate smear assessment have yet to be clinically implemented. In this review, we first provide a historical overview detailing the implementation of hematology analyzers for digital peripheral blood assessment in the clinical laboratory, including the improvements in accuracy, scope, and throughput of current instruments over prior generations. We also describe recent research in digital peripheral blood assessment, particularly in the development of advanced machine learning models that may soon be incorporated into commercial instruments. Next, we provide an overview of recent research in digital assessment of bone marrow aspirate smears and how these approaches could soon lead to development and clinical adoption of instrumentation for automated bone marrow aspirate smear analysis. Finally, we describe the relative advantages and provide our vision for the future of digital assessment of peripheral blood and bone marrow aspirate smears, including what improvements we can soon expect in the hematology laboratory.
Collapse
Affiliation(s)
- Joshua E Lewis
- Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - Olga Pozdnyakova
- Department of Pathology, Brigham and Women's Hospital, Boston, Massachusetts, USA
| |
Collapse
|
8
|
Dai W, Cui Y, Wang P, Wu H, Zhang L, Bian Y, Li Y, Li Y, Hu H, Zhao J, Xu D, Kong D, Wang Y, Xu L. Classification regularized dimensionality reduction improves ultrasound thyroid nodule diagnostic accuracy and inter-observer consistency. Comput Biol Med 2023; 154:106536. [PMID: 36708654 DOI: 10.1016/j.compbiomed.2023.106536] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 12/20/2022] [Accepted: 01/10/2023] [Indexed: 01/13/2023]
Abstract
PROBLEM Convolutional Neural Networks (CNNs) for medical image analysis usually only output a probability value, providing no further information about the original image or inter-relationships between different images. Dimensionality Reduction Techniques (DRTs) are used for visualization of high dimensional medical image data, but they are not intended for discriminative classification analysis. AIM We develop an interactive phenotype distribution field visualization system for medical images to accurately reflect the pathological characteristics of lesions and their similarity to assist radiologists in diagnosis and medical research. METHODS We propose a novel method, Classification Regularized Uniform Manifold Approximation and Projection (UMAP) referred as CReUMAP, combining the advantages of CNN and DRT, to project the extracted feature vector fused with the malignant probability predicted by a CNN to a two-dimensional space, and then apply a spatial segmentation classifier trained on 2614 ultrasound images for prediction of thyroid nodule malignancy and guidance to radiologists. RESULTS The CReUMAP embedding correlates well with the TI-RADS categories of thyroid nodules. The parametric version that embeds external test dataset of 303 images in presence of the training data with known pathological diagnosis improves the benign and malignant nodule diagnostic accuracy (p-value = 0.016) and confidence (p-value = 1.902 × 10-6) of eight radiologists of different experience levels significantly as well as their inter-observer agreements (kappa≥0.75). CReUMAP achieve 90.8% accuracy, 92.1% sensitivity and 88.6% specificity in test set. CONCLUSION CReUMAP embedding is well correlated with the pathological diagnosis of thyroid nodules, and helps radiologists achieve more accurate, confident and consistent diagnosis. It allows a medical center to generate its locally adapted embedding using an already-trained classification model in an updateable manner on an ever-growing local database as long as the extracted feature vectors and predicted diagnostic probabilities of the correspondent classification model can be outputted.
Collapse
Affiliation(s)
- Wenli Dai
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Yan Cui
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Peiyi Wang
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China
| | - Hao Wu
- Department of Ultrasound, The Second Affiliated Hospital of Zhejiang Chinese Medical University, Hangzhou, China
| | - Lei Zhang
- Department of Ultrasound, The Second Affiliated Hospital of Zhejiang Chinese Medical University, Hangzhou, China
| | - Yeping Bian
- Department of Ultrasonography, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer, Chinese Academy of Sciences, Hangzhou, China
| | - Yingying Li
- Department of Special Examinations, Hangzhou Third People's Hospital, Hangzhou, China
| | - Yutao Li
- Department of Ultrasound, Hangzhou First People's Hospital Affiliated to Medical College of Zhejiang University, Hangzhou, China
| | - Hairong Hu
- Demetics Medical Technology, Hangzhou, China
| | - Jiaqi Zhao
- Department of Ultrasound, Shanghai Fourth People's Hospital, School of Medicine, Tongji University, Shanghai, China
| | - Dong Xu
- Department of Ultrasonography, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer, Chinese Academy of Sciences, Hangzhou, China
| | - Dexing Kong
- School of Mathematical Sciences, Zhejiang University, Hangzhou, China; Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China
| | - Yajuan Wang
- Department of Geriatric Medicine & Key Laboratory of Cardiovascular Proteomics of Shandong Province, Qilu Hospital of Shandong University, Jinan, China.
| | - Lei Xu
- Zhejiang Qiushi Institute for Mathematical Medicine, Hangzhou, China.
| |
Collapse
|
9
|
Sulyok M, Luibrand J, Strohäker J, Karacsonyi P, Frauenfeld L, Makky A, Mattern S, Zhao J, Nadalin S, Fend F, Schürch CM. Implementing deep learning models for the classification of Echinococcus multilocularis infection in human liver tissue. Parasit Vectors 2023; 16:29. [PMID: 36694210 PMCID: PMC9875509 DOI: 10.1186/s13071-022-05640-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2022] [Accepted: 12/26/2022] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND The histological diagnosis of alveolar echinococcosis can be challenging. Decision support models based on deep learning (DL) are increasingly used to aid pathologists, but data on the histology of tissue-invasive parasitic infections are missing. The aim of this study was to implement DL methods to classify Echinococcus multilocularis liver lesions and normal liver tissue and assess which regions and structures play the most important role in classification decisions. METHODS We extracted 15,756 echinococcus tiles from 28 patients using 59 whole slide images (WSI); 11,602 tiles of normal liver parenchyma from 18 patients using 33 WSI served as a control group. Different pretrained model architectures were used with a 60-20-20% random splitting. We visualized the predictions using probability-thresholded heat maps of WSI. The area-under-the-curve (AUC) value and other performance metrics were calculated. The GradCAM method was used to calculate and visualize important spatial features. RESULTS The models achieved a high validation and test set accuracy. The calculated AUC values were 1.0 in all models. Pericystic fibrosis and necrotic areas, as well as germinative and laminated layers of the metacestodes played an important role in decision tasks according to the superimposed GradCAM heatmaps. CONCLUSION Deep learning models achieved a high predictive performance in classifying E. multilocularis liver lesions. A possible next step could be to validate the model using other datasets and test it against other pathologic entities as well, such as, for example, Echinococcus granulosus infection.
Collapse
Affiliation(s)
- Mihaly Sulyok
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Julia Luibrand
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Jens Strohäker
- grid.411544.10000 0001 0196 8249Department of Surgery, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Peter Karacsonyi
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Leonie Frauenfeld
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Ahmad Makky
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Sven Mattern
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Jing Zhao
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Silvio Nadalin
- grid.411544.10000 0001 0196 8249Department of Surgery, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Falko Fend
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| | - Christian M. Schürch
- grid.411544.10000 0001 0196 8249Department of Pathology and Neuropathology, University Hospital and Comprehensive Cancer Center Tübingen, Tübingen, Germany
| |
Collapse
|
10
|
Sengar N, Burget R, Dutta MK. A vision transformer based approach for analysis of plasmodium vivax life cycle for malaria prediction using thin blood smear microscopic images. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 224:106996. [PMID: 35843076 DOI: 10.1016/j.cmpb.2022.106996] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Revised: 06/23/2022] [Accepted: 06/29/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND AND OBJECTIVES Microscopic images are an important part for haematologists in diagnosing various diseases in the blood cell. Changes in blood cells are caused by malaria disease, and early diagnosis can prevent the disease from entering its severe stage. METHODS In this paper, an automated non-invasive and efficient deep learning-based framework is developed for multi-class plasmodium vivax life cycle classification and malaria diagnosis. A multi-class microscopic blood cell of different plasmodium vivax life cycle stage dataset is analysed, and a diagnostic framework is designed. Several stages of the disease are examined and augmented through various techniques to make the framework robust in real-time. Generative adversarial network is specially designed to generate extended training samples of various life cycle stages to increase robustness of the resulting model. A special transformer-based neural network vision transformer is designed to improve generalisation capabilities. Microscopic images are classified into multi classes of plasmodium vivax life cycle stage, where the keystone transformer layers extract relevant disease features from microscopic colour images, and the extracted relevant features are used to make predictive diagnostic decisions. RESULTS The capabilities of the vision transformer are computed and analysed by statistical parameters, and the performance of the vision transformer model is compared with baseline architectures, where it was evident that the performance of the vision transformer was significantly better, reaching 90.03% accuracy. CONCLUSIONS A comprehensive comparison of the proposed framework to the state-of-the-art methods proves its efficiency in the classification of plasmodium vivax life cycle for malaria disease identification through thin blood smear microscopic images.
Collapse
Affiliation(s)
| | - Radim Burget
- Brno University of Technology, FEEC, Dept. of Telecommunications, 616 00 Brno, Czech Republic
| | | |
Collapse
|
11
|
Dutta AK, Mageswari RU, Gayathri A, Dallfin Bruxella JM, Ishak MK, Mostafa SM, Hamam H. Barnacles Mating Optimizer with Deep Transfer Learning Enabled Biomedical Malaria Parasite Detection and Classification. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:7776319. [PMID: 35694571 PMCID: PMC9177294 DOI: 10.1155/2022/7776319] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 04/29/2022] [Accepted: 05/07/2022] [Indexed: 01/06/2023]
Abstract
Biomedical engineering involves ideologies and problem-solving methods of engineering to biology and medicine. Malaria is a life-threatening illness, which has gained significant attention among researchers. Since the manual diagnosis of malaria in a clinical setting is tedious, automated tools based on computational intelligence (CI) tools have gained considerable interest. Though earlier studies were focused on the handcrafted features, the diagnostic accuracy can be boosted through deep learning (DL) methods. This study introduces a new Barnacles Mating Optimizer with Deep Transfer Learning Enabled Biomedical Malaria Parasite Detection and Classification (BMODTL-BMPC) model. The presented BMODTL-BMPC model involves the design of intelligent models for the recognition and classification of malaria parasites. Initially, the Gaussian filtering (GF) approach is employed to eradicate noise in blood smear images. Then, Graph cuts (GC) segmentation technique is applied to determine the affected regions in the blood smear images. Moreover, the barnacles mating optimizer (BMO) algorithm with the NasNetLarge model is employed for the feature extraction process. Furthermore, the extreme learning machine (ELM) classification model is employed for the identification and classification of malaria parasites. To assure the enhanced outcomes of the BMODTL-BMPC technique, a wide-ranging experimentation analysis is performed using a benchmark dataset. The experimental results show that the BMODTL-BMPC technique outperforms other recent approaches.
Collapse
Affiliation(s)
- Ashit Kumar Dutta
- Department of Computer Science and Information System, College of Applied Sciences, AlMaarefa University, Riyadh 11597, Saudi Arabia
| | - R. Uma Mageswari
- Department of Computer Science and Engineering, Vardhaman College of Engineering (Autonomous), Hyderabad, Telangana, India
| | - A. Gayathri
- Department of Information Technology, School of Information Technology and Engineering, Vellore Institute of Technology, Vellore, India
| | - J. Mary Dallfin Bruxella
- Department of Computer Science and Information Technology, Kalasalingam Academy of Research and Education, Krishnankoil, India
| | - Mohamad Khairi Ishak
- School of Electrical and Electronic Engineering, Universiti Sains Malaysia, Nibong Tebal 14300, Pulau Pinang, Malaysia
| | - Samih M. Mostafa
- Faculty of Computers and Information, South Valley University, Egypt
| | - Habib Hamam
- Faculty of Engineering, Université de Moncton, Moncton, NB E1A 3E9, Canada
- School of Electrical Engineering, Department of Electrical and Electronic Engineering Science, University of Johannesburg, Johannesburg 2006, South Africa
| |
Collapse
|
12
|
An Empirical Evaluation of Convolutional Networks for Malaria Diagnosis. J Imaging 2022; 8:jimaging8030066. [PMID: 35324621 PMCID: PMC8951236 DOI: 10.3390/jimaging8030066] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Revised: 03/02/2022] [Accepted: 03/04/2022] [Indexed: 01/27/2023] Open
Abstract
Malaria is a globally widespread disease caused by parasitic protozoa transmitted to humans by infected female mosquitoes of Anopheles. It is caused in humans only by the parasite Plasmodium, further classified into four different species. Identifying malaria parasites is possible by analysing digital microscopic blood smears, which is tedious, time-consuming and error prone. So, automation of the process has assumed great importance as it helps the laborious manual process of review and diagnosis. This work focuses on deep learning-based models, by comparing off-the-shelf architectures for classifying healthy and parasite-affected cells, by investigating the four-class classification on the Plasmodium falciparum stages of life and, finally, by evaluating the robustness of the models with cross-dataset experiments on two different datasets. The main contributions to the research in this field can be resumed as follows: (i) comparing off-the-shelf architectures in the task of classifying healthy and parasite-affected cells, (ii) investigating the four-class classification on the P. falciparum stages of life and (iii) evaluating the robustness of the models with cross-dataset experiments. Eleven well-known convolutional neural networks on two public datasets have been exploited. The results show that the networks have great accuracy in binary classification, even though they lack few samples per class. Moreover, the cross-dataset experiments exhibit the need for some further regulations. In particular, ResNet-18 achieved up to 97.68% accuracy in the binary classification, while DenseNet-201 reached 99.40% accuracy on the multiclass classification. The cross-dataset experiments exhibit the limitations of deep learning approaches in such a scenario, even though combining the two datasets permitted DenseNet-201 to reach 97.45% accuracy. Naturally, this needs further investigation to improve the robustness. In general, DenseNet-201 seems to offer the most stable and robust performance, offering as a crucial candidate to further developments and modifications. Moreover, the mobile-oriented architectures showed promising and satisfactory performance in the classification of malaria parasites. The obtained results enable extensive improvements, specifically oriented to the application of object detectors for type and stage of life recognition, even in mobile environments.
Collapse
|
13
|
Ufuktepe DK, Yang F, Kassim YM, Yu H, Maude RJ, Palaniappan K, Jaeger S. Deep Learning-Based Cell Detection and Extraction in Thin Blood Smears for Malaria Diagnosis. IEEE APPLIED IMAGERY PATTERN RECOGNITION WORKSHOP : [PROCEEDINGS]. IEEE APPLIED IMAGERY PATTERN RECOGNITION WORKSHOP 2021; 2021:9762109. [PMID: 36483328 PMCID: PMC7613898 DOI: 10.1109/aipr52630.2021.9762109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
Malaria is a major health threat caused by Plasmodium parasites that infect the red blood cells. Two predominant types of Plasmodium parasites are Plasmodium vivax (P. vivax) and Plasmodium falciparum (P. falciparum). Diagnosis of malaria typically involves visual microscopy examination of blood smears for malaria parasites. This is a tedious, error-prone visual inspection task requiring microscopy expertise which is often lacking in resource-poor settings. To address these problems, attempts have been made in recent years to automate malaria diagnosis using machine learning approaches. Several challenges need to be met for a machine learning approach to be successful in malaria diagnosis. Microscopy images acquired at different sites often vary in color, contrast, and consistency caused by different smear preparation and staining methods. Moreover, touching and overlapping cells complicate the red blood cell detection process, which can lead to inaccurate blood cell counts and thus incorrect parasitemia calculations. In this work, we propose a red blood cell detection and extraction framework to enable processing and analysis of single cells for follow-up processes like counting infected cells or identifying parasite species in thin blood smears. This framework consists of two modules: a cell detection module and a cell extraction module. The cell detection module trains a modified Channel-wise Feature Pyramid Network for Medicine (CFPNet-M) deep learning network that takes the green channel of the image and the color-deconvolution processed image as inputs, and learns a truncated distance transform image of cell annotations. CFPNet-M is chosen due to its low resource requirements, while the distance transform allows achieving more accurate cell counts for dense cells. Once the cells are detected by the network, the cell extraction module is used to extract single cells from the original image and count the number of cells. Our preliminary results based on 193 patients (including 148 P. Falciparum infected patients, and 45 uninfected patients) show that our framework achieves cell count accuracy of 92.2%.
Collapse
Affiliation(s)
| | - Feng Yang
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Yasmin M. Kassim
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Hang Yu
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| | - Richard J. Maude
- Mahidol Oxford Tropical Medicine Research Unit, Mahidol University, Bangkok, Thailand
- Centre for Tropical Medicine and Global Health, Nuffield Department of Medicine, University of Oxford, Oxford, UK
- Harvard TH Chan School of Public Health, Harvard University, Boston, MA, USA
| | | | - Stefan Jaeger
- National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
| |
Collapse
|