1
|
Mu Y, Tizhoosh HR, Dehkharghanian T, Campbell CJV. Whole slide image representation in bone marrow cytology. Comput Biol Med 2023; 166:107530. [PMID: 37837726 DOI: 10.1016/j.compbiomed.2023.107530] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 09/17/2023] [Accepted: 09/27/2023] [Indexed: 10/16/2023]
Abstract
One of the goals of AI-based computational pathology is to generate compact representations of whole slide images (WSIs) that capture the essential information needed for diagnosis. While such approaches have been applied to histopathology, few applications have been reported in cytology. Bone marrow aspirate cytology is the basis for key clinical decisions in hematology. However, visual inspection of aspirate specimens is a tedious and complex process subject to variation in interpretation, and hematopathology expertise is scarce. The ability to generate a compact representation of an aspirate specimen may form the basis for clinical decision-support tools in hematology. In this study, we leverage our previously published end-to-end AI-based system for counting and classifying cells from bone marrow aspirate WSIs, which enables the direct use of individual cells as inputs rather than WSI patches. We then construct bags of individual cell features from each WSI, and apply multiple instance learning to extract their vector representations. To evaluate the quality of our representations, we conducted WSI retrieval and classification tasks. Our results show that we achieved a mAP@10 of 0.58 ±0.02 in WSI-level image retrieval, surpassing the random-retrieval baseline of 0.39 ±0.1. Furthermore, we predicted five diagnostic labels for individual aspirate WSIs with a weighted-average F1 score of 0.57 ±0.03 using a k-nearest-neighbors (k-NN) model, outperforming guessing using empirical class prior probabilities (0.26 ±0.02). We present the first example of exploring trainable mechanisms to generate compact, slide-level representations in bone marrow cytology with deep learning. This method has the potential to summarize complex semantic information in WSIs toward improved diagnostics in hematology, and may eventually support AI-assisted computational pathology approaches.
Collapse
Affiliation(s)
- Youqing Mu
- University of Toronto, Toronto, Canada; McMaster University, Hamilton, Canada
| | - H R Tizhoosh
- Rhazes Lab, Artificial Intelligence & Informatics, Mayo Clinic, Rochester, MN, USA
| | - Taher Dehkharghanian
- McMaster University, Hamilton, Canada; University Health Network, Toronto, Canada
| | - Clinton J V Campbell
- McMaster University, Hamilton, Canada; William Osler Health System, Brampton, Canada.
| |
Collapse
|
2
|
Cheng Z, Li Y. Improved YOLOv7 Algorithm for Detecting Bone Marrow Cells. SENSORS (BASEL, SWITZERLAND) 2023; 23:7640. [PMID: 37688095 PMCID: PMC10490824 DOI: 10.3390/s23177640] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 08/29/2023] [Accepted: 08/31/2023] [Indexed: 09/10/2023]
Abstract
The detection and classification of bone marrow (BM) cells is a critical cornerstone for hematology diagnosis. However, the low accuracy caused by few BM-cell data samples, subtle difference between classes, and small target size, pathologists still need to perform thousands of manual identifications daily. To address the above issues, we propose an improved BM-cell-detection algorithm in this paper, called YOLOv7-CTA. Firstly, to enhance the model's sensitivity to fine-grained features, we design a new module called CoTLAN in the backbone network to enable the model to perform long-term modeling between target feature information. Then, in order to cooperate with the CoTLAN module to pay more attention to the features in the area to be detected, we integrate the coordinate attention (CoordAtt) module between the CoTLAN modules to improve the model's attention to small target features. Finally, we cluster the target boxes of the BM cell dataset based on K-means++ to generate more suitable anchor boxes, which accelerates the convergence of the improved model. In addition, in order to solve the imbalance between positive and negative samples in BM-cell pictures, we use the Focal loss function to replace the multi-class cross entropy. Experimental results demonstrate that the best mean average precision (mAP) of the proposed model reaches 88.6%, which is an improvement of 12.9%, 8.3%, and 6.7% compared with that of the Faster R-CNN model, YOLOv5l model, and YOLOv7 model, respectively. This verifies the effectiveness and superiority of the YOLOv7-CTA model in BM-cell-detection tasks.
Collapse
Affiliation(s)
| | - Yuanyuan Li
- School of Mathematics and Physics, Wuhan Institute of Technology, Wuhan 430205, China
| |
Collapse
|
3
|
Dehkharghanian T, Mu Y, Tizhoosh HR, Campbell CJV. Applied machine learning in hematopathology. Int J Lab Hematol 2023. [PMID: 37257440 DOI: 10.1111/ijlh.14110] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 05/12/2023] [Indexed: 06/02/2023]
Abstract
An increasing number of machine learning applications are being developed and applied to digital pathology, including hematopathology. The goal of these modern computerized tools is often to support diagnostic workflows by extracting and summarizing information from multiple data sources, including digital images of human tissue. Hematopathology is inherently multimodal and can serve as an ideal case study for machine learning applications. However, hematopathology also poses unique challenges compared to other pathology subspecialities when applying machine learning approaches. By modeling the pathologist workflow and thinking process, machine learning algorithms may be designed to address practical and tangible problems in hematopathology. In this article, we discuss the current trends in machine learning in hematopathology. We review currently available machine learning enabled medical devices supporting hematopathology workflows. We then explore current machine learning research trends of the field with a focus on bone marrow cytology and histopathology, and how adoption of new machine learning tools may be enabled through the transition to digital pathology.
Collapse
Affiliation(s)
- Taher Dehkharghanian
- Department of Nephrology, University Health Network, Toronto, Ontario, Canada
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Youqing Mu
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Hamid R Tizhoosh
- Rhazes Lab, Artificial Intelligence and Informatics, Mayo Clinic, Rochester, Minnesota, USA
| | - Clinton J V Campbell
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada
- William Osler Health System, Brampton, Ontario, Canada
| |
Collapse
|
4
|
Yang G, Qin Z, Mu J, Mao H, Mao H, Han M. Efficient diagnosis of hematologic malignancies using bone marrow microscopic images: A method based on MultiPathGAN and MobileViTv2. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 237:107583. [PMID: 37167882 DOI: 10.1016/j.cmpb.2023.107583] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 04/30/2023] [Accepted: 05/03/2023] [Indexed: 05/13/2023]
Abstract
BACKGROUND AND OBJECTIVES Hematologic malignancies, including the associated multiple subtypes, are critically threatening to human health. The timely detection of malignancies is crucial for their effective treatment. In this regard, the examination of bone marrow smears constitutes a crucial step. Nonetheless, the conventional approach to cell identification and enumeration is laborious and time-intensive. Therefore, the present study aimed to develop a method for the efficient diagnosis of these malignancies directly from bone marrow microscopic images. METHODS A deep learning-based framework was developed to facilitate the diagnosis of common hematologic malignancies. First, a total of 2033 microscopic images of bone marrow analysis, including the images for 6 disease types and 1 healthy control, were collected from two Chinese medical websites. Next, the collected images were classified into the training, validation, and test datasets in the ratio of 7:1:2. Subsequently, a method of stain normalization to multi-domains (stain domain augmentation) based on the MultiPathGAN model was developed to equalize the stain styles and expand the image datasets. Afterward, a lightweight hybrid model named MobileViTv2, which integrates the strengths of both CNNs and ViTs, was developed for disease classification. The resulting model was trained and utilized to diagnose patients based on multiple microscopic images of their bone marrow smears, obtained from a cohort of 61 individuals. RESULTS MobileViTv2 exhibited an average accuracy of 94.28% when applied to the test set, with multiple myeloma, acute lymphocytic leukemia, and lymphoma revealed as the three diseases diagnosed with the highest accuracy values of 98%, 96%, and 96%, respectively. Regarding patient-level prediction, the average accuracy of MobileViTv2 was 96.72%. This model outperformed both CNN and ViT models in terms of accuracy, despite utilizing only 9.8 million parameters. When applied to two public datasets, MobileViTv2 exhibited accuracy values of 99.75% and 99.72%, respectively, and outperformed previous methods. CONCLUSIONS The proposed framework could be applied directly to bone marrow microscopic images with different stain styles to efficiently establish the diagnosis of common hematologic malignancies.
Collapse
Affiliation(s)
- Guanghui Yang
- School of Information Science and Engineering, Shandong University, Qingdao 266237, China
| | - Ziqi Qin
- School of Information Science and Engineering, Shandong University, Qingdao 266237, China
| | - Jianmin Mu
- Mudan District Hospital of Traditional Chinese Medicine, Heze 274031, China
| | - Haiting Mao
- Department of Clinical Laboratory, The Second Hospital, Cheeloo College of Medicine, Shandong University, Jinan 250033, China
| | - Huihui Mao
- Department of Clinical Laboratory, The Second Hospital, Cheeloo College of Medicine, Shandong University, Jinan 250033, China
| | - Min Han
- School of Information Science and Engineering, Shandong University, Qingdao 266237, China.
| |
Collapse
|
5
|
Cell Count Differentials by Cytomorphology and Next-Generation Flow Cytometry in Bone Marrow Aspirate: An Evidence-Based Approach. Diagnostics (Basel) 2023; 13:diagnostics13061071. [PMID: 36980379 PMCID: PMC10047335 DOI: 10.3390/diagnostics13061071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 03/06/2023] [Accepted: 03/09/2023] [Indexed: 03/18/2023] Open
Abstract
Despite a lack of evidence, a bone marrow aspirate differential of 500 cells is commonly used in the clinical setting. We aimed to test the performance of 200-cell counts for daily hematological workup. In total, 660 consecutive samples were analyzed recording differentials at 200 and 500 cells. Additionally, immunophenotype results and preanalytical issues were also evaluated. Clinical and statistical differences between both cutoffs and both methods were checked. An independent control group of 122 patients was included. All comparisons between both cutoffs and both methods for all relevant types of cells did not show statistically significant differences. No significant diagnostic discrepancies were demonstrated in the contingency table analysis. This is a real-life study, and some limitations may be pointed out, such as a different sample sizes according to the type of cell in the immunophenotype analysis, the lack of standardization of some preanalytical events, and the relatively small sample size of the control group. The comparisons of differentials by morphology on 200 and 500 cells, as well as by morphology (both cutoffs) and by immunophenotype, are equivalent from the clinical and statistical point of view. The preanalytical issues play a critical role in the assessment of bone marrow aspirate samples.
Collapse
|
6
|
Lewis JE, Shebelut CW, Drumheller BR, Zhang X, Shanmugam N, Attieh M, Horwath MC, Khanna A, Smith GH, Gutman DA, Aljudi A, Cooper LAD, Jaye DL. An Automated Pipeline for Differential Cell Counts on Whole-Slide Bone Marrow Aspirate Smears. Mod Pathol 2023; 36:100003. [PMID: 36853796 PMCID: PMC10310355 DOI: 10.1016/j.modpat.2022.100003] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Revised: 08/10/2022] [Accepted: 09/18/2022] [Indexed: 01/11/2023]
Abstract
The pathologic diagnosis of bone marrow disorders relies in part on the microscopic analysis of bone marrow aspirate (BMA) smears and the manual counting of marrow nucleated cells to obtain a differential cell count (DCC). This manual process has significant limitations, including the analysis of only a small subset of optimal slide areas and nucleated cells, as well as interobserver variability due to differences in cell selection and classification. To address these shortcomings, we developed an automated machine learning-based pipeline for obtaining 11-component DCCs on whole-slide BMAs. This pipeline uses a sequential process of identifying optimal BMA regions with high proportions of marrow nucleated cells, detecting individual cells within these optimal areas, and classifying these cells into 1 of 11 DCC components. Convolutional neural network models were trained on 396,048 BMA region, 28,914 cell boundary, and 1,510,976 cell class images from manual annotations. The resulting automated pipeline produced 11-component DCCs that demonstrated a high statistical and diagnostic concordance with manual DCCs among a heterogeneous group of testing BMA slides with varying pathologies and cellularities. Additionally, we demonstrated that an automated analysis can reduce the intraslide variance in DCCs by analyzing the whole slide and marrow nucleated cells within all optimal regions. Finally, the pipeline outputs of region classification, cell detection, and cell classification can be visualized using whole-slide image analysis software. This study demonstrates the feasibility of a fully automated pipeline for generating DCCs on scanned whole-slide BMA images, with the potential for improving the current standard of practice for utilizing BMA smears in the laboratory analysis of hematologic disorders.
Collapse
Affiliation(s)
- Joshua E Lewis
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Conrad W Shebelut
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Bradley R Drumheller
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Xuebao Zhang
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Nithya Shanmugam
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Michel Attieh
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Michael C Horwath
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Anurag Khanna
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Geoffrey H Smith
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - David A Gutman
- Department of Biomedical Informatics, Emory University, Atlanta, Georgia
| | - Ahmed Aljudi
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia
| | - Lee A D Cooper
- Department of Pathology, Northwestern University, Chicago, Illinois.
| | - David L Jaye
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, Georgia; Winship Cancer Institute, Emory University, Atlanta, Georgia.
| |
Collapse
|
7
|
Wang C, Wei XL, Li CX, Wang YZ, Wu Y, Niu YX, Zhang C, Yu Y. Efficient and Highly Accurate Diagnosis of Malignant Hematological Diseases Based on Whole-Slide Images Using Deep Learning. Front Oncol 2022; 12:879308. [PMID: 35756613 PMCID: PMC9226668 DOI: 10.3389/fonc.2022.879308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/19/2022] [Accepted: 04/27/2022] [Indexed: 11/13/2022] Open
Abstract
Hematopoietic disorders are serious diseases that threaten human health, and the diagnosis of these diseases is essential for treatment. However, traditional diagnosis methods rely on manual operation, which is time consuming and laborious, and examining entire slide is challenging. In this study, we developed a weakly supervised deep learning method for diagnosing malignant hematological diseases requiring only slide-level labels. The method improves efficiency by converting whole-slide image (WSI) patches into low-dimensional feature representations. Then the patch-level features of each WSI are aggregated into slide-level representations by an attention-based network. The model provides final diagnostic predictions based on these slide-level representations. By applying the proposed model to our collection of bone marrow WSIs at different magnifications, we found that an area under the receiver operating characteristic curve of 0.966 on an independent test set can be obtained at 10× magnification. Moreover, the performance on microscopy images can achieve an average accuracy of 94.2% on two publicly available datasets. In conclusion, we have developed a novel method that can achieve fast and accurate diagnosis in different scenarios of hematological disorders.
Collapse
Affiliation(s)
- Chong Wang
- School of Biological Science and Medical Engineering, Beihang University, Beijing, China.,School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Department of Neurobiology, School of Basic Medical Sciences, Beijing Key Laboratory of Neural Regeneration and Repair, Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China
| | - Xiu-Li Wei
- School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Department of Hematology, Xinxiang First People's Hospital, Xinxiang, China
| | - Chen-Xi Li
- School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Department of Hematology, Xinxiang First People's Hospital, Xinxiang, China
| | - Yang-Zhen Wang
- Department of Neurobiology, School of Basic Medical Sciences, Beijing Key Laboratory of Neural Regeneration and Repair, Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China.,School of Life Sciences, Tsinghua University, Beijing, China
| | - Yang Wu
- School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Henan Province Neural Sensing and Control Engineering Technology Research Center, Xinxiang, China
| | - Yan-Xiang Niu
- School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Henan Province Neural Sensing and Control Engineering Technology Research Center, Xinxiang, China
| | - Chen Zhang
- Department of Neurobiology, School of Basic Medical Sciences, Beijing Key Laboratory of Neural Regeneration and Repair, Advanced Innovation Center for Human Brain Protection, Capital Medical University, Beijing, China.,Chinese Institute for Brain Research, Beijing, China
| | - Yi Yu
- School of Medical Engineering, Xinxiang Medical University, Xinxiang, China.,Henan Province Neural Sensing and Control Engineering Technology Research Center, Xinxiang, China
| |
Collapse
|
8
|
Heuser M, Ofran Y, Boissel N, Brunet Mauri S, Craddock C, Janssen J, Wierzbowska A, Buske C. Acute myeloid leukaemia in adult patients: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2020; 31:697-712. [PMID: 32171751 DOI: 10.1016/j.annonc.2020.02.018] [Citation(s) in RCA: 137] [Impact Index Per Article: 34.3] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2020] [Accepted: 02/27/2020] [Indexed: 01/01/2023] Open
Affiliation(s)
- M Heuser
- Hematology, Hemostasis, Oncology and Stem Cell Transplantation, Hannover Medical School, Hannover, Germany
| | - Y Ofran
- Department of Hematology and Bone Marrow Transplantation, Rambam Health Care Campus, Haifa, Israel; The Ruth and Bruce Rappaport Faculty of Medicine, Technion, Israel Institute of Technology, Haifa, Israel
| | - N Boissel
- Department of Hematology, AP-HP, Saint-Louis Hospital, Paris, France; Université de Paris, Paris, France
| | - S Brunet Mauri
- Hospital de la Santa Creu i Sant Pau, IIB-Sant Pau, Barcelona, Spain; Jose Carreras Leukemia Research Institute, Universitat Autònoma de Barcelona, Barcelona, Spain
| | - C Craddock
- Centre for Clinical Haematology, Queen Elizabeth Hospital, Birmingham, UK
| | - J Janssen
- Department of Hematology, Amsterdam University Medical Centers, location VUmc, Amsterdam, The Netherlands
| | - A Wierzbowska
- Department of Hematology, Medical University of Lodz, Lodz, Poland; Copernicus Memorial Hospital, Lodz, Poland
| | - C Buske
- Comprehensive Cancer Center, Institute of Experimental Cancer Research, University Hospital Ulm, Ulm, Germany
| |
Collapse
|
9
|
Chandradevan R, Aljudi AA, Drumheller BR, Kunananthaseelan N, Amgad M, Gutman DA, Cooper LAD, Jaye DL. Machine-based detection and classification for bone marrow aspirate differential counts: initial development focusing on nonneoplastic cells. J Transl Med 2020; 100:98-109. [PMID: 31570774 PMCID: PMC6920560 DOI: 10.1038/s41374-019-0325-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2019] [Revised: 07/30/2019] [Accepted: 09/02/2019] [Indexed: 12/16/2022] Open
Abstract
Bone marrow aspirate (BMA) differential cell counts (DCCs) are critical for the classification of hematologic disorders. While manual counts are considered the gold standard, they are labor intensive, time consuming, and subject to bias. A reliable automated counter has yet to be developed, largely due to the inherent complexity of bone marrow specimens. Digital pathology imaging coupled with machine learning algorithms represents a highly promising emerging technology for this purpose. Yet, training datasets for BMA cellular constituents, critical for building and validating machine learning algorithms, are lacking. Herein, we report our experience creating and employing such datasets to develop a machine learning algorithm to detect and classify BMA cells. Utilizing a web-based system that we developed for annotating and managing digital pathology images, over 10,000 cells from scanned whole slide images of BMA smears were manually annotated, including all classes that comprise the standard clinical DCC. We implemented a two-stage, detection and classification approach that allows design flexibility and improved classification accuracy. In a sixfold cross-validation, our algorithms achieved high overall accuracy in detection (0.959 ± 0.008 precision-recall AUC) and classification (0.982 ± 0.03 ROC AUC) using nonneoplastic samples. Testing on a small set of acute myeloid leukemia and multiple myeloma samples demonstrated similar detection and classification performance. In summary, our algorithms showed promising early results and represent an important initial step in the effort to devise a reliable, objective method to automate DCCs. With further development to include formal clinical validation, such a system has the potential to assist in disease diagnosis and prognosis, and significantly impact clinical practice.
Collapse
Affiliation(s)
| | - Ahmed A Aljudi
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
- Department of Pathology, Children's Healthcare of Atlanta, Atlanta, GA, USA
| | - Bradley R Drumheller
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA
| | | | - Mohamed Amgad
- Department of Biomedical Informatics, Emory University, Atlanta, GA, USA
| | - David A Gutman
- Department of Neurology, Emory University, Atlanta, GA, USA
| | - Lee A D Cooper
- Department of Biomedical Informatics, Emory University, Atlanta, GA, USA.
- Department of Pathology, Northwestern University, Chicago, IL and Robert H. Lurie Comprehensive Cancer Center of Northwestern University, Chicago, IL, USA.
| | - David L Jaye
- Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA, USA.
- Winship Cancer Institute, Emory University, Atlanta, GA, USA.
| |
Collapse
|