Morphogo: An Automatic Bone Marrow Cell Classification System on Digital Images Analyzed by Artificial Intelligence

Advances in Bone Marrow Evaluation

Evaluation of bone marrow aspirate smear and trephine biopsy specimens is critical to the diagnosis of benign and malignant hematologic conditions. Digital pathology has the potential to revolutionize bone marrow assessment through implementation of artificial intelligence for assisted and automated evaluation, but there remain many barriers toward this implementation. This article reviews the current state of digital evaluation of bone marrow aspirate smears and trephine biopsies, recent research using machine learning models for automated specimen analysis, an outline of the advantages and barriers facing clinical implementation of artificial intelligence, and a potential vision of artificial intelligence-associated bone marrow evaluation.

Digital Imaging and AI Pre-classification in Hematology

A leukocyte differential of peripheral blood can be performed using digital imaging coupled with cellular pre-classification by artificial neural networks. Platelet and erythrocyte morphology can be assessed and counts estimated. Systems from a single vendor have been used in clinical practice for several years, with other vendors' systems, in a development. These systems perform comparably to traditional manual optical microscopy, however, it is important to note that they are designed and intended to be operated by a trained morphologist. These systems have several benefits including increased standardization, efficiency, and remote-review capability.

Recent advancements in machine learning for bone marrow cell morphology analysis

As machine learning progresses, techniques such as neural networks, decision trees, and support vector machines are being increasingly applied in the medical domain, especially for tasks involving large datasets, such as cell detection, recognition, classification, and visualization. Within the domain of bone marrow cell morphology analysis, deep learning offers substantial benefits due to its robustness, ability for automatic feature learning, and strong image characterization capabilities. Deep neural networks are a machine learning paradigm specifically tailored for image processing applications. Artificial intelligence serves as a potent tool in supporting the diagnostic process of clinical bone marrow cell morphology. Despite the potential of artificial intelligence to augment clinical diagnostics in this domain, manual analysis of bone marrow cell morphology remains the gold standard and an indispensable tool for identifying, diagnosing, and assessing the efficacy of hematologic disorders. However, the traditional manual approach is not without limitations and shortcomings, necessitating, the exploration of automated solutions for examining and analyzing bone marrow cytomorphology. This review provides a multidimensional account of six bone marrow cell morphology processes: automated bone marrow cell morphology detection, automated bone marrow cell morphology segmentation, automated bone marrow cell morphology identification, automated bone marrow cell morphology classification, automated bone marrow cell morphology enumeration, and automated bone marrow cell morphology diagnosis. Highlighting the attractiveness and potential of machine learning systems based on bone marrow cell morphology, the review synthesizes current research and recent advances in the application of machine learning in this field. The objective of this review is to offer recommendations to hematologists for selecting the most suitable machine learning algorithms to automate bone marrow cell morphology examinations, enabling swift and precise analysis of bone marrow cytopathic trends for early disease identification and diagnosis. Furthermore, the review endeavors to delineate potential future research avenues for machine learning-based applications in bone marrow cell morphology analysis.

Digital morphology compared to the optical microscope: A validation study on reporting bone marrow aspirates

INTRODUCTION

This study aims to evaluate the effectiveness and reliability of the utilization for clinical reporting of the evaluation of digital images of bone marrow aspirates by morphologists and their comparability with the classic microscopic morphological evaluation.

METHODS

We scanned 180 consecutive bone marrow needle aspirates smears using the "Metafer4 VSlide" whole slide imaging (WSI) digital scanning system. We evaluated the statistical comparability and the risk of bias of the microscopic readings with those performed on the screen on the digitized medullary images.

RESULTS

The evaluation of cellularity on the screen was equivalent, with a higher frequency of "normal" than the analysis of digital preparations. The means and medians of the percentage values obtained on the different cell populations with the microscopic and digital reading were comparable as the main categories are concerned, with an average difference equal to 0 for the neutrophilic and eosinophilic granulocytic series, at -0.2% for the total myeloid cells, at 1.2% for the erythroid series, at -0.4% for the lymphocytes and at -0.4% for the blasts. Dysplastic features were consistently identified in 69/71 cell lineages.

CONCLUSION

Our study demonstrated that screen evaluation of digitized bone marrow needle aspirates provides quantitative and qualitative results comparable to traditional microscopic analysis of the corresponding slide smears. Digital images offer significant benefits in reducing the workload of experienced operators, reproducibility and sharing of observations, and image preservation. Even in routine diagnostic activities, their use does not alter the quality of the results obtained in evaluating bone marrow needle aspirates.

High-accuracy morphological identification of bone marrow cells using deep learning-based Morphogo system

Accurate identification and classification of bone marrow (BM) nucleated cell morphology are crucial for the diagnosis of hematological diseases. However, the subjective and time-consuming nature of manual identification by pathologists hinders prompt diagnosis and patient treatment. To address this issue, we developed Morphogo, a convolutional neural network-based system for morphological examination. Morphogo was trained using a vast dataset of over 2.8 million BM nucleated cell images. Its performance was evaluated using 508 BM cases that were categorized into five groups based on the degree of morphological abnormalities, comprising a total of 385,207 BM nucleated cells. The results demonstrated Morphogo's ability to identify over 25 different types of BM nucleated cells, achieving a sensitivity of 80.95%, specificity of 99.48%, positive predictive value of 76.49%, negative predictive value of 99.44%, and an overall accuracy of 99.01%. In most groups, Morphogo cell analysis and Pathologists' proofreading showed high intragroup correlation coefficients for granulocytes, erythrocytes, lymphocytes, monocytes, and plasma cells. These findings further validate the practical applicability of the Morphogo system in clinical practice and emphasize its value in assisting pathologists in diagnosing blood disorders.

Artificial Intelligence-Assisted Diagnostic Cytology and Genomic Testing for Hematologic Disorders

Artificial intelligence (AI) is a rapidly evolving field of computer science that involves the development of computational programs that can mimic human intelligence. In particular, machine learning and deep learning models have enabled the identification and grouping of patterns within data, leading to the development of AI systems that have been applied in various areas of hematology, including digital pathology, alpha thalassemia patient screening, cytogenetics, immunophenotyping, and sequencing. These AI-assisted methods have shown promise in improving diagnostic accuracy and efficiency, identifying novel biomarkers, and predicting treatment outcomes. However, limitations such as limited databases, lack of validation and standardization, systematic errors, and bias prevent AI from completely replacing manual diagnosis in hematology. In addition, the processing of large amounts of patient data and personal information by AI poses potential data privacy issues, necessitating the development of regulations to evaluate AI systems and address ethical concerns in clinical AI systems. Nonetheless, with continued research and development, AI has the potential to revolutionize the field of hematology and improve patient outcomes. To fully realize this potential, however, the challenges facing AI in hematology must be addressed and overcome.

Applied machine learning in hematopathology

An increasing number of machine learning applications are being developed and applied to digital pathology, including hematopathology. The goal of these modern computerized tools is often to support diagnostic workflows by extracting and summarizing information from multiple data sources, including digital images of human tissue. Hematopathology is inherently multimodal and can serve as an ideal case study for machine learning applications. However, hematopathology also poses unique challenges compared to other pathology subspecialities when applying machine learning approaches. By modeling the pathologist workflow and thinking process, machine learning algorithms may be designed to address practical and tangible problems in hematopathology. In this article, we discuss the current trends in machine learning in hematopathology. We review currently available machine learning enabled medical devices supporting hematopathology workflows. We then explore current machine learning research trends of the field with a focus on bone marrow cytology and histopathology, and how adoption of new machine learning tools may be enabled through the transition to digital pathology.

Digital assessment of peripheral blood and bone marrow aspirate smears

The diagnosis of benign and neoplastic hematologic disorders relies on analysis of peripheral blood and bone marrow aspirate smears. As demonstrated by the widespread laboratory adoption of hematology analyzers for automated assessment of peripheral blood, digital analysis of these samples provides many significant benefits compared to relying solely on manual review. Nonetheless, analogous instruments for digital bone marrow aspirate smear assessment have yet to be clinically implemented. In this review, we first provide a historical overview detailing the implementation of hematology analyzers for digital peripheral blood assessment in the clinical laboratory, including the improvements in accuracy, scope, and throughput of current instruments over prior generations. We also describe recent research in digital peripheral blood assessment, particularly in the development of advanced machine learning models that may soon be incorporated into commercial instruments. Next, we provide an overview of recent research in digital assessment of bone marrow aspirate smears and how these approaches could soon lead to development and clinical adoption of instrumentation for automated bone marrow aspirate smear analysis. Finally, we describe the relative advantages and provide our vision for the future of digital assessment of peripheral blood and bone marrow aspirate smears, including what improvements we can soon expect in the hematology laboratory.

Artificial intelligence-assisted diagnosis of hematologic diseases based on bone marrow smears using deep neural networks

BACKGROUND AND OBJECTIVES

The morphological examination of bone marrow (BM) cells is essential in both diagnosing and treating various hematologic diseases. However, it is still done manually with a heavy workload. An artificial intelligence-assisted diagnosis support system of BM cells is highly required to reduce the workloads of examiners and improve the reproducibility of the results.

METHODS

In this paper, we proposed an artificial intelligence-assisted diagnosis support system of morphological examination based on bone marrow smears including cells detection, classification and prediction of leukemia types. For cell detection, we trained the novel YOLOX-s model to locate cells precisely and obtain single cell images. For cell classification, we regarded it as a fine- grained classification task and proposed a novel architecture called MLFL-Net utilizing multi-level features. Furthermore, we predicted the leukemia types on a dataset including 40 normal people (BM transplantation donors) and 40 patients of different kinds of acute leukemia according to the World Health Organization (WHO) standard.

RESULTS

We constructed a large-scale data set of 11,788 fully-annotated micrographs from 728 smears and 131,300 expert-annotated single cell images. With the data set, the detection model achieved 0.9797 AUC and 4.33% box placement error. For cell classification, the total accuracy of our proposed MLFL-Net reached 89.53% which outperformed all the other related models in identifying cell categories. In the meantime, we took acute leukemia as an example to explore the leukemia types prediction procedure of hematological disease. It generated the same diagnostic prediction as the experts gave for 92.5 percent of the cohort.

CONCLUSION

This Artificial Intelligence-assisted system can be implemented to aid in clinical decision making and accelerate diagnosis. The method will contribute to promote the intelligence and modernization of BM cytomorphology, which has vital significance of the development of the medical career.

Cell Count Differentials by Cytomorphology and Next-Generation Flow Cytometry in Bone Marrow Aspirate: An Evidence-Based Approach

Despite a lack of evidence, a bone marrow aspirate differential of 500 cells is commonly used in the clinical setting. We aimed to test the performance of 200-cell counts for daily hematological workup. In total, 660 consecutive samples were analyzed recording differentials at 200 and 500 cells. Additionally, immunophenotype results and preanalytical issues were also evaluated. Clinical and statistical differences between both cutoffs and both methods were checked. An independent control group of 122 patients was included. All comparisons between both cutoffs and both methods for all relevant types of cells did not show statistically significant differences. No significant diagnostic discrepancies were demonstrated in the contingency table analysis. This is a real-life study, and some limitations may be pointed out, such as a different sample sizes according to the type of cell in the immunophenotype analysis, the lack of standardization of some preanalytical events, and the relatively small sample size of the control group. The comparisons of differentials by morphology on 200 and 500 cells, as well as by morphology (both cutoffs) and by immunophenotype, are equivalent from the clinical and statistical point of view. The preanalytical issues play a critical role in the assessment of bone marrow aspirate samples.

Digital pathology and artificial intelligence as the next chapter in diagnostic hematopathology

Digital pathology has a crucial role in diagnostic pathology and is increasingly a technological requirement in the field. Integration of digital slides into the pathology workflow, advanced algorithms, and computer-aided diagnostic techniques extend the frontiers of the pathologist's view beyond the microscopic slide and enable true integration of knowledge and expertise. There is clear potential for artificial intelligence (AI) breakthroughs in pathology and hematopathology. In this review article, we discuss the approach of using machine learning in the diagnosis, classification, and treatment guidelines of hematolymphoid disease, as well as recent progress of artificial intelligence in flow cytometric analysis of hematolymphoid diseases. We review these topics specifically through the potential clinical applications of CellaVision, an automated digital image analyzer of peripheral blood, and Morphogo, a novel artificial intelligence-based bone marrow analyzing system. Adoption of these new technologies will allow pathologists to streamline workflow and achieve faster turnaround time in diagnosing hematological disease.

An Automated Pipeline for Differential Cell Counts on Whole-Slide Bone Marrow Aspirate Smears

The pathologic diagnosis of bone marrow disorders relies in part on the microscopic analysis of bone marrow aspirate (BMA) smears and the manual counting of marrow nucleated cells to obtain a differential cell count (DCC). This manual process has significant limitations, including the analysis of only a small subset of optimal slide areas and nucleated cells, as well as interobserver variability due to differences in cell selection and classification. To address these shortcomings, we developed an automated machine learning-based pipeline for obtaining 11-component DCCs on whole-slide BMAs. This pipeline uses a sequential process of identifying optimal BMA regions with high proportions of marrow nucleated cells, detecting individual cells within these optimal areas, and classifying these cells into 1 of 11 DCC components. Convolutional neural network models were trained on 396,048 BMA region, 28,914 cell boundary, and 1,510,976 cell class images from manual annotations. The resulting automated pipeline produced 11-component DCCs that demonstrated a high statistical and diagnostic concordance with manual DCCs among a heterogeneous group of testing BMA slides with varying pathologies and cellularities. Additionally, we demonstrated that an automated analysis can reduce the intraslide variance in DCCs by analyzing the whole slide and marrow nucleated cells within all optimal regions. Finally, the pipeline outputs of region classification, cell detection, and cell classification can be visualized using whole-slide image analysis software. This study demonstrates the feasibility of a fully automated pipeline for generating DCCs on scanned whole-slide BMA images, with the potential for improving the current standard of practice for utilizing BMA smears in the laboratory analysis of hematologic disorders.

An AI-Aided Diagnostic Framework for Hematologic Neoplasms Based on Morphologic Features and Medical Expertise

A morphologic examination is essential for the diagnosis of hematological diseases. However, its conventional manual operation is time-consuming and laborious. Herein, we attempt to establish an artificial intelligence (AI)-aided diagnostic framework integrating medical expertise. This framework acts as a virtual hematological morphologist (VHM) for diagnosing hematological neoplasms. Two datasets were established as follows: An image dataset was used to train the Faster Region-based Convolutional Neural Network to develop an image-based morphologic feature extraction model. A case dataset containing retrospective morphologic diagnostic data was used to train a support vector machine algorithm to develop a feature-based case identification model based on diagnostic criteria. Integrating these 2 models established a whole-process AI-aided diagnostic framework, namely, VHM, and a 2-stage strategy was applied to practice case diagnosis. The recall and precision of VHM in bone marrow cell classification were 94.65% and 93.95%, respectively. The balanced accuracy, sensitivity, and specificity of VHM were 97.16%, 99.09%, and 92%, respectively, in the differential diagnosis of normal and abnormal cases, and 99.23%, 97.96%, and 100%, respectively, in the precise diagnosis of chronic myelogenous leukemia in chronic phase. This work represents the first attempt, to our knowledge, to extract multimodal morphologic features and to integrate a feature-based case diagnosis model for designing a comprehensive AI-aided morphologic diagnostic framework. The performance of our knowledge-based framework was superior to that of the widely used end-to-end AI-based diagnostic framework in terms of testing accuracy (96.88% vs 68.75%) or generalization ability (97.11% vs 68.75%) in differentiating normal and abnormal cases. The remarkable advantage of VHM is that it follows the logic of clinical diagnostic procedures, making it a reliable and interpretable hematological diagnostic tool.

Evaluation of two semi-supervised learning methods and their combination for automatic classification of bone marrow cells

Differential bone marrow (BM) cell counting is an important test for the diagnosis of various hematological diseases. However, it is difficult to accurately classify BM cells due to non-uniformity and the lack of reproducibility of differential counting. Therefore, automatic classification systems have been developed in which deep learning is used. These systems requires large and accurately labeled datasets for training. To overcome this, we used semi-supervised learning (SSL), in which learning proceeds while labeling. We used three methods: self-training (ST), active learning (AL), and a combination of these methods, and attempted to automatically classify 16 types of BM cell images. ST involves data verification, as in AL, before adding them to the training dataset (confirmed self-training: CST). After 25 rounds of CST, AL, and CST + AL, the initial number of training data increased from 425 to 40,518; 3682; and 47,843, respectively. Accuracies for the test data of 50 images for each cell type were 0.944, 0.941, and 0.976, respectively. Data added with CST or AL showed some imbalances between classes, while CST + AL exhibited fewer imbalances. We suggest that CST + AL, when combined with two SSL methods, is efficient in increasing training data for the development of automatic BM cells classification systems.

Detection of Metastatic Tumor Cells in the Bone Marrow Aspirate Smears by Artificial Intelligence (AI)-Based Morphogo System

Introduction

Metastatic carcinomas of bone marrow (MCBM) are characterized as tumors of non-hematopoietic origin spreading to the bone marrow through blood or lymphatic circulation. The diagnosis is critical for tumor staging, treatment selection and prognostic risk stratification. However, the identification of metastatic carcinoma cells on bone marrow aspiration smears is technically challenging by conventional microscopic screening.

Objective

The aim of this study is to develop an automatic recognition system using deep learning algorithms applied to bone marrow cells image analysis. The system takes advantage of an artificial intelligence (AI)-based method in recognizing metastatic atypical cancer clusters and promoting rapid diagnosis.

Methods

We retrospectively reviewed metastatic non-hematopoietic malignancies in bone marrow aspirate smears collected from 60 cases of patients admitted to Zhongshan Hospital. High resolution digital bone marrow aspirate smear images were generated and automatically analyzed by Morphogo AI based system. Morphogo system was trained and validated using 20748 cell cluster images from randomly selected 50 MCBM patients. 5469 pre-classified cell cluster images from the remaining 10 MCBM patients were used to test the recognition performance between Morphogo and experienced pathologists.

Results

Morphogo exhibited a sensitivity of 56.6%, a specificity of 91.3%, and an accuracy of 82.2% in the recognition of metastatic cancer cells. Morphogo’s classification result was in general agreement with the conventional standard in the diagnosis of metastatic cancer clusters, with a Kappa value of 0.513. The test results between Morphogo and pathologists H1, H2 and H3 agreement demonstrated a reliability coefficient of 0.827. The area under the curve (AUC) for Morphogo to diagnose the cancer cell clusters was 0.865.

Conclusion

In patients with clinical history of cancer, the Morphogo system was validated as a useful screening tool in the identification of metastatic cancer cells in the bone marrow aspirate smears. It has potential clinical application in the diagnostic assessment of metastatic cancers for staging and in screening MCBM during morphology examination when the symptoms of the primary site are indolent.

A Machine Learning Tool Using Digital Microscopy (Morphogo) for the Identification of Abnormal Lymphocytes in the Bone Marrow

Morphological analysis of the bone marrow is an essential step in the diagnosis of hematological disease. The conventional analysis of bone marrow smears is performed under a manual microscope, which is labor-intensive and subject to interobserver variability. The morphological differential diagnosis of abnormal lymphocytes from normal lymphocytes is still challenging. The digital pathology methods integrated with advances in machine learning enable new diagnostic features/algorithms from digital bone marrow cell images in order to optimize classification, thus providing a robust and faster screening diagnostic tool. We have developed a machine learning system, Morphogo, based on algorithms to discriminate abnormal lymphocytes from normal lymphocytes using digital imaging analysis. We retrospectively reviewed 347 cases of bone marrow digital images. Among them, 53 cases had a clinical history and the diagnosis of marrow involvement with lymphoma was confirmed either by morphology or flow cytometry. We split the 53 cases into two groups for training and testing with 43 and 10 cases, respectively. The selected 15,353 cell images were reviewed by pathologists, based on morphological visual appearance, from 43 patients whose diagnosis was confirmed by complementary tests. To expand the range and the precision of recognizing the lymphoid cells in the marrow by automated digital microscopy systems, we developed an algorithm that incorporated color and texture in addition to geometrical cytological features of the variable lymphocyte images which were applied as the training data set. The selected images from the 10 patients were analyzed by the trained artificial intelligence-based recognition system and compared with the final diagnosis rendered by pathologists. The positive predictive value for the identification of the categories of reactive/normal lymphocytes and abnormal lymphoid cells was 99.04%. It seems likely that further training and improvement of the algorithms will facilitate further subclassification of specific lineage subset pathology, e.g., diffuse large B-cell lymphoma from chronic lymphocytic leukemia/small lymphocytic lymphoma, follicular lymphoma, mantle cell lymphoma or even hairy cell leukemia in cases of abnormal malignant lymphocyte classes in the future. This research demonstrated the feasibility of digital pathology and emerging machine learning approaches to automatically diagnose lymphoma cells in the bone marrow based on cytological-histological analyses.

How artificial intelligence might disrupt diagnostics in hematology in the near future

Artificial intelligence (AI) is about to make itself indispensable in the health care sector. Examples of successful applications or promising approaches range from the application of pattern recognition software to pre-process and analyze digital medical images, to deep learning algorithms for subtype or disease classification, and digital twin technology and in silico clinical trials. Moreover, machine-learning techniques are used to identify patterns and anomalies in electronic health records and to perform ad-hoc evaluations of gathered data from wearable health tracking devices for deep longitudinal phenotyping. In the last years, substantial progress has been made in automated image classification, reaching even superhuman level in some instances. Despite the increasing awareness of the importance of the genetic context, the diagnosis in hematology is still mainly based on the evaluation of the phenotype. Either by the analysis of microscopic images of cells in cytomorphology or by the analysis of cell populations in bidimensional plots obtained by flow cytometry. Here, AI algorithms not only spot details that might escape the human eye, but might also identify entirely new ways of interpreting these images. With the introduction of high-throughput next-generation sequencing in molecular genetics, the amount of available information is increasing exponentially, priming the field for the application of machine learning approaches. The goal of all the approaches is to allow personalized and informed interventions, to enhance treatment success, to improve the timeliness and accuracy of diagnoses, and to minimize technically induced misclassifications. The potential of AI-based applications is virtually endless but where do we stand in hematology and how far can we go?