A dataset of microscopic peripheral blood cell images for development of automatic recognition systems

A hybrid approach based on multipath Swin transformer and ConvMixer for white blood cells classification

White blood cells (WBC) play an effective role in the body's defense against parasites, viruses, and bacteria in the human body. Also, WBCs are categorized based on their morphological structures into various subgroups. The number of these WBC types in the blood of non-diseased and diseased people is different. Thus, the study of WBC classification is quite significant for medical diagnosis. Due to the widespread use of deep learning in medical image analysis in recent years, it has also been used in WBC classification. Moreover, the ConvMixer and Swin transformer models, recently introduced, have garnered significant success by attaining efficient long contextual characteristics. Based on this, a new multipath hybrid network is proposed for WBC classification by using ConvMixer and Swin transformer. This proposed model is called Swin Transformer and ConvMixer based Multipath mixer (SC-MP-Mixer). In the SC-MP-Mixer model, firstly, features with strong spatial details are extracted with the ConvMixer. Then Swin transformer effectively handle these features with self-attention mechanism. In addition, the ConvMixer and Swin transformer blocks consist of a multipath structure to obtain better patch representations in the SC-MP-Mixer. To test the performance of the SC-MP-Mixer, experiments were performed on three WBC datasets with 4 (BCCD), 8 (PBC) and 5 (Raabin) classes. The experimental studies resulted in an accuracy of 99.65% for PBC, 98.68% for Raabin, and 95.66% for BCCD. When compared with the studies in the literature and the state-of-the-art models, it was seen that the SC-MP-Mixer had more effective classification results.

A large multi-focus dataset for white blood cell classification

The White Blood Cell (WBC) differential test ranks as the second most frequently performed diagnostic assay. It requires manual confirmation of the peripheral blood smear by experts to identify signs of abnormalities. Automated digital microscopy has emerged as a solution to reduce this labor-intensive process and improve efficiency. Several publicly available datasets provide various WBC subtypes of differing quality and resolution. These datasets have contributed to advancing WBC classification using machine learning techniques. However, digital microscopy of blood cells with high magnification often requires a wider depth of field, posing challenges for automatic digital microscopy that necessitates capturing multiple stacks of focal planes to obtain complete images of specific blood cells. Our dataset provides 25,773 image stacks from 72 patients. The image labels consist of 18 classes encompassing normal and abnormal cells, with two experts reviewing each label. Each image includes 10 z-stacks of cropped 200 by 200 pixel images, captured using a 50X microscope with 400 nm intervals. This study presents a comprehensive multi-focus dataset for WBC classification.

Federated learning using model projection for multi-center disease diagnosis with non-IID data

Multi-center disease diagnosis aims to build a global model for all involved medical centers. Due to privacy concerns, it is infeasible to collect data from multiple centers for training (i.e., centralized learning). Federated Learning (FL) is a decentralized framework that enables multiple clients (e.g., medical centers) to collaboratively train a global model while retaining patient data locally for privacy. However, in practice, the data across medical centers are not independently and identically distributed (Non-IID), causing two challenging issues: (1) catastrophic forgetting at clients, i.e., the local model at clients will forget the knowledge received from the global model after local training, causing reduced performance; and (2) invalid aggregation at the server, i.e., the global model at the server may not be favorable to some clients after model aggregation, resulting in a slow convergence rate. To mitigate these issues, an innovative Federated learning using Model Projection (FedMoP) is proposed, which guarantees: (1) the loss of local model on global data does not increase after local training without accessing the global data so that the performance will not be degenerated; and (2) the loss of global model on local data does not increase after aggregation without accessing local data so that convergence rate can be improved. Extensive experimental results show that our FedMoP outperforms state-of-the-art FL methods in terms of accuracy, convergence rate and communication cost. In particular, our FedMoP also achieves comparable or even higher accuracy than centralized learning. Thus, our FedMoP can ensure privacy protection while outperforming centralized learning in accuracy and communication cost.

CCSI: Continual Class-Specific Impression for data-free class incremental learning

In real-world clinical settings, traditional deep learning-based classification methods struggle with diagnosing newly introduced disease types because they require samples from all disease classes for offline training. Class incremental learning offers a promising solution by adapting a deep network trained on specific disease classes to handle new diseases. However, catastrophic forgetting occurs, decreasing the performance of earlier classes when adapting the model to new data. Prior proposed methodologies to overcome this require perpetual storage of previous samples, posing potential practical concerns regarding privacy and storage regulations in healthcare. To this end, we propose a novel data-free class incremental learning framework that utilizes data synthesis on learned classes instead of data storage from previous classes. Our key contributions include acquiring synthetic data known as Continual Class-Specific Impression (CCSI) for previously inaccessible trained classes and presenting a methodology to effectively utilize this data for updating networks when introducing new classes. We obtain CCSI by employing data inversion over gradients of the trained classification model on previous classes starting from the mean image of each class inspired by common landmarks shared among medical images and utilizing continual normalization layers statistics as a regularizer in this pixel-wise optimization process. Subsequently, we update the network by combining the synthesized data with new class data and incorporate several losses, including an intra-domain contrastive loss to generalize the deep network trained on the synthesized data to real data, a margin loss to increase separation among previous classes and new ones, and a cosine-normalized cross-entropy loss to alleviate the adverse effects of imbalanced distributions in training data. Extensive experiments show that the proposed framework achieves state-of-the-art performance on four of the public MedMNIST datasets and in-house echocardiography cine series, with an improvement in classification accuracy of up to 51% compared to baseline data-free methods. Our code is available at https://github.com/ubc-tea/Continual-Impression-CCSI.

Continual learning in medical image analysis: A survey

In the dynamic realm of practical clinical scenarios, Continual Learning (CL) has gained increasing interest in medical image analysis due to its potential to address major challenges associated with data privacy, model adaptability, memory inefficiency, prediction robustness and detection accuracy. In general, the primary challenge in adapting and advancing CL remains catastrophic forgetting. Beyond this challenge, recent years have witnessed a growing body of work that expands our comprehension and application of continual learning in the medical domain, highlighting its practical significance and intricacy. In this paper, we present an in-depth and up-to-date review of the application of CL in medical image analysis. Our discussion delves into the strategies employed to address specific tasks within the medical domain, categorizing existing CL methods into three settings: Task-Incremental Learning, Class-Incremental Learning, and Domain-Incremental Learning. These settings are further subdivided based on representative learning strategies, allowing us to assess their strengths and weaknesses in the context of various medical scenarios. By establishing a correlation between each medical challenge and the corresponding insights provided by CL, we provide a comprehensive understanding of the potential impact of these techniques. To enhance the utility of our review, we provide an overview of the commonly used benchmark medical datasets and evaluation metrics in the field. Through a comprehensive comparison, we discuss promising future directions for the application of CL in medical image analysis. A comprehensive list of studies is being continuously updated at https://github.com/xw1519/Continual-Learning-Medical-Adaptation.

Application of image recognition technology in pathological diagnosis of blood smears

Traditional manual blood smear diagnosis methods are time-consuming and prone to errors, often relying heavily on the experience of clinical laboratory analysts for accuracy. As breakthroughs in key technologies such as neural networks and deep learning continue to drive digital transformation in the medical field, image recognition technology is increasingly being leveraged to enhance existing medical processes. In recent years, advancements in computer technology have led to improved efficiency in the identification of blood cells in blood smears through the use of image recognition technology. This paper provides a comprehensive summary of the methods and steps involved in utilizing image recognition algorithms for diagnosing diseases in blood smears, with a focus on malaria and leukemia. Furthermore, it offers a forward-looking research direction for the development of a comprehensive blood cell pathological detection system.

Empirical validation of Conformal Prediction for trustworthy skin lesions classification

BACKGROUND AND OBJECTIVE

Uncertainty quantification is a pivotal field that contributes to realizing reliable and robust systems. It becomes instrumental in fortifying safe decisions by providing complementary information, particularly within high-risk applications. existing studies have explored various methods that often operate under specific assumptions or necessitate substantial modifications to the network architecture to effectively account for uncertainties. The objective of this paper is to study Conformal Prediction, an emerging distribution-free uncertainty quantification technique, and provide a comprehensive understanding of the advantages and limitations inherent in various methods within the medical imaging field.

METHODS

In this study, we developed Conformal Prediction, Monte Carlo Dropout, and Evidential Deep Learning approaches to assess uncertainty quantification in deep neural networks. The effectiveness of these methods is evaluated using three public medical imaging datasets focused on detecting pigmented skin lesions and blood cell types.

RESULTS

The experimental results demonstrate a significant enhancement in uncertainty quantification with the utilization of the Conformal Prediction method, surpassing the performance of the other two methods. Furthermore, the results present insights into the effectiveness of each uncertainty method in handling Out-of-Distribution samples from domain-shifted datasets. Our code is available at: github.com/jfayyad/ConformalDx.

CONCLUSIONS

Our conclusion highlights a robust and consistent performance of conformal prediction across diverse testing conditions. This positions it as the preferred choice for decision-making in safety-critical applications.

Comprehensive data analysis of white blood cells with classification and segmentation by using deep learning approaches

Deep learning approaches have frequently been used in the classification and segmentation of human peripheral blood cells. The common feature of previous studies was that they used more than one dataset, but used them separately. No study has been found that combines more than two datasets to use together. In classification, five types of white blood cells were identified by using a mixture of four different datasets. In segmentation, four types of white blood cells were determined, and three different neural networks, including CNN (Convolutional Neural Network), UNet and SegNet, were applied. The classification results of the presented study were compared with those of related studies. The balanced accuracy was 98.03%, and the test accuracy of the train-independent dataset was determined to be 97.27%. For segmentation, accuracy rates of 98.9% for train-dependent dataset and 92.82% for train-independent dataset for the proposed CNN were obtained in both nucleus and cytoplasm detection. In the presented study, the proposed method showed that it could detect white blood cells from a train-independent dataset with high accuracy. Additionally, it is promising as a diagnostic tool that can be used in the clinical field, with successful results in classification and segmentation.

Classification of rotation-invariant biomedical images using equivariant neural networks

Transmission electron microscopy (TEM) is an imaging technique used to visualize and analyze nano-sized structures and objects such as virus particles. Light microscopy can be used to diagnose diseases or characterize e.g. blood cells. Since samples under microscopes exhibit certain symmetries, such as global rotation invariance, equivariant neural networks are presumed to be useful. In this study, a baseline convolutional neural network is constructed in the form of the commonly used VGG16 classifier. Thereafter, it is modified to be equivariant to the p4 symmetry group of rotations of multiples of 90° using group convolutions. This yields a number of benefits on a TEM virus dataset, including higher top validation set accuracy by on average 7.6% and faster convergence during training by on average 23.1% of that of the baseline. Similarly, when training and testing on images of blood cells, the convergence time for the equivariant neural network is 7.9% of that of the baseline. From this it is concluded that augmentation strategies for rotation can be skipped. Furthermore, when modelling the accuracy versus amount of TEM virus training data with a power law, the equivariant network has a slope of - 0.43 compared to - 0.26 of the baseline. Thus the equivariant network learns faster than the baseline when more training data is added. This study extends previous research on equivariant neural networks applied to images which exhibit symmetries to isometric transformations.

Segmentation, feature extraction and classification of leukocytes leveraging neural networks, a comparative study

The gold standard of leukocyte differentiation is a manual examination of blood smears, which is not only time and labor intensive but also susceptible to human error. As to automatic classification, there is still no comparative study of cell segmentation, feature extraction, and cell classification, where a variety of machine and deep learning models are compared with home-developed approaches. In this study, both traditional machine learning of K-means clustering versus deep learning of U-Net, U-Net + ResNet18, and U-Net + ResNet34 were used for cell segmentation, producing segmentation accuracies of 94.36% versus 99.17% for the dataset of CellaVision and 93.20% versus 98.75% for the dataset of BCCD, confirming that deep learning produces higher performance than traditional machine learning in leukocyte classification. In addition, a series of deep-learning approaches, including AlexNet, VGG16, and ResNet18, was adopted to conduct feature extraction and cell classification of leukocytes, producing classification accuracies of 91.31%, 97.83%, and 100% of CellaVision as well as 81.18%, 91.64% and 97.82% of BCCD, confirming the capability of the increased deepness of neural networks in leukocyte classification. As to the demonstrations, this study further conducted cell-type classification of ALL-IDB2 and PCB-HBC datasets, producing high accuracies of 100% and 98.49% among all literature, validating the deep learning model used in this study.

DeepLeish: a deep learning based support system for the detection of Leishmaniasis parasite from Giemsa-stained microscope images

BACKGROUND

Leishmaniasis is a vector-born neglected parasitic disease belonging to the genus Leishmania. Out of the 30 Leishmania species, 21 species cause human infection that affect the skin and the internal organs. Around, 700,000 to 1,000,000 of the newly infected cases and 26,000 to 65,000 deaths are reported worldwide annually. The disease exhibits three clinical presentations, namely, the cutaneous, muco-cutaneous and visceral Leishmaniasis which affects the skin, mucosal membrane and the internal organs, respectively. The relapsing behavior of the disease limits its diagnosis and treatment efficiency. The common diagnostic approaches follow subjective, error-prone, repetitive processes. Despite, an ever pressing need for an accurate detection of Leishmaniasis, the research conducted so far is scarce. In this regard, the main aim of the current research is to develop an artificial intelligence based detection tool for the Leishmaniasis from the Geimsa-stained microscopic images using deep learning method.

METHODS

Stained microscopic images were acquired locally and labeled by experts. The images were augmented using different methods to prevent overfitting and improve the generalizability of the system. Fine-tuned Faster RCNN, SSD, and YOLOV5 models were used for object detection. Mean average precision (MAP), precision, and Recall were calculated to evaluate and compare the performance of the models.

RESULTS

The fine-tuned YOLOV5 outperformed the other models such as Faster RCNN and SSD, with the MAP scores, of 73%, 54% and 57%, respectively.

CONCLUSION

The currently developed YOLOV5 model can be tested in the clinics to assist the laboratorists in diagnosing Leishmaniasis from the microscopic images. Particularly, in low-resourced healthcare facilities, with fewer qualified medical professionals or hematologists, our AI support system can assist in reducing the diagnosing time, workload, and misdiagnosis. Furthermore, the dataset collected by us will be shared with other researchers who seek to improve upon the detection system of the parasite. The current model detects the parasites even in the presence of the monocyte cells, but sometimes, the accuracy decreases due to the differences in the sizes of the parasite cells alongside the blood cells. The incorporation of cascaded networks in future and the quantification of the parasite load, shall overcome the limitations of the currently developed system.

Implementing vision transformer for classifying 2D biomedical images

In recent years, the growth spurt of medical imaging data has led to the development of various machine learning algorithms for various healthcare applications. The MedMNISTv2 dataset, a comprehensive benchmark for 2D biomedical image classification, encompasses diverse medical imaging modalities such as Fundus Camera, Breast Ultrasound, Colon Pathology, Blood Cell Microscope etc. Highly accurate classifications performed on these datasets is crucial for identification of various diseases and determining the course of treatment. This research paper presents a comprehensive analysis of four subsets within the MedMNISTv2 dataset: BloodMNIST, BreastMNIST, PathMNIST and RetinaMNIST. Each of these selected datasets is of diverse data modalities and comes with various sample sizes, and have been selected to analyze the efficiency of the model against diverse data modalities. The study explores the idea of assessing the Vision Transformer Model's ability to capture intricate patterns and features crucial for these medical image classification and thereby transcend the benchmark metrics substantially. The methodology includes pre-processing the input images which is followed by training the ViT-base-patch16-224 model on the mentioned datasets. The performance of the model is assessed using key metrices and by comparing the classification accuracies achieved with the benchmark accuracies. With the assistance of ViT, the new benchmarks achieved for BloodMNIST, BreastMNIST, PathMNIST and RetinaMNIST are 97.90%, 90.38%, 94.62% and 57%, respectively. The study highlights the promise of Vision transformer models in medical image analysis, preparing the way for their adoption and further exploration in healthcare applications, aiming to enhance diagnostic accuracy and assist medical professionals in clinical decision-making.

Suppressing label noise in medical image classification using mixup attention and self-supervised learning

Deep neural networks (DNNs) have been widely applied in medical image classification and achieve remarkable classification performance. These achievements heavily depend on large-scale accurately annotated training data. However, label noise is inevitably introduced in the medical image annotation, as the labeling process heavily relies on the expertise and experience of annotators. Meanwhile, DNNs suffer from overfitting noisy labels, degrading the performance of models. Therefore, in this work, we innovatively devise a noise-robust training approach to mitigate the adverse effects of noisy labels in medical image classification. Specifically, we incorporate contrastive learning and intra-group mixup attention strategies into vanilla supervised learning. The contrastive learning for feature extractor helps to enhance visual representation of DNNs. The intra-group mixup attention module constructs groups and assigns self-attention weights for group-wise samples, and subsequently interpolates massive noisy-suppressed samples through weighted mixup operation. We conduct comparative experiments on both synthetic and real-world noisy medical datasets under various noise levels. Rigorous experiments validate that our noise-robust method with contrastive learning and mixup attention can effectively handle with label noise, and is superior to state-of-the-art methods. An ablation study also shows that both components contribute to boost model performance. The proposed method demonstrates its capability of curb label noise and has certain potential toward real-world clinic applications.

Efficient leukocytes detection and classification in microscopic blood images using convolutional neural network coupled with a dual attention network

Leukocytes, also called White Blood Cells (WBCs) or leucocytes, are the cells that play a pivotal role in human health and are vital indicators of diseases such as malaria, leukemia, AIDS, and other viral infections. WBCs detection and classification in blood smears offers insights to pathologists, aiding diagnosis across medical conditions. Traditional techniques, including manual counting, detection, classification, and visual inspection of microscopic images by medical professionals, pose challenges due to their labor-intensive nature. However, traditional methods are time consuming and sometimes susceptible to errors. Here, we propose a high-performance convolutional neural network (CNN) coupled with a dual-attention network that efficiently detects and classifies WBCs in microscopic thick smear images. The main aim of this study was to enhance clinical hematology systems and expedite medical diagnostic processes. In the proposed technique, we utilized a deep convolutional generative adversarial network (DCGAN) to overcome the limitations imposed by limited training data and employed a dual attention mechanism to improve accuracy, efficiency, and generalization. The proposed technique achieved overall accuracy rates of 99.83%, 99.35%, and 99.60% for the peripheral blood cell (PBC), leukocyte images for segmentation and classification (LISC), and Raabin-WBC benchmark datasets, respectively. Our proposed approach outperforms state-of-the-art methods in terms of accuracy, highlighting the effectiveness of the strategies employed and their potential to enhance diagnostic capabilities and advance real-world healthcare practices and diagnostic systems.

Multi-sample ζ-mixup: richer, more realistic synthetic samples from a p-series interpolant

Modern deep learning training procedures rely on model regularization techniques such as data augmentation methods, which generate training samples that increase the diversity of data and richness of label information. A popular recent method, mixup, uses convex combinations of pairs of original samples to generate new samples. However, as we show in our experiments, mixup  can produce undesirable synthetic samples, where the data is sampled off the manifold and can contain incorrect labels. We propose ζ -mixup, a generalization of mixup  with provably and demonstrably desirable properties that allows convex combinations of T ≥ 2 samples, leading to more realistic and diverse outputs that incorporate information from T original samples by using a p-series interpolant. We show that, compared to mixup, ζ -mixup  better preserves the intrinsic dimensionality of the original datasets, which is a desirable property for training generalizable models. Furthermore, we show that our implementation of ζ -mixup  is faster than mixup, and extensive evaluation on controlled synthetic and 26 diverse real-world natural and medical image classification datasets shows that ζ -mixup  outperforms mixup, CutMix, and traditional data augmentation techniques. The code will be released at https://github.com/kakumarabhishek/zeta-mixup.

Split Learning for Distributed Collaborative Training of Deep Learning Models in Health Informatics

Deep learning continues to rapidly evolve and is now demonstrating remarkable potential for numerous medical prediction tasks. However, realizing deep learning models that generalize across healthcare organizations is challenging. This is due, in part, to the inherent siloed nature of these organizations and patient privacy requirements. To address this problem, we illustrate how split learning can enable collaborative training of deep learning models across disparate and privately maintained health datasets, while keeping the original records and model parameters private. We introduce a new privacy-preserving distributed learning framework that offers a higher level of privacy compared to conventional federated learning. We use several biomedical imaging and electronic health record (EHR) datasets to show that deep learning models trained via split learning can achieve highly similar performance to their centralized and federated counterparts while greatly improving computational efficiency and reducing privacy risks.

Automatic recognition of white blood cell images with memory efficient superpixel metric GNN: SMGNN

An automatic recognizing system of white blood cells can assist hematologists in the diagnosis of many diseases, where accuracy and efficiency are paramount for computer-based systems. In this paper, we presented a new image processing system to recognize the five types of white blood cells in peripheral blood with marked improvement in efficiency when juxtaposed against mainstream methods. The prevailing deep learning segmentation solutions often utilize millions of parameters to extract high-level image features and neglect the incorporation of prior domain knowledge, which consequently consumes substantial computational resources and increases the risk of overfitting, especially when limited medical image samples are available for training. To address these challenges, we proposed a novel memory-efficient strategy that exploits graph structures derived from the images. Specifically, we introduced a lightweight superpixel-based graph neural network (GNN) and broke new ground by introducing superpixel metric learning to segment nucleus and cytoplasm. Remarkably, our proposed segmentation model superpixel metric graph neural network (SMGNN) achieved state of the art segmentation performance while utilizing at most 10000$ \times $ less than the parameters compared to existing approaches. The subsequent segmentation-based cell type classification processes showed satisfactory results that such automatic recognizing algorithms are accurate and efficient to execeute in hematological laboratories. Our code is publicly available at https://github.com/jyh6681/SPXL-GNN.

Automatic base-model selection for white blood cell image classification using meta-learning

Healthcare has benefited from the implementation of deep-learning models to solve medical image classification tasks. For example, White Blood Cell (WBC) image analysis is used to diagnose different pathologies like leukemia. However, medical datasets are mostly imbalanced, inconsistent, and costly to collect. Hence, it is difficult to select an adequate model to overcome the mentioned drawbacks. Therefore, we propose a novel methodology to automatically select models to solve WBC classification tasks. These tasks contain images collected using different staining methods, microscopes, and cameras. The proposed methodology includes meta- and base-level learnings. At the meta-level, we implemented meta-models based on prior-models to acquire meta-knowledge by solving meta-tasks using the shades of gray color constancy method. To determine the best models to solve new WBC tasks we developed an algorithm that uses the meta-knowledge and the Centered Kernel Alignment metric. Next, a learning rate finder method is employed to adapt the selected models. The adapted models (base-models) are used in an ensemble learning approach achieving accuracy and balanced accuracy scores of 98.29 and 97.69 in the Raabin dataset; 100 in the BCCD dataset; 99.57 and 99.51 in the UACH dataset, respectively. The results in all datasets outperform most of the state-of-the-art models, which demonstrates our methodology's advantage of automatically selecting the best model to solve WBC tasks. The findings also indicate that our methodology can be extended to other medical image classification tasks where is difficult to select an adequate deep-learning model to solve new tasks with imbalanced, limited, and out-of-distribution data.

A high-resolution large-scale dataset of pathological and normal white blood cells

Microscopic examination plays a significant role in the initial screening for a variety of hematological, as well as non-hematological, diagnoses. Microscopic blood smear examination that is considered a key diagnostic technique, is in recent clinical practice still performed manually, which is not only time consuming, but can lead to human errors. Although automated and semi-automated systems have been developed in recent years, their high purchasing and maintenance costs make them unaffordable for many medical institutions. Even though much research has been conducted lately to explore more accurate and feasible solutions, most researchers had to deal with a lack of medical data. To address the lack of large-scale databases in this field, we created a high-resolution dataset containing a total of 16027 annotated white blood cells. Moreover, the dataset covers overall 9 types of white blood cells, including clinically significant pathological findings. Since we used high-quality acquisition equipment, the dataset provides one of the highest quality images of blood cells, achieving an approximate resolution of 42 pixels per 1 μm.

MedViT: A robust vision transformer for generalized medical image classification

Convolutional Neural Networks (CNNs) have advanced existing medical systems for automatic disease diagnosis. However, there are still concerns about the reliability of deep medical diagnosis systems against the potential threats of adversarial attacks since inaccurate diagnosis could lead to disastrous consequences in the safety realm. In this study, we propose a highly robust yet efficient CNN-Transformer hybrid model which is equipped with the locality of CNNs as well as the global connectivity of vision Transformers. To mitigate the high quadratic complexity of the self-attention mechanism while jointly attending to information in various representation subspaces, we construct our attention mechanism by means of an efficient convolution operation. Moreover, to alleviate the fragility of our Transformer model against adversarial attacks, we attempt to learn smoother decision boundaries. To this end, we augment the shape information of an image in the high-level feature space by permuting the feature mean and variance within mini-batches. With less computational complexity, our proposed hybrid model demonstrates its high robustness and generalization ability compared to the state-of-the-art studies on a large-scale collection of standardized MedMNIST-2D datasets.

Deep self-supervised transformation learning for leukocyte classification

The scarcity of training annotation is one of the major challenges for the application of deep learning technology in medical image analysis. Recently, self-supervised learning provides a powerful solution to alleviate this challenge by extracting useful features from a large number of unlabeled training data. In this article, we propose a simple and effective self-supervised learning method for leukocyte classification by identifying the different transformations of leukocyte images, without requiring a large batch of negative sampling or specialized architectures. Specifically, a convolutional neural network backbone takes different transformations of leukocyte image as input for feature extraction. Then, a pretext task of self-supervised transformation recognition on the extracted feature is conducted by a classifier, which helps the backbone learn useful representations that generalize well across different leukocyte types and datasets. In the experiment, we systematically study the effect of different transformation compositions on useful leukocyte feature extraction. Compared with five typical baselines of self-supervised image classification, experimental results demonstrate that our method performs better in different evaluation protocols including linear evaluation, domain transfer, and finetuning, which proves the effectiveness of the proposed method.

Few-shot learning using explainable Siamese twin network for the automated classification of blood cells

Automated classification of blood cells from microscopic images is an interesting research area owing to advancements of efficient neural network models. The existing deep learning methods rely on large data for network training and generating such large data could be time-consuming. Further, explainability is required via class activation mapping for better understanding of the model predictions. Therefore, we developed a Siamese twin network (STN) model based on contrastive learning that trains on relatively few images for the classification of healthy peripheral blood cells using EfficientNet-B3 as the base model. Hence, in this study, a total of 17,092 publicly accessible cell histology images were analyzed from which 6% were used for STN training, 6% for few-shot validation, and the rest 88% for few-shot testing. The proposed architecture demonstrates percent accuracies of 97.00, 98.78, 94.59, 95.70, 98.86, 97.09, 99.71, and 96.30 during 8-way 5-shot testing for the classification of basophils, eosinophils, immature granulocytes, erythroblasts, lymphocytes, monocytes, platelets, and neutrophils, respectively. Further, we propose a novel class activation mapping scheme that highlights the important regions in the test image for the STN model interpretability. Overall, the proposed framework could be used for a fully automated self-exploratory classification of healthy peripheral blood cells. The whole proposed framework demonstrates the Siamese twin network training and 8-way k-shot testing. The values indicate the amount of dissimilarity.

Detection of acute promyelocytic leukemia in peripheral blood and bone marrow with annotation-free deep learning

While optical microscopy inspection of blood films and bone marrow aspirates by a hematologist is a crucial step in establishing diagnosis of acute leukemia, especially in low-resource settings where other diagnostic modalities are not available, the task remains time-consuming and prone to human inconsistencies. This has an impact especially in cases of Acute Promyelocytic Leukemia (APL) that require urgent treatment. Integration of automated computational hematopathology into clinical workflows can improve the throughput of these services and reduce cognitive human error. However, a major bottleneck in deploying such systems is a lack of sufficient cell morphological object-labels annotations to train deep learning models. We overcome this by leveraging patient diagnostic labels to train weakly-supervised models that detect different types of acute leukemia. We introduce a deep learning approach, Multiple Instance Learning for Leukocyte Identification (MILLIE), able to perform automated reliable analysis of blood films with minimal supervision. Without being trained to classify individual cells, MILLIE differentiates between acute lymphoblastic and myeloblastic leukemia in blood films. More importantly, MILLIE detects APL in blood films (AUC 0.94 ± 0.04) and in bone marrow aspirates (AUC 0.99 ± 0.01). MILLIE is a viable solution to augment the throughput of clinical pathways that require assessment of blood film microscopy.

A deep learning model for detection of leukocytes under various interference factors

The accurate detection of leukocytes is the basis for the diagnosis of blood system diseases. However, diagnosing leukocyte disorders by doctors is time-consuming and requires extensive experience. Automated detection methods with high accuracy can improve detection efficiency and provide recommendations to inexperienced doctors. Current methods and instruments either fail to automate the identification process fully or have low performance and need suitable leukocyte data sets for further study. To improve the current status, we need to develop more intelligent strategies. This paper investigates fulfilling high-performance automatic detection for leukocytes using a deep learning-based method. We established a new dataset more suitable for leukocyte detection, containing 6273 images (8595 leukocytes) and considering nine common clinical interference factors. Based on the dataset, the performance evaluation of six mainstream detection models is carried out, and a more robust ensemble model is proposed. The mean of average precision (mAP) @IoU = 0.50:0.95 and mean of average recall (mAR)@IoU = 0.50:0.95 of the ensemble model on the test set are 0.853 and 0.922, respectively. The detection performance of poor-quality images is robust. For the first time, it is found that the ensemble model yields an accuracy of 98.84% for detecting incomplete leukocytes. In addition, we also compared the test results of different models and found multiple identical false detections of the models, then provided correct suggestions for the clinic.

Leukocyte deep learning classification assessment using Shapley additive explanations algorithm

INTRODUCTION

A peripheral blood smear is a basic test for hematological disease diagnosis. This test is performed manually in many places worldwide, which requires both time and qualified staff. Large laboratories are equipped with digital morphology analyzers, some of which are based on deep learning methods. However, it is difficult to explain to scientists how they work. In this paper, we proposed to add an explanatory factor to enhance the interpretability of deep learning models in leukocyte classification.

METHODS

10 297 single images of leukocytes obtained from peripheral blood smears were included in this study. Pre-trained and fully trained VGG16 and VGG19 models were used to classify the leukocytes, and Shapley Additive Explanations (SHAP) DeepExplainer was applied to visualize the area of cells that were significant for classification. The output images from the DeepExplainer were compared with cellular elements that are essential to laboratory practice.

RESULTS

The accuracy of our fully trained models was 99.81% for VGG16 and 99.79% for VGG19. It achieved slightly better results than the partially trained model, which scored 98.67% for VGG16 and 98.33% for VGG19. Their SHAP explanations indicated the significance of cellular structures in microscopic examination. Explanations in the pre-trained models have proved the cell and nucleus contours to be relevant to classification, while explanations in the fully trained models pointed to the cytoplasm area.

CONCLUSION

Despite different SHAP DeepExplainer explanations for fully and partially trained models, this method appears to be helpful for the verification of leukocyte classification in automated peripheral blood smear examination.

Automatic generation of artificial images of leukocytes and leukemic cells using generative adversarial networks (syntheticcellgan)

BACKGROUND AND OBJECTIVES

Visual analysis of cell morphology has an important role in the diagnosis of hematological diseases. Morphological cell recognition is a challenge that requires experience and in-depth review by clinical pathologists. Within the new trend of introducing computer-aided diagnostic tools in laboratory medicine, models based on deep learning are being developed for the automatic identification of different types of cells in peripheral blood. In general, well-annotated large image sets are needed to train the models to reach a desired classification performance. This is especially relevant when it comes to discerning between cell images in which morphological differences are subtle and when it comes to low prevalent diseases with the consequent difficulty in collecting cell images. The objective of this work is to develop, train and validate SyntheticCellGAN (SCG), a new system for the automatic generation of artificial images of white blood cells, maintaining morphological characteristics very close to real cells found in practice in clinical laboratories.

METHODS

SCG is designed with two sequential generative adversarial networks. First, a Wasserstein structure is used to transform random noise vectors into low resolution images of basic mononuclear cells. Second, the concept of image-to-image translation is used to build specific models that transform the basic images into high-resolution final images with the realistic morphology of each cell type target: 1) the five groups of normal leukocytes (lymphocytes, monocytes, eosinophils, neutrophils and basophils); 2) atypical promyelocytes and hairy cells, which are two relevant cell types of complex morphology with low abundance in blood smears.

RESULTS

The images of the SCG system are evaluated with four experimental tests. In the first test we evaluated the generated images with quantitative metrics for GANs. In the second test, morphological verification of the artificial images is performed by expert clinical pathologists with 100% accuracy. In the third test, two classifiers based on convolutional neural networks (CNN) previously trained with images of real cells are used. Two sets of artificial images of the SCG system are classified with an accuracy of 95.36% and 94%, respectively. In the fourth test, three CNN classifiers are trained with artificial images of the SCG system. Real cells are identified with an accuracy ranging from 87.7% to 100%.

CONCLUSIONS

The SCG system has proven effective in creating images of all normal leukocytes and two low-prevalence cell classes associated with diseases such as acute promyelocyte leukemia and hairy cell leukemia. Once trained, the system requires low computational cost and can help augment high-quality image datasets to improve automatic recognition model training for clinical laboratory practice.

MedMNIST v2 - A large-scale lightweight benchmark for 2D and 3D biomedical image classification

We introduce MedMNIST v2, a large-scale MNIST-like dataset collection of standardized biomedical images, including 12 datasets for 2D and 6 datasets for 3D. All images are pre-processed into a small size of 28 × 28 (2D) or 28 × 28 × 28 (3D) with the corresponding classification labels so that no background knowledge is required for users. Covering primary data modalities in biomedical images, MedMNIST v2 is designed to perform classification on lightweight 2D and 3D images with various dataset scales (from 100 to 100,000) and diverse tasks (binary/multi-class, ordinal regression, and multi-label). The resulting dataset, consisting of 708,069 2D images and 9,998 3D images in total, could support numerous research/educational purposes in biomedical image analysis, computer vision, and machine learning. We benchmark several baseline methods on MedMNIST v2, including 2D/3D neural networks and open-source/commercial AutoML tools. The data and code are publicly available at https://medmnist.com/ .

RCMNet: A deep learning model assists CAR-T therapy for leukemia

Acute leukemia is a type of blood cancer with a high mortality rate. Current therapeutic methods include bone marrow transplantation, supportive therapy, and chemotherapy. Although a satisfactory remission of the disease can be achieved, the risk of recurrence is still high. Therefore, novel treatments are demanding. Chimeric antigen receptor-T (CAR-T) therapy has emerged as a promising approach to treating and curing acute leukemia. To harness the therapeutic potential of CAR-T cell therapy for blood diseases, reliable cell morphological identification is crucial. Nevertheless, the identification of CAR-T cells is a big challenge posed by their phenotypic similarity with other blood cells. To address this substantial clinical challenge, herein we first construct a CAR-T dataset with 500 original microscopy images after staining. Following that, we create a novel integrated model called RCMNet (ResNet18 with Convolutional Block Attention Module and Multi-Head Self-Attention) that combines the convolutional neural network (CNN) and Transformer. The model shows 99.63% top-1 accuracy on the public dataset. Compared with previous reports, our model obtains satisfactory results for image classification. Although testing on the CAR-T cell dataset, a decent performance is observed, which is attributed to the limited size of the dataset. Transfer learning is adapted for RCMNet and a maximum of 83.36% accuracy is achieved, which is higher than that of other state-of-the-art models. This study evaluates the effectiveness of RCMNet on a big public dataset and translates it to a clinical dataset for diagnostic applications.

AIMIC: Deep Learning for Microscopic Image Classification

BACKGROUND AND OBJECTIVE

Deep learning techniques are powerful tools for image analysis. However, the lack of programming experience makes it difficult for novice users to apply this technology. This project aims to lower the barrier for clinical users to implement deep learning methods in microscopic image classification.

METHODS

In this study, an out-of-the-box software, AIMIC (artificial intelligence-based microscopy image classifier), was developed for users to apply deep learning technology in a code-free manner. The platform was equipped with state-of-the-art deep learning techniques and data preprocessing approaches. Furthermore, we evaluated the built-in networks on four benchmark microscopy image datasets to assist entry-level practitioners in selecting a suitable algorithm.

RESULTS

The entire deep learning pipeline, from training a new network to inferring unseen samples using the trained model, could be implemented on the proposed platform without the need for programming. In the evaluation experiments, the ResNeXt-50-32×4d outperformed other competitor algorithms in terms of average accuracy (96.83%) and average F1-score (96.82%). In addition, the MobileNet-V2 achieved a good balance between the performance (accuracy of 95.72%) and computational cost (inference time of 0.109s for identifying one sample).

CONCLUSIONS

The proposed AI platform allows people without programming experience to use artificial intelligence methods in microscopy image analysis. Besides, the ResNeXt-50-32×4d is a preferable solution for microscopic image classification, and MobileNet-V2 is most likely to be an alternative selection for the scenario when computing resources are limited.

Enhancing classification of cells procured from bone marrow aspirate smears using generative adversarial networks and sequential convolutional neural network

BACKGROUND AND OBJECTIVE

Leukemia represents 30% of all pediatric cancers and is considered the most common malignancy affecting adults and children. Cell differential count obtained from bone marrow aspirate smears is crucial for diagnosing hematologic diseases. Classification of these cell types is an essential task towards analyzing the disease, but it is time-consuming and requires intensive manual intervention. While machine learning has shown excellent outcomes in automating medical diagnosis, it needs ample data to build an efficient model for real-world tasks. This paper aims to generate synthetic data to enhance the classification accuracy of cells obtained from bone marrow aspirate smears.

METHODS

A three-stage architecture has been proposed. We first collaborate with experts from the medical domain to prepare a dataset that consolidates microscopic cell images obtained from bone marrow aspirate smears from three different sources. The second stage involves a generative adversarial networks (GAN) model to generate synthetic microscopic cell images. We propose a GAN model consisting of three networks; generator discriminator and classifier. We train the GAN model with the loss function of Wasserstein GAN with gradient penalty (WGAN-GP). Since our GAN has an additional classifier and was trained using WGAN-GP, we named our model C-WGAN-GP. In the third stage, we propose a sequential convolutional neural network (CNN) to classify cells in the original and synthetic dataset to demonstrate how generating synthetic data and utilizing a simple sequential CNN model can enhance the accuracy of cell classification.

RESULTS

We validated the proposed C-WGAN-GP and sequential CNN model with various evaluation metrics and achieved a classification accuracy of 96.98% using the synthetic dataset. We have presented each cell type's accuracy, specificity, and sensitivity results. The sequential CNN model achieves the highest accuracy for neutrophils with an accuracy rate of 97.5%. The highest value for sensitivity and specificity are 97.1% and 97%. Our proposed GAN model achieved an inception score of 14.52 ± 0.10, significantly better than the existing GAN models.

CONCLUSIONS

Using three network GAN architecture produced more realistic synthetic data than existing models. Sequential CNN model with the synthetic data achieved higher classification accuracy than the original data.

Feature transforms for image data augmentation

A problem with convolutional neural networks (CNNs) is that they require large datasets to obtain adequate robustness; on small datasets, they are prone to overfitting. Many methods have been proposed to overcome this shortcoming with CNNs. In cases where additional samples cannot easily be collected, a common approach is to generate more data points from existing data using an augmentation technique. In image classification, many augmentation approaches utilize simple image manipulation algorithms. In this work, we propose some new methods for data augmentation based on several image transformations: the Fourier transform (FT), the Radon transform (RT), and the discrete cosine transform (DCT). These and other data augmentation methods are considered in order to quantify their effectiveness in creating ensembles of neural networks. The novelty of this research is to consider different strategies for data augmentation to generate training sets from which to train several classifiers which are combined into an ensemble. Specifically, the idea is to create an ensemble based on a kind of bagging of the training set, where each model is trained on a different training set obtained by augmenting the original training set with different approaches. We build ensembles on the data level by adding images generated by combining fourteen augmentation approaches, with three based on FT, RT, and DCT, proposed here for the first time. Pretrained ResNet50 networks are finetuned on training sets that include images derived from each augmentation method. These networks and several fusions are evaluated and compared across eleven benchmarks. Results show that building ensembles on the data level by combining different data augmentation methods produce classifiers that not only compete competitively against the state-of-the-art but often surpass the best approaches reported in the literature.

Threshold estimation based on local minima for nucleus and cytoplasm segmentation

Background

Image segmentation is the process of partitioning an image into separate objects or regions. It is an essential step in image processing to segment the regions of interest for further processing. We propose a method for segmenting the nuclei and cytoplasms from white blood cells (WBCs).

Methods

Initially, the method computes an initial value based on the minimum and maximum values of the input image. Then, a histogram of the input image is computed and approximated to obtain function values. The method searches for the first local maximum and local minimum from the approximated function values in the order of increasing of knots sequence. We approximate the required threshold from the first local minimum and the computed initial value based on defined conditions. The threshold is applied to the input image to binarize it, and then post-processing is performed to obtain the final segmented nucleus. We segment the whole WBC before segmenting the cytoplasm depending on the complexity of the objects in the image. For WBCs that are well separated from red blood cells (RBCs), n thresholds are generated and then produce n thresholded images. Then, a standard Otsu method is used to binarize the average of the produced images. Morphological operations are applied on the binarized image, and then a single-pixel point from the segmented nucleus is used to segment the WBC. For images in which RBCs touch the WBCs, we segment the whole WBC using SLIC and watershed methods. The cytoplasm is obtained by subtracting the segmented nucleus from the segmented WBC.

Results

The method is tested on two different public data sets and the results are compared to the state of art methods. The performance analysis shows that the proposed method segments the nucleus and cytoplasm well.

Conclusion

We propose a method for nucleus and cytoplasm segmentation based on the local minima of the approximated function values from the image’s histogram. The method has demonstrated its utility in segmenting nuclei, WBCs, and cytoplasm, and the results are satisfactory.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12880-022-00801-w.

Synthesis of Microscopic Cell Images Obtained from Bone Marrow Aspirate Smears through Generative Adversarial Networks

Simple Summary

This paper proposes a hybrid generative adversarial networks model—WGAN-GP-AC—to generate synthetic microscopic cell images. We generate the synthetic data for the cell types containing fewer data to obtain a balanced dataset. A balanced dataset would help enhance the classification accuracy of each cell type and help with an easy and quick diagnosis that is critical for leukemia patients. In this work, we combine images from three datasets to form a single concrete dataset with variations of multiple microscopic cell images. We provide experimental results that prove the correlation between the original and our synthetically generated data. We also deliver classification results to showcase that the generated synthetic data can be used for real-life experiments and the advancement of the medical domain.

Abstract

Every year approximately 1.24 million people are diagnosed with blood cancer. While the rate increases each year, the availability of data for each kind of blood cancer remains scarce. It is essential to produce enough data for each blood cell type obtained from bone marrow aspirate smears to diagnose rare types of cancer. Generating data would help easy and quick diagnosis, which are the most critical factors in cancer. Generative adversarial networks (GAN) are the latest emerging framework for generating synthetic images and time-series data. This paper takes microscopic cell images, preprocesses them, and uses a hybrid GAN architecture to generate synthetic images of the cell types containing fewer data. We prepared a single dataset with expert intervention by combining images from three different sources. The final dataset consists of 12 cell types and has 33,177 microscopic cell images. We use the discriminator architecture of auxiliary classifier GAN (AC-GAN) and combine it with the Wasserstein GAN with gradient penalty model (WGAN-GP). We name our model as WGAN-GP-AC. The discriminator in our proposed model works to identify real and generated images and classify every image with a cell type. We provide experimental results demonstrating that our proposed model performs better than existing individual and hybrid GAN models in generating microscopic cell images. We use the generated synthetic data with classification models, and the results prove that the classification rate increases significantly. Classification models achieved 0.95 precision and 0.96 recall value for synthetic data, which is higher than the original, augmented, or combined datasets.

Atypical lymphoid cells circulating in blood in COVID-19 infection: morphology, immunophenotype and prognosis value

AIMS

Atypical lymphocytes circulating in blood have been reported in COVID-19 patients. This study aims to (1) analyse if patients with reactive lymphocytes (COVID-19 RL) show clinical or biological characteristics related to outcome; (2) develop an automatic system to recognise them in an objective way and (3) study their immunophenotype.

METHODS

Clinical and laboratory findings in 36 COVID-19 patients were compared between those showing COVID-19 RL in blood (18) and those without (18). Blood samples were analysed in Advia2120i and stained with May Grünwald-Giemsa. Digital images were acquired in CellaVisionDM96. Convolutional neural networks (CNNs) were used to accurately recognise COVID-19 RL. Immunophenotypic study was performed throughflow cytometry.

RESULTS

Neutrophils, D-dimer, procalcitonin, glomerular filtration rate and total protein values were higher in patients without COVID-19 RL (p<0.05) and four of these patients died. Haemoglobin and lymphocyte counts were higher (p<0.02) and no patients died in the group showing COVID-19 RL. COVID-19 RL showed a distinct deep blue cytoplasm with nucleus mostly in eccentric position. Through two sequential CNNs, they were automatically distinguished from normal lymphocytes and classical RL with sensitivity, specificity and overall accuracy values of 90.5%, 99.4% and 98.7%, respectively. Immunophenotypic analysis revealed COVID-19 RL are mostly activated effector memory CD4 and CD8 T cells.

CONCLUSION

We found that COVID-19 RL are related to a better evolution and prognosis. They can be detected by morphology in the smear review, being the computerised approach proposed useful to enhance a more objective recognition. Their presence suggests an abundant production of virus-specific T cells, thus explaining the better outcome of patients showing these cells circulating in blood.

BloodCaps: A capsule network based model for the multiclassification of human peripheral blood cells

BACKGROUND AND OBJECTIVE

The classification of human peripheral blood cells yields significance in the detection of inflammation, infections and blood cell disorders such as leukemia. Limitations in traditional algorithms for blood cell classification and increased computational processing power have allowed machine learning methods to be utilized for this clinically prevalent task.

METHODS

In the current work, we present BloodCaps, a capsule based model designed for the accurate multiclassification of a diverse and broad spectrum of blood cells.

RESULTS

Implemented on a large-scale dataset of 8 categories of human peripheral blood cells, the proposed architecture achieved an overall accuracy of 99.3%, outperforming convolutional neural networks such as AlexNet(81.5%), VGG16(97.8%), ResNet-18(95.9%) and InceptionV3(98.4%). Furthermore, we devised three new datasets(low-resolution dataset, small dataset, and low-resolution small dataset) from the original dataset, and tested BloodCaps in comparison with AlexNet, VGG16, ResNet-18, and InceptionV3. To further validate the applicability of our proposed model, we tested BloodCaps on additional public datasets such as the All IDB2, BCCD, and Cell Vision datasets. Compared with the reported results, BloodCaps showed the best performance in all three scenarios.

CONCLUSIONS

The proposed method proved superior in octal classification among all three datasets. We believe the proposed method represents a promising tool to improve the diagnostic performance of clinical blood examinations.

Recent evolutions of machine learning applications in clinical laboratory medicine

Machine learning (ML) is gaining increased interest in clinical laboratory medicine, mainly triggered by the decreased cost of generating and storing data using laboratory automation and computational power, and the widespread accessibility of open source tools. Nevertheless, only a handful of ML-based products are currently commercially available for routine clinical laboratory practice. In this review, we start with an introduction to ML by providing an overview of the ML landscape, its general workflow, and the most commonly used algorithms for clinical laboratory applications. Furthermore, we aim to illustrate recent evolutions (2018 to mid-2020) of the techniques used in the clinical laboratory setting and discuss the associated challenges and opportunities. In the field of clinical chemistry, the reviewed applications of ML algorithms include quality review of lab results, automated urine sediment analysis, disease or outcome prediction from routine laboratory parameters, and interpretation of complex biochemical data. In the hematology subdiscipline, we discuss the concepts of automated blood film reporting and malaria diagnosis. At last, we handle a broad range of clinical microbiology applications, such as the reduction of diagnostic workload by laboratory automation, the detection and identification of clinically relevant microorganisms, and the detection of antimicrobial resistance.

A Deep Learning Approach for Segmentation of Red Blood Cell Images and Malaria Detection

Malaria is an endemic life-threating disease caused by the unicellular protozoan parasites of the genus Plasmodium. Confirming the presence of parasites early in all malaria cases ensures species-specific antimalarial treatment, reducing the mortality rate, and points to other illnesses in negative cases. However, the gold standard remains the light microscopy of May-Grünwald–Giemsa (MGG)-stained thin and thick peripheral blood (PB) films. This is a time-consuming procedure, dependent on a pathologist’s skills, meaning that healthcare providers may encounter difficulty in diagnosing malaria in places where it is not endemic. This work presents a novel three-stage pipeline to (1) segment erythrocytes, (2) crop and mask them, and (3) classify them into malaria infected or not. The first and third steps involved the design, training, validation and testing of a Segmentation Neural Network and a Convolutional Neural Network from scratch using a Graphic Processing Unit. Segmentation achieved a global accuracy of 93.72% over the test set and the specificity for malaria detection in red blood cells (RBCs) was 87.04%. This work shows the potential that deep learning has in the digital pathology field and opens the way for future improvements, as well as for broadening the use of the created networks.