A mathematical framework for combining decisions of multiple experts toward accurate and remote diagnosis of malaria using tele-microscopy

Multi-Scale Convolutional Neural Network Ensemble for Multi-Class Arrhythmia Classification

The automated analysis of electrocardiogram (ECG) signals plays a crucial role in the early diagnosis and management of cardiac arrhythmias. The diverse etiology of arrhythmia and the subtle variations in the pathological ECG characteristics pose challenges in designing reliable automated methods. Existing methods mostly use single deep convolutional neural networks (DCNN) based approaches for arrhythmia classification. Such approaches may not be adequate for effectively representing diverse pathological ECG characteristics. This paper presents a novel way of using an ensemble of multiple DCNN classifiers for effective arrhythmia classification named Deep Multi-Scale Convolutional neural network Ensemble (DMSCE). Specifically, we designed multiple scale-dependent DCNN expert classifiers with different receptive fields to encode the scale-specific pathological ECG characteristics and generate the local predictions. A convolutional gating network is designed to compute the dynamic fusion weights for the experts based on their competencies. These weights are used to aggregate the local predictions and generate final diagnosis decisions. Moreover, a new error function with a correlation penalty is formulated to enable interaction and optimal diversity among experts during the training process. The model is evaluated on the PTBXL-2020 12-lead ECG and the CinC-training2017 single-lead ECG datasets and delivers state-of-the-art performance. Average F1-score of 84.5% and 88.3% are obtained for the PTBXL-2020 and the CinC-training2017 datasets, respectively. Impressive performance across various cardiac arrhythmias and the elegant generalization ability for different leads make the method suitable for reliable remote or in-hospital arrhythmia monitoring applications.

Deep Learning Reveals Cancer Metastasis and Therapeutic Antibody Targeting in the Entire Body

Reliable detection of disseminated tumor cells and of the biodistribution of tumor-targeting therapeutic antibodies within the entire body has long been needed to better understand and treat cancer metastasis. Here, we developed an integrated pipeline for automated quantification of cancer metastases and therapeutic antibody targeting, named DeepMACT. First, we enhanced the fluorescent signal of cancer cells more than 100-fold by applying the vDISCO method to image metastasis in transparent mice. Second, we developed deep learning algorithms for automated quantification of metastases with an accuracy matching human expert manual annotation. Deep learning-based quantification in 5 different metastatic cancer models including breast, lung, and pancreatic cancer with distinct organotropisms allowed us to systematically analyze features such as size, shape, spatial distribution, and the degree to which metastases are targeted by a therapeutic monoclonal antibody in entire mice. DeepMACT can thus considerably improve the discovery of effective antibody-based therapeutics at the pre-clinical stage. VIDEO ABSTRACT.

Automated screening of sickle cells using a smartphone-based microscope and deep learning

Sickle cell disease (SCD) is a major public health priority throughout much of the world, affecting millions of people. In many regions, particularly those in resource-limited settings, SCD is not consistently diagnosed. In Africa, where the majority of SCD patients reside, more than 50% of the 0.2-0.3 million children born with SCD each year will die from it; many of these deaths are in fact preventable with correct diagnosis and treatment. Here, we present a deep learning framework which can perform automatic screening of sickle cells in blood smears using a smartphone microscope. This framework uses two distinct, complementary deep neural networks. The first neural network enhances and standardizes the blood smear images captured by the smartphone microscope, spatially and spectrally matching the image quality of a laboratory-grade benchtop microscope. The second network acts on the output of the first image enhancement neural network and is used to perform the semantic segmentation between healthy and sickle cells within a blood smear. These segmented images are then used to rapidly determine the SCD diagnosis per patient. We blindly tested this mobile sickle cell detection method using blood smears from 96 unique patients (including 32 SCD patients) that were imaged by our smartphone microscope, and achieved ~98% accuracy, with an area-under-the-curve of 0.998. With its high accuracy, this mobile and cost-effective method has the potential to be used as a screening tool for SCD and other blood cell disorders in resource-limited settings.

malERA: An updated research agenda for combination interventions and modelling in malaria elimination and eradication

This paper summarises key advances and priorities since the 2011 presentation of the Malaria Eradication Research Agenda (malERA), with a focus on the combinations of intervention tools and strategies for elimination and their evaluation using modelling approaches. With an increasing number of countries embarking on malaria elimination programmes, national and local decisions to select combinations of tools and deployment strategies directed at malaria elimination must address rapidly changing transmission patterns across diverse geographic areas. However, not all of these approaches can be systematically evaluated in the field. Thus, there is potential for modelling to investigate appropriate 'packages' of combined interventions that include various forms of vector control, case management, surveillance, and population-based approaches for different settings, particularly at lower transmission levels. Modelling can help prioritise which intervention packages should be tested in field studies, suggest which intervention package should be used at a particular level or stratum of transmission intensity, estimate the risk of resurgence when scaling down specific interventions after local transmission is interrupted, and evaluate the risk and impact of parasite drug resistance and vector insecticide resistance. However, modelling intervention package deployment against a heterogeneous transmission background is a challenge. Further validation of malaria models should be pursued through an iterative process, whereby field data collected with the deployment of intervention packages is used to refine models and make them progressively more relevant for assessing and predicting elimination outcomes.

Counting malaria parasites with a two-stage EM based algorithm using crowsourced data

Malaria eradication of the worldwide is currently one of the main WHO's global goals. In this work, we focus on the use of human-machine interaction strategies for low-cost fast reliable malaria diagnostic based on a crowdsourced approach. The addressed technical problem consists in detecting spots in images even under very harsh conditions when positive objects are very similar to some artifacts. The clicks or tags delivered by several annotators labeling an image are modeled as a robust finite mixture, and techniques based on the Expectation-Maximization (EM) algorithm are proposed for accurately counting malaria parasites on thick blood smears obtained by microscopic Giemsa-stained techniques. This approach outperforms other traditional methods as it is shown through experimentation with real data.

Applying Faster R-CNN for Object Detection on Malaria Images

Deep learning based models have had great success in object detection, but the state of the art models have not yet been widely applied to biological image data. We apply for the first time an object detection model previously used on natural images to identify cells and recognize their stages in brightfield microscopy images of malaria-infected blood. Many micro-organisms like malaria parasites are still studied by expert manual inspection and hand counting. This type of object detection task is challenging due to factors like variations in cell shape, density, and color, and uncertainty of some cell classes. In addition, annotated data useful for training is scarce, and the class distribution is inherently highly imbalanced due to the dominance of uninfected red blood cells. We use Faster Region-based Convolutional Neural Network (Faster R-CNN), one of the top performing object detection models in recent years, pre-trained on ImageNet but fine tuned with our data, and compare it to a baseline, which is based on a traditional approach consisting of cell segmentation, extraction of several single-cell features, and classification using random forests. To conduct our initial study, we collect and label a dataset of 1300 fields of view consisting of around 100,000 individual cells. We demonstrate that Faster R-CNN outperforms our baseline and put the results in context of human performance.

A portable microfluidic system for rapid measurement of the erythrocyte sedimentation rate

The erythrocyte sedimentation rate (ESR) is a frequently used 30 min or 60 min clinical test for screening of several inflammatory conditions, infections, trauma, and malignant diseases, as well as non-inflammatory conditions including prostate cancer and stroke. Erythrocyte aggregation (EA) is a physiological process where erythrocytes form face-to-face linear structures, called rouleaux, at stasis or low shear rates. In this work, we proposed a method for ESR measurement from EA. We developed a microfluidic opto-electro-mechanical system, using which we experimentally showed a significant correlation (R² = 0.86) between ESR and EA. The microfluidic system was shown to measure ESR from EA using fingerprick blood in 2 min. 40 μl of whole blood is filled in a disposable polycarbonate cartridge which is illuminated with a near infrared emitting diode. Erythrocytes were disaggregated under the effect of a mechanical shear force using a solenoid pinch valve. Following complete disaggregation, transmitted light through the cartridge was measured using a photodetector for 1.5 min. The intensity level is at its lowest at complete disaggregation and highest at complete aggregation. We calculated ESR from the transmitted signal profile. We also developed another microfluidic cartridge specifically for monitoring the EA process in real-time during ESR measurement. The presented system is suitable for ultrafast, low-cost, and low-sample volume measurement of ESR at the point-of-care.

Malaria Diagnosis Using a Mobile Phone Polarized Microscope

Malaria remains a major global health burden, and new methods for low-cost, high-sensitivity, diagnosis are essential, particularly in remote areas with low-resource around the world. In this paper, a cost effective, optical cell-phone based transmission polarized light microscope system is presented for imaging the malaria pigment known as hemozoin. It can be difficult to determine the presence of the pigment from background and other artifacts, even for skilled microscopy technicians. The pigment is much easier to observe using polarized light microscopy. However, implementation of polarized light microscopy lacks widespread adoption because the existing commercial devices have complicated designs, require sophisticated maintenance, tend to be bulky, can be expensive, and would require re-training for existing microscopy technicians. To this end, a high fidelity and high optical resolution cell-phone based polarized light microscopy system is presented which is comparable to larger bench-top polarized microscopy systems but at much lower cost and complexity. The detection of malaria in fixed and stained blood smears is presented using both, a conventional polarized microscope and our cell-phone based system. The cell-phone based polarimetric microscopy design shows the potential to have both the resolution and specificity to detect malaria in a low-cost, easy-to-use, modular platform.

Optofluidic holographic microscopy with custom field of view (FoV) using a linear array detector

Simple and effective imaging strategies are of utmost interest for applications on a lab-on-chip scale. In fact, the majority of diagnostic tools for medical as well as biotechnological studies still employ image-based approaches. Having onboard the chip a compact but powerful imaging apparatus with multiple imaging capabilities, such as 3D dynamic focusing along the optical axis, unlimited field of view (FoV) and double outputs, namely, intensity and quantitative phase-contrast maps of biological objects, is of extreme importance for the next generation of Lab-on-a-Chip (LoC) devices. Here we present a coherent 3D microscopy approach with a holographic modality that is specifically suitable for studying biological samples while they simply flow along microfluidic paths. The LoC device is equipped with a compact linear array detector to capture and generate a new conceptual type of a digital hologram in the space-time domain, named here as Space-Time Digital Hologram (STDH). The reported results show that the method is a promising diagnostic tool for optofluidic investigations of biological specimens.

Red blood cell as an adaptive optofluidic microlens

The perspective of using live cells as lenses could open new revolutionary and intriguing scenarios in the future of biophotonics and biomedical sciences for endoscopic vision, local laser treatments via optical fibres and diagnostics. Here we show that a suspended red blood cell (RBC) behaves as an adaptive liquid-lens at microscale, thus demonstrating its imaging capability and tunable focal length. In fact, thanks to the intrinsic elastic properties, the RBC can swell up from disk volume of 90 fl up to a sphere reaching 150 fl, varying focal length from negative to positive values. These live optofluidic lenses can be fully controlled by triggering the liquid buffer's chemistry. Real-time accurate measurement of tunable focus capability of RBCs is reported through dynamic wavefront characterization, showing agreement with numerical modelling. Moreover, in analogy to adaptive optics testing, blood diagnosis is demonstrated by screening abnormal cells through focal-spot analysis applied to an RBC ensemble as a microlens array.

Stable Atlas-based Mapped Prior (STAMP) machine-learning segmentation for multicenter large-scale MRI data

Machine learning (ML)-based segmentation methods are a common technique in the medical image processing field. In spite of numerous research groups that have investigated ML-based segmentation frameworks, there remains unanswered aspects of performance variability for the choice of two key components: ML algorithm and intensity normalization. This investigation reveals that the choice of those elements plays a major part in determining segmentation accuracy and generalizability. The approach we have used in this study aims to evaluate relative benefits of the two elements within a subcortical MRI segmentation framework. Experiments were conducted to contrast eight machine-learning algorithm configurations and 11 normalization strategies for our brain MR segmentation framework. For the intensity normalization, a Stable Atlas-based Mapped Prior (STAMP) was utilized to take better account of contrast along boundaries of structures. Comparing eight machine learning algorithms on down-sampled segmentation MR data, it was obvious that a significant improvement was obtained using ensemble-based ML algorithms (i.e., random forest) or ANN algorithms. Further investigation between these two algorithms also revealed that the random forest results provided exceptionally good agreement with manual delineations by experts. Additional experiments showed that the effect of STAMP-based intensity normalization also improved the robustness of segmentation for multicenter data sets. The constructed framework obtained good multicenter reliability and was successfully applied on a large multicenter MR data set (n>3000). Less than 10% of automated segmentations were recommended for minimal expert intervention. These results demonstrate the feasibility of using the ML-based segmentation tools for processing large amount of multicenter MR images. We demonstrated dramatically different result profiles in segmentation accuracy according to the choice of ML algorithm and intensity normalization chosen.

Smart-phone based computational microscopy using multi-frame contact imaging on a fiber-optic array

We demonstrate a cellphone based contact microscopy platform, termed Contact Scope, which can image highly dense or connected samples in transmission mode. Weighing approximately 76 grams, this portable and compact microscope is installed on the existing camera unit of a cellphone using an opto-mechanical add-on, where planar samples of interest are placed in contact with the top facet of a tapered fiber-optic array. This glass-based tapered fiber array has ~9 fold higher density of fiber optic cables on its top facet compared to the bottom one and is illuminated by an incoherent light source, e.g., a simple light-emitting-diode (LED). The transmitted light pattern through the object is then sampled by this array of fiber optic cables, delivering a transmission image of the sample onto the other side of the taper, with ~3× magnification in each direction. This magnified image of the object, located at the bottom facet of the fiber array, is then projected onto the CMOS image sensor of the cellphone using two lenses. While keeping the sample and the cellphone camera at a fixed position, the fiber-optic array is then manually rotated with discrete angular increments of e.g., 1-2 degrees. At each angular position of the fiber-optic array, contact images are captured using the cellphone camera, creating a sequence of transmission images for the same sample. These multi-frame images are digitally fused together based on a shift-and-add algorithm through a custom-developed Android application running on the smart-phone, providing the final microscopic image of the sample, visualized through the screen of the phone. This final computation step improves the resolution and also removes spatial artefacts that arise due to non-uniform sampling of the transmission intensity at the fiber optic array surface. We validated the performance of this cellphone based Contact Scope by imaging resolution test charts and blood smears.