Learning-based screening of hematologic disorders using quantitative phase imaging of individual red blood cells

Label-free three-dimensional imaging and quantitative analysis of living fibroblasts and myofibroblasts by holotomographic microscopy

Holotomography (HT) is a cutting-edge fast live-cell quantitative label-free imaging technique. Based on the principle of quantitative phase imaging, it combines holography and tomography to record a three-dimensional map of the refractive index, used as intrinsic optical and quantitative imaging contrast parameter of biological samples, at a sub-micrometer spatial resolution. In this study HT has been employed for the first time to analyze the changes of fibroblasts differentiating towards myofibroblasts - recognized as the main cell player of fibrosis - when cultured in vitro with the pro-fibrotic factor, namely transforming growth factor-β1. In parallel, F-actin, vinculin, α-smooth muscle actin, phospho-myosin light chain 2, type-1 collagen, peroxisome proliferator-activated receptor-gamma coactivator-1α expression and mitochondria were evaluated by confocal laser scanning microscopy. Plasmamembrane passive properties and transient receptor potential canonical channels' currents were also recorded by whole-cell patch-clamp. The fluorescence images and electrophysiological results have been compared to the data obtained by HT and their congruence has been discussed. HT turned out to be a valid approach to morphologically distinguish fibroblasts from well differentiated myofibroblasts while obtaining objective measures concerning volume, surface area, projection area, surface index and dry mass (i.e., the mass of the non-aqueous content inside the cell including proteins and subcellular organelles) of the entire cell, nuclei and nucleoli with the major advantage to monitor outer and inner features in living cells in a non-invasive, rapid and label-free approach. HT might open up new research opportunities in the field of fibrotic diseases. RESEARCH HIGHLIGHTS: Holotomography (HT) is a label-free laser interferometric imaging technology exploiting the intrinsic optical property of cells namely refractive index (RI) to enable a direct imaging and analysis of whole cells or intracellular organelles. HT turned out a valid approach to distinguish morphological features of living unlabeled fibroblasts from differentiated myofibroblasts. HT provided quantitative information concerning volume, surface area, projection area, surface index and dry mass of the entire fibroblasts/myofibroblasts, nuclei and nucleoli.

Fourier space aberration correction for high resolution refractive index imaging using incoherent light

An aberration correction method is introduced for 3D phase deconvolution microscopy. Our technique capitalizes on multiple illumination patterns to iteratively extract Fourier space aberrations, utilizing the overlapping information inherent in these patterns. By refining the point spread function based on the retrieved aberration data, we significantly improve the precision of refractive index deconvolution. We validate the effectiveness of our method on both synthetic and biological three-dimensional samples, achieving notable enhancements in resolution and measurement accuracy. The method's reliability in aberration retrieval is further confirmed through controlled experiments with intentionally induced spherical aberrations, underscoring its potential for wide-ranging applications in microscopy and biomedicine.

The virtual staining method by quantitative phase imaging for label free lymphocytes based on self-supervised iteration cycle-consistent adversarial networks

Quantitative phase imaging (QPI) provides 3D structural and morphological information for label free living cells. Unfortunately, this quantitative phase information cannot meet doctors' diagnostic requirements of the clinical "gold standard," which displays stained cells' pathological states based on 2D color features. To make QPI results satisfy the clinical "gold standard," the virtual staining method by QPI for label free lymphocytes based on self-supervised iteration Cycle-Consistent Adversarial Networks (CycleGANs) is proposed herein. The 3D phase information of QPI is, therefore, trained and transferred to a kind of 2D "virtual staining" image that is well in agreement with "gold standard" results. To solve the problem that unstained QPI and stained "gold standard" results cannot be obtained for the same label free living cell, the self-supervised iteration for the CycleGAN deep learning algorithm is designed to obtain a trained stained result as the ground truth for error evaluation. The structural similarity index of our virtual staining experimental results for 8756 lymphocytes is 0.86. Lymphocytes' area errors after converting to 2D virtual stained results from 3D phase information are less than 3.59%. The mean error of the nuclear to cytoplasmic ratio is 2.69%, and the color deviation from the "gold standard" is less than 6.67%.

Aluminum enhances the oxidative damage of ZnO NMs in the human neuroblastoma SH-SY5Y cell line

Bare and doped zinc oxide nanomaterials (ZnO NMs) are of great interest as multifunctional platforms for biomedical applications. In this study, we systematically investigate the physicochemical properties of Aluminum doped ZnO (AZO) and its bio-interactions with neuroblastoma (SH-SY5Y) and red blood (RBCs) cells. We provide a comprehensive chemical and structural characterization of the NMs. We also evaluated the biocompatibility of AZO NMs using traditional toxicity assays and advanced microscopy techniques. The toxicity of AZO NMs towards SH-SY5Y cells, decreases as a function of Al doping but is higher than the toxicity of ZnO NMs. Our results show that N-acetyl cysteine protects SH-SY5Y cells against reactive oxygen species toxicity induced by AZO NMs. ZnO and AZO NMs do not exert hemolysis in human RBCs at the doses that cause toxicity (IC50) in neuroblastoma cells. The Atomic force microscopy qualitative analysis of the interaction of SH-SY5Y cells with AZO NMs shows evidence that the affinity of the materials with the cells results in morphology changes and diminished interactions between neighboring cells. The holotomographic microscopy analysis demonstrates NMs' internalization in SH-SY5Y cells, changes in their chemical composition, and the role of lipid droplets in the clearance of toxicants.

Label-free cell classification in holographic flow cytometry through an unbiased learning strategy

Nowadays, label-free imaging flow cytometry at the single-cell level is considered the stepforward lab-on-a-chip technology to address challenges in clinical diagnostics, biology, life sciences and healthcare. In this framework, digital holography in microscopy promises to be a powerful imaging modality thanks to its multi-refocusing and label-free quantitative phase imaging capabilities, along with the encoding of the highest information content within the imaged samples. Moreover, the recent achievements of new data analysis tools for cell classification based on deep/machine learning, combined with holographic imaging, are urging these systems toward the effective implementation of point of care devices. However, the generalization capabilities of learning-based models may be limited from biases caused by data obtained from other holographic imaging settings and/or different processing approaches. In this paper, we propose a combination of a Mask R-CNN to detect the cells, a convolutional auto-encoder, used to the image feature extraction and operating on unlabelled data, thus overcoming the bias due to data coming from different experimental settings, and a feedforward neural network for single cell classification, that operates on the above extracted features. We demonstrate the proposed approach in the challenging classification task related to the identification of drug-resistant endometrial cancer cells.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Artificial intelligence-enabled quantitative phase imaging methods for life sciences

Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.

Deep learning-based label-free hematology analysis framework using optical diffraction tomography

Hematology analysis, a common clinical test for screening various diseases, has conventionally required a chemical staining process that is time-consuming and labor-intensive. To reduce the costs of chemical staining, label-free imaging can be utilized in hematology analysis. In this work, we exploit optical diffraction tomography and the fully convolutional one-stage object detector or FCOS, a deep learning architecture for object detection, to develop a label-free hematology analysis framework. Detected cells are classified into four groups: red blood cell, abnormal red blood cell, platelet, and white blood cell. In the results, the trained object detection model showed superior detection performance for blood cells in refractive index tomograms (0.977 mAP) and also showed high accuracy in the four-class classification of blood cells (0.9708 weighted F1 score, 0.9712 total accuracy). For further verification, mean corpuscular volume (MCV) and mean corpuscular hemoglobin (MCH) were compared with values obtained from reference hematology equipment, with our results showing reasonable correlation in both MCV (0.905) and MCH (0.889). This study provides a successful demonstration of the proposed framework in detecting and classifying blood cells using optical diffraction tomography for label-free hematology analysis.

Rapid and stain-free quantification of viral plaque via lens-free holography and deep learning

A plaque assay-the gold-standard method for measuring the concentration of replication-competent lytic virions-requires staining and usually more than 48 h of runtime. Here we show that lens-free holographic imaging and deep learning can be combined to expedite and automate the assay. The compact imaging device captures phase information label-free at a rate of approximately 0.32 gigapixels per hour per well, covers an area of about 30 × 30 mm2 and a 10-fold larger dynamic range of virus concentration than standard assays, and quantifies the infected area and the number of plaque-forming units. For the vesicular stomatitis virus, the automated plaque assay detected the first cell-lysing events caused by viral replication as early as 5 h after incubation, and in less than 20 h it detected plaque-forming units at rates higher than 90% at 100% specificity. Furthermore, it reduced the incubation time of the herpes simplex virus type 1 by about 48 h and that of the encephalomyocarditis virus by about 20 h. The stain-free assay should be amenable for use in virology research, vaccine development and clinical diagnosis.

Single-shot refractive index slice imaging using spectrally multiplexed optical transfer function reshaping

The refractive index (RI) of cells and tissues is crucial in pathophysiology as a noninvasive and quantitative imaging contrast. Although its measurements have been demonstrated using three-dimensional quantitative phase imaging methods, these methods often require bulky interferometric setups or multiple measurements, which limits the measurement sensitivity and speed. Here, we present a single-shot RI imaging method that visualizes the RI of the in-focus region of a sample. By exploiting spectral multiplexing and optical transfer function engineering, three color-coded intensity images of a sample with three optimized illuminations were simultaneously obtained in a single-shot measurement. The measured intensity images were then deconvoluted to obtain the RI image of the in-focus slice of the sample. As a proof of concept, a setup was built using Fresnel lenses and a liquid-crystal display. For validation purposes, we measured microspheres of known RI and cross-validated the results with simulated results. Various static and highly dynamic biological cells were imaged to demonstrate that the proposed method can conduct single-shot RI slice imaging of biological samples with subcellular resolution.

Label-free liquid biopsy through the identification of tumor cells by machine learning-powered tomographic phase imaging flow cytometry

Image-based identification of circulating tumor cells in microfluidic cytometry condition is one of the most challenging perspectives in the Liquid Biopsy scenario. Here we show a machine learning-powered tomographic phase imaging flow cytometry system capable to provide high-throughput 3D phase-contrast tomograms of each single cell. In fact, we show that discrimination of tumor cells against white blood cells is potentially achievable with the aid of artificial intelligence in a label-free flow-cyto-tomography method. We propose a hierarchical machine learning decision-maker, working on a set of features calculated from the 3D tomograms of the cells' refractive index. We prove that 3D morphological features are adequately distinctive to identify tumor cells versus the white blood cell background in the first stage and, moreover, in recognizing the tumor type at the second decision step. Proof-of-concept experiments are shown, in which two different tumor cell lines, namely neuroblastoma cancer cells and ovarian cancer cells, are used against monocytes. The reported results allow claiming the identification of tumor cells with a success rate higher than 97% and with an accuracy over 97% in discriminating between the two cancer cell types, thus opening in a near future the route to a new Liquid Biopsy tool for detecting and classifying circulating tumor cells in blood by stain-free method.

Long-term label-free assessments of individual bacteria using three-dimensional quantitative phase imaging and hydrogel-based immobilization

Three-dimensional (3D) quantitative phase imaging (QPI) enables long-term label-free tomographic imaging and quantitative analysis of live individual bacteria. However, the Brownian motion or motility of bacteria in a liquid medium produces motion artifacts during 3D measurements and hinders precise cell imaging and analysis. Meanwhile, existing cell immobilization methods produce noisy backgrounds and even alter cellular physiology. Here, we introduce a protocol that utilizes hydrogels for high-quality 3D QPI of live bacteria maintaining bacterial physiology. We demonstrate long-term high-resolution quantitative imaging and analysis of individual bacteria, including measuring the biophysical parameters of bacteria and responses to antibiotic treatments.

Stain-free identification of cell nuclei using tomographic phase microscopy in flow cytometry

Quantitative Phase Imaging (QPI) has gained popularity in bioimaging because it can avoid the need for cell staining, which in some cases is difficult or impossible. However, as a result, QPI does not provide labelling of various specific intracellular structures. Here we show a novel computational segmentation method based on statistical inference that makes it possible for QPI techniques to identify the cell nucleus. We demonstrate the approach with refractive index tomograms of stain-free cells reconstructed through the tomographic phase microscopy in flow cytometry mode. In particular, by means of numerical simulations and two cancer cell lines, we demonstrate that the nucleus can be accurately distinguished within the stain-free tomograms. We show that our experimental results are consistent with confocal fluorescence microscopy (FM) data and microfluidic cytofluorimeter outputs. This is a significant step towards extracting specific three-dimensional intracellular structures directly from the phase-contrast data in a typical flow cytometry configuration.

Quantitative Phase Imaging: Recent Advances and Expanding Potential in Biomedicine

Quantitative phase imaging (QPI) is a label-free, wide-field microscopy approach with significant opportunities for biomedical applications. QPI uses the natural phase shift of light as it passes through a transparent object, such as a mammalian cell, to quantify biomass distribution and spatial and temporal changes in biomass. Reported in cell studies more than 60 years ago, ongoing advances in QPI hardware and software are leading to numerous applications in biology, with a dramatic expansion in utility over the past two decades. Today, investigations of cell size, morphology, behavior, cellular viscoelasticity, drug efficacy, biomass accumulation and turnover, and transport mechanics are supporting studies of development, physiology, neural activity, cancer, and additional physiological processes and diseases. Here, we review the field of QPI in biology starting with underlying principles, followed by a discussion of technical approaches currently available or being developed, and end with an examination of the breadth of applications in use or under development. We comment on strengths and shortcomings for the deployment of QPI in key biomedical contexts and conclude with emerging challenges and opportunities based on combining QPI with other methodologies that expand the scope and utility of QPI even further.

Quantitative phase imaging through an ultra-thin lensless fiber endoscope

Quantitative phase imaging (QPI) is a label-free technique providing both morphology and quantitative biophysical information in biomedicine. However, applying such a powerful technique to in vivo pathological diagnosis remains challenging. Multi-core fiber bundles (MCFs) enable ultra-thin probes for in vivo imaging, but current MCF imaging techniques are limited to amplitude imaging modalities. We demonstrate a computational lensless microendoscope that uses an ultra-thin bare MCF to perform quantitative phase imaging with microscale lateral resolution and nanoscale axial sensitivity of the optical path length. The incident complex light field at the measurement side is precisely reconstructed from the far-field speckle pattern at the detection side, enabling digital refocusing in a multi-layer sample without any mechanical movement. The accuracy of the quantitative phase reconstruction is validated by imaging the phase target and hydrogel beads through the MCF. With the proposed imaging modality, three-dimensional imaging of human cancer cells is achieved through the ultra-thin fiber endoscope, promising widespread clinical applications.

Virtual Staining, Segmentation, and Classification of Blood Smears for Label-Free Hematology Analysis

Objective and Impact Statement. We present a fully automated hematological analysis framework based on single-channel (single-wavelength), label-free deep-ultraviolet (UV) microscopy that serves as a fast, cost-effective alternative to conventional hematology analyzers. Introduction. Hematological analysis is essential for the diagnosis and monitoring of several diseases but requires complex systems operated by trained personnel, costly chemical reagents, and lengthy protocols. Label-free techniques eliminate the need for staining or additional preprocessing and can lead to faster analysis and a simpler workflow. In this work, we leverage the unique capabilities of deep-UV microscopy as a label-free, molecular imaging technique to develop a deep learning-based pipeline that enables virtual staining, segmentation, classification, and counting of white blood cells (WBCs) in single-channel images of peripheral blood smears. Methods. We train independent deep networks to virtually stain and segment grayscale images of smears. The segmented images are then used to train a classifier to yield a quantitative five-part WBC differential. Results. Our virtual staining scheme accurately recapitulates the appearance of cells under conventional Giemsa staining, the gold standard in hematology. The trained cellular and nuclear segmentation networks achieve high accuracy, and the classifier can achieve a quantitative five-part differential on unseen test data. Conclusion. This proposed automated hematology analysis framework could greatly simplify and improve current complete blood count and blood smear analysis and lead to the development of a simple, fast, and low-cost, point-of-care hematology analyzer.

Lensless light intensity model for quasi-spherical cell size measurement

Quasi-spherical cell size measurement plays an important role in medical test. Traditional methods such as a microscope and a flow cytometer are either it depends on professionals and cannot be automated, or it is expensive and bulky, which are not suitable for point-of-care test. Lab-on-a-chip technology using the lensless imaging system gives a good solution for obtaining the quasi-spherical cell size. The diffraction effects and the low resolution are the two main problems faced by the lensless imaging system. In this paper, a lensless light intensity model for the quasi-spherical cell size measurement is given. First, the diffraction characteristics of a quasi-spherical cell edge are given. Then, a diffraction model at an arc edge is constructed based on the Fresnel diffraction at a straight edge. Using the diffraction model at an arc edge, we explained the mechanism of the formation of the quasi-spherical cell diffraction fringes. Finally, the light intensity of the first bright ring of the quasi-spherical cell diffraction pattern is used to achieve quasi-spherical cell size measurement. The required equipment and the measurement methods are extremely simple, very suitable for point-of-care test. The experimental results show that the proposed model can realize the statistical measurement of the quasi-spherical cells and the classification of the quasi-spherical cells with a difference of 1 [Formula: see text].

Blood Cell Analysis: From Traditional Methods to Super-Resolution Microscopy

Blood cell analysis is essential for the diagnosis and identification of hematological malignancies. The use of digital microscopy systems has been extended in clinical laboratories. Super-resolution microscopy (SRM) has attracted wide attention in the medical field due to its nanoscale spatial resolution and high sensitivity. It is considered to be a potential method of blood cell analysis that may have more advantages than traditional approaches such as conventional optical microscopy and hematology analyzers in certain examination projects. In this review, we firstly summarize several common blood cell analysis technologies in the clinic, and analyze the advantages and disadvantages of these technologies. Then, we focus on the basic principles and characteristics of three representative SRM techniques, as well as the latest advances in these techniques for blood cell analysis. Finally, we discuss the developmental trend and possible research directions of SRM, and provide some discussions on further development of technologies for blood cell analysis.

Deep learning-based image processing in optical microscopy

Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places.

Graphical abstract

Measurement for the Area of Red Blood Cells From Microscopic Images Based on Image Processing Technology and Its Applications in Aplastic Anemia, Megaloblastic Anemia, and Myelodysplastic Syndrome

Background

Aplastic anemia (AA), megaloblastic anemia (MA), and myelodysplastic syndrome (MDS) were common anemic diseases. Sometimes it was difficult to distinguish patients with these diseases.

Methods

In this article, we proposed one measurement method for the area of red blood cells (RBCs) from microscopic images based on image processing technology and analyzed the differences of the area in 25 patients with AA, 64 patients with MA, and 68 patients with MDS.

Results

The area of RBCs was 44.19 ± 3.88, 42.09 ± 5.35, 52.87 ± 7.68, and 45.75 ± 8.07 μm2 in normal subjects, patients with AA, MA, and MDS, respectively. The coefficients of variation were 8.78%, 10.05%, 14.53%, and 14.00%, respectively, in these groups. The area of RBCs in patients with MA was significantly higher than normal subjects (p < 0.001). Compared with patients with AA and MDS, the area of RBCs in patients with MA was also significantly higher (p < 0.001). The results of correlation analysis between the area of RBCs and mean corpuscular volume (MCV), mean corpuscular hemoglobin (MCH), MCH concentration (MCHC), and red cell distribution width showed no significant correlations (p > 0.05). The area under the curve (AUC) results of the Receiver Operating Characteristic (ROC) curves of RBCs area were 0.421, 0.580, and 0.850, respectively, in patients with AA (p = 0.337), MDS (p = 0.237), and MA (p < 0.001).

Conclusion

Identifying the area of RBCs in peripheral blood smears based on the image processing technology could achieve rapid and efficient diagnostic support for patients with MDS and MA, especially for patients with MA and in combination with MCV. However, a larger sample study is needed to find the cutoff area values.

Differential diagnosis of hereditary anemias from a fraction of blood drop by digital holography and hierarchical machine learning

Anemia affects about the 25% of the global population and can provoke severe diseases, ranging from weakness and dizziness to pregnancy problems, arrhythmias and hearth failures. About 10% of the patients are affected by rare anemias of which 80% are hereditary. Early differential diagnosis of anemia enables prescribing patients a proper treatment and diet, which is effective to mitigate the associated symptoms. Nevertheless, the differential diagnosis of these conditions is often difficult due to shared and overlapping phenotypes. Indeed, the complete blood count and unaided peripheral blood smear observation cannot always provide a reliable differential diagnosis, so that biomedical assays and genetic tests are needed. These procedures are not error-free, require skilled personnel, and severely impact the financial resources of national health systems. Here we show a differential screening system for hereditary anemias that relies on holographic imaging and artificial intelligence. Label-free holographic imaging is aided by a hierarchical machine learning decider that works even in the presence of a very limited dataset but is enough accurate for discerning between different anemia classes with minimal morphological dissimilarities. It is worth to notice that only a few tens of cells from each patient are sufficient to obtain a correct diagnosis, with the advantage of significantly limiting the volume of blood drawn. This work paves the way to a wider use of home screening systems for point of care blood testing and telemedicine with lab-on-chip platforms.

Adaptive Parameter Model for Quasi-Spherical Cell Size Measurement Based on Lensless Imaging System

Many biological cells appear quasi-spherical, such as red blood cells, white blood cells, egg cells, cancer cells, etc. Cell size is an important basis for medical diagnosis. The traditional method is to use a microscope or flow cytometer to obtain the cell size. Either it depends on professionals and cannot be automated, or it is expensive and bulky, which are not suitable for point-of-care test. Lab-on-a-chip technology using a lensless imaging system gives a better solution for obtaining the cell size. In order to deal with the diffraction in the lensless imaging system, the distance between the light source and the cell, the distance between the cell and the CMOS image sensor and optical wavelength need to be accurately measured or controlled, which will greatly increase the complexity of the system, making it difficult to truly apply to point-of-care test. In this paper, an adaptive parameter model for quasi-spherical cell size measurement based on lensless imaging system is given. First, the diffraction theory used in the model is explained. Then, the adaptive algorithm of the system parameter is given. To illustrate the practicality of the algorithm, a quasi-spherical cell size measurement method and a super-resolution algorithm are given. Finally, the experiment proves that the adaptive parameter model is effective can meet the needs of quasi-spherical cell size measurement.

Maximizing content across scales: Moving multimodal microscopy and mesoscopy toward molecular imaging

Molecular imaging aims to depict the molecules in living patients. However, because this aim is still far beyond reach, patchworks of different solutions need to be used to tackle this overarching goal. From the vast toolbox of imaging techniques, we focus on those recent advances in optical microscopy that image molecules and cells at the submicron to centimeter scale. Mesoscopic imaging covers the "imaging gap" between techniques such as confocal microscopy and magnetic resonance imagingthat image entire live samples but with limited resolution. Microscopy focuses on the cellular level; mesoscopy visualizes the organization of molecules and cells into tissues and organs. The correlation between these techniques allows us to combine disciplines ranging from whole body imaging to basic research of model systems. We review current developments focused on improving microscopic and mesoscopic imaging technologies and on hardware and software that push the current sensitivity and resolution boundaries.

Rare bioparticle detection via deep metric learning

Recent deep neural networks have shown superb performance in analyzing bioimages for disease diagnosis and bioparticle classification. Conventional deep neural networks use simple classifiers such as SoftMax to obtain highly accurate results. However, they have limitations in many practical applications that require both low false alarm rate and high recovery rate, e.g., rare bioparticle detection, in which the representative image data is hard to collect, the training data is imbalanced, and the input images in inference time could be different from the training images. Deep metric learning offers a better generatability by using distance information to model the similarity of the images and learning function maps from image pixels to a latent space, playing a vital role in rare object detection. In this paper, we propose a robust model based on a deep metric neural network for rare bioparticle (Cryptosporidium or Giardia) detection in drinking water. Experimental results showed that the deep metric neural network achieved a high accuracy of 99.86% in classification, 98.89% in precision rate, 99.16% in recall rate and zero false alarm rate. The reported model empowers imaging flow cytometry with capabilities of biomedical diagnosis, environmental monitoring, and other biosensing applications.

Red blood cell recognition and posture estimation in microfluidic chip based on lensless imaging

On the one hand, lensless imaging technology has become one of the key technologies to achieve point-of-care testing; on the other hand, microfluidic technology has shown great application potential in the field of biological detection. Using mainstream lensless imaging technology to achieve biological cell imaging in microfluidic chips has technical limitations. In particular, it is more difficult to achieve lensless imaging for non-spherical cells in microfluidic chips such as red blood cells. Achieving red blood cell recognition and posture estimation in a microfluidic chip under the lensless imaging, combined with mainstream lensless imaging technology, can provide more effective red blood cell morphological parameters for medical diagnosis. In this paper, the method for red blood cell recognition and posture estimation in microfluidic chips based on lensless imaging is given. First, the relevant theoretical basis is introduced. Then, the models of red blood cell recognition and posture estimation in microfluidic chips based on lensless imaging are given. The effect of red blood cell flipping on lensless imaging is analyzed in the modeling process. Finally, the effectiveness of the proposed method is verified by experiments. Experiments show that the proposed method can well achieve red blood cell recognition and posture estimation through the shape characteristics of red blood cells.

Deep learning classification of lipid droplets in quantitative phase images

We report the application of supervised machine learning to the automated classification of lipid droplets in label-free, quantitative-phase images. By comparing various machine learning methods commonly used in biomedical imaging and remote sensing, we found convolutional neural networks to outperform others, both quantitatively and qualitatively. We describe our imaging approach, all implemented machine learning methods, and their performance with respect to computational efficiency, required training resources, and relative method performance measured across multiple metrics. Overall, our results indicate that quantitative-phase imaging coupled to machine learning enables accurate lipid droplet classification in single living cells. As such, the present paradigm presents an excellent alternative of the more common fluorescent and Raman imaging modalities by enabling label-free, ultra-low phototoxicity, and deeper insight into the thermodynamics of metabolism of single cells.

Machine-learning-based classification of Stokes-Mueller polarization images for tissue characterization

The microstructural analysis of tissues plays a crucial role in the early detection of abnormal tissue morphology. Polarization microscopy, an optical tool for studying the anisotropic properties of biomolecules, can distinguish normal and malignant tissue features even in the absence of exogenous labelling. To facilitate the quantitative analysis, we developed a polarization-sensitive label-free imaging system based on the Stokes-Mueller calculus. Polarization images of ductal carcinoma tissue samples were obtained using various input polarization states and Stokes-Mueller images were reconstructed using Matlab software. Further, polarization properties, such as degree of linear and circular polarization and anisotropy, were reconstructed from the Stokes images. The Mueller matrix obtained was decomposed using the Lu-Chipman decomposition method to acquire the individual polarization properties of the sample, such as depolarization, diattenuation and retardance. By using the statistical parameters obtained from the polarization images, a support vector machine (SVM) algorithm was trained to facilitate the tissue classification associated with its pathological condition.

Holotomography: Refractive Index as an Intrinsic Imaging Contrast for 3-D Label-Free Live Cell Imaging

Live cell imaging provides essential information in the investigation of cell biology and related pathophysiology. Refractive index (RI) can serve as intrinsic optical imaging contrast for 3-D label-free and quantitative live cell imaging, and provide invaluable information to understand various dynamics of cells and tissues for the study of numerous fields. Recently significant advances have been made in imaging methods and analysis approaches utilizing RI, which are now being transferred to biological and medical research fields, providing novel approaches to investigate the pathophysiology of cells. To provide insight into how RI can be used as an imaging contrast for imaging of biological specimens, here we provide the basic principle of RI-based imaging techniques and summarize recent progress on applications, ranging from microbiology, hematology, infectious diseases, hematology, and histopathology.

Automatic detection and characterization of quantitative phase images of thalassemic red blood cells using a mask region-based convolutional neural network

SIGNIFICANCE

Label-free quantitative phase imaging is a promising technique for the automatic detection of abnormal red blood cells (RBCs) in real time. Although deep-learning techniques can accurately detect abnormal RBCs from quantitative phase images efficiently, their applications in diagnostic testing are limited by the lack of transparency. More interpretable results such as morphological and biochemical characteristics of individual RBCs are highly desirable.

AIM

An end-to-end deep-learning model was developed to efficiently discriminate thalassemic RBCs (tRBCs) from healthy RBCs (hRBCs) in quantitative phase images and segment RBCs for single-cell characterization.

APPROACH

Two-dimensional quantitative phase images of hRBCs and tRBCs were acquired using digital holographic microscopy. A mask region-based convolutional neural network (Mask R-CNN) model was trained to discriminate tRBCs and segment individual RBCs. Characterization of tRBCs was achieved utilizing SHapley Additive exPlanation analysis and canonical correlation analysis on automatically segmented RBC phase images.

RESULTS

The implemented model achieved 97.8% accuracy in detecting tRBCs. Phase-shift statistics showed the highest influence on the correct classification of tRBCs. Associations between the phase-shift features and three-dimensional morphological features were revealed.

CONCLUSIONS

The implemented Mask R-CNN model accurately identified tRBCs and segmented RBCs to provide single-RBC characterization, which has the potential to aid clinical decision-making.

Label-Free Tomographic Imaging of Lipid Droplets in Foam Cells for Machine-Learning-Assisted Therapeutic Evaluation of Targeted Nanodrugs

Lipid droplet (LD) accumulation, a key feature of foam cells, constitutes an attractive target for therapeutic intervention in atherosclerosis. However, despite advances in cellular imaging techniques, current noninvasive and quantitative methods have limited application in living foam cells. Here, using optical diffraction tomography (ODT), we performed quantitative morphological and biophysical analysis of living foam cells in a label-free manner. We identified LDs in foam cells by verifying the specific refractive index using correlative imaging comprising ODT integrated with three-dimensional fluorescence imaging. Through time-lapse monitoring of three-dimensional dynamics of label-free living foam cells, we precisely and quantitatively evaluated the therapeutic effects of a nanodrug (mannose-polyethylene glycol-glycol chitosan-fluorescein isothiocyanate-lobeglitazone; MMR-Lobe) designed to affect the targeted delivery of lobeglitazone to foam cells based on high mannose receptor specificity. Furthermore, by exploiting machine-learning-based image analysis, we further demonstrated therapeutic evaluation at the single-cell level. These findings suggest that refractive index measurement is a promising tool to explore new drugs against LD-related metabolic diseases.

Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging

SIGNIFICANCE

We introduce an application of machine learning trained on optical phase features of epithelial and mesenchymal cells to grade cancer cells' morphologies, relevant to evaluation of cancer phenotype in screening assays and clinical biopsies.

AIM

Our objective was to determine quantitative epithelial and mesenchymal qualities of breast cancer cells through an unbiased, generalizable, and linear score covering the range of observed morphologies.

APPROACH

Digital holographic microscopy was used to generate phase height maps of noncancerous epithelial (Gie-No3B11) and fibroblast (human gingival) cell lines, as well as MDA-MB-231 and MCF-7 breast cancer cell lines. Several machine learning algorithms were evaluated as binary classifiers of the noncancerous cells that graded the cancer cells by transfer learning.

RESULTS

Epithelial and mesenchymal cells were classified with 96% to 100% accuracy. Breast cancer cells had scores in between the noncancer scores, indicating both epithelial and mesenchymal morphological qualities. The MCF-7 cells skewed toward epithelial scores, while MDA-MB-231 cells skewed toward mesenchymal scores. Linear support vector machines (SVMs) produced the most distinct score distributions for each cell line.

CONCLUSIONS

The proposed epithelial-mesenchymal score, derived from linear SVM learning, is a sensitive and quantitative approach for detecting epithelial and mesenchymal characteristics of unknown cells based on well-characterized cell lines. We establish a framework for rapid and accurate morphological evaluation of single cells and subtle phenotypic shifts in imaged cell populations.

Deep learning-based optical field screening for robust optical diffraction tomography

In tomographic reconstruction, the image quality of the reconstructed images can be significantly degraded by defects in the measured two-dimensional (2D) raw image data. Despite the importance of screening defective 2D images for robust tomographic reconstruction, manual inspection and rule-based automation suffer from low-throughput and insufficient accuracy, respectively. Here, we present deep learning-enabled quality control for holographic data to produce robust and high-throughput optical diffraction tomography (ODT). The key idea is to distil the knowledge of an expert into a deep convolutional neural network. We built an extensive database of optical field images with clean/noisy annotations, and then trained a binary-classification network based upon the data. The trained network outperformed visual inspection by non-expert users and a widely used rule-based algorithm, with >90% test accuracy. Subsequently, we confirmed that the superior screening performance significantly improved the tomogram quality. To further confirm the trained model’s performance and generalisability, we evaluated it on unseen biological cell data obtained with a setup that was not used to generate the training dataset. Lastly, we interpreted the trained model using various visualisation techniques that provided the saliency map underlying each model inference. We envision the proposed network would a powerful lightweight module in the tomographic reconstruction pipeline.

Epi-illumination gradient light interference microscopy for imaging opaque structures

Multiple scattering and absorption limit the depth at which biological tissues can be imaged with light. In thick unlabeled specimens, multiple scattering randomizes the phase of the field and absorption attenuates light that travels long optical paths. These obstacles limit the performance of transmission imaging. To mitigate these challenges, we developed an epi-illumination gradient light interference microscope (epi-GLIM) as a label-free phase imaging modality applicable to bulk or opaque samples. Epi-GLIM enables studying turbid structures that are hundreds of microns thick and otherwise opaque to transmitted light. We demonstrate this approach with a variety of man-made and biological samples that are incompatible with imaging in a transmission geometry: semiconductors wafers, specimens on opaque and birefringent substrates, cells in microplates, and bulk tissues. We demonstrate that the epi-GLIM data can be used to solve the inverse scattering problem and reconstruct the tomography of single cells and model organisms.

3D-printable portable open-source platform for low-cost lens-less holographic cellular imaging

Digital holographic microscopy is an emerging, potentially low-cost alternative to conventional light microscopy for micro-object imaging on earth, underwater and in space. Immediate access to micron-scale objects however requires a well-balanced system design and sophisticated reconstruction algorithms, that are commercially available, however not accessible cost-efficiently. Here, we present an open-source implementation of a lens-less digital inline holographic microscope platform, based on off-the-shelf optical, electronic and mechanical components, costing less than $190. It employs a Blu-Ray semiconductor-laser-pickup or a light-emitting-diode, a pinhole, a 3D-printed housing consisting of 3 parts and a single-board portable computer and camera with an open-source implementation of the Fresnel-Kirchhoff routine. We demonstrate 1.55 μm spatial resolution by laser-pickup and 3.91 μm by the light-emitting-diode source. The housing and mechanical components are 3D printed. Both printer and reconstruction software source codes are open. The light-weight microscope allows to image label-free micro-spheres of 6.5 μm diameter, human red-blood-cells of about 8 μm diameter as well as fast-growing plant Nicotiana-tabacum-BY-2 suspension cells with 50 μm sizes. The imaging capability is validated by imaging-contrast quantification involving a standardized test target. The presented 3D-printable portable open-source platform represents a fully-open design, low-cost modular and versatile imaging-solution for use in high- and low-resource areas of the world.

Machine Learning with Optical Phase Signatures for Phenotypic Profiling of Cell Lines

Robust and reproducible profiling of cell lines is essential for phenotypic screening assays. The goals of this study were to determine robust and reproducible optical phase signatures of cell lines for classification with machine learning and to correlate optical phase parameters to motile behavior. Digital holographic microscopy (DHM) reconstructed phase maps of cells from two pairs of cancer and non-cancer cell lines. Seventeen image parameters were extracted from each cell's phase map, used for linear support vector machine learning, and correlated to scratch wound closure and Boyden chamber chemotaxis. The classification accuracy was between 90% and 100% for the six pairwise cell line comparisons. Several phase parameters correlated with wound closure rate and chemotaxis across the four cell lines. The level of cell confluence in culture affected phase parameters in all cell lines tested. Results indicate that optical phase features of cell lines are a robust set of quantitative data of potential utility for phenotypic screening and prediction of motile behavior. © 2019 International Society for Advancement of Cytometry.

Cycle-consistent deep learning approach to coherent noise reduction in optical diffraction tomography

We present a deep neural network to reduce coherent noise in three-dimensional quantitative phase imaging. Inspired by the cycle generative adversarial network, the denoising network was trained to learn a transform between two image domains: clean and noisy refractive index tomograms. The unique feature of this network, distinct from previous machine learning approaches employed in the optical imaging problem, is that it uses unpaired images. The learned network quantitatively demonstrated its performance and generalization capability through denoising experiments of various samples. We concluded by applying our technique to reduce the temporally changing noise emerging from focal drift in time-lapse imaging of biological cells. This reduction cannot be performed using other optical methods for denoising.