Automated red blood cells extraction from holographic images using fully convolutional neural networks

Label-Free Visualization and Morphological Profiling of Neuronal Differentiation and Axonal Degeneration through Quantitative Phase Imaging

Understanding the intricate processes of neuronal growth, degeneration, and neurotoxicity is paramount for unraveling nervous system function and holds significant promise in improving patient outcomes, especially in the context of chemotherapy-induced peripheral neuropathy (CIPN). These processes are influenced by a broad range of entwined events facilitated by chemical, electrical, and mechanical signals. The progress of each process is inherently linked to phenotypic changes in cells. Currently, the primary means of demonstrating morphological changes rely on measurements of neurite outgrowth and axon length. However, conventional techniques for monitoring these processes often require extensive preparation to enable manual or semi-automated measurements. Here, a label-free and non-invasive approach is employed for monitoring neuronal differentiation and degeneration using quantitative phase imaging (QPI). Operating on unlabeled specimens and offering little to no phototoxicity and photobleaching, QPI delivers quantitative maps of optical path length delays that provide an objective measure of cellular morphology and dynamics. This approach enables the visualization and quantification of axon length and other physical properties of dorsal root ganglion (DRG) neuronal cells, allowing greater understanding of neuronal responses to stimuli simulating CIPN conditions. This research paves new avenues for the development of more effective strategies in the clinical management of neurotoxicity.

A dataset of digital holograms of normal and thalassemic cells

Digital holographic microscopy (DHM) is an intriguing medical diagnostic tool due to its label-free and quantitative nature, providing high-contrast images of phase samples. By capturing both intensity and phase information, DHM enables the numerical reconstruction of quantitative phase images. However, the lateral resolution is limited by the diffraction limit, which prompted the recent suggestion of microsphere-assisted DHM to enhance the DHM resolution straightforwardly. The use of such a technique as a medical diagnostic tool requires testing and validation of the proposed assays to prove their feasibility and viability. This paper publishes 760 and 609 microsphere-assisted DHM images of normal and thalassemic red blood cells obtained from a normal and thalassemic male individual, respectively.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Review of research on the instance segmentation of cell images

The instance segmentation of cell images is the basis for conducting cell research and is of great importance for the study and diagnosis of pathologies. To analyze current situations and future developments in the field of cell image instance segmentation, this paper first systematically reviews image segmentation methods based on traditional and deep learning methods. Then, from the three aspects of cell image weak label extraction, cell image instance segmentation, and cell internal structure segmentation, deep-learning-based cell image segmentation methods are analyzed and summarized. Finally, cell image instance segmentation is summarized, and challenges and future developments are discussed.

Resolution and Contrast Enhancement for Lensless Digital Holographic Microscopy and Its Application in Biomedicine

An important imaging technique in biomedicine, the conventional optical microscopy relies on relatively complicated and bulky lens and alignment mechanics. Based on the Gabor holography, the lensless digital holographic microscopy has the advantages of light weight and low cost. It has developed rapidly and received attention in many fields. However, the finite pixel size at the sensor plane limits the spatial resolution. In this study, we first review the principle of lensless digital holography, then go over some methods to improve image contrast and discuss the methods to enhance the image resolution of the lensless holographic image. Moreover, the applications of lensless digital holographic microscopy in biomedicine are reviewed. Finally, we look forward to the future development and prospect of lensless digital holographic technology.

Deep learning for digital holography: a review

Recent years have witnessed the unprecedented progress of deep learning applications in digital holography (DH). Nevertheless, there remain huge potentials in how deep learning can further improve performance and enable new functionalities for DH. Here, we survey recent developments in various DH applications powered by deep learning algorithms. This article starts with a brief introduction to digital holographic imaging, then summarizes the most relevant deep learning techniques for DH, with discussions on their benefits and challenges. We then present case studies covering a wide range of problems and applications in order to highlight research achievements to date. We provide an outlook of several promising directions to widen the use of deep learning in various DH applications.

Self-supervised pretraining for transferable quantitative phase image cell segmentation

In this paper, a novel U-Net-based method for robust adherent cell segmentation for quantitative phase microscopy image is designed and optimised. We designed and evaluated four specific post-processing pipelines. To increase the transferability to different cell types, non-deep learning transfer with adjustable parameters is used in the post-processing step. Additionally, we proposed a self-supervised pretraining technique using nonlabelled data, which is trained to reconstruct multiple image distortions and improved the segmentation performance from 0.67 to 0.70 of object-wise intersection over union. Moreover, we publish a new dataset of manually labelled images suitable for this task together with the unlabelled data for self-supervised pretraining.

Deep Learning-based Phenotypic Assessment of Red Cell Storage Lesions for Safe Transfusions

This study presents a novel approach to automatically perform instant phenotypic assessment of red blood cell (RBC) storage lesion in phase images obtained by digital holographic microscopy. The proposed model combines a generative adversarial network (GAN) with marker-controlled watershed segmentation scheme. The GAN model performed RBC segmentations and classifications to develop ageing markers, and the watershed segmentation was used to completely separate overlapping RBCs. Our approach achieved good segmentation and classification accuracy with a Dices coefficient of 0.94 at a high throughput rate of about 152 cells per second. These results were compared with other deep neural network architectures. Moreover, our image-based deep learning models recognized the morphological changes that occur in RBCs during storage. Our deep learning-based classification results were in good agreement with previous findings on the changes in RBC markers (dominant shapes) affected by storage duration. We believe that our image-based deep learning models can be useful for automated assessment of RBC quality, storage lesions for safe transfusions, and diagnosis of RBC-related diseases.

Clustering-Based Dual Deep Learning Architecture for Detecting Red Blood Cells in Malaria Diagnostic Smears

Computer-assisted algorithms have become a mainstay of biomedical applications to improve accuracy and reproducibility of repetitive tasks like manual segmentation and annotation. We propose a novel pipeline for red blood cell detection and counting in thin blood smear microscopy images, named RBCNet, using a dual deep learning architecture. RBCNet consists of a U-Net first stage for cell-cluster or superpixel segmentation, followed by a second refinement stage Faster R-CNN for detecting small cell objects within the connected component clusters. RBCNet uses cell clustering instead of region proposals, which is robust to cell fragmentation, is highly scalable for detecting small objects or fine scale morphological structures in very large images, can be trained using non-overlapping tiles, and during inference is adaptive to the scale of cell-clusters with a low memory footprint. We tested our method on an archived collection of human malaria smears with nearly 200,000 labeled cells across 965 images from 193 patients, acquired in Bangladesh, with each patient contributing five images. Cell detection accuracy using RBCNet was higher than 97 %. The novel dual cascade RBCNet architecture provides more accurate cell detections because the foreground cell-cluster masks from U-Net adaptively guide the detection stage, resulting in a notably higher true positive and lower false alarm rates, compared to traditional and other deep learning methods. The RBCNet pipeline implements a crucial step towards automated malaria diagnosis.

Quantitative analysis of blood cells from microscopic images using convolutional neural network

Blood cell count provides relevant clinical information about different kinds of disorders. Any deviation in the number of blood cells implies the presence of infection, inflammation, edema, bleeding, and other blood-related issues. Current microscopic methods used for blood cell counting are very tedious and are highly prone to different sources of errors. Besides, these techniques do not provide full information related to blood cells like shape and size, which play important roles in the clinical investigation of serious blood-related diseases. In this paper, deep learning-based automatic classification and quantitative analysis of blood cells are proposed using the YOLOv2 model. The model was trained on 1560 images and 2703-labeled blood cells with different hyper-parameters. It was tested on 26 images containing 1454 red blood cells, 159 platelets, 3 basophils, 12 eosinophils, 24 lymphocytes, 13 monocytes, and 28 neutrophils. The network achieved detection and segmentation of blood cells with an average accuracy of 80.6% and a precision of 88.4%. Quantitative analysis of cells was done following classification, and mean accuracy of 92.96%, 91.96%, 88.736%, and 92.7% has been achieved in the measurement of area, aspect ratio, diameter, and counting of cells respectively.Graphical abstract Graphical abstract where the first picture shows the input image of blood cells seen under a compound light microscope. The second image shows the tools used like OpenCV to pre-process the image. The third image shows the convolutional neural network used to train and perform object detection. The 4th image shows the output of the network in the detection of blood cells. The last images indicate post-processing applied on the output image such as counting of each blood cells using the class label of each detection and quantification of morphological parameters like area, aspect ratio, and diameter of blood cells so that the final result provides the number of each blood cell types (seven) and morphological information providing valuable clinical information.

Automatic detection and characterization of quantitative phase images of thalassemic red blood cells using a mask region-based convolutional neural network

SIGNIFICANCE

Label-free quantitative phase imaging is a promising technique for the automatic detection of abnormal red blood cells (RBCs) in real time. Although deep-learning techniques can accurately detect abnormal RBCs from quantitative phase images efficiently, their applications in diagnostic testing are limited by the lack of transparency. More interpretable results such as morphological and biochemical characteristics of individual RBCs are highly desirable.

AIM

An end-to-end deep-learning model was developed to efficiently discriminate thalassemic RBCs (tRBCs) from healthy RBCs (hRBCs) in quantitative phase images and segment RBCs for single-cell characterization.

APPROACH

Two-dimensional quantitative phase images of hRBCs and tRBCs were acquired using digital holographic microscopy. A mask region-based convolutional neural network (Mask R-CNN) model was trained to discriminate tRBCs and segment individual RBCs. Characterization of tRBCs was achieved utilizing SHapley Additive exPlanation analysis and canonical correlation analysis on automatically segmented RBC phase images.

RESULTS

The implemented model achieved 97.8% accuracy in detecting tRBCs. Phase-shift statistics showed the highest influence on the correct classification of tRBCs. Associations between the phase-shift features and three-dimensional morphological features were revealed.

CONCLUSIONS

The implemented Mask R-CNN model accurately identified tRBCs and segmented RBCs to provide single-RBC characterization, which has the potential to aid clinical decision-making.

Accurate detection of small particles in digital holography using fully convolutional networks

Particle detection is a key procedure in particle field characterization with digital holography. Due to various background noises, spurious small particles might be generated and real small particles might be lost during particle detection. Therefore, accurate small particle detection remains a challenge in the research of energy and combustion. A deep learning method based on modified fully convolutional networks is proposed to detect small opaque particles (e.g., coal particles) on extended focus images. The model is tested by several experiments and proved to have good small particle detection accuracy.

No-search focus prediction at the single cell level in digital holographic imaging with deep convolutional neural network

Digital propagation of an off-axis hologram can provide the quantitative phase-contrast image if the exact distance between the sensor plane (such as CCD) and the reconstruction plane is correctly provided. In this paper, we present a deep-learning convolutional neural network with a regression layer as the top layer to estimate the best reconstruction distance. The experimental results obtained using microsphere beads and red blood cells show that the proposed method can accurately predict the propagation distance from a filtered hologram. The result is compared with the conventional automatic focus-evaluation function. Additionally, our approach can be utilized at the single-cell level, which is useful for cell-to-cell depth measurement and cell adherent studies.

Characterization of Spatial Light Modulator Based on the Phase in Fourier Domain of the Hologram and Its Applications in Coherent Imaging

Although digital holography is used widely at present, the information contained in the digital hologram is still underutilized. For example, the phase values of the Fourier spectra of the hologram are seldom used directly. In this paper, we take full advantage of them for characterizing the phase modulation of a spatial light modulator (SLM). Incident plane light beam is divided into two beams, one of which passes the SLM and interferes with the other one. If an image with a single grey scale loads on the SLM, theoretical analysis proves that the phase of the Fourier spectra of the obtained hologram contains the added phase and a constant part relative to the optical distance. By subtracting the phase for the image with the grey scale of 0 from that for the image with other grey scales, the phase modulation can be characterized. Simulative and experimental results validate that the method is effective. The SLM after characterization is successfully used for coherent imaging, which reconfirms that this method is exact in practice. When compared to the traditional method, the new method is much faster and more convenient.