Reagent-Free and Rapid Assessment of T Cell Activation State Using Diffraction Phase Microscopy and Deep Learning

DHM/SERS reveals cellular morphology and molecular changes during iPSCs-derived activation of astrocytes

The activation of astrocytes derived from induced pluripotent stem cells (iPSCs) is of great significance in neuroscience research, and it is crucial to obtain both cellular morphology and biomolecular information non-destructively in situ, which is still complicated by the traditional optical microscopy and biochemical methods such as immunofluorescence and western blot. In this study, we combined digital holographic microscopy (DHM) and surface-enhanced Raman scattering (SERS) to investigate the activation characteristics of iPSCs-derived astrocytes. It was found that the projected area of activated astrocytes decreased by 67%, while the cell dry mass increased by 23%, and the cells changed from a flat polygonal shape to an elongated star-shaped morphology. SERS analysis further revealed an increase in the intensities of protein spectral peaks (phenylalanine 1001 cm-1, proline 1043 cm-1, etc.) and lipid-related peaks (phosphatidylserine 524 cm-1, triglycerides 1264 cm-1, etc.) decreased in intensity. Principal component analysis-linear discriminant analysis (PCA-LDA) modeling based on spectral data distinguished resting and reactive astrocytes with a high accuracy of 96.5%. The increase in dry mass correlated with the increase in protein content, while the decrease in projected area indicated the adjustment of lipid composition and cell membrane remodeling. Importantly, the results not only reveal the cellular morphology and molecular changes during iPSCs-derived astrocytes activation but also reflect their mapping relationship, thereby providing new insights into diagnosing and treating neurodegenerative diseases.

On the use of deep learning for phase recovery

Phase recovery (PR) refers to calculating the phase of the light field from its intensity measurements. As exemplified from quantitative phase imaging and coherent diffraction imaging to adaptive optics, PR is essential for reconstructing the refractive index distribution or topography of an object and correcting the aberration of an imaging system. In recent years, deep learning (DL), often implemented through deep neural networks, has provided unprecedented support for computational imaging, leading to more efficient solutions for various PR problems. In this review, we first briefly introduce conventional methods for PR. Then, we review how DL provides support for PR from the following three stages, namely, pre-processing, in-processing, and post-processing. We also review how DL is used in phase image processing. Finally, we summarize the work in DL for PR and provide an outlook on how to better use DL to improve the reliability and efficiency of PR. Furthermore, we present a live-updating resource ( https://github.com/kqwang/phase-recovery ) for readers to learn more about PR.

Artificial intelligence-enabled quantitative phase imaging methods for life sciences

Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.

Targeted Enzyme Activity Imaging with Quantitative Phase Microscopy

Quantitative phase imaging (QPI) is a powerful optical imaging modality for label-free, rapid, and three-dimensional (3D) monitoring of cells and tissues. However, molecular imaging of important intracellular biomolecules such as enzymes remains a largely unexplored area for QPI. Herein, we introduce a fundamentally new approach by designing QPI contrast agents that allow sensitive detection of intracellular biomolecules. We report a new class of bio-orthogonal QPI-nanoprobes for in situ high-contrast refractive index (RI) imaging of enzyme activity. The nanoprobes feature silica nanoparticles (SiO2 NPs) having higher RI than endogenous cellular components and surface-anchored cyanobenzothiazole-cysteine (CBT-Cys) conjugated enzyme-responsive peptide sequences. The nanoprobes specifically aggregated in cells with target enzyme activity, increasing intracellular RI and enabling precise visualization of intracellular enzyme activity. We envision that this general design of QPI-nanoprobes could open doors for spatial-temporal mapping of enzyme activity with direct implications for disease diagnosis and evaluating the therapeutic efficacy.

Touch-free optical technologies to streamline the production of T cell therapies

Currently approved adoptive T cell therapy relies on autologous (obtained from the same patient) T cells, which often suffer from poor quality that diminishes treatment efficacy. Due to the heterogeneous nature of T cell quality between and within patients, significant efforts are aimed at optimizing cell manipulation and growth conditions for potent T cell products. We believe that touch-free imaging and sensing technologies are critical to monitor single-cell features during T cell manufacturing to ensure consistent and optimally timed methods for cell manipulation and growth. Here, we discuss emerging label-free optical imaging and sensing methods, along with machine learning techniques that could enable in-line feedback to optimize T cell quality at multiple stages during manufacturing. These methods have the potential to streamline current workflow, accelerate the manufacture of safe high-quality T cell therapies, and improve our understanding of the dynamic, heterogeneous processes of T cell manufacturing.

Automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks

Whole blood cell analysis is widely used in medical applications since its results are indicators for diagnosing a series of diseases. In this work, we report automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. First, a commercial microscope equipped with our developed Phase Real-time Microscope Camera (PhaseRMiC) obtains both bright-field and quantitative phase images. Then, these images are automatically processed by our designed blood smear recognition networks (BSRNet) that recognize erythrocytes, leukocytes and platelets. Finally, blood cell parameters such as counts, shapes and volumes can be extracted according to both quantitative phase images and automatic recognition results. The proposed whole blood cell analysis technique provides high-quality blood cell images and supports accurate blood cell recognition and analysis. Moreover, this approach requires rather simple and cost-effective setups as well as easy and rapid sample preparations. Therefore, this proposed method has great potential application in blood testing aiming at disease diagnostics.

Label-Free Vibrational and Quantitative Phase Microscopy Reveals Remarkable Pathogen-Induced Morphomolecular Divergence in Tumor-Derived Cells

Delineating the molecular and morphological changes that cancer cells undergo in response to extracellular stimuli is crucial for identifying factors that promote tumor progression. Label-free optical imaging offers a potentially promising route for retrieving such single-cell information by generating detailed visualization of the morphology and determining alterations in biomolecular composition. The potential of such nonperturbative morphomolecular microscopy for analyzing microbiota-cancer cell interactions has been surprisingly underappreciated, despite the growing evidence of the critical role of dysbiosis in malignant transformations. Here, using a model system of breast cancer cells, we show that label-free Raman microspectroscopy and quantitative phase microscopy can detect biomolecular and morphological changes in single cells exposed to Bacteroides fragilis toxin (BFT), a toxin secreted by enterotoxigenicB. fragilis. Remarkably, using machine learning to elucidate subtle, but consistent, cellular differences, we found that the morphomolecular differences between BFT-exposed and control breast cancer cells became more accentuated after in vivo passage, corroborating our findings that a short-term BFT exposure imparts a long-term effect on cancer cells and promotes a more invasive phenotype. Complementing more classical labeling techniques, our label-free platform offers a global detection approach with measurements representative of the overall cellular phenotype, paving the way for further investigations into the multifaceted interactions between the cancer cell and the microbiota.

Raman and quantitative phase imaging allow morpho-molecular recognition of malignancy and stages of B-cell acute lymphoblastic leukemia

Acute lymphoblastic leukemia (ALL) is one of the most common malignancies that account for nearly one-third of all pediatric cancers. The current diagnostic assays are time-consuming, labor-intensive, and require expensive reagents. Here, we report a label-free approach featuring diffraction phase imaging and Raman microscopy that can retrieve both morphological and molecular attributes for label-free optical phenotyping of individual B cells. By investigating leukemia cell lines of early and late stages along with the healthy B cells, we show that phase images can capture subtle morphological differences among the healthy, early, and late stages of leukemic cells. By exploiting its biomolecular specificity, we demonstrate that Raman microscopy is capable of accurately identifying not only different stages of leukemia cells but also individual cell lines at each stage. Overall, our study provides a rationale for employing this hybrid modality to screen leukemia cells using the widefield QPI and using Raman microscopy for accurate differentiation of early and late-stage phenotypes. This contrast-free and rapid diagnostic tool exhibits great promise for clinical diagnosis and staging of leukemia in the near future.

Rapid, label-free classification of tumor-reactive T cell killing with quantitative phase microscopy and machine learning

Quantitative phase microscopy (QPM) enables studies of living biological systems without exogenous labels. To increase the utility of QPM, machine-learning methods have been adapted to extract additional information from the quantitative phase data. Previous QPM approaches focused on fluid flow systems or time-lapse images that provide high throughput data for cells at single time points, or of time-lapse images that require delayed post-experiment analyses, respectively. To date, QPM studies have not imaged specific cells over time with rapid, concurrent analyses during image acquisition. In order to study biological phenomena or cellular interactions over time, efficient time-dependent methods that automatically and rapidly identify events of interest are desirable. Here, we present an approach that combines QPM and machine learning to identify tumor-reactive T cell killing of adherent cancer cells rapidly, which could be used for identifying and isolating novel T cells and/or their T cell receptors for studies in cancer immunotherapy. We demonstrate the utility of this method by machine learning model training and validation studies using one melanoma-cognate T cell receptor model system, followed by high classification accuracy in identifying T cell killing in an additional, independent melanoma-cognate T cell receptor model system. This general approach could be useful for studying additional biological systems under label-free conditions over extended periods of examination.

High-speed laser-scanning biological microscopy using FACED

Laser scanning is used in advanced biological microscopy to deliver superior imaging contrast, resolution and sensitivity. However, it is challenging to scale up the scanning speed required for interrogating a large and heterogeneous population of biological specimens or capturing highly dynamic biological processes at high spatiotemporal resolution. Bypassing the speed limitation of traditional mechanical methods, free-space angular-chirp-enhanced delay (FACED) is an all-optical, passive and reconfigurable laser-scanning approach that has been successfully applied in different microscopy modalities at an ultrafast line-scan rate of 1-80 MHz. Optimal FACED imaging performance requires optimized experimental design and implementation to enable specific high-speed applications. In this protocol, we aim to disseminate information allowing FACED to be applied to a broader range of imaging modalities. We provide (i) a comprehensive guide and design specifications for the FACED hardware; (ii) step-by-step optical implementations of the FACED module including the key custom components; and (iii) the overall image acquisition and reconstruction pipeline. We illustrate two practical imaging configurations: multimodal FACED imaging flow cytometry (bright-field, fluorescence and second-harmonic generation) and kHz 2D two-photon fluorescence microscopy. Users with basic experience in optical microscope operation and software engineering should be able to complete the setup of the FACED imaging hardware and software in ~2-3 months.

Deepometry, a framework for applying supervised and weakly supervised deep learning to imaging cytometry

Deep learning offers the potential to extract more than meets the eye from images captured by imaging flow cytometry. This protocol describes the application of deep learning to single-cell images to perform supervised cell classification and weakly supervised learning, using example data from an experiment exploring red blood cell morphology. We describe how to acquire and transform suitable input data as well as the steps required for deep learning training and inference using an open-source web-based application. All steps of the protocol are provided as open-source Python as well as MATLAB runtime scripts, through both command-line and graphic user interfaces. The protocol enables a flexible and friendly environment for morphological phenotyping using supervised and weakly supervised learning and the subsequent exploration of the deep learning features using multi-dimensional visualization tools. The protocol requires 40 h when training from scratch and 1 h when using a pre-trained model.

Toward deep biophysical cytometry: prospects and challenges

The biophysical properties of cells reflect their identities, underpin their homeostatic state in health, and define the pathogenesis of disease. Recent leapfrogging advances in biophysical cytometry now give access to this information, which is obscured in molecular assays, with a discriminative power that was once inconceivable. However, biophysical cytometry should go 'deeper' in terms of exploiting the information-rich cellular biophysical content, generating a molecular knowledge base of cellular biophysical properties, and standardizing the protocols for wider dissemination. Overcoming these barriers, which requires concurrent innovations in microfluidics, optical imaging, and computer vision, could unleash the enormous potential of biophysical cytometry not only for gaining a new mechanistic understanding of biological systems but also for identifying new cost-effective biomarkers of disease.

Coarse Raman and optical diffraction tomographic imaging enable label-free phenotyping of isogenic breast cancer cells of varying metastatic potential

Identification of the metastatic potential represents one of the most important tasks for molecular imaging of cancer. While molecular imaging of metastases has witnessed substantial progress as an area of clinical inquiry, determining precisely what differentiates the metastatic phenotype has proven to be more elusive. In this study, we utilize both the morphological and molecular information provided by 3D optical diffraction tomography and Raman spectroscopy, respectively, to propose a label-free route for optical phenotyping of cancer cells at single-cell resolution. By using an isogenic panel of cell lines derived from MDA-MB-231 breast cancer cells that vary in their metastatic potential, we show that 3D refractive index tomograms can capture subtle morphological differences among the parental, circulating tumor cells, and lung metastatic cells. By leveraging its molecular specificity, we demonstrate that coarse Raman microscopy is capable of rapidly mapping a sufficient number of cells for training a random forest classifier that can accurately predict the metastatic potential of cells at a single-cell level. We also perform multivariate curve resolution alternating least squares decomposition of the spectral dataset to demarcate spectra from cytoplasm and nucleus, and test the feasibility of identifying metastatic phenotypes using the spectra only from the cytoplasmic and nuclear regions. Overall, our study provides a rationale for employing coarse Raman mapping to substantially reduce measurement time thereby enabling the acquisition of reasonably large training datasets that hold the key for label-free single-cell analysis and, consequently, for differentiation of indolent from aggressive phenotypes.

Identification and Staging of B-Cell Acute Lymphoblastic Leukemia Using Quantitative Phase Imaging and Machine Learning

Identification and classification of leukemia cells in a rapid and label-free fashion is clinically challenging and thus presents a prime arena for implementing new diagnostic tools. Quantitative phase imaging, which maps optical path length delays introduced by the specimen, has been demonstrated to discern cellular phenotypes based on differential morphological attributes. Rapid acquisition capability and the availability of label-free images with high information content have enabled researchers to use machine learning (ML) to reveal latent features. We developed a set of ML classifiers, including convolutional neural networks, to discern healthy B cells from lymphoblasts and classify stages of B cell acute lymphoblastic leukemia. Here, we show that the average dry mass and volume of normal B cells are lower than those of cancerous cells and that these morphologic parameters increase further alongside disease progression. We find that the relaxed training requirements of a ML approach are conducive to the classification of cell type, with minimal space, training time, and memory requirements. Our findings pave the way for a larger study on clinical samples of acute lymphoblastic leukemia, with the overarching goal of its broader use in hematopathology, where the prospect of objective diagnoses with minimal sample preparation remains highly desirable.

Classification of cell morphology with quantitative phase microscopy and machine learning

We describe and compare two machine learning approaches for cell classification based on label-free quantitative phase imaging with transport of intensity equation methods. In one approach, we design a multilevel integrated machine learning classifier including various individual models such as artificial neural network, extreme learning machine and generalized logistic regression. In another approach, we apply a pretrained convolutional neural network using transfer learning for the classification. As a validation, we show the performances of both approaches on classification between macrophages cultured in normal gravity and microgravity with quantitative phase imaging. The multilevel integrated classifier achieves average accuracy 93.1%, which is comparable to the average accuracy 93.5% obtained by convolutional neural network. The presented quantitative phase imaging system with two classification approaches could be helpful to biomedical scientists for easy and accurate cell analysis.

Determination of causes of death via spectrochemical analysis of forensic autopsies-based pulmonary edema fluid samples with deep learning algorithm

This study investigated whether infrared spectroscopy combined with a deep learning algorithm could be a useful tool for determining causes of death by analyzing pulmonary edema fluid from forensic autopsies. A newly designed convolutional neural network-based deep learning framework, named DeepIR and eight popular machine learning algorithms, were used to construct classifiers. The prediction performances of these classifiers demonstrated that DeepIR outperformed the machine learning algorithms in establishing classifiers to determine the causes of death. Moreover, DeepIR was generally less dependent on preprocessing procedures than were the machine learning algorithms; it provided the validation accuracy with a narrow range from 0.9661 to 0.9856 and the test accuracy ranging from 0.8774 to 0.9167 on the raw pulmonary edema fluid spectral dataset and the nine preprocessing protocol-based datasets in our study. In conclusion, this study demonstrates that the deep learning-equipped Fourier transform infrared spectroscopy technique has the potential to be an effective aid for determining causes of death.

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

The importance of T cells in immunotherapy has motivated developing technologies to improve therapeutic efficacy. One objective is assessing antigen-induced T cell activation because only functionally active T cells are capable of killing the desired targets. Autofluorescence imaging can distinguish T cell activity states in a non-destructive manner by detecting endogenous changes in metabolic co-enzymes such as NAD(P)H. However, recognizing robust activity patterns is computationally challenging in the absence of exogenous labels. We demonstrate machine learning methods that can accurately classify T cell activity across human donors from NAD(P)H intensity images. Using 8260 cropped single-cell images from six donors, we evaluate classifiers ranging from traditional models that use previously-extracted image features to convolutional neural networks (CNNs) pre-trained on general non-biological images. Adapting pre-trained CNNs for the T cell activity classification task provides substantially better performance than traditional models or a simple CNN trained with the autofluorescence images alone. Visualizing the images with dimension reduction provides intuition into why the CNNs achieve higher accuracy than other approaches. Our image processing and classifier training software is available at https://github.com/gitter-lab/t-cell-classification.