Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Label-Free Assessment of Key Biological Autofluorophores: Material Characteristics and Opportunities for Clinical Applications

Autofluorophores are endogenous fluorescent compounds that naturally occur in the intra and extracellular spaces of all tissues and organs. Most have vital biological functions - like the metabolic cofactors NAD(P)H and FAD+, as well as the structural protein collagen. Others are considered to be waste products - like lipofuscin and advanced glycation end products - which accumulate with age and are associated with cellular dysfunction. Due to their natural fluorescence, these materials have great utility for enabling non-invasive, label-free assays with direct ties to biological function. Numerous technologies, with different advantages and drawbacks, are applied to their assessment, including fluorescence lifetime imaging microscopy, hyperspectral microscopy, and flow cytometry. Here, the applications of label-free autofluorophore assessment are reviewed for clinical and health-research applications, with specific attention to biomaterials, disease detection, surgical guidance, treatment monitoring, and tissue assessment - fields that greatly benefit from non-invasive methodologies capable of continuous, in vivo characterization.

3D convolutional neural networks predict cellular metabolic pathway use from fluorescence lifetime decay data

Fluorescence lifetime imaging of the co-enzyme reduced nicotinamide adenine dinucleotide (NADH) offers a label-free approach for detecting cellular metabolic perturbations. However, the relationships between variations in NADH lifetime and metabolic pathway changes are complex, preventing robust interpretation of NADH lifetime data relative to metabolic phenotypes. Here, a three-dimensional convolutional neural network (3D CNN) trained at the cell level with 3D NAD(P)H lifetime decay images (two spatial dimensions and one time dimension) was developed to identify metabolic pathway usage by cancer cells. NADH fluorescence lifetime images of MCF7 breast cancer cells with three isolated metabolic pathways, glycolysis, oxidative phosphorylation, and glutaminolysis were obtained by a multiphoton fluorescence lifetime microscope and then segmented into individual cells as the input data for the classification models. The 3D CNN models achieved over 90% accuracy in identifying cancer cells reliant on glycolysis, oxidative phosphorylation, or glutaminolysis. Furthermore, the model trained with human breast cancer cell data successfully predicted the differences in metabolic phenotypes of macrophages from control and POLG-mutated mice. These results suggest that the integration of autofluorescence lifetime imaging with 3D CNNs enables intracellular spatial patterns of NADH intensity and temporal dynamics of the lifetime decay to discriminate multiple metabolic phenotypes. Furthermore, the use of 3D CNNs to identify metabolic phenotypes from NADH fluorescence lifetime decay images eliminates the need for time- and expertise-demanding exponential decay fitting procedures. In summary, metabolic-prediction CNNs will enable live-cell and in vivo metabolic measurements with single-cell resolution, filling a current gap in metabolic measurement technologies.

Label-free spatially maintained measurements of metabolic phenotypes in cells

Metabolic reprogramming at a cellular level contributes to many diseases including cancer, yet few assays are capable of measuring metabolic pathway usage by individual cells within living samples. Here, autofluorescence lifetime imaging is combined with single-cell segmentation and machine-learning models to predict the metabolic pathway usage of cancer cells. The metabolic activities of MCF7 breast cancer cells and HepG2 liver cancer cells were controlled by growing the cells in culture media with specific substrates and metabolic inhibitors. Fluorescence lifetime images of two endogenous metabolic coenzymes, reduced nicotinamide adenine dinucleotide (NADH) and oxidized flavin adenine dinucleotide (FAD), were acquired by a multi-photon fluorescence lifetime microscope and analyzed at the cellular level. Quantitative changes of NADH and FAD lifetime components were observed for cells using glycolysis, oxidative phosphorylation, and glutaminolysis. Conventional machine learning models trained with the autofluorescence features classified cells as dependent on glycolytic or oxidative metabolism with 90%-92% accuracy. Furthermore, adapting convolutional neural networks to predict cancer cell metabolic perturbations from the autofluorescence lifetime images provided improved performance, 95% accuracy, over traditional models trained via extracted features. Additionally, the model trained with the lifetime features of cancer cells could be transferred to autofluorescence lifetime images of T cells, with a prediction that 80% of activated T cells were glycolytic, and 97% of quiescent T cells were oxidative. In summary, autofluorescence lifetime imaging combined with machine learning models can detect metabolic perturbations between glycolysis and oxidative metabolism of living samples at a cellular level, providing a label-free technology to study cellular metabolism and metabolic heterogeneity.

Probing organoid metabolism using fluorescence lifetime imaging microscopy (FLIM): The next frontier of drug discovery and disease understanding

Organoid models have been used to address important questions in developmental and cancer biology, tissue repair, advanced modelling of disease and therapies, among other bioengineering applications. Such 3D microenvironmental models can investigate the regulation of cell metabolism, and provide key insights into the mechanisms at the basis of cell growth, differentiation, communication, interactions with the environment and cell death. Their accessibility and complexity, based on 3D spatial and temporal heterogeneity, make organoids suitable for the application of novel, dynamic imaging microscopy methods, such as fluorescence lifetime imaging microscopy (FLIM) and related decay time-assessing readouts. Several biomarkers and assays have been proposed to study cell metabolism by FLIM in various organoid models. Herein, we present an expert-opinion discussion on the principles of FLIM and PLIM, instrumentation and data collection and analysis protocols, and general and emerging biosensor-based approaches, to highlight the pioneering work being performed in this field.

Touch-free optical technologies to streamline the production of T cell therapies

Currently approved adoptive T cell therapy relies on autologous (obtained from the same patient) T cells, which often suffer from poor quality that diminishes treatment efficacy. Due to the heterogeneous nature of T cell quality between and within patients, significant efforts are aimed at optimizing cell manipulation and growth conditions for potent T cell products. We believe that touch-free imaging and sensing technologies are critical to monitor single-cell features during T cell manufacturing to ensure consistent and optimally timed methods for cell manipulation and growth. Here, we discuss emerging label-free optical imaging and sensing methods, along with machine learning techniques that could enable in-line feedback to optimize T cell quality at multiple stages during manufacturing. These methods have the potential to streamline current workflow, accelerate the manufacture of safe high-quality T cell therapies, and improve our understanding of the dynamic, heterogeneous processes of T cell manufacturing.

Label-free optical imaging of cell function and collagen structure for cell-based therapies

Cell-based therapies harness functional cells or tissues to mediate healing and treat disease. Assessment of cellular therapeutics requires methods that are non-destructive to ensure therapies remain viable and uncontaminated for use in patients. Optical imaging of endogenous collagen, by second-harmonic generation, and the metabolic coenzymes NADH and FAD, by autofluorescence microscopy, provides tissue structure and cellular information. Here, we review applications of label-free nonlinear optical imaging of cellular metabolism and collagen second-harmonic generation for assessing cell-based therapies. Additionally, we discuss the potential of label-free imaging for quality control of cell-based therapies, as well as the current limitations and potential future directions of label-free imaging technologies.

Raman spectroscopy and convolutional neural networks for monitoring biochemical radiation response in breast tumour xenografts

Tumour cells exhibit altered metabolic pathways that lead to radiation resistance and disease progression. Raman spectroscopy (RS) is a label-free optical modality that can monitor post-irradiation biomolecular signatures in tumour cells and tissues. Convolutional Neural Networks (CNN) perform automated feature extraction directly from data, with classification accuracy exceeding that of traditional machine learning, in cases where data is abundant and feature extraction is challenging. We are interested in developing a CNN-based predictive model to characterize clinical tumour response to radiation therapy based on their degree of radiosensitivity or radioresistance. In this work, a CNN architecture is built for identifying post-irradiation spectral changes in Raman spectra of tumour tissue. The model was trained to classify irradiated versus non-irradiated tissue using Raman spectra of breast tumour xenografts. The CNN effectively classified the tissue spectra, with accuracies exceeding 92.1% for data collected 3 days post-irradiation, and 85.0% at day 1 post-irradiation. Furthermore, the CNN was evaluated using a leave-one-out- (mouse, section or Raman map) validation approach to investigate its generalization to new test subjects. The CNN retained good predictive accuracy (average accuracies 83.7%, 91.4%, and 92.7%, respectively) when little to no information for a specific subject was given during training. Finally, the classification performance of the CNN was compared to that of a previously developed model based on group and basis restricted non-negative matrix factorization and random forest (GBR-NMF-RF) classification. We found that CNN yielded higher classification accuracy, sensitivity, and specificity in mice assessed 3 days post-irradiation, as compared with the GBR-NMF-RF approach. Overall, the CNN can detect biochemical spectral changes in tumour tissue at an early time point following irradiation, without the need for previous manual feature extraction. This study lays the foundation for developing a predictive framework for patient radiation response monitoring.

An approachable, flexible and practical machine learning workshop for biologists

Summary

The increasing prevalence and importance of machine learning in biological research have created a need for machine learning training resources tailored towards biological researchers. However, existing resources are often inaccessible, infeasible or inappropriate for biologists because they require significant computational and mathematical knowledge, demand an unrealistic time-investment or teach skills primarily for computational researchers. We created the Machine Learning for Biologists (ML4Bio) workshop, a short, intensive workshop that empowers biological researchers to comprehend machine learning applications and pursue machine learning collaborations in their own research. The ML4Bio workshop focuses on classification and was designed around three principles: (i) emphasizing preparedness over fluency or expertise, (ii) necessitating minimal coding and mathematical background and (iii) requiring low time investment. It incorporates active learning methods and custom open-source software that allows participants to explore machine learning workflows. After multiple sessions to improve workshop design, we performed a study on three workshop sessions. Despite some confusion around identifying subtle methodological flaws in machine learning workflows, participants generally reported that the workshop met their goals, provided them with valuable skills and knowledge and greatly increased their beliefs that they could engage in research that uses machine learning. ML4Bio is an educational tool for biological researchers, and its creation and evaluation provide valuable insight into tailoring educational resources for active researchers in different domains.

Availability and implementation

Workshop materials are available at https://github.com/carpentries-incubator/ml4bio-workshop and the ml4bio software is available at https://github.com/gitter-lab/ml4bio.

Supplementary information

Supplementary data are available at Bioinformatics online.

Disease-Relevant Single Cell Photonic Signatures Identify S100β Stem Cells and their Myogenic Progeny in Vascular Lesions

A hallmark of subclinical atherosclerosis is the accumulation of vascular smooth muscle cell (SMC)-like cells leading to intimal thickening and lesion formation. While medial SMCs contribute to vascular lesions, the involvement of resident vascular stem cells (vSCs) remains unclear. We evaluated single cell photonics as a discriminator of cell phenotype in vitro before the presence of vSC within vascular lesions was assessed ex vivo using supervised machine learning and further validated using lineage tracing analysis. Using a novel lab-on-a-Disk(Load) platform, label-free single cell photonic emissions from normal and injured vessels ex vivo were interrogated and compared to freshly isolated aortic SMCs, cultured Movas SMCs, macrophages, B-cells, S100β⁺ mVSc, bone marrow derived mesenchymal stem cells (MSC) and their respective myogenic progeny across five broadband light wavelengths (λ465 - λ670 ± 20 nm). We found that profiles were of sufficient coverage, specificity, and quality to clearly distinguish medial SMCs from different vascular beds (carotid vs aorta), discriminate normal carotid medial SMCs from lesional SMC-like cells ex vivo following flow restriction, and identify SMC differentiation of a series of multipotent stem cells following treatment with transforming growth factor beta 1 (TGF- β1), the Notch ligand Jagged1, and Sonic Hedgehog using multivariate analysis, in part, due to photonic emissions from enhanced collagen III and elastin expression. Supervised machine learning supported genetic lineage tracing analysis of S100β⁺ vSCs and identified the presence of S100β⁺vSC-derived myogenic progeny within vascular lesions. We conclude disease-relevant photonic signatures may have predictive value for vascular disease.

Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to distinguish the unique molecular environment of fluorophores. FLIM measures the time a fluorophore remains in an excited state before emitting a photon, and detects molecular variations of fluorophores that are not apparent with spectral techniques alone. FLIM is sensitive to multiple biomedical processes including disease progression and drug efficacy.

AIM

We provide an overview of FLIM principles, instrumentation, and analysis while highlighting the latest developments and biological applications.

APPROACH

This review covers FLIM principles and theory, including advantages over intensity-based fluorescence measurements. Fundamentals of FLIM instrumentation in time- and frequency-domains are summarized, along with recent developments. Image segmentation and analysis strategies that quantify spatial and molecular features of cellular heterogeneity are reviewed. Finally, representative applications are provided including high-resolution FLIM of cell- and organelle-level molecular changes, use of exogenous and endogenous fluorophores, and imaging protein-protein interactions with Förster resonance energy transfer (FRET). Advantages and limitations of FLIM are also discussed.

CONCLUSIONS

FLIM is advantageous for probing molecular environments of fluorophores to inform on fluorophore behavior that cannot be elucidated with intensity measurements alone. Development of FLIM technologies, analysis, and applications will further advance biological research and clinical assessments.