Nonparametric empirical Bayesian framework for fluorescence-lifetime imaging microscopy

Building Fluorescence Lifetime Maps Photon-by-Photon by Leveraging Spatial Correlations

Fluorescence lifetime imaging microscopy (FLIM) has become a standard tool in the quantitative characterization of subcellular environments. However, quantitative FLIM analyses face several challenges. First, spatial correlations between pixels are often ignored as signal from individual pixels is analyzed independently thereby limiting spatial resolution. Second, existing methods deduce photon ratios instead of absolute lifetime maps. Next, the number of fluorophore species contributing to the signal is unknown, while excited state lifetimes with <1 ns difference are difficult to discriminate. Finally, existing analyses require high photon budgets and often cannot rigorously propagate experimental uncertainty into values over lifetime maps and number of species involved. To overcome all of these challenges simultaneously and self-consistently at once, we propose the first doubly nonparametric framework. That is, we learn the number of species (using Beta-Bernoulli process priors) and absolute maps of these fluorophore species (using Gaussian process priors) by leveraging information from pulses not leading to observed photon. We benchmark our framework using a broad range of synthetic and experimental data and demonstrate its robustness across a number of scenarios including cases where we recover lifetime differences between species as small as 0.3 ns with merely 1000 photons.

Fluorescence lifetime: Beating the IRF and interpulse window

Fluorescence lifetime imaging captures the spatial distribution of chemical species across cellular environments employing pulsed illumination confocal setups. However, quantitative interpretation of lifetime data continues to face critical challenges. For instance, fluorescent species with known in vitro excited-state lifetimes may split into multiple species with unique lifetimes when introduced into complex living environments. What is more, mixtures of species, which may be both endogenous and introduced into the sample, may exhibit 1) very similar lifetimes as well as 2) wide ranges of lifetimes including lifetimes shorter than the instrumental response function or whose duration may be long enough to be comparable to the interpulse window. By contrast, existing methods of analysis are optimized for well-separated and intermediate lifetimes. Here, we broaden the applicability of fluorescence lifetime analysis by simultaneously treating unknown mixtures of arbitrary lifetimes-outside the intermediate, Goldilocks, zone-for data drawn from a single confocal spot leveraging the tools of Bayesian nonparametrics (BNP). We benchmark our algorithm, termed BNP lifetime analysis, using a range of synthetic and experimental data. Moreover, we show that the BNP lifetime analysis method can distinguish and deduce lifetimes using photon counts as small as 500.

A method for the fast and photon-efficient analysis of time-domain fluorescence lifetime image data over large dynamic ranges

Fluorescence lifetime imaging (FLIM) allows the quantification of sub‐cellular processes in situ, in living cells. A number of approaches have been developed to extract the lifetime from time‐domain FLIM data, but they are often limited in terms of speed, photon efficiency, precision or the dynamic range of lifetimes they can measure. Here, we focus on one of the best performing methods in the field, the centre‐of‐mass method (CMM), that conveys advantages in terms of speed and photon efficiency over others. In this paper, however, we identify a loss of photon efficiency of CMM for short lifetimes when background noise is present. We subsequently present a new development and generalization of CMM that provides for the rapid and accurate extraction of fluorescence lifetime over a large lifetime dynamic range. We provide software tools to simulate, validate and analyse FLIM data sets and compare the performance of our approach against the standard CMM and the commonly employed least‐square minimization (LSM) methods. Our method features a better photon efficiency than standard CMM and LSM and is robust in the presence of background noise. The algorithm is applicable to any time‐domain FLIM data set.

Fast Analysis of Time-Domain Fluorescence Lifetime Imaging via Extreme Learning Machine

We present a fast and accurate analytical method for fluorescence lifetime imaging microscopy (FLIM), using the extreme learning machine (ELM). We used extensive metrics to evaluate ELM and existing algorithms. First, we compared these algorithms using synthetic datasets. The results indicate that ELM can obtain higher fidelity, even in low-photon conditions. Afterwards, we used ELM to retrieve lifetime components from human prostate cancer cells loaded with gold nanosensors, showing that ELM also outperforms the iterative fitting and non-fitting algorithms. By comparing ELM with a computational efficient neural network, ELM achieves comparable accuracy with less training and inference time. As there is no back-propagation process for ELM during the training phase, the training speed is much higher than existing neural network approaches. The proposed strategy is promising for edge computing with online training.

Spatial resolution improved fluorescence lifetime imaging via deep learning

We present a deep learning approach to obtain high-resolution (HR) fluorescence lifetime images from low-resolution (LR) images acquired from fluorescence lifetime imaging (FLIM) systems. We first proposed a theoretical method for training neural networks to generate massive semi-synthetic FLIM data with various cellular morphologies, a sizeable dynamic lifetime range, and complex decay components. We then developed a degrading model to obtain LR-HR pairs and created a hybrid neural network, the spatial resolution improved FLIM net (SRI-FLIMnet) to simultaneously estimate fluorescence lifetimes and realize the nonlinear transformation from LR to HR images. The evaluative results demonstrate SRI-FLIMnet's superior performance in reconstructing spatial information from limited pixel resolution. We also verified SRI-FLIMnet using experimental images of bacterial infected mouse raw macrophage cells. Results show that the proposed data generation method and SRI-FLIMnet efficiently achieve superior spatial resolution for FLIM applications. Our study provides a solution for fast obtaining HR FLIM images.

High Resolution Fluorescence Lifetime Maps from Minimal Photon Counts

Fluorescence lifetime imaging microscopy (FLIM) may reveal subcellular spatial lifetime maps of key molecular species. Yet, such a quantitative picture of life necessarily demands high photon budgets at every pixel under the current analysis paradigm, thereby increasing acquisition time and photodamage to the sample. Motivated by recent developments in computational statistics, we provide a direct means to update our knowledge of the lifetime maps of species of different lifetimes from direct photon arrivals, while accounting for experimental features such as arbitrary forms of the instrument response function (IRF) and exploiting information from empty laser pulses not resulting in photon detection. Our ability to construct lifetime maps holds for arbitrary lifetimes, from short lifetimes (comparable to the IRF) to lifetimes exceeding interpulse times. As our method is highly data efficient, for the same amount of data normally used to determine lifetimes and photon ratios, working within the Bayesian paradigm, we report direct blind unmixing of lifetimes with subnanosecond resolution and subpixel spatial resolution using standard raster scan FLIM images. We demonstrate our method using a wide range of simulated and experimental data.

Label-free sensing of cells with fluorescence lifetime imaging: The quest for metabolic heterogeneity

Molecular, morphological, and physiological heterogeneity is the inherent property of cells which governs differences in their response to external influence. Tumor cell metabolic heterogeneity is of a special interest due to its clinical relevance to tumor progression and therapeutic outcomes. Rapid, sensitive, and noninvasive assessment of metabolic heterogeneity of cells is a great demand for biomedical sciences. Fluorescence lifetime imaging (FLIM), which is an all-optical technique, is an emerging tool for sensing and quantifying cellular metabolism by measuring fluorescence decay parameters of endogenous fluorophores, such as NAD(P)H. To achieve accurate discrimination between metabolically diverse cellular subpopulations, appropriate approaches to FLIM data collection and analysis are needed. In this paper, the unique capability of FLIM to attain the overarching goal of discriminating metabolic heterogeneity is demonstrated. This has been achieved using an approach to data analysis based on the nonparametric analysis, which revealed a much better sensitivity to the presence of metabolically distinct subpopulations compared to more traditional approaches of FLIM measurements and analysis. The approach was further validated for imaging cultured cancer cells treated with chemotherapy. These results pave the way for accurate detection and quantification of cellular metabolic heterogeneity using FLIM, which will be valuable for assessing therapeutic vulnerabilities and predicting clinical outcomes.

Complex wavelet filter improves FLIM phasors for photon starved imaging experiments

Fluorescence lifetime imaging microscopy (FLIM) with phasor analysis provides easy visualization and analysis of fluorophores' lifetimes which is valuable for multiple applications including metabolic imaging, STED imaging, FRET imaging and functional imaging. However, FLIM imaging typically suffers from low photon budgets, leading to unfavorable signal to noise ratios which in many cases prevent extraction of information from the data. Traditionally, median filters are applied in phasor analysis to tackle this problem. This unfortunately degrades high spatial frequency FLIM information in the phasor analysis. These high spatial frequency components are typically edges of features and puncta, which applies to membranes, mitochondria, granules and small organelles in a biological sample. To tackle this problem, we propose a filtering strategy with complex wavelet filtering and Anscombe transform for FLIM phasor analysis. This filtering strategy preserves fine structures and reports accurate lifetimes in photon starved FLIM imaging. Moreover, this filter outperforms median filters and makes FLIM imaging with lower laser power and faster imaging possible.

FLIMJ: An open-source ImageJ toolkit for fluorescence lifetime image data analysis

In the field of fluorescence microscopy, there is continued demand for dynamic technologies that can exploit the complete information from every pixel of an image. One imaging technique with proven ability for yielding additional information from fluorescence imaging is Fluorescence Lifetime Imaging Microscopy (FLIM). FLIM allows for the measurement of how long a fluorophore stays in an excited energy state, and this measurement is affected by changes in its chemical microenvironment, such as proximity to other fluorophores, pH, and hydrophobic regions. This ability to provide information about the microenvironment has made FLIM a powerful tool for cellular imaging studies ranging from metabolic measurement to measuring distances between proteins. The increased use of FLIM has necessitated the development of computational tools for integrating FLIM analysis with image and data processing. To address this need, we have created FLIMJ, an ImageJ plugin and toolkit that allows for easy use and development of extensible image analysis workflows with FLIM data. Built on the FLIMLib decay curve fitting library and the ImageJ Ops framework, FLIMJ offers FLIM fitting routines with seamless integration with many other ImageJ components, and the ability to be extended to create complex FLIM analysis workflows. Building on ImageJ Ops also enables FLIMJ's routines to be used with Jupyter notebooks and integrate naturally with science-friendly programming in, e.g., Python and Groovy. We show the extensibility of FLIMJ in two analysis scenarios: lifetime-based image segmentation and image colocalization. We also validate the fitting routines by comparing them against industry FLIM analysis standards.

Direct Photon-by-Photon Analysis of Time-Resolved Pulsed Excitation Data using Bayesian Nonparametrics

Lifetimes of chemical species are typically estimated by either fitting time-correlated single-photon counting (TCSPC) histograms or phasor analysis from time-resolved photon arrivals. While both methods yield lifetimes in a computationally efficient manner, their performance is limited by choices made on the number of distinct chemical species contributing photons. However, the number of species is encoded in the photon arrival times collected for each illuminated spot and need not be set by hand a priori. Here, we propose a direct photon-by-photon analysis of data drawn from pulsed excitation experiments to infer, simultaneously and self-consistently, the number of species and their associated lifetimes from a few thousand photons. We do so by leveraging new mathematical tools within the Bayesian nonparametric. We benchmark our method for both simulated and experimental data for 1-4 species.

Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to distinguish the unique molecular environment of fluorophores. FLIM measures the time a fluorophore remains in an excited state before emitting a photon, and detects molecular variations of fluorophores that are not apparent with spectral techniques alone. FLIM is sensitive to multiple biomedical processes including disease progression and drug efficacy.

AIM

We provide an overview of FLIM principles, instrumentation, and analysis while highlighting the latest developments and biological applications.

APPROACH

This review covers FLIM principles and theory, including advantages over intensity-based fluorescence measurements. Fundamentals of FLIM instrumentation in time- and frequency-domains are summarized, along with recent developments. Image segmentation and analysis strategies that quantify spatial and molecular features of cellular heterogeneity are reviewed. Finally, representative applications are provided including high-resolution FLIM of cell- and organelle-level molecular changes, use of exogenous and endogenous fluorophores, and imaging protein-protein interactions with Förster resonance energy transfer (FRET). Advantages and limitations of FLIM are also discussed.

CONCLUSIONS

FLIM is advantageous for probing molecular environments of fluorophores to inform on fluorophore behavior that cannot be elucidated with intensity measurements alone. Development of FLIM technologies, analysis, and applications will further advance biological research and clinical assessments.

Review: Clinical in vivo multiphoton FLIM tomography

Fluorescence Lifetime Imaging (FLIM) in life sciences based on ultrashort laser scanning microscopy and time-correlated single photon counting (TCSPC) started 30 years ago in Jena/East-Germany. One decade later, first two-photon FLIM images of a human finger were taken with a lab microscope based on a tunable femtosecond Ti:sapphire laser. In 2002/2003, first clinical non-invasive two-photon FLIM studies on patients with dermatological disorders were performed using a novel multiphoton tomograph. Current in vivo two-photon FLIM studies on human subjects are based on TCSPC and focus on (i) patients with skin inflammation and skin cancer as well as brain tumors, (ii) cosmetic research on volunteers to evaluate anti-ageing cremes, (iii) pharmaceutical research on volunteers to gain information on in situ pharmacokinetics, and (iv) space medicine to study non-invasively skin modifications on astronauts during long-term space flights. Two-photon FLIM studies on volunteers and patients are performed with multiphoton FLIM tomographs using near infrared femtosecond laser technology that provide rapid non-invasive and label-free intratissue autofluorescence biopsies with picosecond temporal resolution.