FLIMJ: An open-source ImageJ toolkit for fluorescence lifetime image data analysis

3D convolutional neural networks predict cellular metabolic pathway use from fluorescence lifetime decay data

Fluorescence lifetime imaging of the co-enzyme reduced nicotinamide adenine dinucleotide (NADH) offers a label-free approach for detecting cellular metabolic perturbations. However, the relationships between variations in NADH lifetime and metabolic pathway changes are complex, preventing robust interpretation of NADH lifetime data relative to metabolic phenotypes. Here, a three-dimensional convolutional neural network (3D CNN) trained at the cell level with 3D NAD(P)H lifetime decay images (two spatial dimensions and one time dimension) was developed to identify metabolic pathway usage by cancer cells. NADH fluorescence lifetime images of MCF7 breast cancer cells with three isolated metabolic pathways, glycolysis, oxidative phosphorylation, and glutaminolysis were obtained by a multiphoton fluorescence lifetime microscope and then segmented into individual cells as the input data for the classification models. The 3D CNN models achieved over 90% accuracy in identifying cancer cells reliant on glycolysis, oxidative phosphorylation, or glutaminolysis. Furthermore, the model trained with human breast cancer cell data successfully predicted the differences in metabolic phenotypes of macrophages from control and POLG-mutated mice. These results suggest that the integration of autofluorescence lifetime imaging with 3D CNNs enables intracellular spatial patterns of NADH intensity and temporal dynamics of the lifetime decay to discriminate multiple metabolic phenotypes. Furthermore, the use of 3D CNNs to identify metabolic phenotypes from NADH fluorescence lifetime decay images eliminates the need for time- and expertise-demanding exponential decay fitting procedures. In summary, metabolic-prediction CNNs will enable live-cell and in vivo metabolic measurements with single-cell resolution, filling a current gap in metabolic measurement technologies.

Single-photon microscopy to study biomolecular condensates

Biomolecular condensates serve as membrane-less compartments within cells, concentrating proteins and nucleic acids to facilitate precise spatial and temporal orchestration of various biological processes. The diversity of these processes and the substantial variability in condensate characteristics present a formidable challenge for quantifying their molecular dynamics, surpassing the capabilities of conventional microscopy. Here, we show that our single-photon microscope provides a comprehensive live-cell spectroscopy and imaging framework for investigating biomolecular condensation. Leveraging a single-photon detector array, single-photon microscopy enhances the potential of quantitative confocal microscopy by providing access to fluorescence signals at the single-photon level. Our platform incorporates photon spatiotemporal tagging, which allowed us to perform time-lapse super-resolved imaging for molecular sub-diffraction environment organization with simultaneous monitoring of molecular mobility, interactions, and nano-environment properties through fluorescence lifetime fluctuation spectroscopy. This integrated correlative study reveals the dynamics and interactions of RNA-binding proteins involved in forming stress granules, a specific type of biomolecular condensates, across a wide range of spatial and temporal scales. Our versatile framework opens up avenues for exploring a broad spectrum of biomolecular processes beyond the formation of membrane-less organelles.

Real-time open-source FLIM analysis

Fluorescence lifetime imaging microscopy (FLIM) provides valuable quantitative insights into fluorophores' chemical microenvironment. Due to long computation times and the lack of accessible, open-source real-time analysis toolkits, traditional analysis of FLIM data, particularly with the widely used time-correlated single-photon counting (TCSPC) approach, typically occurs after acquisition. As a result, uncertainties about the quality of FLIM data persist even after collection, frequently necessitating the extension of imaging sessions. Unfortunately, prolonged sessions not only risk missing important biological events but also cause photobleaching and photodamage. We present the first open-source program designed for real-time FLIM analysis during specimen scanning to address these challenges. Our approach combines acquisition with real-time computational and visualization capabilities, allowing us to assess FLIM data quality on the fly. Our open-source real-time FLIM viewer, integrated as a Napari plugin, displays phasor analysis and rapid lifetime determination (RLD) results computed from real-time data transmitted by acquisition software such as the open-source Micro-Manager-based OpenScan package. Our method facilitates early identification of FLIM signatures and data quality assessment by providing preliminary analysis during acquisition. This not only speeds up the imaging process, but it is especially useful when imaging sensitive live biological samples.

FLUTE: A Python GUI for interactive phasor analysis of FLIM data

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique used to probe the local environment of fluorophores. The fit-free phasor approach to FLIM data is increasingly being used due to its ease of interpretation. To date, no open-source graphical user interface (GUI) for phasor analysis of FLIM data is available in Python, thus limiting the widespread use of phasor analysis in biomedical research. Here, we present Fluorescence Lifetime Ultimate Explorer (FLUTE), a Python GUI that is designed to fill this gap. FLUTE simplifies and automates many aspects of the analysis of FLIM data acquired in the time domain, such as calibrating the FLIM data, performing interactive exploration of the phasor plot, displaying phasor plots and FLIM images with different lifetime contrasts simultaneously, and calculating the distance from known molecular species. After applying desired filters and thresholds, the final edited datasets can be exported for further user-specific analysis. FLUTE has been tested using several FLIM datasets including autofluorescence of zebrafish embryos and in vitro cells. In summary, our user-friendly GUI extends the advantages of phasor plotting by making the data visualization and analysis easy and interactive, allows for analysis of large FLIM datasets, and accelerates FLIM analysis for non-specialized labs.

Probing organoid metabolism using fluorescence lifetime imaging microscopy (FLIM): The next frontier of drug discovery and disease understanding

Organoid models have been used to address important questions in developmental and cancer biology, tissue repair, advanced modelling of disease and therapies, among other bioengineering applications. Such 3D microenvironmental models can investigate the regulation of cell metabolism, and provide key insights into the mechanisms at the basis of cell growth, differentiation, communication, interactions with the environment and cell death. Their accessibility and complexity, based on 3D spatial and temporal heterogeneity, make organoids suitable for the application of novel, dynamic imaging microscopy methods, such as fluorescence lifetime imaging microscopy (FLIM) and related decay time-assessing readouts. Several biomarkers and assays have been proposed to study cell metabolism by FLIM in various organoid models. Herein, we present an expert-opinion discussion on the principles of FLIM and PLIM, instrumentation and data collection and analysis protocols, and general and emerging biosensor-based approaches, to highlight the pioneering work being performed in this field.

Visualising varnish removal for conservation of paintings by fluorescence lifetime imaging (FLIM)

The removal of varnish from the surface is a key step in painting conservation. Varnish removal is traditionally monitored by examining the painting surface under ultraviolet illumination. We show here that by imaging the fluorescence lifetime instead, much better contrast, sensitivity, and specificity can be achieved. For this purpose, we developed a lightweight (4.8 kg) portable instrument for macroscopic fluorescence lifetime imaging (FLIM). It is based on a time-correlated single-photon avalanche diode (SPAD) camera to acquire the FLIM images and a pulsed 440 nm diode laser to excite the varnish fluorescence. A historical model painting was examined to demonstrate the capabilities of the system. We found that the FLIM images provided information on the distribution of the varnish on the painting surface with greater sensitivity, specificity, and contrast compared to the traditional ultraviolet illumination photography. The distribution of the varnish and other painting materials was assessed using FLIM during and after varnish removal with different solvent application methods. Monitoring of the varnish removal process between successive solvent applications by a swab revealed an evolving image contrast as a function of the cleaning progress. FLIM of dammar and mastic resin varnishes identified characteristic changes to their fluorescence lifetimes depending on their ageing conditions. Thus, FLIM has a potential to become a powerful and versatile tool to visualise varnish removal from paintings.

Graphical Abstract

Metabolic response of microglia to amyloid deposition during Alzheimer's disease progression in a mouse model

Alzheimer's disease (AD) drives metabolic changes in the central nervous system (CNS). In AD microglia are activated and proliferate in response to amyloid β plaques. To further characterize the metabolic changes in microglia associated with plaque deposition in situ, we examined cortical tissue from 2, 4, and 8-month-old wild type and 5XFAD mice, a mouse model of plaque deposition. 5XFAD mice exhibited progressive microgliosis and plaque deposition as well as changes in microglial morphology and neuronal dystrophy. Multiphoton-based fluorescent lifetime imaging microscopy (FLIM) metabolic measurements showed that older mice had an increased amount of free NAD(P)H, indicative of a shift towards glycolysis. Interestingly in 5XFAD mice, we also found an abundant previously undescribed third fluorescence component that suggests an alternate NAD(P)H binding partner associated with pathology. This work demonstrates that FLIM in combination with other quantitative imaging methods, is a promising label-free tool for understanding the mechanisms of AD pathology.

The BrightEyes-TTM as an open-source time-tagging module for democratising single-photon microscopy

Fluorescence laser-scanning microscopy (LSM) is experiencing a revolution thanks to new single-photon (SP) array detectors, which give access to an entirely new set of single-photon information. Together with the blooming of new SP LSM techniques and the development of tailored SP array detectors, there is a growing need for (i) DAQ systems capable of handling the high-throughput and high-resolution photon information generated by these detectors, and (ii) incorporating these DAQ protocols in existing fluorescence LSMs. We developed an open-source, low-cost, multi-channel time-tagging module (TTM) based on a field-programmable gate array that can tag in parallel multiple single-photon events, with 30 ps precision, and multiple synchronisation events, with 4 ns precision. We use the TTM to demonstrate live-cell super-resolved fluorescence lifetime image scanning microscopy and fluorescence lifetime fluctuation spectroscopy. We expect that our BrightEyes-TTM will support the microscopy community in spreading SP-LSM in many life science laboratories.

A method for the fast and photon-efficient analysis of time-domain fluorescence lifetime image data over large dynamic ranges

Fluorescence lifetime imaging (FLIM) allows the quantification of sub‐cellular processes in situ, in living cells. A number of approaches have been developed to extract the lifetime from time‐domain FLIM data, but they are often limited in terms of speed, photon efficiency, precision or the dynamic range of lifetimes they can measure. Here, we focus on one of the best performing methods in the field, the centre‐of‐mass method (CMM), that conveys advantages in terms of speed and photon efficiency over others. In this paper, however, we identify a loss of photon efficiency of CMM for short lifetimes when background noise is present. We subsequently present a new development and generalization of CMM that provides for the rapid and accurate extraction of fluorescence lifetime over a large lifetime dynamic range. We provide software tools to simulate, validate and analyse FLIM data sets and compare the performance of our approach against the standard CMM and the commonly employed least‐square minimization (LSM) methods. Our method features a better photon efficiency than standard CMM and LSM and is robust in the presence of background noise. The algorithm is applicable to any time‐domain FLIM data set.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

COVID-19-Related Daily Stress Processes in College-Aged Adults: Examining the Role of Depressive Symptom Severity

Exposure to daily stressors specific to the COVID-19 pandemic (e.g., threat of infection) is associated with emotional distress, heightened stress reactivity, and increased depressive symptomology. Herein, we examined whether current depressive symptomology modulates the association between COVID-19-related daily stressor exposure and negative affective reactivity in young, otherwise healthy, college-aged adults. Fifty-eight adults (21 men; 22±3years) completed a daily web-based interview for eight consecutive days to assess COVID-19-related daily stress exposure and emotional responsiveness (September–November 2020). Depressive symptom severity was assessed using the Patient Health Questionnaire-9 (PHQ-9), and a score of ≥10 (range: 0–27) was used to define adults with a depressive episode (n=20). Participants reported at least one COVID-19-related stressor on 35.8% of interview days. Depressive symptomology did not predict the likelihood of exposure to a COVID-19-related stressor (p=0.46; OR=1.52; 95% CI: 0.492–4.718). However, negative affect (NA) was greater on days with an exposure to any COVID-19-specific daily stressor in adults with moderate-to-severe depressive symptoms (b=0.28, SE=0.093, p=0.003) but not in those without (b=0.009, SE=0.074, p=0.90), such that negative affective reactivity to COVID-19-related stressors was amplified in adults with a current depressive episode (p=0.019). Depressive symptomology did not moderate positive affective reactivity (p=0.686). Taken together, these data suggest that exposure to daily stressors related to COVID-19 further worsens NA in adults with a current depressive episode, potentially rendering them more susceptible to adverse mental health outcomes during the pandemic.

The Relationship Between Perceived Stress, State-Trait Anxiety, and Sleep Quality Among University Graduates in China During the COVID-19 Pandemic

The study aimed to investigate the relationship among perceived stress, state-trait anxiety, and sleep quality of graduates to provide a reference for improving their psychological status and attitude adjustment of job-searching during the COVID-19 pandemic. The research was conducted in a descriptive cross-sectional online survey between May 2020 and August 2020. The data were collected from 1,200 participants by using the personal information form prepared by the researchers in line with the literature, the Perceived Stress Scale, the State-Trait Anxiety Inventory, and the Pittsburgh Sleep Quality Index (PSQI). Among the surveyed participants, 47.67% were female, and 10.92% were medical students. The mean perceived stress, state anxiety, trait anxiety, and sleep quality were moderate and found as 31.4±6.69, 46.67±5.80, 49.45±5.54, and 5.94±2.47, respectively. The detection rates of state anxiety and trait anxiety were 48.63 and 49.50%, respectively. There was no significant difference in the detection rate of state anxiety and trait anxiety among different genders and majors (p >0.05). The detection rate of state anxiety and trait anxiety of rural family students was higher than that of urban family students (p <0.01). The score on the PSQI was positively associated with the scores on the perceived stress, state anxiety, and trait anxiety scales (p <0.001 for each model). Sleep quality was associated with increased perceived stress, state anxiety, and trait anxiety among graduates in China. Collectively, the study revealed the relationship between perceived stress, state-trait anxiety, and sleep quality among university graduates in China during the COVID-19 pandemic. Our results offer novel practical implications for all circles of the society to ensure students' health under the context of the COVID-19 epidemic.

New Extensibility and Scripting Tools in the ImageJ Ecosystem

ImageJ provides a framework for image processing across scientific domains while being fully open source. Over the years ImageJ has been substantially extended to support novel applications in scientific imaging as they emerge, particularly in the area of biological microscopy, with functionality made more accessible via the Fiji distribution of ImageJ. Within this software ecosystem, work has been done to extend the accessibility of ImageJ to utilize scripting, macros, and plugins in a variety of programming scenarios, e.g., from Groovy and Python and in Jupyter notebooks and cloud computing. We provide five protocols that demonstrate the extensibility of ImageJ for various workflows in image processing. We focus first on Fluorescence Lifetime Imaging Microscopy (FLIM) data, since this requires significant processing to provide quantitative insights into the microenvironments of cells. Second, we show how ImageJ can now be utilized for common image processing techniques, specifically image deconvolution and inversion, while highlighting the new, built-in features of ImageJ-particularly its capacity to run completely headless and the Ops matching feature that selects the optimal algorithm for a given function and data input, thereby enabling processing speedup. Collectively, these protocols can be used as a basis for automating biological image processing workflows. © 2021 Wiley Periodicals LLC. Basic Protocol 1: Using PyImageJ for FLIM data processing Alternate Protocol: Groovy FLIMJ in Jupyter Notebooks Basic Protocol 2: Using ImageJ Ops for image deconvolution Support Protocol 1: Using ImageJ Ops matching feature for image inversion Support Protocol 2: Headless ImageJ deconvolution.

Recent innovations in fluorescence lifetime imaging microscopy for biology and medicine

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) measures the decay rate of fluorophores, thus providing insights into molecular interactions. FLIM is a powerful molecular imaging technique that is widely used in biology and medicine.

AIM

This perspective highlights some of the major advances in FLIM instrumentation, analysis, and biological and clinical applications that we have found impactful over the last year.

APPROACH

Innovations in FLIM instrumentation resulted in faster acquisition speeds, rapid imaging over large fields of view, and integration with complementary modalities such as single-molecule microscopy or light-sheet microscopy. There were significant developments in FLIM analysis with machine learning approaches to enhance processing speeds, fit-free techniques to analyze images without a priori knowledge, and open-source analysis resources. The advantages and limitations of these recent instrumentation and analysis techniques are summarized. Finally, applications of FLIM in the last year include label-free imaging in biology, ophthalmology, and intraoperative imaging, FLIM of new fluorescent probes, and lifetime-based Förster resonance energy transfer measurements.

CONCLUSIONS

A large number of high-quality publications over the last year signifies the growing interest in FLIM and ensures continued technological improvements and expanding applications in biomedical research.

Joint regression-classification deep learning framework for analyzing fluorescence lifetime images using NADH and FAD

In this paper, we develop a deep neural network based joint classification-regression approach to identify microglia, a resident central nervous system macrophage, in the brain using fluorescence lifetime imaging microscopy (FLIM) data. Microglia are responsible for several key aspects of brain development and neurodegenerative diseases. Accurate detection of microglia is key to understanding their role and function in the CNS, and has been studied extensively in recent years. In this paper, we propose a joint classification-regression scheme that can incorporate fluorescence lifetime data from two different autofluorescent metabolic co-enzymes, FAD and NADH, in the same model. This approach not only represents the lifetime data more accurately but also provides the classification engine a more diverse data source. Furthermore, the two components of model can be trained jointly which combines the strengths of the regression and classification methods. We demonstrate the efficacy of our method using datasets generated using mouse brain tissue which show that our joint learning model outperforms results on the coenzymes taken independently, providing an efficient way to classify microglia from other cells.

Hyperdimensional Imaging Contrast Using an Optical Fiber

Fluorescence properties of a molecule can be used to study the structural and functional nature of biological processes. Physical properties, including fluorescence lifetime, emission spectrum, emission polarization, and others, help researchers probe a molecule, produce desired effects, and infer causes and consequences. Correlative imaging techniques such as hyperdimensional imaging microscopy (HDIM) combine the physical properties and biochemical states of a fluorophore. Here we present a fiber-based imaging system that can generate hyper-dimensional contrast by combining multiple fluorescence properties into a single fluorescence lifetime decay curve. Fluorescence lifetime imaging microscopy (FLIM) with controlled excitation polarization and temporally dispersed emission can generate a spectrally coded, polarization-filtered lifetime distribution for a pixel. This HDIM scheme generates a better contrast between different molecules than that from individual techniques. This setup uses only a single detector and is simpler to implement, modular, cost-efficient, and adaptable to any existing FLIM microscope. We present higher contrast data from Arabidopsis thaliana epidermal cells based on intrinsic anthocyanin emission properties under multiphoton excitation. This work lays the foundation for an alternative hyperdimensional imaging system and demonstrates that contrast-based imaging is useful to study cellular heterogeneity in biological samples.