Net-FLICS: fast quantitative wide-field fluorescence lifetime imaging with compressed sensing - a deep learning approach

Persistent Luminescence Lifetime-Based Near-Infrared Nanoplatform via Deep Learning for High-Fidelity Biosensing of Hypochlorite

In light of deep tissue penetration and ultralow background, near-infrared (NIR) persistent luminescence (PersL) bioprobes have become powerful tools for bioapplications. However, the inhomogeneous signal attenuation may significantly limit its application for precise biosensing owing to tissue absorption and scattering. In this work, a PersL lifetime-based nanoplatform via deep learning was proposed for high-fidelity bioimaging and biosensing in vivo. The persistent luminescence imaging network (PLI-Net), which consisted of a 3D-deep convolutional neural network (3D-CNN) and the PersL imaging system, was logically constructed to accurately extract the lifetime feature from the profile of PersL intensity-based decay images. Significantly, the NIR PersL nanomaterials represented by Zn1+xGa2-2xSnxO4: 0.4 % Cr (ZGSO) were precisely adjusted over their lifetime, enabling the PersL lifetime-based imaging with high-contrast signals. Inspired by the adjustable and reliable PersL lifetime imaging of ZGSO NPs, a proof-of-concept PersL nanoplatform was further developed and showed exceptional analytical performance for hypochlorite detection via a luminescence resonance energy transfer process. Remarkably, on the merits of the dependable and anti-interference PersL lifetimes, this PersL lifetime-based nanoprobe provided highly sensitive and accurate imaging of both endogenous and exogenous hypochlorite. This breakthrough opened up a new way for the development of high-fidelity biosensing in complex matrix systems.

3D convolutional neural networks predict cellular metabolic pathway use from fluorescence lifetime decay data

Fluorescence lifetime imaging of the co-enzyme reduced nicotinamide adenine dinucleotide (NADH) offers a label-free approach for detecting cellular metabolic perturbations. However, the relationships between variations in NADH lifetime and metabolic pathway changes are complex, preventing robust interpretation of NADH lifetime data relative to metabolic phenotypes. Here, a three-dimensional convolutional neural network (3D CNN) trained at the cell level with 3D NAD(P)H lifetime decay images (two spatial dimensions and one time dimension) was developed to identify metabolic pathway usage by cancer cells. NADH fluorescence lifetime images of MCF7 breast cancer cells with three isolated metabolic pathways, glycolysis, oxidative phosphorylation, and glutaminolysis were obtained by a multiphoton fluorescence lifetime microscope and then segmented into individual cells as the input data for the classification models. The 3D CNN models achieved over 90% accuracy in identifying cancer cells reliant on glycolysis, oxidative phosphorylation, or glutaminolysis. Furthermore, the model trained with human breast cancer cell data successfully predicted the differences in metabolic phenotypes of macrophages from control and POLG-mutated mice. These results suggest that the integration of autofluorescence lifetime imaging with 3D CNNs enables intracellular spatial patterns of NADH intensity and temporal dynamics of the lifetime decay to discriminate multiple metabolic phenotypes. Furthermore, the use of 3D CNNs to identify metabolic phenotypes from NADH fluorescence lifetime decay images eliminates the need for time- and expertise-demanding exponential decay fitting procedures. In summary, metabolic-prediction CNNs will enable live-cell and in vivo metabolic measurements with single-cell resolution, filling a current gap in metabolic measurement technologies.

Deep-LASI, single-molecule data analysis software

By avoiding ensemble averaging, single-molecule methods provide novel means of extracting mechanistic insights into function of material and molecules at the nanoscale. However, one of the big limitations is the vast amount of data required for analyzing and extracting the desired information, which is time-consuming and user dependent. Here, we introduce Deep-LASI, a software suite for the manual and automatic analysis of single-molecule traces, interactions, and the underlying kinetics. The software can handle data from one-, two- and three-color fluorescence data, and was particularly designed for the analysis of two- and three-color single-molecule fluorescence resonance energy transfer experiments. The functionalities of the software include: the registration of multiple-channels, trace sorting and categorization, determination of the photobleaching steps, calculation of fluorescence resonance energy transfer correction factors, and kinetic analyses based on hidden Markov modeling or deep neural networks. After a kinetic analysis, the ensuing transition density plots are generated, which can be used for further quantification of the kinetic parameters of the system. Each step in the workflow can be performed manually or with the support of machine learning algorithms. Upon reading in the initial data set, it is also possible to perform the remaining analysis steps automatically without additional supervision. Hence, the time dedicated to the analysis of single-molecule experiments can be reduced from days/weeks to minutes. After a thorough description of the functionalities of the software, we also demonstrate the capabilities of the software via the analysis of a previously published dynamic three-color DNA origami structure fluctuating between three states. With the drastic time reduction in data analysis, new types of experiments become realistically possible that complement our currently available palette of methodologies for investigating the nanoworld.

Coupling a recurrent neural network to SPAD TCSPC systems for real-time fluorescence lifetime imaging

Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, and robustness. Inspired by the concept of Edge Artificial Intelligence (Edge AI), we propose a robust approach that enables fast FLI with no degradation of accuracy. This approach couples a recurrent neural network (RNN), which is trained to estimate the fluorescence lifetime directly from raw timestamps without building histograms, to SPAD TCSPC systems, thereby drastically reducing transfer data volumes and hardware resource utilization, and enabling real-time FLI acquisition. We train two variants of the RNN on a synthetic dataset and compare the results to those obtained using center-of-mass method (CMM) and least squares fitting (LS fitting). Results demonstrate that two RNN variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are comparable to CMM and LS fitting in terms of accuracy, while outperforming them in the presence of background noise by a large margin. To explore the ultimate limits of the approach, we derive the Cramer-Rao lower bound of the measurement, showing that RNN yields lifetime estimations with near-optimal precision. To demonstrate real-time operation, we build a FLI microscope based on an existing SPAD TCSPC system comprising a 32[Formula: see text]32 SPAD sensor named Piccolo. Four quantized GRU cores, capable of processing up to 4 million photons per second, are deployed on the Xilinx Kintex-7 FPGA that controls the Piccolo. Powered by the GRU, the FLI setup can retrieve real-time fluorescence lifetime images at up to 10 frames per second. The proposed FLI system is promising and ideally suited for biomedical applications, including biological imaging, biomedical diagnostics, and fluorescence-assisted surgery, etc.

Applications of machine learning in time-domain fluorescence lifetime imaging: a review

Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.

Fast data fitting scheme for compressive multispectral fluorescence lifetime imaging

A single-pixel camera combined with compressive sensing techniques is a promising fluorescence microscope scheme for acquiring a multidimensional dataset (space, spectrum, and lifetime) and for reducing the measurement time with respect to conventional microscope schemes. However, upon completing the acquisition, a computational step is necessary for image reconstruction and data analysis, which can be time-consuming, potentially canceling out the beneficial effect of compressive sensing. In this work, we propose and experimentally validate a fast-fit workflow based on global analysis and multiple linear fits, which significantly reduces the computation time from tens of minutes to less than 1 s. Moreover, as the method is interlaced with the measurement flow, it can be applied in parallel with the acquisitions.

Molecularly Engineered Room-Temperature Phosphorescence for Biomedical Application: From the Visible toward Second Near-Infrared Window

Phosphorescence, characterized by luminescent lifetimes significantly longer than that of biological autofluorescence under ambient environment, is of great value for biomedical applications. Academic evidence of fluorescence imaging indicates that virtually all imaging metrics (sensitivity, resolution, and penetration depths) are improved when progressing into longer wavelength regions, especially the recently reported second near-infrared (NIR-II, 1000-1700 nm) window. Although the emission wavelength of probes does matter, it is not clear whether the guideline of "the longer the wavelength, the better the imaging effect" is still suitable for developing phosphorescent probes. For tissue-specific bioimaging, long-lived probes, even if they emit visible phosphorescence, enable accurate visualization of large deep tissues. For studies dealing with bioimaging of tiny biological architectures or dynamic physiopathological activities, the prerequisite is rigorous planning of long-wavelength phosphorescence, being aware of the cooperative contribution of long wavelengths and long lifetimes for improving the spatiotemporal resolution, penetration depth, and sensitivity of bioimaging. In this Review, emerging molecular engineering methods of room-temperature phosphorescence are discussed through the lens of photophysical mechanisms. We highlight the roles of phosphorescence with emission from visible to NIR-II windows toward bioapplications. To appreciate such advances, challenges and prospects in rapidly growing studies of room-temperature phosphorescence are described.

Building Fluorescence Lifetime Maps Photon-by-Photon by Leveraging Spatial Correlations

Fluorescence lifetime imaging microscopy (FLIM) has become a standard tool in the quantitative characterization of subcellular environments. However, quantitative FLIM analyses face several challenges. First, spatial correlations between pixels are often ignored as signal from individual pixels is analyzed independently thereby limiting spatial resolution. Second, existing methods deduce photon ratios instead of absolute lifetime maps. Next, the number of fluorophore species contributing to the signal is unknown, while excited state lifetimes with <1 ns difference are difficult to discriminate. Finally, existing analyses require high photon budgets and often cannot rigorously propagate experimental uncertainty into values over lifetime maps and number of species involved. To overcome all of these challenges simultaneously and self-consistently at once, we propose the first doubly nonparametric framework. That is, we learn the number of species (using Beta-Bernoulli process priors) and absolute maps of these fluorophore species (using Gaussian process priors) by leveraging information from pulses not leading to observed photon. We benchmark our framework using a broad range of synthetic and experimental data and demonstrate its robustness across a number of scenarios including cases where we recover lifetime differences between species as small as 0.3 ns with merely 1000 photons.

Deep learning-based fusion of widefield diffuse optical tomography and micro-CT structural priors for accurate 3D reconstructions

Widefield illumination and detection strategies leveraging structured light have enabled fast and robust probing of tissue properties over large surface areas and volumes. However, when applied to diffuse optical tomography (DOT) applications, they still require a time-consuming and expert-centric solving of an ill-posed inverse problem. Deep learning (DL) models have been recently proposed to facilitate this challenging step. Herein, we expand on a previously reported deep neural network (DNN) -based architecture (modified AUTOMAP - ModAM) for accurate and fast reconstructions of the absorption coefficient in 3D DOT based on a structured light illumination and detection scheme. Furthermore, we evaluate the improved performances when incorporating a micro-CT structural prior in the DNN-based workflow, named Z-AUTOMAP. This Z-AUTOMAP significantly improves the widefield imaging process's spatial resolution, especially in the transverse direction. The reported DL-based strategies are validated both in silico and in experimental phantom studies using spectral micro-CT priors. Overall, this is the first successful demonstration of micro-CT and DOT fusion using deep learning, greatly enhancing the prospect of rapid data-integration strategies, often demanded in challenging pre-clinical scenarios.

Fluorescence lifetime: Beating the IRF and interpulse window

Fluorescence lifetime imaging captures the spatial distribution of chemical species across cellular environments employing pulsed illumination confocal setups. However, quantitative interpretation of lifetime data continues to face critical challenges. For instance, fluorescent species with known in vitro excited-state lifetimes may split into multiple species with unique lifetimes when introduced into complex living environments. What is more, mixtures of species, which may be both endogenous and introduced into the sample, may exhibit 1) very similar lifetimes as well as 2) wide ranges of lifetimes including lifetimes shorter than the instrumental response function or whose duration may be long enough to be comparable to the interpulse window. By contrast, existing methods of analysis are optimized for well-separated and intermediate lifetimes. Here, we broaden the applicability of fluorescence lifetime analysis by simultaneously treating unknown mixtures of arbitrary lifetimes-outside the intermediate, Goldilocks, zone-for data drawn from a single confocal spot leveraging the tools of Bayesian nonparametrics (BNP). We benchmark our algorithm, termed BNP lifetime analysis, using a range of synthetic and experimental data. Moreover, we show that the BNP lifetime analysis method can distinguish and deduce lifetimes using photon counts as small as 500.

Nanosecond compressive fluorescence lifetime microscopy imaging via the RATS method with a direct reconstruction of lifetime maps

The RAndom Temporal Signals (RATS) method has proven to be a useful and versatile method for measuring photoluminescence (PL) dynamics and fluorescence lifetime imaging (FLIM). Here, we present two fundamental development steps in the method. First, we demonstrate that by using random digital laser modulation in RATS, it is possible to implement the measurement of PL dynamics with temporal resolution in units of nanoseconds. Secondly, we propose an alternative approach to evaluating FLIM measurements based on a single-pixel camera experiment. In contrast to the standard evaluation, which requires a lengthy iterative reconstruction of PL maps for each timepoint, here we use a limited set of predetermined PL lifetimes and calculate the amplitude maps corresponding to each lifetime. The alternative approach significantly saves post-processing time and, in addition, in a system with noise present, it shows better stability in terms of the accuracy of the FLIM spectrogram. Besides simulations that confirmed the functionality of the extension, we implemented the new advancements into a microscope optical setup for mapping PL dynamics on the micrometer scale. The presented principles were also verified experimentally by mapping a LuAG:Ce crystal surface.

Two-Photon Imaging for Non-Invasive Corneal Examination

Two-photon imaging (TPI) microscopy, namely, two-photon excited fluorescence (TPEF), fluorescence lifetime imaging (FLIM), and second-harmonic generation (SHG) modalities, has emerged in the past years as a powerful tool for the examination of biological tissues. These modalities rely on different contrast mechanisms and are often used simultaneously to provide complementary information on morphology, metabolism, and structural properties of the imaged tissue. The cornea, being a transparent tissue, rich in collagen and with several cellular layers, is well-suited to be imaged by TPI microscopy. In this review, we discuss the physical principles behind TPI as well as its instrumentation. We also provide an overview of the current advances in TPI instrumentation and image analysis. We describe how TPI can be leveraged to retrieve unique information on the cornea and to complement the information provided by current clinical devices. The present state of corneal TPI is outlined. Finally, we discuss the obstacles that must be overcome and offer perspectives and outlooks to make clinical TPI of the human cornea a reality.

Computational macroscopic lifetime imaging and concentration unmixing of autofluorescence

Single-pixel computational imaging can leverage highly sensitive detectors that concurrently acquire data across spectral and temporal domains. For molecular imaging, such methodology enables to collect rich intensity and lifetime multiplexed fluorescence datasets. Herein we report on the application of a single-pixel structured light-based platform for macroscopic imaging of tissue autofluorescence. The super-continuum visible excitation and hyperspectral single-pixel detection allow for parallel characterization of autofluorescence intensity and lifetime. Furthermore, we exploit a deep learning based data processing pipeline, to perform autofluorescence unmixing while yielding the autofluorophores' concentrations. The full scheme (setup and processing) is validated in silico and in vitro with clinically relevant autofluorophores flavin adenine dinucleotide, riboflavin, and protoporphyrin. The presented results demonstrate the potential of the methodology for macroscopically quantifying the intensity and lifetime of autofluorophores, with higher specificity for cases of mixed emissions, which are ubiquitous in autofluorescence and multiplexed in vivo imaging.

Suppression of natural lens fluorescence in fundus autofluorescence measurements: review of hardware solutions

Fluorescence lifetime imaging ophthalmoscopy (FLIO), a technique for investigating metabolic changes in the eye ground, can reveal the first signs of diseases related to metabolism. The fluorescence of the natural lens overlies the fundus fluorescence. Although the influence of natural lens fluorescence can be somewhat decreased with mathematical models, excluding this influence during the measurement by using hardware enables more exact estimation of the fundus fluorescence. Here, we analyze four 1-photon excitation hardware solutions to suppress the influence of natural lens fluorescence: aperture stop separation, confocal scanning laser ophthalmoscopy, combined confocal scanning laser ophthalmoscopy and aperture stop separation, and dual point confocal scanning laser ophthalmoscopy. The effect of each principle is demonstrated in examples. The best suppression is provided by the dual point principle, realized with a confocal scanning laser ophthalmoscope. In this case, in addition to the fluorescence of the whole eye, the fluorescence of the anterior part of the eye is detected from a non-excited spot of the fundus. The intensity and time-resolved fluorescence spectral data of the fundus are derived through the subtraction of the simultaneously measured fluorescence of the excited and non-excited spots. Advantages of future 2-photon fluorescence excitation are also discussed. This study provides the first quantitative evaluation of hardware principles to suppress the fluorescence of the natural lens during measurements of fundus autofluorescence.

Imaging approaches for monitoring three-dimensional cell and tissue culture systems

The past decade has seen an increasing demand for more complex, reproducible and physiologically relevant tissue cultures that can mimic the structural and biological features of living tissues. Monitoring the viability, development and responses of such tissues in real-time are challenging due to the complexities of cell culture physical characteristics and the environments in which these cultures need to be maintained in. Significant developments in optics, such as optical manipulation, improved detection and data analysis, have made optical imaging a preferred choice for many three-dimensional (3D) cell culture monitoring applications. The aim of this review is to discuss the challenges associated with imaging and monitoring 3D tissues and cell culture, and highlight topical label-free imaging tools that enable bioengineers and biophysicists to non-invasively characterise engineered living tissues.

Fast Analysis of Time-Domain Fluorescence Lifetime Imaging via Extreme Learning Machine

We present a fast and accurate analytical method for fluorescence lifetime imaging microscopy (FLIM), using the extreme learning machine (ELM). We used extensive metrics to evaluate ELM and existing algorithms. First, we compared these algorithms using synthetic datasets. The results indicate that ELM can obtain higher fidelity, even in low-photon conditions. Afterwards, we used ELM to retrieve lifetime components from human prostate cancer cells loaded with gold nanosensors, showing that ELM also outperforms the iterative fitting and non-fitting algorithms. By comparing ELM with a computational efficient neural network, ELM achieves comparable accuracy with less training and inference time. As there is no back-propagation process for ELM during the training phase, the training speed is much higher than existing neural network approaches. The proposed strategy is promising for edge computing with online training.

Framework for denoising Monte Carlo photon transport simulations using deep learning

SIGNIFICANCE

The Monte Carlo (MC) method is widely used as the gold-standard for modeling light propagation inside turbid media, such as human tissues, but combating its inherent stochastic noise requires one to simulate a large number photons, resulting in high computational burdens.

AIM

We aim to develop an effective image denoising technique using deep learning (DL) to dramatically improve the low-photon MC simulation result quality, equivalently bringing further acceleration to the MC method.

APPROACH

We developed a cascade-network combining DnCNN with UNet, while extending a range of established image denoising neural-network architectures, including DnCNN, UNet, DRUNet, and deep residual-learning for denoising MC renderings (ResMCNet), in handling three-dimensional MC data and compared their performances against model-based denoising algorithms. We also developed a simple yet effective approach to creating synthetic datasets that can be used to train DL-based MC denoisers.

RESULTS

Overall, DL-based image denoising algorithms exhibit significantly higher image quality improvements over traditional model-based denoising algorithms. Among the tested DL denoisers, our cascade network yields a 14 to 19 dB improvement in signal-to-noise ratio, which is equivalent to simulating 25  ×   to 78  ×   more photons. Other DL-based methods yielded similar results, with our method performing noticeably better with low-photon inputs and ResMCNet along with DRUNet performing better with high-photon inputs. Our cascade network achieved the highest quality when denoising complex domains, including brain and mouse atlases.

CONCLUSIONS

Incorporating state-of-the-art DL denoising techniques can equivalently reduce the computation time of MC simulations by one to two orders of magnitude. Our open-source MC denoising codes and data can be freely accessed at http://mcx.space/.

Monte Carlo-based data generation for efficient deep learning reconstruction of macroscopic diffuse optical tomography and topography applications

SIGNIFICANCE

Deep learning (DL) models are being increasingly developed to map sensor data to the image domain directly. However, DL methodologies are data-driven and require large and diverse data sets to provide robust and accurate image formation performances. For research modalities such as 2D/3D diffuse optical imaging, the lack of large publicly available data sets and the wide variety of instrumentation designs, data types, and applications leads to unique challenges in obtaining well-controlled data sets for training and validation. Meanwhile, great efforts over the last four decades have focused on developing accurate and computationally efficient light propagation models that are flexible enough to simulate a wide variety of experimental conditions.

AIM

Recent developments in Monte Carlo (MC)-based modeling offer the unique advantage of simulating accurately light propagation spatially, temporally, and over an extensive range of optical parameters, including minimally to highly scattering tissue within a computationally efficient platform. Herein, we demonstrate how such MC platforms, namely "Monte Carlo eXtreme" and "Mesh-based Monte Carlo," can be leveraged to generate large and representative data sets for training the DL model efficiently.

APPROACH

We propose data generator pipeline strategies using these platforms and demonstrate their potential in fluorescence optical topography, fluorescence optical tomography, and single-pixel diffuse optical tomography. These applications represent a large variety in instrumentation design, sample properties, and contrast function.

RESULTS

DL models trained using the MC-based in silico datasets, validated further with experimental data not used during training, show accurate and promising results.

CONCLUSION

Overall, these MC-based data generation pipelines are expected to support the development of DL models for rapid, robust, and user-friendly image formation in a wide variety of applications.

Deep learning-based image processing in optical microscopy

Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places.

Graphical abstract

High Resolution Fluorescence Lifetime Maps from Minimal Photon Counts

Fluorescence lifetime imaging microscopy (FLIM) may reveal subcellular spatial lifetime maps of key molecular species. Yet, such a quantitative picture of life necessarily demands high photon budgets at every pixel under the current analysis paradigm, thereby increasing acquisition time and photodamage to the sample. Motivated by recent developments in computational statistics, we provide a direct means to update our knowledge of the lifetime maps of species of different lifetimes from direct photon arrivals, while accounting for experimental features such as arbitrary forms of the instrument response function (IRF) and exploiting information from empty laser pulses not resulting in photon detection. Our ability to construct lifetime maps holds for arbitrary lifetimes, from short lifetimes (comparable to the IRF) to lifetimes exceeding interpulse times. As our method is highly data efficient, for the same amount of data normally used to determine lifetimes and photon ratios, working within the Bayesian paradigm, we report direct blind unmixing of lifetimes with subnanosecond resolution and subpixel spatial resolution using standard raster scan FLIM images. We demonstrate our method using a wide range of simulated and experimental data.

3D k-space reflectance fluorescence tomography via deep learning

We report on the potential to perform image reconstruction in 3D k-space reflectance fluorescence tomography (FT) using deep learning (DL). Herein, we adopt a modified AUTOMAP architecture and develop a training methodology that leverages an open-source Monte-Carlo-based simulator to generate a large dataset. Using an enhanced EMNIST (EEMNIST) dataset as an embedded contrast function allows us to train the network efficiently. The optical strategy utilizes k-space illumination in a reflectance configuration to probe tissue in the mesoscopic regime with high sensitivity and resolution. The proposed DL model training and validation is performed with both in silico data and a phantom experiment. Overall, our results indicate that the approach can correctly reconstruct both single and multiple fluorescent embedding(s) in a 3D volume. Furthermore, the presented technique is shown to outperform the traditional approaches [least-squares (LSQ) and total-variation minimization (TVAL)], especially at higher depths. We, therefore, expect the proposed computational technique to have future implications in preclinical studies.

Deep learning in macroscopic diffuse optical imaging

SIGNIFICANCE

Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.

AIM

We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).

APPROACH

First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.

RESULTS

The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.

CONCLUSIONS

The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells

Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool to quantify molecular compositions and study molecular states in complex cellular environment as the lifetime readings are not biased by fluorophore concentration or excitation power. However, the current methods to generate FLIM images are either computationally intensive or unreliable when the number of photons acquired at each pixel is low. Here we introduce a new deep learning-based method termed flimGANE (fluorescence lifetime imaging based on Generative Adversarial Network Estimation) that can rapidly generate accurate and high-quality FLIM images even in the photon-starved conditions. We demonstrated our model is up to 2,800 times faster than the gold standard time-domain maximum likelihood estimation (TD_MLE) and that flimGANE provides a more accurate analysis of low-photon-count histograms in barcode identification, cellular structure visualization, Förster resonance energy transfer characterization, and metabolic state analysis in live cells. With its advantages in speed and reliability, flimGANE is particularly useful in fundamental biological research and clinical applications, where high-speed analysis is critical.

Compressed sensing in fluorescence microscopy

Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.

Macroscopic Fluorescence Lifetime Imaging for Monitoring of Drug-Target Engagement

Precision medicine promises to improve therapeutic efficacy while reducing adverse effects, especially in oncology. However, despite great progresses in recent years, precision medicine for cancer treatment is not always part of routine care. Indeed, the ability to specifically tailor therapies to distinct patient profiles requires still significant improvements in targeted therapy development as well as decreases in drug treatment failures. In this regard, preclinical animal research is fundamental to advance our understanding of tumor biology, and diagnostic and therapeutic response. Most importantly, the ability to measure drug-target engagement accurately in live and intact animals is critical in guiding the development and optimization of targeted therapy. However, a major limitation of preclinical molecular imaging modalities is their lack of capability to directly and quantitatively discriminate between drug accumulation and drug-target engagement at the pathological site. Recently, we have developed Macroscopic Fluorescence Lifetime Imaging (MFLI) as a unique feature of optical imaging to quantitate in vivo drug-target engagement. MFLI quantitatively reports on nanoscale interactions via lifetime-sensing of Förster Resonance Energy Transfer (FRET) in live, intact animals. Hence, MFLI FRET acts as a direct reporter of receptor dimerization and target engagement via the measurement of the fraction of labeled-donor entity undergoing binding to its respective receptor. MFLI is expected to greatly impact preclinical imaging and also adjacent fields such as image-guided surgery and drug development.

3D denoised completion network for deep single-pixel reconstruction of hyperspectral images

Single-pixel imaging acquires an image by measuring its coefficients in a transform domain, thanks to a spatial light modulator. However, as measurements are sequential, only a few coefficients can be measured in the real-time applications. Therefore, single-pixel reconstruction is usually an underdetermined inverse problem that requires regularization to obtain an appropriate solution. Combined with a spectral detector, the concept of single-pixel imaging allows for hyperspectral imaging. While each channel can be reconstructed independently, we propose to exploit the spectral redundancy between channels to regularize the reconstruction problem. In particular, we introduce a denoised completion network that includes 3D convolution filters. Contrary to black-box approaches, our network combines the classical Tikhonov theory with the deep learning methodology, leading to an explainable network. Considering both simulated and experimental data, we demonstrate that the proposed approach yields hyperspectral images with higher quantitative metrics than the approaches developed for grayscale images.

Multivariate Curve Resolution Slicing of Multiexponential Time-Resolved Spectroscopy Fluorescence Data

Time-resolved fluorescence spectroscopy (TRFS), i.e., measurement of fluorescence decay curves for different excitation and/or emission wavelengths, provides specific and sensitive local information on molecules and on their environment. However, TRFS relies on multiexponential data fitting to derive fluorescence lifetimes from the measured decay curves and the time resolution of the technique is limited by the instrumental response function (IRF). We propose here a multivariate curve resolution (MCR) approach based on data slicing to perform tailored and fit-free analysis of multiexponential fluorescence decay curves. MCR slicing, taking as a basic framework the multivariate curve resolution-alternating least-squares (MCR-ALS) soft-modeling algorithm, relies on a hybrid bilinear/trilinear data decomposition. A key feature of the method is that it enables the recovery of individual components characterized by decay profiles that are only partially describable by monoexponential functions. For TRFS data, not only pure multiexponential tail information but also shorter time delay information can be decomposed, where the signal deviates from the ideal exponential behavior due to the limited time resolution. The accuracy of the proposed approach is validated by analyzing mixtures of three commercial dyes and characterizing the mixture composition, lifetimes, and associated contributions, even in situations where only ternary mixture samples are available. MCR slicing is also applied to the analysis of TRFS data obtained on a photoswitchable fluorescent protein (rsEGFP2). Three fluorescence lifetimes are extracted, along with the profile of the IRF, highlighting that decomposition of complex systems, for which individual isomers are characterized by different exponential decays, can also be achieved.

Robustness and accuracy improvement of data processing with 2D neural networks for transient absorption dynamics

To achieve the goal of efficiently analyzing transient absorption spectra without arbitrary assumption and to overcome the limitations of conventional methods in fitting ability and highly noised backgrounds, it is essential to develop new tools to achieve more accurate and robust prediction based on the intrinsic properties of a spectrum even under strong noise. In this work, Lasso regression and neural network were combined to achieve an effective fitting. Compared to the conventional global fitting method, our network could automatically determine the exponential form on each wave unit, in which the accuracy was as high as 97%. Thereafter, the lifetime with the corresponding amplitude ratio could be easily predicted by the neural network on each wave unit. This kind of prediction is difficult to achieve by global fitting due to the limitation of computational resources. Furthermore, more accurate fitting even under weak signals could be achieved for the mean square error (MSE) decreasing by more than 100 times on average compared to conventional global fitting methods. Attributed to its improved accuracy and robustness, our developed algorithm could be readily applied to analyze time-resolved transient spectra.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Mesoscopic fluorescence lifetime imaging: Fundamental principles, clinical applications and future directions

Fluorescence lifetime imaging (FLIm) is an optical spectroscopic imaging technique capable of real-time assessments of tissue properties in clinical settings. Label-free FLIm is sensitive to changes in tissue structure and biochemistry resulting from pathological conditions, thus providing optical contrast to identify and monitor the progression of disease. Technical and methodological advances over the last two decades have enabled the development of FLIm instrumentation for real-time, in situ, mesoscopic imaging compatible with standard clinical workflows. Herein, we review the fundamental working principles of mesoscopic FLIm, discuss the technical characteristics of current clinical FLIm instrumentation, highlight the most commonly used analytical methods to interpret fluorescence lifetime data and discuss the recent applications of FLIm in surgical oncology and cardiovascular diagnostics. Finally, we conclude with an outlook on the future directions of clinical FLIm.

Single-pixel image reconstruction from experimental data using neural networks

Single-pixel cameras that measure image coefficients have various promising applications, in particular for hyper-spectral imaging. Here, we investigate deep neural networks that when fed with experimental data can output high-quality images in real time. Assuming that the measurements are corrupted by mixed Poisson-Gaussian noise, we propose to map the raw data from the measurement domain to the image domain based on a Tikhonov regularization. This step can be implemented as the first layer of a deep neural network, followed by any architecture of layers that acts in the image domain. We also describe a framework for training the network in the presence of noise. In particular, our approach includes an estimation of the image intensity and experimental parameters, together with a normalization scheme that allows varying noise levels to be handled during training and testing. Finally, we present results from simulations and experimental acquisitions with varying noise levels. Our approach yields images with improved peak signal-to-noise ratios, even for noise levels that were foreseen during the training of the networks, which makes the approach particularly suitable to deal with experimental data. Furthermore, while this approach focuses on single-pixel imaging, it can be adapted for other computational optics problems.

Luminescence lifetime imaging of three-dimensional biological objects

A major focus of current biological studies is to fill the knowledge gaps between cell, tissue and organism scales. To this end, a wide array of contemporary optical analytical tools enable multiparameter quantitative imaging of live and fixed cells, three-dimensional (3D) systems, tissues, organs and organisms in the context of their complex spatiotemporal biological and molecular features. In particular, the modalities of luminescence lifetime imaging, comprising fluorescence lifetime imaging (FLI) and phosphorescence lifetime imaging microscopy (PLIM), in synergy with Förster resonance energy transfer (FRET) assays, provide a wealth of information. On the application side, the luminescence lifetime of endogenous molecules inside cells and tissues, overexpressed fluorescent protein fusion biosensor constructs or probes delivered externally provide molecular insights at multiple scales into protein-protein interaction networks, cellular metabolism, dynamics of molecular oxygen and hypoxia, physiologically important ions, and other physical and physiological parameters. Luminescence lifetime imaging offers a unique window into the physiological and structural environment of cells and tissues, enabling a new level of functional and molecular analysis in addition to providing 3D spatially resolved and longitudinal measurements that can range from microscopic to macroscopic scale. We provide an overview of luminescence lifetime imaging and summarize key biological applications from cells and tissues to organisms.

A Review of New High-Throughput Methods Designed for Fluorescence Lifetime Sensing From Cells and Tissues

Though much of the interest in fluorescence in the past has been on measuring spectral qualities such as wavelength and intensity, there are two other highly useful intrinsic properties of fluorescence: lifetime (or decay) and anisotropy (or polarization). Each has its own set of unique advantages, limitations, and challenges in detection when it comes to use in biological studies. This review will focus on the property of fluorescence lifetime, providing a brief background on instrumentation and theory, and examine the recent advancements and applications of measuring lifetime in the fields of high-throughput fluorescence lifetime imaging microscopy (HT-FLIM) and time-resolved flow cytometry (TRFC). In addition, the crossover of these two methods and their outlooks will be discussed.

Deep Learning for Biospectroscopy and Biospectral Imaging: State-of-the-Art and Perspectives

With the advances in instrumentation and sampling techniques, there is an explosive growth of data from molecular and cellular samples. The call to extract more information from the large data sets has greatly challenged the conventional chemometrics method. Deep learning, which utilizes very large data sets for finding hidden features therein and for making accurate predictions for a wide range of applications, has been applied in an unbelievable pace in biospectroscopy and biospectral imaging in the recent 3 years. In this Feature, we first introduce the background and basic knowledge of deep learning. We then focus on the emerging applications of deep learning in the data preprocessing, feature detection, and modeling of the biological samples for spectral analysis and spectroscopic imaging. Finally, we highlight the challenges and limitations in deep learning and the outlook for future directions.

Hollow molecularly imprinted fluorescent sensor using europium complex as functional monomer for the detection of trace 2,4,6-trichlorophenol in real water samples

As an important environmental indicator, 2,4,6-trichlorophenol (2,4,6-TCP) was proved extremely harmful to human body. In this article, hollow molecularly imprinted fluorescent polymers (@MIPs) for the selective detection of 2,4,6-TCP were devised and fabricated by sacrificial skeleton method based on SiO₂ nanoparticles. As the most innovation, highly luminescent europium complex Eu(MAA)₃phen played the role of both fluorophores and functional monomers of the MIPs. The obtained @MIPs showed monodispersity and the average particle size was around 130 nm. It had a linear fluorescent response within the concentration range 10-100 nmol L^-1 with the correlation coefficient calculated as 0.99625, and the limit of detection was identified as 2.41 nmol L^-1. The results show that Eu(MAA)₃phen as a fluorophore has high luminescent properties, and as a functional monomer, it can improve the selectivity and anti-interference performance of MIPs. Furthermore, the hollow structure made it possible that the imprinted specific recognition sites distributed on both inner and outer surfaces of @MIPs. The experimental results showed that these @MIPs could be employed to the selective detection of chlorophenols under low concentration. And this work will provide a reference for further optimization of fluorescent imprinted sensors.

Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation

Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080-human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.

High compression deep learning based single-pixel hyperspectral macroscopic fluorescence lifetime imaging in vivo

Single pixel imaging frameworks facilitate the acquisition of high-dimensional optical data in biological applications with photon starved conditions. However, they are still limited to slow acquisition times and low pixel resolution. Herein, we propose a convolutional neural network for fluorescence lifetime imaging with compressed sensing at high compression (NetFLICS-CR), which enables in vivo applications at enhanced resolution, acquisition and processing speeds, without the need for experimental training datasets. NetFLICS-CR produces intensity and lifetime reconstructions at 128 × 128 pixel resolution over 16 spectral channels while using only up to 1% of the required measurements, therefore reducing acquisition times from ∼2.5 hours at 50% compression to ∼3 minutes at 99% compression. Its potential is demonstrated in silico, in vitro and for mice in vivo through the monitoring of receptor-ligand interactions in liver and bladder and further imaging of intracellular delivery of the clinical drug Trastuzumab to HER2-positive breast tumor xenografts. The data acquisition time and resolution improvement through NetFLICS-CR, facilitate the translation of single pixel macroscopic flurorescence lifetime imaging (SP-MFLI) for in vivo monitoring of lifetime properties and drug uptake.

Deep learning model for ultrafast quantification of blood flow in diffuse correlation spectroscopy

Diffuse correlation spectroscopy (DCS) is increasingly used in the optical imaging field to assess blood flow in humans due to its non-invasive, real-time characteristics and its ability to provide label-free, bedside monitoring of blood flow changes. Previous DCS studies have utilized a traditional curve fitting of the analytical or Monte Carlo models to extract the blood flow changes, which are computationally demanding and less accurate when the signal to noise ratio decreases. Here, we present a deep learning model that eliminates this bottleneck by solving the inverse problem more than 2300% faster, with equivalent or improved accuracy compared to the nonlinear fitting with an analytical method. The proposed deep learning inverse model will enable real-time and accurate tissue blood flow quantification with the DCS technique.

Macroscopic fluorescence lifetime topography enhanced via spatial frequency domain imaging

We report on a macroscopic fluorescence lifetime imaging (MFLI) topography computational framework based around machine learning with the main goal of retrieving the depth of fluorescent inclusions deeply seated in bio-tissues. This approach leverages the depth-resolved information inherent to time-resolved fluorescence data sets coupled with the retrieval of in situ optical properties as obtained via spatial frequency domain imaging (SFDI). Specifically, a Siamese network architecture is proposed with optical properties (OPs) and time-resolved fluorescence decays as input followed by simultaneous retrieval of lifetime maps and depth profiles. We validate our approach using comprehensive in silico data sets as well as with a phantom experiment. Overall, our results demonstrate that our approach can retrieve the depth of fluorescence inclusions, especially when coupled with optical properties estimation, with high accuracy. We expect the presented computational approach to find great utility in applications such as optical-guided surgery.

An Adaptive Feature Selection Algorithm for Fuzzy Clustering Image Segmentation Based on Embedded Neighbourhood Information Constraints

This paper addresses the lack of robustness of feature selection algorithms for fuzzy clustering segmentation with the Gaussian mixture model. Assuming that the neighbourhood pixels and the centre pixels obey the same distribution, a Markov method is introduced to construct the prior probability distribution and achieve the membership degree regularisation constraint for clustering sample points. Then, a noise smoothing factor is introduced to optimise the prior probability constraint. Second, a power index is constructed by combining the classification membership degree and prior probability since the Kullback–Leibler (KL) divergence of the noise smoothing factor is used to supervise the prior probability; this probability is embedded into Fuzzy Superpixels Fuzzy C-means (FSFCM) as a regular factor. This paper proposes a fuzzy clustering image segmentation algorithm based on an adaptive feature selection Gaussian mixture model with neighbourhood information constraints. To verify the segmentation performance and anti-noise robustness of the improved algorithm, the fuzzy C-means clustering algorithm Fuzzy C-means (FCM), FSFCM, Spatially Variant Finite Mixture Model (SVFMM), EGFMM, extended Gaussian mixture model (EGMM), adaptive feature selection robust fuzzy clustering segmentation algorithm (AFSFCM), fast and robust spatially constrained Gaussian mixture model (GMM) for image segmentation (FRSCGMM), and improve method are used to segment grey images containing Gaussian noise, salt-and-pepper noise, multiplicative noise and mixed noise. The peak signal-to-noise ratio (PSNR) and the error rate (MCR) are used as the theoretical basis for assessing the segmentation results. The improved algorithm indicators proposed in this paper are optimised. The improved algorithm yields increases of 0.1272–12.9803 dB, 1.5501–13.4396 dB, 1.9113–11.2613 dB and 1.0233–10.2804 dB over the other methods, and the Misclassification rate (MSR) decreases by 0.32–37.32%, 5.02–41.05%, 0.3–21.79% and 0.9–30.95% compared to that with the other algorithms. It is verified that the segmentation results of the improved algorithm have good regional consistency and strong anti-noise robustness, and they meet the needs of noisy image segmentation.

UNMIX-ME: spectral and lifetime fluorescence unmixing via deep learning

Hyperspectral fluorescence lifetime imaging allows for the simultaneous acquisition of spectrally resolved temporal fluorescence emission decays. In turn, the acquired rich multidimensional data set enables simultaneous imaging of multiple fluorescent species for a comprehensive molecular assessment of biotissues. However, to enable quantitative imaging, inherent spectral overlap between the considered fluorescent probes and potential bleed-through must be considered. Such a task is performed via either spectral or lifetime unmixing, typically independently. Herein, we present "UNMIX-ME" (unmix multiple emissions), a deep learning-based fluorescence unmixing routine, capable of quantitative fluorophore unmixing by simultaneously using both spectral and temporal signatures. UNMIX-ME was trained and validated using an in silico framework replicating the data acquisition process of a compressive hyperspectral fluorescent lifetime imaging platform (HMFLI). It was benchmarked against a conventional LSQ method for tri and quadri-exponential simulated samples. Last, UNMIX-ME's potential was assessed for NIR FRET in vitro and in vivo preclinical applications.

Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to distinguish the unique molecular environment of fluorophores. FLIM measures the time a fluorophore remains in an excited state before emitting a photon, and detects molecular variations of fluorophores that are not apparent with spectral techniques alone. FLIM is sensitive to multiple biomedical processes including disease progression and drug efficacy.

AIM

We provide an overview of FLIM principles, instrumentation, and analysis while highlighting the latest developments and biological applications.

APPROACH

This review covers FLIM principles and theory, including advantages over intensity-based fluorescence measurements. Fundamentals of FLIM instrumentation in time- and frequency-domains are summarized, along with recent developments. Image segmentation and analysis strategies that quantify spatial and molecular features of cellular heterogeneity are reviewed. Finally, representative applications are provided including high-resolution FLIM of cell- and organelle-level molecular changes, use of exogenous and endogenous fluorophores, and imaging protein-protein interactions with Förster resonance energy transfer (FRET). Advantages and limitations of FLIM are also discussed.

CONCLUSIONS

FLIM is advantageous for probing molecular environments of fluorophores to inform on fluorophore behavior that cannot be elucidated with intensity measurements alone. Development of FLIM technologies, analysis, and applications will further advance biological research and clinical assessments.

A Local Neighborhood Robust Fuzzy Clustering Image Segmentation Algorithm Based on an Adaptive Feature Selection Gaussian Mixture Model

Since the fuzzy local information C-means (FLICM) segmentation algorithm cannot take into account the impact of different features on clustering segmentation results, a local fuzzy clustering segmentation algorithm based on a feature selection Gaussian mixture model was proposed. First, the constraints of the membership degree on the spatial distance were added to the local information function. Second, the feature saliency was introduced into the objective function. By using the Lagrange multiplier method, the optimal expression of the objective function was solved. Neighborhood weighting information was added to the iteration expression of the classification membership degree to obtain a local feature selection based on feature selection. Each of the improved FLICM algorithm, the fuzzy C-means with spatial constraints (FCM_S) algorithm, and the original FLICM algorithm were then used to cluster and segment the interference images of Gaussian noise, salt-and-pepper noise, multiplicative noise, and mixed noise. The performances of the peak signal-to-noise ratio and error rate of the segmentation results were compared with each other. At the same time, the iteration time and number of iterations used to converge the objective function of the algorithm were compared. In summary, the improved algorithm significantly improved the ability of image noise suppression under strong noise interference, improved the efficiency of operation, facilitated remote sensing image capture under strong noise interference, and promoted the development of a robust anti-noise fuzzy clustering algorithm.

Fast fit-free analysis of fluorescence lifetime imaging via deep learning

Fluorescence lifetime imaging (FLI) provides unique quantitative information in biomedical and molecular biology studies but relies on complex data-fitting techniques to derive the quantities of interest. Herein, we propose a fit-free approach in FLI image formation that is based on deep learning (DL) to quantify fluorescence decays simultaneously over a whole image and at fast speeds. We report on a deep neural network (DNN) architecture, named fluorescence lifetime imaging network (FLI-Net) that is designed and trained for different classes of experiments, including visible FLI and near-infrared (NIR) FLI microscopy (FLIM) and NIR gated macroscopy FLI (MFLI). FLI-Net outputs quantitatively the spatially resolved lifetime-based parameters that are typically employed in the field. We validate the utility of the FLI-Net framework by performing quantitative microscopic and preclinical lifetime-based studies across the visible and NIR spectra, as well as across the 2 main data acquisition technologies. These results demonstrate that FLI-Net is well suited to accurately quantify complex fluorescence lifetimes in cells and, in real time, in intact animals without any parameter settings. Hence, FLI-Net paves the way to reproducible and quantitative lifetime studies at unprecedented speeds, for improved dissemination and impact of FLI in many important biomedical applications ranging from fundamental discoveries in molecular and cellular biology to clinical translation.

Emerging role of machine learning in light-matter interaction

Machine learning has provided a huge wave of innovation in multiple fields, including computer vision, medical diagnosis, life sciences, molecular design, and instrumental development. This perspective focuses on the implementation of machine learning in dealing with light-matter interaction, which governs those fields involving materials discovery, optical characterizations, and photonics technologies. We highlight the role of machine learning in accelerating technology development and boosting scientific innovation in the aforementioned aspects. We provide future directions for advanced computing techniques via multidisciplinary efforts that can help to transform optical materials into imaging probes, information carriers and photonics devices.