Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging

Net-FLICS: fast quantitative wide-field fluorescence lifetime imaging with compressed sensing - a deep learning approach

Macroscopic fluorescence lifetime imaging (MFLI) via compressed sensed (CS) measurements enables efficient and accurate quantification of molecular interactions in vivo over a large field of view (FOV). However, the current data-processing workflow is slow, complex and performs poorly under photon-starved conditions. In this paper, we propose Net-FLICS, a novel image reconstruction method based on a convolutional neural network (CNN), to directly reconstruct the intensity and lifetime images from raw time-resolved CS data. By carefully designing a large simulated dataset, Net-FLICS is successfully trained and achieves outstanding reconstruction performance on both in vitro and in vivo experimental data and even superior results at low photon count levels for lifetime quantification.

Single-pixel phase and fluorescence microscope

Multimodal microscopes either use multiple cameras or a single camera to multiplex different modes spatially. The former needs expertise demanding alignment and the latter suffers from limited spatial resolution. Here, we report an alignment-free full-resolution simultaneous fluorescence and phase imaging approach using single-pixel detectors. By combining reference-free interferometry with single-pixel imaging scheme, we employ structured illumination to encode the phase and fluorescence of the sample into two single-pixel detection arms, and then conduct reconstruction computationally from the illumination patterns and recorded correlated measurements. The recovered fluorescence and phase images are inherently aligned thanks to single-pixel imaging scheme. To validate the proposed method, we built a proof-of-concept setup for first imaging the phase of an etched glass with given etching depth and then imaging the phase and fluorescence of the quantum dot sample. This method holds great potential for multispectral fluorescence microscopy with additional single-pixel detectors or a spectrometer. Besides, this cost-efficient multimodal system might find broad applications in biomedical science and material science.

Hyperspectral wide-field time domain single-pixel diffuse optical tomography platform

We present the design and comprehensive instrumental characterization of a time domain diffuse optical tomography (TD-DOT) platform based on wide-field illumination and wide-field hyperspectral time-resolved single-pixel detection for functional and molecular imaging in turbid media. The proposed platform combines two digital micro-mirror devices (DMDs) to generate structured light and a spectrally resolved multi-anode photomultiplier tube (PMT) detector in time domain for hyperspectral data acquisition over 16 wavelength channels based on the time-correlated single-photon counting (TCSPC) technique. The design of the proposed platform is described in detail and its characteristics in spatial, temporal and spectral dimensions are calibrated and presented. The performance of the system is further validated through a phantom study where two absorbers in glass tubes with spectral contrast are mapped in a turbid medium of ~20 mm thickness. The method presented here offers the potential of accelerating the imaging process and improving reconstruction results in TD-DOT and thus facilitates its wide spread use in preclinical and clinical in vivo imaging scenarios.

Direct approach to compute Jacobians for diffuse optical tomography using perturbation Monte Carlo-based photon "replay"

Perturbation Monte Carlo (pMC) has been previously proposed to rapidly recompute optical measurements when small perturbations of optical properties are considered, but it was largely restricted to changes associated with prior tissue segments or regions-of-interest. In this work, we expand pMC to compute spatially and temporally resolved sensitivity profiles, i.e. the Jacobians, for diffuse optical tomography (DOT) applications. By recording the pseudo random number generator (PRNG) seeds of each detected photon, we are able to "replay" all detected photons to directly create the 3D sensitivity profiles for both absorption and scattering coefficients. We validate the replay-based Jacobians against the traditional adjoint Monte Carlo (aMC) method, and demonstrate the feasibility of using this approach for efficient 3D image reconstructions using in vitro hyperspectral wide-field DOT measurements. The strengths and limitations of the replay approach regarding its computational efficiency and accuracy are discussed, in comparison with aMC, for point-detector systems as well as wide-field pattern-based and hyperspectral imaging systems. The replay approach has been implemented in both of our open-source MC simulators - MCX and MMC (http://mcx.space).

Comparison of illumination geometry for lifetime-based measurements in whole-body preclinical imaging

Macroscopic fluorescence lifetime imaging (MFLI) has been proved to be an accurate tool to quantify Förster resonance energy transfer (FRET) lifetime-based assessment of receptor-ligand engagement in vitro and in vivo. Herein, we report on the quantitative comparison of MFLI for whole-body preclinical studies in transmittance and reflectance geometries. The comparative study was conducted for both in vitro and in vivo conditions. FRET quantification performance in both geometries was similar in high fluorescence concentration samples. However, the reflectance geometry performed better at low fluorescence concentration. In addition, reflectance geometry could acquire subsurface imaging of the main whole-body organs of small animals without being compromised by tissue attenuation.

Quantitative imaging of receptor-ligand engagement in intact live animals

Maintaining an intact tumor environment is critical for quantitation of receptor-ligand engagement in a targeted drug development pipeline. However, measuring receptor-ligand engagement in vivo and non-invasively in preclinical settings is extremely challenging. We found that quantitation of intracellular receptor-ligand binding can be achieved using whole-body macroscopic lifetime-based Förster Resonance Energy Transfer (FRET) imaging in intact, live animals bearing tumor xenografts. We determined that FRET levels report on ligand binding to transferrin receptors conversely to raw fluorescence intensity. FRET levels in heterogeneous tumors correlate with intracellular ligand binding but strikingly, not with ubiquitously used ex vivo receptor expression assessment. Hence, MFLI-FRET provides a direct measurement of systemic delivery, target availability and intracellular drug delivery in preclinical studies. Here, we have used MFLI to measure FRET longitudinally in intact and live animals. MFLI-FRET is well-suited for guiding the development of targeted drug therapy in heterogeneous tumors in intact, live small animals.

Assessing patterns for compressive fluorescence lifetime imaging

A novel hyperspectral single pixel system was used to compare different compressive basis patterns for intensity imaging, lifetime imaging, and FRET quantification. Six popular basis patterns were compared experimentally in a phantom containing two fluorescent dyes. The basis patterns that performed best for lifetime quantification were used to measure FRET occurrence in well-plate samples with varying acceptor-donor ratios. The ABS-WP approach using Haar patterns and the compressive sensing approach with Hadamard Ranked patterns displayed the best overall performances at a 50% compression ratio.

Review of structured light in diffuse optical imaging

Diffuse optical imaging probes deep living tissue enabling structural, functional, metabolic, and molecular imaging. Recently, due to the availability of spatial light modulators, wide-field quantitative diffuse optical techniques have been implemented, which benefit greatly from structured light methodologies. Such implementations facilitate the quantification and characterization of depth-resolved optical and physiological properties of thick and deep tissue at fast acquisition speeds. We summarize the current state of work and applications in the three main techniques leveraging structured light: spatial frequency-domain imaging, optical tomography, and single-pixel imaging. The theory, measurement, and analysis of spatial frequency-domain imaging are described. Then, advanced theories, processing, and imaging systems are summarized. Preclinical and clinical applications on physiological measurements for guidance and diagnosis are summarized. General theory and method development of tomographic approaches as well as applications including fluorescence molecular tomography are introduced. Lastly, recent developments of single-pixel imaging methodologies and applications are reviewed.

Real-time single-pixel video imaging with Fourier domain regularization

We present a closed-form image reconstruction method for single-pixel imaging based on the generalized inverse of the measurement matrix. Its numerical cost scales proportionally with the number of measured samples. Regularization of the inverse problem is obtained by minimizing the norms of the convolution between the reconstructed image and a set of spatial filters. The final reconstruction formula can be expressed in terms of matrix pseudoinverse. At high compression, this approach is an interesting alternative to the methods of compressive sensing based on l1-norm optimization, which are too slow for real-time applications. For instance, we demonstrate experimental single-pixel detection with real-time reconstruction obtained in parallel with measurement at a frame rate of 11 Hz for highly compressive measurements with a resolution of 256 × 256. To this end, we preselect the sampling functions to match the average spectrum obtained with an image database. The sampling functions are selected from the Walsh-Hadamard basis, from the discrete cosine basis, or from a subset of Morlet wavelets convolved with white noise. We show that by incorporating the quadratic criterion into the closed-form reconstruction formula, we can use binary rather than continuous sampling and reach similar reconstruction quality as is obtained by minimizing the total variation. This makes it possible to use cosine- or Morlet-based sampling with digital micromirror devices without advanced binarization methods.

Improving mesoscopic fluorescence molecular tomography via preconditioning and regularization

Mesoscopic fluorescence molecular tomography (MFMT) is a novel imaging technique capable of obtaining 3-D distribution of molecular probes inside biological tissues at depths of a few millimeters with a resolution up to ~100 μm. However, the ill-conditioned nature of the MFMT inverse problem severely deteriorates its reconstruction performances. Furthermore, dense spatial sampling and fine discretization of the imaging volume required for high resolution reconstructions make the sensitivity matrix (Jacobian) highly correlated, which prevents even advanced algorithms from achieving optimal solutions. In this work, we propose two computational methods to respectively increase the incoherence of the sensitivity matrix and improve the convergence rate of the inverse solver. We first apply a compressed sensing (CS) based preconditioner on either the whole sensitivity matrix or sub sensitivity matrices to reduce the coherence between columns of the sensitivity matrix. Then we employed a regularization method based on the weight iterative improvement method (WIIM) to mitigate the ill-condition of the sensitivity matrix and to drive the iterative optimization process towards convergence at a faster rate. We performed numerical simulations and phantom experiments to validate the effectiveness of the proposed strategies. In both in silico and in vitro cases, we were able to improve the quality of MFMT reconstructions significantly.

Time-resolved multispectral imaging based on an adaptive single-pixel camera

Time-resolved multispectral imaging has many applications in different fields, which range from characterization of biological tissues to environmental monitoring. In particular, optical techniques, such as lidar and fluorescence lifetime imaging, require imaging at the subnanosecond scales over an extended area. In this paper, we demonstrate experimentally a time-resolved multispectral acquisition scheme based on single-pixel imaging. Single-pixel imaging is an emerging paradigm that provides low-cost high-quality images. Here, we use an adaptive strategy that allows acquisition and image reconstruction times to be reduced drastically or full basis scans. Adaptive time-resolved multispectral imaging scheme can have significant applications in biological imaging, at scales from macroscopic to microscopic.

Doubling the pixel count limitation of single-pixel imaging via sinusoidal amplitude modulation

We demonstrate a single-pixel imaging (SPI) method that can achieve pixel resolution beyond the physical limitation of the spatial light modulator (SLM), by adopting sinusoidal amplitude modulation and frequency filtering. Through light field analysis, we observe that the induced intensity with a squared value of the amplitude contains higher frequency components. By filtering out the zero frequency of the sinusoidal amplitude in the Fourier domain, we can separate out the higher frequency components, which enables SPI with higher resolving ability and thus beyond the limitation of the SLM. Further, to address the speed issue in grayscale spatial light modulation, we propose a fast implementation scheme with tens-of-kilohertz refresh rate. Specifically, we use a digital micromirror device (DMD) working at the full frame rate to conduct binarized sinusoidal patterning in the spatial domain and pinhole filtering eliminating the binarization error in the Fourier domain. For experimental validation, we build a single-pixel microscope to retrieve 1200 × 1200-pixel images via a sub-megapixel DMD, and the setup achieves comparable performance to array sensor microscopy and provides additional sectioning ability.

Multiplexed fluorescence lifetime imaging by concentration-dependent quenching

This study sought to use the undesirable concentration-dependent quenching to propose a simple multiplexed imaging analysis for histopathological identification of different stained tissues. To verify this point, the relationship between the fluorescence lifetime and eosin concentration was obtained. At low concentrations, the fluorescence lifetimes of eosin were independent of the concentration (<0.25 μg ml^-1). At moderate concentrations (0.25-1 μg ml^-1), eosin was quenched and its fluorescence lifetime was shortened gradually. Interestingly, the fluorescence of eosin was still quenched when the concentration exceeded 1 μg ml^-1, but its corresponding fluorescence lifetimes increase with increased concentration (>100 μg ml^-1). To further verify that multiplexed imaging of different tissues could be achieved only by eosin, we used fluorescence lifetime imaging microscopy (FLIM) to measure fluorescence lifetimes from hematoxylin and eosin (H&E) stained sections. Working directly on an average fluorescence lifetime (τ_m) histogram for lifetime-based separation easily achieved multiplexed imaging in situ. H&E stained erythrocytes, smooth muscles, collagen and artificial structures on a prepared microscopic slide could be identified without the need of alternating laser excitation, using hyperspectral systems and special staining or multi-labeled immunofluorescence. Using only eosin, different types of tissues could be distinguished by eosin concentration-dependent quenching. Hence, eosin fluorescence lifetimes potentially simplify multiplexed imaging and may have potential applications for pathological diagnosis.

Identification and characterization of different tissues in blood vessel by multiplexed fluorescence lifetimes

Herein, fluorescence lifetime imaging microscopy (FLIM) was used to directly measure eosin fluorescence lifetimes from H&E-stained umbilical artery, and a further utilization of eosin for high-content and multi-target analysis was proposed for the first time.

Improving mesoscopic fluorescence molecular tomography through data reduction

Mesoscopic fluorescence molecular tomography (MFMT) is a novel imaging technique that aims at obtaining the 3-D distribution of molecular probes inside biological tissues at depths of a few millimeters. To achieve high resolution, around 100-150μm scale in turbid samples, dense spatial sampling strategies are required. However, a large number of optodes leads to sizable forward and inverse problems that can be challenging to compute efficiently. In this work, we propose a two-step data reduction strategy to accelerate the inverse problem and improve robustness. First, data selection is performed via signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) criteria. Then principal component analysis (PCA) is applied to further reduce the size of the sensitivity matrix. We perform numerical simulations and phantom experiments to validate the effectiveness of the proposed strategy. In both in silico and in vitro cases, we are able to significantly improve the quality of MFMT reconstructions while reducing the computation times by close to a factor of two.

Dental optical tomography with upconversion nanoparticles-a feasibility study

Upconversion nanoparticles (UCNPs) have the unique ability to emit multiple colors upon excitation by near-infrared (NIR) light. Herein, we investigate the potential use of UCNPs as contrast agents for dental optical tomography, with a focus on monitoring the status of fillings after dental restoration. The potential of performing tomographic imaging using UCNP emission of visible or NIR light is established. This in silico and ex vivo study paves the way toward employing UCNPs as theranostic agents for dental applications.