Single-photon peak event detection (SPEED): a computational method for fast photon counting in fluorescence lifetime imaging microscopy

Inspiring a convergent engineering approach to measure and model the tissue microenvironment

Understanding the molecular and physical complexity of the tissue microenvironment (TiME) in the context of its spatiotemporal organization has remained an enduring challenge. Recent advances in engineering and data science are now promising the ability to study the structure, functions, and dynamics of the TiME in unprecedented detail; however, many advances still occur in silos that rarely integrate information to study the TiME in its full detail. This review provides an integrative overview of the engineering principles underlying chemical, optical, electrical, mechanical, and computational science to probe, sense, model, and fabricate the TiME. In individual sections, we first summarize the underlying principles, capabilities, and scope of emerging technologies, the breakthrough discoveries enabled by each technology and recent, promising innovations. We provide perspectives on the potential of these advances in answering critical questions about the TiME and its role in various disease and developmental processes. Finally, we present an integrative view that appreciates the major scientific and educational aspects in the study of the TiME.

Advancements in fluorescence lifetime imaging microscopy Instrumentation: Towards high speed and 3D

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging tool offering molecular specific insights into samples through the measurement of fluorescence decay time, with promising applications in diverse research fields. However, to acquire two-dimensional lifetime images, conventional FLIM relies on extensive scanning in both the spatial and temporal domain, resulting in much slower acquisition rates compared to intensity-based approaches. This problem is further magnified in three-dimensional imaging, as it necessitates additional scanning along the depth axis. Recent advancements have aimed to enhance the speed and three-dimensional imaging capabilities of FLIM. This review explores the progress made in addressing these challenges and discusses potential directions for future developments in FLIM instrumentation.

Analog multiplexing of a laser clock and computational photon counting for fast fluorescence lifetime imaging microscopy

The dynamic range and fluctuations of fluorescence intensities and lifetimes in biological samples are large, demanding fast, precise, and versatile techniques. Among the high-speed fluorescence lifetime imaging microscopy (FLIM) techniques, directly sampling the output of analog single-photon detectors at GHz rates combined with computational photon counting can handle a larger range of photon rates. Traditionally, the laser clock is not sampled explicitly in fast FLIM; rather the detection is synchronized to the laser clock so that the excitation pulse train can be inferred from the cumulative photon statistics of several pixels. This has two disadvantages for sparse or weakly fluorescent samples: inconsistencies in inferring the laser clock within a frame and inaccuracies in aligning the decay curves from different frames for averaging. The data throughput is also very inefficient in systems with repetition rates much larger than the fluorescence lifetime due to significant silent regions where no photons are expected. We present a method for registering the photon arrival times to the excitation using time-domain multiplexing for fast FLIM. The laser clock is multiplexed with photocurrents into the silent region. Our technique does not add to the existing data bottleneck, has the sub-nanosecond dead time of computational photon counting based fast FLIM, works with various detectors, lasers, and electronics, and eliminates the errors in lifetime estimation in photon-starved conditions. We demonstrate this concept on two multiphoton setups of different laser repetition rates for single and multichannel FLIM multiplexed into a single digitizer channel for real-time imaging of biological samples.

Temporally optimized and spectrally shaped hyperspectral coherent anti-Stokes Raman scattering microscopy

Coherent anti-Stokes Raman scattering (CARS) microscopy offers label-free chemical contrasts based on molecular vibrations. Hyperspectral CARS (HS-CARS) microscopy enables comprehensive microscale chemical characterization of biological samples. Various HS-CARS methods have been developed with individual advantages and disadvantages. We present what we believe to be a new temporally optimized and spectrally shaped (TOSS) HS-CARS method to overcome the limitations of existing techniques by providing precise control of the spatial and temporal profiles of the excitation beams for efficient and accurate measurements. This method uniquely uses Fourier transform pulse shaping based on a two-dimensional spatial light modulator to control the phase and amplitude of the excitation beams. TOSS-HS-CARS achieves fast, stable, and flexible acquisition, minimizes photodamage, and is highly adaptable to a multimodal multiphoton imaging system.

Development of a high-speed line-scanning fluorescence lifetime imaging microscope for biological imaging

We report the development of a novel line-scanning microscope capable of acquiring high-speed time-correlated single-photon counting (TCSPC)-based fluorescence lifetime imaging microscopy (FLIM) imaging. The system consists of a laser-line focus, which is optically conjugated to a 1024 × 8 single-photon avalanche diode (SPAD)-based line-imaging complementary metal-oxide semiconductor (CMOS), with 23.78 µm pixel pitch at 49.31% fill factor. Incorporation of on-chip histogramming on the line-sensor enables acquisition rates 33 times faster than our previously reported bespoke high-speed FLIM platforms. We demonstrate the imaging capability of the high-speed FLIM platform in a number of biological applications.

Fluorescence lifetime tracking and imaging of single moving particles assisted by a low-photon-count analysis algorithm

Fluorescence lifetime imaging microscopy (FLIM) has been widely used in the field of biological research because of its high specificity, sensitivity, and quantitative ability in the sensing cellular microenvironment. The most commonly used FLIM technology is based on time-correlated single photon counting (TCSPC). Although the TCSPC method has the highest temporal resolution, the data acquisition time is usually long, and the imaging speed is slow. In this work, we proposed a fast FLIM technology for fluorescence lifetime tracking and imaging of single moving particles, named single particle tracking FLIM (SPT-FLIM). We used feedback-controlled addressing scanning and Mosaic FLIM mode imaging to reduce the number of scanned pixels and the data readout time, respectively. Moreover, we developed a compressed sensing analysis algorithm based on alternating descent conditional gradient (ADCG) for low-photon-count data. We applied the ADCG-FLIM algorithm on both simulated and experimental datasets to evaluate its performance. The results showed that ADCG-FLIM could achieve reliable lifetime estimation with high accuracy and precision in the case of a photon count less than 100. By reducing the photon count requirement for each pixel from, typically, 1000 to 100, the acquisition time for a single frame lifetime image could be significantly shortened, and the imaging speed could be improved to a great extent. On this basis, we obtained lifetime trajectories of moving fluorescent beads using the SPT-FLIM technique. Overall, our work offers a powerful tool for fluorescence lifetime tracking and imaging of single moving particles, which will promote the application of TCSPC-FLIM in biological research.

Nanosecond compressive fluorescence lifetime microscopy imaging via the RATS method with a direct reconstruction of lifetime maps

The RAndom Temporal Signals (RATS) method has proven to be a useful and versatile method for measuring photoluminescence (PL) dynamics and fluorescence lifetime imaging (FLIM). Here, we present two fundamental development steps in the method. First, we demonstrate that by using random digital laser modulation in RATS, it is possible to implement the measurement of PL dynamics with temporal resolution in units of nanoseconds. Secondly, we propose an alternative approach to evaluating FLIM measurements based on a single-pixel camera experiment. In contrast to the standard evaluation, which requires a lengthy iterative reconstruction of PL maps for each timepoint, here we use a limited set of predetermined PL lifetimes and calculate the amplitude maps corresponding to each lifetime. The alternative approach significantly saves post-processing time and, in addition, in a system with noise present, it shows better stability in terms of the accuracy of the FLIM spectrogram. Besides simulations that confirmed the functionality of the extension, we implemented the new advancements into a microscope optical setup for mapping PL dynamics on the micrometer scale. The presented principles were also verified experimentally by mapping a LuAG:Ce crystal surface.

Tracking the binding of multi-functional fluorescent tags for Alzheimer's disease using quantitative multiphoton microscopy

A recent theranostic approach to address Alzheimer's disease (AD) utilizes multifunctional targets that both tag and negate the toxicity of AD biomarkers. These compounds, which emit fluorescence with both an activation and a spectral shift in the presence of Aβ, were previously characterized with traditional fluorescence imaging for binary characterization. However, these multifunctional compounds have broad and dynamic emission spectra that are dependent on factors such as the local environment, presence of Aβ deposits, etc. Since quantitative multiphoton microscopy is sensitive to the binding dynamics of molecules, we characterized the performance of two such compounds, LS-4 and ZY-12-OMe, using Simultaneous Label-free Autofluorescence Multi-harmonic (SLAM) microscopy and Fast Optical Coherence, Autofluorescence Lifetime imaging and Second harmonic generation (FOCALS) microscopy. This study shows that the combination of quantitative multiphoton imaging with multifunctional tags for AD offers new insights into the interaction of these tags with AD biomarkers and the theranostic mechanisms.

Computational Photon Counting Using Multithreshold Peak Detection for Fast Fluorescence Lifetime Imaging Microscopy

Time-resolved photon counting methods have a finite bandwidth that restricts the acquisition speed of techniques like fluorescence lifetime imaging microscopy (FLIM). To enable faster imaging, computational methods can be employed to count photons when the output of a detector is directly digitized at a high sampling rate. Here, we present computational photon counting using a hybrid photodetector in conjunction with multithreshold peak detection to count instances where one or more photons arrive at the detector within the detector response time. This method can be used to distinguish up to five photon counts per digitized point, whereas previous demonstrations of computational photon counting on data acquired with photomultiplier tubes have only counted one photon at a time. We demonstrate in both freely moving C. elegans and a human breast cancer cell line undergoing apoptosis that this novel multithreshold peak detection method can accurately characterize the intensity and fluorescence lifetime of samples producing photon rates up to 223%, higher than previously demonstrated photon counting FLIM systems.

Label-free multimodal nonlinear optical imaging of needle biopsy cores for intraoperative cancer diagnosis

SIGNIFICANCE

Needle biopsy (NB) procedures are important for the initial diagnosis of many types of cancer. However, the possibility of NB specimens being unable to provide diagnostic information, (i.e., non-diagnostic sampling) and the time-consuming histological evaluation process can cause delays in diagnoses that affect patient care.

AIM

We aim to demonstrate the advantages of this label-free multimodal nonlinear optical imaging (NLOI) technique as a non-destructive point-of-procedure evaluation method for NB tissue cores, for the visualization and characterization of the tissue microenvironment.

APPROACH

A portable, label-free, multimodal NLOI system combined second-harmonic generation (SHG) and third-harmonic generation and two- and three-photon autofluorescence (2PF, 3PF) microscopy. It was used for intraoperative imaging of fresh NB tissue cores acquired during canine cancer surgeries, which involved liver, lung, and mammary tumors as well as soft-tissue sarcoma; in total, eight canine patients were recruited. An added tissue culture chamber enabled the use of this NLOI system for longitudinal imaging of fresh NB tissue cores taken from an induced rat mammary tumor and healthy mouse livers.

RESULTS

The intraoperative NLOI system was used to assess fresh canine NB specimens during veterinary cancer surgeries. Histology-like morphological features were visualized by the combination of four NLOI modalities at the point-of-procedure. The NLOI results provided quantitative information on the tissue microenvironment such as the collagen fiber orientation using Fourier-domain SHG analysis and metabolic profiling by optical redox ratio (ORR) defined by 2PF/(2PF + 3PF). The analyses showed that the canine mammary tumor had more randomly oriented collagen fibers compared to the tumor margin, and hepatocarcinoma had a wider distribution of ORR with a lower mean value compared to the liver fibrosis and the normal-appearing liver. Moreover, the loss of metabolic information during tissue degradation of fresh murine NB specimens was shown by overall intensity decreases in all channels and an increase of mean ORR from 0.94 (standard deviation 0.099) to 0.97 (standard deviation 0.077) during 1-h longitudinal imaging of a rat mammary tumor NB specimen. The tissue response to staurosporine (STS), an apoptotic inducer, from fresh murine liver NB specimens was also observed. The mean ORR decreased from 0.86 to 0.74 in the first 40 min and then increased to 0.8 during the rest of the hour of imaging, compared to the imaging results without the addition of STS, which showed a continuous increase of ORR from 0.72 to 0.75.

CONCLUSIONS

A label-free, multimodal NLOI platform reveals microstructural and metabolic information of the fresh NB cores during intraoperative cancer imaging. This system has been demonstrated on animal models to show its potential to provide a more comprehensive histological assessment and a better understanding of the unperturbed tumor microenvironment. Considering tissue degradation, or loss of viability upon fixation, this intraoperative NLOI system has the advantage of immediate assessment of freshly excised tissue specimens at the point of procedure.

Computational optical sensing and imaging 2021: feature issue introduction

This Feature Issue includes 2 reviews and 34 research articles that highlight recent works in the field of Computational Optical Sensing and Imaging. Many of the works were presented at the 2021 OSA Topical Meeting on Computational Optical Sensing and Imaging, held virtually from July 19 to July 23, 2021. Articles in the feature issue cover a broad scope of computational imaging topics, such as microscopy, 3D imaging, phase retrieval, non-line-of-sight imaging, imaging through scattering media, ghost imaging, compressed sensing, and applications with new types of sensors. Deep learning approaches for computational imaging and sensing are also a focus of this feature issue.

Computational Optical Sensing and Imaging 2021: introduction to the feature issue

This feature issue includes two reviews and 34 research papers that highlight recent works in the field of computational optical sensing and imaging. Many of the works were presented at the 2021 Optica (formerly OSA) Topical Meeting on Computational Optical Sensing and Imaging, held virtually from 19 July to 23 July 2021. Papers in the feature issue cover a broad scope of computational imaging topics, such as microscopy, 3D imaging, phase retrieval, non-line-of-sight imaging, imaging through scattering media, ghost imaging, compressed sensing, and applications with new types of sensors. Deep learning approaches for computational imaging and sensing are also a focus of this feature issue.