High-speed compressed-sensing fluorescence lifetime imaging microscopy of live cells

EventLFM: event camera integrated Fourier light field microscopy for ultrafast 3D imaging

Ultrafast 3D imaging is indispensable for visualizing complex and dynamic biological processes. Conventional scanning-based techniques necessitate an inherent trade-off between acquisition speed and space-bandwidth product (SBP). Emerging single-shot 3D wide-field techniques offer a promising alternative but are bottlenecked by the synchronous readout constraints of conventional CMOS systems, thus restricting data throughput to maintain high SBP at limited frame rates. To address this, we introduce EventLFM, a straightforward and cost-effective system that overcomes these challenges by integrating an event camera with Fourier light field microscopy (LFM), a state-of-the-art single-shot 3D wide-field imaging technique. The event camera operates on a novel asynchronous readout architecture, thereby bypassing the frame rate limitations inherent to conventional CMOS systems. We further develop a simple and robust event-driven LFM reconstruction algorithm that can reliably reconstruct 3D dynamics from the unique spatiotemporal measurements captured by EventLFM. Experimental results demonstrate that EventLFM can robustly reconstruct fast-moving and rapidly blinking 3D fluorescent samples at kHz frame rates. Furthermore, we highlight EventLFM's capability for imaging of blinking neuronal signals in scattering mouse brain tissues and 3D tracking of GFP-labeled neurons in freely moving C. elegans. We believe that the combined ultrafast speed and large 3D SBP offered by EventLFM may open up new possibilities across many biomedical applications.

High-speed optical imaging with sCMOS pixel reassignment

Fluorescence microscopy has undergone rapid advancements, offering unprecedented visualization of biological events and shedding light on the intricate mechanisms governing living organisms. However, the exploration of rapid biological dynamics still poses a significant challenge due to the limitations of current digital camera architectures and the inherent compromise between imaging speed and other capabilities. Here, we introduce sHAPR, a high-speed acquisition technique that leverages the operating principles of sCMOS cameras to capture fast cellular and subcellular processes. sHAPR harnesses custom fiber optics to convert microscopy images into one-dimensional recordings, enabling acquisition at the maximum camera readout rate, typically between 25 and 250 kHz. We have demonstrated the utility of sHAPR with a variety of phantom and dynamic systems, including high-throughput flow cytometry, cardiomyocyte contraction, and neuronal calcium waves, using a standard epi-fluorescence microscope. sHAPR is highly adaptable and can be integrated into existing microscopy systems without requiring extensive platform modifications. This method pushes the boundaries of current fluorescence imaging capabilities, opening up new avenues for investigating high-speed biological phenomena.

Investigating the Photophysical Aspects of a Naphthalene-Based Excited-State Proton Transfer Dye 1-(1H-Benzo[d]imidazol-2-yl)naphthalen-2-ol: pH-Dependent Modulation of Photodynamics

The steady state and time-resolved photophysical behavior of a proton transfer dye 1-(1H-benzo[d]imidazol-2-yl)naphthalen-2-ol (H-BINO) was investigated. The excited state intramolecular proton transfer (ESIPT) reaction in H-BINO is predominant in nonpolar solutions (toluene and DCM) with a lifetime of ∼1.0 ns. However, in polar media (DMF and MeOH), the excited state photodynamics is characterized by a complex equilibrium of emission from the locally excited state (0.1-2.3 ns), the phototautomer (0.5-1.2 ns), and the anionic emission (2.1-5.4 ns). In the solid state, emission from the various aggregated states dictates the photobehavior. Interestingly, the photodynamics in aqueous solution changes starkly as a function of pH with the anionic (2.1 ns) and phototautomeric (0.5-1.0 ns) emissions guiding the photodynamics as the pH of the medium increases. Optimized structural parameters at the proton donor and acceptor sites for the enol and keto forms and the calculated potential energy curve along the proton transfer coordinate at the density functional theory (DFT) level with the B3LYP/6-311G++(d,p) theory support a favorable and barrierless ESIPT process. The current results will surely boost the ongoing research on small molecule emissive materials.

From Cell Populations to Molecular Complexes: Multiplexed Multimodal Microscopy to Explore p53-53BP1 Molecular Interaction

Surpassing the diffraction barrier revolutionized modern fluorescence microscopy. However, intrinsic limitations in statistical sampling, the number of simultaneously analyzable channels, hardware requirements, and sample preparation procedures still represent an obstacle to its widespread diffusion in applicative biomedical research. Here, we present a novel pipeline based on automated multimodal microscopy and super-resolution techniques employing easily available materials and instruments and completed with open-source image-analysis software developed in our laboratory. The results show the potential impact of single-molecule localization microscopy (SMLM) on the study of biomolecules' interactions and the localization of macromolecular complexes. As a demonstrative application, we explored the basis of p53-53BP1 interactions, showing the formation of a putative macromolecular complex between the two proteins and the basal transcription machinery in situ, thus providing visual proof of the direct role of 53BP1 in sustaining p53 transactivation function. Moreover, high-content SMLM provided evidence of the presence of a 53BP1 complex on the cell cytoskeleton and in the mitochondrial space, thus suggesting the existence of novel alternative 53BP1 functions to support p53 activity.

Tutorial on compressed ultrafast photography

Significance

Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges.

Aim

This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect.

Approach

A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement.

Conclusions

CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

Introducing the Special Issue Honoring Lihong V. Wang, Pioneer in Biomedical Optics

The editorial concludes the JBO Special Issue Honoring Lihong V. Wang, outlining Prof. Wang's salient contributions to advancing the field of biomedical optics.

Near-Infrared Fluorescence Imaging in Preclinical Models of Glioblastoma

Cancer is a public health problem requiring ongoing research to improve current treatments and discover novel therapies. More accurate imaging would facilitate such research. Near-infrared fluorescence has been developed as a non-invasive imaging technique capable of visualizing and measuring biological processes at the molecular level in living subjects. In this work, we evaluate the tumor activity in two preclinical glioblastoma models by using fluorochrome (IRDye 800CW) coupled to different molecules: tripeptide Arg-Gly-Asp (RGD), 2-amino-2-deoxy-D-glucose (2-DG), and polyethylene glycol (PEG). These molecules interact with pathological conditions of tumors, including their overexpression of αvβ3 integrins (RGD), elevated glucose uptake (2-DG), and enhanced permeability and retention effect (PEG). IRDye 800CW RGD gave the best in vivo fluorescence signal from the tumor area, which contrasted well with the low fluorescence intensity of healthy tissue. In the ex vivo imaging (dissected tumor), the accumulation of IRDye 800CW RGD could be appreciated at the tumor site. Glioblastoma tumors were presently detected with specificity and sensitivity by utilizing IRDye 800CW RGD, a near-infrared fluorophore combined with a marker of αvβ3 integrin expression. Further research is needed on its capacity to monitor tumor growth in glioblastoma after chemotherapy.

Dual-potential electrochemiluminescence cytosensor based on a metal-organic framework and ABEI-PEI-Au@AgNPs for the simultaneous determination of phosphatidylserine and epidermal growth factor receptors on an apoptotic cell surface

A new electrochemiluminescence (ECL) cytosensor is proposed for the simultaneous determination of phosphatidylserine (PS) and epidermal growth factor receptor (EGFR) based on the ECL signals of metal-organic framework-5 (MOF-5) loaded CdS quantum dots and N-(aminobutyl)-N-(ethylisoluminol)-polyethylenimine capped Au and Ag nanoparticles. Apoptosis promotes the exposure of PS and reduces the expression of EGFR in cell membranes. Two spatially resolved areas on dual-disk glassy carbon electrodes were designed to eliminate the interference from different ECL probes. Using HepG2 cells treated with resveratrol to induce apoptosis, the cytosensor exhibited high sensitivity, simplicity, and high reproducibility, demonstrating its potential in drug screening and rapid apoptotic cell detection. The strategy reported provides a promising platform for the highly sensitive cytosensing and convenient screening of clinically relevant anticancer drugs.

Spatial resolution enhancement in photon-starved STED imaging using deep learning-based fluorescence lifetime analysis

As a super-resolution imaging method, stimulated emission depletion (STED) microscopy has unraveled fine intracellular structures and provided insights into nanoscale organizations in cells. Although image resolution can be further enhanced by continuously increasing the STED-beam power, the resulting photodamage and phototoxicity are major issues for real-world applications of STED microscopy. Here we demonstrate that, with 50% less STED-beam power, the STED image resolution can be improved up to 1.45-fold using the separation of photons by a lifetime tuning (SPLIT) scheme combined with a deep learning-based phasor analysis algorithm termed flimGANE (fluorescence lifetime imaging based on a generative adversarial network). This work offers a new approach for STED imaging in situations where only a limited photon budget is available.

Compact and robust deep learning architecture for fluorescence lifetime imaging and FPGA implementation

This paper reports a bespoke adder-based deep learning network for time-domain fluorescence lifetime imaging (FLIM). By leveraging thel1-norm extraction method, we propose a 1D Fluorescence Lifetime AdderNet (FLAN) without multiplication-based convolutions to reduce the computational complexity. Further, we compressed fluorescence decays in temporal dimension using a log-scale merging technique to discard redundant temporal information derived as log-scaling FLAN (FLAN+LS). FLAN+LS achieves 0.11 and 0.23 compression ratios compared with FLAN and a conventional 1D convolutional neural network (1D CNN) while maintaining high accuracy in retrieving lifetimes. We extensively evaluated FLAN and FLAN+LS using synthetic and real data. A traditional fitting method and other non-fitting, high-accuracy algorithms were compared with our networks for synthetic data. Our networks attained a minor reconstruction error in different photon-count scenarios. For real data, we used fluorescent beads' data acquired by a confocal microscope to validate the effectiveness of real fluorophores, and our networks can differentiate beads with different lifetimes. Additionally, we implemented the network architecture on a field-programmable gate array (FPGA) with a post-quantization technique to shorten the bit-width, thereby improving computing efficiency. FLAN+LS on hardware achieves the highest computing efficiency compared to 1D CNN and FLAN. We also discussed the applicability of our network and hardware architecture for other time-resolved biomedical applications using photon-efficient, time-resolved sensors.

Granger causality for compressively sensed sparse signals

Compressed sensing is a scheme that allows for sparse signals to be acquired, transmitted, and stored using far fewer measurements than done by conventional means employing the Nyquist sampling theorem. Since many naturally occurring signals are sparse (in some domain), compressed sensing has rapidly seen popularity in a number of applied physics and engineering applications, particularly in designing signal and image acquisition strategies, e.g., magnetic resonance imaging, quantum state tomography, scanning tunneling microscopy, and analog to digital conversion technologies. Contemporaneously, causal inference has become an important tool for the analysis and understanding of processes and their interactions in many disciplines of science, especially those dealing with complex systems. Direct causal analysis for compressively sensed data is required to avoid the task of reconstructing the compressed data. Also, for some sparse signals, such as for sparse temporal data, it may be difficult to discover causal relations directly using available data-driven or model-free causality estimation techniques. In this work, we provide a mathematical proof that structured compressed sensing matrices, specifically circulant and Toeplitz, preserve causal relationships in the compressed signal domain, as measured by Granger causality (GC). We then verify this theorem on a number of bivariate and multivariate coupled sparse signal simulations which are compressed using these matrices. We also demonstrate a real world application of network causal connectivity estimation from sparse neural spike train recordings from rat prefrontal cortex. In addition to demonstrating the effectiveness of structured matrices for GC estimation from sparse signals, we also show a computational time advantage of the proposed strategy for causal inference from compressed signals of both sparse and regular autoregressive processes as compared to standard GC estimation from original signals.

Compressed fluorescence lifetime imaging via combined TV-based and deep priors

Compressed fluorescence lifetime imaging (Compressed-FLIM) is a novel Snapshot compressive imaging (SCI) method for single-shot widefield FLIM. This approach has the advantages of high temporal resolution and deep frame sequences, allowing for the analysis of FLIM signals that follow complex decay models. However, the precision of Compressed-FLIM is limited by reconstruction algorithms. To improve the reconstruction accuracy of Compressed-FLIM in dealing with large-scale FLIM problem, we developed a more effective combined prior model 3DTG_p V_net, based on the Plug and Play (PnP) framework. Extensive numerical simulations indicate the proposed method eliminates reconstruction artifacts caused by the Deep denoiser networks. Moreover, it improves the reconstructed accuracy by around 4dB (peak signal-to-noise ratio; PSNR) over the state-of-the-art TV+FFDNet in test data sets. We conducted the single-shot FLIM experiment with different Rhodamine reagents and the results show that in practice, the proposed algorithm has promising reconstruction performance and more negligible lifetime bias.

Molecular to Supramolecular Self-Assembled Luminogens for Tracking the Intracellular Organelle Dynamics

Deciphering the dynamics of intracellular organelles has gained immense attention due to their subtle control over diverse, complex biological processes such as cellular metabolism, energy homeostasis, and autophagy. In this context, molecular materials, including small-organic fluorescent probes and their supramolecular self-assembled nano-/microarchitectures, have been employed to explore the diverse intracellular biological events. However, only a handful of fluorescent probes and self-assembled emissive structures have been successfully used to track different organelle's movements, circumventing the issues related to water solubility and long-term photostability. Thus, the water-soluble molecular fluorescent probes and the water-dispersible supramolecular self-assemblies have emerged as promising candidates to explore the trafficking of the organelles under diverse physiological conditions. In this review, we have delineated the recent progress of fluorescent probes and their supramolecular self-assemblies for the elucidation of the dynamics of diverse cellular organelles with a special emphasis on lysosomes, lipid droplets, and mitochondria. Recent advancement in fluorescence lifetime and super-resolution microscopy imaging has also been discussed to investigate the dynamics of organelles. In addition, the fabrication of the next-generation molecular to supramolecular self-assembled luminogens for probing the variation of microenvironments during the trafficking process has been outlined.

Imaging approaches for monitoring three-dimensional cell and tissue culture systems

The past decade has seen an increasing demand for more complex, reproducible and physiologically relevant tissue cultures that can mimic the structural and biological features of living tissues. Monitoring the viability, development and responses of such tissues in real-time are challenging due to the complexities of cell culture physical characteristics and the environments in which these cultures need to be maintained in. Significant developments in optics, such as optical manipulation, improved detection and data analysis, have made optical imaging a preferred choice for many three-dimensional (3D) cell culture monitoring applications. The aim of this review is to discuss the challenges associated with imaging and monitoring 3D tissues and cell culture, and highlight topical label-free imaging tools that enable bioengineers and biophysicists to non-invasively characterise engineered living tissues.

Enhancing the Throughput of FT Mass Spectrometry Imaging Using Joint Compressed Sensing and Subspace Modeling

Mass spectrometry imaging (MSI) allows for untargeted mapping of the chemical composition of tissues with attomole detection limits. MSI using Fourier transform (FT)-based mass spectrometers, such as FT-ion cyclotron resonance (FT-ICR), grants the ability to examine the chemical space with unmatched mass resolution and mass accuracy. However, direct imaging of large tissue samples using FT-ICR is slow. In this work, we present an approach that combines the subspace modeling of ICR temporal signals with compressed sensing to accelerate high-resolution FT-ICR MSI. A joint subspace and spatial sparsity constrained model computationally reconstructs high-resolution MSI data from the sparsely sampled transients with reduced duration, allowing a significant reduction in imaging time. Simulation studies and experimental implementation of the proposed method in investigation of brain tissues demonstrate a 10-fold enhancement in throughput of FT-ICR MSI, without the need for instrumental or hardware modifications.

Activatable Peptides for Rapid and Simple Visualization of Protease Activity Secreted in Living Cells

Activity-based monitoring of cell-secreted proteases has gained significant interest due to the implication of these substances in diverse cellular functions. Here, we demonstrated a cell-based method of monitoring protease activity using fluorescent cell-permeable peptides. The activatable peptide consists of anionic (EEEE), cleavable, and cationic sequences (RRRR) that enable intracellular delivery by matrix metalloproteinase-2 (MMP2), which is secreted by living cancer cells. Compared to HT-29 cells (MMP2-negative), HT-1080 cells (MMP2-positive) showed a strong fluorescence response to the short fluorescent peptide via cell-secreted protease activation. Our approach is expected to find applications for the rapid visualization of protease activity in living cells.

Lifetime-based analysis of binary fluorophores mixtures in the low photon count limit

Single biomolecule sensing often requires the quantification of multiple fluorescent species. Here, we theoretically and experimentally use time-resolved fluorescence via Time Correlated Single Photon Counting (TCSPC) to accurately quantify fluorescent species with similar chromatic signatures. A modified maximum likelihood estimator is introduced to include two fluorophore species, with compensation of the instrument response function. We apply this algorithm to simulated data of a simplified two-fluorescent species model, as well as to experimental data of fluorophores' mixtures and to a model protein, doubly labeled with different fluorophores' ratio. We show that 100 to 200 photons per fluorophore, in a 10-ms timescale, are sufficient to provide an accurate estimation of the dyes' ratio on the model protein. Our results provide estimation for the desired photon integration time toward implementation of TCSPC in systems with fast occurring events, such as translocation of biomolecules through nanopores or single-molecule burst analyses experiments.

•

Exact ratios of emission-similar dyes in binary mixtures were quantified by TCSPC

•

MLE-based analysis with IRF compensation was implemented for two fluorescent dyes

•

Dual dye bioconjugation on a model protein was quantified at limited photon counts

Scan-less machine-learning-enabled incoherent microscopy for minimally-invasive deep-brain imaging

Deep-brain microscopy is strongly limited by the size of the imaging probe, both in terms of achievable resolution and potential trauma due to surgery. Here, we show that a segment of an ultra-thin multi-mode fiber (cannula) can replace the bulky microscope objective inside the brain. By creating a self-consistent deep neural network that is trained to reconstruct anthropocentric images from the raw signal transported by the cannula, we demonstrate a single-cell resolution (< 10μm), depth sectioning resolution of 40 μm, and field of view of 200 μm, all with green-fluorescent-protein labelled neurons imaged at depths as large as 1.4 mm from the brain surface. Since ground-truth images at these depths are challenging to obtain in vivo, we propose a novel ensemble method that averages the reconstructed images from disparate deep-neural-network architectures. Finally, we demonstrate dynamic imaging of moving GCaMp-labelled C. elegans worms. Our approach dramatically simplifies deep-brain microscopy.

Deep Compressed Imaging via Optimized-Pattern Scanning

The need for high-speed imaging in applications such as biomedicine, surveillance and consumer electronics has called for new developments of imaging systems. While the industrial effort continuously pushes the advance of silicon focal plane array image sensors, imaging through a single-pixel detector has gained significant interests thanks to the development of computational algorithms. Here, we present a new imaging modality, Deep Compressed Imaging via Optimized-Pattern Scanning (DeCIOPS), which can significantly increase the acquisition speed for a single-detector-based imaging system. We project and scan an illumination pattern across the object and collect the sampling signal with a single-pixel detector. We develop an innovative end-to-end optimized auto-encoder, using a deep neural network and compressed sensing algorithm, to optimize the illumination pattern, which allows us to reconstruct faithfully the image from a small number of samples, and with a high frame rate. Compared with the conventional switching-mask based single-pixel camera and point scanning imaging systems, our method achieves a much higher imaging speed, while retaining a similar imaging quality. We experimentally validated this imaging modality in the settings of both continuous-wave (CW) illumination and pulsed light illumination and showed high-quality image reconstructions with a high compressed sampling rate. This new compressed sensing modality could be widely applied in different imaging systems, enabling new applications which require high imaging speed.

Theoretical investigations of a modified compressed ultrafast photography method suitable for single-shot fluorescence lifetime imaging

A single-shot fluorescence lifetime imaging (FLIM) method based on the compressed ultrafast photography (CUP) is proposed, named space-restricted CUP (srCUP). srCUP is suitable for imaging objects moving slowly (<∼150/Mmm/s, M is the magnification of the objective lens) in the field of view with the intensity changing within nanoseconds in a measurement window around 10 ns. We used synthetic datasets to explore the performances of srCUP compared with CUP and TCUP (a variant of CUP). srCUP not only provides superior reconstruction performances, but its reconstruction speed is also twofold and threefold faster than CUP and TCUP, respectively. The lifetime determination performances were assessed by estimating lifetime components, amplitude- and intensity-weighted average lifetimes (τ_A and τ_I), with the reconstructed scenes using the least squares method based on a bi-exponential model. srCUP has the best accuracy and precision for lifetime determinations with a relative bias less than 7% and a coefficient of variation less than 7% for τ_A, and a relative bias less than 10% and a coefficient of variation less than 11% for τ_I.