Wide-field time-gated SPAD imager for phasor-based FLIM applications

Underwater Single-Photon 3D Reconstruction Algorithm Based on K-Nearest Neighbor

The high sensitivity and picosecond time resolution of single-photon avalanche diodes (SPADs) can improve the operational range and imaging accuracy of underwater detection systems. When an underwater SPAD imaging system is used to detect targets, backward-scattering caused by particles in water often results in the poor quality of the reconstructed underwater image. Although methods such as simple pixel accumulation have been proven to be effective for time-photon histogram reconstruction, they perform unsatisfactorily in a highly scattering environment. Therefore, new reconstruction methods are necessary for underwater SPAD detection to obtain high-resolution images. In this paper, we propose an algorithm that reconstructs high-resolution depth profiles of underwater targets from a time-photon histogram by employing the K-nearest neighbor (KNN) to classify multiple targets and the background. The results contribute to the performance of pixel accumulation and depth estimation algorithms such as pixel cross-correlation and ManiPoP. We use public experimental data sets and underwater simulation data to verify the effectiveness of the proposed algorithm. The results of our algorithm show that the root mean square errors (RMSEs) of land targets and simulated underwater targets are reduced by 57.12% and 23.45%, respectively, achieving high-resolution single-photon depth profile reconstruction.

Towards optical measurements of membrane potential values in Bacillus subtilis using fluorescence lifetime

Membrane potential (MP) changes can provide a simple readout of bacterial functional and metabolic state or stress levels. While several optical methods exist for measuring fast changes in MP in excitable cells, there is a dearth of such methods for absolute and precise measurements of steady-state membrane potentials (MPs) in bacterial cells. Conventional electrode-based methods for the measurement of MP are not suitable for calibrating optical methods in small bacterial cells. While optical measurement based on Nernstian indicators have been successfully used, they do not provide absolute or precise quantification of MP or its changes. We present a novel, calibrated MP recording approach to address this gap. Our method is based on (i) a unique VoltageFluor (VF) optical transducer, whose fluorescence lifetime varies as a function of MP via photoinduced electron transfer (PeT) and (ii) a quantitative phasor-FLIM analysis for high-throughput readout. This method allows MP changes to be easily recorded, quantified and visualized. Using our preliminary Bacillus subtilis-specific MP versus VF lifetime calibration, we estimated the MP for unperturbed B. subtilis cells to be -65 mV and that for chemically depolarized cells as -14 mV. Our work paves the way for deeper insights into bacterial electrophysiology and bioelectricity research.

Multiplexed near infrared fluorescence lifetime imaging in turbid media

Significance

Fluorescence lifetime imaging (FLI) plays a pivotal role in enhancing our understanding of biological systems, providing a valuable tool for non-invasive exploration of biomolecular and cellular dynamics, both in vitro and in vivo. Its ability to selectively target and multiplex various entities, alongside heightened sensitivity and specificity, offers rapid and cost-effective insights.

Aim

Our aim is to investigate the multiplexing capabilities of near-infrared (NIR) FLI within a scattering medium that mimics biological tissues. We strive to develop a comprehensive understanding of FLI's potential for multiplexing diverse targets within a complex, tissue-like environment.

Approach

We introduce an innovative Monte Carlo (MC) simulation approach that accurately describes the scattering behavior of fluorescent photons within turbid media. Applying phasor analyses, we enable the multiplexing of distinct targets within a single FLI image. Leveraging the state-of-the-art single-photon avalanche diode (SPAD) time-gated camera, SPAD512S, we conduct experimental wide-field FLI in the NIR regime.

Results

Our study demonstrates the successful multiplexing of dual targets within a single FLI image, reaching a depth of 1 cm within tissue-like phantoms. Through our novel MC simulation approach and phasor analyses, we showcase the effectiveness of our methodology in overcoming the challenges posed by scattering media.

Conclusions

This research underscores the potential of NIR FLI for multiplexing applications in complex biological environments. By combining advanced simulation techniques with cutting-edge experimental tools, we introduce significant results in the non-invasive exploration of biomolecular dynamics, to advance the field of FLI research.

Fluorescence Lifetime Measurements and Analyses: Protocols Using Flow Cytometry and High-Throughput Microscopy

This chapter focuses on applications and protocols that involve the measurement of the fluorescence lifetime as an informative cytometric parameter. The timing of fluorescence decay has been well-studied for cell counting, sorting, and imaging. Therefore, provided herein is an overview of the techniques used, how they enhance cytometry protocols, and the modern techniques used for lifetime analysis. The background and theory behind fluorescence decay kinetic measurements in cells is first discussed followed by the history of the development of time-resolved flow cytometry. These sections are followed by a review of applications that benefit from the quantitative nature of fluorescence lifetimes as a photophysical trait. Lastly, perspectives on the modern ways in which the fluorescence lifetime is scanned at high throughputs which include high-speed microscopy and machine learning are provided.

Deep learning approach for denoising low-SNR correlation plenoptic images

Correlation Plenoptic Imaging (CPI) is a novel volumetric imaging technique that uses two sensors and the spatio-temporal correlations of light to detect both the spatial distribution and the direction of light. This novel approach to plenoptic imaging enables refocusing and 3D imaging with significant enhancement of both resolution and depth of field. However, CPI is generally slower than conventional approaches due to the need to acquire sufficient statistics for measuring correlations with an acceptable signal-to-noise ratio (SNR). We address this issue by implementing a Deep Learning application to improve image quality with undersampled frame statistics. We employ a set of experimental images reconstructed by a standard CPI architecture, at three different sampling ratios, and use it to feed a CNN model pre-trained through the transfer learning paradigm U-Net architecture with VGG-19 net for the encoding part. We find that our model reaches a Structural Similarity (SSIM) index value close to 1 both for the test sample (SSIM = [Formula: see text]) and in 5-fold cross validation (SSIM = [Formula: see text]); the results are also shown to outperform classic denoising methods, in particular for images with lower SNR. The proposed work represents the first application of Artificial Intelligence in the field of CPI and demonstrates its high potential: speeding-up the acquisition by a factor 20 over the fastest CPI so far demonstrated, enabling recording potentially 200 volumetric images per second. The presented results open the way to scanning-free real-time volumetric imaging at video rate, which is expected to achieve a substantial influence in various applications scenarios, from monitoring neuronal activity to machine vision and security.

FLUTE: A Python GUI for interactive phasor analysis of FLIM data

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique used to probe the local environment of fluorophores. The fit-free phasor approach to FLIM data is increasingly being used due to its ease of interpretation. To date, no open-source graphical user interface (GUI) for phasor analysis of FLIM data is available in Python, thus limiting the widespread use of phasor analysis in biomedical research. Here, we present Fluorescence Lifetime Ultimate Explorer (FLUTE), a Python GUI that is designed to fill this gap. FLUTE simplifies and automates many aspects of the analysis of FLIM data acquired in the time domain, such as calibrating the FLIM data, performing interactive exploration of the phasor plot, displaying phasor plots and FLIM images with different lifetime contrasts simultaneously, and calculating the distance from known molecular species. After applying desired filters and thresholds, the final edited datasets can be exported for further user-specific analysis. FLUTE has been tested using several FLIM datasets including autofluorescence of zebrafish embryos and in vitro cells. In summary, our user-friendly GUI extends the advantages of phasor plotting by making the data visualization and analysis easy and interactive, allows for analysis of large FLIM datasets, and accelerates FLIM analysis for non-specialized labs.

Correlated-photon imaging at 10 volumetric images per second

The correlation properties of light provide an outstanding tool to overcome the limitations of traditional imaging techniques. A relevant case is represented by correlation plenoptic imaging (CPI), a quantum-inspired volumetric imaging protocol employing spatio-temporally correlated photons from either entangled or chaotic sources to address the main limitations of conventional light-field imaging, namely, the poor spatial resolution and the reduced change of perspective for 3D imaging. However, the application potential of high-resolution imaging modalities relying on photon correlations is limited, in practice, by the need to collect a large number of frames. This creates a gap, unacceptable for many relevant tasks, between the time performance of correlated-light imaging and that of traditional imaging methods. In this article, we address this issue by exploiting the photon number correlations intrinsic in chaotic light, combined with a cutting-edge ultrafast sensor made of a large array of single-photon avalanche diodes (SPADs). This combination of source and sensor is embedded within a novel single-lens CPI scheme enabling to acquire 10 volumetric images per second. Our results place correlated-photon imaging at a competitive edge and prove its potential in practical applications.

Challenges and prospects for multi-chip microlens imprints on front-side illuminated SPAD imagers

The overall sensitivity of frontside-illuminated, silicon single-photon avalanche diode (SPAD) arrays has often suffered from fill factor limitations. The fill factor loss can however be recovered by employing microlenses, whereby the challenges specific to SPAD arrays are represented by large pixel pitch (> 10 µm), low native fill factor (as low as ∼10%), and large size (up to 10 mm). In this work we report on the implementation of refractive microlenses by means of photoresist masters, used to fabricate molds for imprints of UV curable hybrid polymers deposited on SPAD arrays. Replications were successfully carried out for the first time, to the best of our knowledge, at wafer reticle level on different designs in the same technology and on single large SPAD arrays with very thin residual layers (∼10 µm), as needed for better efficiency at higher numerical aperture (NA > 0.25). In general, concentration factors within 15-20% of the simulation results were obtained for the smaller arrays (32×32 and 512×1), achieving for example an effective fill factor of 75.6-83.2% for a 28.5 µm pixel pitch with a native fill factor of 28%. A concentration factor up to 4.2 was measured on large 512×512 arrays with a pixel pitch of 16.38 µm and a native fill factor of 10.5%, whereas improved simulation tools could give a better estimate of the actual concentration factor. Spectral measurements were also carried out, resulting in good and uniform transmission in the visible and NIR.

Visualising varnish removal for conservation of paintings by fluorescence lifetime imaging (FLIM)

The removal of varnish from the surface is a key step in painting conservation. Varnish removal is traditionally monitored by examining the painting surface under ultraviolet illumination. We show here that by imaging the fluorescence lifetime instead, much better contrast, sensitivity, and specificity can be achieved. For this purpose, we developed a lightweight (4.8 kg) portable instrument for macroscopic fluorescence lifetime imaging (FLIM). It is based on a time-correlated single-photon avalanche diode (SPAD) camera to acquire the FLIM images and a pulsed 440 nm diode laser to excite the varnish fluorescence. A historical model painting was examined to demonstrate the capabilities of the system. We found that the FLIM images provided information on the distribution of the varnish on the painting surface with greater sensitivity, specificity, and contrast compared to the traditional ultraviolet illumination photography. The distribution of the varnish and other painting materials was assessed using FLIM during and after varnish removal with different solvent application methods. Monitoring of the varnish removal process between successive solvent applications by a swab revealed an evolving image contrast as a function of the cleaning progress. FLIM of dammar and mastic resin varnishes identified characteristic changes to their fluorescence lifetimes depending on their ageing conditions. Thus, FLIM has a potential to become a powerful and versatile tool to visualise varnish removal from paintings.

Graphical Abstract

Development of a high-speed line-scanning fluorescence lifetime imaging microscope for biological imaging

We report the development of a novel line-scanning microscope capable of acquiring high-speed time-correlated single-photon counting (TCSPC)-based fluorescence lifetime imaging microscopy (FLIM) imaging. The system consists of a laser-line focus, which is optically conjugated to a 1024 × 8 single-photon avalanche diode (SPAD)-based line-imaging complementary metal-oxide semiconductor (CMOS), with 23.78 µm pixel pitch at 49.31% fill factor. Incorporation of on-chip histogramming on the line-sensor enables acquisition rates 33 times faster than our previously reported bespoke high-speed FLIM platforms. We demonstrate the imaging capability of the high-speed FLIM platform in a number of biological applications.

NIR Fluorescence lifetime macroscopic imaging with a novel time-gated SPAD camera

SwissSPAD3 is the latest of a family of widefield time-gated SPAD imagers developed for fluorescence lifetime imaging (FLI) applications. Its distinctive features are (i) the ability to define shorter gates than its predecessors (width W < 1 ns), (ii) support for laser repetition rates up to at least 80 MHz and (iii) a dual-gate architecture providing an effective duty cycle of 100%. We present widefield macroscopic FLI measurements of short lifetime NIR dyes, analyzed using the phasor approach. The results are compared with those previously obtained with SwissSPAD2 and to theoretical predictions.

Light-field microscopy with correlated beams for high-resolution volumetric imaging

Light-field microscopy represents a promising solution for microscopic volumetric imaging, thanks to its capability to encode information on multiple planes in a single acquisition. This is achieved through its peculiar simultaneous capture of information on light spatial distribution and propagation direction. However, state-of-the-art light-field microscopes suffer from a detrimental loss of spatial resolution compared to standard microscopes. In this article, we experimentally demonstrate the working principle of a new scheme, called Correlation Light-field Microscopy (CLM), where the correlation between two light beams is exploited to achieve volumetric imaging with a resolution that is only limited by diffraction. In CLM, a correlation image is obtained by measuring intensity correlations between a large number of pairs of ultra-short frames; each pair of frames is illuminated by the two correlated beams, and is exposed for a time comparable with the source coherence time. We experimentally show the capability of CLM to recover the information contained in out-of-focus planes within three-dimensional test targets and biomedical phantoms. In particular, we demonstrate the improvement of the depth of field enabled by CLM with respect to a conventional microscope characterized by the same resolution. Moreover, the multiple perspectives contained in a single correlation image enable reconstructing over 50 distinguishable transverse planes within a 1 mm³ sample.

Improving the performance of rapid lifetime determination for wide-field time-gated imaging in live cells

In biological research, rapid wide-field fluorescence lifetime imaging has become an important imaging tool. However, the biological samples with weak fluorescence signals and lower sensitivity often suffer from very low precision in lifetime determinations which restricts its widespread utilization in many bioimaging applications. To address this issue, a method is presented in this paper to substantially enhance the precision of rapid lifetime determination (RLD). It expedites the discrimination of fluorescence lifetimes, even for the weak signals coming from the cells, stained with long-lived biocompatible AIS/ZnS QDs. The proposed method works in two phases. The first phase deals with the systematic noise analysis based on the signal and contrast of the images in a time-gated imaging system, wherein acquiring the high-quality imaging data through optimization of hardware parameters improves the overall system performance. In the second phase, the chosen images are treated using total variation denoising method combined with the Max/Min filtering method for extracting the region of interest to reconstruct the intensity images for RLD. We performed several experiments on live cells to demonstrate the improvements in imaging performance by the systematic optimizations and data treatment. Obtained results demonstrated a great enhancement in signal-to-noise and contrast-to-noise ratios beside witnessing an obvious improvement in RLD for weak signals. This approach can be used not only to improve the quality of time-gated imaging data but also for efficient fluorescence lifetime imaging of live biological samples without compromising imaging speed and light exposure.

Single-Molecule Fluorescence Lifetime Imaging Using Wide-Field and Confocal-Laser Scanning Microscopy: A Comparative Analysis

A recent addition to the toolbox of super-resolution microscopy methods is fluorescence-lifetime single-molecule localization microscopy (FL-SMLM). The synergy of SMLM and fluorescence-lifetime imaging microscopy (FLIM) combines superior image resolution with lifetime information and can be realized using two complementary experimental approaches: confocal-laser scanning microscopy (CLSM) or wide-field microscopy. Here, we systematically and comprehensively compare these two novel FL-SMLM approaches in different spectral regions. For wide-field FL-SMLM, we use a commercial lifetime camera, and for CLSM-based FL-SMLM we employ a home-built system equipped with a rapid scan unit and a single-photon detector. We characterize the performances of the two systems in localizing single emitters in 3D by combining FL-SMLM with metal-induced energy transfer (MIET) for localization along the third dimension and in the lifetime-based multiplexed bioimaging using DNA-PAINT. Finally, we discuss advantages and disadvantages of wide-field and confocal FL-SMLM and provide practical advice on rational FL-SMLM experiment design.

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera

Near-infrared (NIR) fluorescence lifetime imaging (FLI) provides a unique contrast mechanism to monitor biological parameters and molecular events in vivo. Single-photon avalanche diode (SPAD) cameras have been recently demonstrated in FLI microscopy (FLIM) applications, but their suitability for in vivo macroscopic FLI (MFLI) in deep tissues remains to be demonstrated. Herein, we report in vivo NIR MFLI measurement with SwissSPAD2, a large time-gated SPAD camera. We first benchmark its performance in well-controlled in vitro experiments, ranging from monitoring environmental effects on fluorescence lifetime, to quantifying Förster resonant energy transfer (FRET) between dyes. Next, we use it for in vivo studies of target-drug engagement in live and intact tumor xenografts using FRET. Information obtained with SwissSPAD2 was successfully compared to that obtained with a gated intensified charge-coupled device (ICCD) camera, using two different approaches. Our results demonstrate that SPAD cameras offer a powerful technology for in vivo preclinical applications in the NIR window.

NIR Fluorescence lifetime macroscopic imaging with a time-gated SPAD camera

The performance of SwissSPAD2 (SS2), a large scale, widefield time-gated CMOS SPAD imager developed for fluorescence lifetime imaging, has recently been described in the context of visible range and fluorescence lifetime imaging microscopy (FLIM) of dyes with lifetimes in the 2.5 - 4 ns range. Here, we explore its capabilities in the NIR regime relevant for small animal imaging, where its sensitivity is lower and typical NIR fluorescent dye lifetimes are much shorter (1 ns or less). We carry out this study in a simple macroscopic imaging setup based on a compact NIR picosecond pulsed laser, an engineered diffuser-based illumination optics, and NIR optimized imaging lens suitable for well-plate or small animal imaging. Because laser repetition rates can vary between models, but the synchronization signal frequency accepted by SS2 is fixed to 20 MHz, we first checked that a simple frequency-division scheme enables data recording for different laser repetition rates. Next, we acquired data using different time gate widths, including gates with duration longer than the laser period, and analyzed the resulting data using both standard nonlinear least-square fit (NLSF) and phasor analysis. We show that the fixed synchronization rate and large gate widths characterizing SS2 (10 ns and over) are not an obstacle to accurately extracting lifetime in the 1 ns range and to distinguishing between close lifetimes. In summary, SS2 and similar very large gated SPAD imagers appear as a versatile alternative to other widefield time-resolved detectors for NIR fluorescence lifetime imaging, including preclinical molecular applications.

Theoretical minimum uncertainty of single-molecule localizations using a single-photon avalanche diode array

Single-photon avalanche diode (SPAD) arrays can be used for single-molecule localization microscopy (SMLM) because of their high frame rate and lack of readout noise. SPAD arrays have a binary frame output, which means photon arrivals should be described as a binomial process rather than a Poissonian process. Consequentially, the theoretical minimum uncertainty of the localizations is not accurately predicted by the Poissonian Cramér-Rao lower bound (CRLB). Here, we derive a binomial CRLB and benchmark it using simulated and experimental data. We show that if the expected photon count is larger than one for all pixels within one standard deviation of a Gaussian point spread function, the binomial CRLB gives a 46% higher theoretical uncertainty than the Poissonian CRLB. For typical SMLM photon fluxes, where no saturation occurs, the binomial CRLB predicts the same uncertainty as the Poissonian CRLB. Therefore, the binomial CRLB can be used to predict and benchmark localization uncertainty for SMLM with SPAD arrays for all practical emitter intensities.

Antioxidant assessment of agricultural produce using fluorescence techniques: a review

The study of bioactive compounds like food antioxidants is getting huge attention and curiosity by researchers and other relevant stakeholders (e.g., food and pharmaceutical industries) due to their health benefits. However, the currently available protocols to estimate the antioxidant activity of foods are time-consuming, destructive, require complex procedures for sample preparation, need technical persons, and not possible for real-time application, which are very important for large-scale or industrial applications. On the other hand, fluorescence spectroscopy and imaging techniques are relatively new, fast, mostly nondestructive, and possible to apply real-time to detect the antioxidants of foods. However, there is no review article on fluorescence techniques for estimating antioxidants in agricultural produces. Therefore, the present review comprehensively summarizes the overview of fluorescence phenomena, techniques (i.e., spectroscopy and computer vision), and their potential to monitor antioxidants in fruits and vegetables. Finally, opportunities and challenges of fluorescence techniques are described toward developing next-generation protocols for antioxidants measurement. Fluorescence techniques (both spectroscopy and imaging) are simpler and faster than available traditional methods of antioxidants measurement. Moreover, the fluorescence imaging technique has the potential to apply in real-time antioxidant identification in agricultural produce such as fruits and vegetables. Therefore, this technique might be used as a next-generation protocol for qualitative and quantitative antioxidants measurement after improvements like new material technologies for sensor (detector) and light sources for higher sensitivity and reduce the cost of implementing real-world applications.

Towards Quantum 3D Imaging Devices

We review the advancement of the research toward the design and implementation of quantum plenoptic cameras, radically novel 3D imaging devices that exploit both momentum–position entanglement and photon–number correlations to provide the typical refocusing and ultra-fast, scanning-free, 3D imaging capability of plenoptic devices, along with dramatically enhanced performances, unattainable in standard plenoptic cameras: diffraction-limited resolution, large depth of focus, and ultra-low noise. To further increase the volumetric resolution beyond the Rayleigh diffraction limit, and achieve the quantum limit, we are also developing dedicated protocols based on quantum Fisher information. However, for the quantum advantages of the proposed devices to be effective and appealing to end-users, two main challenges need to be tackled. First, due to the large number of frames required for correlation measurements to provide an acceptable signal-to-noise ratio, quantum plenoptic imaging (QPI) would require, if implemented with commercially available high-resolution cameras, acquisition times ranging from tens of seconds to a few minutes. Second, the elaboration of this large amount of data, in order to retrieve 3D images or refocusing 2D images, requires high-performance and time-consuming computation. To address these challenges, we are developing high-resolution single-photon avalanche photodiode (SPAD) arrays and high-performance low-level programming of ultra-fast electronics, combined with compressive sensing and quantum tomography algorithms, with the aim to reduce both the acquisition and the elaboration time by two orders of magnitude. Routes toward exploitation of the QPI devices will also be discussed.

An Overview of CMOS Photodetectors Utilizing Current-Assistance for Swift and Efficient Photo-Carrier Detection

This review paper presents an assortment of research on a family of photodetectors which use the same base mechanism, current assistance, for the operation. Current assistance is used to create a drift field in the semiconductor, more specifically silicon, in order to improve the bandwidth and the quantum efficiency. Based on the detector and application, the drift field can be static or modulated. Applications include 3D imaging (both direct and indirect time-of-flight), optical receivers and fluorescence lifetime imaging. This work discusses the current-assistance principle, the various photodetectors using this principle and a comparison is made with other state-of-the-art photodetectors used for the same application.

Simultaneous multi-spectral, single-photon fluorescence imaging using a plasmonic colour filter array

We present the first realisation of simultaneous multi-spectral fluorescence imaging using a single-photon avalanche diode (SPAD) array, where the spectral unmixing is facilitated by a plasmonic metasurface mosaic colour filter array (CFA). A 64 × 64 pixel format silicon SPAD array is used to record widefield fluorescence and brightfield data from four biological samples. A plasmonic metasurface composed of an arrangement of circular and elliptical nanoholes etched into an aluminium thin film deposited on a glass substrate provides the high transmission efficiency CFA, enabling a bespoke spectral unmixing algorithm to reconstruct high fidelity, full colour images from as few as ∼3 photons per pixel. This approach points the way toward real-time, single-photon sensitive multi-spectral fluorescence imaging. Furthermore, this is possible without additional bulky components such as a filter wheel, prism or diffraction grating, nor the need for multiple sample exposures or multiple detectors.

An overview of continuous and discrete phasor analysis of binned or time-gated periodic decays

Time-resolved analysis of periodically excited luminescence decays by the phasor method in the presence of time-gating or binning is revisited. Analytical expressions for discrete configurations of square gates are derived and the locus of the phasors of such modified periodic single-exponential decays is compared to the canonical universal semicircle. The effects of IRF offset, decay truncation and gate shape are also discussed. Finally, modified expressions for the phase and modulus lifetimes are provided for some simple cases. A discussion of a modified phasor calibration approach is presented.

Continuous and discrete phasor analysis of binned or time-gated periodic decays

The time-resolved analysis of periodically excited luminescence decays by the phasor method in the presence of time-gating or binning is revisited. Analytical expressions for discrete configurations of square gates are derived, and the locus of the phasors of such modified periodic single-exponential decays is compared to the canonical universal semicircle. The effects of instrument response function offset, decay truncation, and gate shape are also discussed. Finally, modified expressions for the phase and modulus lifetimes are provided for some simple cases. A discussion of a modified phasor calibration approach is presented, and an illustration of the new concepts with examples from the literature concludes this work.

Current-Assisted SPAD with Improved p-n Junction and Enhanced NIR Performance

Single-photon avalanche diodes (SPADs) fabricated in conventional CMOS processes typically have limited near infra-red (NIR) sensitivity. This is the consequence of isolating the SPADs in a lowly-doped deep N-type well. In this work, we present a second improved version of the “current-assisted” single-photon avalanche diode, fabricated in a conventional 350 nm CMOS process, having good NIR sensitivity owing to 14 μm thick epilayer for photon absorption. The presented device has a photon absorption area of 30 × 30 µm², with a much smaller central active area for avalanche multiplication. The photo-electrons generated in the absorption area are guided swiftly towards the central area with a drift field created by the “current-assistance” principle. The central active avalanche area has a cylindrical p-n junction as opposed to the square geometry from the previous iteration. The presented device shows improved performance in all aspects, most notably in photon detection probability. The p-n junction capacitance is estimated to be ~1 fF and on-chip passive quenching with source followers is employed to conserve the small capacitance for bringing monitoring signals off-chip. Device physics simulations are presented along with measured dark count rate (DCR), timing jitter, after-pulsing probability (APP) and photon detection probability (PDP). The presented device has a peak PDP of 22.2% at a wavelength of 600 nm and a timing jitter of 220 ps at a wavelength of 750 nm.

Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation

Fluorescence lifetime imaging microscopy (FLIM) is a key technology that provides direct insight into cell metabolism, cell dynamics and protein activity. However, determining the lifetimes of different fluorescent proteins requires the detection of a relatively large number of photons, hence slowing down total acquisition times. Moreover, there are many cases, for example in studies of cell collectives, where wide-field imaging is desired. We report scan-less wide-field FLIM based on a 0.5 MP resolution, time-gated Single Photon Avalanche Diode (SPAD) camera, with acquisition rates up to 1 Hz. Fluorescence lifetime estimation is performed via a pre-trained artificial neural network with 1000-fold improvement in processing times compared to standard least squares fitting techniques. We utilised our system to image HT1080-human fibrosarcoma cell line as well as Convallaria. The results show promise for real-time FLIM and a viable route towards multi-megapixel fluorescence lifetime images, with a proof-of-principle mosaic image shown with 3.6 MP.

Direct Photon-by-Photon Analysis of Time-Resolved Pulsed Excitation Data using Bayesian Nonparametrics

Lifetimes of chemical species are typically estimated by either fitting time-correlated single-photon counting (TCSPC) histograms or phasor analysis from time-resolved photon arrivals. While both methods yield lifetimes in a computationally efficient manner, their performance is limited by choices made on the number of distinct chemical species contributing photons. However, the number of species is encoded in the photon arrival times collected for each illuminated spot and need not be set by hand a priori. Here, we propose a direct photon-by-photon analysis of data drawn from pulsed excitation experiments to infer, simultaneously and self-consistently, the number of species and their associated lifetimes from a few thousand photons. We do so by leveraging new mathematical tools within the Bayesian nonparametric. We benchmark our method for both simulated and experimental data for 1-4 species.

Fluorescence lifetime imaging microscopy: fundamentals and advances in instrumentation, analysis, and applications

SIGNIFICANCE

Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique to distinguish the unique molecular environment of fluorophores. FLIM measures the time a fluorophore remains in an excited state before emitting a photon, and detects molecular variations of fluorophores that are not apparent with spectral techniques alone. FLIM is sensitive to multiple biomedical processes including disease progression and drug efficacy.

AIM

We provide an overview of FLIM principles, instrumentation, and analysis while highlighting the latest developments and biological applications.

APPROACH

This review covers FLIM principles and theory, including advantages over intensity-based fluorescence measurements. Fundamentals of FLIM instrumentation in time- and frequency-domains are summarized, along with recent developments. Image segmentation and analysis strategies that quantify spatial and molecular features of cellular heterogeneity are reviewed. Finally, representative applications are provided including high-resolution FLIM of cell- and organelle-level molecular changes, use of exogenous and endogenous fluorophores, and imaging protein-protein interactions with Förster resonance energy transfer (FRET). Advantages and limitations of FLIM are also discussed.

CONCLUSIONS

FLIM is advantageous for probing molecular environments of fluorophores to inform on fluorophore behavior that cannot be elucidated with intensity measurements alone. Development of FLIM technologies, analysis, and applications will further advance biological research and clinical assessments.

Wide-Field Fluorescence Lifetime Imaging of Single Molecules

Fluorescence lifetime imaging (FLIM) has become an important microscopy technique in bioimaging. The two most important of its applications are lifetime-multiplexing for imaging many different structures in parallel, and lifetime-based measurements of Förster resonance energy transfer. There are two principal FLIM techniques, one based on confocal-laser scanning microscopy (CLSM) and time-correlated single-photon counting (TCSPC) and the other based on wide-field microscopy and phase fluorometry. Although the first approach (CLSM-TCSPC) assures high sensitivity and allows one to detect single molecules, it is slow and has a small photon yield. The second allows, in principal, high frame rates (by 2-3 orders of magnitude faster than CLSM), but it suffers from low sensitivity, which precludes its application for single-molecule imaging. Here, we demonstrate that a novel wide-field TCSPC camera (LINCam25, Photonscore GmbH) can be successfully used for single-molecule FLIM, although its quantum yield of detection in the red spectral region is only ∼5%. This is due to the virtually absent background and readout noise of the camera, assuring high signal-to-noise ratio even at low detection efficiency. We performed single-molecule FLIM of different red fluorophores, and we use the lifetime information for successfully distinguishing between different molecular species. Finally, we demonstrate single-molecule metal-induced energy transfer (MIET) imaging which is a first step for three-dimensional single-molecule localization microscopy (SMLM) with nanometer resolution.

Single-Photon, Time-Gated, Phasor-Based Fluorescence Lifetime Imaging through Highly Scattering Medium

Fluorescence lifetime imaging (FLI) is increasingly recognized as a powerful tool for biochemical and cellular investigations, including in vivo applications. Fluorescence lifetime is an intrinsic characteristic of any fluorescent dye which, to a large extent, does not depend on excitation intensity and signal level. In particular, it allows distinguishing dyes with similar emission spectra, offering additional multiplexing capabilities. However, in vivo FLI in the visible range is complicated by the contamination by (i) tissue autofluorescence, which decreases contrast, and by (ii) light scattering and absorption in tissues, which significantly reduce fluorescence intensity and modify the temporal profile of the signal. Here, we demonstrate how these issues can be accounted for and overcome, using a new time-gated single-photon avalanche diode array camera, SwissSPAD2, combined with phasor analysis to provide a simple and fast visual method for lifetime imaging. In particular, we show how phasor dispersion increases with increasing scattering and/or decreasing fluorescence intensity. Next, we show that as long as the fluorescence signal of interest is larger than the phantom autofluorescence, the presence of a distinct lifetime can be clearly identified with appropriate background correction. We use these results to demonstrate the detection of A459 cells expressing the fluorescent protein mCyRFP1 through highly scattering and autofluorescent phantom layers. These results showcase the possibility to perform FLI in challenging conditions, using standard, bright, visible fluorophore or fluorescence proteins.