Two-photon image-scanning microscopy with SPAD array and blind image reconstruction

Deep Mouse Brain Two-Photon Near-Infrared Fluorescence Imaging Using a Superconducting Nanowire Single-Photon Detector Array

Two-photon microscopy (2PM) has become an important tool in biology to study the structure and function of intact tissues in vivo. However, adult mammalian tissues such as the mouse brain are highly scattering, thereby putting fundamental limits on the achievable imaging depth, which typically reside at around 600-800 μm. In principle, shifting both the excitation as well as (fluorescence) emission light to the shortwave near-infrared (SWIR, 1000-1700 nm) region promises substantially deeper imaging in 2PM, yet this shift has proven challenging in the past due to the limited availability of detectors and probes in this wavelength region. To overcome these limitations and fully capitalize on the SWIR region, in this work, we introduce a novel array of superconducting nanowire single-photon detectors (SNSPDs) and associated custom detection electronics for use in near-infrared 2PM. The SNSPD array exhibits high efficiency and dynamic range as well as low dark-count rates over a wide wavelength range. Additionally, the electronics and software permit a seamless integration into typical 2PM systems. Together with an organic fluorescent dye emitting at 1105 nm, we report imaging depth of >1.1 mm in the in vivo mouse brain, limited mostly by available labeling density and laser properties. Our work establishes a promising, and ultimately scalable, new detector technology for SWIR 2PM that facilitates deep tissue biological imaging.

4D Single-particle tracking with asynchronous read-out single-photon avalanche diode array detector

Single-particle tracking techniques enable investigation of the complex functions and interactions of individual particles in biological environments. Many such techniques exist, each demonstrating trade-offs between spatiotemporal resolution, spatial and temporal range, technical complexity, and information content. To mitigate these trade-offs, we enhanced a confocal laser scanning microscope with an asynchronous read-out single-photon avalanche diode array detector. This detector provides an image of the particle's emission, precisely reflecting its position within the excitation volume. This localization is utilized in a real-time feedback system to drive the microscope scanning mechanism and ensure the particle remains centered inside the excitation volume. As each pixel is an independent single-photon detector, single-particle tracking is combined with fluorescence lifetime measurement. Our system achieves 40 nm lateral and 60 nm axial localization precision with 100 photons and sub-millisecond temporal sampling for real-time tracking. Offline tracking can refine this precision to the microsecond scale. We validated the system's spatiotemporal resolution by tracking fluorescent beads with diffusion coefficients up to 10 μm2/s. Additionally, we investigated the movement of lysosomes in living SK-N-BE cells and measured the fluorescence lifetime of the marker expressed on a membrane protein. We expect that this implementation will open other correlative imaging and tracking studies.

More than double the fun with two-photon excitation microscopy

For generations researchers have been observing the dynamic processes of life through the lens of a microscope. This has offered tremendous insights into biological phenomena that span multiple orders of time- and length-scales ranging from the pure magic of molecular reorganization at the membrane of immune cells, to cell migration and differentiation during development or wound healing. Standard fluorescence microscopy techniques offer glimpses at such processes in vitro, however, when applied in intact systems, they are challenged by reduced signal strengths and signal-to-noise ratios that result from deeper imaging. As a remedy, two-photon excitation (TPE) microscopy takes a special place, because it allows us to investigate processes in vivo, in their natural environment, even in a living animal. Here, we review the fundamental principles underlying TPE aimed at basic and advanced microscopy users interested in adopting TPE for intravital imaging. We focus on applications in neurobiology, present current trends towards faster, wider and deeper imaging, discuss the combination with photon counting technologies for metabolic imaging and spectroscopy, as well as highlight outstanding issues and drawbacks in development and application of these methodologies.

Image scanning microscopy: a vectorial physical optics analysis

Image scanning microscopy (ISM) achieves resolution beyond the diffraction limit by a factor of 2. However, prior ISM research predominantly employs scalar diffraction theory, neglecting critical physical effects such as polarization, aberrations, and Stokes shift. This paper presents a comprehensive vectorial ISM point spread function (PSF) model that accounts for these phenomena. By considering the effect of polarization in emission and excitation paths, as well as aberrations and Stokes shift, our model provides a more accurate representation of ISM. We analyze the differences between scalar and vectorial theories in ISM and investigate the impact of pinhole size and aberration strength on resolution. At a numerical aperture of 1.2, the full width half maximum (FWHM) discrepancy between scalar and vectorial ISM PSFs can reach 45 nm, representing a 30% deviation from the vectorial model. Additionally, we explore multiphoton excitation in ISM and observe increased FWHM for 2-photon and 3-photon excitation compared to 1-photon excitation. The FWHM of the 2-photon excitation ISM PSF increases by 20% and the FWHM of the 3-photon excitation ISM PSF increases by 28% compared to the 1-photon excitation ISM. In addition, we found that the optimal sweep factor for 2-photon ISM is 1.22, and the optimal sweep factor of 3-photon ISM is 1.12 instead of the 2 predicted by the one-photon scalar ISM theory. Our work improves the understanding of ISM and contributes to its advancement as a high-resolution imaging technique.

Single-photon microscopy to study biomolecular condensates

Biomolecular condensates serve as membrane-less compartments within cells, concentrating proteins and nucleic acids to facilitate precise spatial and temporal orchestration of various biological processes. The diversity of these processes and the substantial variability in condensate characteristics present a formidable challenge for quantifying their molecular dynamics, surpassing the capabilities of conventional microscopy. Here, we show that our single-photon microscope provides a comprehensive live-cell spectroscopy and imaging framework for investigating biomolecular condensation. Leveraging a single-photon detector array, single-photon microscopy enhances the potential of quantitative confocal microscopy by providing access to fluorescence signals at the single-photon level. Our platform incorporates photon spatiotemporal tagging, which allowed us to perform time-lapse super-resolved imaging for molecular sub-diffraction environment organization with simultaneous monitoring of molecular mobility, interactions, and nano-environment properties through fluorescence lifetime fluctuation spectroscopy. This integrated correlative study reveals the dynamics and interactions of RNA-binding proteins involved in forming stress granules, a specific type of biomolecular condensates, across a wide range of spatial and temporal scales. Our versatile framework opens up avenues for exploring a broad spectrum of biomolecular processes beyond the formation of membrane-less organelles.

Tutorial: multiphoton microscopy to advance neuroscience research

Multiphoton microscopy (MPM) employs ultrafast infrared lasers for high-resolution deep three-dimensional imaging of live biological samples. The goal of this tutorial is to provide a practical guide to MPM imaging for novice microscopy developers and life-science users. Principles of MPM, microscope setup, and labeling strategies are discussed. Use of MPM to achieve unprecedented imaging depth of whole mounted explants and intravital imaging via implantable glass windows of the mammalian nervous system is demonstrated.

Focus image scanning microscopy for sharp and gentle super-resolved microscopy

To date, the feasibility of super-resolution microscopy for imaging live and thick samples is still limited. Stimulated emission depletion (STED) microscopy requires high-intensity illumination to achieve sub-diffraction resolution, potentially introducing photodamage to live specimens. Moreover, the out-of-focus background may degrade the signal stemming from the focal plane. Here, we propose a new method to mitigate these limitations without drawbacks. First, we enhance a STED microscope with a detector array, enabling image scanning microscopy (ISM). Therefore, we implement STED-ISM, a method that exploits the working principle of ISM to reduce the depletion intensity and achieve a target resolution. Later, we develop Focus-ISM, a strategy to improve the optical sectioning and remove the background of any ISM-based imaging technique, with or without a STED beam. The proposed approach requires minimal architectural changes to a conventional microscope but provides substantial advantages for live and thick sample imaging.

The BrightEyes-TTM as an open-source time-tagging module for democratising single-photon microscopy

Fluorescence laser-scanning microscopy (LSM) is experiencing a revolution thanks to new single-photon (SP) array detectors, which give access to an entirely new set of single-photon information. Together with the blooming of new SP LSM techniques and the development of tailored SP array detectors, there is a growing need for (i) DAQ systems capable of handling the high-throughput and high-resolution photon information generated by these detectors, and (ii) incorporating these DAQ protocols in existing fluorescence LSMs. We developed an open-source, low-cost, multi-channel time-tagging module (TTM) based on a field-programmable gate array that can tag in parallel multiple single-photon events, with 30 ps precision, and multiple synchronisation events, with 4 ns precision. We use the TTM to demonstrate live-cell super-resolved fluorescence lifetime image scanning microscopy and fluorescence lifetime fluctuation spectroscopy. We expect that our BrightEyes-TTM will support the microscopy community in spreading SP-LSM in many life science laboratories.

In vivo super-resolution of the brain - How to visualize the hidden nanoplasticity?

Super-resolution fluorescence microscopy has entered most biological laboratories worldwide and its benefit is undisputable. Its application to brain imaging, for example in living mice, enables the study of sub-cellular structural plasticity and brain function directly in a living mammal. The demands of brain imaging on the different super-resolution microscopy techniques (STED, RESOLFT, SIM, ISM) and labeling strategies are discussed here as well as the challenges of the required cranial window preparation. Applications of super-resolution in the anesthetized mouse brain enlighten the stability and plasticity of synaptic nanostructures. These studies show the potential of in vivo super-resolution imaging and justify its application more widely in vivo to investigate the role of nanostructures in memory and learning.

Multiphoton microscopy for label-free multicolor imaging of peripheral nerve

SIGNIFICANCE

Means for quantitation of myelinated fibers in peripheral nerve may guide diagnosis and clinical decision making in management of peripheral nerve disorders. Multiphoton microscopy techniques such as the third-harmonic generation enable label-free in vivo imaging of peripheral nerves.

AIM

Develop a multiphoton microscope based on a custom high-power infrared fiber laser for label-free imaging of peripheral nerve.

APPROACH

A cost-effective multiphoton microscope employing a single fiber laser source at 1300 nm was designed and used for stain-free multicolor imaging of murine and human peripheral nerve.

RESULTS

Second-harmonic generation signal from collagen centered about 650-nm delineated neural connective tissue, whereas third-harmonic general signal centered about 433-nm delineated myelin and other lipids. In sciatic nerve from transgenic reporter mice expressing yellow fluorescent protein within peripheral neurons, three-photon-excitation with emission peak at 527-nm delineated axoplasm. The signal obtained from unlabeled axially sectioned samples was adequate for segmentation of myelinated fibers using commercial image processing software. In unlabeled whole mount specimens, imaging depths over 100-μm were achieved.

CONCLUSIONS

A multiphoton microscope powered by a fiber laser enables stain-free histomorphometry of mammalian peripheral nerve. The simplicity of the microscope design carries potential for clinical translation to inform decision making in peripheral nerve disorders.

Saturated-excitation image scanning microscopy

Image scanning microscopy (ISM) overcomes the trade-off between spatial resolution and signal volume in confocal microscopy by rearranging the signal distribution on a two-dimensional detector array to achieve a spatial resolution close to the theoretical limit achievable by infinitesimal pinhole detection without sacrificing the detected signal intensity. In this paper, we improved the spatial resolution of ISM in three dimensions by exploiting saturated excitation (SAX) of fluorescence. We theoretically investigated the imaging properties of ISM, when the fluorescence signals are nonlinearly induced by SAX, and show combined SAX-ISM fluorescence imaging to demonstrate the improvement of the spatial resolution in three dimensions. In addition, we confirmed that the SNR of SAX-ISM imaging of fluorescent beads and biological samples, which is one of the challenges in conventional SAX microscopy, was improved.

Cooled SPAD array detector for low light-dose fluorescence laser scanning microscopy

The single-photon timing and sensitivity performance and the imaging ability of asynchronous-readout single-photon avalanche diode (SPAD) array detectors have opened up enormous perspectives in fluorescence (lifetime) laser scanning microscopy (FLSM), such as super-resolution image scanning microscopy and high-information content fluorescence fluctuation spectroscopy. However, the strengths of these FLSM techniques depend on the many different characteristics of the detector, such as dark noise, photon-detection efficiency, after-pulsing probability, and optical cross talk, whose overall optimization is typically a trade-off between these characteristics. To mitigate this trade-off, we present, to our knowledge, a novel SPAD array detector with an active cooling system that substantially reduces the dark noise without significantly deteriorating any other detector characteristics. In particular, we show that lowering the temperature of the sensor to −15°C significantly improves the signal/noise ratio due to a 10-fold decrease in the dark count rate compared with room temperature. As a result, for imaging, the laser power can be decreased by more than a factor of three, which is particularly beneficial for live-cell super-resolution imaging, as demonstrated in fixed and living cells expressing green-fluorescent-protein-tagged proteins. For fluorescence fluctuation spectroscopy, together with the benefit of the reduced laser power, we show that cooling the detector is necessary to remove artifacts in the correlation function, such as spurious negative correlations observed in the hot elements of the detector, i.e., elements for which dark noise is substantially higher than the median value. Overall, this detector represents a further step toward the integration of SPAD array detectors in any FLSM system.

Deep learning enables confocal laser-scanning microscopy with enhanced resolution

Theoretical resolution enhancement of confocal laser-scanning microscopy (CLSM) is sacrificed for the best compromise between optical sectioning and the signal-to-noise ratio (SNR). The pixel reassignment reconstruction algorithm can improve the effective spatial resolution of CLSM to its theoretical limit. However, current implementations are not versatile and are time-consuming or technically complex. Here we present a parameter-free post-processing strategy for laser-scanning microscopy based on deep learning, which enables a spatial resolution enhancement by a factor of ∼1.3, compared to conventional CLSM. To speed up the training process for experimental data, transfer learning, combined with a hybrid dataset consisting of simulated synthetic and experimental images, is employed. The overall resolution and SNR improvement, validated by quantitative evaluation metrics, allowed us to correctly infer the fine structures of real experimental images.

The Development of Microscopy for Super-Resolution: Confocal Microscopy, and Image Scanning Microscopy

Optical methods of super-resolution microscopy, such as confocal microscopy, structured illumination, nonlinear microscopy, and image scanning microscopy are reviewed. These methods avoid strong invasive interaction with a sample, allowing the observation of delicate biological samples. The meaning of resolution and the basic principles and different approaches to superresolution are discussed.

Noise and Breakdown Characterization of SPAD Detectors with Time-Gated Photon-Counting Operation

Being ready-to-detect over a certain portion of time makes the time-gated single-photon avalanche diode (SPAD) an attractive candidate for low-noise photon-counting applications. A careful SPAD noise and performance characterization, however, is critical to avoid time-consuming experimental optimization and redesign iterations for such applications. Here, we present an extensive empirical study of the breakdown voltage, as well as the dark-count and afterpulsing noise mechanisms for a fully integrated time-gated SPAD detector in 0.35-μm CMOS based on experimental data acquired in a dark condition. An "effective" SPAD breakdown voltage is introduced to enable efficient characterization and modeling of the dark-count and afterpulsing probabilities with respect to the excess bias voltage and the gating duration time. The presented breakdown and noise models will allow for accurate modeling and optimization of SPAD-based detector designs, where the SPAD noise can impose severe trade-offs with speed and sensitivity as is shown via an example.

Structured illumination microscopy and image scanning microscopy: a review and comparison of imaging properties

Structured illumination microscopy and image scanning microscopy are two microscopical tech- niques, rapidly increasing in practical application, that can result in improvement in transverse spatial resolution, and/or improvement in axial imaging performance. The history and principles of these techniques are reviewed, and the imaging properties of the two methods compared. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

cSPARCOM: Multi-detector reconstruction by confocal super-resolution correlation microscopy

Image scanning microscopy (ISM), an upgraded successor of the ubiquitous confocal microscope, facilitates up to two-fold improvement in lateral resolution, and has become an indispensable element in the toolbox of the bio-imaging community. Recently, super-resolution optical fluctuation image scanning microscopy (SOFISM) integrated the analysis of intensity-fluctuations information into the basic ISM architecture, to enhance its resolving power. Both of these techniques typically rely on pixel-reassignment as a fundamental processing step, in which the parallax of different detector elements to the sample is compensated by laterally shifting the point spread function (PSF). Here, we propose an alternative analysis approach, based on the recent high-performing sparsity-based super-resolution correlation microscopy (SPARCOM) method. Through measurements of DNA origami nano-rulers and fixed cells labeled with organic dye, we experimentally show that confocal SPARCOM (cSPARCOM), which circumvents pixel-reassignment altogether, provides enhanced resolution compared to pixel-reassigned based analysis. Thus, cSPARCOM further promotes the effectiveness of ISM, and particularly that of correlation based ISM implementations such as SOFISM, where the PSF deviates significantly from spatial invariance.

Confocal-based fluorescence fluctuation spectroscopy with a SPAD array detector

The combination of confocal laser-scanning microscopy (CLSM) and fluorescence fluctuation spectroscopy (FFS) is a powerful tool in studying fast, sub-resolution biomolecular processes in living cells. A detector array can further enhance CLSM-based FFS techniques, as it allows the simultaneous acquisition of several samples-essentially images-of the CLSM detection volume. However, the detector arrays that have previously been proposed for this purpose require tedious data corrections and preclude the combination of FFS with single-photon techniques, such as fluorescence lifetime imaging. Here, we solve these limitations by integrating a novel single-photon-avalanche-diode (SPAD) array detector in a CLSM system. We validate this new implementation on a series of FFS analyses: spot-variation fluorescence correlation spectroscopy, pair-correlation function analysis, and image-derived mean squared displacement analysis. We predict that the unique combination of spatial and temporal information provided by our detector will make the proposed architecture the method of choice for CLSM-based FFS.

Image scanning microscopy with a long depth of focus generated by an annular radially polarized beam

Image scanning microscopy (ISM) is a promising tool for bioimaging owing to its integration of signal to noise ratio (SNR) and super resolution superior to that obtained in confocal scanning microscopy. In this paper, we introduce the annular radially polarized beam to the ISM, which yields an axially extended excitation focus and enhanced resolution, providing a new possibility to obtain the whole information of thick specimen with a single scan. We present the basic principle and a rigorous theoretical model for ISM with annular radially polarized beam (ISM-aRP). Results show that the resolution of ISM-aRP can be enhanced by 4% compared with that in conventional ISM, and the axial extent of the focus is longer than 6λ. The projected view of the simulated fluorescent beads suspension specimen demonstrates the validity of ISM-aRP to obtain the whole information of volume sample. Moreover, this simple method can be easily integrated into the commercial laser scanning microscopy systems.

Object detection neural network improves Fourier ptychography reconstruction

High resolution microscopy is heavily dependent on superb optical elements and superresolution microscopy even more so. Correcting unavoidable optical aberrations during post-processing is an elegant method to reduce the optical system's complexity. A prime method that promises superresolution, aberration correction, and quantitative phase imaging is Fourier ptychography. This microscopy technique combines many images of the sample, recorded at differing illumination angles akin to computed tomography and uses error minimisation between the recorded images with those generated by a forward model. The more precise knowledge of those illumination angles is available for the image formation forward model, the better the result. Therefore, illumination estimation from the raw data is an important step and supports correct phase recovery and aberration correction. Here, we derive how illumination estimation can be cast as an object detection problem that permits the use of a fast convolutional neural network (CNN) for this task. We find that faster-RCNN delivers highly robust results and outperforms classical approaches by far with an up to 3-fold reduction in estimation errors. Intriguingly, we find that conventionally beneficial smoothing and filtering of raw data is counterproductive in this type of application. We present a detailed analysis of the network's performance and provide all our developed software openly.

Image scanning microscopy with multiphoton excitation or Bessel beam illumination

Image scanning microscopy is a technique of confocal microscopy in which the confocal pinhole is replaced by a detector array, and the image is reconstructed most straightforwardly by pixel reassignment. In the fluorescence mode, the detector array collects most of the fluorescent light, so the signal-to-noise ratio is much improved compared with confocal microscopy with a small pinhole, while the resolution is improved compared with conventional fluorescence microscopy. Here we consider two cases in which the illumination and detection point spread functions are dissimilar: illumination with a Bessel beam and multiphoton microscopy. It has been shown previously that for Bessel beam illumination in image scanning microscopy with a large array, the imaging performance is degraded. On the other hand, it is also known that the resolution of confocal microscopy is improved by Bessel beam illumination. Here we analyze image scanning microscopy with Bessel beam illumination together with a small array and show that an improvement in transverse resolution (width of the point spread function) by a factor of 1.78 compared with a conventional fluorescence microscope can be obtained. We also examine the behavior of image scanning microscopy in two- or three-photon fluorescence and for two-photon excitation also with Bessel beam illumination. The combination of the optical sectioning effect of image scanning microscopy with multiphoton microscopy reduces background from the sample surface, which can increase penetration depth. For a detector array size of two Airy units, the resolution of two-photon image scanning microscopy is a factor 1.85 better and the peak of the point spread function 2.84 times higher than in nonconfocal two-photon fluorescence. The resolution of three-photon image scanning microscopy is a factor 2.10 better, and the peak of the point spread function is 3.77 times higher than in nonconfocal three-photon fluorescence. The resolution of two-photon image scanning microscopy with Bessel beam illumination is a factor 2.13 better than in standard two-photon fluorescence. Axial resolution and optical sectioning in two-photon or three-photon fluorescence are also improved by using the image scanning modality.

In vivo two-photon microscopic observation and ablation in deeper brain regions realized by modifications of excitation beam diameter and immersion liquid

In vivo two-photon microscopy utilizing a nonlinear optical process enables, in living mouse brains, not only the visualization of morphologies and functions of neural networks in deep regions but also their optical manipulation at targeted sites with high spatial precision. Because the two-photon excitation efficiency is proportional to the square of the photon density of the excitation laser light at the focal position, optical aberrations induced by specimens mainly limit the maximum depth of observations or that of manipulations in the microscopy. To increase the two-photon excitation efficiency, we developed a method for evaluating the focal volume in living mouse brains. With this method, we modified the beam diameter of the excitation laser light and the value of the refractive index in the immersion liquid to maximize the excitation photon density at the focal position. These two modifications allowed the successful visualization of the finer structures of hippocampal CA1 neurons, as well as the intracellular calcium dynamics in cortical layer V astrocytes, even with our conventional two-photon microscopy system. Furthermore, it enabled focal laser ablation dissection of both single apical and single basal dendrites of cortical layer V pyramidal neurons. These simple modifications would enable us to investigate the contributions of single cells or single dendrites to the functions of local cortical networks.