Enhancement of lateral resolution and optical sectioning capability of two-photon fluorescence microscopy by combining temporal-focusing with structured illumination

Cascaded domain multiphoton spatial frequency modulation imaging

Significance

Multiphoton microscopy is a powerful imaging tool for biomedical applications. A variety of techniques and respective benefits exist for multiphoton microscopy, but an enhanced resolution is especially desired. Additionally multiphoton microscopy requires ultrafast pulses for excitation, so optimization of the pulse duration at the sample is critical for strong signals.

Aim

We aim to perform enhanced resolution imaging that is robust to scattering using a structured illumination technique while also providing a rapid and easily repeatable means to optimize group delay dispersion (GDD) compensation through to the sample.

Approach

Spatial frequency modulation imaging (SPIFI) is used in two domains: the spatial domain (SD) and the wavelength domain (WD). The WD-SPIFI system is an in-line tool enabling GDD optimization that considers all material through to the sample. The SD-SPIFI system follows and enables enhanced resolution imaging.

Results

The WD-SPIFI dispersion optimization performance is confirmed with independent pulse characterization, enabling rapid optimization of pulses for imaging with the SD-SPIFI system. The SD-SPIFI system demonstrates enhanced resolution imaging without the use of photon counting enabled by signal to noise improvements due to the WD-SPIFI system.

Conclusions

Implementing SPIFI in-line in two domains enables full-path dispersion compensation optimization through to the sample for enhanced resolution multiphoton microscopy.

Temporal focusing multiphoton microscopy with cross-modality multi-stage 3D U-Net for fast and clear bioimaging

Temporal focusing multiphoton excitation microscopy (TFMPEM) enables fast widefield biotissue imaging with optical sectioning. However, under widefield illumination, the imaging performance is severely degraded by scattering effects, which induce signal crosstalk and a low signal-to-noise ratio in the detection process, particularly when imaging deep layers. Accordingly, the present study proposes a cross-modality learning-based neural network method for performing image registration and restoration. In the proposed method, the point-scanning multiphoton excitation microscopy images are registered to the TFMPEM images by an unsupervised U-Net model based on a global linear affine transformation process and local VoxelMorph registration network. A multi-stage 3D U-Net model with a cross-stage feature fusion mechanism and self-supervised attention module is then used to infer in-vitro fixed TFMPEM volumetric images. The experimental results obtained for in-vitro drosophila mushroom body (MB) images show that the proposed method improves the structure similarity index measures (SSIMs) of the TFMPEM images acquired with a 10-ms exposure time from 0.38 to 0.93 and 0.80 for shallow- and deep-layer images, respectively. A 3D U-Net model, pretrained on in-vitro images, is further trained using a small in-vivo MB image dataset. The transfer learning network improves the SSIMs of in-vivo drosophila MB images captured with a 1-ms exposure time to 0.97 and 0.94 for shallow and deep layers, respectively.

Image improvement of temporal focusing multiphoton microscopy via superior spatial modulation excitation and Hilbert-Huang transform decomposition

Temporal focusing-based multiphoton excitation microscopy (TFMPEM) just provides the advantage of widefield optical sectioning ability with axial resolution of several micrometers. However, under the plane excitation, the photons emitted from the molecules in turbid tissues undergo scattering, resulting in complicated background noise and an impaired widefield image quality. Accordingly, this study constructs a general and comprehensive numerical model of TFMPEM utilizing Fourier optics and performs simulations to determine the superior spatial frequency and orientation of the structured pattern which maximize the axial excitation confinement. It is shown experimentally that the optimized pattern minimizes the intensity of the out-of-focus signal, and hence improves the quality of the image reconstructed using the Hilbert transform (HT). However, the square-like reflection components on digital micromirror device leads to pattern residuals in the demodulated image when applying high spatial frequency of structured pattern. Accordingly, the HT is replaced with Hilbert-Huang transform (HHT) in order to sift out the low-frequency background noise and pattern residuals in the demodulation process. The experimental results obtained using a kidney tissue sample show that the HHT yields a significant improvement in the TFMPEM image quality.

Label-Free Autofluorescence-Detected Mid-Infrared Photothermal Microscopy of Pharmaceutical Materials

Label-free autofluorescence-detected photothermal mid-IR (AF-PTIR) microscopy is demonstrated experimentally and applied to test the distribution of active pharmaceutical ingredients (APIs) in a mixture containing representative pharmaceutical excipients. Two-photon excited UV-fluorescence (TPE-UVF) supports autofluorescence of native aromatic moieties using visible-light optics. Thermal modulation of the fluorescence quantum yield serves to report on infrared absorption, enabling infrared spectroscopy in the fingerprint region with a spatial resolution dictated by fluorescence. AF-PTIR provides high selectivity and sensitivity in image contrast for aromatic APIs, complementing broadly applicable optical photothermal IR (O-PTIR) microscopy based on photothermal modulation of refractive index/scattering. Mapping the API distribution is critical in designing processes for powdered dosage form manufacturing, with high spatial variance potentially producing variability in both delivered dosage and product efficacy. The ubiquity of aromatic moieties within API candidates suggests the viability of AF-PTIR in combination with O-PTIR to improve the confidence of chemical classification in spatially heterogeneous dosage forms.

Adaptive optics with spatio-temporal lock-in detection for temporal focusing microscopy

Wavefront distortion in temporal focusing microscopy (TFM) results in a distorted temporal profile of the excitation pulses owing to spatio-temporal coupling. Since the pulse duration is dramatically changed in the excitation volume, it is difficult to correct the temporal profile for a thick sample. Here, we demonstrate adaptive optics (AO) correction in a thick sample. We apply structured illumination microscopy (SIM) to an AO correction in wide-field TFM to decrease the change in the pulse duration in the signal detection volume. The AO correction with SIM was very successful in a thick sample for which AO correction with TFM failed.

Influence of the lateral size of a hologram on the reconstructed image in digital light-in-flight recording by holography

Digital light-in-flight recording by holography is a promising technique for observing a propagating ultrashort light pulse as a motion picture. A typical reconstruction process of digital light-in-flight recording by holography, we extract holograms without considering the relationship between the lateral size of the extracted hologram (sub-hologram) and the size of an area where the propagating ultrashort light pulse and an image sensor overlap. The area records the image of the ultrashort light pulse at a certain moment. In this study, by considering the size of the small interference fringe image, we assessed the influence of the lateral size of the hologram on the reconstructed image. We defined the size of the area in which the interference fringe image at a moment is recorded. Then, we examined the reconstructed images by changing the lateral size of the sub-hologram. As a result, we found that the lateral size of the hologram does not affect the size of the reconstructed image but the spatial resolution of the reconstructed images.

Temporal focusing multiphoton microscopy with optimized parallel multiline scanning for fast biotissue imaging

SIGNIFICANCE

Line scanning-based temporal focusing multiphoton microscopy (TFMPM) has superior axial excitation confinement (AEC) compared to conventional widefield TFMPM, but the frame rate is limited due to the limitation of the single line-to-line scanning mechanism. The development of the multiline scanning-based TFMPM requires only eight multiline patterns for full-field uniform multiphoton excitation and it still maintains superior AEC.

AIM

The optimized parallel multiline scanning TFMPM is developed, and the performance is verified with theoretical simulation. The system provides a sharp AEC equivalent to the line scanning-based TFMPM, but fewer scans are required.

APPROACH

A digital micromirror device is integrated in the TFMPM system and generates the multiline pattern for excitation. Based on the result of single-line pattern with sharp AEC, we can further model the multiline pattern to find the best structure that has the highest duty cycle together with the best AEC performance.

RESULTS

The AEC is experimentally improved to 1.7  μm from the 3.5  μm of conventional TFMPM. The adopted multiline pattern is akin to a pulse-width-modulation pattern with a spatial period of four times the diffraction-limited line width. In other words, ideally only four π  /  2 spatial phase-shift scans are required to form a full two-dimensional image with superior AEC instead of image-size-dependent line-to-line scanning.

CONCLUSIONS

We have demonstrated the developed parallel multiline scanning-based TFMPM has the multiline pattern for sharp AEC and the least scans required for full-field uniform excitation. In the experimental results, the temporal focusing-based multiphoton images of disordered biotissue of mouse skin with improved axial resolution due to the near-theoretical limit AEC are shown to clearly reduce background scattering.

Three-Dimensional Single-Molecule Localization Microscopy in Whole-Cell and Tissue Specimens

Super-resolution microscopy techniques are versatile and powerful tools for visualizing organelle structures, interactions, and protein functions in biomedical research. However, whole-cell and tissue specimens challenge the achievable resolution and depth of nanoscopy methods. We focus on three-dimensional single-molecule localization microscopy and review some of the major roadblocks and developing solutions to resolving thick volumes of cells and tissues at the nanoscale in three dimensions. These challenges include background fluorescence, system- and sample-induced aberrations, and information carried by photons, as well as drift correction, volume reconstruction, and photobleaching mitigation. We also highlight examples of innovations that have demonstrated significant breakthroughs in addressing the abovementioned challenges together with their core concepts as well as their trade-offs.

Structured illumination imaging without grating rotation based on mirror operation on 1D Fourier spectrum

Structured illumination microscopy (SIM) is a rapidly developing a super-resolution optical microscopy technique. With SIM, the grating is needed in order to rotate several angles for illuminating the sample in different directions. Multiple rotations reduce the imaging speed and grating rotation angle errors damage the image recovery quality. We introduce mirror transformation on one-dimension (1D) Fourier spectrum to SIM for resolving the problems of low imaging speed and severe impact on image reconstruction quality by grating rotation angle errors. When mirror operation and SIM are combined, the grating is placed at an orientation for obtaining three shadow images. The three shadow images are acquired by CCD at three different phase shift for a direction of grating. Thus, the SIM imaging speed is faster and the effect on image reconstruction quality by grating rotation angle errors is greatly reduced.

Interferometric temporal focusing microscopy using three-photon excitation fluorescence

Super-resolution microscopy has become a powerful tool for biological research. However, its spatial resolution and imaging depth are limited, largely due to background light. Interferometric temporal focusing (ITF) microscopy, which combines structured illumination microscopy and three-photon excitation fluorescence microscopy, can overcome these limitations. Here, we demonstrate ITF microscopy using three-photon excitation fluorescence, which has a spatial resolution of 106 nm at an imaging depth of 100 µm with an excitation wavelength of 1060 nm.

High-resolution imaging in two-photon excitation microscopy using in situ estimations of the point spread function

We present a technique for improving the spatial resolution of two-photon excitation microscopy; our technique combines annular illumination with an in situ estimation of the point spread function (PSF) used for deconvolution. For the in situ estimation of the PSF, we developed a technique called autocorrelation scanning, in which a sample is imaged by the scanning of two excitation foci that are overlapped over various distances. The image series obtained with the variation of the distance between the two foci provides the autocorrelation function of the PSF, which can be used to estimate the PSF at specific positions within a sample. We proved the principle and the effectiveness of this technique through observations of a fluorescent biological sample, and we confirmed that the improvement in the spatial resolution was ~1.7 times that of typical two-photon excitation microscopy by observing a mouse brain phantom at a depth of 200 µm.

Temporal focusing-based widefield multiphoton microscopy with spatially modulated illumination for biotissue imaging

A developed temporal focusing-based multiphoton excitation microscope (TFMPEM) has a digital micromirror device (DMD) which is adopted not only as a blazed grating for light spatial dispersion but also for patterned illumination simultaneously. Herein, the TFMPEM has been extended to implement spatially modulated illumination at structured frequency and orientation to increase the beam coverage at the back-focal aperture of the objective lens. The axial excitation confinement (AEC) of TFMPEM can be condensed from 3.0 μm to 1.5 μm for a 50 % improvement. By using the TFMPEM with HiLo technique as two structured illuminations at the same spatial frequency but different orientation, reconstructed biotissue images according to the condensed AEC structured illumination are shown obviously superior in contrast and better scattering suppression. Picture: TPEF images of the eosin-stained mouse cerebellar cortex by conventional TFMPEM (left), and the TFMPEM with HiLo technique as 1.09 μm^-1 spatially modulated illumination at 90° (center) and 0° (right) orientations.

Fast two-snapshot structured illumination for temporal focusing microscopy with enhanced axial resolution

We present a new two-snapshot structured light illumination (SLI) reconstruction algorithm for fast image acquisition. The new algorithm, which only requires two mutually π phase-shifted raw structured images, is implemented on a custom-built temporal focusing fluorescence microscope (TFFM) to enhance its axial resolution via a digital micromirror device (DMD). First, the orientation of the modulated sinusoidal fringe patterns is automatically identified via spatial frequency vector detection. Subsequently, the modulated in-focal-plane images are obtained via rotation and subtraction. Lastly, a parallel amplitude demodulation method, derived based on Hilbert transform, is applied to complete the decoding processes. To demonstrate the new SLI algorithm, a TFFM is custom-constructed, where a DMD replaces the generic blazed grating in the system and simultaneously functions as a diffraction grating and a programmable binary mask, generating arbitrary fringe patterns. The experimental results show promising depth-discrimination capability with an axial resolution enhancement factor of 1.25, which matches well with the theoretical estimation, i.e, 1.27. Imaging experiments on pollen grain and mouse kidney samples have been performed. The results indicate that the two-snapshot algorithm presents comparable contrast reconstruction and optical cross-sectioning capability than those adopting the conventional root-mean-square (RMS) reconstruction method. The two-snapshot method can be readily applied to any sinusoidally modulated illumination systems to realize high-speed 3D imaging as less frames are required for each in-focal-plane image restoration, i.e., the image acquisition speed is improved by 2.5 times for any two-photon systems.

Temporal focusing microscopy using three-photon excitation fluorescence with a 92-fs Yb-fiber chirped pulse amplifier

Temporal focusing (TF) microscopy is a wide-field two-photon excitation fluorescence (2PEF) microscopy technique, the optical sectioning capability of which is lower than that of point-scanning 2PEF microscopy. Here we demonstrate TF microscopy using three-photon excitation fluorescence (3PEF), which enhances the optical sectioning capability. As an excitation light source for the 3PEF, we developed an Yb-fiber chirped pulse amplifier, which produces 92-fs 9.0-μJ 1060-nm pulses at a repetition rate of 200 kHz. The optical sectioning capability was improved by a factor of 1.3 compared with that of 2PEF-TF microscopy. We also demonstrate dual-color imaging with both 2PEF and 3PEF.

Two-color temporal focusing multiphoton excitation imaging with tunable-wavelength excitation

Wavelength tunable temporal focusing multiphoton excitation microscopy (TFMPEM) is conducted to visualize optical sectioning images of multiple fluorophore–labeled specimens through the optimal two-photon excitation (TPE) of each type of fluorophore. The tunable range of excitation wavelength was determined by the groove density of the grating, the diffraction angle, the focal length of lenses, and the shifting distance of the first lens in the beam expander. Based on a consideration of the trade-off between the tunable-wavelength range and axial resolution of temporal focusing multiphoton excitation imaging, the presented system demonstrated a tunable-wavelength range from 770 to 920 nm using a diffraction grating with groove density of 830 ?? lines / mm . TPE fluorescence imaging examination of a fluorescent thin film indicated that the width of the axial confined excitation was 3.0 ± 0.7 ?? ? m and the shifting distance of the temporal focal plane was less than 0.95 ?? ? m within the presented wavelength tunable range. Fast different wavelength excitation and three-dimensionally rendered imaging of Hela cell mitochondria and cytoskeletons and mouse muscle fibers were demonstrated. Significantly, the proposed system can improve the quality of two-color TFMPEM images through different excitation wavelengths to obtain higher-quality fluorescent signals in multiple-fluorophore measurements.

Three-dimensional spatiotemporal focusing of holographic patterns

Two-photon excitation with temporally focused pulses can be combined with phase-modulation approaches, such as computer-generated holography and generalized phase contrast, to efficiently distribute light into two-dimensional, axially confined, user-defined shapes. Adding lens-phase modulations to 2D-phase holograms enables remote axial pattern displacement as well as simultaneous pattern generation in multiple distinct planes. However, the axial confinement linearly degrades with lateral shape area in previous reports where axially shifted holographic shapes were not temporally focused. Here we report an optical system using two spatial light modulators to independently control transverse- and axial-target light distribution. This approach enables simultaneous axial translation of single or multiple spatiotemporally focused patterns across the sample volume while achieving the axial confinement of temporal focusing. We use the system's capability to photoconvert tens of Kaede-expressing neurons with single-cell resolution in live zebrafish larvae.

Three-dimensional computer-generated holography cannot be implemented with temporal focusing. Here, Hernandez et al. use two spatial light modulators to control transverse- and axial-target light distribution, generating spatiotemporally focused patterns with uniform light distribution throughout the entire volume.

Fast volumetric imaging with patterned illumination via digital micro-mirror device-based temporal focusing multiphoton microscopy

Temporal focusing multiphoton microscopy (TFMPM) has the advantage of area excitation in an axial confinement of only a few microns; hence, it can offer fast three-dimensional (3D) multiphoton imaging. Herein, fast volumetric imaging via a developed digital micromirror device (DMD)-based TFMPM has been realized through the synchronization of an electron multiplying charge-coupled device (EMCCD) with a dynamic piezoelectric stage for axial scanning. The volumetric imaging rate can achieve 30 volumes per second according to the EMCCD frame rate of more than 400 frames per second, which allows for the 3D Brownian motion of one-micron fluorescent beads to be spatially observed. Furthermore, it is demonstrated that the dynamic HiLo structural multiphoton microscope can reject background noise by way of the fast volumetric imaging with high-speed DMD patterned illumination.

Two-dimensional spatiotemporal focusing of femtosecond pulses and its applications in microscopy

We demonstrate and theoretically analyze the two-dimensional spatiotemporal focusing of femtosecond pulses by utilizing a two-dimensional spectral disperser. Compared with spatiotemporal focusing with a diffraction grating, it can achieve widefield illumination with better sectioning ability for a multiphoton excitation process. By utilizing paraxial approximation, our analytical method improves the axial confinement ability and identifies that the free spectra range (FSR) of the two-dimensional spectral disperser affects the out-of-focus multiphoton excitation intensity due to the temporal self-imaging effect. Based on our numerical simulation, a FSR of 50 GHz is necessary to reduce the out-of-focus two-photon excitation by 2 orders of magnitude compared with that in a grating-based spatiotemporal focusing scheme for a 90-fs excitation laser pulse. We build a two-dimensional spatiotemporal focusing microscope using a virtually imaged phased array and achieve an axial resolution of 1.3 μm, which outperforms the resolution of conventional spatiotemporal focusing using a grating by a factor of 1.7, and demonstrate better image contrast inside a tissue-like phantom.

Super-resolution two-photon microscopy via scanning patterned illumination

We developed two-photon scanning patterned illumination microscopy (2P-SPIM) for super-resolution two-photon imaging. Our approach used a traditional two-photon microscopy setup with temporally modulated excitation to create patterned illumination fields. Combing nine different illuminations and structured illumination reconstruction, super-resolution imaging was achieved in two-photon microscopy. Using 2P-SPIM we achieved a lateral resolution of 141 nm, which represents an improvement by a factor of 1.9 over the corresponding diffraction limit. We further demonstrated super-resolution cellular imaging by 2P-SPIM to image actin cytoskeleton in mammalian cells and three-dimensional imaging in highly scattering retinal tissue.

Resolution enhancement of two-photon microscopy via intensity-modulated laser scanning structured illumination

Conventional structured illumination microscopy (SIM) with wide-field illumination is an applicable tool to provide resolution enhancement. And yet its applications in thick specimens are still full of challenges. By combing the structured illumination concept with two-photon excitation, a laser scanning two-photon structured illumination microscope (LSTP-SIM) was constructed to gain ∼1.42-fold lateral resolution enhancement in contrast to two-photon fluorescence microscopy. With a point-scanning geometry, an acoustic-optical modulator was used to modulate temporally the excitation intensity in order to produce the structured illumination pattern. The theoretical models of image formation and image reconstruction were clearly established. Simulation and experiments were both performed to show the capability of this system to enhance the lateral resolution. Combined with the inherent optical sectioning power of the two-photon excitation, LSTP-SIM would have the potential for applications in optically-thick specimens.

Nonlinear structured-illumination enhanced temporal focusing multiphoton excitation microscopy with a digital micromirror device

In this study, the light diffraction of temporal focusing multiphoton excitation microscopy (TFMPEM) and the excitation patterning of nonlinear structured-illumination microscopy (NSIM) can be simultaneously and accurately implemented via a single high-resolution digital micromirror device. The lateral and axial spatial resolutions of the TFMPEM are remarkably improved through the second-order NSIM and projected structured light, respectively. The experimental results demonstrate that the lateral and axial resolutions are enhanced from 397 nm to 168 nm (2.4-fold) and from 2.33 μm to 1.22 μm (1.9-fold), respectively, in full width at the half maximum. Furthermore, a three-dimensionally rendered image of a cytoskeleton cell featuring ~25 nm microtubules is improved, with other microtubules at a distance near the lateral resolution of 168 nm also able to be distinguished.

Accumulating advantages, reducing limitations: multimodal nonlinear imaging in biomedical sciences - the synergy of multiple contrast mechanisms

Multimodal nonlinear microscopy has matured during the past decades to one of the key imaging modalities in life science and biomedicine due to its unique capabilities of label-free visualization of tissue structure and chemical composition, high depth penetration, intrinsic 3D sectioning, diffraction limited resolution and low phototoxicity. This review briefly summarizes first recent advances in the field regarding the methodology, e.g., contrast mechanisms and signal characteristics used for contrast generation as well as novel image processing approaches. The second part deals with technologic developments emphasizing improvements in penetration depth, imaging speed, spatial resolution and nonlinear labeling strategies. The third part focuses on recent applications in life science fundamental research and biomedical diagnostics as well as future clinical applications.