Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle

HiLo microscopy with caustic illumination

HiLo microscopy is an optical sectioning structured illumination microscopy technique based on computationally combining two images: one with uniform illumination and the other with structured illumination. The most widely used structured illumination in HiLo microscopy is random speckle patterns, due to their simplicity and resilience to tissue scattering. Here, we present a novel HiLo microscopy strategy based on random caustic patterns. Building on an off-the-shelf diffuser and a low-coherence LED source, we demonstrate that caustic HiLo can achieve 4.5 µm optical sectioning capability with a 20× 0.75 NA objective. In addition, with the distinct intensity statistical properties of caustic patterns, we show that our caustic HiLo outperforms speckle HiLo, achieving enhanced optical sectioning capability and preservation of fine features by imaging scattering fixed brain sections of 100 µm, 300 µm, and 500 µm thicknesses. We anticipate that this new structured illumination technique may find various biomedical imaging applications.

Smartphone-based optical sectioning (SOS) microscopy with a telecentric design for fluorescence imaging

We propose a smartphone-based optical sectioning (SOS) microscope based on the HiLo technique, with a single smartphone replacing a high-cost illumination source and a camera sensor. We built our SOS with off-the-shelf optical, mechanical cage systems with 3D-printed adapters to seamlessly integrate the smartphone with the SOS main body. The liquid light guide can be integrated with the adapter, guiding the smartphone's LED light to the digital mirror device (DMD) with neglectable loss. We used an electrically tuneable lens (ETL) instead of a mechanical translation stage to realise low-cost axial scanning. The ETL was conjugated to the objective lens's back pupil plane (BPP) to construct a telecentric design by a 4f configuration to maintain the lateral magnification for different axial positions. SOS has a 571.5 µm telecentric scanning range and an 11.7 µm axial resolution. The broadband smartphone LED torch can effectively excite fluorescent polystyrene (PS) beads. We successfully used SOS for high-contrast fluorescent PS beads imaging with different wavelengths and optical sectioning imaging of multilayer fluorescent PS beads. To our knowledge, the proposed SOS is the first smartphone-based HiLo optical sectioning microscopy (£1965), which can save around £7035 compared with a traditional HiLo system (£9000). It is a powerful tool for biomedical research in resource-limited areas.

In Vivo Intelligent Fluorescence Endo-Microscopy by Varifocal Meta-Device and Deep Learning

Endo-microscopy is crucial for real-time 3D visualization of internal tissues and subcellular structures. Conventional methods rely on axial movement of optical components for precise focus adjustment, limiting miniaturization and complicating procedures. Meta-device, composed of artificial nanostructures, is an emerging optical flat device that can freely manipulate the phase and amplitude of light. Here, an intelligent fluorescence endo-microscope is developed based on varifocal meta-lens and deep learning (DL). The breakthrough enables in vivo 3D imaging of mouse brains, where varifocal meta-lens focal length adjusts through relative rotation angle. The system offers key advantages such as invariant magnification, a large field-of-view, and optical sectioning at a maximum focal length tuning range of ≈2 mm with 3 µm lateral resolution. Using a DL network, image acquisition time and system complexity are significantly reduced, and in vivo high-resolution brain images of detailed vessels and surrounding perivascular space are clearly observed within 0.1 s (≈50 times faster). The approach will benefit various surgical procedures, such as gastrointestinal biopsies, neural imaging, brain surgery, etc.

Bio-friendly long-term subcellular dynamic recording by self-supervised image enhancement microscopy

Fluorescence microscopy has become an indispensable tool for revealing the dynamic regulation of cells and organelles. However, stochastic noise inherently restricts optical interrogation quality and exacerbates observation fidelity when balancing the joint demands of high frame rate, long-term recording and low phototoxicity. Here we propose DeepSeMi, a self-supervised-learning-based denoising framework capable of increasing signal-to-noise ratio by over 12 dB across various conditions. With the introduction of newly designed eccentric blind-spot convolution filters, DeepSeMi effectively denoises images with no loss of spatiotemporal resolution. In combination with confocal microscopy, DeepSeMi allows for recording organelle interactions in four colors at high frame rates across tens of thousands of frames, monitoring migrasomes and retractosomes over a half day, and imaging ultra-phototoxicity-sensitive Dictyostelium cells over thousands of frames. Through comprehensive validations across various samples and instruments, we prove DeepSeMi to be a versatile and biocompatible tool for breaking the shot-noise limit.

Multiplane HiLo microscopy with speckle illumination and non-local means denoising

Significance

HiLo microscopy synthesizes an optically sectioned image from two images, one obtained with uniform and another with patterned illumination, such as laser speckle. Speckle-based HiLo has the advantage of being robust to aberrations but is susceptible to residual speckle noise that is difficult to control. We present a computational method to reduce this residual noise without undermining resolution. In addition, we improve the versatility of HiLo microscopy by enabling simultaneous multiplane imaging (here nine planes).

Aim

Our goal is to perform fast, high-contrast, multiplane imaging with a conventional camera-based fluorescence microscope.

Approach

Multiplane HiLo imaging is achieved with the use of a single camera and z-splitter prism. Speckle noise reduction is based on the application of a non-local means (NLM) denoising method to perform ensemble averaging of speckle grains.

Results

We demonstrate the capabilities of multiplane HiLo with NLM denoising both with synthesized data and by imaging cardiac and brain activity in zebrafish larvae at 40 Hz frame rates.

Conclusions

Multiplane HiLo microscopy aided by NLM denoising provides a simple tool for fast optically sectioned volumetric imaging that can be of general utility for fluorescence imaging applications.

Image scanning lensless fiber-bundle endomicroscopy

Fiber-based confocal endomicroscopy has shown great promise for minimally-invasive deep-tissue imaging. Despite its advantages, confocal fiber-bundle endoscopy inherently suffers from undersampling due to the spacing between fiber cores, and low collection efficiency when the target is not in proximity to the distal fiber facet. Here, we demonstrate an adaptation of image-scanning microscopy (ISM) to lensless fiber bundle endoscopy, doubling the spatial sampling frequency and significantly improving collection efficiency. Our approach only requires replacing the confocal detector with a camera. It improves the spatial resolution for targets placed at a distance from the fiber tip, and addresses the fundamental challenge of aliasing/pixelization artifacts.

Cascaded domain multiphoton spatial frequency modulation imaging

Significance

Multiphoton microscopy is a powerful imaging tool for biomedical applications. A variety of techniques and respective benefits exist for multiphoton microscopy, but an enhanced resolution is especially desired. Additionally multiphoton microscopy requires ultrafast pulses for excitation, so optimization of the pulse duration at the sample is critical for strong signals.

Aim

We aim to perform enhanced resolution imaging that is robust to scattering using a structured illumination technique while also providing a rapid and easily repeatable means to optimize group delay dispersion (GDD) compensation through to the sample.

Approach

Spatial frequency modulation imaging (SPIFI) is used in two domains: the spatial domain (SD) and the wavelength domain (WD). The WD-SPIFI system is an in-line tool enabling GDD optimization that considers all material through to the sample. The SD-SPIFI system follows and enables enhanced resolution imaging.

Results

The WD-SPIFI dispersion optimization performance is confirmed with independent pulse characterization, enabling rapid optimization of pulses for imaging with the SD-SPIFI system. The SD-SPIFI system demonstrates enhanced resolution imaging without the use of photon counting enabled by signal to noise improvements due to the WD-SPIFI system.

Conclusions

Implementing SPIFI in-line in two domains enables full-path dispersion compensation optimization through to the sample for enhanced resolution multiphoton microscopy.

Background inhibited and speed-loss-free volumetric imaging in vivo based on structured-illumination Fourier light field microscopy

Benefiting from its advantages in fast volumetric imaging for recording biodynamics, Fourier light field microscopy (FLFM) has a wide range of applications in biomedical research, especially in neuroscience. However, the imaging quality of the FLFM is always deteriorated by both the out-of-focus background and the strong scattering in biological samples. Here we propose a structured-illumination and interleaved-reconstruction based Fourier light field microscopy (SI-FLFM), in which we can filter out the background fluorescence in FLFM without sacrificing imaging speed. We demonstrate the superiority of our SI-FLFM in high-speed, background-inhibited volumetric imaging of various biodynamics in larval zebrafish and mice in vivo. The signal-to-background ratio (SBR) is improved by tens of times. And the volumetric imaging speed can be up to 40 Hz, avoiding artifacts caused by temporal under-sampling in conventional structured illumination microscopy. These suggest that our SI-FLFM is suitable for applications of weak fluorescence signals but high imaging speed requirements.

Rapid slide-free and non-destructive histological imaging using wide-field optical-sectioning microscopy

Histopathology based on formalin-fixed and paraffin-embedded tissues has long been the gold standard for surgical margin assessment (SMA). However, routine pathological practice is lengthy and laborious, failing to guide surgeons intraoperatively. In this report, we propose a practical and low-cost histological imaging method with wide-field optical-sectioning microscopy (i.e., High-and-Low-frequency (HiLo) microscopy). HiLo can achieve rapid and non-destructive imaging of freshly-excised tissues at an extremely high acquisition speed of 5 cm²/min with a spatial resolution of 1.3 µm (lateral) and 5.8 µm (axial), showing great potential as an SMA tool that can provide immediate feedback to surgeons and pathologists for intraoperative decision-making. We demonstrate that HiLo enables rapid extraction of diagnostic features for different subtypes of human lung adenocarcinoma and hepatocellular carcinoma, producing surface images of rough specimens with large field-of-views and cellular features that are comparable to the clinical standard. Our results show promising clinical translations of HiLo microscopy to improve the current standard of care.

Comparison of Two- and Three-Beam Interference Pattern Generation in Structured Illumination Microscopy

Structured illumination microscopy (SIM) provides wide-field optical sectioning in the focal plane by modulating the imaging information using fringe pattern illumination. For generating the fringe pattern illumination, a digital micro-mirror device (DMD) is commonly used due to its flexibility and fast refresh rate. However, the benefit of different pattern generation, for example, the two-beam interference mode and the three-beam interference mode, has not been clearly investigated. In this study, we systematically analyze the optical sectioning provided by the two-beam inference mode and the three-beam interference mode of DMD. The theoretical analysis and imaging results show that the two-beam interference mode is suitable for fast imaging of the superficial dynamic target due to reduced number of phase shifts needed to form the image, and the three-beam interference mode is ideal for imaging three-dimensional volume due to its superior optical sectioning by the improved modulation of the illumination patterns. These results, we believe, will provide better guidance for the use of DMD for SIM imaging and also for the choice of beam patterns in SIM application in the future.

HiLo Based Line Scanning Temporal Focusing Microscopy for High-Speed, Deep Tissue Imaging

High-speed, optical-sectioning imaging is highly desired in biomedical studies, as most bio-structures and bio-dynamics are in three-dimensions. Compared to point-scanning techniques, line scanning temporal focusing microscopy (LSTFM) is a promising method that can achieve high temporal resolution while maintaining a deep penetration depth. However, the contrast and axial confinement would still be deteriorated in scattering tissue imaging. Here, we propose a HiLo-based LSTFM, utilizing structured illumination to inhibit the fluorescence background and, thus, enhance the image contrast and axial confinement in deep imaging. We demonstrate the superiority of our method by performing volumetric imaging of neurons and dynamical imaging of microglia in mouse brains in vivo.

Single-scan HiLo with line-illumination strategy for optical section imaging of thick tissues

Optical sectioning has been widely employed for inhibiting out-of-focus backgrounds in three-dimensional (3D) imaging of biological samples. However, point scanning imaging or multiple acquisitions for wide-field optical sectioning in epi-illumination microscopy remains time-consuming for large-scale imaging. In this paper, we propose a single-scan optical sectioning method based on the hybrid illumination (HiLo) algorithm with a line-illumination strategy. Our method combines HiLo background inhibition with confocal slit detection. It thereby offers a higher optical sectioning capability than wide-field HiLo and line-confocal imaging without extra modulation and multiple data acquisition. To demonstrate the optical-sectioning capability of our system, we imaged a thin fluorescent plane and different fluorescence-labeled mouse tissue. Our method shows an excellent background inhibition in thick tissue and thus potentially provides an alternative tool for 3D imaging of large-scale biological tissue.

Deep learning based one-shot optically-sectioned structured illumination microscopy for surface measurement

Optically-sectioned structured illumination microscopy (OS-SIM) is broadly used for biological imaging and engineering surface measurement owing to its simple, low-cost, scanning-free experimental setup and excellent optical sectioning capability. However, the efficiency of current optically-sectioned methods in OS-SIM is yet limited for surface measurement because a set of wide-field images under uniform or structured illumination are needed to derive an optical section at each scanning height. In this paper, a deep-learning-based one-shot optically-sectioned method, called Deep-OS-SIM, is proposed to improve the efficiency of OS-SIM for surface measurement. Specifically, we develop a convolutional neural network (CNN) to learn the statistical invariance of optical sectioning across structured illumination images. By taking full advantage of the high entropy properties of structured illumination images to train the CNN, fast convergence and low training error are achieved in our method even for low-textured surfaces. The well-trained CNN is then applied to a plane mirror for testing, demonstrating the ability of the method to reconstruct high-quality optical sectioning from only one instead of two or three raw structured illumination frames. Further measurement experiments on a standard step and milled surface show that the proposed method has similar accuracy to OS-SIM techniques but with higher imaging speed.

Telecentric design for digital-scanning-based HiLo optical sectioning endomicroscopy with an electrically tunable lens

Confocal endoscopy has been widely used to obtain fine optically sectioned images. However, confocal endomicroscopic images are formed by point-by-point scanning in both lateral and axial directions, which results in long image acquisition time. Here, an endomicroscope with telecentric configuration is presented to achieve nonmechanical and rapid axial scanning for volumetric fluorescence imaging. In our system, optical sectioning in wide-field fashion is obtained through HiLo imaging with a digital micromirror device. Axial scanning, without mechanical moving parts, is conducted by digital focus adjustment using an electrically tunable lens, offering constant magnification and contrast. We demonstrate imaging performance of our system with optically sectioned images using fluorescently labeled beads, as well as ex vivo mice cardiac tissue samples. Our system provides multiple advantages, in terms of improved scanning range, and reduced image acquisition time, which shows great potentials for three-dimensional biopsies of volumetric biological samples.

Simultaneous multi-color optical sectioning fluorescence microscopy with wavelength-coded volume holographic gratings

Optical sectioning fluorescence microscopy provides high contrast images of volumetric samples and has been widely used for many biological applications. However, simultaneously acquiring multi-color fluorescence images require additional optical elements and devices, which are bulky, wavelength specific, and not cost-effective. In this paper, wavelength-coded volume holographic gratings (WC-VHGs) based optical sectioning fluorescence microscopy is proposed to simultaneously offer multi-color fluorescence images with fine out-of-focus background rejection. Due to wavelength degeneracy, multiplexed WC-VHGs are capable of acquiring multi-wavelength fluorescence images in a single shot, and displaying the laterally separated multi-wavelength images onto CCD. In our system optical sectioning capability is achieved through speckle illumination and HiLo imaging method. To demonstrate imaging characteristics of our system, dual-wavelength fluorescence images of both standard fluorescent microspheres and ex vivo mT/mG mice cardiac tissue are presented. Current results may find important applications in hyperspectral imaging for biomedical research.

Light-Sheet Microscopy for Surface Topography Measurements and Quantitative Analysis

A novel light-sheet microscopy (LSM) system that uses the laser triangulation method to quantitatively calculate and analyze the surface topography of opaque samples is discussed. A spatial resolution of at least 10 μm in z-direction, 10 μm in x-direction and 25 μm in y-direction with a large field-of-view (FOV) is achieved. A set of sample measurements that verify the system′s functionality in various applications are presented. The system has a simple mechanical structure, such that the spatial resolution is easily improved by replacement of the objective, and a linear calibration formula, which enables convenient system calibration. As implemented, the system has strong potential for, e.g., industrial sample line inspections, however, since the method utilizes reflected/scattered light, it also has the potential for three-dimensional analysis of translucent and layered structures.

Four-dimensional visualization of zebrafish cardiovascular and vessel dynamics by a structured illumination microscope with electrically tunable lens

We established a four-dimensional (4D) microscopy method using structured illumination for optical axial imaging with an electrically tunable lens. With its fast imaging capability, the dynamics of the cardiovascular system of the zebrafish and cerebral vessels were imaged based on the coverage of two stacks (25 layers) per second with lateral /axial resolutions of 0.6 µm and 1.8 µm, respectively. Time lapse imaging clearly shows the contractile-relaxation response of the beating heart at different cardiac phases and with different mobilities of blood vessels in different regions. This new 4D technique will facilitate in vivo imaging of organ function, generation, as well as drug responses in small-sized animals.

Multi-plane, wide-field fluorescent microscopy for biodynamic imaging in vivo

Wide-field fluorescent microscopy (WFFM) is widely employed in biomedical studies, due to its inherent advantages in high-speed imaging of biological dynamics noninvasively and specifically. However, WFFM suffers from the loss of axial resolution and the poor resistance to light scattering in deep tissue imaging. Here we propose a novel WFFM which has the capability in optical sectioning and volumetric imaging. We perform speckle illumination with a digital-micromirror-device for optical sectioning and employ an electrically tunable lens for defocusing modulation so as to quickly switch the image planes. We demonstrate its applications in multi-plane, wide-field imaging of biological dynamics in both zebrafish brains and mouse brains in vivo.

Line scanning, fiber bundle fluorescence HiLo endomicroscopy with confocal slit detection

Fiber bundle fluorescence endomicroscopy is an effective method for in vivo imaging of biological tissue samples. Line-scanning confocal laser endomicroscopy realizes confocal imaging at a much higher frame rate compared to the point scanning system, but with reduced optical sectioning. To address this problem, we describe a fiber bundle endomicroscopy system that utilizes the HiLo technique to enhance the optical sectioning while still maintaining high image acquisition rates. Confocal HiLo endomicroscopy is achieved by synchronizing the scanning hybrid-illumination laser line with the rolling shutter of a CMOS camera. An evident improvement of axial sectioning is achieved as compared to the line-scanning confocal endomicroscopy without the HiLo technique. Comparisons are also made with epifluorescence endomicroscopy with and without HiLo. The optical sectioning enhancement is demonstrated on lens tissue as well as porcine kidney tissue.

A 2-dimensional ratchet model describes assembly initiation of a specialized bacterial cell surface

Bacterial spores are dormant cells that are encased in a thick protein shell, the "coat," which participates in protecting the organism's DNA from environmental insults. The coat is composed of dozens of proteins that assemble in an orchestrated fashion during sporulation. In Bacillus subtilis, 2 proteins initiate coat assembly: SpoVM, which preferentially binds to micron-scale convex membranes and marks the surface of the developing spore as the site for coat assembly; and SpoIVA, a structural protein recruited by SpoVM that uses ATP hydrolysis to drive its irreversible polymerization around the developing spore. Here, we describe the initiation of coat assembly by SpoVM and SpoIVA. Using single-molecule fluorescence microscopy in vivo in sporulating cells and in vitro on synthetic spores, we report that SpoVM's localization is primarily driven by a lower off-rate on membranes of preferred curvature in the absence of other coat proteins. Recruitment and polymerization of SpoIVA results in the entrapment of SpoVM on the forespore surface. Using experimentally derived reaction parameters, we show that a 2-dimensional ratchet model can describe the interdependent localization dynamics of SpoVM and SpoIVA, wherein SpoVM displays a longer residence time on the forespore surface, which favors recruitment of SpoIVA to that location. Localized SpoIVA polymerization in turn prevents further sampling of other membranes by prelocalized SpoVM molecules. Our model therefore describes the dynamics of structural proteins as they localize and assemble at the correct place and time within a cell to form a supramolecular complex.

Accurate surface profilometry using differential optical sectioning microscopy with structured illumination

A differential optical sectioning microscopy with structured-illumination (DOSM-SI) with enhanced axial precision is explored in this paper for three-dimensional (3D) measurement. As the segment of data on the linear region of the contrast depth response curve (CDR) is very sensitive to variation of the height information, the DOSM-SI introduces a new CDR2 with an axial shift to intersect the linear region of the CDR1, which is achieved by using two charge-coupled detectors (CCDs) in the optical path. The CCD1 is located on the imaging plane and the CCD2 is displaced from the imaging plane. The difference between the CDR1 and CDR2 for each pixel is defined as the differential depth response curve (DCDR). Further, the zero-crossing point of the DCDR for each pixel is accurately extracted using the line-fitting technique, and finally, the sample surface can be reconstructed with a high resolution and precision. Since the slope around the zero-crossing point of the DCDR is apparently larger than that of near the focal position, an enhanced resolution and precision can be realized in DOSM-SI. The experiments and theoretical analysis are elaborated to demonstrate the feasibility of DOSM-SI.

Endogenous SHG and 2PEF coherence imaging of substructures in neurons in 3D

Neuronal morphology, long-distance transport and signalling critically depend on the organization of microtubules in the cytoskeleton. Second harmonic generation (SHG) imaging has been recognized as a potentially powerful tool for in situ label-free neuroimaging with specific sensitivity to microtubules. We study here the structural organization of microtubules in living neurons using a wide-field multiphoton microscope that performs 3D imaging using a structured illumination. This microscope allows label-free high throughput imaging of living mammalian neurons. We show that we can image structural correlations by taking advantage of the structured illumination and the coherence of the emitted light. The result allows us to study the microtubule organization throughout the development of the neuron and to differentiate between the regions of the cytoskeleton in the matured neuron.

Fast confocal fluorescence imaging in freely behaving mice

Fluorescence imaging in the brain of freely behaving mice is challenging due to severe miniaturization constraints. In particular, the ability to image a large field of view at high temporal resolution and with efficient out-of-focus background rejection still raises technical difficulties. Here, we present a novel fiberscope system that provides fast (up to 200 Hz) background-free fluorescence imaging in freely behaving mice over a field of view of diameter 230 μm. The fiberscope is composed of a custom-made multipoint-scanning confocal microscope coupled to the animal with an image guide and a micro-objective. By simultaneously registering a multipoint-scanning confocal image and a conventional widefield image, we subtracted the residual out-of-focus background and provided a background-free confocal image. Illumination and detection pinholes were created using a digital micromirror device, providing high adaptability to the sample structure and imaging conditions. Using this novel imaging tool, we demonstrated fast fluorescence imaging of microvasculature up to 120 μm deep in the mouse cortex, with an out-of-focus background reduced by two orders of magnitude compared with widefield microscopy. Taking advantage of the high acquisition rate (200 Hz), we measured red blood cell velocity in the cortical microvasculature and showed an increase in awake, unrestrained mice compared with anaesthetized animals.

Speckle illumination holographic non-scanning fluorescence endoscopy

Optical sectioning endoscopy such as confocal endoscopy offers capabilities to obtain three-dimensional (3D) information from various biological samples by discriminating between the desired in-focus signals and out-of-focus background. However, in general confocal images are formed through point-by-point scanning and the scanning time is proportional to the 3D space-bandwidth product. Recently, structured illumination endoscopy has been utilized for optically sectioned wide-field imaging, but it still needs axial scanning to acquire images from different depths of focal plane. Here, we report wide-field, multiplane, optical sectioning endoscopic imaging, incorporating 3D active speckle-based illumination and multiplexed volume holographic gratings, to simultaneously obtain images of fluorescently labeled tissue structures from different depths, without the need of scanning. We present the design, and implementation, as well as experimental data, demonstrating this endoscopic system's ability to obtain optically sectioned multiplane fluorescent images of tissue samples, with cellular level resolution in wide-field fashion, and no need for mechanical or optical axial scanning.(A) Schematic drawing of the SIHN endoscopy to simultaneously acquire multiplane images from different depths. (B) Uniform, and (C) SIHN illuminated images of standard fluorescence beads (25 μm in diameter) for the two axial planes. (D) Intensity profile on fluorescently labeled signal (ie, in-focus) and background (ie, out-of-focus) of microspheres.

A frequency domain SIM reconstruction algorithm using reduced number of images

Conventional two-dimensional structured illumination microscopy (SIM) requires 9 raw images to reconstruct a super-resolved image. In order to increase the frame rate of 2DSIM, attempts are being made to reduce the number of raw SIM images. However, all the proposed SIM reconstruction algorithms (SIM-RA) capable of reconstructing super-resolution (SR) image with a reduced number of raw SIM images operate in the spatial domain. Here, we present a frequency domain SIM-RA based on ordinary least squares technique, which enables reconstruction of SR image using 4 raw SIM images. Unlike the spatial domain RA, which produces the SR image through iterative convergence, the presented RA provides a single step solution. It also reveals the fundamental limitation of least number of raw images required for resolution doubling in SIM.

Doubling the pixel count limitation of single-pixel imaging via sinusoidal amplitude modulation

We demonstrate a single-pixel imaging (SPI) method that can achieve pixel resolution beyond the physical limitation of the spatial light modulator (SLM), by adopting sinusoidal amplitude modulation and frequency filtering. Through light field analysis, we observe that the induced intensity with a squared value of the amplitude contains higher frequency components. By filtering out the zero frequency of the sinusoidal amplitude in the Fourier domain, we can separate out the higher frequency components, which enables SPI with higher resolving ability and thus beyond the limitation of the SLM. Further, to address the speed issue in grayscale spatial light modulation, we propose a fast implementation scheme with tens-of-kilohertz refresh rate. Specifically, we use a digital micromirror device (DMD) working at the full frame rate to conduct binarized sinusoidal patterning in the spatial domain and pinhole filtering eliminating the binarization error in the Fourier domain. For experimental validation, we build a single-pixel microscope to retrieve 1200 × 1200-pixel images via a sub-megapixel DMD, and the setup achieves comparable performance to array sensor microscopy and provides additional sectioning ability.

Temporal focusing-based widefield multiphoton microscopy with spatially modulated illumination for biotissue imaging

A developed temporal focusing-based multiphoton excitation microscope (TFMPEM) has a digital micromirror device (DMD) which is adopted not only as a blazed grating for light spatial dispersion but also for patterned illumination simultaneously. Herein, the TFMPEM has been extended to implement spatially modulated illumination at structured frequency and orientation to increase the beam coverage at the back-focal aperture of the objective lens. The axial excitation confinement (AEC) of TFMPEM can be condensed from 3.0 μm to 1.5 μm for a 50 % improvement. By using the TFMPEM with HiLo technique as two structured illuminations at the same spatial frequency but different orientation, reconstructed biotissue images according to the condensed AEC structured illumination are shown obviously superior in contrast and better scattering suppression. Picture: TPEF images of the eosin-stained mouse cerebellar cortex by conventional TFMPEM (left), and the TFMPEM with HiLo technique as 1.09 μm^-1 spatially modulated illumination at 90° (center) and 0° (right) orientations.

Augmented line-scan focal modulation microscopy for multi-dimensional imaging of zebrafish heart in vivo

Multi-dimensional fluorescence imaging of live animal models demands strong optical sectioning, high spatial resolution, fast image acquisition, and minimal photobleaching. While conventional laser scanning microscopes are capable of deep penetration and sub-cellular resolution, they are generally too slow and causing excessive photobleaching for volumetric or time-lapse imaging. We demonstrate the performance of an augmented line-scan focal modulation microscope (aLSFMM), a high-speed imaging platform that affords above video-rate imaging speed by the use of line scanning. Exceptional background rejection is accomplished by combining a confocal slit with focal modulation. The image quality is further improved by merging the information from simultaneously acquired focal modulation and confocal images. Such a hybrid imaging scheme makes it possible to use very low power excitation light in high-speed imaging, and therefore leads to reduced photobleaching that is desirable for three-dimensional (3D) and four-dimensional (4D) in vivo image acquisition.

Optical imaging of surface chemistry and dynamics in confinement

We imaged the interfacial structure and dynamics of water in a microscopically confined geometry, in three dimensions and on millisecond time scales, with a structurally illuminated wide-field second harmonic microscope. The second harmonic images reported on the orientational order of interfacial water, induced by charge-dipole interactions between water molecules and surface charges. The images were converted into surface potential maps. Spatially resolved surface acid dissociation constant (pK_a,s) values were determined for the silica deprotonation reaction by following pH-induced chemical changes on the curved and confined surfaces of a glass microcapillary immersed in aqueous solutions. These values ranged from 2.3 to 10.7 along the wall of a single capillary because of surface heterogeneities. Water molecules that rotate along an oscillating external electric field were also imaged.

Tolerancing the alignment of large-core optical fibers, fiber bundles and light guides using a Fourier approach

Optical fiber technology is found in a wide variety of applications to flexibly relay light between two points, enabling information transfer across long distances and allowing access to hard-to-reach areas. Large-core optical fibers and light guides find frequent use in illumination and spectroscopic applications, for example, endoscopy and high-resolution astronomical spectroscopy. Proper alignment is critical for maximizing throughput in optical fiber coupling systems; however, there currently are no formal approaches to tolerancing the alignment of a light-guide coupling system. Here, we propose a Fourier alignment sensitivity (FAS) algorithm to determine the optimal tolerances on the alignment of a light guide by computing the alignment sensitivity. The algorithm shows excellent agreement with both simulated and experimentally measured values and improves on the computation time of equivalent ray-tracing simulations by two orders of magnitude. We then apply FAS to tolerance and fabricate a coupling system, which is shown to meet specifications, thus validating FAS as a tolerancing technique. These results indicate that FAS is a flexible and rapid means to quantify the alignment sensitivity of a light guide, widely informing the design and tolerancing of coupling systems.

In vivo volumetric fluorescence sectioning microscopy with mechanical-scan-free hybrid illumination imaging

Optical sectioning microscopy in wide-field fashion has been widely used to obtain three-dimensional images of biological samples; however, it requires scanning in depth and considerable time to acquire multiple depth information of a volumetric sample. In this paper, in vivo optical sectioning microscopy with volumetric hybrid illumination, with no mechanical moving parts, is presented. The proposed system is configured such that the optical sectioning is provided by hybrid illumination using a digital micro-mirror device (DMD) for uniform and non-uniform pattern projection, while the depth of imaging planes is varied by using an electrically tunable-focus lens with invariant magnification and resolution. We present and characterize the design, implementation, and experimentally demonstrate the proposed system's ability through 3D imaging of in vivo Canenorhabditis elegans' growth cones.

Dual-modality endomicroscopy with co-registered fluorescence and phase contrast

We describe a dual-modality laser scanning endomicroscope that provides simultaneous fluorescence contrast based on confocal laser endomicroscopy (CLE) and phase-gradient contrast based on scanning oblique back-scattering microscopy (sOBM). The probe consists of a 2.6mm-diameter micro-objective attached to a 30,000-core flexible fiber bundle. The dual contrasts are inherently co-registered, providing complementary information on labeled and un-labeled sample structure. Proof of principle demonstrations are presented with ex-vivo mouse colon tissue.

Sensorless adaptive optics implementation in widefield optical sectioning microscopy inside in vivo Drosophila brain

We present an implementation of a sensorless adaptive optics loop in a widefield fluorescence microscope. This setup is designed to compensate for aberrations induced by the sample on both excitation and emission pathways. It allows fast optical sectioning inside a living Drosophila brain. We present a detailed characterization of the system performances. We prove that the gain brought to optical sectioning by realizing structured illumination microscopy with adaptive optics down to 50 μm deep inside living Drosophila brain.

Fast Calcium Imaging with Optical Sectioning via HiLo Microscopy

Imaging intracellular calcium concentration via reporters that change their fluorescence properties upon binding of calcium, referred to as calcium imaging, has revolutionized our way to probe neuronal activity non-invasively. To reach neurons densely located deep in the tissue, optical sectioning at high rate of acquisition is necessary but difficult to achieve in a cost effective manner. Here we implement an accessible solution relying on HiLo microscopy to provide robust optical sectioning with a high frame rate in vivo. We show that large calcium signals can be recorded from dense neuronal populations at high acquisition rates. We quantify the optical sectioning capabilities and demonstrate the benefits of HiLo microscopy compared to wide-field microscopy for calcium imaging and 3D reconstruction. We apply HiLo microscopy to functional calcium imaging at 100 frames per second deep in biological tissues. This approach enables us to discriminate neuronal activity of motor neurons from different depths in the spinal cord of zebrafish embryos. We observe distinct time courses of calcium signals in somata and axons. We show that our method enables to remove large fluctuations of the background fluorescence. All together our setup can be implemented to provide efficient optical sectioning in vivo at low cost on a wide range of existing microscopes.

Full-color structured illumination optical sectioning microscopy

In merits of super-resolved resolution and fast speed of three-dimensional (3D) optical sectioning capability, structured illumination microscopy (SIM) has found variety of applications in biomedical imaging. So far, most SIM systems use monochrome CCD or CMOS cameras to acquire images and discard the natural color information of the specimens. Although multicolor integration scheme are employed, multiple excitation sources and detectors are required and the spectral information is limited to a few of wavelengths. Here, we report a new method for full-color SIM with a color digital camera. A data processing algorithm based on HSV (Hue, Saturation, and Value) color space is proposed, in which the recorded color raw images are processed in the Hue, Saturation, Value color channels, and then reconstructed to a 3D image with full color. We demonstrated some 3D optical sectioning results on samples such as mixed pollen grains, insects, micro-chips and the surface of coins. The presented technique is applicable to some circumstance where color information plays crucial roles, such as in materials science and surface morphology.

Contrast-based sensorless adaptive optics for retinal imaging

Conventional adaptive optics ophthalmoscopes use wavefront sensing methods to characterize ocular aberrations for real-time correction. However, there are important situations in which the wavefront sensing step is susceptible to difficulties that affect the accuracy of the correction. To circumvent these, wavefront sensorless adaptive optics (or non-wavefront sensing AO; NS-AO) imaging has recently been developed and has been applied to point-scanning based retinal imaging modalities. In this study we show, for the first time, contrast-based NS-AO ophthalmoscopy for full-frame in vivo imaging of human and animal eyes. We suggest a robust image quality metric that could be used for any imaging modality, and test its performance against other metrics using (physical) model eyes.

High speed, line-scanning, fiber bundle fluorescence confocal endomicroscopy for improved mosaicking

A significant limitation of fiber bundle endomicroscopy systems is that the field of view tends to be small, usually only several hundred micrometers in diameter. Image mosaicking techniques can increase the effective image size, but require careful manipulation of the probe to ensure sufficient overlap between adjacent frames. For confocal endomicroscopes, which typically have frame rates on the order of 10 fps, this is particularly challenging. In this paper we demonstrate that line-scanning confocal endomicroscopy can, by use of a high speed linear CCD camera, achieve a frame rate of 120 fps while maintaining sufficient resolution and signal-to-noise ratio to allow imaging of topically stained gastrointestinal tissues. This leads to improved performance of a cross-correlation based mosaicking algorithm when compared with lower frame-rate systems.

Double-exposure optical sectioning structured illumination microscopy based on Hilbert transform reconstruction

Structured illumination microscopy (SIM) with axially optical sectioning capability has found widespread applications in three-dimensional live cell imaging in recent years, since it combines high sensitivity, short image acquisition time, and high spatial resolution. To obtain one sectioned slice, three raw images with a fixed phase-shift, normally 2π/3, are generally required. In this paper, we report a data processing algorithm based on the one-dimensional Hilbert transform, which needs only two raw images with arbitrary phase-shift for each single slice. The proposed algorithm is different from the previous two-dimensional Hilbert spiral transform algorithm in theory. The presented algorithm has the advantages of simpler data processing procedure, faster computation speed and better reconstructed image quality. The validity of the scheme is verified by imaging biological samples in our developed DMD-based LED-illumination SIM system.

Talbot holographic illumination nonscanning (THIN) fluorescence microscopy

Optical sectioning techniques offer the ability to acquire three-dimensional information from various organ tissues by discriminating between the desired in-focus and out-of-focus (background) signals. Alternative techniques to confocal, such as active structured illumination, exist for fast optically sectioned images, but they require individual axial planes to be imaged consecutively. In this article, an imaging technique (THIN), by utilizing active Talbot illumination in 3D and multiplexed holographic Bragg filters for depth discrimination, is demonstrated for imaging in vivo 3D biopsy without mechanical or optical axial scanning.

Improving high resolution retinal image quality using speckle illumination HiLo imaging

Retinal image quality from flood illumination adaptive optics (AO) ophthalmoscopes is adversely affected by out-of-focus light scatter due to the lack of confocality. This effect is more pronounced in small eyes, such as that of rodents, because the requisite high optical power confers a large dioptric thickness to the retina. A recently-developed structured illumination microscopy (SIM) technique called HiLo imaging has been shown to reduce the effect of out-of-focus light scatter in flood illumination microscopes and produce pseudo-confocal images with significantly improved image quality. In this work, we adopted the HiLo technique to a flood AO ophthalmoscope and performed AO imaging in both (physical) model and live rat eyes. The improvement in image quality from HiLo imaging is shown both qualitatively and quantitatively by using spatial spectral analysis.

Optically-sectioned two-shot structured illumination microscopy with Hilbert-Huang processing

We introduce a fast, simple, adaptive and experimentally robust method for reconstructing background-rejected optically-sectioned images using two-shot structured illumination microscopy. Our innovative data demodulation method needs two grid-illumination images mutually phase shifted by π (half a grid period) but precise phase displacement between two frames is not required. Upon frames subtraction the input pattern with increased grid modulation is obtained. The first demodulation stage comprises two-dimensional data processing based on the empirical mode decomposition for the object spatial frequency selection (noise reduction and bias term removal). The second stage consists in calculating high contrast image using the two-dimensional spiral Hilbert transform. Our algorithm effectiveness is compared with the results calculated for the same input data using structured-illumination (SIM) and HiLo microscopy methods. The input data were collected for studying highly scattering tissue samples in reflectance mode. Results of our approach compare very favorably with SIM and HiLo techniques.

Needle-based fluorescence endomicroscopy via structured illumination with a plastic, achromatic objective

In order to diagnose cancer, a sample must be removed, prepared, and examined under a microscope, which is expensive, invasive, and time consuming. Fiber optic fluorescence endomicroscopy, where an image guide is used to obtain high-resolution images of tissue in vivo, has shown promise as an alternative to conventional biopsies. However, the resolution of standard endomicroscopy is limited by the fiber bundle sampling frequency and out-of-focus light. A system is presented which incorporates a plastic, achromatic objective to increase the sampling and which provides optical sectioning via structured illumination to reject background light. An image is relayed from the sample by a fiber bundle with the custom 2.1-mm outer diameter objective lens integrated to the distal tip. The objective is corrected for the excitation and the emission wavelengths of proflavine (452 and 515 nm). It magnifies the object onto the fiber bundle to improve the system's lateral resolution by increasing the sampling. The plastic lenses were fabricated via single-point diamond turning and assembled using a zero alignment technique. Ex vivo images of normal and neoplastic murine mammary tissues stained with proflavine are captured. The system achieves higher contrast and resolves smaller features than standard fluorescence endomicroscopy.

Optimization of a widefield structured illumination microscope for non-destructive assessment and quantification of nuclear features in tumor margins of a primary mouse model of sarcoma

Cancer is associated with specific cellular morphological changes, such as increased nuclear size and crowding from rapidly proliferating cells. In situ tissue imaging using fluorescent stains may be useful for intraoperative detection of residual cancer in surgical tumor margins. We developed a widefield fluorescence structured illumination microscope (SIM) system with a single-shot FOV of 2.1×1.6 mm (3.4 mm²) and sub-cellular resolution (4.4 µm). The objectives of this work were to measure the relationship between illumination pattern frequency and optical sectioning strength and signal-to-noise ratio in turbid (i.e. thick) samples for selection of the optimum frequency, and to determine feasibility for detecting residual cancer on tumor resection margins, using a genetically engineered primary mouse model of sarcoma. The SIM system was tested in tissue mimicking solid phantoms with various scattering levels to determine impact of both turbidity and illumination frequency on two SIM metrics, optical section thickness and modulation depth. To demonstrate preclinical feasibility, ex vivo 50 µm frozen sections and fresh intact thick tissue samples excised from a primary mouse model of sarcoma were stained with acridine orange, which stains cell nuclei, skeletal muscle, and collagenous stroma. The cell nuclei were segmented using a high-pass filter algorithm, which allowed quantification of nuclear density. The results showed that the optimal illumination frequency was 31.7 µm⁻¹ used in conjunction with a 4×0.1 NA objective ( = 0.165). This yielded an optical section thickness of 128 µm and an 8.9×contrast enhancement over uniform illumination. We successfully demonstrated the ability to resolve cell nuclei in situ achieved via SIM, which allowed segmentation of nuclei from heterogeneous tissues in the presence of considerable background fluorescence. Specifically, we demonstrate that optical sectioning of fresh intact thick tissues performed equivalently in regards to nuclear density quantification, to physical frozen sectioning and standard microscopy.

Imaging Fourier transform endospectroscopy for in vivo and in situ multispectral imaging

We report the design and implementation of a multispectral imaging Fourier transform endospectroscopy (IFTES) system. The IFTES system employs a flexible fiber bundle catheter coupled to a home-built imaging Fourier transform spectroscope. The instrument enables the performance of non- or minimally invasive subsurface imaging and multispectral imaging at the cellular level in vivo and in situ. A maximum spectral resolution of 0.2 nm at 632.8 nm and a lateral resolution of 4.4 μm were proved. Preliminary results of a standard resolution target, ex-vivo small animal tissue, single wavelength laser, fluorescence solution, in-vivo mouse skin, microspheres mixture, and in-vivo transgenic mouse brain were given to demonstrate the potential of the technique.

Depth-resolved cellular microrheology using HiLo microscopy

It is increasingly important to measure cell mechanical properties in three-dimensional environments. Particle tracking microrheology (PTM) can measure cellular viscoelastic properties; however, out-of-plane data can introduce artifacts into these measurements. We developed a technique that employs HiLo microscopy to reduce out-of-plane contributions. This method eliminated signals from 90% of probes 0.5 μm or further from the focal plane, while retaining all in-plane probes. We used this technique to characterize live-cell bilayers and found that there were significant, frequency-dependent changes to the extracted cell moduli when compared to conventional analysis. Our results indicate that removal of out-of-plane information is vital for accurate assessments of cell mechanical properties.

Fast optically sectioned fluorescence HiLo endomicroscopy

We describe a nonscanning, fiber bundle endomicroscope that performs optically sectioned fluorescence imaging with fast frame rates and real-time processing. Our sectioning technique is based on HiLo imaging, wherein two widefield images are acquired under uniform and structured illumination and numerically processed to reject out-of-focus background. This work is an improvement upon an earlier demonstration of widefield optical sectioning through a flexible fiber bundle. The improved device features lateral and axial resolutions of 2.6 and 17 μm, respectively, a net frame rate of 9.5 Hz obtained by real-time image processing with a graphics processing unit (GPU) and significantly reduced motion artifacts obtained by the use of a double-shutter camera. We demonstrate the performance of our system with optically sectioned images and videos of a fluorescently labeled chorioallantoic membrane (CAM) in the developing G. gallus embryo. HiLo endomicroscopy is a candidate technique for low-cost, high-speed clinical optical biopsies.

Single-shot optical sectioning using two-color probes in HiLo fluorescence microscopy

We describe a wide-field fluorescence microscope setup which combines HiLo microscopy technique with the use of a two-color fluorescent probe. It allows one-shot fluorescence optical sectioning of thick biological moving sample which is illuminated simultaneously with a flat and a structured pattern at two different wavelengths. Both homogenous and structured fluorescence images are spectrally separated at detection and combined similarly with the HiLo microscopy technique. We present optically sectioned full-field images of Xenopus laevis embryos acquired at 25 images/s frame rate.

Analyzing speckle contrast for HiLo microscopy optimization

HiLo microscopy is a recently developed technique that provides both optical sectioning and fast imaging with a simple implementation and at a very low cost. The methodology combines widefield and speckled illumination images to obtain one optically sectioned image. Hence, the characteristics of such speckle illumination ultimately determine the quality of HiLo images and the overall performance of the method. In this work, we study how speckle contrast influence local variations of fluorescence intensity and brightness profiles of thick samples. We present this article as a guide to adjust the parameters of the system for optimizing the capabilities of this novel technology.

Optically sectioned in vivo imaging with speckle illumination HiLo microscopy

We present a simple wide-field imaging technique, called HiLo microscopy, that is capable of producing optically sectioned images in real time, comparable in quality to confocal laser scanning microscopy. The technique is based on the fusion of two raw images, one acquired with speckle illumination and another with standard uniform illumination. The fusion can be numerically adjusted, using a single parameter, to produce optically sectioned images of varying thicknesses with the same raw data. Direct comparison between our HiLo microscope and a commercial confocal laser scanning microscope is made on the basis of sectioning strength and imaging performance. Specifically, we show that HiLo and confocal 3-D imaging of a GFP-labeled mouse brain hippocampus are comparable in quality. Moreover, HiLo microscopy is capable of faster, near video rate imaging over larger fields of view than attainable with standard confocal microscopes. The goal of this paper is to advertise the simplicity, robustness, and versatility of HiLo microscopy, which we highlight with in vivo imaging of common model organisms including planaria, C. elegans, and zebrafish.