Super-resolution scanning laser microscopy through virtually structured detection

Modulated pattern scanning microscopy

In confocal microscopy, the effective optical transfer function (OTF_eff) with Gaussian plane wave illumination covers very few high-frequency components, which prohibits further improvement of the resolution. We propose modulated pattern scanning microscopy (MPSM) to achieve super-resolution imaging. In MPSM, the phase of the illumination beam is modulated to reassign the OTF_eff in the Fourier domain. The phase mask is designed using an optimization algorithm to obtain the fluorescence emission pattern with rich high-frequency components. Then, the postprocessing algorithms are adapted to retrieve the super-resolved images from the modulated recordings. Simulation and experiment demonstrate that MPSM increases the resolution approximately 1.3 times better than confocal microscopy. Compared with conventional deconvolution, MPSM exhibits a higher signal-to-noise ratio.

Resolution-enhanced OCT and expanded framework of information capacity and resolution in coherent imaging

Spatial resolution in conventional optical microscopy has traditionally been treated as a fixed parameter of the optical system. Here, we present an approach to enhance transverse resolution in beam-scanned optical coherence tomography (OCT) beyond its aberration-free resolution limit, without any modification to the optical system. Based on the theorem of invariance of information capacity, resolution-enhanced (RE)-OCT navigates the exchange of information between resolution and signal-to-noise ratio (SNR) by exploiting efficient noise suppression via coherent averaging and a simple computational bandwidth expansion procedure. We demonstrate a resolution enhancement of 1.5 × relative to the aberration-free limit while maintaining comparable SNR in silicone phantom. We show that RE-OCT can significantly enhance the visualization of fine microstructural features in collagen gel and ex vivo mouse brain. Beyond RE-OCT, our analysis in the spatial-frequency domain leads to an expanded framework of information capacity and resolution in coherent imaging that contributes new implications to the theory of coherent imaging. RE-OCT can be readily implemented on most OCT systems worldwide, immediately unlocking information that is beyond their current imaging capabilities, and so has the potential for widespread impact in the numerous areas in which OCT is utilized, including the basic sciences and translational medicine.

The Development of Microscopy for Super-Resolution: Confocal Microscopy, and Image Scanning Microscopy

Optical methods of super-resolution microscopy, such as confocal microscopy, structured illumination, nonlinear microscopy, and image scanning microscopy are reviewed. These methods avoid strong invasive interaction with a sample, allowing the observation of delicate biological samples. The meaning of resolution and the basic principles and different approaches to superresolution are discussed.

Review of bio-optical imaging systems with a high space-bandwidth product

Optical imaging has served as a primary method to collect information about biosystems across scales-from functionalities of tissues to morphological structures of cells and even at biomolecular levels. However, to adequately characterize a complex biosystem, an imaging system with a number of resolvable points, referred to as a space-bandwidth product (SBP), in excess of one billion is typically needed. Since a gigapixel-scale far exceeds the capacity of current optical imagers, compromises must be made to obtain either a low spatial resolution or a narrow field-of-view (FOV). The problem originates from constituent refractive optics-the larger the aperture, the more challenging the correction of lens aberrations. Therefore, it is impractical for a conventional optical imaging system to achieve an SBP over hundreds of millions. To address this unmet need, a variety of high-SBP imagers have emerged over the past decade, enabling an unprecedented resolution and FOV beyond the limit of conventional optics. We provide a comprehensive survey of high-SBP imaging techniques, exploring their underlying principles and applications in bioimaging.

Virtually structured detection enables super-resolution ophthalmoscopy of rod and cone photoreceptors in human retina

BACKGROUND

High resolution imaging is desirable for advanced study and clinical management of retinal diseases. However, spatial resolution of retinal imaging has been limited due to available numerical aperture and optical aberration of the ocular optics. This study is to develop and validate virtually structured detection (VSD) to surpass diffraction limit for resolution improvement in in vivo retinal imaging of awake human.

METHODS

A rapid line scanning laser ophthalmoscope (SLO) was constructed for in vivo retinal imaging. A high speed (25,000 kHz) camera was used for recording the two-dimensional (2D) light reflectance profile, corresponding to each focused line illumination. VSD was implemented to the 2D light reflectance profiles for super-resolution reconstruction. Because each 2D light reflectance profile was recorded within 40 μs, the intra-frame blur due to eye movements can be ignored. Digital registration was implemented to further compensate for inter-frame eye movements, before the VSD processing. Based on digital processing, the modulation transfer function (MTF) of the imaging system was derived for objective identification of the cut-off frequency of ocular optics, which is essential for robust VSD processing to ensure reliable super-resolution imaging. Dynamic motility analysis of the super-resolution images was implemented to further enhance the imaging contrast of retinal rod and cone photoreceptors.

RESULTS

The VSD based super-resolution SLO significantly improved image quality compared with equivalent wide-field imaging. In vivo observation of individual retinal photoreceptors has been demonstrated unambiguously. Dynamic motility analysis of the super-resolution images enhanced the contrast of retinal rod and cone photoreceptors, and revealed sub-cellular structures in cone photoreceptors.

CONCLUSIONS

In conjunction with rapid line-scan imaging and digital registration to minimize the effect of eye movements, VSD enabled resolution improvement to observe individual retinal photoreceptors without the involvement of adaptive optics (AO). An objective method has been developed to identify MTF to enable quantitative estimation of the cut-off frequency required for robust VSD processing.

High-throughput optical sectioning via line-scanning imaging with digital structured modulation

Optical sectioning with high-throughput, a high signal-to-noise ratio (SNR), and submicrometer resolution is crucial, but challenging, to three-dimensional visualization of large biological tissue samples. Here we propose line-scanning imaging with digital structured modulation for optical sectioning. Our method generates images with a significantly improved SNR, compared to wide-field structured illumination microscopy (WF-SIM), without residual modulation artifacts. We image a 14.5mm×11.5mm horizontal view of mouse brain tissue at a pixel resolution of 0.32µm×0.32µm in 101 s, which, compared to WF-SIM, represents a significant improvement on imaging throughput. These results provide development opportunities for high-throughput, high-resolution large-area optical imaging methods.

Super-resolution ophthalmoscopy: Virtually structured detection for resolution improvement in retinal imaging

Quantitative retinal imaging is essential for advanced study and clinical management of eye diseases. However, spatial resolution of retinal imaging has been limited due to available numerical aperture and optical aberration of the ocular optics. Structured illumination microscopy has been established to break the diffraction-limit resolution in conventional light microscopy. However, practical implementation of structured illumination microscopy for in vivo ophthalmoscopy of the retina is challenging due to inevitable eye movements that can produce phase artifacts. Recently, we have demonstrated the feasibility of using virtually structured detection as one alternative to structured illumination microscopy for super-resolution imaging. By providing the flexibility of digital compensation of eye movements, the virtually structured detection provides a feasible, phase-artifact-free strategy to achieve super-resolution ophthalmoscopy. In this article, we summarize the technical rationale of virtually structured detection, and its implementations for super-resolution imaging of freshly isolated retinas, intact animals, and awake human subjects.

Rapid Image Reconstruction of Structured Illumination Microscopy Directly in the Spatial Domain

Super-resolution structured illumination microscopy (SIM) routinely performs image reconstruction in the frequency domain using an approach termed frequency-domain reconstruction (FDR). Due to multiple Fourier transforms between the spatial and frequency domains, SIM suffers from low reconstruction speed, constraining its applications in real-time, dynamic imaging. To overcome this limitation, we developed a new method for SIM image reconstruction, termed spatial domain reconstruction (SDR). SDR is intrinsically simpler than FDR, does not require Fourier transforms and the theory predicts it to be a rapid image reconstruction method. Results show that SDR reconstructs a super-resolution image 7-fold faster than FDR, producing images that are equal to either FDR or the widely-used FairSIM. We provide a proof-of-principle using mobile fluorescent beads to demonstrate the utility of SDR in imaging moving objects. Consequently, replacement of the FDR approach with SDR significantly enhances SIM as the desired method for live-cell, instant super-resolution imaging. This means that SDR-SIM is a "What You See Is What You Get" approach to super-resolution imaging.

Virtual single-pixel imaging-based deconvolution method for spatial resolution improvement in wide-field fluorescence microscopy

Deconvolution technique has been widely used in fluorescence microscopy to restore fine structures of biological samples. However, conventional deconvolution methods usually achieve little contrast enhancement in dense structures that have the intervals close to the Rayleigh criterion. Herein, we developed a novel deconvolution method, termed virtual single-pixel imaging (v-SPI). Differing from existing deconvolution methods, v-SPI aims to retrieve the less blurred image directly, not the sample distribution which cannot be actually obtained. And the result can be retrieved simply by solving a linear matrix in spatial domain. In addition, the proposed method has no requirement of calibrating parameters of microscope system. Simulation and experimental results demonstrated that the proposed v-SPI method can enhance the contrast of dense structures significantly and acquire a 24% increase in resolution.

Structured illumination imaging without grating rotation based on mirror operation on 1D Fourier spectrum

Structured illumination microscopy (SIM) is a rapidly developing a super-resolution optical microscopy technique. With SIM, the grating is needed in order to rotate several angles for illuminating the sample in different directions. Multiple rotations reduce the imaging speed and grating rotation angle errors damage the image recovery quality. We introduce mirror transformation on one-dimension (1D) Fourier spectrum to SIM for resolving the problems of low imaging speed and severe impact on image reconstruction quality by grating rotation angle errors. When mirror operation and SIM are combined, the grating is placed at an orientation for obtaining three shadow images. The three shadow images are acquired by CCD at three different phase shift for a direction of grating. Thus, the SIM imaging speed is faster and the effect on image reconstruction quality by grating rotation angle errors is greatly reduced.

Nonlinear Focal Modulation Microscopy

We demonstrate nonlinear focal modulation microscopy (NFOMM) to achieve superresolution imaging. Traditional approaches to superresolution that utilize point scanning often rely on spatially reducing the size of the emission pattern by directly narrowing (e.g., through minimizing the detection pinhole in Airyscan, Zeiss) or indirectly peeling its outer profiles [e.g., through depleting the outer emission region in stimulated emission depletion (STED) microscopy]. We show that an alternative conceptualization that focuses on maximizing the optical system's frequency shifting ability offers advantages in further improving resolution while reducing system complexity. In NFOMM, a spatial light modulator and a suitably intense laser illumination are used to implement nonlinear focal-field modulation to achieve a transverse spatial resolution of ∼60  nm (∼λ/10). We show that NFOMM is comparable with STED microscopy and suitable for fundamental biology studies, as evidenced in imaging nuclear pore complexes, tubulin and vimentin in Vero cells. Since NFOMM is readily implemented as an add-on module to a laser-scanning microscope, we anticipate wide utility of this new imaging technique.

Rapid super-resolution confocal microscopy through virtual structured detection based on serial scanning in array

To improve the imaging speed of a confocal microscope with virtual structured detection, we have designed an optical system with rigid coordination control of the CCD, galvanometer scanner, and laser diode. In this system, the width of the coherent transfer function expands, which enhances the lateral resolution by a factor of 1.4. Also, the temporal image sequence is transformed to a spatial one so that multiple images can be acquired during a single exposure period of the CCD. This method increases the system imaging speed 25-fold at least, and an even higher speed can be achieved by further increasing the number of spots recorded during a single exposure period.

Design of diffractive optical elements for subdiffraction spot arrays with high light efficiency

Spot arrays beyond the diffraction limit are required in many optical applications, and the shaping of a light beam into subdiffraction spot arrays can be implemented by diffractive optical elements (DOEs). However, the low light efficiency of spot arrays is undesired in many applications. In this paper, a modified Gerchberg-Saxton algorithm is presented for generating DOEs to realize subdiffraction spot arrays with higher light efficiency. In the simulation, the spot size is reduced to approximately 70%-90% of the diffraction-limited spot, and the corresponding light efficiency is within the range of 20% to 50%. The experimental results are also shown to demonstrate the effectiveness of the proposed algorithm.

Connectomics of synaptic microcircuits: lessons from the outer retina

Photoreceptors form a sophisticated synaptic complex with bipolar and horizontal cells, transmitting the signals generated by the phototransduction cascade to downstream retinal circuitry. The cone photoreceptor synapse shows several characteristic anatomical connectivity motifs that shape signal transfer: typically, ON-cone bipolar cells receive photoreceptor input through invaginating synapses; OFF-cone bipolar cells form basal synapses with photoreceptors. Both ON- and OFF-cone bipolar cells are believed to sample from all cone photoreceptors within their dendritic span. Electron microscopy and immunolabelling studies have established the robustness of these motifs, but have been limited by trade-offs in sample size and spatial resolution, respectively, constraining precise quantitative investigation to a few individual cells. 3D-serial electron microscopy overcomes these limitations and has permitted complete sets of neurons to be reconstructed over a comparatively large section of retinal tissue. Although the published mouse dataset lacks labels for synaptic structures, the characteristic anatomical motifs at the photoreceptor synapse can be exploited to identify putative synaptic contacts, which has enabled the development of a quantitative description of outer retinal connectivity. This revealed unexpected exceptions to classical motifs, including substantial interaction between rod and cone pathways at the photoreceptor synapse, sparse photoreceptor sampling and atypical contacts. Here, we summarize what was learned from this study in a more general context: we consider both the implications and limitations of the study and identify promising avenues for future research.

Image formation in image scanning microscopy, including the case of two-photon excitation

The effect of combining the image scanning microscopy (ISM) technique with two-photon fluorescence microscopy is analyzed. The effective spatial frequency cutoff can be doubled, as compared with conventional two-photon fluorescence microscopy, and the magnitude of the optical transfer function near the cutoff of conventional two-photon microscopy is increased by orders of magnitude. For the two-photon case, it is found that the optimum pixel reassignment factor in ISM is not equal to one half, as is often assumed in single-photon fluoresence image scanning microscopy, because the excitation and detection point spread functions are different. The optimum reassignment factor depends on the noise level, and in general the useful cutoff spatial frequency is about 1.8 times that for conventional two-photon microscopy. The effect of altering the reassignment factor in single-photon fluorescence ISM with a Stokes shift is also investigated. Illumination using pupil filters, such as by a Bessel beam, is considered. Using a ring detector array is found to result in good imaging behavior, exhibiting a sharpening of the point spread function by a factor of 1.7 compared with conventional fluorescence. Image formation in ISM can be considered in a four-dimensional spatial frequency space, giving new insight into the imaging properties. This approach is related to phase space representations such as the Wigner distribution function and the ambiguity function. A noniterative algorithm for image restoration is proposed.

In vivo super-resolution retinal imaging through virtually structured detection

High resolution is important for sensitive detection of subtle distortions of retinal morphology at an early stage of eye diseases. We demonstrate virtually structured detection (VSD) as a feasible method to achieve in vivo super-resolution ophthalmoscopy. A line-scanning strategy was employed to achieve a super-resolution imaging speed up to 127 ?? frames / s with a frame size of 512 × 512 ?? pixels . The proof-of-concept experiment was performed on anesthetized frogs. VSD-based super-resolution images reveal individual photoreceptors and nerve fiber bundles unambiguously. Both image contrast and signal-to-noise ratio are significantly improved due to the VSD implementation.

Image recombination transform algorithm for superresolution structured illumination microscopy

Structured illumination microscopy (SIM) is an attractive choice for fast superresolution imaging. The generation of structured illumination patterns made by interference of laser beams is broadly employed to obtain high modulation depth of patterns, while the polarizations of the laser beams must be elaborately controlled to guarantee the high contrast of interference intensity, which brings a more complex configuration for the polarization control. The emerging pattern projection strategy is much more compact, but the modulation depth of patterns is deteriorated by the optical transfer function of the optical system, especially in high spatial frequency near the diffraction limit. Therefore, the traditional superresolution reconstruction algorithm for interference-based SIM will suffer from many artifacts in the case of projection-based SIM that possesses a low modulation depth. Here, we propose an alternative reconstruction algorithm based on image recombination transform, which provides an alternative solution to address this problem even in a weak modulation depth. We demonstrated the effectiveness of this algorithm in the multicolor superresolution imaging of bovine pulmonary arterial endothelial cells in our developed projection-based SIM system, which applies a computer controlled digital micromirror device for fast fringe generation and multicolor light-emitting diodes for illumination. The merit of the system incorporated with the proposed algorithm allows for a low excitation intensity fluorescence imaging even less than 1??W/cm2, which is beneficial for the long-term, in vivo superresolved imaging of live cells and tissues.

Virtual k-Space Modulation Optical Microscopy

We report a novel superresolution microscopy approach for imaging fluorescence samples. The reported approach, termed virtual k-space modulation optical microscopy (VIKMOM), is able to improve the lateral resolution by a factor of 2, reduce the background level, improve the optical sectioning effect and correct for unknown optical aberrations. In the acquisition process of VIKMOM, we used a scanning confocal microscope setup with a 2D detector array to capture sample information at each scanned x-y position. In the recovery process of VIKMOM, we first modulated the captured data by virtual k-space coding and then employed a ptychography-inspired procedure to recover the sample information and correct for unknown optical aberrations. We demonstrated the performance of the reported approach by imaging fluorescent beads, fixed bovine pulmonary artery endothelial (BPAE) cells, and living human astrocytes (HA). As the VIKMOM approach is fully compatible with conventional confocal microscope setups, it may provide a turn-key solution for imaging biological samples with ∼100  nm lateral resolution, in two or three dimensions, with improved optical sectioning capabilities and aberration correcting.

Confocal laser scanning microscopy with spatiotemporal structured illumination

Confocal laser scanning microscopy (CLSM), which is widely utilized in the biological and biomedical sciences, is limited in spatial resolution due to diffraction to about half the light wavelength. Here we have combined structured illumination with CLSM to enhance its spatial resolution. To this end, we have used a spatial light modulator (SLM) to generate fringe patterns of different orientations and phase shifts in the excitation spot without any mechanical movement. We have achieved 1.8 and 1.7 times enhanced lateral and axial resolutions, respectively, by synthesizing the object spectrum along different illumination directions. This technique is thus a promising tool for high-resolution morphological or fluorescence imaging, especially in deep tissue.

Intrinsic optical signal imaging of retinal physiology: a review

Intrinsic optical signal (IOS) imaging promises to be a noninvasive method for high-resolution examination of retinal physiology, which can advance the study and diagnosis of eye diseases. While specialized optical instruments are desirable for functional IOS imaging of retinal physiology, in depth understanding of multiple IOS sources in the complex retinal neural network is essential for optimizing instrument designs. We provide a brief overview of IOS studies and relationships in rod outer segment suspensions, isolated retinas, and intact eyes. Recent developments of line-scan confocal and functional optical coherence tomography (OCT) instruments have allowed in vivo IOS mapping of photoreceptor physiology. Further improvements of the line-scan confocal and functional OCT systems may provide a feasible solution to pursue functional IOS mapping of human photoreceptors. Some interesting IOSs have already been detected in inner retinal layers, but better development of the IOS instruments and software algorithms is required to achieve optimal physiological assessment of inner retinal neurons.

Rapid super-resolution line-scanning microscopy through virtually structured detection

Virtually structured detection (VSD) has been demonstrated to break the diffraction limit in scanning laser microscopy (SLM). VSD provides an easy, low-cost, and phase-artifact-free strategy to achieve super-resolution imaging. However, practical application of this method is challenging due to a limited image acquisition speed. We report here the combination of VSD and line-scanning microscopy (LSM) to improve the image acquisition speed. A motorized dove prism was used to achieve automatic control of four-angle (i.e., 0°, 45°, 90°, and 135°) scanning, thus ensuring isotropic resolution improvement. Both an optical resolution target and a living frog eyecup were used to verify resolution enhancement.

Resolution enhancement of two-photon microscopy via intensity-modulated laser scanning structured illumination

Conventional structured illumination microscopy (SIM) with wide-field illumination is an applicable tool to provide resolution enhancement. And yet its applications in thick specimens are still full of challenges. By combing the structured illumination concept with two-photon excitation, a laser scanning two-photon structured illumination microscope (LSTP-SIM) was constructed to gain ∼1.42-fold lateral resolution enhancement in contrast to two-photon fluorescence microscopy. With a point-scanning geometry, an acoustic-optical modulator was used to modulate temporally the excitation intensity in order to produce the structured illumination pattern. The theoretical models of image formation and image reconstruction were clearly established. Simulation and experiments were both performed to show the capability of this system to enhance the lateral resolution. Combined with the inherent optical sectioning power of the two-photon excitation, LSTP-SIM would have the potential for applications in optically-thick specimens.

Super-Resolution Scanning Laser Microscopy Based on Virtually Structured Detection

Light microscopy plays a key role in biological studies and medical diagnosis. The spatial resolution of conventional optical microscopes is limited to approximately half the wavelength of the illumination light as a result of the diffraction limit. Several approaches-including confocal microscopy, stimulated emission depletion microscopy, stochastic optical reconstruction microscopy, photoactivated localization microscopy, and structured illumination microscopy-have been established to achieve super-resolution imaging. However, none of these methods is suitable for the super-resolution ophthalmoscopy of retinal structures because of laser safety issues and inevitable eye movements. We recently experimentally validated virtually structured detection (VSD) as an alternative strategy to extend the diffraction limit. Without the complexity of structured illumination, VSD provides an easy, low-cost, and phase artifact-free strategy to achieve super-resolution in scanning laser microscopy. In this article we summarize the basic principles of the VSD method, review our demonstrated single-point and line-scan super-resolution systems, and discuss both technical challenges and the potential of VSD-based instrumentation for super-resolution ophthalmoscopy of the retina.

Resolution analysis in computational imaging with patterned illumination and bucket detection

In computational imaging by pattern projection, a sequence of microstructured light patterns codified onto a programmable spatial light modulator is used to sample an object. The patterns are used as generalized measurement modes where the object information is expressed. In this Letter, we show that the resolution of the recovered image is only limited by the numerical aperture of the projecting optics regardless of the quality of the collection optics. We provide proof-of-principle experiments where the single-pixel detection strategy outperforms the resolution achieved using a conventional optical array detector for optical imaging. It is advantageous in the presence of real-world conditions, such as optical aberrations and optical imperfections in between the sample and the sensor. We provide experimental verification of image retrieval even when an optical diffuser prevents imaging with a megapixel array camera.

Breaking diffraction limit of lateral resolution in optical coherence tomography

Quantitative imaging of biomedical specimens is essential in biomedical study and diagnosis. Given excellent capability in three-dimensional (3D) imaging, optical coherence tomography (OCT) has been extensively used in ophthalmic imaging, vascular medicine, dermatological study, etc. Lateral resolution of the OCT is light diffraction limited, which precludes the feasibility of quantitative assessment of individual cells. In this paper, we demonstrated the feasibility of breaking diffraction-limit in OCT imaging through virtually structured detection (VSD). OCT examination of optical resolution target verified resolution doubling in the VSD based OCT imaging. Super-resolution OCT identification of individual frog photoreceptors was demonstrated to verify the potential of resolution enhancement in retinal imaging. We anticipate that further development of the VSD based OCT promises an easy, low cost strategy to achieve sub-cellular resolution tomography of the retina and other biological systems.