High frame-rate multichannel beam-scanning microscopy based on Lissajous trajectories

CRASH2p: Closed-loop Two Photon Imaging in a Freely Moving Animal

Direct measurement of neural activity in freely moving animals is essential for understanding how the brain controls and represents behaviors. Genetically encoded calcium indicators report neural activity as changes in fluorescence intensity, but brain motion confounds quantitative measurement of fluorescence. Translation, rotation, and deformation of the brain and the movements of intervening scattering or auto-fluorescent tissue all alter the amount of fluorescent light captured by a microscope. Compared to single-photon approaches, two photon microscopy is less sensitive to scattering and off-target fluorescence, but more sensitive to motion, and two photon imaging has always required anchoring the microscope to the brain. We developed a closed-loop resonant axial-scanning high-speed two photon (CRASH2p) microscope for real-time 3D motion correction in unrestrained animals, without implantation of reference markers. We complemented CRASH2p with a novel scanning strategy and a multi-stage registration pipeline. We performed volumetric ratiometrically corrected functional imaging in the CNS of freely moving Drosophila larvae and discovered previously unknown neural correlates of behavior.

High-speed forward-viewing optical coherence tomography probe based on Lissajous sampling and sparse reconstruction

We present a novel endoscopy probe using optical coherence tomography (OCT) that combines sparse Lissajous scanning and compressed sensing (CS) for faster data collection. This compact probe is only 4 mm in diameter and achieves a large field of view (FOV) of 2.25 mm2 and a 10 mm working distance. Unlike traditional OCT systems that use bulky raster scanning, our design features a dual-axis piezoelectric mechanism for efficient Lissajous pattern scanning. It employs compressive data reconstruction algorithms that minimize data collection requirements for efficient, high-speed imaging. This approach significantly enhances imaging speed by over 40%, substantially improving miniaturization and performance for endoscopic applications.

MEMS-based two-photon microscopy with Lissajous scanning and image reconstruction under a feed-forward control strategy

Two-photon microscopy (TPM) based on two-dimensional micro-electro-mechanical (MEMS) system mirrors shows promising applications in biomedicine and the life sciences. To improve the imaging quality and real-time performance of TPM, this paper proposes Lissajous scanning control and image reconstruction under a feed-forward control strategy, a dual-parameter alternating drive control algorithm and segmented phase synchronization mechanism, and pipe-lined fusion-mean filtering and median filtering to suppress image noise. A 10 fps frame rate (512 × 512 pixels), a 140 µm × 140 µm field of view, and a 0.62 µm lateral resolution were achieved. The imaging capability of MEMS-based Lissajous scanning TPM was verified by ex vivo and in vivo biological tissue imaging.

Sparse Lissajous scanning reflectance confocal microscope with an adjustable field of view and fast iterative Fourier filtering reconstruction

Medical imaging devices are becoming increasingly compact, necessitating optimization research into different methods of actuation. Actuation influences important parameters of the imaging device such as size, weight, frame rate, field of view (FOV), and image reconstruction for imaging devices point scanning techniques. Current literature around piezoelectric fiber cantilever actuators focuses on device optimization with a fixed FOV but neglects adjustability. In this paper, we introduce an adjustable FOV piezoelectric fiber cantilever microscope and provide a characterization and optimization procedure. To overcome calibration challenges, we utilize a position sensitive detector (PSD) and address trade-offs between FOV and sparsity with a novel inpainting technique. Our work demonstrates the potential for scanner operation when sparsity and distortion dominate the FOV, extending the usable FOV for this form of actuation and others that currently only operate under ideal imaging conditions.

Dual-resonant scanning multiphoton microscope with ultrasound lens and resonant mirror for rapid volumetric imaging

A dual-resonant scanning multiphoton (DRSM) microscope incorporating a tunable acoustic gradient index of refraction lens with a resonant mirror is developed for high-speed volumetric imaging. In the proposed microscope, the pulse train signal of a femtosecond laser is used to trigger an embedded field programmable gate array to sample the multiphoton excited fluorescence signal at the rate of one pixel per laser pulse. It is shown that a frame rate of around 8000 Hz can be obtained in the x-z plane for an image region with a size of 256 × 80 pixels. Moreover, a volumetric imaging rate of over 30 Hz can be obtained for a large image volume of 343 × 343 × 120 μm3 with an image size of 256 × 256 × 80 voxels. Moreover, a volumetric imaging rate of over 30 Hz can be obtained for a large image volume of 256 × 256 × 80 voxels, which represents 343 × 343 × 120 μm3 in field-of-view. The rapid volumetric imaging rate eliminates the aliasing effect for observed temporal frequencies lower than 15 Hz. The practical feasibility of the DRSM microscope is demonstrated by observing the mushroom bodies of a drosophila brain and performing 3D dynamic observations of moving 10-μm fluorescent beads.

High-resolution ptychographic imaging enabled by high-speed multi-pass scanning

As a coherent diffraction imaging technique, ptychography provides high-spatial resolution beyond Rayleigh's criterion of the focusing optics, but it is also sensitively affected by the decoherence coming from the spatial and temporal variations in the experiment. Here we show that high-speed ptychographic data acquisition with short exposure can effectively reduce the impact from experimental variations. To reach a cumulative dose required for a given resolution, we further demonstrate that a continuous multi-pass scan via high-speed ptychography can achieve high-resolution imaging. This low-dose scan strategy is shown to be more dose-efficient, and has potential for radiation-sensitive sample studies and time-resolved imaging.

Handheld laser scanning microscope catheter for real-time and in vivo confocal microscopy using a high definition high frame rate Lissajous MEMS mirror

A handheld confocal microscope using a rapid MEMS scanning mirror facilitates real-time optical biopsy for simple cancer diagnosis. Here we report a handheld confocal microscope catheter using high definition and high frame rate (HDHF) Lissajous scanning MEMS mirror. The broad resonant frequency region of the fast axis on the MEMS mirror with a low Q-factor facilitates the flexible selection of scanning frequencies. HDHF Lissajous scanning was achieved by selecting the scanning frequencies with high greatest common divisor (GCD) and high total lobe number. The MEMS mirror was fully packaged into a handheld configuration, which was coupled to a home-built confocal imaging system. The confocal microscope catheter allows fluorescence imaging of in vivo and ex vivo mouse tissues with 30 Hz frame rate and 95.4% fill factor at 256 × 256 pixels image, where the lateral resolution is 4.35 μm and the field-of-view (FOV) is 330 μm × 330 μm. This compact confocal microscope can provide diverse handheld microscopic applications for real-time, on-demand, and in vivo optical biopsy.

Correction of non-uniform angular velocity and sub-pixel jitter in optical scanning

Optical scanners are widely used in high-resolution scientific, medical, and industrial devices. The accuracy and precision of these instruments are often limited by angular speed fluctuations due to rotational inertia and by poor synchronization between scanning and light detection, respectively. Here we demonstrate that both problems can be mitigated by recording scanner orientation in synchrony with light detection, followed by data resampling. This approach is illustrated with synthetic and experimental data from a point-scanning microscope with a resonant scanner and a non-resonant scanner. Fitting of the resonant scanner orientation data to a cosine model was used to correct image warping and sampling jitter, as well as to precisely interleave image lines collected during the clockwise and counterclockwise resonant scanner portions of the rotation cycle. Vertical scanner orientation data interpolation was used to correct image distortion due to angular speed fluctuations following abrupt control signal changes.

Resonant scanning design and control for fast spatial sampling

Two-dimensional, resonant scanners have been utilized in a large variety of imaging modules due to their compact form, low power consumption, large angular range, and high speed. However, resonant scanners have problems with non-optimal and inflexible scanning patterns and inherent phase uncertainty, which limit practical applications. Here we propose methods for optimized design and control of the scanning trajectory of two-dimensional resonant scanners under various physical constraints, including high frame-rate and limited actuation amplitude. First, we propose an analytical design rule for uniform spatial sampling. We demonstrate theoretically and experimentally that by expanding the design space, the proposed designs outperform previous designs in terms of scanning range and fill factor. Second, we show that we can create flexible scanning patterns that allow focusing on user-defined Regions-of-Interest (RoI) by modulation of the scanning parameters. The scanning parameters are found by an optimization algorithm. In simulations, we demonstrate the benefits of these designs with standard metrics and higher-level computer vision tasks (LiDAR odometry and 3D object detection). Finally, we experimentally implement and verify both unmodulated and modulated scanning modes using a two-dimensional, resonant MEMS scanner. Central to the implementations is high bandwidth monitoring of the phase of the angular scans in both dimensions. This task is carried out with a position-sensitive photodetector combined with high-bandwidth electronics, enabling fast spatial sampling at [Formula: see text] Hz frame-rate.

Adaptive Lissajous scanning pattern design by phase modulation

This paper proposes a phase modulation method for Lissajous scanning systems, which provides adaptive scan pattern design without changing the frame rate or the field of view. Based on a rigorous analysis of Lissajous scanning, phase modulation constrains and a method for pixel calculation are derived. An accurate and simple metric for resolution calculation is proposed based on the area spanned by neighboring pixels and used for scan pattern optimization also considering the scanner dynamics. The methods are implemented using MEMS mirrors for verification of the adaptive pattern shaping, where a 5-fold resolution improvement in a defined region of interest is demonstrated.

Upscaling X-ray nanoimaging to macroscopic specimens

Upscaling X-ray nanoimaging to macroscopic specimens has the potential for providing insights across multiple length scales, but its feasibility has long been an open question. By combining the imaging requirements and existing proof-of-principle examples in large-specimen preparation, data acquisition and reconstruction algorithms, the authors provide imaging time estimates for howX-ray nanoimaging can be scaled to macroscopic specimens. To arrive at this estimate, a phase contrast imaging model that includes plural scattering effects is used to calculate the required exposure and corresponding radiation dose. The coherent X-ray flux anticipated from upcoming diffraction-limited light sources is then considered. This imaging time estimation is in particular applied to the case of the connectomes of whole mouse brains. To image the connectome of the whole mouse brain, electron microscopy connectomics might require years, whereas optimized X-ray microscopy connectomics could reduce this to one week. Furthermore, this analysis points to challenges that need to be overcome (such as increased X-ray detector frame rate) and opportunities that advances in artificial-intelligence-based 'smart' scanning might provide. While the technical advances required are daunting, it is shown that X-ray microscopy is indeed potentially applicable to nanoimaging of millimetre- or even centimetre-size specimens.

Volumetric Lissajous confocal microscopy with tunable spatiotemporal resolution

Dynamic biological systems present challenges to existing three-dimensional (3D) optical microscopes because of their continuous temporal and spatial changes. Most techniques are rigid in adapting the acquisition parameters over time, as in confocal microscopy, where a laser beam is sequentially scanned at a predefined spatial sampling rate and pixel dwell time. Such lack of tunability forces a user to provide scan parameters, which may not be optimal, based on the best assumption before an acquisition starts. Here, we developed volumetric Lissajous confocal microscopy to achieve unsurpassed 3D scanning speed with a tunable sampling rate. The system combines an acoustic liquid lens for continuous axial focus translation with a resonant scanning mirror. Accordingly, the excitation beam follows a dynamic Lissajous trajectory enabling sub-millisecond acquisitions of image series containing 3D information at a sub-Nyquist sampling rate. By temporal accumulation and/or advanced interpolation algorithms, the volumetric imaging rate is selectable using a post-processing step at the desired spatiotemporal resolution for events of interest. We demonstrate multicolor and calcium imaging over volumes of tens of cubic microns with 3D acquisition speeds of 30 Hz and frame rates up to 5 kHz.

A Versatile Illumination System for Real-Time Terahertz Imaging

Terahertz technologies are attracting strong interest from high-end industrial fields, and particularly for non-destructive-testing purposes. Currently lacking compactness, integrability as well as adaptability for those implementations, the development and commercialisation of more efficient sources and detectors progressively ensure the transition toward applicative implementations, especially for real-time full-field imaging. In this work, a flexible illumination system, based on fast beam steering has been developed and characterized. Its primary goal is to suppress interferences induced by the coherence length of certain terahertz sources, spoiling terahertz images. The second goal is to ensure an enhanced signal-to-noise ratio on the detector side by the full use and optimized distribution of the available power. This system provides a homogeneous and adjustable illumination through a simplified setup to guarantee optimum real-time imaging capabilities, tailored to the sample under inspection. Working toward industrial implementations, different illumination process are conveniently assessed as a result of the versatility of this method.

Software-Based Phase Control, Video-Rate Imaging, and Real-Time Mosaicing With a Lissajous-Scanned Confocal Microscope

We present software-based methods for automatic phase control and for mosaicing high-speed, Lissajous-scanned images. To achieve imaging speeds fast enough for mosaicing, we first increase the image update rate tenfold from 3 to 30 Hz, then vertically interpolate each sparse image in real-time to eliminate fixed pattern noise. We validate our methods by imaging fluorescent beads and automatically maintaining phase control over the course of one hour. We then image fixed mouse brain tissues at varying update rates and compare the resulting mosaics. Using reconstructed image data as feedback for phase control eliminates the need for phase sensors and feedback controllers, enabling long-term imaging experiments without additional hardware. Mosaicing subsampled images results in video-rate imaging speeds, nearly fully recovered spatial resolution, and millimeter-scale fields of view.

Design rules for dense and rapid Lissajous scanning

Lissajous microscanners are very popular in compact laser-scanning applications, such as solid-state light detection and ranging (LIDAR), owing to their high-quality factor and low power consumption. In the Lissajous scanner driven by a two-axis micro-electro-mechanical system scanning mirror (MEMS-SM), the design theory is insufficient to meet the temporal and spatial resolution at the same time. In this paper, the greatest common divisor of the two-axis driving frequency is used as the temporal resolution, the concept of the fill factor (FF) is used to describe the spatial resolution of the scanner, and a general algorithm for calculating the FF is presented. Combined with the characteristics of the Lissajous trajectory, three design rules of the general Lissajous scanner are proposed, and the design theory of the Lissajous scanner enabling MEMS LIDAR is perfected. Experimental results show that the proposed design rules can effectively meet the LIDAR design requirements.

2D Au-Coated Resonant MEMS Scanner for NIR Fluorescence Intraoperative Confocal Microscope

The electrostatic MEMS scanner plays an important role in the miniaturization of the microscopic imaging system. We have developed a new two-dimensional (2D) parametrically-resonant MEMS scanner with patterned Au coating (>90% reflectivity at an NIR 785-nm wavelength), for a near-infrared (NIR) fluorescence intraoperative confocal microscopic imaging system with a compact form factor. A silicon-on-insulator (SOI)-wafer based dicing-free microfabrication process has been developed for mass-production with high yield. Based on an in-plane comb-drive configuration, the resonant MEMS scanner performs 2D Lissajous pattern scanning with a large mechanical scanning angle (MSA, ±4°) on each axis at low driving voltage (36 V). A large field-of-view (FOV) has been achieved by using a post-objective scanning architecture of the confocal microscope. We have integrated the new MEMS scanner into a custom-made NIR fluorescence intraoperative confocal microscope with an outer diameter of 5.5 mm at its distal-end. Axial scanning has been achieved by using a piezoelectric actuator-based driving mechanism. We have successfully demonstrated ex vivo 2D imaging on human tissue specimens with up to five frames/s. The 2D resonant MEMS scanner can potentially be utilized for many applications, including multiphoton microendoscopy and wide-field endoscopy.

Design and realization of a wide field of view infrared scanning system with an integrated micro-electromechanical system mirror

We present a wide field of view (FOV) infrared scanning system, designed for single-pixel near-infrared thermal imaging. The scanning system consisted of a two-axis micro-electromechanical system (MEMS) mirror that was incorporated within the lens. The optical system consisted of two groups of lenses and a silicon avalanche photodiode. The system was designed for both the production of thermal images and also to utilize the techniques of radiation thermometry to measure the absolute temperature of targets from 500°C to 1100°C. Our system has the potential for real-time image acquisition, with improved data acquisition electronics. The FOV of our scanning system was ±30° when fully utilizing the MEMS mirror's scanning angle of ±5°. The pixel FOV (calculated from the distance to target size ratio) was 100:1. The image quality was analyzed, including the modulation transfer function, spot diagrams, ray fan plots, lateral chromatic aberrations, distortion, relative illumination, and size-of-source effect. The instrument was fabricated in our laboratory, and one of the thermal images, which was taken with the new lens, is presented as an example of the instrument optical performance.

Spectroscopic stimulated Raman scattering imaging of highly dynamic specimens through matrix completion

Spectroscopic stimulated Raman scattering (SRS) imaging generates chemical maps of intrinsic molecules, with no need for prior knowledge. Despite great advances in instrumentation, the acquisition speed for a spectroscopic SRS image stack is fundamentally bounded by the pixel integration time. In this work, we report three-dimensional sparsely sampled spectroscopic SRS imaging that measures ~20% of pixels throughout the stack. In conjunction with related work in low-rank matrix completion (e.g., the Netflix Prize), we develop a regularized non-negative matrix factorization algorithm to decompose the sub-sampled image stack into spectral signatures and concentration maps. This design enables an acquisition speed of 0.8 s per image stack, with 50 frames in the spectral domain and 40,000 pixels in the spatial domain, which is faster than the conventional raster laser-scanning scheme by one order of magnitude. Such speed allows real-time metabolic imaging of living fungi suspended in a growth medium while effectively maintaining the spatial and spectral resolutions. This work is expected to promote broad application of matrix completion in spectroscopic laser-scanning imaging.

Frequency selection rule for high definition and high frame rate Lissajous scanning

Lissajous microscanners are very attractive in compact laser scanning applications such as endomicroscopy or pro-projection display owing to high mechanical stability and low operating voltages. The scanning frequency serves as a critical factor for determining the scanning imaging quality. Here we report the selection rule of scanning frequencies that can realize high definition and high frame-rate (HDHF) full-repeated Lissajous scanning imaging. The fill factor (FF) monotonically increases with the total lobe number of a Lissajous curve, i.e., the sum of scanning frequencies divided by the great common divisor (GCD) of bi-axial scanning frequencies. The frames per second (FPS), called the pattern repeated rate or the frame rate, linearly increases with GCD. HDHF Lissajous scanning is achieved at the bi-axial scanning frequencies, where the GCD has the maximum value among various sets of the scanning frequencies satisfying the total lobe number for a target FF. Based on this selection rule, the experimental results clearly demonstrate that conventional Lissajous scanners substantially increase both FF and FPS by slightly modulating the scanning frequencies at near the resonance within the resonance bandwidth of a Lissajous scanner. This selection rule provides a new guideline for HDHF Lissajous scanning in compact laser scanning systems.

Locating and Visualizing Crystals for X-Ray Diffraction Experiments

Macromolecular crystallography has advanced from using macroscopic crystals, which might be >1 mm on a side, to crystals that are essentially invisible to the naked eye, or even under a standard laboratory microscope. As crystallography requires recognizing crystals when they are produced, and then placing them in an X-ray, electron, or neutron beam, this provides challenges, particularly in the case of advanced X-ray sources, where beams have very small cross sections and crystals may be vanishingly small. Methods for visualizing crystals are reviewed here, and examples of different types of cases are presented, including: standard crystals, crystals grown in mesophase, in situ crystallography, and crystals grown for X-ray Free Electron Laser or Micro Electron Diffraction experiments. As most techniques have limitations, it is desirable to have a range of complementary techniques available to identify and locate crystals. Ideally, a given technique should not cause sample damage, but sometimes it is necessary to use techniques where damage can only be minimized. For extreme circumstances, the act of probing location may be coincident with collecting X-ray diffraction data. Future challenges and directions are also discussed.

Image Quality Analysis and Optical Performance Requirement for Micromirror-Based Lissajous Scanning Displays

Micromirror-based scanning displays have been the focus of a variety of applications. Lissajous scanning displays have advantages in terms of power consumption; however, the image quality is not good enough. The main reason for this is the varying size and the contrast ratio of pixels at different positions of the image. In this paper, the Lissajous scanning trajectory is analyzed and a new method based on the diamond pixel is introduced to Lissajous displays. The optical performance of micromirrors is discussed. A display system demonstrator is built, and tests of resolution and contrast ratio are conducted. The test results show that the new Lissajous scanning method can be used in displays by using diamond pixels and image quality remains stable at different positions.

Multi-channel beam-scanning imaging at kHz frame rates by Lissajous trajectory microscopy

A beam-scanning microscope based on Lissajous trajectory imaging is described for achieving streaming 2D imaging with continuous frame rates up to 1.4 kHz. The microscope utilizes two fast-scan resonant mirrors to direct the optical beam on a circuitous trajectory through the field of view. By separating the full Lissajous trajectory time-domain data into sub-trajectories (partial, undersampled trajectories) effective frame-rates much higher than the repeat time of the Lissajous trajectory are achieved with many unsampled pixels present. A model-based image reconstruction (MBIR) 3D in-painting algorithm is then used to interpolate the missing data for the unsampled pixels to recover full images. The MBIR algorithm uses a maximum a posteriori estimation with a generalized Gaussian Markov random field prior model for image interpolation. Because images are acquired using photomultiplier tubes or photodiodes, parallelization for multi-channel imaging is straightforward. Preliminary results show that when combined with the MBIR in-painting algorithm, this technique has the ability to generate kHz frame rate images across 6 total dimensions of space, time, and polarization for SHG, TPEF, and confocal reflective birefringence data on a multimodal imaging platform for biomedical imaging. The use of a multi-channel data acquisition card allows for multimodal imaging with perfect image overlay. Image blur due to sample motion was also reduced by using higher frame rates.