Volumetric HiLo microscopy employing an electrically tunable lens

Simple, non-mechanical and automatic calibration approach for axial-scanning microscopy with an electrically tunable lens

We describe a simple and robust calibration approach for axial-scanning microscopy that realizes axial focus shifts with an electrically tunable lens (ETL). We demonstrate the calibration approach based on a microscope with an ETL placed close to the rear stop of the objective lens. By introducing a target-consisted of repeating lines at one known frequency and placed at a ~45° angle to the imaging path, the calibration method captures multiple images at different ETL currents and calibrates the dependence of the axial focus shift on the ETL current by evaluating the sharpness of the captured images. It calibrates the dependence of the magnification of the microscope on the ETL current by measuring the period of the repeating lines in the captured images. The experimental results show that different from the conventional calibration procedure, the proposed scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the ETL current. This might facilitate imaging studies that require the measurement of fine structures in a 3D volume. We also show the calibration procedure can be used to estimate the radius of a conner-arc sample, fabricated using laser micromachining. We believe that this easy-to-use calibration approach may facilitate use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods. RESEARCH HIGHLIGHTS: The proposed calibration scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the electrically tunable lens (ETL) current. It might facilitate imaging studies that require the measurement of fine structures in a 3D volume, and the use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods.

Superresolution structured illumination microscopy reconstruction algorithms: a review

Structured illumination microscopy (SIM) has become the standard for next-generation wide-field microscopy, offering ultrahigh imaging speed, superresolution, a large field-of-view, and long-term imaging. Over the past decade, SIM hardware and software have flourished, leading to successful applications in various biological questions. However, unlocking the full potential of SIM system hardware requires the development of advanced reconstruction algorithms. Here, we introduce the basic theory of two SIM algorithms, namely, optical sectioning SIM (OS-SIM) and superresolution SIM (SR-SIM), and summarize their implementation modalities. We then provide a brief overview of existing OS-SIM processing algorithms and review the development of SR-SIM reconstruction algorithms, focusing primarily on 2D-SIM, 3D-SIM, and blind-SIM. To showcase the state-of-the-art development of SIM systems and assist users in selecting a commercial SIM system for a specific application, we compare the features of representative off-the-shelf SIM systems. Finally, we provide perspectives on the potential future developments of SIM.

Higher order wavefront correction and axial scanning in a single fast and compact piezo-driven adaptive lens

We present a compact adaptive glass membrane lens for higher order wavefront correction and axial scanning, driven by integrated segmented piezoelectric actuators. The membrane can be deformed in a combination of rotational symmetry providing focus control of up to ± 6 m^-1 and spherical aberration correction of up to 5 wavelengths and different discrete symmetries to correct higher order aberrations such as astigmatism, coma and trefoil by up to 10 wavelengths. Our design provides a large clear aperture of 12 mm at an outer diameter of the actuator of 18 mm, a thickness of 2 mm and a response time of less than 2 ms.

Chromatic aberration correction employing reinforcement learning

In fluorescence microscopy a multitude of labels are used that bind to different structures of biological samples. These often require excitation at different wavelengths and lead to different emission wavelengths. The presence of different wavelengths can induce chromatic aberrations, both in the optical system and induced by the sample. These lead to a detuning of the optical system, as the focal positions shift in a wavelength dependent manner and finally to a decrease in the spatial resolution. We present the correction of chromatic aberrations by using an electrical tunable achromatic lens driven by reinforcement learning. The tunable achromatic lens consists of two lens chambers filled with different optical oils and sealed with deformable glass membranes. By deforming the membranes of both chambers in a targeted manner, the chromatic aberrations present in the system can be manipulated to tackle both systematic and sample induced aberrations. We demonstrate chromatic aberration correction of up to 2200 mm and shift of the focal spot positions of 4000 mm. For control of this non-linear system with four input voltages, several reinforcement learning agents are trained and compared. The experimental results show that the trained agent can correct system and sample induced aberration and thereby improve the imaging quality, this is demonstrated using biomedical samples. In this case human thyroid was used for demonstration.

Reliability of tunable lenses: feedback sensors and the influence of temperature, orientation, and vibrations

We compare different aspects of the robustness to environmental conditions of two different types of piezo-actuated fluid-membrane lenses: a silicone membrane lens, where the piezo actuator indirectly deforms the flexible membrane through fluid displacement, and a glass membrane lens, where the piezo actuator directly deforms the stiff membrane. While both lenses operated reliably over the temperature range of 0°-75°C, there was a significant effect on their actuation characteristics, which can be well described through a simple model. The silicone lens in particular showed a variation in focal power of up to 0.1m^{-1 ∘}C^-1. We demonstrated that integrated pressure and temperature sensors can provide feedback for focal power, however, limited by the response time of the elastomers in the lenses, with polyurethane in the support structures of the glass membrane lens being more critical than the silicone. Studying the mechanical effects, the silicone membrane lens showed a gravity-induced coma and tilt, and a reduced imaging quality with the Strehl ratio decreasing from 0.89 to 0.31 at a vibration frequency of 100 Hz and an acceleration of 3g. The glass membrane lens was unaffected by gravity, and the Strehl ratio decreased from 0.92 to 0.73 at a vibration of 100 Hz, 3g. Overall, the stiffer glass membrane lens is more robust against environmental influences.

Tunable doublets: piezoelectric glass membrane lenses with an achromatic and spherical aberration control

We present two versions of tunable achromatic doublets based on each two piezoelectrically actuated glass membranes that create the surface of fluid volumes with different dispersions: a straightforward back-to-back and a more intricate stack of the fluid volumes. In both cases, we can control the chromatic focal shift and focal power independently by a suitable combination of actuation voltages on both active membranes. The doublets have a large aperture of 12 mm at an outer diameter of the actuator of 18 mm, an overall thickness of 3 mm and a short response time of around 0.5 ms and, in addition, provide spherical aberration correction. The two designs have an achromatic focal power range of ±2.2 m^-1 and ±3.2 m^-1 or, for the purpose of actively correcting chromatic errors, a chromatic focal shift at vanishing combined focal power of up to ±0.08 m^-1 and ±0.12 m^-1.

Rapid slide-free and non-destructive histological imaging using wide-field optical-sectioning microscopy

Histopathology based on formalin-fixed and paraffin-embedded tissues has long been the gold standard for surgical margin assessment (SMA). However, routine pathological practice is lengthy and laborious, failing to guide surgeons intraoperatively. In this report, we propose a practical and low-cost histological imaging method with wide-field optical-sectioning microscopy (i.e., High-and-Low-frequency (HiLo) microscopy). HiLo can achieve rapid and non-destructive imaging of freshly-excised tissues at an extremely high acquisition speed of 5 cm²/min with a spatial resolution of 1.3 µm (lateral) and 5.8 µm (axial), showing great potential as an SMA tool that can provide immediate feedback to surgeons and pathologists for intraoperative decision-making. We demonstrate that HiLo enables rapid extraction of diagnostic features for different subtypes of human lung adenocarcinoma and hepatocellular carcinoma, producing surface images of rough specimens with large field-of-views and cellular features that are comparable to the clinical standard. Our results show promising clinical translations of HiLo microscopy to improve the current standard of care.

Assignment of Focus Position with Convolutional Neural Networks in Adaptive Lens Based Axial Scanning for Confocal Microscopy

Adaptive lenses offer axial scanning without mechanical translation and thus are promising to replace mechanical-movement-based axial scanning in microscopy. The scan is accomplished by sweeping the applied voltage. However, the relation between the applied voltage and the resulting axial focus position is not unambiguous. Adaptive lenses suffer from hysteresis effects, and their behaviour depends on environmental conditions. This is especially a hurdle when complex adaptive lenses are used that offer additional functionalities and are controlled with more degrees of freedom. In such case, a common approach is to iterate the voltage and monitor the adaptive lens. Here, we introduce an alternative approach which provides a single shot estimation of the current axial focus position by a convolutional neural network. We use the experimental data of our custom confocal microscope for training and validation. This leads to fast scanning without photo bleaching of the sample and opens the door to automatized and aberration-free smart microscopy. Applications in different types of laser-scanning microscopes are possible. However, maybe the training procedure of the neural network must be adapted for some use cases.

3DM: deep decomposition and deconvolution microscopy for rapid neural activity imaging

We report the development of deep decomposition and deconvolution microscopy (3DM), a computational microscopy method for the volumetric imaging of neural activity. 3DM overcomes the major challenge of deconvolution microscopy, the ill-posed inverse problem. We take advantage of the temporal sparsity of neural activity to reformulate and solve the inverse problem using two neural networks which perform sparse decomposition and deconvolution. We demonstrate the capability of 3DM via in vivo imaging of the neural activity of a whole larval zebrafish brain with a field of view of 1040 µm × 400 µm × 235 µm and with estimated lateral and axial resolutions of 1.7 µm and 5.4 µm, respectively, at imaging rates of up to 4.2 volumes per second.

Reduction of spherical and chromatic aberration in axial-scanning optical systems with tunable lenses

Optical systems with integrated tunable lenses allow for rapid axial-scanning without mechanical translation of the components. However, changing the power of the tunable lens typically upsets aberration balancing across the system, introducing spherical and chromatic aberrations that limit the usable axial range. This study develops an analytical approximation for the tuning-induced spherical and axial chromatic aberration of a general optical system containing a tunable lens element. The resulting model indicates that systems can be simultaneously corrected for both tuning-induced spherical and chromatic aberrations by controlling the lateral magnification, coma, and pupil lateral color prior to the tunable surface. These insights are then used to design a realizable axial-scanning microscope system with a high numerical aperture and diffraction-limited performance over a wide field of view and deep axial range.

Single-scan HiLo with line-illumination strategy for optical section imaging of thick tissues

Optical sectioning has been widely employed for inhibiting out-of-focus backgrounds in three-dimensional (3D) imaging of biological samples. However, point scanning imaging or multiple acquisitions for wide-field optical sectioning in epi-illumination microscopy remains time-consuming for large-scale imaging. In this paper, we propose a single-scan optical sectioning method based on the hybrid illumination (HiLo) algorithm with a line-illumination strategy. Our method combines HiLo background inhibition with confocal slit detection. It thereby offers a higher optical sectioning capability than wide-field HiLo and line-confocal imaging without extra modulation and multiple data acquisition. To demonstrate the optical-sectioning capability of our system, we imaged a thin fluorescent plane and different fluorescence-labeled mouse tissue. Our method shows an excellent background inhibition in thick tissue and thus potentially provides an alternative tool for 3D imaging of large-scale biological tissue.

Diffraction-limited axial scanning in thick biological tissue with an aberration-correcting adaptive lens

Diffraction-limited deep focusing into biological tissue is challenging due to aberrations that lead to a broadening of the focal spot. The diffraction limit can be restored by employing aberration correction for example with a deformable mirror. However, this results in a bulky setup due to the required beam folding. We propose a bi-actuator adaptive lens that simultaneously enables axial scanning and the correction of specimen-induced spherical aberrations with a compact setup. Using the bi-actuator lens in a confocal microscope, we show diffraction-limited axial scanning up to 340 μm deep inside a phantom specimen. The application of this technique to in vivo measurements of zebrafish embryos with reporter-gene-driven fluorescence in a thyroid gland reveals substructures of the thyroid follicles, indicating that the bi-actuator adaptive lens is a meaningful supplement to the existing adaptive optics toolset.

Confocal fluorescence microscopy for geometry parameter measurements of submerged micro-structures

Due to the challenging environment of micro-manufacturing techniques where the workpiece is submerged in a fluid, a contactless in situ capable measurement is required for quality control. However, the in situ conditions and the small specimen dimensions hinder the use of conventional metrology. Confocal fluorescence microscopy is shown to enable step height measurements of a specimen submerged in a 2.6 mm thick fluid layer with an uncertainty of 8.8 μm by fitting a model of the fluorescence intensity to the measured signal. To ascertain the potential of the proposed measurement approach, the minimal achievable uncertainty of 0.07 μm for a shot-noise-limited signal is derived.

Extended depth of field for single biomolecule optical imaging-force spectroscopy

Real-time optical imaging combined with single-molecule manipulation broadens the horizons for acquiring information about the spatiotemporal localization and the mechanical details of target molecules. To obtain an optical signal outside the focal plane without unintended interruption of the force signal in single-molecule optical imaging-force spectroscopy, we developed an optical method to extend the depth of field in a high numerical aperture objective (≥ 1.2), required to visualize a single fluorophore. By axial scanning, using an electrically tunable lens with a fixed sample, we were successfully able to visualize the epidermal growth factor receptor (EGFR) moving along the three-dimensionally elongated filamentous actin bundles connecting cells (intercellular nanotube), while another EGFR on the intercellular nanotube was trapped by optical tweezers in living cells. Our approach is simple, fast and inexpensive, but it is powerful for imaging target molecules axially in single-molecule optical imaging-force spectroscopy.

Electrically tunable binary phase Fresnel lens based on a dielectric elastomer actuator

We propose and demonstrate an all-solid-state tunable binary phase Fresnel lens with electrically controllable focal length. The lens is composed of a binary phase Fresnel zone plate, a circular acrylic frame, and a dielectric elastomer (DE) actuator which is made of a thin DE layer and two compliant electrodes using silver nanowires. Under electric potential, the actuator produces in-plane deformation in a radial direction that can compress the Fresnel zones. The electrically-induced deformation compresses the Fresnel zones to be contracted as high as 9.1% and changes the focal length, getting shorter from 20.0 cm to 14.5 cm. The measured change in the focal length of the fabricated lens is consistent with the result estimated from numerical simulation.

Fundamental uncertainty limit for speckle displacement measurements

The basic metrological task in speckle photography is to quantify displacements of speckle patterns, allowing for instance the investigation of the mechanical load and modification of objects with rough surfaces. However, the fundamental limit of the measurement uncertainty due to photon shot noise is unknown. For this reason, the Cramér-Rao bound (CRB) is derived for speckle displacement measurements, representing the squared minimal achievable measurement uncertainty. As result, the CRB for speckle patterns is only two times the CRB for an ideal point light source. Hence, speckle photography is an optimal measurement approach for contactless displacement measurements on rough surfaces. In agreement with a derivation from Heisenberg's uncertainty principle, the CRB depends on the number of detected photons and the diffraction limit of the imaging system described by the speckle size. The theoretical results are verified and validated, demonstrating the capability for displacement measurements with nanometer resolution.

Transmission of Multiple Signals through an Optical Fiber Using Wavefront Shaping

The transmission of multiple independent optical signals through a multimode fiber is accomplished using wavefront shaping in order to compensate for the light distortion during the propagation within the fiber. Our methodology is based on digital optical phase conjugation employing only a single spatial light modulator, where the optical wavefront is individually modulated at different regions of the modulator, one region per light signal. Digital optical phase conjugation approaches are considered to be faster than other wavefront shaping approaches, where (for example) a complete determination of the wave propagation behavior of the fiber is performed. In contrast, the presented approach is time-efficient since it only requires one calibration per light signal. The proposed method is potentially appropriate for spatial division multiplexing in communications engineering. Further application fields are endoscopic light delivery in biophotonics, especially in optogenetics, where single cells in biological tissue have to be selectively illuminated with high spatial and temporal resolution.

Spiral phase mask shadow-imaging for 3D-measurement of flow fields

Particle tracking velocimetry (PTV) is a valuable tool for microfluidic analysis. Especially mixing processes and the environmental interaction of fluids on a microscopic scale are of particular importance for pharmaceutical and biomedical applications. However, currently applied techniques suffer from the lag of instantaneous depth information. Here we present a scan-free, shadow-imaging PTV-technique for 3D trajectory and velocity measurement of flow fields in micro-channels with 2 µm spatial resolution. By using an incoherent light source, one camera and a spatial light modulator (LCoS-SLM) that generates double-images of the seeding particle shadows, it is a simply applicable and highly scalable technique.

In vivo volumetric fluorescence sectioning microscopy with mechanical-scan-free hybrid illumination imaging

Optical sectioning microscopy in wide-field fashion has been widely used to obtain three-dimensional images of biological samples; however, it requires scanning in depth and considerable time to acquire multiple depth information of a volumetric sample. In this paper, in vivo optical sectioning microscopy with volumetric hybrid illumination, with no mechanical moving parts, is presented. The proposed system is configured such that the optical sectioning is provided by hybrid illumination using a digital micro-mirror device (DMD) for uniform and non-uniform pattern projection, while the depth of imaging planes is varied by using an electrically tunable-focus lens with invariant magnification and resolution. We present and characterize the design, implementation, and experimentally demonstrate the proposed system's ability through 3D imaging of in vivo Canenorhabditis elegans' growth cones.