Direct wavefront sensing in adaptive optical microscopy using backscattered light

Adaptive optics for optical microscopy [Invited]

Optical microscopy is widely used to visualize fine structures. When applied to bioimaging, its performance is often degraded by sample-induced aberrations. In recent years, adaptive optics (AO), originally developed to correct for atmosphere-associated aberrations, has been applied to a wide range of microscopy modalities, enabling high- or super-resolution imaging of biological structure and function in complex tissues. Here, we review classic and recently developed AO techniques and their applications in optical microscopy.

Recent advances in optical imaging through deep tissue: imaging probes and techniques

Optical imaging has been essential for scientific observations to date, however its biomedical applications has been restricted due to its poor penetration through tissues. In living tissue, signal attenuation and limited imaging depth caused by the wave distortion occur because of scattering and absorption of light by various molecules including hemoglobin, pigments, and water. To overcome this, methodologies have been proposed in the various fields, which can be mainly categorized into two stategies: developing new imaging probes and optical techniques. For example, imaging probes with long wavelength like NIR-II region are advantageous in tissue penetration. Bioluminescence and chemiluminescence can generate light without excitation, minimizing background signals. Afterglow imaging also has high a signal-to-background ratio because excitation light is off during imaging. Methodologies of adaptive optics (AO) and studies of complex media have been established and have produced various techniques such as direct wavefront sensing to rapidly measure and correct the wave distortion and indirect wavefront sensing involving modal and zonal methods to correct complex aberrations. Matrix-based approaches have been used to correct the high-order optical modes by numerical post-processing without any hardware feedback. These newly developed imaging probes and optical techniques enable successful optical imaging through deep tissue. In this review, we discuss recent advances for multi-scale optical imaging within deep tissue, which can provide reseachers multi-disciplinary understanding and broad perspectives in diverse fields including biophotonics for the purpose of translational medicine and convergence science.

Multi-segmented feature coupling for jointly reconstructing initial pressure and speed of sound in photoacoustic computed tomography

SIGNIFICANCE

Photoacoustic computed tomography (PACT) is a fast-growing imaging modality. In PACT, the image quality is degraded due to the unknown distribution of the speed of sound (SoS). Emerging initial pressure (IP) and SoS joint-reconstruction methods promise reduced artifacts in PACT. However, previous joint-reconstruction methods have some deficiencies. A more effective method has promising prospects in preclinical applications.

AIM

We propose a multi-segmented feature coupling (MSFC) method for SoS-IP joint reconstruction in PACT.

APPROACH

In the proposed method, the ultrasound detectors were divided into multiple sub-arrays with each sub-array and its opposite counterpart considered to be a pair. The delay and sum algorithm was then used to reconstruct two images based on a subarray pair and estimated a direction-specific SoS, based on image correlation and the orientation of the subarrays. Once the data generated by all pairs of subarrays were processed, an image that was optimized in terms of minimal feature splitting in all directions was generated. Further, based on the direction-specific SoS, a model-based method was used to directly reconstruct the SoS distribution.

RESULTS

Both phantom and animal experiments demonstrated feasibility and showed promising results compared with conventional methods, with less splitting and blurring and fewer distortions.

CONCLUSIONS

The developed MSFC method shows promising results for both IP and SoS reconstruction. The MSFC method will help to optimize the image quality of PACT in clinical applications.

Simultaneous realisation of zonal and modal wavefront sensing using programmable multiplexed grating patterns

In the present work, we propose a programmable multiplexed grating-based wavefront sensor (MGWS) to realise zonal and modal wavefront sensing approaches simultaneously. This is implemented by employing different bit-planes of a color image such that zonal wavefront sensing is performed with enhanced spatial resolution and modal wavefront sensing is performed to measure a large number of aberration modes present in the incident wavefront, simultaneously. We present proof-of-concept simulation results that demonstrate the working of the proposed MGWS and its ability to compensate for the presence of large number of aberration modes significantly, in comparison to either of the sensing approaches when used independently. Further, simulation results are included to quantify the same by considering an optical imaging system to image an array of two-dimensional bead objects. The proposed sensor is flexible in easy switching between either of the sensing approaches and the number of bit-planes can be increased conveniently to further improve the performance of the proposed MGWS.

Closed-loop wavefront sensing and correction in the mouse brain with computed optical coherence microscopy

Optical coherence microscopy (OCM) uses interferometric detection to capture the complex optical field with high sensitivity, which enables computational wavefront retrieval using back-scattered light from the sample. Compared to a conventional wavefront sensor, aberration sensing with OCM via computational adaptive optics (CAO) leverages coherence and confocal gating to obtain signals from the focus with less cross-talk from other depths or transverse locations within the field-of-view. Here, we present an investigation of the performance of CAO-based aberration sensing in simulation, bead phantoms, and ex vivo mouse brain tissue. We demonstrate that, due to the influence of the double-pass confocal OCM imaging geometry on the shape of computed pupil functions, computational sensing of high-order aberrations can suffer from signal attenuation in certain spatial-frequency bands and shape similarity with lower order counterparts. However, by sensing and correcting only low-order aberrations (astigmatism, coma, and trefoil), we still successfully corrected tissue-induced aberrations, leading to 3× increase in OCM signal intensity at a depth of ∼0.9 mm in a freshly dissected ex vivo mouse brain.

Adaptive photoacoustic computed tomography

For many optical imaging modalities, image qualities are inevitably degraded by wavefront distortions caused by varying light speed. In optical microscopy and astronomy, adaptive optics (AO) has long been applied to compensate for such unwanted aberrations. Photoacoustic computed tomography (PACT), despite relying on the ultrasonic wave for image formation, suffers from the acoustic version of the same problem. However, this problem has traditionally been regarded as an inverse problem of jointly reconstructing both the initial pressure and the sound speed distributions. In this work, we proposed a method similar to indirect wavefront sensing in AO. We argued that wavefront distortions can be extracted and corrected by a frequency domain analysis of local images. In addition to an adaptively reconstructed aberration-free image, the speed of sound map can be subsequently estimated. We demonstrated the method by in silico, phantom, and in vivo experiments.

Adaptive optics for high-resolution imaging

Adaptive optics (AO) is a technique that corrects for optical aberrations. It was originally proposed to correct for the blurring effect of atmospheric turbulence on images in ground-based telescopes and was instrumental in the work that resulted in the Nobel prize-winning discovery of a supermassive compact object at the centre of our galaxy. When AO is used to correct for the eye's imperfect optics, retinal changes at the cellular level can be detected, allowing us to study the operation of the visual system and to assess ocular health in the microscopic domain. By correcting for sample-induced blur in microscopy, AO has pushed the boundaries of imaging in thick tissue specimens, such as when observing neuronal processes in the brain. In this primer, we focus on the application of AO for high-resolution imaging in astronomy, vision science and microscopy. We begin with an overview of the general principles of AO and its main components, which include methods to measure the aberrations, devices for aberration correction, and how these components are linked in operation. We present results and applications from each field along with reproducibility considerations and limitations. Finally, we discuss future directions.

Multi-layer Shack-Hartmann wavefront sensing in the point source regime

The Shack-Hartmann wavefront sensor (SHWS) is often operated under the assumption that the sensed light can be described by a single wavefront. In biological tissues and other multi-layered samples, secondary wavefronts from axially and/or transversely displaced regions can lead to artifactual aberrations. Here, we evaluate these artifactual aberrations in a simulated ophthalmic SHWS by modeling the beacons that would be generated by a two-layer retina in human and mouse eyes. Then, we propose formulae for calculating a minimum SHWS centroid integration area to mitigate these aberrations by an order of magnitude, potentially benefiting SHWS-based metrology and adaptive optics systems such as those used for retinal imaging and microscopy.

Microscope-AOtools: a generalised adaptive optics implementation

Aberrations arising from sources such as sample heterogeneity and refractive index mismatches are constant problems in biological imaging. These aberrations reduce image quality and the achievable depth of imaging, particularly in super-resolution microscopy techniques. Adaptive optics (AO) technology has been proven to be effective in correcting for these aberrations, thereby improving the image quality. However, it has not been widely adopted by the biological imaging community due, in part, to difficulty in set-up and operation of AO. The methods for doing so are not novel or unknown, but new users often waste time and effort reimplementing existing methods for their specific set-ups, hardware, sample types, etc. Microscope-AOtools offers a robust, easy-to-use implementation of the essential methods for set-up and use of AO elements and techniques. These methods are constructed in a generalised manner that can utilise a range of adaptive optics elements, wavefront sensing techniques and sensorless AO correction methods. Furthermore, the methods are designed to be easily extensible as new techniques arise, leading to a streamlined pipeline for new AO technology and techniques to be adopted by the wider microscopy community.

Wavefront correction for adaptive optics with reflected light and deep neural networks

Light scattering and aberrations limit optical microscopy in biological tissue, which motivates the development of adaptive optics techniques. Here, we develop a method for wavefront correction in adaptive optics with reflected light and deep neural networks compatible with an epi-detection configuration. Large datasets of sample aberrations which consist of excitation and detection path aberrations as well as the corresponding reflected focus images are generated. These datasets are used for training deep neural networks. After training, these networks can disentangle and independently correct excitation and detection aberrations based on reflected light images recorded from scattering samples. A similar deep learning approach is also demonstrated with scattering guide stars. The predicted aberration corrections are validated using two photon imaging.

Improved model-based wavefront sensorless adaptive optics for extended objects using N + 2 images

The original model-based wavefront sensorless adaptive optics (WFSless AO) for extended objects uses the low spatial frequency content of images as the metric function and employs 2N + 1 images to correct N Lukosz aberration modes. We propose an improved method that uses the same metric but requires only N + 2 images to correct N aberration modes. The N + 2 method can achieve comparable corrective accuracy but requiring much smaller number of images compared with the 2N + 1 method. The N + 2 method is not only more efficient, but also provides the flexibility of choosing arbitrary basis modes by involving the non-orthogonality between modes in a linear least-squares optimization process. It is demonstrated that the deformable mirror's influence functions modes (IFM), orthogonal mirror modes (OMM) and fitted Zernike modes (FZM) all can be used as modal basis for the N + 2 method to generate biased images. It is also proved that the N + 2 method is robust to image sampling rate and image noise, showing good prospects of applications in various imaging systems.

Optical properties of electrically controlled arc-electrode liquid-crystal microlens array for wavefront measurement and adjustment

An electrically controlled arc-electrode liquid-crystal microlens array (AE-LCMLA), with tuning and swing focus, is proposed, which can be utilized to replace the traditional mechanically controlled microlenses and also cooperate with photosensitive arrays to solve the problems of measuring and further adjusting a strong distortion wavefront. The top patterned electrode of a single LC microlens is composed of three arc-electrodes distributed symmetrically around a central microhole for constructing the key controlling structures of the LC cavity in the AE-LCMLA. All the arc-electrodes are individually controlled, and then the focal spot of each microlens can be moved freely in a three-dimensional fashion including along the optical axial direction and over the focal plane by simply adjusting the driving signal voltage applied over each arc-electrode, independently. The featured performances of the AE-LCMLA in a wavelength range of ∼501-561  nm are the driving signal voltage being relatively low (less than ∼11  V_rms), the focal length tuning range being from ∼2.54  mm to ∼3.50  mm, the maximum focus swing distance being ∼52.92  μm, and the focus swing ratio K being ∼20‰.

Automated sensorless single-shot closed-loop adaptive optics microscopy with feedback from computational adaptive optics

Traditional wavefront-sensor-based adaptive optics (AO) techniques face numerous challenges that cause poor performance in scattering samples. Sensorless closed-loop AO techniques overcome these challenges by optimizing an image metric at different states of a deformable mirror (DM). This requires acquisition of a series of images continuously for optimization - an arduous task in dynamic in vivo samples. We present a technique where the different states of the DM are instead simulated using computational adaptive optics (CAO). The optimal wavefront is estimated by performing CAO on an initial volume to minimize an image metric, and then the pattern is translated to the DM. In this paper, we have demonstrated this technique on a spectral-domain optical coherence microscope for three applications: real-time depth-wise aberration correction, single-shot volumetric aberration correction, and extension of depth-of-focus. Our technique overcomes the disadvantages of sensor-based AO, reduces the number of image acquisitions compared to traditional sensorless AO, and retains the advantages of both computational and hardware-based AO.

IsoSense: frequency enhanced sensorless adaptive optics through structured illumination

We present IsoSense, a wavefront sensing method that mitigates sample dependency in image-based sensorless adaptive optics applications in microscopy. Our method employs structured illumination to create additional high spatial frequencies in the image through custom illumination patterns. This improves the reliability of image quality metric calculations and enables sensorless wavefront measurement even in samples with sparse spatial frequency content. We demonstrate the feasibility of IsoSense for aberration correction in a deformable-mirror-based structured illumination super-resolution fluorescence microscope.

Adaptive optics in spectroscopy and densely labeled-fluorescence applications

Adaptive optics systems have been integrated in many imaging modalities in order to correct for aberrations that are introduced by samples and optical elements. Usually, the optical system has access to a guide star (i.e., a point-like structure that is smaller than the diffraction limit). This guide star can be used as a beacon for adaptive optics enhancement. In contrast, for spectroscopy and densely-labeled fluorescent samples, the signal is diffused throughout the entire beam path and is not confined to a well-defined point-like structure. Here, we show analytically and experimentally that, in these scenarios, adaptive optics systems are expected to yield significantly lower signal enhancement than when a guide star is available. We discuss adaptive optics' performance degradation for different imaging modalities (e.g., confocal, multi-photon microscopy) and identify solutions to overcome low signal enhancements.

Depth of field extension and objective space depth measurement based on wavefront imaging

When all the parts of the wavefront imaging system are kept static after wavefront measuring, the target's images are blurry, because the depth of field (DOF) of the system affects the imaging quality. In this paper, the method for extending the DOF of the wavefront imaging system through an integrated architecture of a liquid-crystal microlens array (LCMLA) powered by electricity and a common photosensitive array, is presented. The DOF can be extended remarkably only by stitching together several sub-images of the LCMLA. The problem that the wavefronts and imaging results are insensitive to the objective depth is also solved. Optimal driving voltage signals are found out according to Sobel mean gradient to efficiently calibrate the depth of objective space in order to quantitatively measure the depth. The approach indicates a viable way to effectively extend the DOF of imaging micro-systems and to measure the geometrical depth of targets at the same time.

Combined hardware and computational optical wavefront correction

In many optical imaging applications, it is necessary to overcome aberrations to obtain high-resolution images. Aberration correction can be performed by either physically modifying the optical wavefront using hardware components, or by modifying the wavefront during image reconstruction using computational imaging. Here we address a longstanding issue in computational imaging: photons that are not collected cannot be corrected. This severely restricts the applications of computational wavefront correction. Additionally, performance limitations of hardware wavefront correction leave many aberrations uncorrected. We combine hardware and computational correction to address the shortcomings of each method. Coherent optical backscattering data is collected using high-speed optical coherence tomography, with aberrations corrected at the time of acquisition using a wavefront sensor and deformable mirror to maximize photon collection. Remaining aberrations are corrected by digitally modifying the coherently-measured wavefront during imaging reconstruction. This strategy obtains high-resolution images with improved signal-to-noise ratio of in vivo human photoreceptor cells with more complete correction of ocular aberrations, and increased flexibility to image at multiple retinal depths, field locations, and time points. While our approach is not restricted to retinal imaging, this application is one of the most challenging for computational imaging due to the large aberrations of the dilated pupil, time-varying aberrations, and unavoidable eye motion. In contrast with previous computational imaging work, we have imaged single photoreceptors and their waveguide modes in fully dilated eyes with a single acquisition. Combined hardware and computational wavefront correction improves the image sharpness of existing adaptive optics systems, and broadens the potential applications of computational imaging methods.

Wavefront measurement using computational adaptive optics

In many optical imaging applications, it is necessary to correct for aberrations to obtain high quality images. Optical coherence tomography (OCT) provides access to the amplitude and phase of the backscattered optical field for three-dimensional (3D) imaging samples. Computational adaptive optics (CAO) modifies the phase of the OCT data in the spatial frequency domain to correct optical aberrations without using a deformable mirror, as is commonly done in hardware-based adaptive optics (AO). This provides improvement of image quality throughout the 3D volume, enabling imaging across greater depth ranges and in highly aberrated samples. However, the CAO aberration correction has a complicated relation to the imaging pupil and is not a direct measurement of the pupil aberrations. Here we present new methods for recovering the wavefront aberrations directly from the OCT data without the use of hardware adaptive optics. This enables both computational measurement and correction of optical aberrations.

Adaptive optical microscopy for neurobiology

With the ability to correct for the aberrations introduced by biological specimens, adaptive optics-a method originally developed for astronomical telescopes-has been applied to optical microscopy to recover diffraction-limited imaging performance deep within living tissue. In particular, this technology has been used to improve image quality and provide a more accurate characterization of both structure and function of neurons in a variety of living organisms. Among its many highlights, adaptive optical microscopy has made it possible to image large volumes with diffraction-limited resolution in zebrafish larval brains, to resolve dendritic spines over 600μm deep in the mouse brain, and to more accurately characterize the orientation tuning properties of thalamic boutons in the primary visual cortex of awake mice.

Wavefront control in adaptive microscopy using Shack-Hartmann sensors with arbitrarily shaped pupils

In adaptive optical microscopy of thick biological tissue, strong scattering and aberrations can change the effective pupil shape by rendering some Shack-Hartmann spots unusable. The change of pupil shape leads to a change of wavefront reconstruction or control matrix that should be updated accordingly. Modified slope and modal wavefront control methods based on measurements of a Shack-Hartmann wavefront sensor are proposed to accommodate an arbitrarily shaped pupil. Furthermore, we present partial wavefront control methods that remove specific aberration modes like tip, tilt and defocus from the control loop. The proposed control methods were investigated and compared by simulation using experimentally obtained aberration data. The performance was then tested experimentally through closed-loop aberration corrections using an obscured pupil.

High-resolution imaging in two-photon excitation microscopy using in situ estimations of the point spread function

We present a technique for improving the spatial resolution of two-photon excitation microscopy; our technique combines annular illumination with an in situ estimation of the point spread function (PSF) used for deconvolution. For the in situ estimation of the PSF, we developed a technique called autocorrelation scanning, in which a sample is imaged by the scanning of two excitation foci that are overlapped over various distances. The image series obtained with the variation of the distance between the two foci provides the autocorrelation function of the PSF, which can be used to estimate the PSF at specific positions within a sample. We proved the principle and the effectiveness of this technique through observations of a fluorescent biological sample, and we confirmed that the improvement in the spatial resolution was ~1.7 times that of typical two-photon excitation microscopy by observing a mouse brain phantom at a depth of 200 µm.

Improving Focal Photostimulation of Cortical Neurons with Pre-derived Wavefront Correction

Recent progress in neuroscience to image and investigate brain function has been made possible by impressive developments in optogenetic and opto-molecular tools. Such research requires advances in optical techniques for the delivery of light through brain tissue with high spatial resolution. The tissue causes distortions to the wavefront of the incoming light which broadens the focus and consequently reduces the intensity and degrades the resolution. Such effects are detrimental in techniques requiring focal stimulation. Adaptive wavefront correction has been demonstrated to compensate for these distortions. However, iterative derivation of the corrective wavefront introduces time constraints that limit its applicability to probe living cells. Here, we demonstrate that we can pre-determine and generalize a small set of Zernike modes to correct for aberrations of the light propagating through specific brain regions. A priori identification of a corrective wavefront is a direct and fast technique that improves the quality of the focus without the need for iterative adaptive wavefront correction. We verify our technique by measuring the efficiency of two-photon photolysis of caged neurotransmitters along the dendrites of a whole-cell patched neuron. Our results show that encoding the selected Zernike modes on the excitation light can improve light propagation through brain slices of rats as observed by the neuron's evoked excitatory post-synaptic potential in response to localized focal uncaging at the spines of the neuron's dendrites.

Optimization-based wavefront sensorless adaptive optics for multiphoton microscopy

Optical aberrations have detrimental effects in multiphoton microscopy. These effects can be curtailed by implementing model-based wavefront sensorless adaptive optics, which only requires the addition of a wavefront shaping device, such as a deformable mirror (DM) to an existing microscope. The aberration correction is achieved by maximizing a suitable image quality metric. We implement a model-based aberration correction algorithm in a second-harmonic microscope. The tip, tilt, and defocus aberrations are removed from the basis functions used for the control of the DM, as these aberrations induce distortions in the acquired images. We compute the parameters of a quadratic polynomial that is used to model the image quality metric directly from experimental input-output measurements. Finally, we apply the aberration correction by maximizing the image quality metric using the least-squares estimate of the unknown aberration.

Snapshot coherence-gated direct wavefront sensing for multi-photon microscopy

Deep imaging in turbid media such as biological tissue is challenging due to scattering and optical aberrations. Adaptive optics has the potential to compensate the tissue aberrations. We present a wavefront sensing scheme for multi-photon scanning microscopes using the pulsed, near-infrared light reflected back from the sample utilising coherence gating and a confocal pinhole to isolate the light from a layer of interest. By interfering the back-reflected light with a tilted reference beam, we create a fringe pattern with a known spatial carrier frequency in an image of the back-aperture plane of the microscope objective. The wavefront aberrations distort this fringe pattern and thereby imprint themselves at the carrier frequency, which allows us to separate the aberrations in the Fourier domain from low spatial frequency noise. A Fourier analysis of the modulated fringes combined with a virtual Shack-Hartmann sensor for smoothing yields a modal representation of the wavefront suitable for correction. We show results with this method correcting both DM-induced and sample-induced aberrations in rat tail collagen fibres as well as a Hoechst-stained MCF-7 spheroid of cancer cells.

Wavefront sensorless adaptive optics optical coherence tomography for in vivo retinal imaging in mice

We present wavefront sensorless adaptive optics (WSAO) Fourier domain optical coherence tomography (FD-OCT) for in vivo small animal retinal imaging. WSAO is attractive especially for mouse retinal imaging because it simplifies optical design and eliminates the need for wavefront sensing, which is difficult in the small animal eye. GPU accelerated processing of the OCT data permitted real-time extraction of image quality metrics (intensity) for arbitrarily selected retinal layers to be optimized. Modal control of a commercially available segmented deformable mirror (IrisAO Inc.) provided rapid convergence using a sequential search algorithm. Image quality improvements with WSAO OCT are presented for both pigmented and albino mouse retinal data, acquired in vivo.