Adaptive optics confocal microscopy using direct wavefront sensing

Adaptive optics in an oblique plane microscope

Adaptive optics (AO) can restore diffraction-limited performance when imaging beyond superficial cell layers in vivo and in vitro, and as such, is of interest for advanced 3D microscopy methods such as light-sheet fluorescence microscopy (LSFM). In a typical LSFM system, the illumination and detection paths are separate and subject to different optical aberrations. To achieve optimal microscope performance, it is necessary to sense and correct these aberrations in both light paths, resulting in a complex microscope system. Here, we show that in an oblique plane microscope (OPM), a type of LSFM with a single primary objective lens, the same deformable mirror can correct both illumination and fluorescence detection. Besides reducing the complexity, we show that AO in OPM also restores the relative alignment of the light-sheet and focal plane, and that a projection imaging mode can stabilize and improve the wavefront correction in a sensorless AO format. We demonstrate OPM with AO on fluorescent nanospheres and by imaging the vasculature and cancer cells in zebrafish embryos embedded in a glass capillary, restoring diffraction limited resolution and improving the signal strength twofold.

Adaptive Optics in an Oblique Plane Microscope

Adaptive optics (AO) can restore diffraction limited performance when imaging beyond superficial cell layers in vivo and in vitro, and as such is of interest for advanced 3D microscopy methods such as light-sheet fluorescence microscopy (LSFM). In a typical LSFM system, the illumination and detection paths are separate and subject to different optical aberrations. To achieve optimal microscope performance, it is necessary to sense and correct these aberrations in both light paths, resulting in a complex microscope system. Here, we show that in an oblique plane microscope (OPM), a type of LSFM with a single primary objective lens, the same deformable mirror can correct both the illumination and fluorescence detection. Besides reducing the complexity, we show that AO in OPM also restores the relative alignment of the light-sheet and focal plane, and that a projection imaging mode can stabilize and improve the wavefront correction in a sensorless AO format. We demonstrate OPM with AO on fluorescent nanospheres and by imaging the vasculature and cancer cells in zebrafish embryos embedded in a glass capillary, restoring diffraction limited resolution and improving the signal strength twofold.

Geometric transformation adaptive optics (GTAO) for volumetric deep brain imaging through gradient-index lenses

The advance of genetic function indicators has enabled the observation of neuronal activities at single-cell resolutions. A major challenge for the applications on mammalian brains is the limited optical access depth. Currently, the method of choice to access deep brain structures is to insert miniature optical components. Among these validated miniature optics, the gradient-index (GRIN) lens has been widely employed for its compactness and simplicity. However, due to strong fourth-order astigmatism, GRIN lenses suffer from a small imaging field of view, which severely limits the measurement throughput and success rate. To overcome these challenges, we developed geometric transformation adaptive optics (GTAO), which enables adaptable achromatic large-volume correction through GRIN lenses. We demonstrate its major advances through in vivo structural and functional imaging of mouse brains. The results suggest that GTAO can serve as a versatile solution to enable large-volume recording of deep brain structures and activities through GRIN lenses.

Single-shot digital optical fluorescence phase conjugation through forward multiple-scattering samples

Aberrations and multiple scattering in biological tissues critically distort light beams into highly complex speckle patterns. In this regard, digital optical phase conjugation (DOPC) is a promising technique enabling in-depth focusing. However, DOPC becomes challenging when using fluorescent guide stars for four main reasons: the low photon budget available, the large spectral bandwidth of the fluorescent signal, the Stokes shift between the emission and the excitation wavelength, and the absence of reference beam preventing holographic measurement. Here, we demonstrate the possibility to focus a laser beam through multiple-scattering samples by measuring speckle fields in a single acquisition step with a reference-free, high-resolution wavefront sensor. By taking advantage of the large spectral bandwidth of forward multiply scattering samples, digital fluorescence phase conjugation is achieved to focus a laser beam at the excitation wavelength while measuring the broadband speckle field arising from a micrometer-sized fluorescent bead.

Phase diversity-based wavefront sensing for fluorescence microscopy

Fluorescence microscopy is an invaluable tool in biology, yet its performance is compromised when the wavefront of light is distorted due to optical imperfections or the refractile nature of the sample. Such optical aberrations can dramatically lower the information content of images by degrading image contrast, resolution, and signal. Adaptive optics (AO) methods can sense and subsequently cancel the aberrated wavefront, but are too complex, inefficient, slow, or expensive for routine adoption by most labs. Here we introduce a rapid, sensitive, and robust wavefront sensing scheme based on phase diversity, a method successfully deployed in astronomy but underused in microscopy. Our method enables accurate wavefront sensing to less than λ/35 root mean square (RMS) error with few measurements, and AO with no additional hardware besides a corrective element. After validating the method with simulations, we demonstrate calibration of a deformable mirror > 100-fold faster than comparable methods (corresponding to wavefront sensing on the ~100 ms scale), and sensing and subsequent correction of severe aberrations (RMS wavefront distortion exceeding λ/2), restoring diffraction-limited imaging on extended biological samples.

Adaptive Optics Microscopy with Wavefront Sensing Based on Neighbor Correlation

Complex structures in living cells and tissues induce wavefront errors when light waves pass through them, and images observed with optical microscopes are undesirably blurred. This problem is especially serious for living plant cells because images are strikingly degraded even within a single cell. Adaptive optics (AO) is expected to be a solution to this problem by correcting such wavefront errors, thus enabling high-resolution imaging. In particular, scene-based AO involves wavefront sensing based on the image correlation between subapertures in a Shack-Hartmann wavefront sensor and thus does not require an intense point light source. However, the complex 3D structures of living cells often cause low correlation between subimages, leading to loss of accuracy in wavefront sensing. This paper proposes a novel method for scene-based sensing using only image correlations between adjacent subapertures. The method can minimize changes between subimages to be correlated and thus prevent inaccuracy in phase estimation. Using an artificial test target mimicking the optical properties of a layer of living plant cells, an imaging performance with a Strehl ratio of approximately 0.5 was confirmed. Upon observation of chloroplast autofluorescence inside living leaf cells of the moss Physcomitrium patens, recovered resolution images were successfully obtained even with complex biological structures. Under bright-field illumination, the proposed method outperformed the conventional method, demonstrating the future potential of this method for label- and damage-free AO microscopy. Several points for improvement in terms of the effect of AO correction are discussed.

Tandem aberration correction optics (TACO) in wide-field structured illumination microscopy

Structured illumination microscopy (SIM) is a powerful super-resolution imaging technique that uses patterned illumination to down-modulate high spatial-frequency information of samples. However, the presence of spatially-dependent aberrations can severely disrupt the illumination pattern, limiting the quality of SIM imaging. Conventional adaptive optics (AO) techniques that employ wavefront correctors at the pupil plane are not capable of effectively correcting these spatially-dependent aberrations. We introduce the Tandem Aberration Correction Optics (TACO) approach that combines both pupil AO and conjugate AO for aberration correction in SIM. TACO incorporates a deformable mirror (DM) for pupil AO in the detection path to correct for global aberrations, while a spatial light modulator (SLM) is placed at the plane conjugate to the aberration source near the sample plane, termed conjugate AO, to compensate spatially-varying aberrations in the illumination path. Our numerical simulations and experimental results show that the TACO approach can recover the illumination pattern close to an ideal condition, even when severely misshaped by aberrations, resulting in high-quality super-resolution SIM reconstruction. The TACO approach resolves a critical traditional shortcoming of aberration correction for structured illumination. This advance significantly expands the application of SIM imaging in the study of complex, particularly biological, samples and should be effective in other wide-field microscopies.

Investigation of PZT Materials for Reliable Piezostack Deformable Mirror with Modular Design

This article presents a study of the electrophysical properties of a piezoceramic material for use in adaptive optics. The key characteristics that may be important for the manufacturing of piezoelectric deformable mirrors are the following: piezoelectric constants (d31, d33, d15), capacitance, elastic compliance values s for different crystal directions, and the dielectric loss tangent (tgδ). Based on PZT ceramics, the PKP-12 material was developed with high values of the dielectric constant, piezoelectric modulus, and electromechanical coupling coefficients. The deformable mirror control elements are made from the resulting material-piezoceramic combs with five individual actuators in a row. In this case, the stroke of the actuator is in the range of 4.1-4.3 microns and the capacitance of the actuator is about 12 nF.

AlN-based piezoelectric MEMS deformable mirror

An AlN-based piezoelectric micro-electromechanical system (MEMS) continuous membrane deformable mirror (DM) prototype is presented for the first time. Its effective aperture diameter is 5 mm and it is equipped with 25 independently controlled actuators. Owing to the advantages associated with the AlN piezoelectric thin-film technology, attractive characteristics including CMOS compatible fabrication, bidirectional linear and negligible hysteresis actuation, and excellent linear superposition control capability have been successfully demonstrated. Moreover, good optical aberration correction performance is also validated via the surface contour fitting experiment to the Zernike polynomials up to the first 14 orders despite the non-optimized device structure design, representing great application perspective.

Efficient framework of solving time-gated reflection matrix for imaging through turbid medium

Imaging through turbid medium is a long pursuit in many research fields, such as biomedicine, astronomy and automatic vehicle, in which the reflection matrix-based method is a promising solution. However, the epi-detection geometry suffers from round-trip distortion and it is challenging to isolate the input and output aberrations in non-ideal cases due to system imperfections and measurement noises. Here, we present an efficient framework based on single scattering accumulation together with phase unwrapping that can accurately separate input and output aberrations from the noise-affected reflection matrix. We propose to only correct the output aberration while suppressing the input aberration by incoherent averaging. The proposed method is faster in convergence and more robust against noise, avoiding precise and tedious system adjustments. In both simulations and experiments, we demonstrate the diffraction-limited resolution capability under optical thickness beyond 10 scattering mean free paths, showing the potential of applications in neuroscience and dermatology.

Adaptive optics for optical microscopy [Invited]

Optical microscopy is widely used to visualize fine structures. When applied to bioimaging, its performance is often degraded by sample-induced aberrations. In recent years, adaptive optics (AO), originally developed to correct for atmosphere-associated aberrations, has been applied to a wide range of microscopy modalities, enabling high- or super-resolution imaging of biological structure and function in complex tissues. Here, we review classic and recently developed AO techniques and their applications in optical microscopy.

Recent advances in optical imaging through deep tissue: imaging probes and techniques

Optical imaging has been essential for scientific observations to date, however its biomedical applications has been restricted due to its poor penetration through tissues. In living tissue, signal attenuation and limited imaging depth caused by the wave distortion occur because of scattering and absorption of light by various molecules including hemoglobin, pigments, and water. To overcome this, methodologies have been proposed in the various fields, which can be mainly categorized into two stategies: developing new imaging probes and optical techniques. For example, imaging probes with long wavelength like NIR-II region are advantageous in tissue penetration. Bioluminescence and chemiluminescence can generate light without excitation, minimizing background signals. Afterglow imaging also has high a signal-to-background ratio because excitation light is off during imaging. Methodologies of adaptive optics (AO) and studies of complex media have been established and have produced various techniques such as direct wavefront sensing to rapidly measure and correct the wave distortion and indirect wavefront sensing involving modal and zonal methods to correct complex aberrations. Matrix-based approaches have been used to correct the high-order optical modes by numerical post-processing without any hardware feedback. These newly developed imaging probes and optical techniques enable successful optical imaging through deep tissue. In this review, we discuss recent advances for multi-scale optical imaging within deep tissue, which can provide reseachers multi-disciplinary understanding and broad perspectives in diverse fields including biophotonics for the purpose of translational medicine and convergence science.

Improved photoacoustic images via wavefront shaping modulation based on the scattering structure

Multispectral optoacoustic tomography (MSOT) has become the dominant technical solution for photoacoustic imaging (PAI). However, the laser source of fiber output in the current MSOT method is typically a TEM00 Gaussian beam, which is prone to artifacts and incomplete due to the uneven distribution of the irradiated light intensity. Here, we propose a novel method to improve the quality of photoacoustic image reconstruction by modulating the wavefront shaping of the incident laser beam based on the designed scattering structure. In the experiment, we add the designed scattering structure to the current hemispherical photoacoustic transducer array device. Through experiments and simulations, we investigate and compare the effects of different scattering structures on laser intensity modulation. The results show that an ED1-C20 diffusion structure with a scattering angle of 20 degrees has the most effective modulation of the beam intensity distribution. And we choose gold nanoparticles of 50-100 nanometers (nm) diameters and index finger capillary vessels respectively as the medium of PAI. We obtain the highest ratio of PAI area increases of gold nanoparticles and index finger to devices compare without scattering structure is 29.69% and 634.94%, respectively. Experimental results demonstrate that our method is significantly higher quality than traditional methods, which has great potential for theoretical application in medical PAI.

Through-skull brain imaging in vivo at visible wavelengths via dimensionality reduction adaptive-optical microscopy

Compensation of sample-induced optical aberrations is crucial for visualizing microscopic structures deep within biological tissues. However, strong multiple scattering poses a fundamental limitation for identifying and correcting the tissue-induced aberrations. Here, we introduce a label-free deep-tissue imaging technique termed dimensionality reduction adaptive-optical microscopy (DReAM) to selectively attenuate multiple scattering. We established a theoretical framework in which dimensionality reduction of a time-gated reflection matrix can attenuate uncorrelated multiple scattering while retaining a single-scattering signal with a strong wave correlation, irrespective of sample-induced aberrations. We performed mouse brain imaging in vivo through the intact skull with the probe beam at visible wavelengths. Despite the strong scattering and aberrations, DReAM offered a 17-fold enhancement of single scattering-to-multiple scattering ratio and provided high-contrast images of neural fibers in the brain cortex with the diffraction-limited spatial resolution of 412 nanometers and a 33-fold enhanced Strehl ratio.

Multi-segmented feature coupling for jointly reconstructing initial pressure and speed of sound in photoacoustic computed tomography

SIGNIFICANCE

Photoacoustic computed tomography (PACT) is a fast-growing imaging modality. In PACT, the image quality is degraded due to the unknown distribution of the speed of sound (SoS). Emerging initial pressure (IP) and SoS joint-reconstruction methods promise reduced artifacts in PACT. However, previous joint-reconstruction methods have some deficiencies. A more effective method has promising prospects in preclinical applications.

AIM

We propose a multi-segmented feature coupling (MSFC) method for SoS-IP joint reconstruction in PACT.

APPROACH

In the proposed method, the ultrasound detectors were divided into multiple sub-arrays with each sub-array and its opposite counterpart considered to be a pair. The delay and sum algorithm was then used to reconstruct two images based on a subarray pair and estimated a direction-specific SoS, based on image correlation and the orientation of the subarrays. Once the data generated by all pairs of subarrays were processed, an image that was optimized in terms of minimal feature splitting in all directions was generated. Further, based on the direction-specific SoS, a model-based method was used to directly reconstruct the SoS distribution.

RESULTS

Both phantom and animal experiments demonstrated feasibility and showed promising results compared with conventional methods, with less splitting and blurring and fewer distortions.

CONCLUSIONS

The developed MSFC method shows promising results for both IP and SoS reconstruction. The MSFC method will help to optimize the image quality of PACT in clinical applications.

Active Aberration Correction with Adaptive Coefficient SPGD Algorithm for Laser Scanning Confocal Microscope

A laser scanning confocal microscope (LSCM) is an effective scientific instrument for studying sub-micron structures, and it has been widely used in the field of biological detection. However, the illumination depth of LSCMs is limited due to the optical aberrations introduced by living biological tissue, which acts as an optical medium with a non-uniform refractive index, resulting in a significant dispersion of the focus of LSCM illumination light and, hence, a loss in the resolution of the image. In this study, to minimize the effect of optical aberrations, an image-based adaptive optics technology using an optimized stochastic parallel gradient descent (SPGD) algorithm with an adaptive coefficient is applied to the optical path of an LSCM system. The effectiveness of the proposed aberration correction approach is experimentally evaluated in the LSCM system. The results illustrate that the proposed adaptive optics system with an adaptive coefficient SPGD algorithm can effectively reduce the interference caused by aberrations during depth imaging.

Enabling single-molecule localization microscopy in turbid food emulsions

Turbidity poses a major challenge for the microscopic characterization of food systems. Local mismatches in refractive indices, for example, lead to significant image deterioration along sample depth. To mitigate the issue of turbidity and to increase the accessible optical resolution in food microscopy, we added adaptive optics (AO) and flat-field illumination to our previously published open microscopy framework, the miCube. In the detection path, we implemented AO via a deformable mirror to compensate aberrations and to modulate the emission wavefront enabling the engineering of point spread functions (PSFs) for single-molecule localization microscopy (SMLM) in three dimensions. As a model system for a non-transparent food colloid such as mayonnaise, we designed an oil-in-water emulsion containing the ferric ion binding protein phosvitin commonly present in egg yolk. We targeted phosvitin with fluorescently labelled primary antibodies and used PSF engineering to obtain two- and three-dimensional images of phosvitin covered oil droplets with sub 100 nm resolution. Our data indicated that phosvitin is homogeneously distributed at the interface. With the possibility to obtain super-resolved images in depth, our work paves the way for localizing biomacromolecules at heterogeneous colloidal interfaces in food emulsions.

This article is part of the Theo Murphy meeting issue ‘Super-resolution structured illumination microscopy (part 2)’.

High-accuracy, direct aberration determination using self-attention-armed deep convolutional neural networks

Optical microscopes have long been essential for many scientific disciplines. However, the resolution and contrast of such microscopic images are dramatically affected by aberrations. In this study, compacted with adaptive optics, we propose a machine learning technique, called the "phase-retrieval deep convolutional neural networks (PRDCNNs)". This aberration determination architecture is direct and exhibits high accuracy and certain generalization ability. Notably, its performance surpasses those of similar, existing methods, with fewer fluctuations and greater robustness against noise. We anticipate future application of the proposed PRDCNNs to super-resolution microscopes. This article is protected by copyright. All rights reserved.

Moving beyond velocity: Opportunities and challenges to quantify immune cell behavior

The analysis of cellular behavior using intravital multi-photon microscopy has contributed substantially to our understanding of the priming and effector phases of immune responses. Yet, many questions remain unanswered and unexplored. Though advancements in intravital imaging techniques and animal models continue to drive new discoveries, continued improvements in analysis methods are needed to extract detailed information about cellular behavior. Focusing on dendritic cell (DC) and T cell interactions as an exemplar, here we discuss key limitations for intravital imaging studies and review and explore alternative approaches to quantify immune cell behavior. We touch upon current developments in deep learning models, as well as established methods from unrelated fields such as ecology to detect and track objects over time. As developments in open-source software make it possible to process and interactively view larger datasets, the challenge for the field will be to determine how best to combine intravital imaging with multi-parameter imaging of larger tissue regions to discover new facets of leukocyte dynamics and how these contribute to immune responses.

Extended range and aberration-free autofocusing via remote focusing and sequence-dependent learning

Rapid autofocusing over long distances is critical for tracking 3D topological variations and sample motion in real time. Taking advantage of a deformable mirror and Shack-Hartmann wavefront sensor, remote focusing can permit fast axial scanning with simultaneous correction of system-induced aberrations. Here, we report an autofocusing technique that combines remote focusing with sequence-dependent learning via a bidirectional long short term memory network. A 120 µm autofocusing range was achieved in a compact reflectance confocal microscope both in air and in refractive-index-mismatched media, with similar performance under arbitrary-thickness liquid layers up to 1 mm. The technique was validated on sample types not used for network training, as well as for tracking of continuous axial motion. These results demonstrate that the proposed technique is suitable for real-time aberration-free autofocusing over a large axial range, and provides unique advantages for biomedical, holographic and other related applications.

COACH-based Shack-Hartmann wavefront sensor with an array of phase coded masks

Shack-Hartmann wavefront sensors (SHWS) are generally used to measure the wavefront shape of light beams. Measurement accuracy and the sensitivity of these sensors are important factors for better wavefront sensing. In this paper, we demonstrate a new type of SHWS with better measurement accuracy than the regular SHWS. The lenslet array in the regular SHWS is replaced with an array of coded phase masks, and the principle of coded aperture correlation holography (COACH) is used for wavefront reconstruction. Sharper correlation peaks achieved by COACH improve the accuracy of the estimated local slopes of the measured wavefront and consequently improve the reconstruction accuracy of the overall wavefront. Experimental results confirm that the proposed method provides a lower mean square wavefront error by one order of magnitude in comparison to the regular SHWS.

Visualizing the Invisible: Advanced Optical Microscopy as a Tool to Measure Biomechanical Forces

The importance of mechanical force in biology is evident across diverse length scales, ranging from tissue morphogenesis during embryo development to mechanotransduction across single adhesion proteins at the cell surface. Consequently, many force measurement techniques rely on optical microscopy to measure forces being applied by cells on their environment, to visualize specimen deformations due to external forces, or even to directly apply a physical perturbation to the sample via photoablation or optogenetic tools. Recent developments in advanced microscopy offer improved approaches to enhance spatiotemporal resolution, imaging depth, and sample viability. These advances can be coupled with already existing force measurement methods to improve sensitivity, duration and speed, amongst other parameters. However, gaining access to advanced microscopy instrumentation and the expertise necessary to extract meaningful insights from these techniques is an unavoidable hurdle. In this Live Cell Imaging special issue Review, we survey common microscopy-based force measurement techniques and examine how they can be bolstered by emerging microscopy methods. We further explore challenges related to the accompanying data analysis in biomechanical studies and discuss the various resources available to tackle the global issue of technology dissemination, an important avenue for biologists to gain access to pre-commercial instruments that can be leveraged for biomechanical studies.

2.5D microscopy with polarization independent SLM for enhanced detection efficiency and aberration correction

Fast, volumetric imaging by fluorescence microscopy is essential in studying biological phenomena and cellular functions. Recently, single-shot 2.5D microscopy showed promising results for high-throughput quantitative subcellular analysis via extended depth of field imaging without sequential z-scanning; however, the detection efficiency was limited and it lacked depth-induced aberration correction. Here we report that a spatial light modulator (SLM) in a polarization insensitive configuration can significantly improve the detection efficiency of 2.5D microscopy, while also compensating for aberrations at large imaging depths caused by the refractive index mismatch between the sample and the immersion medium. We highlight the improved efficiency via quantitative single-molecule RNA imaging of mammalian cells with a 2-fold improvement in the fluorescence intensity compared to a conventional SLM-based microscopy. We demonstrate the aberration correction capabilities and extended depth of field by imaging thick specimens with fewer z-scanning steps.

Pupil-aberration calibration with controlled illumination for quantitative phase imaging

Quantitative phase imaging (QPI) exploits sample-induced changes in the optical field to analyze biological specimens in a label-free manner. However, the quantitative nature of QPI makes it susceptible to optical aberrations. We propose a method for calibrating pupil aberrations by imaging a sample of interest. The proposed method recovers pupil information by utilizing the cross-spectral density between optical fields at different incident angles and allows both thin and weakly scattering three-dimensional samples for calibration. We experimentally validate the proposed method by imaging various samples, including a resolution target, breast tissue, and a polystyrene bead, and demonstrate aberration-free two- and three-dimensional QPI.

Subcellular three-dimensional imaging deep through multicellular thick samples by structured illumination microscopy and adaptive optics

Structured Illumination Microscopy enables live imaging with sub-diffraction resolution. Unfortunately, optical aberrations can lead to loss of resolution and artifacts in Structured Illumination Microscopy rendering the technique unusable in samples thicker than a single cell. Here we report on the combination of Adaptive Optics and Structured Illumination Microscopy enabling imaging with 150 nm lateral and 570 nm axial resolution at a depth of 80 µm through Caenorhabditis elegans. We demonstrate that Adaptive Optics improves the three-dimensional resolution, especially along the axial direction, and reduces artifacts, successfully realizing 3D-Structured Illumination Microscopy in a variety of biological samples.

Single cell capture, isolation, and long-term in-situ imaging using quantitative self-interference spectroscopy

Single cell research with microfluidic chip is of vital importance in biomedical studies and clinical medicine. Simultaneous microfluidic cell manipulations and long-term cell monitoring needs further investigations due to the lack of label-free quantitative imaging techniques and systems. In this work, single cell capture, isolation and long-term in-situ monitoring was realized with a microfluidic cell chip, compact cell incubator and quantitative self-interference spectroscopy. The proposed imaging method could obtain quantitative and dynamic refractive index distribution in living cells. And the designed microfluidic chip could capture and isolate single cells. The customized incubator could support cell growth conditions when single cell was captured in microfluidic chip. According to the results, single cells could be trapped, transferred and pushed into the culture chamber with the microfluidic chip. The incubator could culture single cells in the chip for 120 h. The refractive index sensitivity of the proposed quantitative imaging method was 0.0282 and the relative error was merely 0.04%.

Adaptive photoacoustic computed tomography

For many optical imaging modalities, image qualities are inevitably degraded by wavefront distortions caused by varying light speed. In optical microscopy and astronomy, adaptive optics (AO) has long been applied to compensate for such unwanted aberrations. Photoacoustic computed tomography (PACT), despite relying on the ultrasonic wave for image formation, suffers from the acoustic version of the same problem. However, this problem has traditionally been regarded as an inverse problem of jointly reconstructing both the initial pressure and the sound speed distributions. In this work, we proposed a method similar to indirect wavefront sensing in AO. We argued that wavefront distortions can be extracted and corrected by a frequency domain analysis of local images. In addition to an adaptively reconstructed aberration-free image, the speed of sound map can be subsequently estimated. We demonstrated the method by in silico, phantom, and in vivo experiments.

Multi-layer Shack-Hartmann wavefront sensing in the point source regime

The Shack-Hartmann wavefront sensor (SHWS) is often operated under the assumption that the sensed light can be described by a single wavefront. In biological tissues and other multi-layered samples, secondary wavefronts from axially and/or transversely displaced regions can lead to artifactual aberrations. Here, we evaluate these artifactual aberrations in a simulated ophthalmic SHWS by modeling the beacons that would be generated by a two-layer retina in human and mouse eyes. Then, we propose formulae for calculating a minimum SHWS centroid integration area to mitigate these aberrations by an order of magnitude, potentially benefiting SHWS-based metrology and adaptive optics systems such as those used for retinal imaging and microscopy.

Imaging performance of microscopy adaptive-optics system using scene-based wavefront sensing

SIGNIFICANCE

A scene-based adaptive-optics (AO) system is developed and a method for investigating its imaging performance is proposed. The system enables derivation of Strehl ratios from observed images via collaboration with computer simulations. The resultant Strehl ratios are comparable with those of other current AO systems.

AIM

For versatile and noninvasive AO microscopy, a scene-based wavefront-sensing technique working on a Shack-Hartmann wavefront sensor is developed in a modal control system. The purpose of the research is to clarify the imaging performance of the AO system via the derivation of Strehl ratios from observed images toward applications in microscopy of living cells and tissues.

APPROACH

Two imaging metrics that can be directly measured from observed images (i.e., an energy concentration ratio and unbiased maximum ratio) are defined and related to the Strehl ratio via computer simulations. Experiments are conducted using artificial targets to measure the imaging metrics, which are then converted to Strehl ratios.

RESULTS

The resultant Strehl ratios are >0.7 and 0.5 in the cases of defocus and higher aberrations, respectively. The half-widths at half-maximum of the AO-corrected bead images are favorably comparable to those of on-focus images under simple defocus aberration, and the AO system works both under bright-field illumination and on fluorescent bead images.

CONCLUSIONS

The proposed scene-based AO system is expected to work with a Strehl ratio of more than 0.5 when applied to high-resolution live imaging of cells and tissues under bright-field and fluorescence microscopies.

Aberration-corrected three-dimensional non-inertial scanning for femtosecond lasers

Large aberrations are induced by non-collimated light when the convergence or divergence of the incident beam on the back-pupil plane of the objective lens is adjusted for 3D non-inertial scanning. These aberrations significantly degrade the focus quality and decrease the peak intensity of the femtosecond laser focal spot. Here, we describe an aberration-corrected 3D non-inertial scanning method for femtosecond lasers based on a digital micromirror device (DMD) that is used for both beam scanning and aberration correction. An imaging setup is used to detect the focal spot in the 3D space, and an iterative optimization algorithm is used to optimize the focal spot. We demonstrate the application of our proposed approach in two-photon imaging. With correction for the 200-µm out-of-focal plane, the optical axial resolution improves from 7.67 to 3.25 µm, and the intensity of the fluorescence signal exhibits an almost fivefold improvement when a 40× objective lens is used. This aberration-corrected 3D non-inertial scanning method for femtosecond lasers offers a new approach for a variety of potential applications, including nonlinear optical imaging, microfabrication, and optical storage.

Microscope-AOtools: a generalised adaptive optics implementation

Aberrations arising from sources such as sample heterogeneity and refractive index mismatches are constant problems in biological imaging. These aberrations reduce image quality and the achievable depth of imaging, particularly in super-resolution microscopy techniques. Adaptive optics (AO) technology has been proven to be effective in correcting for these aberrations, thereby improving the image quality. However, it has not been widely adopted by the biological imaging community due, in part, to difficulty in set-up and operation of AO. The methods for doing so are not novel or unknown, but new users often waste time and effort reimplementing existing methods for their specific set-ups, hardware, sample types, etc. Microscope-AOtools offers a robust, easy-to-use implementation of the essential methods for set-up and use of AO elements and techniques. These methods are constructed in a generalised manner that can utilise a range of adaptive optics elements, wavefront sensing techniques and sensorless AO correction methods. Furthermore, the methods are designed to be easily extensible as new techniques arise, leading to a streamlined pipeline for new AO technology and techniques to be adopted by the wider microscopy community.

Deep learning assisted Shack-Hartmann wavefront sensor for direct wavefront detection

The conventional Shack-Hartmann wavefront sensor (SHWS) requires wavefront slope measurements of every micro-lens for wavefront reconstruction. In this Letter, we applied deep learning on the SHWS to directly predict the wavefront distributions without wavefront slope measurements. The results show that our method could provide a lower root mean square wavefront error in high detection speed. The performance of the proposed method is also evaluated on challenging wavefronts, while the conventional approaches perform insufficiently. This Letter provides a new approach, to the best of our knowledge, to perform direct wavefront detection in SHWS-based applications.

Distortion matrix concept for deep optical imaging in scattering media

In optical imaging, light propagation is affected by the inhomogeneities of the medium. Sample-induced aberrations and multiple scattering can strongly degrade the image resolution and contrast. On the basis of a dynamic correction of the incident and/or reflected wavefronts, adaptive optics has been used to compensate for those aberrations. However, it only applies to spatially invariant aberrations or to thin aberrating layers. Here, we propose a global and noninvasive approach based on the distortion matrix concept. This matrix basically connects any focusing point of the image with the distorted part of its wavefront in reflection. A singular value decomposition of the distortion matrix allows to correct for high-order aberrations and forward multiple scattering over multiple isoplanatic modes. Proof-of-concept experiments are performed through biological tissues including a turbid cornea. We demonstrate a Strehl ratio enhancement up to 2500 and recover a diffraction-limited resolution until a depth of 10 scattering mean free paths.

Conformal convolutional neural network (CCNN) for single-shot sensorless wavefront sensing

Wavefront sensing technique is essential in deep tissue imaging, which guides spatial light modulator to compensate wavefront distortion for better imaging quality. Recently, convolutional neural network (CNN) based sensorless wavefront sensing methods have achieved remarkable speed advantages via single-shot measurement methodology. However, the low efficiency of convolutional filters dealing with circular point-spread-function (PSF) features makes them less accurate. In this paper, we propose a conformal convolutional neural network (CCNN) that boosts the performance by pre-processing circular features into rectangular ones through conformal mapping. The proposed conformal mapping reduces the number of convolutional filters that need to describe a circular feature, thus enables the neural network to recognize PSF features more efficiently. We demonstrate our CCNN could improve the wavefront sensing accuracy over 15% compared to a traditional CNN through simulations and validate the accuracy improvement in experiments. The improved performances make the proposed method promising in high-speed deep tissue imaging.

Direct determination of aberration functions in microscopy by an artificial neural network

Adaptive optics relies on the fast and accurate determination of aberrations but is often hindered by wavefront sensor limitations or lengthy optimization algorithms. Deep learning by artificial neural networks has recently been shown to provide determination of aberration coefficients from various microscope metrics. Here we numerically investigate the direct determination of aberration functions in the pupil plane of a high numerical aperture microscope using an artificial neural network. We show that an aberration function can be determined from fluorescent guide stars and used to improve the Strehl ratio without the need for reconstruction from Zernike polynomial coefficients.

Extended field-of-view adaptive optics in microscopy via numerical field segmentation

Sample-induced optical aberrations in microscopy are, in general, field dependent, limiting their correction via pupil adaptive optics (AO) to the center of the available field-of-view (FoV). This is a major hindrance, particularly for deep tissue imaging, where AO has a significant impact. We present a new wide-field AO microscopy scheme, in which the deformable element is located at the pupil plane of the objective. To maintain high-quality correction across its entirety, the FoV is partitioned into small segments, and a separate aberration estimation is performed for each via a modal-decomposition-based indirect wavefront sensing algorithm. A final full-field image is synthesized by stitching of the partitions corrected consecutively and independently via their respective measured aberrations. The performance and limitations of the method are experimentally explored on synthetic samples imaged via a custom-developed AO fluorescence microscope featuring an optofluidic refractive wavefront modulator.

Intensity-enhanced deep network wavefront reconstruction in Shack-Hartmann sensors

The Shack-Hartmann wavefront sensor (SH-WFS) is known to produce incorrect measurements of the wavefront gradient in the presence of non-uniform illumination. Moreover, the most common least-squares phase reconstructors cannot accurately reconstruct the wavefront in the presence of branch points. We therefore developed the intensity/slopes network (ISNet), a deep convolutional-neural-network-based reconstructor that uses both the wavefront gradient information and the intensity of the SH-WFS's subapertures to provide better wavefront reconstruction. We trained the network on simulated data with multiple levels of turbulence and compared the performance of our reconstructor to several other reconstruction techniques. ISNet produced the lowest wavefront error of the reconstructors we evaluated and operated at a speed suitable for real-time applications, enabling the use of the SH-WFS in stronger turbulence than was previously possible.

Dynamic conjugate F-SHARP microscopy

Optical microscopy is an indispensable tool in biomedical sciences, but its reach in deep tissues is limited due to aberrations and scattering. This problem can be overcome by wavefront-shaping techniques, albeit at limited fields of view (FOVs). Inspired by astronomical imaging, conjugate wavefront shaping can lead to an increased field of view in microscopy, but this correction is limited to a set depth and cannot be dynamically adapted. Here, we present a conjugate wavefront-shaping scheme based on focus scanning holographic aberration probing (F-SHARP). We combine it with a compact implementation that can be readily adapted to a variety of commercial and home-built two-photon microscopes. We demonstrate the power of the method by imaging with high resolution over extended FOV (>80 µm) deeper than 400 μm inside a mouse brain through a thinned skull.

Learning-based Shack-Hartmann wavefront sensor for high-order aberration detection

We present a learning-based Shack-Hartmann wavefront sensor (SHWS) to achieve the high-order aberration detection without image segmentation or centroid positioning. Zernike coefficient amplitudes of aberrations measured from biological samples are referred and expanded to generate the training datasets. With one SHWS pattern inputted, up to 120^th Zernike modes could be predicted within 10.9 ms with 95.56% model accuracy by a personal computer. The statistical experimental results show that compared with traditional modal-based SHWS, the root mean squared error in phase residuals of this method is reduced by ∼40.54% and the Strehl ratio of the point spread functions is improved by ∼27.31%. The aberration detection performance of this method is also validated on a mouse brain slice with 300 µm thickness and the median improvement of peak-to-background ratio of this method is ∼30% to 40% compared with traditional SHWS. With the high detection accuracy, simple processes, fast prediction speed and good compatibility, this work offers a potential approach to improve the wavefront sensing ability of SHWS, which could be combined with an existing adaptive optics system and be further applied in biological applications.

Imaging, Visualization, and Computation in Developmental Biology

Embryonic development is highly complex and dynamic, requiring the coordination of numerous molecular and cellular events at precise times and places. Advances in imaging technology have made it possible to follow developmental processes at cellular, tissue, and organ levels over time as they take place in the intact embryo. Parallel innovations of in vivo probes permit imaging to report on molecular, physiological, and anatomical events of embryogenesis, but the resulting multidimensional data sets pose significant challenges for extracting knowledge. In this review, we discuss recent and emerging advances in imaging technologies, in vivo labeling, and data processing that offer the greatest potential for jointly deciphering the intricate cellular dynamics and the underlying molecular mechanisms. Our discussion of the emerging area of “image-omics” highlights both the challenges of data analysis and the promise of more fully embracing computation and data science for rapidly advancing our understanding of biology.

Highly Sensitive Shack-Hartmann Wavefront Sensor: Application to Non-Transparent Tissue Mimic Imaging with Adaptive Light-Sheet Fluorescence Microscopy

High-quality in-depth imaging of three-dimensional samples remains a major challenge in modern microscopy. Selective plane illumination microscopy (SPIM) is a widely used technique that enables imaging of living tissues with subcellular resolution. However, scattering, absorption, and optical aberrations limit the depth at which useful imaging can be done. Adaptive optics (AOs) is a method capable of measuring and correcting aberrations in different kinds of fluorescence microscopes, thereby improving the performance of the optical system. We have incorporated a wavefront sensor adaptive optics scheme to SPIM (_WAOSPIM) to correct aberrations induced by optically-thick samples, such as multi-cellular tumor spheroids (MCTS). Two-photon fluorescence provides us with a tool to produce a weak non-linear guide star (NGS) in any region of the field of view. The faintness of NGS; however, led us to develop a high-sensitivity Shack–Hartmann wavefront sensor (SHWS). This paper describes this newly developed SHWS and shows the correction capabilities of _WAOSPIM using NGS in thick, inhomogeneous samples like MCTS. We report improvements of up to 79% for spatial frequencies corresponding to cellular and subcellular size features.

Direct wavefront sensing enables functional imaging of infragranular axons and spines

We advance two-photon microscopy for near-diffraction-limited imaging up to 850 µm below the pia in awake mice. Our approach combines direct wavefront sensing of light from a guidestar (formed by descanned fluorescence from Cy5.5-conjugated dextran in brain microvessels) with adaptive optics to compensate for tissue-induced aberrations in the wavefront. We achieve high signal-to-noise ratios in recordings of glutamate release from thalamocortical axons and calcium transients in spines of layer 5b basal dendrites during active tactile sensing.

Label-free tomography of living cellular nanoarchitecture using hyperspectral self-interference microscopy

Quantitative phase imaging (QPI) is the most ideal method for achieving long-term cellular tomography because it is label free and quantitative. However, for current QPI instruments, interference signals from different layers overlay with each other and impede nanoscale optical sectioning. Integrated incubators and improved configurations also require further investigation for QPI instruments. In this work, hyperspectral self-reflectance microscopy is proposed to achieve label-free tomography of living cellular nanoarchitecture. The optical description and tomography reconstruction algorithm were proposed so that the quantitative morphological structure of the entire living cell can be acquired with 89.2 nm axial resolution and 1.91 nm optical path difference sensitivity. A cell incubator was integrated to culture living cells for in situ measurement and expensive precise optical components were not needed. The proposed system can reveal native and dynamic cellular nanoscale structure, providing an alternative approach for long-term monitoring and quantitative analysis of living cells.

Inversion-free image recovery from strong aberration using a minimally sampled transmission matrix

A transmission matrix (TM), a characteristic response for an input-output relation of an optical system, has been used for achieving diffraction-limited and aberration-free images through highly-aberrant imaging systems. However, its requirement of acquiring a huge-size TM along with its heavy computational load limit its widespread applications. Here we propose a method for TM-based image reconstruction, which is more efficient in terms of data manipulation and computational time. Only 10% of the TM elements for a fish-eye (FE) lens with strong aberration were sampled compared to that required for the image reconstruction by the conventional inversion method. The missing information was filled in by an iterative interpolation algorithm working in k-space. In addition, as a replacement of the time-consuming matrix inversion process, a phase pattern was created from the minimally sampled TM in order to compensate for the angle-dependent phase retardation caused by the FE lens. The focal distortion could be corrected by applying the phase correction pattern to the angular spectrums of the measured object images. The remaining spatial distortion could also be determined through the geometrical transformation also determined by the minimally sampled TM elements. Through the use of these procedures, the object image can be reconstructed 55 times faster than through the use of the usual inversion method using the full-sized TM, without compromising the reconstruction performances.

Aberration correction method based on double-helix point spread function

Point spread function (PSF) engineering has met with lots of interest in various optical imaging techniques, including super-resolution microscopy, microparticle tracking, and extended depth-of-field microscopy. The intensity distributions of the modified PSFs often suffer from deteriorations caused by system aberrations, which greatly degrade the image contrast, resolution, or localization precision. We present an aberration correction method using a spiral-phase-based double-helix PSF as an aberration indicator, which is sensitive and quantitatively correlated to the spherical aberration, coma, and astigmatism. Superior to the routine iteration-based correction methods, the presented approach is iteration-free and the aberration coefficients can be directly calculated with the measured parameters, relieving the computing burden. The validity of the method is verified by both examining the intensity distribution of the conventional Gaussian PSF in three dimensions and observing muntjac skin fibroblast cells. This iteration-free correction method has a potential application in PSF engineering systems equipped with a spatial light modulator.

Optical imaging featuring both long working distance and high spatial resolution by correcting the aberration of a large aperture lens

High-resolution optical imaging within thick objects has been a challenging task due to the short working distance of conventional high numerical aperture (NA) objective lenses. Lenses with a large physical diameter and thus a large aperture, such as microscope condenser lenses, can feature both a large NA and a long working distance. However, such lenses suffer from strong aberrations. To overcome this problem, we present a method to correct the aberrations of a transmission-mode imaging system that is composed of two condensers. The proposed method separately identifies and corrects aberrations of illumination and collection lenses of up to 1.2 NA by iteratively optimizing the total intensity of the synthetic aperture images in the forward and phase-conjugation processes. At a source wavelength of 785 nm, we demonstrated a spatial resolution of 372 nm at extremely long working distances of up to 1.6 mm, an order of magnitude improvement in comparison to conventional objective lenses. Our method of converting microscope condensers to high-quality objectives may facilitate increases in the imaging depths of super-resolution and expansion microscopes.

Pupil mask diversity for image correction in microscopy

Three-dimensional microscopy suffers from sample-induced aberrations that reduce the resolution and lead to misinterpretations of the object distribution. In this paper, the resolution of a three-dimensional fluorescent microscope is significantly improved by introducing an amplitude diversity in the form of a binary amplitude mask positioned in several different orientations within the pupil, followed by computer processing of the diversity images. The method has proved to be fast, easy to implement, and cost-effective in high-resolution imaging of casper fli:GFP zebrafish.

Brillouin micro-spectroscopy through aberrations via sensorless adaptive optics

Brillouin spectroscopy is a powerful optical technique for non-contact viscoelastic characterizations which has recently found applications in three-dimensional mapping of biological samples. Brillouin spectroscopy performances are rapidly degraded by optical aberrations and have therefore been limited to homogenous transparent samples. In this work, we developed an adaptive optics (AO) configuration designed for Brillouin scattering spectroscopy to engineer the incident wavefront and correct for aberrations. Our configuration does not require direct wavefront sensing and the injection of a "guide-star"; hence, it can be implemented without the need for sample pre-treatment. We used our AO-Brillouin spectrometer in aberrated phantoms and biological samples and obtained improved precision and resolution of Brillouin spectral analysis; we demonstrated 2.5-fold enhancement in Brillouin signal strength and 1.4-fold improvement in axial resolution because of the correction of optical aberrations.

Adaptive optical microscopy for neurobiology

With the ability to correct for the aberrations introduced by biological specimens, adaptive optics-a method originally developed for astronomical telescopes-has been applied to optical microscopy to recover diffraction-limited imaging performance deep within living tissue. In particular, this technology has been used to improve image quality and provide a more accurate characterization of both structure and function of neurons in a variety of living organisms. Among its many highlights, adaptive optical microscopy has made it possible to image large volumes with diffraction-limited resolution in zebrafish larval brains, to resolve dendritic spines over 600μm deep in the mouse brain, and to more accurately characterize the orientation tuning properties of thalamic boutons in the primary visual cortex of awake mice.

Wavefront control in adaptive microscopy using Shack-Hartmann sensors with arbitrarily shaped pupils

In adaptive optical microscopy of thick biological tissue, strong scattering and aberrations can change the effective pupil shape by rendering some Shack-Hartmann spots unusable. The change of pupil shape leads to a change of wavefront reconstruction or control matrix that should be updated accordingly. Modified slope and modal wavefront control methods based on measurements of a Shack-Hartmann wavefront sensor are proposed to accommodate an arbitrarily shaped pupil. Furthermore, we present partial wavefront control methods that remove specific aberration modes like tip, tilt and defocus from the control loop. The proposed control methods were investigated and compared by simulation using experimentally obtained aberration data. The performance was then tested experimentally through closed-loop aberration corrections using an obscured pupil.