Methods for the characterization of deformable membrane mirrors

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution - one that can be readily transferred between microscope modalities - has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a "black box", but provided physical insights on internal workings, which could influence future designs.

Deflectometry based calibration of a deformable mirror for aberration correction and remote focusing in microscopy

Adaptive optics (AO) techniques enhance the capability of optical microscopy through precise control of wavefront modulations to compensate phase aberrations and improves image quality. However, the aberration correction is often limited due to the lack of dynamic range in existing calibration methods, such as interferometry or Shack-Hartmann (SH) wavefront sensors. Here, we use deflectometry (DF) as a calibration method for a deformable mirror (DM) to extend the available range of aberration correction. We characterised the dynamic range and accuracy of the DF-based calibration of DMs depending on the spatial frequency of the test pattern used in DF. We also demonstrated the capability of large magnitude phase control for remote-focusing over a range larger than was possible with SH sensing.

SOLEIL: single-objective lens inclined light sheet localization microscopy

High-NA light sheet illumination can improve the resolution of single-molecule localization microscopy (SMLM) by reducing the background fluorescence. These approaches currently require custom-made sample holders or additional specialized objectives, which makes the sample mounting or the optical system complex and therefore reduces the usability of these approaches. Here, we developed a single-objective lens-inclined light sheet microscope (SOLEIL) that is capable of 2D and 3D SMLM in thick samples. SOLEIL combines oblique illumination with point spread function PSF engineering to enable dSTORM imaging in a wide variety of samples. SOLEIL is compatible with standard sample holders and off-the-shelve optics and standard high NA objectives. To accomplish optimal optical sectioning we show that there is an ideal oblique angle and sheet thickness. Furthermore, to show what optical sectioning delivers for SMLM we benchmark SOLEIL against widefield and HILO microscopy with several biological samples. SOLEIL delivers in 15 μm thick Caco2-BBE cells a 374% higher intensity to background ratio and a 54% improvement in the estimated CRLB compared to widefield illumination, and a 184% higher intensity to background ratio and a 20% improvement in the estimated CRLB compared to HILO illumination.

Efficient wavefront sensorless adaptive optics based on large dynamic crosstalk-free holographic modal wavefront sensing

The correction of wavefront sensorless adaptive optics (WFSless AO) can be significantly accelerated by using a holographic modal wavefront sensor (HMWFS). The HMWFS is realized by a computer-generated hologram (CGH) into which all aberration modes to be detected are encoded and only a single-shot image is required for simultaneous measurement of multiple modes. The conventional HMWFS suffers from a quite limited dynamic range and severe inter-modal crosstalk which deteriorates the sensing accuracy. We proposed a novel HMWFS with a large dynamic range and no crosstalk and validated its performance by simulation and experiment. In the improved HMWFS scheme, the aberration is represented by Lukosz modes whose gradients are orthogonal and the modal coefficients can be estimated independently. Instead of using a binary CGH in conventional HMWFS, a kinoform CGH with high diffraction efficiency is adopted in the improved HMWFS. The kinoform CGH is produced by a phase-only liquid-crystal spatial light modulator (LCSLM) which also serves as a wavefront corrector in our WFSless AO system.

Three-dimensional adaptive optical nanoscopy for thick specimen imaging at sub-50-nm resolution

Understanding cellular organization demands the best possible spatial resolution in all three dimensions. In fluorescence microscopy, this is achieved by 4Pi nanoscopy methods that combine the concepts of using two opposing objectives for optimal diffraction-limited 3D resolution with switching fluorescent molecules between bright and dark states to break the diffraction limit. However, optical aberrations have limited these nanoscopes to thin samples and prevented their application in thick specimens. Here we have developed an improved iso-stimulated emission depletion nanoscope, which uses an advanced adaptive optics strategy to achieve sub-50-nm isotropic resolution of structures such as neuronal synapses and ring canals previously inaccessible in tissue. The adaptive optics scheme presented in this work is generally applicable to any microscope with a similar beam path geometry involving two opposing objectives to optimize resolution when imaging deep in aberrating specimens.

Microscope-AOtools: a generalised adaptive optics implementation

Aberrations arising from sources such as sample heterogeneity and refractive index mismatches are constant problems in biological imaging. These aberrations reduce image quality and the achievable depth of imaging, particularly in super-resolution microscopy techniques. Adaptive optics (AO) technology has been proven to be effective in correcting for these aberrations, thereby improving the image quality. However, it has not been widely adopted by the biological imaging community due, in part, to difficulty in set-up and operation of AO. The methods for doing so are not novel or unknown, but new users often waste time and effort reimplementing existing methods for their specific set-ups, hardware, sample types, etc. Microscope-AOtools offers a robust, easy-to-use implementation of the essential methods for set-up and use of AO elements and techniques. These methods are constructed in a generalised manner that can utilise a range of adaptive optics elements, wavefront sensing techniques and sensorless AO correction methods. Furthermore, the methods are designed to be easily extensible as new techniques arise, leading to a streamlined pipeline for new AO technology and techniques to be adopted by the wider microscopy community.

Compact adaptive interferometer for unknown freeform surfaces with large departure

The newest experimental validation report of the coverage for the rotationally non-symmetric departure of a freeform surface in adaptive interferometry is about 20 µm. A compact adaptive interferometer is introduced to test unknown freeform surfaces with larger departure. The cascaded DMs (woofer and tweeter) can effectively double the measurable rotationally non-symmetric departure, to ∼80 µm using current DM technology. With a constrained decoupling control algorithm, the woofer and tweeter can averagely share the aberrations without coupling. DM surface monitoring is addressed by a time-division-monitoring (TDM) technique, which avoids separate monitoring devices and configurations and thus makes a compact configuration. Measurements of two different surfaces are presented: a nearly flat freeform with ∼40 um departure, and an off-axis paraboloid with ∼50 um of asymmetric departure.

Nanodiamonds enable adaptive-optics enhanced, super-resolution, two-photon excitation microscopy

Particles of diamond in the 5-100 nm size range, known as nanodiamond (ND), have shown promise as robust fluorophores for optical imaging. We demonstrate here that, due to their photostability, they are not only suitable for two-photon imaging, but also allow significant resolution enhancement when combined with computational super-resolution techniques. We observe a resolution of 42.5 nm when processing two-photon images with the Super-Resolution Radial Fluctuations algorithm. We show manipulation of the point-spread function of the microscope using adaptive optics. This demonstrates how the photostability of ND can also be of use when characterizing adaptive optics technologies or testing the resilience of super-resolution or aberration correction algorithms.

IsoSense: frequency enhanced sensorless adaptive optics through structured illumination

We present IsoSense, a wavefront sensing method that mitigates sample dependency in image-based sensorless adaptive optics applications in microscopy. Our method employs structured illumination to create additional high spatial frequencies in the image through custom illumination patterns. This improves the reliability of image quality metric calculations and enables sensorless wavefront measurement even in samples with sparse spatial frequency content. We demonstrate the feasibility of IsoSense for aberration correction in a deformable-mirror-based structured illumination super-resolution fluorescence microscope.

Optimization-based real-time open-loop control of an optofluidic refractive phase modulator

We present a novel open-loop control method for an electrostatically actuated optofluidic refractive phase modulator, and demonstrate its performance for high-order aberration correction. Contrary to conventional electrostatic deformable mirrors, an optofluidic modulator is capable of bidirectional (push-pull) actuation through hydro-mechanical coupling. Control methods based on matrix pseudo-inversion, the common approach used for deformable mirrors, thus perform sub-optimally for such a device. Instead, we formulate the task of finding driving voltages for a given desired wavefront shape as an optimization problem with inequality constraints that can be solved using an interior-point method in real time. We show that this optimization problem is a convex one and that its solution represents a global minimum in residual wavefront error. We use the new method to control both the refractive phase modulator and a conventional electrostatic deformable mirror, and experimentally demonstrate improved correction fidelity for both.

Extended-image-based correction of aberrations using a deformable mirror with hysteresis

With a view to the next generation of large space telescopes, we investigate guide-star-free, image-based aberration correction using a unimorph deformable mirror in a plane conjugate to the primary mirror. We designed and built a high-resolution imaging testbed to evaluate control algorithms. In this paper we use an algorithm based on the heuristic hill climbing technique and compare the correction in three different domains, namely the voltage domain, the domain of the Zernike modes, and the domain of the singular modes of the deformable mirror. Through our systematic experimental study, we found that successive control in two domains effectively counteracts uncompensated hysteresis of the deformable mirror.

Rapid adaptive remote focusing microscope for sensing of volumetric neural activity

The ability to record neural activity in the brain of a living organism at cellular resolution is of great importance for defining the neural circuit mechanisms that direct behavior. Here we present an adaptive two-photon microscope optimized for extraction of neural signals over volumes in intact Drosophila brains, even in the presence of specimen motion. High speed volume imaging was made possible through reduction of spatial resolution while maintaining the light collection efficiency of a high resolution, high numerical aperture microscope. This enabled simultaneous recording of odor-evoked calcium transients in a defined volume of mushroom body Kenyon cell bodies in a live fruit fly.

Pulse front adaptive optics: a new method for control of ultrashort laser pulses

Ultrafast lasers enable a wide range of physics research and the manipulation of short pulses is a critical part of the ultrafast tool kit. Current methods of laser pulse shaping are usually considered separately in either the spatial or the temporal domain, but laser pulses are complex entities existing in four dimensions, so full freedom of manipulation requires advanced forms of spatiotemporal control. We demonstrate through a combination of adaptable diffractive and reflective optical elements - a liquid crystal spatial light modulator (SLM) and a deformable mirror (DM) - decoupled spatial control over the pulse front (temporal group delay) and phase front of an ultra-short pulse was enabled. Pulse front modulation was confirmed through autocorrelation measurements. This new adaptive optics technique, for the first time enabling in principle arbitrary shaping of the pulse front, promises to offer a further level of control for ultrafast lasers.

Optimization-based wavefront sensorless adaptive optics for multiphoton microscopy

Optical aberrations have detrimental effects in multiphoton microscopy. These effects can be curtailed by implementing model-based wavefront sensorless adaptive optics, which only requires the addition of a wavefront shaping device, such as a deformable mirror (DM) to an existing microscope. The aberration correction is achieved by maximizing a suitable image quality metric. We implement a model-based aberration correction algorithm in a second-harmonic microscope. The tip, tilt, and defocus aberrations are removed from the basis functions used for the control of the DM, as these aberrations induce distortions in the acquired images. We compute the parameters of a quadratic polynomial that is used to model the image quality metric directly from experimental input-output measurements. Finally, we apply the aberration correction by maximizing the image quality metric using the least-squares estimate of the unknown aberration.

Direct wavefront sensing in adaptive optical microscopy using backscattered light

Adaptive optics has been used to compensate the detrimental effects of aberrations in a range of high-resolution microscopes. We investigate how backscattered laser illumination can be used as the source for direct wavefront sensing using a pinhole-filtered Shack-Hartmann wavefront sensor. It is found that the sensor produces linear response to input aberrations for a given specimen. The gradient of this response is dependent upon experimental configuration and specimen structure. Cross sensitivity between modes is also observed. The double pass nature of the microscope system leads in general to lower sensitivity to odd-symmetry aberration modes. The results show that there is potential for use of this type of wavefront sensing in microscopes.

Three dimensional laser microfabrication in diamond using a dual adaptive optics system

Femtosecond laser fabrication of controlled three dimensional structures deep in the bulk of diamond is facilitated by a dual adaptive optics system. A deformable mirror is used in parallel with a liquid crystal spatial light modulator to compensate the extreme aberrations caused by the refractive index mismatch between the diamond and the objective immersion medium. It is shown that aberration compensation is essential for the generation of controlled micron-scale features at depths greater than 200 μm, and the dual adaptive optics approach demonstrates increased fabrication efficiency relative to experiments using a single adaptive element.

Simple characterisation of a deformable mirror inside a high numerical aperture microscope using phase diversity

We present a simple and versatile scheme for characterising amplitude and phase modulation by an active element, such as a deformable mirror, in the pupil plane of a high NA microscope. By placing a mirror in the vicinity of the focal plane of the objective and recording images of the reflected focal spot on a camera, we show that reliable measurements of the influence function of the mirror actuators in the pupil plane of the objective can be obtained using an iterative electric field retrieval algorithm. Compared to direct wavefront sensors, the proposed method allows characterisation for a variety of objectives with different NA and pupil sizes without modification of the setup, requires minimal space inside the microscope, and can be used with pulsed sources such as used for multiphoton microscopy. In order to validate our method, we compare our data to the results obtained with a Shack-Hartmann wavefront sensor, and show that comparable precision is achieved.

Performance analysis of multiplexed phase computer-generated hologram for modal wavefront sensing

We investigate the performance and capability of a holographic modal wavefront sensor (HMWS) that is based on a multiplexed phase computer-generated hologram (MPCGH). The theoretical treatments of the HMWS are presented with scalar diffraction approximations and Fourier analysis. Several MPCGHs have been designed with different linear carrier frequencies, by using of the multiplexed coding scheme we have proposed, and by coding some common Zernike modes. The numerical simulation is carried out to investigate the performance of the HMWS to detect particular aberration mode(s), by considering the effect of different carrier frequency selections and the capability of coding a large number of modes. The results exhibit the expected characteristics of a corresponding symmetric spot pair, and indicate that the wavefront distorted by a particular Zernike mode(s) can be retrieved immediately through solving the amplitude of each mode coded in MPCGHs through the response curves of the HMWS.

Long-term imaging of mouse embryos using adaptive harmonic generation microscopy

We present a detailed description of an adaptive harmonic generation (HG) microscope and culture techniques that permit long-term, three-dimensional imaging of mouse embryos. HG signal from both pre- and postimplantation stage (0.5-5.5 day-old) mouse embryos are fully characterized. The second HG images reveal central spindles during cytokinesis whereas third HG images show several features, such as lipid droplets, nucleoli, and plasma membranes. The embryos are found to develop normally during one-day-long discontinuous HG imaging, permitting the observation of several dynamic events, such as morula compaction and blastocyst formation.

Optical pattern recognition via adaptive spatial homodyne detection

We present an experimental demonstration of an optical pattern recognition scheme based on spatial homodyne detection. Our scheme is adaptive, all-optical, utilizes a single-element photo-detector, and provides a single parameter readout to quantify the efficacy of pattern recognition, thereby allowing very fast pattern recognition speeds. The spatial homodyne detector was applied to the identification of one- and two-dimensional phase profiles.

Image-based adaptive optics for two-photon microscopy

We demonstrate wavefront sensorless aberration correction in a two-photon excited fluorescence microscope. Using analysis of the image-formation process, we have developed an optimized correction scheme permitting image-quality improvement with minimal additional exposure of the sample. We show that, as a result, our correction process induces little photobleaching and significantly improves the quality of images of biological samples. In particular, increased visibility of small structures is demonstrated. Finally, we illustrate the use of this technique on various fresh and fixed biological tissues.

Optical compensation of distorted data image caused by interference fringe distortion in holographic data storage

Photopolymer materials shrink because of photopolymerization. This shrinkage distorts the recorded interference fringes in a medium made of such material, which in turn degrades the reconstructed image quality. Adaptive optics controlled by a genetic algorithm was developed to optimize the wavefront of the reference beam while reproducing in order to compensate for the interference fringe distortion. We defined a fitness measure for this genetic algorithm that involves the mean brightness and coefficients of the variations of bit data "1" and "0". In an experiment, the adaptive optics improved the reconstructed image to the extent that data could be reproduced from the entire area of the image, and the signal to noise ratio of the reproduced data could be improved.

Compact adaptive optics line scanning ophthalmoscope

We have developed a compact retinal imager that integrates adaptive optics (AO) into a line scanning ophthalmoscope (LSO). The bench-top AO-LSO instrument significantly reduces the size, complexity, and cost of research AO scanning laser ophthalmoscopes (AOSLOs), for the purpose of moving adaptive optics imaging more rapidly into routine clinical use. The AO-LSO produces high resolution retinal images with only one moving part and a significantly reduced instrument footprint and number of optical components. The AO-LSO has a moderate field of view (5.5 deg), which allows montages of the macula or other targets to be obtained more quickly and efficiently. In a preliminary human subjects investigation, photoreceptors could be resolved and counted within approximately 0.5 mm of the fovea. Photoreceptor counts matched closely to previously reported histology. The capillaries surrounding the foveal avascular zone could be resolved, as well as cells flowing within them. Individual nerve fiber bundles could be resolved, especially near the optic nerve head, as well as other structures such as the lamina cribrosa. In addition to instrument design, fabrication, and testing, software algorithms were developed for automated image registration and cone counting.

Adaptive optics for structured illumination microscopy

We implement wave front sensor-less adaptive optics in a structured illumination microscope. We investigate how the image formation process in this type of microscope is affected by aberrations. It is found that aberrations can be classified into two groups, those that affect imaging of the illumination pattern and those that have no influence on this pattern. We derive a set of aberration modes ideally suited to this application and use these modes as the basis for an efficient aberration correction scheme. Each mode is corrected independently through the sequential optimisation of an image quality metric. Aberration corrected imaging is demonstrated using fixed fluorescent specimens. Images are further improved using differential aberration imaging for reduction of background fluorescence.

Adaptive optics in microscopy

The imaging properties of optical microscopes are often compromised by aberrations that reduce image resolution and contrast. Adaptive optics technology has been employed in various systems to correct these aberrations and restore performance. This has required various departures from the traditional adaptive optics schemes that are used in astronomy. This review discusses the sources of aberrations, their effects and their correction with adaptive optics, particularly in confocal and two-photon microscopes. Different methods of wavefront sensing, indirect aberration measurement and aberration correction devices are discussed. Applications of adaptive optics in the related areas of optical data storage, optical tweezers and micro/nanofabrication are also reviewed.