Wavefront sensor-less adaptive optics using deep reinforcement learning

Image metric-based multi-observation single-step deep deterministic policy gradient for sensorless adaptive optics

Sensorless adaptive optics (SAO) has been widely used across diverse fields such as astronomy, microscopy, and ophthalmology. Recent advances have proved the feasibility of using the deep deterministic policy gradient (DDPG) for image metric-based SAO, achieving fast correction speeds compared to the coordinate search Zernike mode hill climbing (ZMHC) method. In this work, we present a multi-observation single-step DDPG (MOSS-DDPG) optimization framework for SAO on a confocal scanning laser ophthalmoscope (SLO) system with particular consideration for applications in preclinical retinal imaging. MOSS-DDPG optimizes N target Zernike coefficients in a single-step manner based on 2N + 1 observations of the image sharpness metric values. Through in silico simulations, MOSS-DDPG has demonstrated the capability to quickly achieve diffraction-limited resolution performance with long short-term memory (LSTM) network implementation. In situ tests suggest that knowledge learned through simulation adapts swiftly to imperfections in the real system by transfer learning, exhibiting comparable in situ performance to the ZMHC method with a greater than tenfold reduction in the required number of iterations.

Piston Error Automatic Correction for Segmented Mirrors via Deep Reinforcement Learning

The segmented mirror co-phase error identification technique based on supervised learning methods has the advantages of simple application conditions, no dependence on custom sensors, a fast calculation speed, and low computing power requirements compared with other methods. However, it is often difficult to obtain a high accuracy in practical application situations with this method because of the difference between the training model and the actual model. The reinforcement learning algorithm does not need to model the real system when operating the system. However, it still retains the advantages of supervised learning. Thus, in this paper, we placed a mask on the pupil plane of the segmented telescope optical system. Moreover, based on the wide spectrum, point spread function, and modulation transfer function of the optical system and deep reinforcement learning-without modeling the optical system-a large-range and high-precision piston error automatic co-phase method with multiple-submirror parallelization was proposed. Finally, we carried out relevant simulation experiments, and the results indicate that the method is effective.

Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution - one that can be readily transferred between microscope modalities - has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a "black box", but provided physical insights on internal workings, which could influence future designs.

Adaptive optics for optical microscopy [Invited]

Optical microscopy is widely used to visualize fine structures. When applied to bioimaging, its performance is often degraded by sample-induced aberrations. In recent years, adaptive optics (AO), originally developed to correct for atmosphere-associated aberrations, has been applied to a wide range of microscopy modalities, enabling high- or super-resolution imaging of biological structure and function in complex tissues. Here, we review classic and recently developed AO techniques and their applications in optical microscopy.

Generalization of learned Fourier-based phase-diversity wavefront sensing

Proper initialization of the nonlinear optimization is important to avoid local minima in phase diversity wavefront sensing (PDWS). An effective neural network based on low-frequency coefficients in the Fourier domain has proved effective to determine a better estimate of the unknown aberrations. However, the network relies significantly on the training settings, such as imaging object and optical system parameters, resulting in a weak generalization ability. Here we propose a generalized Fourier-based PDWS method by combining an object-independent network with a system-independent image processing procedure. We demonstrate that a network trained with a specific setting can be applied to any image regardless of the actual settings. Experimental results show that a network trained with one setting can be applied to images with four other settings. For 1000 aberrations with RMS wavefront errors bounded within [0.2 λ, 0.4 λ], the mean RMS residual errors are 0.032 λ, 0.039 λ, 0.035 λ, and 0.037 λ, respectively, and 98.9% of the RMS residual errors are less than 0.05 λ.

Reverse translation of artificial intelligence in glaucoma: Connecting basic science with clinical applications

Artificial intelligence (AI) has been approved for biomedical research in diverse areas from bedside clinical studies to benchtop basic scientific research. For ophthalmic research, in particular glaucoma, AI applications are rapidly growing for potential clinical translation given the vast data available and the introduction of federated learning. Conversely, AI for basic science remains limited despite its useful power in providing mechanistic insight. In this perspective, we discuss recent progress, opportunities, and challenges in the application of AI in glaucoma for scientific discoveries. Specifically, we focus on the research paradigm of reverse translation, in which clinical data are first used for patient-centered hypothesis generation followed by transitioning into basic science studies for hypothesis validation. We elaborate on several distinctive areas of research opportunities for reverse translation of AI in glaucoma including disease risk and progression prediction, pathology characterization, and sub-phenotype identification. We conclude with current challenges and future opportunities for AI research in basic science for glaucoma such as inter-species diversity, AI model generalizability and explainability, as well as AI applications using advanced ocular imaging and genomic data.

Using beam-offset optical coherence tomography to reconstruct backscattered photon profiles in scattering media

Raster scanning imaging technologies capture least scattered photons (LSPs) and reject multiple scattered photons (MSPs) in backscattered photons to image the underlying structures of a scattering medium. However, MSPs can still squeeze into the images, resulting in limited imaging depth, degraded contrast, and significantly reduced lateral resolution. Great efforts have been made to understand how MSPs affect imaging performance through modeling, but the techniques for visualizing the backscattered photon profile (BSPP) in scattering media during imaging are unavailable. Here, a method of reconstructing BSPP is demonstrated using beam-offset optical coherence tomography (OCT), in which OCT images are acquired at offset positions from the illumination beam. The separation of LSPs and MSPs based on the BSPP enables quantification of imaging depth, contrast, and lateral resolution, as well as access to the depth-resolved modulated transfer function (MTF). This approach presents great opportunities for better retrieving tissue optical properties, correctly interpreting images, or directly using MTF as the feedback for adaptive optical imaging.

Advances in Optical Coherence Tomography Imaging Technology and Techniques for Choroidal and Retinal Disorders

Optical coherence tomography (OCT) imaging has played a pivotal role in the field of retina. This light-based, non-invasive imaging modality provides high-quality, cross-sectional analysis of the retina and has revolutionized the diagnosis and management of retinal and choroidal diseases. Since its introduction in the early 1990s, OCT technology has continued to advance to provide quicker acquisition times and higher resolution. In this manuscript, we discuss some of the most recent advances in OCT technology and techniques for choroidal and retinal diseases. The emerging innovations discussed include wide-field OCT, adaptive optics OCT, polarization sensitive OCT, full-field OCT, hand-held OCT, intraoperative OCT, at-home OCT, and more. The applications of these rising OCT systems and techniques will allow for a closer monitoring of chorioretinal diseases and treatment response, more robust analysis in basic science research, and further insights into surgical management. In addition, these innovations to optimize visualization of the choroid and retina offer a promising future for advancing our understanding of the pathophysiology of chorioretinal diseases.

Application of Adaptive Optics in Ophthalmology

The eye, the photoreceptive organ used to perceive the external environment, is of great importance to humans. It has been proven that some diseases in humans are accompanied by fundus changes; therefore, the health status of people may be interpreted from retinal images. However, the human eye is not a perfect refractive system for the existence of ocular aberrations. These aberrations not only affect the ability of human visual discrimination and recognition, but restrict the observation of the fine structures of human eye and reduce the possibility of exploring the mechanisms of eye disease. Adaptive optics (AO) is a technique that corrects optical wavefront aberrations. Once integrated into ophthalmoscopes, AO enables retinal imaging at the cellular level. This paper illustrates the principle of AO in correcting wavefront aberrations in human eyes, and then reviews the applications and advances of AO in ophthalmology, including the adaptive optics fundus camera (AO-FC), the adaptive optics scanning laser ophthalmoscope (AO-SLO), the adaptive optics optical coherence tomography (AO-OCT), and their combined multimodal imaging technologies. The future development trend of AO in ophthalmology is also prospected.