Learning-based focusing through scattering media

Regularized cost function in wavefront shaping for advancing the contrast of structured light

The cost function in the iterative optimization algorithms is one of the sensitive optimization controllers that plays a crucial role in feedback based wavefront shaping for constructing well-resolved complex structured light through scattering media. There has been a trade-off between resolution and the contrast enhancement of the structured light in wavefront shaping. We have developed an ℓ 2-norm based quadratic cost function (L2QN) and proposed a regularized cost function (RCF) for advancing the contrast and maintaining the high resolution of structured light. Both the simulations and experiments have been performed, and it has been found that the proposed RCF significantly advances the contrast and structural uniformity for focusing light through scattering media as well as for diffused reflection mode. The potential applications of the method demonstrated in this study can be extended into holographic displays, structured light illumination microscopy, photo-lithography, photothermal treatments, dosimetry, laser materials processing, and energy control inside and outside an incubation system.

Extrapolated speckle-correlation imaging with an untrained deep neural network

We present a method for speckle-correlation imaging with an extended field of view to observe spatially non-sparse objects. In speckle-correlation imaging, an object is recovered from a non-invasively captured image through a scattering medium by assuming shift-invariance of the optical process called the memory effect. The field of view of speckle-correlation imaging is limited by the size of the memory effect, and it can be extended by extrapolating the speckle correlation in the reconstruction process. However, spatially sparse objects are assumed in the inversion process because of its severe ill-posedness. To address this issue, we introduce a deep image prior, which regularizes the image statistics by using the structure of an untrained convolutional neural network, to speckle-correlation imaging. We experimentally demonstrated the proposed method and showed the possibility of extending the method to imaging through scattering media.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media. Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution. However, the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications. In addition, the components of an optical system are usually designed and manufactured for a fixed function or performance. Recent advances in wavefront shaping have demonstrated that scattering- or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium. This offers unprecedented opportunities in many applications to achieve controllable optical delivery or detection at depths or dynamically configurable functionalities by using scattering media to substitute conventional optical components. In this article, the recent progress of wavefront shaping in multidisciplinary fields is reviewed, from optical focusing and imaging with scattering media, functionalized devices, modulation of mode coupling, and nonlinearity in multimode fiber to multimode fiber-based applications. Apart from insights into the underlying principles and recent advances in wavefront shaping implementations, practical limitations and roadmap for future development are discussed in depth. Looking back and looking forward, it is believed that wavefront shaping holds a bright future that will open new avenues for noninvasive or minimally invasive optical interactions and arbitrary control inside deep tissues. The high degree of freedom with multiple scattering will also provide unprecedented opportunities to develop novel optical devices based on a single scattering medium (generic or customized) that can outperform traditional optical components.

Incoherent optical phase conjugation

Optical phase conjugation is a known technique for optically reproducing an object behind a scattering medium. Here we present digital optical phase conjugation through scattering media with spatially and temporally incoherent light. This enables us to eliminate the inevitable light coherence and the need for interferometric measurement for optical phase conjugation. Moreover, we show a method for suppressing background noise, which is critical in incoherent optical phase conjugation. We numerically and experimentally demonstrate the proposed method with background suppression.

Spectral speckle-correlation imaging

We present a method for single-shot spectrally resolved imaging through scattering media by using the spectral memory effect of speckles. In our method, a single speckle pattern from a multi-colored object is captured through scattering media with a monochrome image sensor. The color object is recovered by correlation of the captured speckle and a three-dimensional phase retrieval process. The proposed method was experimentally demonstrated by using point sources with different emission spectra located between diffusers. This study paves the way for non-invasive and low-cost spectral imaging through scattering media.

Three-dimensional deeply generated holography [Invited]

In this paper, we present a noniterative method for 3D computer-generated holography based on deep learning. A convolutional neural network is adapted for directly generating a hologram to reproduce a 3D intensity pattern in a given class. We experimentally demonstrated the proposed method with optical reproductions of multiple layers based on phase-only Fourier holography. Our method is noniterative, but it achieves a reproduction quality comparable with that of iterative methods for a given class.

Visual data detection through side-scattering in a multimode optical fiber

Light propagation in optical fibers is accompanied by random omnidirectional scattering. The small fraction of coherent guided light that escapes outside the cladding of the fiber forms a speckle pattern. Here, visual information imaged into the input facet of a multimode fiber with a transparent buffer is retrieved, using a convolutional neural network, from the side-scattered light at several locations along the fiber. This demonstration can promote the development of distributed optical imaging systems and optical links interfaced via the sides of the fiber.

Deeply coded aperture for lensless imaging

In this Letter, we present a method for jointly designing a coded aperture and a convolutional neural network for reconstructing an object from a single-shot lensless measurement. The coded aperture and the reconstruction network are connected with a deep learning framework in which the coded aperture is placed as a first convolutional layer. Our co-optimization method was experimentally demonstrated with a fully convolutional network, and its performance was compared to a coded aperture with a modified uniformly redundant array.

Single-shot noninvasive three-dimensional imaging through scattering media

We present a method for single-shot three-dimensional imaging through scattering media with a three-dimensional memory effect. In the proposed computational process, a captured speckle image is two-dimensionally correlated with different scales, and the object is three-dimensionally recovered with three-dimensional phase retrieval. Our method was experimentally demonstrated with a lensless setup and was compared with a multishot approach used in our previous work [Opt. Lett.44, 2526 (2019)OPLEDP0146-959210.1364/OL.44.002526].

Image reconstruction through dynamic scattering media based on deep learning

Under complex scattering conditions, it is very difficult to capture clear object images hidden behind the media by modelling the inverse problem. With regard to dynamic scattering media, the challenge increases. For solving the inverse problem, we propose a new class-specific image reconstruction algorithm. The method based on deep learning classifies blurred scattering images according to scattering conditions and then recovers to clear images hidden behind the media. The deep learning network is used to learn the mapping relationship between the object and the scattering image rather than characterizing the scattering media explicitly or parametrically. 25000 scattering images are obtained under five sets of dynamic scattering condition to verify the feasibility of the proposed method. In addition, the generalizability of the method has been verified successfully. Compared with common CNN method, it's confirmed that our algorithm has better performance in reconstructing higher-quality images. Furthermore, for a given scattering image with unknown scattering condition, the closest scattering condition information can be given by classification network, and then the corresponding clear image is restored by reconstruction network.

Noninvasive three-dimensional imaging through scattering media by three-dimensional speckle correlation

We present a method for noninvasive three-dimensional imaging through scattering media by using a three-dimensional memory effect in scattering phenomena. In the proposed method, an object in a scattering medium is reconstructed from a three-dimensional autocorrelation of speckle images captured by axially scanning an image sensor, based on a three-dimensional phase retrieval algorithm. We experimentally demonstrated our method with a lensless setup by using a three-dimensionally printed object between diffusers.

Deep learning wavefront sensing

We present a new class of wavefront sensors by extending their design space based on machine learning. This approach simplifies both the optical hardware and image processing in wavefront sensing. We experimentally demonstrated a variety of image-based wavefront sensing architectures that can directly estimate Zernike coefficients of aberrated wavefronts from a single intensity image by using a convolutional neural network. We also demonstrated that the proposed deep learning wavefront sensor can be trained to estimate wavefront aberrations stimulated by a point source and even extended sources.

Light scattering control in transmission and reflection with neural networks

Scattering often limits the controlled delivery of light in applications such as biomedical imaging, optogenetics, optical trapping, and fiber-optic communication or imaging. Such scattering can be controlled by appropriately shaping the light wavefront entering the material. Here, we develop a machine-learning approach for light control. Using pairs of binary intensity patterns and intensity measurements we train neural networks (NNs) to provide the wavefront corrections necessary to shape the beam after the scatterer. Additionally, we demonstrate that NNs can be used to find a functional relationship between transmitted and reflected speckle patterns. Establishing the validity of this relationship, we focus and scan in transmission through opaque media using reflected light. Our approach shows the versatility of NNs for light shaping, for efficiently and flexibly correcting for scattering, and in particular the feasibility of transmission control based on reflected light.

Imaging through scattering media using speckle pattern classification based support vector regression

Imaging through scattering media is a common practice in many applications of biomedical imaging. Object image would deteriorate into unrecognizable speckle pattern when scattering media is presented. Many methods have been investigated to reconstruct the object image when only speckle pattern is available. In this paper, we demonstrate a method of single-shot imaging through scattering media. This method is based on classification and support vector regression of the measured speckle pattern. We prove the possibility of speckle pattern classification and related formulas are presented. The specified and limited imaging capability without speckle pattern classification is demonstrated. Our proposed approach, that is, speckle pattern classification based support vector regression method, makes up the deficiency. Experimental results show that, with our approach, speckle patterns could be utilized for classification when object images are unavailable, and object images can be reconstructed with high fidelity. The proposed approach for imaging through scattering media is expected to be applicable to various sensing schemes.

Deep-learning-generated holography

We present a method for computer-generated holography based on deep learning. The inverse process of light propagation is regressed with a number of computationally generated speckle data sets. This method enables noniterative calculation of computer-generated holograms (CGHs). The proposed method was experimentally verified with a phase-only CGH.

Learning-based single-shot superresolution in diffractive imaging

We present a method of retrieving a superresolved object field from a single captured intensity image in diffraction-limited diffractive imaging based on machine learning. In this method, the inverse process of diffractive imaging is regressed by using a number of pairs, each consisting of object and captured images. The proposed method is experimentally demonstrated by using a lensless imaging setup with or without scattering media.