Imaging of objects through a thin scattering layer using a spectrally and spatially separated reference

Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media

Optical imaging in scattering media is important to many fields but remains challenging. Recent methods have focused on imaging through thin scattering layers or thicker scattering media with prior knowledge of the sample, but this still limits practical applications. Here, we report an imaging method named 'speckle kinetography' that enables high-resolution imaging in unknown scattering media with thicknesses up to about 6 transport mean free paths. Speckle kinetography non-invasively records a series of incoherent speckle images accompanied by object motion and the inherently retained object information is extracted through an overlapping speckle correlation algorithm to construct the object's autocorrelation for imaging. Under single-colour light-emitting diode, white light, and fluorescence illumination, we experimentally demonstrate 1 μm resolution imaging and tracking of objects moving in scattering samples, while reducing the requirements for prior knowledge. We anticipate this method will enable imaging in currently inaccessible scenarios.

High-throughput imaging through dynamic scattering media based on speckle de-blurring

Effectively imaging through dynamic scattering media is of great importance and challenge. Some imaging methods based on physical or learning models have been designed for object reconstruction. However, with an increase in exposure time or more drastic changes in the scattering medium, the speckle pattern superimposed during camera integration time undergoes more significant changes, resulting in a modification of the collected speckle structure and increased blurring, which brings significant challenges to the reconstruction. Here, the clearer structural information of blurred speckles is unearthed with a presented speckle de-blurring algorithm, and a high-throughput imaging method through rapidly changing scattering media is proposed for reconstruction under long exposure. For the problem of varying blur degrees in different regions of the speckle, a block-based method is proposed to divide the speckle into distinct sub-speckles, which can realize the reconstruction of hidden objects. The imaging of hidden objects with different complexity through dynamic scattering media is demonstrated, and the reconstruction results are improved significantly for speckles with different blur degrees, which verifies the effectiveness of the method. This method is a high-throughput approach that enables non-invasive imaging solely through the collection of a single speckle. It directly operates on blurred speckles, making it suitable for traditional speckle-correlation methods and deep learning (DL) methods. This provides a new way of thinking about solving practical scattering imaging challenges.

Central wavelength estimation in spectral imaging behind a diffuser via deep learning

Multispectral imaging through scattering media is an important practical issue in the field of sensing. The light from a scattering medium is expected to carry information about the spectral properties of the medium, as well as geometrical information. Because spatial and spectral information of the object is encoded in speckle images, the information about the structure and spectrum of the object behind the scattering medium can be estimated from those images. Here we propose a deep learning-based strategy that can estimate the central wavelength from speckle images captured with a monochrome camera. When objects behind scattering media are illuminated with narrowband light having different spectra with different spectral peaks, deep learning of speckle images acquired at different central wavelengths can extend the spectral region to reconstruct images and estimate the central wavelengths of the illumination light. The proposed method achieves central wavelength estimation in 1 nm steps for objects whose central wavelength varies in a range of 100 nm. Because our method can achieve image reconstruction and central wavelength estimation in a single shot using a monochrome camera, this technique will pave the way for multispectral imaging through scattering media.

Using speckle correlations for single-shot 3D imaging

Recovery of a 3D object behind a scattering medium is an important problem in many fields, including biomedical and defense applications. Speckle correlation imaging can recover objects in a single shot but contains no depth information. To date, its extension to 3D recovery has relied on multiple measurements, multi-spectral light, or pre-calibration of the speckle with a reference object. Here, we show that the presence of a point source behind the scatterer enables single-shot reconstruction of multiple objects at multiple depths. The method relies on speckle scaling from the axial memory effect, in addition to the transverse one, and recovers objects directly, without the need for phase retrieval. We provide simulation and experimental results to show object reconstructions at different depths with a single-shot measurement. We also provide theoretical principles describing the region where speckle scales with axial distance and its effects on the depth of field. Our technique will be useful where a natural point source exists, such as fluorescence imaging or car headlights in fog.

Locating through dynamic scattering media based on speckle correlations

In complex imaging settings, optical scattering often prohibits the formation of a clear target image, and instead, only a speckle without the original spatial structure information is obtained. Scattering seriously interferes with the locating of targets; especially, when the scattering medium is dynamic, the dynamic nature leads to rapid decorrelation of optical information in time, and the challenge increases. Here, a locating method is proposed to detect the target hidden behind a dynamic scattering medium, which uses the a priori information of a known reference object in the neighborhood of the target. The research further designs an automatic calibration method to simplify the locating process, and analyzes the factors affecting positioning accuracy. The proposed method enables us to predict the position of a target from the autocorrelation of the captured speckle pattern; the angle and distance deviations of the target are all within 2.5%. This approach can locate a target using only a single-shot speckle pattern, and it is beneficial for target localization in dynamic scattering conditions.

Prior-free imaging unknown target through unknown scattering medium

Imaging through scattering medium based on deep learning has been extensively studied. However, existing methods mainly utilize paired data-prior and lack physical-process fusion, and it is difficult to reconstruct hidden targets without the trained networks. This paper proposes an unsupervised neural network that integrates the universal physical process. The reconstruction process of the network is irrelevant to the system and only requires one frame speckle pattern and unpaired targets. The proposed network enables online optimization by using physical process instead of fitting data. Thus, large-scale paired data no longer need to be obtained to train the network in advance, and the proposed method does not need prior information. The optimization of the network is a physical-based process rather than a data mapping process, and the proposed method also increases the insufficient generalization ability of the learning-based method in scattering medium and targets. The universal applicability of the proposed method to different optical systems increases the likelihood that the method will be used in practice.

Investigation of image plane for image reconstruction of objects through diffusers via deep learning

SIGNIFICANCE

The imaging of objects hidden in light-scattering media is a vital practical task in a wide range of applications, including biological imaging. Deep-learning-based methods have been used to reconstruct images behind scattering media under complex scattering conditions, but improvements in the quality of the reconstructed images are required.

AIM

To investigate the effect of image plane on the accuracy of reconstructed images.

APPROACH

Light reflected from an object passing through glass diffusers is captured by changing the image plane of an optical imaging system. Images are reconstructed by deep learning, and evaluated in terms of structural similarity index measure, classification accuracy of digital images, and training and testing error curves.

RESULTS

The reconstruction accuracy was improved for the case in which the diffuser was imaged, compared to the case where the object was imaged. The training and testing error curves show that the loss converged to lower values in fewer epochs when the diffuser was imaged.

CONCLUSIONS

The proposed approach demonstrates an improvement in the accuracy of the reconstruction of objects hidden through glass diffusers by imaging glass diffuser surfaces, and can be applied to objects at unknown locations in a scattering medium.

Invasive and Non-Invasive Observation of Occluded Fast Transient Events: Computational Tools

Industrial processes involving thermal plasma such as cutting, welding, laser machining with ultra-short laser pulses (nonequilibrium conditions), high temperature melting using electrical discharge or ion-beams, etc., generate non-repeatable fast transient events which can reveal valuable information about the processes. In such industrial environments containing high temperature and radiation, it is often difficult to install conventional lens-based imaging windows and components to observe such events. In this study, we compare imaging requirements and performances with invasive and non-invasive modes when a fast transient event is occluded by a metal window consisting of numerous holes punched through it. Simulation studies were carried out for metal windows with different types of patterns, reconstructed for both invasive and non-invasive modes and compared. Sparks were generated by rapid electrical discharge behind a metal window consisting of thousands of punched through-holes and the time sequence was recorded using a high-speed camera. The time sequence was reconstructed with and without the spatio-spectral point spread functions and compared. Commented MATLAB codes are provided for both invasive and non-invasive modes of reconstruction.

Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect

A speckle image formed by scattering lights can be decoded by recently invented techniques, owing to the optical memory effect, thereby enabling the observation of a hidden object behind a thin scattering medium. However, the range of three-dimensional OME is typically small; therefore, both the field of view and depth of field are limited. We propose a method that can significantly and simultaneously improve both values for a specific scenario, where one object moves around the other position-fixed object. The effectiveness of the proposed scheme is demonstrated through a set of experiments.

Spectral speckle-correlation imaging

We present a method for single-shot spectrally resolved imaging through scattering media by using the spectral memory effect of speckles. In our method, a single speckle pattern from a multi-colored object is captured through scattering media with a monochrome image sensor. The color object is recovered by correlation of the captured speckle and a three-dimensional phase retrieval process. The proposed method was experimentally demonstrated by using point sources with different emission spectra located between diffusers. This study paves the way for non-invasive and low-cost spectral imaging through scattering media.

Three-dimensional broadband light beam manipulation in forward scattering samples

Focusing light into highly disordered biological tissue is a major challenge in optical microscopy and biomedical imaging due to scattering. However, correlations in the scattering matrix, known as "memory effects", can be used to improve imaging capabilities. Here we discuss theoretically and numerically the possibility to achieve three-dimensional ultrashort laser focusing and scanning inside forward scattering media, beyond the scattering mean free path, by simultaneously taking advantage of the angular and the chromato-axial memory effects. The numerical model is presented in details, is validated within the state of the art theoretical and experimental framework and is finally used to propose a scheme for focusing ultra-short laser pulses in depth through forward scattering media.

Non-invasive optical imaging using the extension of the Fourier-domain shower-curtain effect

Optical imaging for non-self-luminous objects surrounded by complex scattering environments is scientifically challenging and technologically important. We propose a non-invasive imaging method by externally sending the illuminating light through the scattering medium and by detecting and analyzing the speckle patterns. The imaging of the object is recovered by extending the application scope of the Fourier-domain shower-curtain effect. It is found that the imaging depth is substantially extended and that faster imaging restoration is realized with the improved illumination scheme assisted with optical lenses, hence making it possible to apply the non-invasive optical imaging technique for practical applications.

Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect

Strong scattering medium brings great difficulties to image objects. Optical memory effect makes it possible to image through strong random scattering medium in a limited angle field-of-view (FOV). The limitation of FOV results in a limited optical memory effect range, which prevents the optical memory effect to be applied to real imaging applications. In this paper, a kind of practical convolutional neural network called PDSNet (Pragmatic De-scatter ConvNet) is constructed to image objects hidden behind different scattering media. The proposed method can expand at least 40 times of the optical memory effect range with a average PSNR above 24dB, and enable to image complex objects in real time, even for objects with untrained scales. The provided experiments can verify its accurateness and efficiency.

A compact single channel interferometer to study vortex beam propagation through scattering layers

We propose and demonstrate a single channel interferometer that can be used to study how vortex beams propagate through a scatterer. The interferometer consists of a multifunctional diffractive optical element (MDOE) synthesized by the spatial random multiplexing of a Fresnel zone plate and a spiral Fresnel zone plate with different focal lengths. The MDOE generates two co-propagating beams, such that only the beam carrying orbital angular momentum is modulated by an annular stack of thin scatterers located at the focal plane of the Fresnel zone plate, while the other beam passes through the centre of the annulus without any modulation. The interference pattern is recorded at the focal plane of the spiral Fresnel zone plate. The scattering of vortex beams through stacks consisting of different number of thin scatterers was studied using the proposed optical setup. Conflicting results have been reported earlier on whether higher or lower charge beams suffer more deterioration. The proposed interferometer provides a relatively simple and compact means of experimentally studying propagation of vortex beams through scattering medium.

Spatial light modulator aided noninvasive imaging through scattering layers

We propose and demonstrate a new imaging technique to noninvasively see through scattering layers with the aid of a spatial light modulator (SLM). A relay system projects the incoherent light pattern emitting from the scattering layer onto the SLM. Two coded phase masks are displayed, one after another, on the SLM to modulate the projected scattered field and the two corresponding intensity patterns are recorded by a digital camera. The above procedure helps to achieve two goals. Firstly, since the coded phase masks are digitally synthesized, the point spread function of the imaging system can be engineered such that the image retrieval becomes more reliable. Secondly, the two recorded intensity patterns are subtracted one from the other and by that the background noise of the recovered image is minimized. The above two advantages along with a modified phase retrieval algorithm enable a relatively easier and accurate convergence to the image of the covered object.

Focusing large spectral bandwidths through scattering media

Wavefront shaping is a powerful method to refocus light through a scattering medium. Its application to large spectral bandwidths or multiple wavelengths refocusing for nonlinear bio-imaging in-depth is however limited by spectral decorrelations. In this work, we demonstrate ways to access a large spectral memory of a refocus in thin scattering media and thick forward-scattering biological tissues. First, we show that the accessible spectral bandwidth through a scattering medium involves an axial spatio-spectral coupling, which can be minimized when working in a confocal geometry. Second, we show that this bandwidth can be further enlarged when working in a broadband excitation regime. These results open important prospects for multispectral nonlinear imaging through scattering media.

Multiple-image double-encryption via 2D rotations of a random phase mask with spatially incoherent illumination

Optical image encryption technique has become extremely important in these years. However, most of the proposed multiple-image encryption systems are illuminated with coherent light source. Here we present a multiple-image double-encryption method with spatially incoherent illumination. The first-encryption of multiple images is based on the speckle rotation decorrelation property, and the second-encryption of images' order is based on the speckle shift decorrelation out of the angular memory-effect range. The double-encryption via two-dimensional rotations of the random phase mask enhances the security and keeps the simplicity of the cryptosystem. The capacity of the ciphertext is greatly increased by multiplexing, and further increased after crosstalk noise removal. The use of incoherent light source reduces the requirements for experimental conditions, and makes the cryptosystem easy to implement in various application scenarios.

Depth-resolved and auto-focus imaging through scattering layer with wavelength compensation

Imaging techniques through turbid materials have been extensively studied in recent years. The challenge now is to recover objects in a large field of view with depth-resolving ability. We present a method to image through a thin scattering layer automatically with the depth of the object detectable. By revealing the wavelength-depth-matching relation based on the axial memory effect, this method can automatically search the optimal wavelength of the reference light and compute the depth of the object. The no-reference image quality assessment function and rule-based searching algorithm are used in the searching process. The proposed method is promising for dynamic object tracking.