Single-shot large field of view imaging with scattering media by spatial demultiplexing

Widefield functional speckle-correlation optical scattering mesoscopy toward hemodynamic imaging

Speckle-correlation optical scattering imaging (SCOSI) has shown the potential for non-invasive biomedical diagnostic applications, which directly utilizes the scattering patterns to reconstruct the deep and non-line-of-sight objects. However, the course of the translation of this technique to preclinical biomedical imaging applications has been postponed by the following two facts: 1) the field of view of SCOSI was significantly limited by the optical memory effect, and 2) the molecular-tagged functional imaging of the biological tissues remains largely unexplored. In this work, a proof-of-concept design of the first-generation widefield functional SCOSI (WF-SCOSI) system was presented for simultaneously achieving mesoscopic mapping of fluid morphology and flow rate, which was realized by implementing the concepts of scanning synthesis and fluorescence scattering flowmetry. The ex vivo imaging results of the fluorescence-labeled large-scale blood vessel network phantom underneath the strong scatters demonstrated the effectiveness of WF-SCOSI toward non-invasive hemodynamic imaging applications.

High-throughput imaging through dynamic scattering media based on speckle de-blurring

Effectively imaging through dynamic scattering media is of great importance and challenge. Some imaging methods based on physical or learning models have been designed for object reconstruction. However, with an increase in exposure time or more drastic changes in the scattering medium, the speckle pattern superimposed during camera integration time undergoes more significant changes, resulting in a modification of the collected speckle structure and increased blurring, which brings significant challenges to the reconstruction. Here, the clearer structural information of blurred speckles is unearthed with a presented speckle de-blurring algorithm, and a high-throughput imaging method through rapidly changing scattering media is proposed for reconstruction under long exposure. For the problem of varying blur degrees in different regions of the speckle, a block-based method is proposed to divide the speckle into distinct sub-speckles, which can realize the reconstruction of hidden objects. The imaging of hidden objects with different complexity through dynamic scattering media is demonstrated, and the reconstruction results are improved significantly for speckles with different blur degrees, which verifies the effectiveness of the method. This method is a high-throughput approach that enables non-invasive imaging solely through the collection of a single speckle. It directly operates on blurred speckles, making it suitable for traditional speckle-correlation methods and deep learning (DL) methods. This provides a new way of thinking about solving practical scattering imaging challenges.

Multi-target object scattering imaging with intensity correlation of structured illumination

Imaging through scattering layers based on the optical memory effect (OME) concept has been widely investigated in recent years. Among many scattering scenarios, it is very important to recover hidden targets with proper spatial distribution in the scene where multiple targets out of the OME range exist. In this Letter, we put forward a method for multi-target object scattering imaging. With the help of intensity correlation between the structured illumination patterns and recorded speckle images, the relative position of all hidden targets can be obtained and the movement of the targets within the OME range can be tracked. We experimentally implement scattering imaging with 16 targets and the motion tracking of them. Our results present a significant advance in a large field of view scattering imaging with multiple targets.

Speckle autocorrelation separation for multi-target scattering imaging

Imaging through scattering media remains a big challenge in optics while the single-shot non-invasive speckle autocorrelation technique (SAT) is well-known as a promising way to handle it. However, it usually cannot recover a large-scale target or multiple isolated small ones due to the limited effective range of the optical memory effect (OME). In this paper, we propose a multi-target scattering imaging scheme by combining the traditional SA algorithm with a Deep Learning (DL) strategy. The basic idea is to extract each autocorrelation component of every target from the autocorrelation result of a mixed speckle using a suitable DL method. Once we get all the expected autocorrelation components, a typical phase retrieval algorithm (PRA) could be applied to reveal the shapes of all those corresponding small targets. In our experimental demonstration, up to five isolated targets are successfully recovered.

Locating through dynamic scattering media based on speckle correlations

In complex imaging settings, optical scattering often prohibits the formation of a clear target image, and instead, only a speckle without the original spatial structure information is obtained. Scattering seriously interferes with the locating of targets; especially, when the scattering medium is dynamic, the dynamic nature leads to rapid decorrelation of optical information in time, and the challenge increases. Here, a locating method is proposed to detect the target hidden behind a dynamic scattering medium, which uses the a priori information of a known reference object in the neighborhood of the target. The research further designs an automatic calibration method to simplify the locating process, and analyzes the factors affecting positioning accuracy. The proposed method enables us to predict the position of a target from the autocorrelation of the captured speckle pattern; the angle and distance deviations of the target are all within 2.5%. This approach can locate a target using only a single-shot speckle pattern, and it is beneficial for target localization in dynamic scattering conditions.

Non-invasive imaging through scattering medium and around corners beyond 3D memory effect

The three-dimensional (3D) memory effect (ME) has been shown to exist in a variety of scattering scenes. Limited by the scope of ME, speckle correlation technology only can be applied in a small imaging field of view (FOV) with a small depth of field (DOF). In this Letter, an untrained neural network is constructed and used as an optimization tool to restore the targets beyond the 3D ME range. The autocorrelation consistency relationship and the generative adversarial strategy are combined. Only single frame speckle and unaligned real targets are needed for online optimization; therefore, the neural network does not need to train in advance. Furthermore, the proposed method does not need to conduct additional modulation for the system. This method can reconstruct not only hidden targets behind the scattering medium, but also targets around corners. The combination strategy of the generative adversarial framework with physical priors used to decouple the aliasing information and reconstruct the target will provide inspiration for the field of computational imaging.

Efficient color imaging through unknown opaque scattering layers via physics-aware learning

Color imaging with scattered light is crucial to many practical applications and becomes one of the focuses in optical imaging fields. More physics theories have been introduced in the deep learning (DL) approach for the optical tasks and improve the imaging capability a lot. Here, an efficient color imaging method is proposed in reconstructing complex objects hidden behind unknown opaque scattering layers, which can obtain high reconstruction fidelity in spatial structure and accurate restoration in color information by training with only one diffuser. More information is excavated by utilizing the scattering redundancy and promotes the physics-aware DL approach to reconstruct the color objects hidden behind unknown opaque scattering layers with robust generalization capability by an efficient means. This approach gives impetus to color imaging through dynamic scattering media and provides an enlightening reference for solving complex inverse problems based on physics-aware DL methods.

Non-line-of-sight imaging under white-light illumination: a two-step deep learning approach

Non-line-of-sight (NLOS) imaging has received considerable attentions for its ability to recover occluded objects from an indirect view. Various NLOS imaging techniques have been demonstrated recently. Here, we propose a white-light NLOS imaging method that is equipped only with an ordinary camera, and not necessary to operate under active coherent illumination as in other existing NLOS systems. The central idea is to incorporate speckle correlation-based model into a deep neural network (DNN), and form a two-step DNN strategy that endeavors to learn the optimization of the scattered pattern autocorrelation and object image reconstruction, respectively. Optical experiments are carried out to demonstrate the proposed method.

Single-shot imaging through scattering media under strong ambient light interference

Speckle correlation imaging (SCI) has found tremendous versatility compared with other scattering imaging approaches due to its single-shot data acquisition strategy, relatively simple optical setup, and high-fidelity reconstruction performance. However, this simplicity requires SCI experiments to be performed strictly in a darkroom condition. As background noise increases, the speckle contrast rapidly decreases, making precise interpretation of the data extremely difficult. Here, we demonstrate a method by refining the speckle in the autocorrelation domain to achieve high-performance single-shot imaging. Experiment results prove that our method is adapted to estimate objects in a low signal-to-background ratio (SBR) circumstance even if the SBR is about -23dB. Laboratory and outdoor SCI experiments are performed.

Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect

A speckle image formed by scattering lights can be decoded by recently invented techniques, owing to the optical memory effect, thereby enabling the observation of a hidden object behind a thin scattering medium. However, the range of three-dimensional OME is typically small; therefore, both the field of view and depth of field are limited. We propose a method that can significantly and simultaneously improve both values for a specific scenario, where one object moves around the other position-fixed object. The effectiveness of the proposed scheme is demonstrated through a set of experiments.

Non-invasive super-resolution imaging through dynamic scattering media

Super-resolution imaging has been revolutionizing technical analysis in various fields from biological to physical sciences. However, many objects are hidden by strongly scattering media such as biological tissues that scramble light paths, create speckle patterns and hinder object’s visualization, let alone super-resolution imaging. Here, we demonstrate non-invasive super-resolution imaging through scattering media based on a stochastic optical scattering localization imaging (SOSLI) technique. After capturing multiple speckle patterns of photo-switchable point sources, our computational approach utilizes the speckle correlation property of scattering media to retrieve an image with a 100-nm resolution, an eight-fold enhancement compared to the diffraction limit. More importantly, we demonstrate our SOSLI to do non-invasive super-resolution imaging through not only static scattering media, but also dynamic scattering media with strong decorrelation such as biological tissues. Our approach paves the way to non-invasively visualize various samples behind scattering media at nanometer levels of detail.

The authors introduce stochastic optical scattering localization imaging (SOSLI) for non-invasive super-resolution imaging through scattering media. They capture multiple speckle patterns of photo-switchable point sources and use the speckle correlation to retrieve images with 100 nm resolution.

Multitarget imaging through scattering media beyond the 3D optical memory effect

A robust method for efficient spatial separation optical imaging through scattering media regardless of the three-dimensional (3D) optical memory effect is proposed. In this method, the problems of imaging dealiasing, decomposition, and separation of speckle patterns are solved by employing independent component analysis. Multitarget imaging behind a scattering layer with diverse spatial positions is observed experimentally, for the first time, to the best of our knowledge. In this work, we demonstrate that, by knowing the number of targets and keeping each subtargets' size in the optical memory effect range while isolating them beyond this range without overlap in the axial direction, speckle dealiasing and multitarget imaging are solved effectively. The strategy provides a potentially useful means for incoherent imaging through scattering media in a wide class of fields such as optical microscopy, biomedical imaging, and astronomical imaging.

Achromatic metasurface doublet with a wide incident angle for light focusing

Benefiting from the excellent capabilities of arbitrarily controlling the phase, amplitude and polarization of the electromagnetic wave, metasurfaces have attracted much attention and brought forward the revolution of fields ranging from device fabrications to optical applications. Cascaded metasurfaces have been demonstrated to correct the monochromatic aberration and enable a near-diffraction-limited focusing spot over a wide incident angle. However, they can only work under the design wavelength and suffer from the axial chromatic aberration at another wavelength. Here, an achromatic metasurface doublet is proposed to eliminate the axial achromatic aberration and enable high-quality focusing with a wide incident angle at distinct wavelengths. It consists of square nanopillar arrays with spatially varying width to simultaneously realize wavelength-dependent phase controls. The constructed metasurface doublet is further verified numerically and near-diffraction-limited foci are exactly formed at the same plane with an incident angle up to 20° for design wavelengths. We expect that our proposed approach can find optical applications in the fields of holograms, photograms, microscopy and machine vision.

Extending the imaging range through scattering layers to the entire correlation range

A method of extending the imaging range through scattering layers around a reference point (RP) is realized. Objects within the entire correlation range of the RP can be totally recovered. By scanning the light source, objects within the memory effect (ME) range of the RP are completely recovered with high quality. By combining the shift of a camera to move the object to the center of observation view, objects far away from the RP are retrieved with an improved signal-to-noise ratio. The extended imaging range is about 3.5 times the ME range and more than 16 times the imaging range with normal static illumination. The RP can be imprecisely placed at a distance from the objects instead of precisely replacing them owing to the extended imaging range. This simple-system method forcefully breaks the limitation of the ME range and is very easy to implement in practical applications, which is meaningful for the research in scattering imaging.

Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect

Strong scattering medium brings great difficulties to image objects. Optical memory effect makes it possible to image through strong random scattering medium in a limited angle field-of-view (FOV). The limitation of FOV results in a limited optical memory effect range, which prevents the optical memory effect to be applied to real imaging applications. In this paper, a kind of practical convolutional neural network called PDSNet (Pragmatic De-scatter ConvNet) is constructed to image objects hidden behind different scattering media. The proposed method can expand at least 40 times of the optical memory effect range with a average PSNR above 24dB, and enable to image complex objects in real time, even for objects with untrained scales. The provided experiments can verify its accurateness and efficiency.

Single-shot multi-view imaging enabled by scattering lens

Imaging three-dimensional (3D) objects has been realized by methods such as binocular stereo vision and multi-view imaging. These methods, however, needs multiple cameras or multiple shots to get elemental images. In this paper, we develop a single-shot multi-view imaging technique by utilizing the natural randomness of scattering media. By exploiting the memory effect and uncorrelated point spread functions (PSF) among scattering media, we demonstrate that both stereo imaging with large disparity and up to seven-view imaging of a 3D object can be reconstructed from only one speckle pattern by deconvolution. The elemental images are consistent with 3D object projection and images taken by multi-shot imaging. Our technique provides a feasible method to capture multi-view imaging with short acquisition time and easy calibration.

Fast 3D movement of a laser focusing spot behind scattering media by utilizing optical memory effect and optical conjugate planes

Controlling light propagation intentionally through turbid media such as ground glass or biological tissue has been demonstrated for many useful applications. Due to random scattering effect, one of the important goals is to draw a desired shape behind turbid media with a swift and precise method. Feedback wavefront shaping method which is known as a very effective approach to focus the light, is restricted by slow optimization process for obtaining multiple spots. Here we propose a technique to implement feedback wavefront shaping with optical memory effect and optical 4f system to speedy move focus spot and form shapes in 3D space behind scattering media. Starting with only one optimization process to achieve a focusing spot, the advantages of the optical configuration and full digital control allow us to move the focus spot with high quality at the speed of SLM frame rate. Multiple focusing spots can be achieved simultaneously by combining multiple phase patterns on a single SLM. By inheriting the phase patterns in the initial focusing process, we can enhance the intensity of the focusing spot at the edge of memory effect in with 50% reduction in optimization time. With a new focusing spot, we have two partially overlapped memory effect regions, expanding our 3D scanning range. With fast wavefront shaping devices, our proposed technique could potentially find appealing applications with biological tissues.