Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media

Memory-less scattering imaging with ultrafast convolutional optical neural networks

The optical memory effect in complex scattering media including turbid tissue and speckle layers has been a critical foundation for macroscopic and microscopic imaging methods. However, image reconstruction from strong scattering media without the optical memory effect has not been achieved. Here, we demonstrate image reconstruction through scattering layers where no optical memory effect exists, by developing a multistage convolutional optical neural network (ONN) integrated with multiple parallel kernels operating at the speed of light. Training this Fourier optics-based, parallel, one-step convolutional ONN with the strong scattering process for direct feature extraction, we achieve memory-less image reconstruction with a field of view enlarged by a factor up to 271. This device is dynamically reconfigurable for ultrafast multitask image reconstruction with a computational power of 1.57 peta-operations per second (POPS). Our achievement establishes an ultrafast and high energy-efficient optical machine learning platform for graphic processing.

Non-invasive accelerated imaging through a scattering medium via multi-stage complexity guidance

The research of scattering imaging is of great significance to the development of various fields, but the existing scattering imaging methods are difficult to combine for the advantages of non-invasiveness, real-time imaging, and high quality. In this paper, a new, to our knowledge, scattering imaging technique is proposed that optimizes the traditional autocorrelation imaging technique by multi-stage complexity guidance and the initial acceleration module. We introduce the complexity difference index into the phase iterative recovery step for effective complexity guidance, and add the initial module based on error-reduction iteration to realize a fast startup. A series of experiments is carried out to test the performance of the new technique. The results show that the proposed technique significantly improves the scattering reconstruction speed. Meanwhile, the accuracy and clarity of the reconstructed image are significantly higher than the traditional method in terms of fast imaging. Moreover, this technique has better robustness to noise compared to the traditional autocorrelation imaging technique. The experimental code for this paper is available on GitHub.

Adaptive imaging through dense dynamic scattering media using transfer learning

Imaging through scattering media is a long-standing challenge in optical imaging, holding substantial importance in fields like biology, transportation, and remote sensing. Recent advancements in learning-based methods allow accurate and rapid imaging through optically thick scattering media. However, the practical application of data-driven deep learning faces substantial hurdles due to its inherent limitations in generalization, especially in scenarios such as imaging through highly non-static scattering media. Here we utilize the concept of transfer learning toward adaptive imaging through dense dynamic scattering media. Our approach specifically involves using a known segment of the imaging target to fine-tune the pre-trained de-scattering model. Since the training data of downstream tasks used for transfer learning can be acquired simultaneously with the current test data, our method can achieve clear imaging under varying scattering conditions. Experiment results show that the proposed approach (with transfer learning) is capable of providing more than 5dB improvements when optical thickness varies from 11.6 to 13.1 compared with the conventional deep learning approach (without transfer learning). Our method holds promise for applications in video surveillance and beacon guidance under dense dynamic scattering conditions.

Acousto-optic modulator-based improvement in imaging through scattering media

Reduced visibility is a common problem when light traverses through a scattering medium, and it becomes difficult to identify an object in such scenarios. What we believe to be a novel proof-of-principle technique for improving image visibility based on the quadrature lock-in discrimination algorithm in which the demodulation is performed using an acousto-optic modulator is presented here. A significant improvement in image visibility is achieved using a series of frames. We have also performed systematic imaging by varying the camera parameters, such as exposure time, frame rate, and series length, to investigate their effect on enhancing image visibility.

Imaging through scattering media under strong ambient light interference via the lock-in process

Scattered light imaging techniques leveraging memory effects have been extensively investigated, yet most approaches are limited to operating in predominantly dark environments. The introduction of additional optical noise disrupts the fine structure of the original speckle pattern, undermining spatial correlation and resulting in imaging failure. In this study, we present a high-performance imaging method that integrates a lock-in process to overcome this limitation. Our experimental results demonstrate that the proposed technique enables successful imaging of targets in low signal-to-background ratio (SBR) environments, even at SBR levels as low as -28.0 dB. Furthermore, the method allows for the directional separation of targets with distinct modulation frequencies. This innovative approach has the potential to significantly expand the applicability of scattering imaging techniques by eliminating the constraints of dark field environments, thereby enhancing the convenience of in vivo microscopy and daytime astronomical observations.

Enhanced secrecy in optical communication using speckle from multiple scattering layers

We study the secrecy of an optical communication system with two scattering layers, to hide both the sender and receiver, by measuring the correlation of the intermediate speckle generated between the two layers. The binary message is modulated as spatially shaped wavefronts, and the high number of transmission modes of the scattering layers allows for many uncorrelated incident wavefronts to send the same message, making it difficult for an attacker to intercept or decode the message and thus increasing secrecy. We collect 50,000 intermediate speckle patterns and analyze their correlation distribution using the Kolmogorov-Smirnov (K-S) test. We search for further correlations using the K-Means and Hierarchical unsupervised classification algorithms. We find no correlation between the intermediate speckle and the message, suggesting a person-in-the-middle attack is not possible. This method is compatible with any digital encryption method and is applicable for codifications in optical wireless communication (OWC).

Central wavelength estimation in spectral imaging behind a diffuser via deep learning

Multispectral imaging through scattering media is an important practical issue in the field of sensing. The light from a scattering medium is expected to carry information about the spectral properties of the medium, as well as geometrical information. Because spatial and spectral information of the object is encoded in speckle images, the information about the structure and spectrum of the object behind the scattering medium can be estimated from those images. Here we propose a deep learning-based strategy that can estimate the central wavelength from speckle images captured with a monochrome camera. When objects behind scattering media are illuminated with narrowband light having different spectra with different spectral peaks, deep learning of speckle images acquired at different central wavelengths can extend the spectral region to reconstruct images and estimate the central wavelengths of the illumination light. The proposed method achieves central wavelength estimation in 1 nm steps for objects whose central wavelength varies in a range of 100 nm. Because our method can achieve image reconstruction and central wavelength estimation in a single shot using a monochrome camera, this technique will pave the way for multispectral imaging through scattering media.

Imaging and positioning through scattering media with double-helix point spread function engineering

Significance

Double-helix point spread function (DH-PSF) microscopy has been developed for three-dimensional (3D) localization and imaging at super-resolution but usually in environments with no or weak scattering. To date, super-resolution imaging through turbid media has not been reported.

Aim

We aim to explore the potential of DH-PSF microscopy in the imaging and localization of targets in scattering environments for improved 3D localization accuracy and imaging quality.

Approach

The conventional DH-PSF method was modified to accommodate the scanning strategy combined with a deconvolution algorithm. The localization of a fluorescent microsphere is determined by the center of the corresponding double spot, and the image is reconstructed from the scanned data by deconvoluting the DH-PSF.

Results

The resolution, i.e., the localization accuracy, was calibrated to 13 nm in the transverse plane and 51 nm in the axial direction. Penetration thickness could reach an optical thickness (OT) of 5. Proof-of-concept imaging and the 3D localization of fluorescent microspheres through an eggshell membrane and an inner epidermal membrane of an onion are presented to demonstrate the super-resolution and optical sectioning capabilities.

Conclusions

Modified DH-PSF microscopy can image and localize targets buried in scattering media using super-resolution. Combining fluorescent dyes, nanoparticles, and quantum dots, among other fluorescent probes, the proposed method may provide a simple solution for visualizing deeper and clearer in/through scattering media, making in situ super-resolution microscopy possible for various demanding applications.

All-optical image classification through unknown random diffusers using a single-pixel diffractive network

Classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. These methods demand relatively large-scale computing using deep neural networks running on digital computers. Here, we present an all-optical processor to directly classify unknown objects through unknown, random phase diffusers using broadband illumination detected with a single pixel. A set of transmissive diffractive layers, optimized using deep learning, forms a physical network that all-optically maps the spatial information of an input object behind a random diffuser into the power spectrum of the output light detected through a single pixel at the output plane of the diffractive network. We numerically demonstrated the accuracy of this framework using broadband radiation to classify unknown handwritten digits through random new diffusers, never used during the training phase, and achieved a blind testing accuracy of 87.74 ± 1.12%. We also experimentally validated our single-pixel broadband diffractive network by classifying handwritten digits "0" and "1" through a random diffuser using terahertz waves and a 3D-printed diffractive network. This single-pixel all-optical object classification system through random diffusers is based on passive diffractive layers that process broadband input light and can operate at any part of the electromagnetic spectrum by simply scaling the diffractive features proportional to the wavelength range of interest. These results have various potential applications in, e.g., biomedical imaging, security, robotics, and autonomous driving.

Speckle autocorrelation separation for multi-target scattering imaging

Imaging through scattering media remains a big challenge in optics while the single-shot non-invasive speckle autocorrelation technique (SAT) is well-known as a promising way to handle it. However, it usually cannot recover a large-scale target or multiple isolated small ones due to the limited effective range of the optical memory effect (OME). In this paper, we propose a multi-target scattering imaging scheme by combining the traditional SA algorithm with a Deep Learning (DL) strategy. The basic idea is to extract each autocorrelation component of every target from the autocorrelation result of a mixed speckle using a suitable DL method. Once we get all the expected autocorrelation components, a typical phase retrieval algorithm (PRA) could be applied to reveal the shapes of all those corresponding small targets. In our experimental demonstration, up to five isolated targets are successfully recovered.

cGAN-assisted imaging through stationary scattering media

Analyzing images taken through scattering media is challenging, owing to speckle decorrelations from perturbations in the media. For in-line imaging modalities, which are appealing because they are compact, require no moving parts, and are robust, negating the effects of such scattering becomes particularly challenging. Here we explore the use of conditional generative adversarial networks (cGANs) to mitigate the effects of the additional scatterers in in-line geometries, including digital holographic microscopy. Using light scattering simulations and experiments on objects of interest with and without additional scatterers, we find that cGANs can be quickly trained with minuscule datasets and can also efficiently learn the one-to-one statistical mapping between the cross-domain input-output image pairs. Importantly, the output images are faithful enough to enable quantitative feature extraction. We also show that with rapid training using only 20 image pairs, it is possible to negate this undesired scattering to accurately localize diffraction-limited impulses with high spatial accuracy, therefore transforming a shift variant system to a linear shift invariant (LSI) system.

Investigation of image plane for image reconstruction of objects through diffusers via deep learning

SIGNIFICANCE

The imaging of objects hidden in light-scattering media is a vital practical task in a wide range of applications, including biological imaging. Deep-learning-based methods have been used to reconstruct images behind scattering media under complex scattering conditions, but improvements in the quality of the reconstructed images are required.

AIM

To investigate the effect of image plane on the accuracy of reconstructed images.

APPROACH

Light reflected from an object passing through glass diffusers is captured by changing the image plane of an optical imaging system. Images are reconstructed by deep learning, and evaluated in terms of structural similarity index measure, classification accuracy of digital images, and training and testing error curves.

RESULTS

The reconstruction accuracy was improved for the case in which the diffuser was imaged, compared to the case where the object was imaged. The training and testing error curves show that the loss converged to lower values in fewer epochs when the diffuser was imaged.

CONCLUSIONS

The proposed approach demonstrates an improvement in the accuracy of the reconstruction of objects hidden through glass diffusers by imaging glass diffuser surfaces, and can be applied to objects at unknown locations in a scattering medium.

Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect by speckle-based difference strategy

The shape of two objects hidden behind a thin scattering medium is retrieved by the presented method. One of the two objects keeps stationary, while the other one is supposed to be gradually moving, and the Euclidean distance between them is always beyond the range of the 3D optical memory effect. We capture two speckle patterns to image the two isolated objects by using a developed speckle-differential-based strategy and the traditional speckle autocorrelation technique. The feasibility of our method is demonstrated by theoretical analysis and a set of experiments.

Simultaneously improving multiple imaging parameters with scattering media

Traditional optical imaging systems can provide high-quality imaging with a complicated and expensive optical design by eliminating aberrations. With the help of an optical memory effect, rather than independently improving a single imaging parameter, the simultaneous improvement of several imaging parameters by adding scattering media to the imaging systems is, to the best of our knowledge, demonstrated for the first time. As an example, in a simple single lens imaging system, in addition to the depth of field being greatly improved, spherical aberration, coma aberration, and chromatic aberration are simultaneously eliminated by placing a scattering medium between the lens and the camera. The results indicate the potential applications of scattering media in many fields such as optical imaging, optical measurements, and biomedical applications.

Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect

A speckle image formed by scattering lights can be decoded by recently invented techniques, owing to the optical memory effect, thereby enabling the observation of a hidden object behind a thin scattering medium. However, the range of three-dimensional OME is typically small; therefore, both the field of view and depth of field are limited. We propose a method that can significantly and simultaneously improve both values for a specific scenario, where one object moves around the other position-fixed object. The effectiveness of the proposed scheme is demonstrated through a set of experiments.

Non-invasive super-resolution imaging through dynamic scattering media

Super-resolution imaging has been revolutionizing technical analysis in various fields from biological to physical sciences. However, many objects are hidden by strongly scattering media such as biological tissues that scramble light paths, create speckle patterns and hinder object’s visualization, let alone super-resolution imaging. Here, we demonstrate non-invasive super-resolution imaging through scattering media based on a stochastic optical scattering localization imaging (SOSLI) technique. After capturing multiple speckle patterns of photo-switchable point sources, our computational approach utilizes the speckle correlation property of scattering media to retrieve an image with a 100-nm resolution, an eight-fold enhancement compared to the diffraction limit. More importantly, we demonstrate our SOSLI to do non-invasive super-resolution imaging through not only static scattering media, but also dynamic scattering media with strong decorrelation such as biological tissues. Our approach paves the way to non-invasively visualize various samples behind scattering media at nanometer levels of detail.

The authors introduce stochastic optical scattering localization imaging (SOSLI) for non-invasive super-resolution imaging through scattering media. They capture multiple speckle patterns of photo-switchable point sources and use the speckle correlation to retrieve images with 100 nm resolution.

Complex wavefront sensing based on coherent diffraction imaging using vortex modulation

Phase retrieval seeks to reconstruct the phase from the measured intensity, which is an ill-posed problem. A phase retrieval problem can be solved with physical constraints by modulating the investigated complex wavefront. Orbital angular momentum has been recently employed as a type of reliable modulation. The topological charge l is robust during propagation when there is atmospheric turbulence. In this work, topological modulation is used to solve the phase retrieval problem. Topological modulation offers an effective dynamic range of intensity constraints for reconstruction. The maximum intensity value of the spectrum is reduced by a factor of 173 under topological modulation when l is 50. The phase is iteratively reconstructed without a priori knowledge. The stagnation problem during the iteration can be avoided using multiple topological modulations.

Complex wavefront sensing based on alternative structured phase modulation

Spatial light modulators (SLMs), which generate varying phase modulation, are widely used in coherent diffraction imaging. Random patterns are uploaded on the SLM to modulate the measured wavefront. However, a random pattern is highly complex and requires a reliable SLM. In addition, the uncorrelated terms generated from the random modulations need to be sufficiently captured using an imaging sensor with a high signal-to-noise ratio (SNR) to avoid stagnation during iterations. We propose an alternative structured phase modulation (ASPM) method. The modulations are composed of orthogonally placed phase bars that introduce uncorrelated modulations. The ASPM modulation can act as the phase grating; in addition, the modulated intensities are concentrated, which can be captured with a high SNR. The complexity of the ASPM patterns is significantly reduced, which is helpful for utilizing the SLM to generate reliable phase modulation.

Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network

Coherent imaging through scatter is a challenging task. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach can make high-quality and highly generalizable predictions through unseen diffusers. Here, we propose a new deep neural network model that is agnostic to a broader class of perturbations including scatterer change, displacements, and system defocus up to 10× depth of field. In addition, we develop a new analysis framework for interpreting the mechanism of our deep learning model and visualizing its generalizability based on an unsupervised dimension reduction technique. We show that our model can unmix the scattering-specific information and extract the object-specific information and achieve generalization under different scattering conditions. Our work paves the way to a robust and interpretable deep learning approach to imaging through scattering media.

Non-invasive single-shot recovery of a point-spread function of a memory effect based scattering imaging system

Accessing the point-spread function (PSF) of a complex optical system is important for a variety of imaging applications. However, placing an invasive point source is often impractical, and estimating it blindly with multiple frames is slow and requires a complex nonlinear optimization. Here, we introduce a simple single-shot method to noninvasively recover the accurate PSF of an isoplanatic imaging system, in the context of multiple light scattering. Our approach is based on the reconstruction of any unknown sparse hidden object using the autocorrelation imaging technique, followed by a deconvolution with a blur kernel derived from the statistics of a speckle pattern. A deconvolution on the camera image then retrieves the accurate PSF of the system, enabling further imaging applications. We demonstrate numerically and experimentally the effectiveness of this approach compared to previous deconvolution techniques.

Imaging through opacity using a near-infrared low-spatial-coherence fiber light source

Memory-effect-based speckle correlation is one of the most practical techniques for imaging through scattering opaque media, where a light source with low spatial coherence and moderate bandwidth plays a pivotal role. Usually, a rapidly rotating diffuser is applied to make the light source spatially decoherent. Here, an all-fiber-based low-spatial-coherence light source is proposed and demonstrated for such speckle-correlated imaging. The illumination structure is greatly simplified, the lightening efficiency is enhanced, and the wavelength is extended to the near-infrared band, which is favorable for a larger memory effect range and deeper penetrating depth through opacity. Moreover, the proposed local illumination method can identify the orientation of the object, which has not been revealed by former methods. This work would facilitate the research in optical biomedical imaging and broaden the applications of multimode random fiber lasers.

Multitarget imaging through scattering media beyond the 3D optical memory effect

A robust method for efficient spatial separation optical imaging through scattering media regardless of the three-dimensional (3D) optical memory effect is proposed. In this method, the problems of imaging dealiasing, decomposition, and separation of speckle patterns are solved by employing independent component analysis. Multitarget imaging behind a scattering layer with diverse spatial positions is observed experimentally, for the first time, to the best of our knowledge. In this work, we demonstrate that, by knowing the number of targets and keeping each subtargets' size in the optical memory effect range while isolating them beyond this range without overlap in the axial direction, speckle dealiasing and multitarget imaging are solved effectively. The strategy provides a potentially useful means for incoherent imaging through scattering media in a wide class of fields such as optical microscopy, biomedical imaging, and astronomical imaging.

Imaging around corners in the mid-infrared using speckle correlations

Speckle correlation imaging offers the ability to see objects through diffusive materials and around corners. Imaging self-illuminating thermal objects in non-line-of-sight scenarios is of particular interest. Here, using bispectrum and phase retrieval methods, we demonstrate speckle correlation imaging of mid-infrared objects through diffusers and around corners at resolutions near the diffraction limit. The images agree well with those recorded by conventional cameras with line-of-sight to the same objects.

Extending the imaging range through scattering layers to the entire correlation range

A method of extending the imaging range through scattering layers around a reference point (RP) is realized. Objects within the entire correlation range of the RP can be totally recovered. By scanning the light source, objects within the memory effect (ME) range of the RP are completely recovered with high quality. By combining the shift of a camera to move the object to the center of observation view, objects far away from the RP are retrieved with an improved signal-to-noise ratio. The extended imaging range is about 3.5 times the ME range and more than 16 times the imaging range with normal static illumination. The RP can be imprecisely placed at a distance from the objects instead of precisely replacing them owing to the extended imaging range. This simple-system method forcefully breaks the limitation of the ME range and is very easy to implement in practical applications, which is meaningful for the research in scattering imaging.

Spatial light modulator aided noninvasive imaging through scattering layers

We propose and demonstrate a new imaging technique to noninvasively see through scattering layers with the aid of a spatial light modulator (SLM). A relay system projects the incoherent light pattern emitting from the scattering layer onto the SLM. Two coded phase masks are displayed, one after another, on the SLM to modulate the projected scattered field and the two corresponding intensity patterns are recorded by a digital camera. The above procedure helps to achieve two goals. Firstly, since the coded phase masks are digitally synthesized, the point spread function of the imaging system can be engineered such that the image retrieval becomes more reliable. Secondly, the two recorded intensity patterns are subtracted one from the other and by that the background noise of the recovered image is minimized. The above two advantages along with a modified phase retrieval algorithm enable a relatively easier and accurate convergence to the image of the covered object.

Experimental characterization of the speckle pattern at the output of a multimode optical fiber

Speckle patterns produced by coherent waves interfering with each other are undesirable in many imaging applications (for example, in laser projection systems) but on the other hand, they contain useful information that can be exploited (for example, for blood flow analysis or reconstruction of the object that generates the speckle). It is therefore important to understand how speckle can be enhanced or reduced by tailoring the coherence of laser light. Using a conventional semiconductor laser and a multimode optical fiber we study experimentally how the speckle pattern depends on the laser pump current and on the image acquisition settings. By varying the pump current from below to above the lasing threshold, and simultaneously tuning the image exposure time to compensate for the change in brightness, we find conditions that allow for recorded images with similar average intensity, but with speckle contrast (the standard deviation of the intensity over the average intensity) as low as 0.16, or as high as 0.99.

Multiple-image double-encryption via 2D rotations of a random phase mask with spatially incoherent illumination

Optical image encryption technique has become extremely important in these years. However, most of the proposed multiple-image encryption systems are illuminated with coherent light source. Here we present a multiple-image double-encryption method with spatially incoherent illumination. The first-encryption of multiple images is based on the speckle rotation decorrelation property, and the second-encryption of images' order is based on the speckle shift decorrelation out of the angular memory-effect range. The double-encryption via two-dimensional rotations of the random phase mask enhances the security and keeps the simplicity of the cryptosystem. The capacity of the ciphertext is greatly increased by multiplexing, and further increased after crosstalk noise removal. The use of incoherent light source reduces the requirements for experimental conditions, and makes the cryptosystem easy to implement in various application scenarios.

Speckle rotation decorrelation based single-shot video through scattering media

Optical imaging and tracking moving objects through scattering media is a challenge with important applications. However, previous works suffer from time-consuming recovery process, object complexity limit, or object information lost. Here we present a method based on the speckle rotation decorrelation property. The rotational speckles detected at short intervals are uncorrelated and multiplexed in a single-shot camera image. Object frames of the video are recovered by cross-correlation deconvolution of the camera image with a computationally rotated point spread function. The near real-time recovery provides sharp object image frames with accurate object relative positions, exact movement velocity, and continuous motion trails. This multiplexing technique has important implications for a wide range of real-world imaging scenarios.

Deep-penetration fluorescence imaging through dense yeast cells suspensions using Airy beams

We propose, to the best of our knowledge, a new method to image fluorescent objects through turbid media based on Airy beam scanning. This is achieved by using the nondiffractive nature of Airy beams, namely their ability to maintain their shape while penetrating through a highly scattering medium. We show that our technique can image fluorescent objects immersed in turbid media with higher resolution and signal to noise than confocal imaging. As proof of principle, we demonstrate imaging of 1 μm sized fluorescent beads through a dense suspension of yeast cells with an attenuation coefficient of 51  cm^-1 at a depth of 90 μm. Finally, we demonstrate that our technique can also provide the depth of the imaged object without any additional sectioning.

Compressive three-dimensional super-resolution microscopy with speckle-saturated fluorescence excitation

Nonlinear structured illumination microscopy (nSIM) is an effective approach for super-resolution wide-field fluorescence microscopy with a theoretically unlimited resolution. In nSIM, carefully designed, highly-contrasted illumination patterns are combined with the saturation of an optical transition to enable sub-diffraction imaging. While the technique proved useful for two-dimensional imaging, extending it to three-dimensions is challenging due to the fading of organic fluorophores under intense cycling conditions. Here, we present a compressed sensing approach that allows 3D sub-diffraction nSIM of cultured cells by saturating fluorescence excitation. Exploiting the natural orthogonality of speckles at different axial planes, 3D probing of the sample is achieved by a single two-dimensional scan. Fluorescence contrast under saturated excitation is ensured by the inherent high density of intensity minima associated with optical vortices in polarized speckle patterns. Compressed speckle microscopy is thus a simple approach that enables 3D super-resolved nSIM imaging with potentially considerably reduced acquisition time and photobleaching.

Prior-information-free single-shot scattering imaging beyond the memory effect

Imaging beyond the memory effect (ME) is critical to seeing through the scattering media. Methods proposed before have suffered from invasive point spread function measurement or the availability of prior information of the imaging targets. In this Letter, we propose a prior-information-free single-shot scattering imaging method to exceed the ME range. The autocorrelation of each imaging target is separated blindly from the autocorrelation of the recorded dual-target speckle via Fourier spectrum guessing and iterative energy constrained compensation. Working together with phase retrieval, dual targets exceeding the ME range can be reconstructed via a single shot. The effectiveness of the algorithm is verified by simulated experiments and a real imaging system.

Review of 3D Imaging by Coded Aperture Correlation Holography (COACH)

Coded aperture correlation holography (COACH) is a relatively new technique to record holograms of incoherently illuminated scenes. In this review, we survey the main milestones in the COACH topic from two main points of view. First, we review the prime architectures of optical hologram recorders in the family of COACH systems. Second, we discuss some of the key applications of these recorders in the field of imaging in general, and for 3D super-resolution imaging, partial aperture imaging, and seeing through scattering medium, in particular. We summarize this overview with a general perspective on this research topic and its prospective directions.

Improving reconstruction of speckle correlation imaging by using a modified phase retrieval algorithm with the number of nonzero-pixels constraint

Speckle correlation imaging (SCI) has been considered one of the most promising techniques for computational imaging through a scattering medium. However, the image quality is not always acceptable in conventional SCI, especially when a complex object is involved. In this work, a modified phase retrieval algorithm is introduced to significantly improve the imaging quality of SCI. In the proposed scheme, nonzero-pixel constraints, rather than the real and nonnegative constraints, are employed as the object domain constraints of the iterative algorithm in the image reconstruction process. Experimental results are presented to show the performance enhancement of this scheme, inclusive of less iterations, better image quality, and higher reliability, in comparison with the conventional SCI method.

An improved Richardson-Lucy iterative algorithm for C-scan image restoration and inclusion size measurement

The accuracy of measuring inclusion size in direct C-scan image of immersion ultrasonic testing is restricted by the lateral resolution of the focused transducer, even if a high frequency is used, and the blurred edge due to scattering of sound waves at inclusions. In this work, an improved image restoration method that is based on the Richardson-Lucy (RL) iterative algorithm is proposed, which is used to restore the C-scan image and improve the accuracy of inclusion size measurement in immersion ultrasonic testing. For the improved RL iterative algorithm, the point spread function (PSF) is derived based on the multi-Gaussian beam model and Kirchhoff approximation, which considers the propagation properties of sound waves at water-steel interface and the spectral characteristics of the transducer with high frequency. In order to determine the final iteration number, the relationship between final iteration number and size of the inclusion in the image is established by restoring the simulated C-scan image and further calibrated with size correction factor. The size correction factor considers the effect of sound attenuation and electro-mechanical transformation encountered in practical testing equipment. Experimental results show that the inclusion sizes measured in restored C-scan images agree well with the optical micrograph results, which prove the effectiveness of the proposed method.

Single-shot large field of view imaging with scattering media by spatial demultiplexing

Benefiting from the memory effect (ME) for speckle intensity correlations, only one single-shot speckle pattern can be used for the high-quality recovery of objects. However, ME gives a restriction to the field of view (FOV) for imaging with scattering media. Objects beyond the ME region cannot be recovered and produce unwanted speckle patterns, which cause reduction in the speckle contrast and recovery quality. Nevertheless, all the spatial information from a large object is embedded in a single speckle image. Here, we extract the spatial information from these unavoidable speckle patterns and enlarge the FOV of the imaging system with scattering media. Regional point spreading functions, which are fixed and only need to be recorded once for all-time use, are employed to recover corresponding spatial regions of an object by deconvolution. Then, an automatic weighted averaging in an iterative process is performed to obtain the object with significantly enlarged FOV. Our results present an important advancement of imaging techniques with strongly scattering media.

Imaging through scattering medium by adaptive non-linear digital processing

Scattering media have always posed obstacles for imaging through them. In this study, we propose a single exposure, spatially incoherent and interferenceless method capable of imaging multi-plane objects through scattering media using only a single lens and a digital camera. A point object and a resolution chart are precisely placed at the same axial location, and light scattered from them is focused onto an image sensor using a spherical lens. For both cases, intensity patterns are recorded under identical conditions using only a single camera shot. The final image is obtained by an adaptive non-linear cross-correlation between the response functions of the point object and of the resolution chart. The clear and sharp reconstructed image demonstrates the validity of the method.

Non-linear adaptive three-dimensional imaging with interferenceless coded aperture correlation holography (I-COACH)

Interferenceless coded aperture correlation holography (I-COACH) is an incoherent digital holography technique for imaging 3D objects without two-wave interference. In I-COACH, the object beam is modulated by a pseudorandom coded phase mask (CPM) and propagates to the camera where its intensity pattern is recorded. The image of the object is reconstructed by a cross-correlation of the object intensity pattern with a point intensity response of the system, whereas the light from both the object and the point, are modulated by the same CPM. In order to recover the image of the object without bias level and background noise, multiple intensity recordings are necessary for both objects as well as the point object, which in turn significantly reduces the time resolution of imaging. In this study, a non-linear reconstruction technique is developed to reconstruct the image of the object with only a single camera shot. Furthermore, the proposed technique is adaptive to different experimental conditions in the sense of finding different optimal parameters for each experiment. The new method has been implemented on a regular I-COACH system in both transmission as well as reflection illumination modes.

Imaging through scattering media with the auxiliary of a known reference object

Imaging through scattering media has been one of the main challenges in optics, and are encountered in many different disciplines of sciences, ranging from biology, mesoscopic physics to astronomy. Recently, various methods have been proposed. In this manuscript, we propose a robust method for imaging through scattering media in a reflective geometry, a scenario widely encountered in non-invasive and marker-free biological imaging. The proposed method relies on the a priori information of a known reference object in the neighborhood of the target, and uses it as an auxiliary to reconstruct the target image. We show that the target image can be analytically reconstructed from the autocorrelation of the recorded speckle if the reference is point-like, otherwise, deconvolution with the reference speckle should be performed. We experimentally demonstrate the proposed method in a proof-of-concept system with an LED illumination through a thick ground glass.

Non-invasive imaging through strongly scattering media based on speckle pattern estimation and deconvolution

Imaging through scattering media is still a formidable challenge with widespread applications ranging from biomedical imaging to remote sensing. Recent research progresses provide several feasible solutions, which are hampered by limited complexity of targets, invasiveness of data collection process and lack of robustness for reconstruction. In this paper, we show that the complex to-be-observed targets can be non-invasively reconstructed with fine details. Training targets, which can be directly reconstructed by speckle correlation and phase retrieval, are utilized as the input of the proposed speckle pattern estimation model, in which speckle modeling and constrained least square optimization are applied to estimate the distribution of the speckle pattern. Reconstructions for to-be-observed targets are realized by deconvoluting the estimated speckle pattern from the acquired integrated intensity matrices (IIMs). The qualities of reconstructed results are ensured by the stable statistical property and memory effect of laser speckle patterns. Experimental results show that the proposed method can reconstruct complex targets in high quality and the reconstruction performance is robust even much less data are acquired.

Imaging of objects through a thin scattering layer using a spectrally and spatially separated reference

Incoherently illuminated or luminescent objects give rise to a low-contrast speckle-like pattern when observed through a thin diffusive medium, as such a medium effectively convolves their shape with a speckle-like point spread function (PSF). This point spread function can be extracted in the presence of a reference object of known shape. Here it is shown that reference objects that are both spatially and spectrally separated from the object of interest can be used to obtain an approximation of the point spread function. The crucial observation, corroborated by analytical calculations, is that the spectrally shifted point spread function is strongly correlated to a spatially scaled one. With the approximate point spread function thus obtained, the speckle-like pattern is deconvolved to produce a clear and sharp image of the object on a speckle-like background of low intensity.

Wide field fluorescence epi-microscopy behind a scattering medium enabled by speckle correlations

Fluorescence microscopy is widely used in biological imaging, however scattering from tissues strongly limits its applicability to a shallow depth. In this work we adapt a methodology inspired from stellar speckle interferometry, and exploit the optical memory effect to enable fluorescence microscopy through a turbid layer. We demonstrate efficient reconstruction of micrometer-size fluorescent objects behind a scattering medium in epi-microscopy, and study the specificities of this imaging modality (magnification, field of view, resolution) as compared to traditional microscopy. Using a modified phase retrieval algorithm to reconstruct fluorescent objects from speckle images, we demonstrate robust reconstructions even in relatively low signal to noise conditions. This modality is particularly appropriate for imaging in biological media, which are known to exhibit relatively large optical memory ranges compatible with tens of micrometers size field of views, and large spectral bandwidths compatible with emission fluorescence spectra of tens of nanometers widths.

Imaging through scattering layers exceeding memory effect range with spatial-correlation-achieved point-spread-function

We propose to measure intensity transmission matrices or point-spread-function (PSF) of diffusers via spatial-correlation, with no scanning or interferometric detection required. With the measured PSF, we report optical imaging based on the memory effect that allows tracking of moving objects through a scattering medium. Our technique enlarges the limited effective range of traditional imaging techniques based on the memory effect, and substitutes time-consuming iterative algorithms by a fast cross-correlation deconvolution method to greatly reduce time consumption for image reconstruction.

Extended depth-resolved imaging through a thin scattering medium with PSF manipulation

Human ability to visualize an image is usually hindered by optical scattering. Recent extensive studies have promoted imaging technique through turbid materials to a reality where color image can be restored behind scattering media in real time. The big challenge now is to recover objects in a large field of view with depth resolving ability. Based on the existing research results, we systematically study the physical relationship between speckles generated from objects at different planes. By manipulating a given single point spread function, depth-resolved imaging through a thin scattering medium can be extended beyond the original depth of field (DOF). Experimental testing of standard scattering media shows that the DOF can be extended up to 5 times and the physical mechanism is depicted. This extended DOF is benefit to 3D imaging through scattering environment, and it is expected to have important applications in science, technology, bio-medical, security and defense.

3D Imaging through Scatterers with Interferenceless Optical System

Imaging through a scattering medium is a challenging task. We propose and demonstrate an interferenceless incoherent opto-digital technique for 3D imaging through a scatterer with a single lens and a digital camera. The light diffracted from a point object is modulated by a scattering mask. The modulated wavefront is projected on an image sensor using a spherical lens and the impulse response is recorded. An object is placed at the same axial location as the point object and another intensity pattern is recorded with identical experimental conditions and with the same scattering mask. The image of the object is reconstructed by a cross-correlation between a reconstructing function and the object hologram. For 3D imaging, a library of reconstructing functions are created corresponding to different axial locations. The different planes of the object are reconstructed by a cross-correlation of the object hologram with the corresponding reconstructing functions.

Non-diffracting beams for label-free imaging through turbid media

We propose a new method to image through dynamically changing turbid media based on the scanning of non-diffractive laser beams. We use computer-generated holograms to create Airy beams and compare quantitatively the characteristics of their propagation in clear and turbid media. Imaging contrast is achieved by relative reflection of the scanned beams from the imaged surface. We implement our method to demonstrate experimentally our ability to image a chromium surface on a glass slide through 270 μm of highly scattering milk/water mixtures with a resolution of several microns.

Imaging through a thin scattering layer and jointly retrieving the point-spread-function using phase-diversity

Recently introduced angular-memory-effect based techniques enable non-invasive imaging of objects hidden behind thin scattering layers. However, both the speckle-correlation and the bispectrum analysis are based on the statistical average of large amounts of speckle grains, which determines that they can hardly access the important information of the point-spread-function (PSF) of a highly scattering imaging system. Here, inspired by notions used in astronomy, we present a phase-diversity speckle imaging scheme, based on recording a sequence of intensity speckle patterns at various imaging planes, and experimentally demonstrate that in addition to being able to retrieve the image of hidden objects, we can also simultaneously estimate the pupil function and the PSF of a highly scattering imaging system without any guide-star nor reference.