Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media

Tracking the transmission matrix of a moving fiber with the transmitted data

During data transmission, the dynamic change of a scattering medium will make the measured transmission matrix (TM) invalid, so it is necessary to repeatedly measure the TM to achieve a long-time data transmission, which requires stopping the data transmission process frequently to measure the TM and leads to a reduction in the communication capacity. To solve this problem, we propose a TM tracking method during data transmission. In the case of more than three discrete levels of phase modulation, this method can realize the calibration of the TM with the intensity pictures captured by the camera and the recovered data, so it does not require stopping the data transmission process to measure the TM and thus avoids the loss of communication capacity. We have proved the feasibility of this method through simulations and experiments and realized the continuous transmission of random data and image data through a moving fiber with high accuracy.

Memory-less scattering imaging with ultrafast convolutional optical neural networks

The optical memory effect in complex scattering media including turbid tissue and speckle layers has been a critical foundation for macroscopic and microscopic imaging methods. However, image reconstruction from strong scattering media without the optical memory effect has not been achieved. Here, we demonstrate image reconstruction through scattering layers where no optical memory effect exists, by developing a multistage convolutional optical neural network (ONN) integrated with multiple parallel kernels operating at the speed of light. Training this Fourier optics-based, parallel, one-step convolutional ONN with the strong scattering process for direct feature extraction, we achieve memory-less image reconstruction with a field of view enlarged by a factor up to 271. This device is dynamically reconfigurable for ultrafast multitask image reconstruction with a computational power of 1.57 peta-operations per second (POPS). Our achievement establishes an ultrafast and high energy-efficient optical machine learning platform for graphic processing.

Large-scale photonic computing with nonlinear disordered media

Neural networks find widespread use in scientific and technological applications, yet their implementations in conventional computers have encountered bottlenecks due to ever-expanding computational needs. Photonic computing is a promising neuromorphic platform with potential advantages of massive parallelism, ultralow latency and reduced energy consumption but mostly for computing linear operations. Here we demonstrate a large-scale, high-performance nonlinear photonic neural system based on a disordered polycrystalline slab composed of lithium niobate nanocrystals. Mediated by random quasi-phase-matching and multiple scattering, linear and nonlinear optical speckle features are generated as the interplay between the simultaneous linear random scattering and the second-harmonic generation, defining a complex neural network in which the second-order nonlinearity acts as internal nonlinear activation functions. Benchmarked against linear random projection, such nonlinear mapping embedded with rich physical computational operations shows improved performance across a large collection of machine learning tasks in image classification, regression and graph classification. Demonstrating up to 27,648 input and 3,500 nonlinear output nodes, the combination of optical nonlinearity and random scattering serves as a scalable computing engine for diverse applications.

Construction of vector vortex beams on hybrid-order Poincaré sphere through highly scattering media

Vector vortex beams (VVBs) have attracted extensive attention due to their unique properties and their wide applications in fields such as optical manipulation and optical imaging. However, the wavefronts of the vector vortex beams are highly scrambled when they encounter highly scattering media (HSM), such as thick biological tissues, which greatly prevents the applications of VVBs behind HSM. To address this issue, we propose a scheme to construct VVBs of freewill position on the surface of hybrid-order Poincaré sphere (HyOPS) through HSM. With the measurement of two orthogonal scalar transmission matrices, the conjugated wavefronts for constructing orbital angular momentum beams with arbitrary topological charge in right and left circularly polarized states through HSM can be calculated, respectively. When an input wavefront superimposed by the two conjugated wavefronts with an appropriate ratio and phase delay, impinges on the HSM, the desired VVB can be created through HSM. To demonstrate the viability of our scheme, a series of VVBs on different locations of various HyOPSs have been reconstructed through a ZnO scattering layer experimentally. Furthermore, to characterize the polarization distribution of the generated beams, the polarization maps of these beams are derived by measuring the four Stokes parameters, which agree well with the theoretical distributions. This work will promote the applications of VVBs in highly scattering environments.

Neural network enabled fringe projection through scattering media

The projection of fringes plays an essential role in many applications, such as fringe projection profilometry and structured illumination microscopy. However, these capabilities are significantly constrained in environments affected by optical scattering. Although recent developments in wavefront shaping have effectively generated high-fidelity focal points and relatively simple structured images amidst scattering, the ability to project fringes that cover half of the projection area has not yet been achieved. To address this limitation, this study presents a fringe projector enabled by a neural network, capable of projecting fringes with variable periodicities and orientation angles through scattering media. We tested this projector on two types of scattering media: ground glass diffusers and multimode fibers. For these scattering media, the average Pearson's correlation coefficients between the projected fringes and their designed configurations are 86.9% and 79.7%, respectively. These results demonstrate the effectiveness of the proposed neural network enabled fringe projector. This advancement is expected to broaden the scope of fringe-based imaging techniques, making it feasible to employ them in conditions previously hindered by scattering effects.

Non-line-of-sight optical wireless communication system enabled by wavefront shaping for multi-user indoor access

In this Letter, we experimentally investigate a non-line-of-sight (NLOS) optical wireless communication (OWC) system that utilizes wavefront shaping techniques to realize simultaneous data transmission for multiple users. Wavefront shaping techniques are employed to address the issue of low intensity of diffusely reflected light at the receiver in NLOS scenarios for indoor high-speed access. To achieve communication path planning and tracing for two different users in free-space optical communication, the pixels of the spatial light modulator (SLM) are divided into two halves to separately manipulate the wavefront of two independent data carriers centered at different wavelengths. The maximum received optical power can be effectively improved by more than 15 dB with the wavefront shaping technique. To avoid power enhancement of non-target wavelength, the wavelength difference of two different users is experimentally studied. The difference in power enhancement ratio (DPER) is increased with the wavelength difference, and 14.95 dB DPER is obtained with a 10 nm wavelength difference. Under the aforementioned wavelength planning strategy, successful transmission and reception of 2 × 160 Gbit/s 16-QAM signals for two users with coherent detection is achieved using wavelengths of 1550 and 1560 nm in an indoor access scenario.

Non-invasive accelerated imaging through a scattering medium via multi-stage complexity guidance

The research of scattering imaging is of great significance to the development of various fields, but the existing scattering imaging methods are difficult to combine for the advantages of non-invasiveness, real-time imaging, and high quality. In this paper, a new, to our knowledge, scattering imaging technique is proposed that optimizes the traditional autocorrelation imaging technique by multi-stage complexity guidance and the initial acceleration module. We introduce the complexity difference index into the phase iterative recovery step for effective complexity guidance, and add the initial module based on error-reduction iteration to realize a fast startup. A series of experiments is carried out to test the performance of the new technique. The results show that the proposed technique significantly improves the scattering reconstruction speed. Meanwhile, the accuracy and clarity of the reconstructed image are significantly higher than the traditional method in terms of fast imaging. Moreover, this technique has better robustness to noise compared to the traditional autocorrelation imaging technique. The experimental code for this paper is available on GitHub.

Neural étendue expander for ultra-wide-angle high-fidelity holographic display

Holographic displays can generate light fields by dynamically modulating the wavefront of a coherent beam of light using a spatial light modulator, promising rich virtual and augmented reality applications. However, the limited spatial resolution of existing dynamic spatial light modulators imposes a tight bound on the diffraction angle. As a result, modern holographic displays possess low étendue, which is the product of the display area and the maximum solid angle of diffracted light. The low étendue forces a sacrifice of either the field-of-view (FOV) or the display size. In this work, we lift this limitation by presenting neural étendue expanders. This new breed of optical elements, which is learned from a natural image dataset, enables higher diffraction angles for ultra-wide FOV while maintaining both a compact form factor and the fidelity of displayed contents to human viewers. With neural étendue expanders, we experimentally achieve 64 × étendue expansion of natural images in full color, expanding the FOV by an order of magnitude horizontally and vertically, with high-fidelity reconstruction quality (measured in PSNR) over 29 dB on retinal-resolution images.

Adaptive noise-resilient deep learning for image reconstruction in multimode fiber scattering

This research offers a comprehensive exploration of three pivotal aspects within the realm of fiber optics and piezoelectric materials. The study delves into the influence of voltage variation on piezoelectric displacement, examines the effects of bending multimode fiber (MMF) on data transmission, and scrutinizes the performance of an autoencoder in MMF image reconstruction with and without additional noise. To assess the impact of voltage variation on piezoelectric displacement, experiments were conducted by applying varying voltages to a piezoelectric material, meticulously measuring its radial displacement. The results revealed a notable increase in displacement with higher voltage, presenting implications for fiber stability and overall performance. Additionally, the investigation into the effects of bending MMF on data transmission highlighted that the bending process causes the fiber to become leaky and radiate power radially, potentially affecting data transmission. This crucial insight emphasizes the necessity for further research to optimize data transmission in practical fiber systems. Furthermore, the performance of an autoencoder model was evaluated using a dataset of MMF images, in diverse scenarios. The autoencoder exhibited impressive accuracy in reconstructing MMF images with high fidelity. The results underscore the significance of ongoing research in these domains, propelling advancements in fiber optic technology.

Adaptive imaging through dense dynamic scattering media using transfer learning

Imaging through scattering media is a long-standing challenge in optical imaging, holding substantial importance in fields like biology, transportation, and remote sensing. Recent advancements in learning-based methods allow accurate and rapid imaging through optically thick scattering media. However, the practical application of data-driven deep learning faces substantial hurdles due to its inherent limitations in generalization, especially in scenarios such as imaging through highly non-static scattering media. Here we utilize the concept of transfer learning toward adaptive imaging through dense dynamic scattering media. Our approach specifically involves using a known segment of the imaging target to fine-tune the pre-trained de-scattering model. Since the training data of downstream tasks used for transfer learning can be acquired simultaneously with the current test data, our method can achieve clear imaging under varying scattering conditions. Experiment results show that the proposed approach (with transfer learning) is capable of providing more than 5dB improvements when optical thickness varies from 11.6 to 13.1 compared with the conventional deep learning approach (without transfer learning). Our method holds promise for applications in video surveillance and beacon guidance under dense dynamic scattering conditions.

Single-pixel imaging through non-homogeneous turbid media with adaptive illumination

The presence of scattering media limits the quality of images obtained by optical systems. Single-pixel imaging techniques based on structured illumination are highly tolerant to the presence of scattering between the object and the sensor, but very sensitive when the scattering medium is between the light source and the object. This makes it difficult to develop single-pixel imaging techniques for the case of objects immersed in scattering media. We present what we believe to be a new system for imaging objects through inhomogeneous scattering media in an epi-illumination configuration. It works in an adaptive way by combining diffuse optical imaging (DOI) and single pixel imaging (SPI) techniques in two stages. First, the turbid media is characterized by projecting light patterns with an LED array and applying DOI techniques. Second, the LED array is programmed to project light only through the less scattering areas of the media, while simultaneously using a digital micromirror device (DMD) to project light patterns onto the target using Hadamard basis coding functions. With this adaptive technique, we are able to obtain images of targets through two different scattering media with better quality than using conventional illumination. We also show that the system works with fluorescent targets.

Widefield functional speckle-correlation optical scattering mesoscopy toward hemodynamic imaging

Speckle-correlation optical scattering imaging (SCOSI) has shown the potential for non-invasive biomedical diagnostic applications, which directly utilizes the scattering patterns to reconstruct the deep and non-line-of-sight objects. However, the course of the translation of this technique to preclinical biomedical imaging applications has been postponed by the following two facts: 1) the field of view of SCOSI was significantly limited by the optical memory effect, and 2) the molecular-tagged functional imaging of the biological tissues remains largely unexplored. In this work, a proof-of-concept design of the first-generation widefield functional SCOSI (WF-SCOSI) system was presented for simultaneously achieving mesoscopic mapping of fluid morphology and flow rate, which was realized by implementing the concepts of scanning synthesis and fluorescence scattering flowmetry. The ex vivo imaging results of the fluorescence-labeled large-scale blood vessel network phantom underneath the strong scatters demonstrated the effectiveness of WF-SCOSI toward non-invasive hemodynamic imaging applications.

Vectorial holography over a multimode fiber

Vectorial holography through a strongly scattering medium can facilitate various applications in optics and photonics. However, the realization of vectorial holography with arbitrary distribution of optical intensity is still limited because of experimental noise during the calibration of vectorial transmission matrix (TM) and reconstruction noise during the retrieval of input wavefront for a given holographic target. Herein, we propose and experimentally demonstrate the vectorial holography with arbitrary distribution of optical intensity over a multimode fiber (MMF) using the Tikhonov regularization. By optimizing the noise factor, the performance of vectorial holography over an MMF is improved compared with the conjugate transpose and inverse TM methods. Our results might shed new light on the optical communication and detection mediated by MMFs.

Velocities of transmission eigenchannels and diffusion

The diffusion model is used to calculate both the time-averaged flow of particles in stochastic media and the propagation of waves averaged over ensembles of disordered static configurations. For classical waves exciting static disordered samples, such as a layer of paint or a tissue sample, the flux transmitted through the sample may be dramatically enhanced or suppressed relative to predictions of diffusion theory when the sample is excited by a waveform corresponding to a transmission eigenchannel. Even so, it is widely assumed that the velocity of waves is irretrievably randomized in scattering media. Here we demonstrate in microwave measurements and numerical simulations that the statistics of velocity of different transmission eigenchannels are distinct and remains so on all length scales and are identical on the incident and output surfaces. The interplay between eigenchannel velocities and transmission eigenvalues determines the energy density within the medium, the diffusion coefficient, and the dynamics of propagation. The diffusion coefficient and all scattering parameters, including the scattering mean free path, oscillate with the width of the sample as the number and shape of the propagating channels in the medium change.

High-security learning-based optical encryption assisted by disordered metasurface

Artificial intelligence has gained significant attention for exploiting optical scattering for optical encryption. Conventional scattering media are inevitably influenced by instability or perturbations, and hence unsuitable for long-term scenarios. Additionally, the plaintext can be easily compromised due to the single channel within the medium and one-to-one mapping between input and output. To mitigate these issues, a stable spin-multiplexing disordered metasurface (DM) with numerous polarized transmission channels serves as the scattering medium, and a double-secure procedure with superposition of plaintext and security key achieves two-to-one mapping between input and output. In attack analysis, when the ciphertext, security key, and incident polarization are all correct, the plaintext can be decrypted. This system demonstrates excellent decryption efficiency over extended periods in noisy environments. The DM, functioning as an ultra-stable and active speckle generator, coupled with the double-secure approach, creates a highly secure speckle-based cryptosystem with immense potentials for practical applications.

Generating Airy beams through multimode fibres

Focussing light through a multimode fibre (MMF) is the basis of holographic endoscopes, which currently enable detailed imaging of deep tissue. Achieving high fidelity and purity diffraction-limited foci has been shown to be possible, when fully controlling the amplitude, phase, and two orthogonal polarisation states of the input field. Yet, generating more complex field distributions with similar performance remains to be assessed. Here, we demonstrate the generation of Airy beams through an MMF containing in excess of 90 % of the optical power delivered by the fibre. We discuss two distinct methods for generating optical landscapes: the direct field and the Fourier domain synthesis. Moreover, we showcase the flexibility of the Fourier domain synthesis to modify the generated beam.

Terahertz Spatiotemporal Wave Synthesis in Random Systems

Complex media have emerged as a powerful and robust framework to control light-matter interactions designed for task-specific optical functionalities. Studies on wavefront shaping through disordered systems have demonstrated optical wave manipulation capabilities beyond conventional optics, including aberration-free and subwavelength focusing. However, achieving arbitrary and simultaneous control over the spatial and temporal features of light remains challenging. In particular, no practical solution exists for field-level arbitrary spatiotemporal control of wave packets. A new paradigm shift has emerged in the terahertz frequency domain, offering methods for absolute time-domain measurements of the scattered electric field, enabling direct field-based wave synthesis. In this work, we report the experimental demonstration of field-level control of single-cycle terahertz pulses on arbitrary spatial points through complex disordered media.

Non-orthogonal optical multiplexing empowered by deep learning

Orthogonality among channels is a canonical basis for optical multiplexing featured with division multiplexing, which substantially reduce the complexity of signal post-processing in demultiplexing. However, it inevitably imposes an upper limit of capacity for multiplexing. Herein, we report on non-orthogonal optical multiplexing over a multimode fiber (MMF) leveraged by a deep neural network, termed speckle light field retrieval network (SLRnet), where it can learn the complicated mapping relation between multiple non-orthogonal input light field encoded with information and their corresponding single intensity output. As a proof-of-principle experimental demonstration, it is shown that the SLRnet can effectively solve the ill-posed problem of non-orthogonal optical multiplexing over an MMF, where multiple non-orthogonal input signals mediated by the same polarization, wavelength and spatial position can be explicitly retrieved utilizing a single-shot speckle output with fidelity as high as ~ 98%. Our results resemble an important step for harnessing non-orthogonal channels for high capacity optical multiplexing.

Diffractive optical computing in free space

Structured optical materials create new computing paradigms using photons, with transformative impact on various fields, including machine learning, computer vision, imaging, telecommunications, and sensing. This Perspective sheds light on the potential of free-space optical systems based on engineered surfaces for advancing optical computing. Manipulating light in unprecedented ways, emerging structured surfaces enable all-optical implementation of various mathematical functions and machine learning tasks. Diffractive networks, in particular, bring deep-learning principles into the design and operation of free-space optical systems to create new functionalities. Metasurfaces consisting of deeply subwavelength units are achieving exotic optical responses that provide independent control over different properties of light and can bring major advances in computational throughput and data-transfer bandwidth of free-space optical processors. Unlike integrated photonics-based optoelectronic systems that demand preprocessed inputs, free-space optical processors have direct access to all the optical degrees of freedom that carry information about an input scene/object without needing digital recovery or preprocessing of information. To realize the full potential of free-space optical computing architectures, diffractive surfaces and metasurfaces need to advance symbiotically and co-evolve in their designs, 3D fabrication/integration, cascadability, and computing accuracy to serve the needs of next-generation machine vision, computational imaging, mathematical computing, and telecommunication technologies.

Optimizing multi-user indoor sound communications with acoustic reconfigurable metasurfaces

Sound in indoor spaces forms a complex wavefield due to multiple scattering encountered by the sound. Indoor acoustic communication involving multiple sources and receivers thus inevitably suffers from cross-talks. Here, we demonstrate the isolation of acoustic communication channels in a room by wavefield shaping using acoustic reconfigurable metasurfaces (ARMs) controlled by optimization protocols based on communication theories. The ARMs have 200 electrically switchable units, each selectively offering 0 or π phase shifts in the reflected waves. The sound field is reshaped for maximal Shannon capacity and minimal cross-talk simultaneously. We demonstrate diverse acoustic functionalities over a spectrum much larger than the coherence bandwidth of the room, including multi-channel, multi-spectral channel isolations, and frequency-multiplexed acoustic communication. Our work shows that wavefield shaping in complex media can offer new strategies for future acoustic engineering.

Tight focusing through scattering media via a Bessel-basis transmission matrix

The transmission matrix (TM) is a powerful tool for focusing light through scattering media. Here, we demonstrate a Bessel-basis TM that enables tight focusing through the scattering media and reduces the full width at half maximum of the focus by 23% on average, as compared to the normally used Hadamard-basis TM. To measure the Bessel-basis TM, we establish a common-path inter-mode interferometer (IMI), which can fully utilize the pixels of the spatial light modulator, leading to an enhancement in the peak-to-background intensity ratio (PBR) of the focus. Experimental results suggest that the Bessel-basis TM can achieve a tighter focus behind the scattering media, and the PBR of the focus obtained by the IMI is around 14.3% higher than that achieved using the normal peripheral reference interferometry.

Feature decoupled knowledge distillation enabled lightweight image transmission through multimode fibers

Multimode fibers (MMF) show tremendous potential in transmitting high-capacity spatial information. However, the quality of multimode transmission is quite sensitive to inherent scattering characteristics of MMF and almost inevitable external perturbations. Previous research has shown that deep learning may break through this limitation, while deep neural networks are intricately designed with huge computational complexity. In this study, we propose a novel feature decoupled knowledge distillation (KD) framework for lightweight image transmission through MMF. In this framework, the frequency-principle-inspired feature decoupled module significantly improves image transmission quality and the lightweight student model can reach the performance of the sophisticated teacher model through KD. This work represents the first effort, to the best of our knowledge, that successfully applies a KD-based framework for image transmission through scattering media. Experimental results demonstrate that even with up to 93.4% reduction in model computational complexity, we can still achieve averaged Structure Similarity Index Measure (SSIM) of 0.76, 0.85, and 0.90 in Fashion-MNIST, EMNIST, and MNIST images respectively, which are very close to the performance of cumbersome teacher models. This work dramatically reduces the complexity of high-fidelity image transmission through MMF and holds broad prospects for applications in resource-constrained environments and hardware implementations.

Single-shot digital optical fluorescence phase conjugation through forward multiple-scattering samples

Aberrations and multiple scattering in biological tissues critically distort light beams into highly complex speckle patterns. In this regard, digital optical phase conjugation (DOPC) is a promising technique enabling in-depth focusing. However, DOPC becomes challenging when using fluorescent guide stars for four main reasons: the low photon budget available, the large spectral bandwidth of the fluorescent signal, the Stokes shift between the emission and the excitation wavelength, and the absence of reference beam preventing holographic measurement. Here, we demonstrate the possibility to focus a laser beam through multiple-scattering samples by measuring speckle fields in a single acquisition step with a reference-free, high-resolution wavefront sensor. By taking advantage of the large spectral bandwidth of forward multiply scattering samples, digital fluorescence phase conjugation is achieved to focus a laser beam at the excitation wavelength while measuring the broadband speckle field arising from a micrometer-sized fluorescent bead.

Single-pixel transmission matrix recovery via two-photon fluorescence

Imaging at depth in opaque materials has long been a challenge. Recently, wavefront shaping has enabled notable advance for deep imaging. Nevertheless, most noninvasive wavefront-shaping methods require cameras, lack the sensitivity for deep imaging under weak optical signals, or can only focus on a single "guidestar." Here, we retrieve the transmission matrix (TM) noninvasively using two-photon fluorescence exploiting a single-pixel detection combined with a computational framework, allowing to achieve single-target focus on multiple guidestars spread beyond the memory effect range. In addition, if we assume that memory effect correlations exist in the TM, we are able to substantially reduce the number of measurements needed.

High-fidelity multi-channel optical information transmission through scattering media

High-fidelity optical information transmission through strongly scattering media is challenging, but is crucial for the applications such as the free-space optical communication in a haze or fog. Binarizing optical information can somehow suppress the disruptions caused by light scattering. However, this method gives a compromised communication throughput. Here, we propose high-fidelity multiplexing anti-scattering transmission (MAST). MAST encodes multiple bits into a complex-valued pattern, loads the complex-valued pattern to an optical field through modulation, and finally employs a scattering matrix-assisted retrieval technique to reconstruct the original information from the speckle patterns. In our demonstration, we multiplexed three channels and MAST achieved a high-fidelity transmission of 3072 (= 1024× 3) bits data per transmission and average transmission error as small as 0.06%.

Wavefront shaping improves the transparency of the scattering media: a review

Significance

Wavefront shaping (WFS) can compensate for distortions by optimizing the wavefront of the input light or reversing the transmission matrix of the media. It is a promising field of research. A thorough understanding of principles and developments of WFS is important for optical research.

Aim

To provide insight into WFS for researchers who deal with scattering in biomedicine, imaging, and optical communication, our study summarizes the basic principles and methods of WFS and reviews recent progress.

Approach

The basic principles, methods of WFS, and the latest applications of WFS in focusing, imaging, and multimode fiber (MMF) endoscopy are described. The practical challenges and prospects of future development are also discussed.

Results

Data-driven learning-based methods are opening up new possibilities for WFS. High-resolution imaging through MMFs can support small-diameter endoscopy in the future.

Conclusion

The rapid development of WFS over the past decade has shown that the best solution is not to avoid scattering but to find ways to correct it or even use it. WFS with faster speed, more optical modes, and more modulation degrees of freedom will continue to drive exciting developments in various fields.

Progressive acceleration in the genetic algorithm by intragenerational mutation

The intragenerational mutation of the genetic algorithm (IMGA) is proposed to actively broaden the searching space during the optimization process. The searching space is aggressively increased by expanding the variation of mutation rates of all individuals within each generation, leading to the reduction of the required number of iterations, improving the convergence speed and the enhancement factor.

Deep learning based coherent diffraction imaging of dynamic scattering media

The ptychographic iterative engine (PIE) is a lensless coherent diffraction imaging algorithm known for its simplicity, easy to use, scalability, and fast convergence. However, practical applications often encounter interference in imaging results caused by non-static scattering media, such as dense fog, seawater target detection and medical biology diagnosis. To address this challenge, we propose a novel approach using computational deep learning for dynamic scattering medium image reconstruction, enabling lens-free coherent diffraction imaging through dynamic scattering media. Through extensive analysis, we evaluate the effectiveness of the neural network for PIE image recovery under varying scattering medium concentration conditions. We also test scattering images obtained by hybrid training with different concentrations of scattering medium to assess the generalisation ability of the neural network. The experimental results demonstrate that our proposed method achieve PIE lens-free imaging under non-static scattering media interference. This coherent diffraction imaging method, based on transmission through dynamic scattering media, opens up new possibilities for practical applications of PIE and fosters its development in complex environments. Its significance extends to fields like atmospheric pollution, seawater target detection and medical biology diagnosis, providing valuable references for research in these domains.

Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media

Optical imaging in scattering media is important to many fields but remains challenging. Recent methods have focused on imaging through thin scattering layers or thicker scattering media with prior knowledge of the sample, but this still limits practical applications. Here, we report an imaging method named 'speckle kinetography' that enables high-resolution imaging in unknown scattering media with thicknesses up to about 6 transport mean free paths. Speckle kinetography non-invasively records a series of incoherent speckle images accompanied by object motion and the inherently retained object information is extracted through an overlapping speckle correlation algorithm to construct the object's autocorrelation for imaging. Under single-colour light-emitting diode, white light, and fluorescence illumination, we experimentally demonstrate 1 μm resolution imaging and tracking of objects moving in scattering samples, while reducing the requirements for prior knowledge. We anticipate this method will enable imaging in currently inaccessible scenarios.

Non-linear optical scattering PUF: enhancing security against modeling attacks for authentication systems

With the rapid expansion of the Internet of Things (IoT), ensuring the security of personal and group information has become increasingly crucial. However, conventional optical scattering physical unclonable function (OS-PUF) faces challenges due to its linear scattering behavior. In this article, we propose a non-linear OS-PUF (NOS-PUF) that integrates electro-optic materials. By leveraging random refractive index fluctuations generated by the NOS-PUF, we mitigate modeling attacks based on the OS-PUF and bolster the overall security of the authentication process. Moreover, we introduce a novel modeling attack methodology based on scattering invariant modes (SIMs) that poses a significant threat to conventional OS-PUF and NOS-PUF authentication systems. Through extensive simulations, we demonstrate that our NOS-PUF achieves a remarkably lower false accept rate for modeling attacks utilizing SIMs, surpassing the entropy limit imposed by the Gabor filtering algorithm by more than five orders of magnitude. These results highlight the heightened security and increased information entropy offered by the proposed NOS-PUF, making it particularly suitable for applications demanding robust and high-security authentication measures.

Spatially resolved readout of a Fabry-Perot ultrasound sensor interrogated through a multimode optical fiber using wavefront shaping

The spatially resolved interrogation of a Fabry-Perot ultrasound sensor using a laser beam focused through a multimode fiber is demonstrated. To scan the beam across the sensor as required to read it out, optical wavefront shaping was employed to compensate for the scrambling of light in the fiber. By providing a means to map ultrasound through inexpensive, lightweight fibers, this could lead to new ultrasonic and photoacoustic imaging systems, such as endoscopes and flexible handheld probes.

Integration of high-resolution imaging through scattering medium into a disposable micro-endoscope via projection of 2D spots-array

The objective of this research includes integration of high-resolution imaging through scattering medium, such as blood, into a disposable micro-endoscope. A fiber laser integrated into the micro-endoscope as part of its illumination channel, allows to project a tunable array of spots of light onto an object, that is located behind the scattering medium. We have a laser fiber as part of the illumination channel of a disposable micro-endoscope. By using proper optics, we convert the temporal modulation of the laser into spatial distribution. Thus, the result is generation of spatial spots when using a pulsed laser. The detection channel is a holographic recording of the collected back scattered light, that allows extraction of the electrical field. By time integrating the field we obtain the realization of the spatial array of illumination spots formed on top of the inspected object and behind the scattering medium. By changing the temporal modulation of the illumination laser (changing its temporal photonic signals), we can tune the positions of the spots in the illumination array. If the distance between the projected spots is larger than the imaging resolution, then by applying localization microscopy algorithms together with scanning of the position of the spots in the array, will yield a high-resolution reconstruction of the inspected object. We theoretically and experimentally demonstrate the discussed operation principle and show the potential of the proposed concept as a modality in medical endoscopic procedures.

Multi-channel data transmission through a multimode fiber based on OAM phase encoding

Data transmission based on the transmission matrix method has realized the multiplexing of a large number of orbital angular momentum (OAM) modes under scattering, which encodes the data by modulating the amplitude of the OAM modes. However, this amplitude modulation (amplitude encoding) method has obvious cross talk when the number of output modes is small, resulting in a non-negligible bit error rate. Here, a multi-channel data transmission method based on OAM phase modulation (phase encoding) under scattering is proposed. This method can resist the multiple-scattering effect of multimode fibers and realize accurate data transmission with very few rows of camera pixels for output mode measurement, which is suitable for high-speed data transmission under scattering. Experimentally, we have achieved a bit error rate of less than 0.005% in the data transmission of a color image through a 60 m multimode fiber with only 2 rows of camera pixels for output mode measurement. Experiments also showed that the proposed method has a higher stability than amplitude encoding when the proportion of "1" or "0" in the code changes.

Extrapolated speckle-correlation imaging with an untrained deep neural network

We present a method for speckle-correlation imaging with an extended field of view to observe spatially non-sparse objects. In speckle-correlation imaging, an object is recovered from a non-invasively captured image through a scattering medium by assuming shift-invariance of the optical process called the memory effect. The field of view of speckle-correlation imaging is limited by the size of the memory effect, and it can be extended by extrapolating the speckle correlation in the reconstruction process. However, spatially sparse objects are assumed in the inversion process because of its severe ill-posedness. To address this issue, we introduce a deep image prior, which regularizes the image statistics by using the structure of an untrained convolutional neural network, to speckle-correlation imaging. We experimentally demonstrated the proposed method and showed the possibility of extending the method to imaging through scattering media.

Tracing multiple scattering trajectories for deep optical imaging in scattering media

Multiple light scattering hampers imaging objects in complex scattering media. Approaches used in real practices mainly aim to filter out multiple scattering obscuring the ballistic waves that travel straight through the scattering medium. Here, we propose a method that makes the deterministic use of multiple scattering for microscopic imaging of an object embedded deep within scattering media. The proposed method finds a stack of multiple complex phase plates that generate similar light trajectories as the original scattering medium. By implementing the inverse scattering using the identified phase plates, our method rectifies multiple scattering and amplifies ballistic waves by almost 600 times. This leads to a significant increase in imaging depth-more than three times the scattering mean free path-as well as the correction of image distortions. Our study marks an important milestone in solving the long-standing high-order inverse scattering problems.

Learning diffractive optical communication around arbitrary opaque occlusions

Free-space optical communication becomes challenging when an occlusion blocks the light path. Here, we demonstrate a direct communication scheme, passing optical information around a fully opaque, arbitrarily shaped occlusion that partially or entirely occludes the transmitter's field-of-view. In this scheme, an electronic neural network encoder and a passive, all-optical diffractive network-based decoder are jointly trained using deep learning to transfer the optical information of interest around the opaque occlusion of an arbitrary shape. Following its training, the encoder-decoder pair can communicate any arbitrary optical information around opaque occlusions, where the information decoding occurs at the speed of light propagation through passive light-matter interactions, with resilience against various unknown changes in the occlusion shape and size. We also validate this framework experimentally in the terahertz spectrum using a 3D-printed diffractive decoder. Scalable for operation in any wavelength regime, this scheme could be particularly useful in emerging high data-rate free-space communication systems.

High-throughput imaging through dynamic scattering media based on speckle de-blurring

Effectively imaging through dynamic scattering media is of great importance and challenge. Some imaging methods based on physical or learning models have been designed for object reconstruction. However, with an increase in exposure time or more drastic changes in the scattering medium, the speckle pattern superimposed during camera integration time undergoes more significant changes, resulting in a modification of the collected speckle structure and increased blurring, which brings significant challenges to the reconstruction. Here, the clearer structural information of blurred speckles is unearthed with a presented speckle de-blurring algorithm, and a high-throughput imaging method through rapidly changing scattering media is proposed for reconstruction under long exposure. For the problem of varying blur degrees in different regions of the speckle, a block-based method is proposed to divide the speckle into distinct sub-speckles, which can realize the reconstruction of hidden objects. The imaging of hidden objects with different complexity through dynamic scattering media is demonstrated, and the reconstruction results are improved significantly for speckles with different blur degrees, which verifies the effectiveness of the method. This method is a high-throughput approach that enables non-invasive imaging solely through the collection of a single speckle. It directly operates on blurred speckles, making it suitable for traditional speckle-correlation methods and deep learning (DL) methods. This provides a new way of thinking about solving practical scattering imaging challenges.

Projecting colorful images through scattering media via deep learning

The existence of scatterers in the optical path has been the major obstacle that prohibits one from projecting images through solid walls, turbid water, clouds, and fog. Recent developments in wavefront shaping and neural networks demonstrate effective compensation for scattering effects, showing the promise to project clear images against strong scattering. However, previous studies were mainly restricted to projecting greyscale images using monochromatic light, mainly due to the increased complexity of simultaneously controlling multiple wavelengths. In this work, we fill this blank by developing a projector network, which enables the projection of colorful images through scattering media with three primary colors. To validate the performance of the projector network, we experimentally demonstrated projecting colorful images obtained from the MINST dataset through two stacked diffusers. Quantitatively, the averaged intensity Pearson's correlation coefficient for 1,000 test colorful images reaches about 90.6%, indicating the superiority of the developed network. We anticipate that the projector network can be beneficial to a variety of display applications in scattering environments.

Imaging through a scattering medium via model-driven deep learning

Imaging through a scattering medium is of great significance in many areas. Especially, speckle correlation imaging has been valued for its noninvasiveness. In this work, we report a deep learning solution that incorporates the physical model and an additional regularization for high-fidelity speckle correlation imaging. Without large-scale data to train, the physical model and regularization prior provide a correct direction for neural network to precisely reconstruct hidden objects from speckle under different scattering scenarios and noise levels. Experimental results demonstrate that the proposed method presents a significant advance in improving generalization and combating the invasion of noise.

Formation of multiple complex light structures simultaneously in 3D volume using a single binary phase mask

Complex structure formation inside or through turbid media is a challenging task due to refractive index inhomogeneity, random light scattering, and speckle noise formation. In this article, we have coupled the data regression model in the R-squared metric and used its advantages as a fitness function in the genetic algorithm to advance the resolution and structural uniformity. As a compatible system with the binary genetic algorithm, we have presented a cost-effective iterative wavefront shaping system-design with binary phase modulation using an affordable ferroelectric liquid crystal (FLC) based binary-phase spatial light modulator (SLM). R-squared metric in the genetic algorithm is analyzed to optimize the binary phase mask, and the prototype system based on iterative binary phase modulation has been validated with a 120-grit ground glass diffuser and fresh chicken tissues of thickness 307 [Formula: see text] and 812 [Formula: see text]. The detailed results show that the proposed cost-effective wavefront shaping system with data regression model assisted R-squared fitness function can construct high-resolution multiple complex hetero-structures simultaneously in 3D volume using an optimized single phase-mask.

Low-cost wavefront shaping via the third-order correlation of light fields

In this Letter, inspired by the ghost imaging technique, we propose a wavefront shaping technique based on the third-order correlation of light fields (TCLF). Theoretically, we prove that if the light field fluctuation can be modeled by a complex Gaussian random process with a non-zero mean, the conjugate complex amplitude of the object and a focusing phase factor can be obtained by TCLF when using a single-point detector, which can support wavefront shaping. Experiments demonstrate that TCLF can achieve high-resolution wavefront shaping for scattered fields and scattering-assisted holography without additional operations such as optimization and phase shifting.

Efficiently scanning a focus behind scattering media beyond memory effect by wavefront tilting and re-optimization

One of the main challenges in the wavefront shaping technique is to enable controllable light propagation through scattering media. However, the scanning of the focus generated by wavefront shaping is limited to a small range determined by the optical memory effect (ME). Here, we propose and demonstrate efficiently scanning a focus behind scattering media beyond the ME region using the wavefront tilting and re-optimization (WFT&RO) method. After scanning an initial focus to a desired position by wavefront tilting, our approach utilizes the scanned focus at a new position as the "guide star" to do wavefront re-optimization, which can not only enhance the intensity of the focus to the value before scanning but also accelerate the optimization speed. Repeat such a process, we can theoretically fast scan the focus to any position beyond the ME region while maintaining a relatively uniform intensity. We experimentally demonstrate the power of the method by scanning a focus with uniform intensity values through an optical diffuser within a range that is at least 5 folds larger than the ME region. Additionally, for the case of two cascaded optical diffusers, the scanning range achieved is at least 7 folds larger than the ME region. Our method holds promising implications for applications such as imaging through media, where the ability to control light through scattering media is crucial.

Referenceless characterization of complex media using physics-informed neural networks

In this work, we present a method to characterize the transmission matrices of complex scattering media using a physics-informed, multi-plane neural network (MPNN) without the requirement of a known optical reference field. We use this method to accurately measure the transmission matrix of a commercial multi-mode fiber without the problems of output-phase ambiguity and dark spots, leading to up to 58% improvement in focusing efficiency compared with phase-stepping holography. We demonstrate how our method is significantly more noise-robust than phase-stepping holography and show how it can be generalized to characterize a cascade of transmission matrices, allowing one to control the propagation of light between independent scattering media. This work presents an essential tool for accurate light control through complex media, with applications ranging from classical optical networks, biomedical imaging, to quantum information processing.

High-fidelity and high-robustness free-space ghost transmission in complex media with coherent light source using physics-driven untrained neural network

It is well recognized that it is challenging to realize high-fidelity and high-robustness ghost transmission through complex media in free space using coherent light source. In this paper, we report a new method to realize high-fidelity and high-robustness ghost transmission through complex media by generating random amplitude-only patterns as 2D information carriers using physics-driven untrained neural network (UNN). The random patterns are generated to encode analog signals (i.e., ghost) without any training datasets and labeled data, and are used as information carriers in a free-space optical channel. Coherent light source modulated by the random patterns propagates through complex media, and a single-pixel detector is utilized to collect light intensities at the receiving end. A series of optical experiments have been conducted to verify the proposed approach. Experimental results demonstrate that the proposed method can realize high-fidelity and high-robustness analog-signal (ghost) transmission in complex environments, e.g., around a corner, or dynamic and turbid water. The proposed approach using the designed physics-driven UNN could open an avenue for high-fidelity free-space ghost transmission through complex media.

Learning-based correction with Gaussian constraints for ghost imaging through dynamic scattering media

In this Letter, we propose a learning-based correction method to realize ghost imaging (GI) through dynamic scattering media using deep neural networks with Gaussian constraints. The proposed method learns the wave-scattering mechanism in dynamic scattering environments and rectifies physically existing dynamic scaling factors in the optical channel. The corrected realizations obey a Gaussian distribution and can be used to recover high-quality ghost images. Experimental results demonstrate effectiveness and robustness of the proposed learning-based correction method when imaging through dynamic scattering media is conducted. In addition, only the half number of realizations is needed in dynamic scattering environments, compared with that used in the temporally corrected GI method. The proposed scheme provides a novel, to the best of our knowledge, insight into GI and could be a promising and powerful tool for optical imaging through dynamic scattering media.

Reference-less wavefront shaping in a Hopfield-like rough intensity landscape

This study introduces a new digital-micromirror based binary-phase wavefront shaping technique, which allows the measurement of the full coupling matrix of a disordered medium without a reference and enables to focusing transmitted light. The coupling matrix takes on a bi-dyadic structure, similar to a Hopfield memory matrix containing two memory patterns. Sequential wavefront optimization in this configuration often stalls due to a rough intensity landscape, resulting in a non-optimal state. To overcome this issue, we propose the Complete Couplings Mapping method, which consistently reaches the theoretically expected maximum intensity.

Controlling the transmission of broadband light through scattering media using a digital micromirror device

Wavefront shaping has emerged as a valuable technique in complex photonics, wherein the various eigenmodes of the disordered medium are selectively excited to control the overall transmission through the medium. The process utilizes active optical devices such as liquid crystal-based spatial light modulators (LC-SLM), deformable mirrors (DM), and digital micromirror devices (DMD). Among these, the latter is preferred for imaging through dynamic scattering media such as living biological tissues due to their high-speed refresh rate and increased resolution. This study employs a genetic algorithm along with binary amplitude modulation generated by a digital micromirror device to spatially and spectrally control the large spectral bandwidth through a scattering medium. We illustrate spatial single-point focusing of broadband light, multipoint focusing of broadband light, and programmable spectral filtering of the same through disordered samples.

Complex amplitude field recovery of a scattering media obstructed object with multi-captured images

An iterative-based method for recovering the complex amplitude field behind scattering media is presented in this Letter. This method compensates the random phase modulation of scattering media by using multiple captured scattered light fields. Complex amplitude reconstruction with local iterative averaging of scattered light fields, and double weighted feedback is efficiently applied. Two feasible types of system setups, with varying detector positions and wavelength, are proposed. Simulations and proof-of-concept experiments are employed to demonstrate the effectiveness of the proposed method in reconstructing complex amplitude of a hidden target.

Near-infrared speckle wavemeter based on nonlinear frequency conversion

The wavemeter is an important instrument for spectrum analysis, widely used in spectral calibration, remote sensing, atomic physics, and high-precision metrology. However, near-infrared (NIR) wavemeters require infrared-sensitive detectors that are expensive and less sensitive compared to silicon-based visible light detectors. To circumvent these limitations, we propose an NIR speckle wavemeter based on nonlinear frequency conversion. We combine a scattering medium and the deep learning technique to invert the nonlinear mapping of the NIR wavelength and speckles in the visible wave band. With the outstanding performance of deep learning, a high-precision wavelength resolution of 1 pm is achievable in our experiment. We further demonstrate the robustness of our system and show that the recognition of power parameters and multi-spectral lines is also feasible. The proposed method offers a convenient and flexible way to measure NIR light, and it offers the possibility of cost reduction in miniaturized wavemeter systems.

Optical coherent detection through multi-scattering media by wave-mixing cleaning effect in liquid-crystal OASLM

Liquid-crystal (LC) optically addressable spatial light modulators (OASLMs) allow control of the phase and amplitude of optical beams. By performing wave mixing in an OASLM, we show that coherent phase detection can be achieved for light beams passing through highly scattering media, such as foam layers with several cm thicknesses. Thanks to the adaptive response of our OASLM, the phase information on the speckle signal is transferred at the output of the OASLM to the plane wave reference beam, allowing the cleaning of optical distortions and the direct measurement of amplitude phase modulations with a small diameter single photodiode. A good signal-to-noise ratio (SNR) is demonstrated for foam thickness up to 3 cm. These properties, together with the recently demonstrated sub-ms response time of our OASLM, make the method compatible with foreseen applications for imaging in biomedical tissues and turbid media.