Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue

Neurophotonics beyond the surface: unmasking the brain's complexity exploiting optical scattering

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

High-fidelity and high-speed wavefront shaping by leveraging complex media

High-precision light manipulation is crucial for delivering information through complex media. However, existing spatial light modulation devices face a fundamental speed-fidelity tradeoff. Digital micromirror devices have emerged as a promising candidate for high-speed wavefront shaping but at the cost of compromised fidelity due to the limited control degrees of freedom. Here, we leverage the sparse-to-random transformation through complex media to overcome the dimensionality limitation of spatial light modulation devices. We demonstrate that pattern compression by sparsity-constrained wavefront optimization allows sparse and robust wavefront representations in complex media, improving the projection fidelity without sacrificing frame rate, hardware complexity, or optimization time. Our method is generalizable to different pattern types and complex media, supporting consistent performance with up to 89% and 126% improvements in projection accuracy and speckle suppression, respectively. The proposed optimization framework could enable high-fidelity high-speed wavefront shaping through different scattering media and platforms without changes to the existing holographic setups, facilitating a wide range of physics and real-world applications.

Non-invasive and noise-robust light focusing using confocal wavefront shaping

Wavefront-shaping is a promising approach for imaging fluorescent targets deep inside scattering tissue despite strong aberrations. It enables focusing an incoming illumination into a single spot inside tissue, as well as correcting the outgoing light scattered from the tissue. Previously, wavefront shaping modulations have been successively estimated using feedback from strong fluorescent beads, which have been manually added to a sample. However, such algorithms do not generalize to neurons whose emission is orders of magnitude weaker. We suggest a wavefront shaping approach that works with a confocal modulation of both the illumination and imaging arms. Since the aberrations are corrected in the optics before the detector, the low photon budget is directed into a single sensor spot and detected with high signal-noise ratio. We derive a score function for modulation evaluation from mathematical principles, and successfully use it to image fluorescence neurons, despite scattering through thick tissue.

Optical Analysis Method for Turbid Media Based on Wedge-Shaped Cells at Different Angles

The wedge-shaped sample cell, by offering a comprehensive representation of scattering information in turbid media, significantly enhances the informational content conveyed by spectral images compared to flat sample cells. To further refine the accuracy of turbid medium component detection utilizing wedge-shaped sample cells, this work undertakes modeling and analysis of the influence of different wedge angles on detection precision. In this study, employing a 5° gradient in the incident angle of light, we investigate the impact of incident angles ranging from 10 to 45° on the turbid medium component analysis. Validation experiments are performed by utilizing solutions of Indian ink and fat emulsion at varying ratios. Experimental findings demonstrate that under identical experimental conditions, the wedge-shaped sample cell model at an incident angle of 35° yields optimal analysis results. Utilizing partial least-squares regression (PLSR) for the corresponding optical parameters, the highest value of Rp reached 0.980, with an RMSEP of 0.002. When compared to the model with a 30° incident angle, Rp increased by 0.033, and RMSEP decreased by 0.008. In comparison to the flat sample cell model, Rp increased by 0.041, and RMSEP decreased by 0.004. This study, through continuous variation of wedge angles and PLSR modeling and prediction, further enhances the accuracy of turbid medium component detection, laying an experimental foundation for subsequent analysis of turbid medium components based on wedge-shaped sample cells.

Neurophotonics beyond the Surface: Unmasking the Brain's Complexity Exploiting Optical Scattering

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Acoustic hologram-induced virtual in vivo enhanced waveguide (AH-VIEW)

Optical imaging and phototherapy in deep tissues face notable challenges due to light scattering. We use encoded acoustic holograms to generate three-dimensional acoustic fields within the target medium, enabling instantaneous and robust modulation of the volumetric refractive index, thereby noninvasively controlling the trajectory of light. Through this approach, we achieved a remarkable 24.3% increase in tissue heating rate in vitro photothermal effect tests on porcine skin. In vivo photoacoustic imaging of mouse brain vasculature exhibits an improved signal-to-noise ratio through the intact scalp and skull. These findings demonstrate that our strategy can effectively suppress light scattering in complex biological tissues by inducing low-angle scattering, achieving an effective depth reaching the millimeter scale. The versatility of this strategy extends its potential applications to neuroscience, lithography, and additive manufacturing.

Nonlinear optical response of heme solutions

Heme is the prosthetic group for cytochrome that exists in nearly all living organisms and serves as a vital component of human red blood cells (RBCs). Tunable optical nonlinearity in suspensions of RBCs has been demonstrated previously, however, the nonlinear optical response of a pure heme (without membrane structure) solution has not been studied to our knowledge. In this work, we show optical nonlinearity in two common kinds of heme (i.e., hemin and hematin) solutions by a series of experiments and numerical simulations. We find that the mechanism of nonlinearity in heme solutions is distinct from that observed in the RBC suspensions where the nonlinearity can be easily tuned through optical power, concentration, and the solution properties. In particular, we observe an unusual phenomenon wherein the heme solution exhibits negative optical nonlinearity and render self-collimation of a focused beam at specific optical powers, enabling shape-preserving propagation of light to long distances. Our results may have potential applications in optical imaging and medical diagnosis through blood.

Tight focusing through scattering media via a Bessel-basis transmission matrix

The transmission matrix (TM) is a powerful tool for focusing light through scattering media. Here, we demonstrate a Bessel-basis TM that enables tight focusing through the scattering media and reduces the full width at half maximum of the focus by 23% on average, as compared to the normally used Hadamard-basis TM. To measure the Bessel-basis TM, we establish a common-path inter-mode interferometer (IMI), which can fully utilize the pixels of the spatial light modulator, leading to an enhancement in the peak-to-background intensity ratio (PBR) of the focus. Experimental results suggest that the Bessel-basis TM can achieve a tighter focus behind the scattering media, and the PBR of the focus obtained by the IMI is around 14.3% higher than that achieved using the normal peripheral reference interferometry.

Single-shot digital optical fluorescence phase conjugation through forward multiple-scattering samples

Aberrations and multiple scattering in biological tissues critically distort light beams into highly complex speckle patterns. In this regard, digital optical phase conjugation (DOPC) is a promising technique enabling in-depth focusing. However, DOPC becomes challenging when using fluorescent guide stars for four main reasons: the low photon budget available, the large spectral bandwidth of the fluorescent signal, the Stokes shift between the emission and the excitation wavelength, and the absence of reference beam preventing holographic measurement. Here, we demonstrate the possibility to focus a laser beam through multiple-scattering samples by measuring speckle fields in a single acquisition step with a reference-free, high-resolution wavefront sensor. By taking advantage of the large spectral bandwidth of forward multiply scattering samples, digital fluorescence phase conjugation is achieved to focus a laser beam at the excitation wavelength while measuring the broadband speckle field arising from a micrometer-sized fluorescent bead.

Single-pixel transmission matrix recovery via two-photon fluorescence

Imaging at depth in opaque materials has long been a challenge. Recently, wavefront shaping has enabled notable advance for deep imaging. Nevertheless, most noninvasive wavefront-shaping methods require cameras, lack the sensitivity for deep imaging under weak optical signals, or can only focus on a single "guidestar." Here, we retrieve the transmission matrix (TM) noninvasively using two-photon fluorescence exploiting a single-pixel detection combined with a computational framework, allowing to achieve single-target focus on multiple guidestars spread beyond the memory effect range. In addition, if we assume that memory effect correlations exist in the TM, we are able to substantially reduce the number of measurements needed.

Dynamic polarization fusion network (DPFN) for imaging in different scattering systems

Deep learning has broad applications in imaging through scattering media. Polarization, as a distinctive characteristic of light, exhibits superior stability compared to light intensity within scattering media. Consequently, the de-scattering network trained using polarization is expected to achieve enhanced performance and generalization. For getting optimal outcomes in diverse scattering conditions, it makes sense to train expert networks tailored for each corresponding condition. Nonetheless, it is often unfeasible to acquire the corresponding data for every possible condition. And, due to the uniqueness of polarization, different polarization information representation methods have different sensitivity to different environments. As another of the most direct approaches, a generalist network can be trained with a range of polarization data from various scattering situations, however, it requires a larger network to capture the diversity of the data and a larger training set to prevent overfitting. Here, in order to achieve flexible adaptation to diverse environmental conditions and facilitate the selection of optimal polarization characteristics, we introduce a dynamic learning framework. This framework dynamically adjusts the weights assigned to different polarization components, thus effectively accommodating a wide range of scattering conditions. The proposed architecture incorporates a Gating Network (GTN) that efficiently integrates multiple polarization features and dynamically determines the suitable polarization information for various scenarios. Experimental result demonstrates that the network exhibits robust generalization capabilities across continuous scattering conditions.

Fully Active Delivery of Nanodrugs In Vivo via Remote Optical Manipulation

The active delivery of nanodrugs has been a bottleneck problem in nanomedicine. While modification of nanodrugs with targeting agents can enhance their retention at the lesion location, the transportation of nanodrugs in the circulation system is still a passive process. The navigation of nanodrugs with external forces such as magnetic field has been shown to be effective for active delivery, but the existing techniques are limited to specific materials like magnetic nanoparticles. In this study, an alternative actuation method is proposed based on optical manipulation for remote navigation of nanodrugs in vivo, which is compatible with most of the common drug carriers and exhibits significantly higher manipulation precision. By the programmable scanning of the laser beam, the motion trajectory and velocity of the nanodrugs can be precisely controlled in real time, making it possible for intelligent drug delivery, such as inverse-flow transportation, selective entry into specific vascular branch, and dynamic circumvention across obstacles. In addition, the controlled mass delivery of nanodrugs can be realized through indirect actuation by the microflow field. The developed optical manipulation method provides a new solution for the active delivery of nanodrugs, with promising potential for the treatment of blood diseases such as leukemia and thrombosis.

Wavefront shaping improves the transparency of the scattering media: a review

Significance

Wavefront shaping (WFS) can compensate for distortions by optimizing the wavefront of the input light or reversing the transmission matrix of the media. It is a promising field of research. A thorough understanding of principles and developments of WFS is important for optical research.

Aim

To provide insight into WFS for researchers who deal with scattering in biomedicine, imaging, and optical communication, our study summarizes the basic principles and methods of WFS and reviews recent progress.

Approach

The basic principles, methods of WFS, and the latest applications of WFS in focusing, imaging, and multimode fiber (MMF) endoscopy are described. The practical challenges and prospects of future development are also discussed.

Results

Data-driven learning-based methods are opening up new possibilities for WFS. High-resolution imaging through MMFs can support small-diameter endoscopy in the future.

Conclusion

The rapid development of WFS over the past decade has shown that the best solution is not to avoid scattering but to find ways to correct it or even use it. WFS with faster speed, more optical modes, and more modulation degrees of freedom will continue to drive exciting developments in various fields.

Fast wavefront shaping for two-photon brain imaging with multipatch correction

Nonlinear fluorescence microscopy promotes in-vivo optical imaging of cellular structure at diffraction-limited resolution deep inside scattering biological tissues. Active compensation of tissue-induced aberrations and light scattering through adaptive wavefront correction further extends the accessible depth by restoring high resolution at large depth. However, those corrections are only valid over a very limited field of view within the angular memory effect. To overcome this limitation, we introduce an acousto-optic light modulation technique for fluorescence imaging with simultaneous wavefront correction at pixel scan speed. Biaxial wavefront corrections are first learned by adaptive optimization at multiple locations in the image field. During image acquisition, the learned corrections are then switched on the fly according to the position of the excitation focus during the raster scan. The proposed microscope is applied to in vivo transcranial neuron imaging and demonstrates multi-patch correction of thinned skull-induced aberrations and scattering at 40-kHz data acquisition speed.

Wavefront shaping through a free-form scattering object

Wavefront shaping is a technique to study and control light transport inside scattering media. Wavefront shaping is considered to be applicable to any complex material, yet in most previous studies, the only sample geometries that are studied are slabs or wave-guides. In this paper, we study how macroscopic changes in the sample shape affect light scattering using the wavefront shaping technique. Using a flexible scattering material, we optimize the intensity of light in a focusing spot using wavefront shaping and record the optimized pattern, comparing the enhancement for different curvatures and beam radii. We validate our hypothesis that wavefront shaping has a similar enhancement regardless of the free-form shape of the sample and thus offers relevant potential for industrial applications. We propose a new figure of merit to evaluate the performance of wavefront shaping for different shapes. Surprisingly, based on this figure of merit, we observe that for this particular sample, wavefront shaping has a slightly better performance for a free-form shape than for a slab shape.

Overlapping speckle correlation algorithm for high-resolution imaging and tracking of objects in unknown scattering media

Optical imaging in scattering media is important to many fields but remains challenging. Recent methods have focused on imaging through thin scattering layers or thicker scattering media with prior knowledge of the sample, but this still limits practical applications. Here, we report an imaging method named 'speckle kinetography' that enables high-resolution imaging in unknown scattering media with thicknesses up to about 6 transport mean free paths. Speckle kinetography non-invasively records a series of incoherent speckle images accompanied by object motion and the inherently retained object information is extracted through an overlapping speckle correlation algorithm to construct the object's autocorrelation for imaging. Under single-colour light-emitting diode, white light, and fluorescence illumination, we experimentally demonstrate 1 μm resolution imaging and tracking of objects moving in scattering samples, while reducing the requirements for prior knowledge. We anticipate this method will enable imaging in currently inaccessible scenarios.

Super-resolution imaging through scattering media based on improved triple correlation recursion and deterministic iterative estimation

Iterative phase retrieval algorithms are commonly used in computational techniques and optimization methods to obtain the reconstruction of objects hidden behind opaque scattering media. However, these methods are susceptible to converging to incorrect local minima, and the calculation results tend to be unstable. In this paper, a triple-correlation-based super-resolution imaging (TCSI) framework is proposed to achieve single-shot imaging of unknown objects hidden behind the scattering medium. The amplitude spectrum of the object is obtained by a speckle correlation (SC) method. Iterative relaxation recursion (IRR) sufficiently extracts object information from the triple correlation (TC) of the speckle patterns, serving as the prior initial guess for the iterative estimation algorithm (IE) to obtain a deterministic phase spectrum. Blur correction (BC) is then applied to the diffraction-limited image to achieve super-resolution imaging. Experimental results demonstrate that the flexible framework could effectively overcome the influence of speckle resolution and outperform traditional methods in terms of performance. Our approach provides a basis for non-invasively visualizing various samples behind scattering media.

Optically Programmable Living Microrouter in Vivo

With high reconfigurability and swarming intelligence, programmable medical micromachines (PMMs) represent a revolution in microrobots for executing complex coordinated tasks, especially for dynamic routing of various targets along their respective routes. However, it is difficult to achieve a biocompatible implantation into the body due to their exogenous building blocks. Herein, a living microrouter based on an organic integration of endogenous red blood cells (RBCs), programmable scanning optical tweezers and flexible optofluidic strategy is reported. By harvesting energy from a designed optical force landscape, five RBCs are optically rotated in a controlled velocity and direction, under which, a specific actuation flow is achieved to exert the well-defined hydrodynamic forces on various biological targets, thus enabling a selective routing by integrating three successive functions, i.e., dynamic input, inner processing, and controlled output. Benefited from the optofluidic manipulation, various blood cells, such as the platelets and white blood cells, are transported toward the damaged vessel and cell debris for the dynamic hemostasis and targeted clearance, respectively. Moreover, the microrouter enables a precise transport of nanodrugs for active and targeted delivery in a large quantity. The proposed RBC microrouter might provide a biocompatible medical platform for cell separation, drug delivery, and immunotherapy.

Extrapolated speckle-correlation imaging with an untrained deep neural network

We present a method for speckle-correlation imaging with an extended field of view to observe spatially non-sparse objects. In speckle-correlation imaging, an object is recovered from a non-invasively captured image through a scattering medium by assuming shift-invariance of the optical process called the memory effect. The field of view of speckle-correlation imaging is limited by the size of the memory effect, and it can be extended by extrapolating the speckle correlation in the reconstruction process. However, spatially sparse objects are assumed in the inversion process because of its severe ill-posedness. To address this issue, we introduce a deep image prior, which regularizes the image statistics by using the structure of an untrained convolutional neural network, to speckle-correlation imaging. We experimentally demonstrated the proposed method and showed the possibility of extending the method to imaging through scattering media.

Three-dimensional operando optical imaging of particle and electrolyte heterogeneities inside Li-ion batteries

Understanding (de)lithiation heterogeneities in battery materials is key to ensure optimal electrochemical performance. However, this remains challenging due to the three-dimensional morphology of electrode particles, the involvement of both solid- and liquid-phase reactants and a range of relevant timescales (seconds to hours). Here we overcome this problem and demonstrate the use of confocal microscopy for the simultaneous three-dimensional operando measurement of lithium-ion dynamics in individual agglomerate particles, and the electrolyte in batteries. We examine two technologically important cathode materials: LixCoO2 and LixNi0.8Mn0.1Co0.1O2. The surface-to-core transport velocity of Li-phase fronts and volume changes are captured as a function of cycling rate. Additionally, we visualize heterogeneities in the bulk and at agglomerate surfaces during cycling, and image microscopic liquid electrolyte concentration gradients. We discover that surface-limited reactions and intra-agglomerate competing rates control (de)lithiation and structural heterogeneities in agglomerate-based electrodes. Importantly, the conditions under which optical imaging can be performed inside the complex environments of battery electrodes are outlined.

Extreme transport of light in spheroids of tumor cells

Extreme waves are intense and unexpected wavepackets ubiquitous in complex systems. In optics, these rogue waves are promising as robust and noise-resistant beams for probing and manipulating the underlying material. Localizing large optical power is crucial especially in biomedical systems, where, however, extremely intense beams have not yet been observed. We here discover that tumor-cell spheroids manifest optical rogue waves when illuminated by randomly modulated laser beams. The intensity of light transmitted through bio-printed three-dimensional tumor models follows a signature Weibull statistical distribution, where extreme events correspond to spatially-localized optical modes propagating within the cell network. Experiments varying the input beam power and size indicate that the rogue waves have a nonlinear origin. We show that these nonlinear optical filaments form high-transmission channels with enhanced transmission. They deliver large optical power through the tumor spheroid, and can be exploited to achieve a local temperature increase controlled by the input wave shape. Our findings shed light on optical propagation in biological aggregates and demonstrate how nonlinear extreme event formation allows light concentration in deep tissues, paving the way to using rogue waves in biomedical applications, such as light-activated therapies.

Enhanced light focusing inside scattering media with shaped ultrasound

Light focusing is the primary enabler of various scientific and industrial processes including laser materials processing and microscopy. However, the scattering of light limits the depth at which current methods can operate inside heterogeneous media such as biological tissue, liquid emulsions, and composite materials. Several approaches have been developed to address this issue, but they typically come at the cost of losing spatial or temporal resolution, or increased invasiveness. Here, we show that ultrasound waves featuring a Bessel-like profile can locally modulate the optical properties of a turbid medium to facilitate light guiding. Supported by wave optics and Monte Carlo simulations, we demonstrate how ultrasound enhances light focusing a factor of 7 compared to conventional methods based on placing optical elements outside the complex medium. Combined with point-by-point scanning, images of samples immersed in turbid media with an optical density up to 15, similar to that of weakly scattering biological tissue, can be reconstructed. The quasi-instantaneous generation of the shaped-ultrasound waves, together with the possibility to use transmission and reflection architectures, can pave the way for the real-time control of light inside living tissue.

Single-shot decoherence polarization gated imaging through turbid media

We propose a method for imaging through a turbid medium by using a single-shot decoherence polarization gate (DPG). The DPG is made up of a polarizer, an analyzer, and a weakly scattering medium. Contrary to intuition, we discover that the preferential utilization of sparsely scattered photons by introducing weakly scattering mediums can lead to better image quality. The experimental results show that the visibilities of the images acquired from the DPG imaging method are obviously improved. The contrast of the bar can be increased by 50% by the DPG imaging technique. Furthermore, we study the effect of the volume concentration of the weakly scattering medium on the speckle suppression and the enhancement of the visibilities of the images. The variances of the contrasts of the image show that there exists an optimum optical depth (∼0.8) of the weakly scattering medium for DPG imaging through a specific turbid medium.

NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media

Diffraction-limited optical imaging through scattering media has the potential to transform many applications such as airborne and space-based imaging (through the atmosphere), bioimaging (through skin and human tissue), and fiber-based imaging (through fiber bundles). Existing wavefront shaping methods can image through scattering media and other obscurants by optically correcting wavefront aberrations using high-resolution spatial light modulators-but these methods generally require (i) guidestars, (ii) controlled illumination, (iii) point scanning, and/or (iv) statics scenes and aberrations. We propose neural wavefront shaping (NeuWS), a scanning-free wavefront shaping technique that integrates maximum likelihood estimation, measurement modulation, and neural signal representations to reconstruct diffraction-limited images through strong static and dynamic scattering media without guidestars, sparse targets, controlled illumination, nor specialized image sensors. We experimentally demonstrate guidestar-free, wide field-of-view, high-resolution, diffraction-limited imaging of extended, nonsparse, and static/dynamic scenes captured through static/dynamic aberrations.

Functional imaging through scattering medium via fluorescence speckle demixing and localization

Recently, fluorescence-based optical techniques have emerged as a powerful tool to probe information in the mammalian brain. However, tissue heterogeneities prevent clear imaging of deep neuron bodies due to light scattering. While several up-to-date approaches based on ballistic light allow to retrieve information at shallow depths inside the brain, non-invasive localization and functional imaging at depth still remains a challenge. It was recently shown that functional signals from time-varying fluorescent emitters located behind scattering samples could be retrieved by using a matrix factorization algorithm. Here we show that the seemingly information-less, low-contrast fluorescent speckle patterns recovered by the algorithm can be used to locate each individual emitter, even in the presence of background fluorescence. We test our approach by imaging the temporal activity of large groups of fluorescent sources behind different scattering phantoms mimicking biological tissues, and through a brain slice with a thickness of ∼200 µm.

Optical focusing inside scattering media with iterative time-reversed ultrasonically encoded near-infrared light

Focusing light inside scattering media is a long-sought goal in optics. Time-reversed ultrasonically encoded (TRUE) focusing, which combines the advantages of biological transparency of the ultrasound and the high efficiency of digital optical phase conjugation (DOPC) based wavefront shaping, has been proposed to tackle this problem. By invoking repeated acousto-optic interactions, iterative TRUE (iTRUE) focusing can further break the resolution barrier imposed by the acoustic diffraction limit, showing great potential for deep-tissue biomedical applications. However, stringent requirements on system alignment prohibit the practical use of iTRUE focusing, especially for biomedical applications at the near-infrared spectral window. In this work, we fill this blank by developing an alignment protocol that is suitable for iTRUE focusing with a near-infrared light source. This protocol mainly contains three steps, including rough alignment with manual adjustment, fine-tuning with a high-precision motorized stage, and digital compensation through Zernike polynomials. Using this protocol, an optical focus with a peak-to-background ratio (PBR) of up to 70% of the theoretical value can be achieved. By using a 5-MHz ultrasonic transducer, we demonstrated the first iTRUE focusing using near-infrared light at 1053 nm, enabling the formation of an optical focus inside a scattering medium composed of stacked scattering films and a mirror. Quantitatively, the size of the focus decreased from roughly 1 mm to 160 µm within a few consecutive iterations and a PBR up to 70 was finally achieved. We anticipate that the capability of focusing near-infrared light inside scattering media, along with the reported alignment protocol, can be beneficial to a variety of applications in biomedical optics.

High-definition image transmission through dynamically perturbed multimode fiber by a self-attention based neural network

We implement faithful multimode fiber (MMF) image transmission by a self-attention-based neural network. Compared with a real-valued artificial neural network (ANN) based on a convolutional neural network (CNN), our method utilizes a self-attention mechanism to achieve a higher image quality. The enhancement measure (EME) and structural similarity (SSIM) of the dataset collected in the experiment improved by 0.79 and 0.04; the total number of parameters can be reduced by up to 25%. To enhance the robustness of the neural network to MMF bending in image transmission, we use a simulation dataset to prove that the hybrid training method is helpful in MMF transmission of a high-definition image. Our findings may pave the way for simpler and more robust single-MMF image transmission schemes with hybrid training; SSIM on datasets under different disturbances improve by 0.18. This system has the potential to be applied to various high-demand image transmission tasks, such as endoscopy.

Imaging and positioning through scattering media with double-helix point spread function engineering

Significance

Double-helix point spread function (DH-PSF) microscopy has been developed for three-dimensional (3D) localization and imaging at super-resolution but usually in environments with no or weak scattering. To date, super-resolution imaging through turbid media has not been reported.

Aim

We aim to explore the potential of DH-PSF microscopy in the imaging and localization of targets in scattering environments for improved 3D localization accuracy and imaging quality.

Approach

The conventional DH-PSF method was modified to accommodate the scanning strategy combined with a deconvolution algorithm. The localization of a fluorescent microsphere is determined by the center of the corresponding double spot, and the image is reconstructed from the scanned data by deconvoluting the DH-PSF.

Results

The resolution, i.e., the localization accuracy, was calibrated to 13 nm in the transverse plane and 51 nm in the axial direction. Penetration thickness could reach an optical thickness (OT) of 5. Proof-of-concept imaging and the 3D localization of fluorescent microspheres through an eggshell membrane and an inner epidermal membrane of an onion are presented to demonstrate the super-resolution and optical sectioning capabilities.

Conclusions

Modified DH-PSF microscopy can image and localize targets buried in scattering media using super-resolution. Combining fluorescent dyes, nanoparticles, and quantum dots, among other fluorescent probes, the proposed method may provide a simple solution for visualizing deeper and clearer in/through scattering media, making in situ super-resolution microscopy possible for various demanding applications.

Adaptive optics for optical microscopy [Invited]

Optical microscopy is widely used to visualize fine structures. When applied to bioimaging, its performance is often degraded by sample-induced aberrations. In recent years, adaptive optics (AO), originally developed to correct for atmosphere-associated aberrations, has been applied to a wide range of microscopy modalities, enabling high- or super-resolution imaging of biological structure and function in complex tissues. Here, we review classic and recently developed AO techniques and their applications in optical microscopy.

Using speckle correlations for single-shot 3D imaging

Recovery of a 3D object behind a scattering medium is an important problem in many fields, including biomedical and defense applications. Speckle correlation imaging can recover objects in a single shot but contains no depth information. To date, its extension to 3D recovery has relied on multiple measurements, multi-spectral light, or pre-calibration of the speckle with a reference object. Here, we show that the presence of a point source behind the scatterer enables single-shot reconstruction of multiple objects at multiple depths. The method relies on speckle scaling from the axial memory effect, in addition to the transverse one, and recovers objects directly, without the need for phase retrieval. We provide simulation and experimental results to show object reconstructions at different depths with a single-shot measurement. We also provide theoretical principles describing the region where speckle scales with axial distance and its effects on the depth of field. Our technique will be useful where a natural point source exists, such as fluorescence imaging or car headlights in fog.

High-gain and high-speed wavefront shaping through scattering media

Wavefront shaping (WFS) is emerging as a promising tool for controlling and focusing light in complex scattering media. The shaping system's speed, the energy gain of the corrected wavefronts, and the control degrees of freedom (DOF) are the most important metrics for WFS, especially for highly scattering and dynamic samples. Despite recent advances, current methods suffer from trade-offs that limit satisfactory performance to only one or two of these metrics. Here, we report a WFS technique that simultaneously achieves high speed, high energy gain, and high control DOF. By combining photorefractive crystal-based analog optical phase conjugation (AOPC) and stimulated emission light amplification, our technique achieves an energy gain approaching unity, more than three orders of magnitude larger than conventional AOPC. The response time of ~10 μs with about 106 control modes corresponds to an average mode time of about 0.01 ns/mode, which is more than 50 times lower than some of the fastest WFS systems to date. We anticipate that this technique will be instrumental in overcoming the optical diffusion limit in photonics and translate WFS techniques to real-world applications.

All-optical image classification through unknown random diffusers using a single-pixel diffractive network

Classification of an object behind a random and unknown scattering medium sets a challenging task for computational imaging and machine vision fields. Recent deep learning-based approaches demonstrated the classification of objects using diffuser-distorted patterns collected by an image sensor. These methods demand relatively large-scale computing using deep neural networks running on digital computers. Here, we present an all-optical processor to directly classify unknown objects through unknown, random phase diffusers using broadband illumination detected with a single pixel. A set of transmissive diffractive layers, optimized using deep learning, forms a physical network that all-optically maps the spatial information of an input object behind a random diffuser into the power spectrum of the output light detected through a single pixel at the output plane of the diffractive network. We numerically demonstrated the accuracy of this framework using broadband radiation to classify unknown handwritten digits through random new diffusers, never used during the training phase, and achieved a blind testing accuracy of 87.74 ± 1.12%. We also experimentally validated our single-pixel broadband diffractive network by classifying handwritten digits "0" and "1" through a random diffuser using terahertz waves and a 3D-printed diffractive network. This single-pixel all-optical object classification system through random diffusers is based on passive diffractive layers that process broadband input light and can operate at any part of the electromagnetic spectrum by simply scaling the diffractive features proportional to the wavelength range of interest. These results have various potential applications in, e.g., biomedical imaging, security, robotics, and autonomous driving.

Optimal Cooling of Multiple Levitated Particles through Far-Field Wavefront Shaping

Light forces can be harnessed to levitate mesoscopic objects and cool them down toward their motional quantum ground state. Roadblocks on the way to scale up levitation from a single to multiple particles in close proximity are the requirements to constantly monitor the particles' positions as well as to engineer light fields that react fast and appropriately to their movements. Here, we present an approach that solves both problems at once. By exploiting the information stored in a time-dependent scattering matrix, we introduce a formalism enabling the identification of spatially modulated wavefronts, which simultaneously cool down multiple objects of arbitrary shapes. An experimental implementation is suggested based on stroboscopic scattering-matrix measurements and time-adaptive injections of modulated light fields.

Functional imaging through scattering medium via fluorescence speckle demixing and localization

Recently, fluorescence-based optical techniques have emerged as a powerful tool to probe information in the mammalian brain. However, tissue heterogeneities prevent clear imaging of deep neuron bodies due to light scattering. While several up-to-date approaches based on ballistic light allow to retrieve information at shallow depths inside the brain, non-invasive localization and functional imaging at depth still remains a challenge. It was recently shown that functional signals from time-varying fluorescent emitters located behind scattering samples could be retrieved by using a matrix factorization algorithm. Here we show that the seemingly information-less, low-contrast fluorescent speckle patterns recovered by the algorithm can be used to locate each individual emitter, even in the presence of background fluorescence. We test our approach by imaging the temporal activity of large groups of fluorescent sources behind different scattering phantoms mimicking biological tissues, and through a brain slice with a thickness of ~200 micron.

Speckle autocorrelation separation for multi-target scattering imaging

Imaging through scattering media remains a big challenge in optics while the single-shot non-invasive speckle autocorrelation technique (SAT) is well-known as a promising way to handle it. However, it usually cannot recover a large-scale target or multiple isolated small ones due to the limited effective range of the optical memory effect (OME). In this paper, we propose a multi-target scattering imaging scheme by combining the traditional SA algorithm with a Deep Learning (DL) strategy. The basic idea is to extract each autocorrelation component of every target from the autocorrelation result of a mixed speckle using a suitable DL method. Once we get all the expected autocorrelation components, a typical phase retrieval algorithm (PRA) could be applied to reveal the shapes of all those corresponding small targets. In our experimental demonstration, up to five isolated targets are successfully recovered.

Brillouin Interaction between Two Optical Modes Selectively Excited in Weakly Guiding Multimode Optical Fibers

A multimode optical fiber supports excitation and propagation of a pure single optical mode, i.e., the field pattern that satisfies the boundary conditions and does not change along the fiber. When two counterpropagating pure optical modes are excited, they could interact through the stimulated Brillouin scattering (SBS) process. Here, we present a simple theoretical formalism describing SBS interaction between two individual optical modes selectively excited in an acoustically isotropic multimode optical fiber. Employing a weakly guiding step-index fiber approach, we have built an analytical expression for the spatial distribution of the sound field amplitude in the fiber core and explored the features of SBS gain spectra, describing the interaction between modes of different orders. In this way, we give a clear insight into the sound propagation effects accompanying SBS in multimode optical fibers, and demonstrate their specific contributions to the SBS gain spectrum.

Efficient dispersion modeling in optical multimode fiber

Dispersion remains an enduring challenge for the characterization of wavelength-dependent transmission through optical multimode fiber (MMF). Beyond a small spectral correlation width, a change in wavelength elicits a seemingly independent distribution of the transmitted field. Here we report on a parametric dispersion model that describes mode mixing in MMF as an exponential map and extends the concept of principal modes to describe the fiber's spectrally resolved transmission matrix (TM). We present computational methods to fit the model to measurements at only a few, judiciously selected, discrete wavelengths. We validate the model in various MMF and demonstrate an accurate estimation of the full TM across a broad spectral bandwidth, approaching the bandwidth of the best-performing principal modes, and exceeding the original spectral correlation width by more than two orders of magnitude. The model allows us to conveniently study the spectral behavior of principal modes, and obviates the need for dense spectral measurements, enabling highly efficient reconstruction of the multispectral TM of MMF.

Self-attention module in a multi-scale improved U-net (SAM-MIU-net) motivating high-performance polarization scattering imaging

Polarization imaging has outstanding advantages in the field of scattering imaging, which still encounters great challenges in heavy scattering media systems even though there are helps from deep learning technology. In this paper, we propose a self-attention module (SAM) in multi-scale improved U-net (SAM-MIU-net) for the polarization scattering imaging, which can extract a new combination of multidimensional information from targets effectively. The proposed SAM-MIU-net can focus on the stable feature carried by polarization characteristics of the target, so as to enhance the expression of the available features, and make it easier to extract polarization features which help to recover the detail of targets for the polarization scattering imaging. Meanwhile, the SAM's effectiveness has been verified in a series of experiments. Based on proposed SAM-MIU-net, we have investigated the generalization abilities for the targets' structures and materials, and the imaging distances between the targets and the ground glass. Experimental results demonstrate that our proposed SAM-MIU-net can achieve high-precision reconstruction of target information under incoherent light conditions for the polarization scattering imaging.

Light scattering control with the two-step focusing method based on neural networks and multi-pixel coding

Focusing light through scattering media is essential for high-resolution optical imaging and deep penetration. Here, a two-step focusing method based on neural networks (NNs) and multi-pixel coding is proposed to achieve high-quality focusing with theoretical maximum enhancement. In the first step, a single-layer neural network (SLNN) is used to obtain the initial mask, which can be used to focus with a moderate enhancement. In the second step, we use multi-pixel coding to encode the initial mask. The coded masks and their corresponding speckle patterns are used to train another SLNN to get the final mask and achieve high-quality focusing. In this experiment, for a mask of 16 × 16 modulation units, in the case of using 8 pixels in a modulation unit, focus with the enhancement of 40.3 (only 0.44 less than the theoretical value) has been achieved with 3000 pictures (1000 pictures in the first step and 2000 pictures in the second step). Compared with the case of employing only the initial mask and the direct multi-pixel encoded mask, the enhancement is increased by 220% and 24%. The proposed method provides a new idea for improving the focusing effect through the scattering media using NNs.

High-speed single-exposure time-reversed ultrasonically encoded optical focusing against dynamic scattering

Focusing light deep inside live scattering tissue promises to revolutionize biophotonics by enabling deep tissue noninvasive optical imaging, manipulation, and therapy. By combining with guide stars, wavefront shaping is emerging as a powerful tool to make scattering media optically transparent. However, for in vivo biomedical applications, the speeds of existing techniques are still too slow to accommodate the fast speckle decorrelation of live tissue. To address this key bottleneck, we develop a quaternary phase encoding scheme to enable single-exposure time-reversed ultrasonically encode optical focusing with full-phase modulations. Specifically, we focus light inside dynamic scattering media with an average mode time down to 29 ns, which indicates that more than 10⁴ effective spatial modes can be controlled within 1 millisecond. With this technique, we demonstrate in vivo light focusing in between a highly opaque adult zebrafish of 5.1 millimeters in thickness and a ground glass diffuser. Our work presents an important step toward in vivo deep tissue applications of wavefront shaping.

One-to-all lightweight Fourier channel attention convolutional neural network for speckle reconstructions

Speckle reconstruction is a classical inverse problem in computational imaging. Inspired by the memory effect of the scattering medium, deep learning methods reveal excellent performance in extracting the correlation of speckle patterns. Nowadays, advanced models generally include more than 10M parameters and mostly pay more attention to the spatial feature information. However, the frequency domain of images also contains precise hierarchical representations. Here we propose a one-to-all lightweight Fourier channel attention convolutional neural network (FCACNN) with Fourier channel attention and the res-connected bottleneck structure. Compared with the state-of-the-art model, i.e., self-attention armed convolutional neural network (SACNN), our architecture has better feature extraction and reconstruction ability. The Pearson correlation coefficient and Jaccard index scores of FCACNN increased by at least 5.2% and 13.6% compared with task-related models. And the parameter number of the lightweight FCACNN is only 1.15M. Furthermore, the validation results show that the one-to-all model, FCACNN, has excellent generalization capability on unseen speckle patterns such as handwritten letters and Quickdraws.

Scattering correcting wavefront shaping for three-photon microscopy

Three-photon (3P) microscopy is getting traction due to its superior performance in deep tissues. Yet, aberrations and light scattering still pose one of the main limitations in the attainable depth ranges for high-resolution imaging. Here, we show scattering correcting wavefront shaping with a simple continuous optimization algorithm, guided by the integrated 3P fluorescence signal. We demonstrate focusing and imaging behind scattering layers and investigate convergence trajectories for different sample geometries and feedback non-linearities. Furthermore, we show imaging through a mouse skull and demonstrate a novel, to the best of our knowledge, fast phase estimation scheme that substantially increases the speed at which the optimal correction can be found.

Piezoelectric-based optical modulator for miniaturized wireless medical implants

Optical links for medical implants have recently been explored as an attractive option primarily because it provides a route to ultrasmall wireless implant systems. Existing devices for optical communication either are not CMOS compatible, require large bias voltages to operate, or consume substantial amounts of power. Here, we present a high-Q CMOS-compatible electro-optic modulator that enables establishing an optical data uplink to implants. The modulator acts as a pF-scale capacitor, requires no bias voltage, and operates at CMOS voltages of down to 0.5V. We believe this technology would provide a path towards the realization of millimeter (mm)- and sub-mm scale wireless implants for use in bio-sensing applications.

Simple method to modulate the scattered light field under strong disturbance

In this Letter, we propose a simple and robust method that we have named an optimal accumulation algorithm (OAA) to modulate a scattered light field. Compared with the simulated annealing algorithm (SAA) and genetic algorithm (GA), the OAA is very robust, that is to say it has a strong anti-disturbance capability. In experiments, the scattered light field through ground glass and a polystyrene suspension was modulated, where the polystyrene suspension supported a dynamic random disturbance. It was found that, even if the suspension is too thick to see the ballistic light, the OAA can still modulate the scattered field effectively, while the SAA and GA completely failed. In addition, the OAA is so simple that only addition and comparison are needed, and it can achieve multi-target modulation.

Using beam-offset optical coherence tomography to reconstruct backscattered photon profiles in scattering media

Raster scanning imaging technologies capture least scattered photons (LSPs) and reject multiple scattered photons (MSPs) in backscattered photons to image the underlying structures of a scattering medium. However, MSPs can still squeeze into the images, resulting in limited imaging depth, degraded contrast, and significantly reduced lateral resolution. Great efforts have been made to understand how MSPs affect imaging performance through modeling, but the techniques for visualizing the backscattered photon profile (BSPP) in scattering media during imaging are unavailable. Here, a method of reconstructing BSPP is demonstrated using beam-offset optical coherence tomography (OCT), in which OCT images are acquired at offset positions from the illumination beam. The separation of LSPs and MSPs based on the BSPP enables quantification of imaging depth, contrast, and lateral resolution, as well as access to the depth-resolved modulated transfer function (MTF). This approach presents great opportunities for better retrieving tissue optical properties, correctly interpreting images, or directly using MTF as the feedback for adaptive optical imaging.

Coherent enhancement of optical remission in diffusive media

Remitted waves are used for sensing and imaging in diverse diffusive media from the Earth's crust to the human brain. Separating the source and detector increases the penetration depth of light, but the signal strength decreases rapidly, leading to a poor signal-to-noise ratio. Here, we show, experimentally and numerically, that wavefront shaping a laser beam incident on a diffusive sample enables an enhancement of remission by an order of magnitude at depths of up to 10 transport mean free paths. We develop a theoretical model which predicts the maximal remission enhancement. Our analysis reveals a significant improvement in the sensitivity of remitted waves to local changes of absorption deep inside diffusive media. This work illustrates the potential of coherent wavefront control for noninvasive diffuse wave imaging applications, such as diffuse optical tomography and functional near-infrared spectroscopy.

Nano-electromechanical spatial light modulator enabled by asymmetric resonant dielectric metasurfaces

Spatial light modulators (SLMs) play essential roles in various free-space optical technologies, offering spatio-temporal control of amplitude, phase, or polarization of light. Beyond conventional SLMs based on liquid crystals or microelectromechanical systems, active metasurfaces are considered as promising SLM platforms because they could simultaneously provide high-speed and small pixel size. However, the active metasurfaces reported so far have achieved either limited phase modulation or low efficiency. Here, we propose nano-electromechanically tunable asymmetric dielectric metasurfaces as a platform for reflective SLMs. Exploiting the strong asymmetric radiation of perturbed high-order Mie resonances, the metasurfaces experimentally achieve a phase-shift close to 290^∘, over 50% reflectivity, and a wavelength-scale pixel size. Electrical control of diffraction patterns is also achieved by displacing the Mie resonators using nano-electro-mechanical forces. This work paves the ways for future exploration of the asymmetric metasurfaces and for their application to the next-generation SLMs.

This work experimentally demonstrates nano-electromechanically tunable asymmetric dielectric metasurfaces. The metasurfaces enable large phase tuning, high reflection, a wavelength-scale pixel size, and electrical control of diffraction patterns.

Feedback-based wavefront shaping for weak light with lock-in beat frequency detection

Feedback-based wavefront shaping is a promising and versatile technique for enhancing the contrast of a target signal through highly scattering media. However, this technique can fail for low optical signals such as fluorescence and Raman signals or in a reflection setup because the trend in weak feedback signals can be easily overwhelmed by noise. To address this challenge, we develop a technique based on a single acousto-optic deflector (AOD) to create a signal with a selected beat frequency from optical signals that can serve as feedback, in which the phase distribution of various radio frequency components of the driving signal for the AOD is optimized for wavefront shaping. By shifting incident light frequency with the AOD, the feedback signal at a selected beat frequency can be measured with a high signal-to-noise ratio (SNR) by a lock-in amplifier, thus enabling the enhancement of weak target signals through highly scattering media. It is found that the method of lock-in beat frequency detection can significantly improve fluorescence imaging and Raman spectral measurements in a reflection setup, and thus could be potentially used for in vivo measurements.

Sum rules for energy deposition eigenchannels in scattering systems

In a random-scattering system, the deposition matrix maps the incident wavefront onto the internal field distribution across a target volume. The corresponding eigenchannels have been used to enhance the wave energy delivered to the target. Here, we find the sum rules for the eigenvalues and eigenchannels of the deposition matrix in any system geometry: including two- and three-dimensional scattering systems, as well as narrow waveguides and wide slabs. We derive a number of constraints on the eigenchannel intensity distributions inside the system as well as the corresponding eigenvalues. Our results are general and applicable to random systems of arbitrary scattering strength as well as different types of waves including electromagnetic waves, acoustic waves, and matter waves.

Improved photoacoustic images via wavefront shaping modulation based on the scattering structure

Multispectral optoacoustic tomography (MSOT) has become the dominant technical solution for photoacoustic imaging (PAI). However, the laser source of fiber output in the current MSOT method is typically a TEM00 Gaussian beam, which is prone to artifacts and incomplete due to the uneven distribution of the irradiated light intensity. Here, we propose a novel method to improve the quality of photoacoustic image reconstruction by modulating the wavefront shaping of the incident laser beam based on the designed scattering structure. In the experiment, we add the designed scattering structure to the current hemispherical photoacoustic transducer array device. Through experiments and simulations, we investigate and compare the effects of different scattering structures on laser intensity modulation. The results show that an ED1-C20 diffusion structure with a scattering angle of 20 degrees has the most effective modulation of the beam intensity distribution. And we choose gold nanoparticles of 50-100 nanometers (nm) diameters and index finger capillary vessels respectively as the medium of PAI. We obtain the highest ratio of PAI area increases of gold nanoparticles and index finger to devices compare without scattering structure is 29.69% and 634.94%, respectively. Experimental results demonstrate that our method is significantly higher quality than traditional methods, which has great potential for theoretical application in medical PAI.