Distortion matrix concept for deep optical imaging in scattering media

Removing Artifacts in Transcranial Photoacoustic Imaging With Polarized Self-Attention Dense-UNet

OBJECTIVE

Photoacoustic imaging (PAI) is a promising transcranial imaging technique. However, the distortion of photoacoustic signals induced by the skull significantly influences its imaging quality. We aimed to use deep learning for removing artifacts in PAI.

METHODS

In this study, we propose a polarized self-attention dense U-Net, termed PSAD-UNet, to correct the distortion and accurately recover imaged objects beneath bone plates. To evaluate the performance of the proposed method, a series of experiments was performed using a custom-built PAI system.

RESULTS

The experimental results showed that the proposed PSAD-UNet method could effectively implement transcranial PAI through a one- or two-layer bone plate. Compared with the conventional delay-and-sum and classical U-Net methods, PSAD-UNet can diminish the influence of bone plates and provide high-quality PAI results in terms of structural similarity and peak signal-to-noise ratio. The 3-D experimental results further confirm the feasibility of PSAD-UNet in 3-D transcranial imaging.

CONCLUSION

PSAD-UNet paves the way for implementing transcranial PAI with high imaging accuracy, which reveals broad application prospects in preclinical and clinical fields.

Harnessing forward multiple scattering for optical imaging deep inside an opaque medium

As light travels through a disordered medium such as biological tissues, it undergoes multiple scattering events. This phenomenon is detrimental to in-depth optical microscopy, as it causes a drastic degradation of contrast, resolution and brightness of the resulting image beyond a few scattering mean free paths. However, the information about the inner reflectivity of the sample is not lost; only scrambled. To recover this information, a matrix approach of optical imaging can be fruitful. Here, we report on a de-scanned measurement of a high-dimension reflection matrix R via low coherence interferometry. Then, we show how a set of independent focusing laws can be extracted for each medium voxel through an iterative multi-scale analysis of wave distortions contained in R. It enables an optimal and local compensation of forward multiple scattering paths and provides a three-dimensional confocal image of the sample as the latter one had become digitally transparent. The proof-of-concept experiment is performed on a human opaque cornea and an extension of the penetration depth by a factor five is demonstrated compared to the state-of-the-art.

Acoustic hologram-induced virtual in vivo enhanced waveguide (AH-VIEW)

Optical imaging and phototherapy in deep tissues face notable challenges due to light scattering. We use encoded acoustic holograms to generate three-dimensional acoustic fields within the target medium, enabling instantaneous and robust modulation of the volumetric refractive index, thereby noninvasively controlling the trajectory of light. Through this approach, we achieved a remarkable 24.3% increase in tissue heating rate in vitro photothermal effect tests on porcine skin. In vivo photoacoustic imaging of mouse brain vasculature exhibits an improved signal-to-noise ratio through the intact scalp and skull. These findings demonstrate that our strategy can effectively suppress light scattering in complex biological tissues by inducing low-angle scattering, achieving an effective depth reaching the millimeter scale. The versatility of this strategy extends its potential applications to neuroscience, lithography, and additive manufacturing.

Single-shot digital optical fluorescence phase conjugation through forward multiple-scattering samples

Aberrations and multiple scattering in biological tissues critically distort light beams into highly complex speckle patterns. In this regard, digital optical phase conjugation (DOPC) is a promising technique enabling in-depth focusing. However, DOPC becomes challenging when using fluorescent guide stars for four main reasons: the low photon budget available, the large spectral bandwidth of the fluorescent signal, the Stokes shift between the emission and the excitation wavelength, and the absence of reference beam preventing holographic measurement. Here, we demonstrate the possibility to focus a laser beam through multiple-scattering samples by measuring speckle fields in a single acquisition step with a reference-free, high-resolution wavefront sensor. By taking advantage of the large spectral bandwidth of forward multiply scattering samples, digital fluorescence phase conjugation is achieved to focus a laser beam at the excitation wavelength while measuring the broadband speckle field arising from a micrometer-sized fluorescent bead.

Three-dimensional ultrasound matrix imaging

Matrix imaging paves the way towards a next revolution in wave physics. Based on the response matrix recorded between a set of sensors, it enables an optimized compensation of aberration phenomena and multiple scattering events that usually drastically hinder the focusing process in heterogeneous media. Although it gave rise to spectacular results in optical microscopy or seismic imaging, the success of matrix imaging has been so far relatively limited with ultrasonic waves because wave control is generally only performed with a linear array of transducers. In this paper, we extend ultrasound matrix imaging to a 3D geometry. Switching from a 1D to a 2D probe enables a much sharper estimation of the transmission matrix that links each transducer and each medium voxel. Here, we first present an experimental proof of concept on a tissue-mimicking phantom through ex-vivo tissues and then, show the potential of 3D matrix imaging for transcranial applications.

Aberration free synthetic aperture second harmonic generation holography

Second harmonic generation (SHG) microscopy is a valuable tool for optical microscopy. SHG microscopy is normally performed as a point scanning imaging method, which lacks phase information and is limited in spatial resolution by the spatial frequency support of the illumination optics. In addition, aberrations in the illumination are difficult to remove. We propose and demonstrate SHG holographic synthetic aperture holographic imaging in both the forward (transmission) and backward (epi) imaging geometries. By taking a set of holograms with varying incident angle plane wave illumination, the spatial frequency support is increased and the input and output pupil phase aberrations are estimated and corrected - producing diffraction limited SHG imaging that combines the spatial frequency support of the input and output optics. The phase correction algorithm is computationally efficient and robust and can be applied to any set of measured field imaging data.

Enhanced secrecy in optical communication using speckle from multiple scattering layers

We study the secrecy of an optical communication system with two scattering layers, to hide both the sender and receiver, by measuring the correlation of the intermediate speckle generated between the two layers. The binary message is modulated as spatially shaped wavefronts, and the high number of transmission modes of the scattering layers allows for many uncorrelated incident wavefronts to send the same message, making it difficult for an attacker to intercept or decode the message and thus increasing secrecy. We collect 50,000 intermediate speckle patterns and analyze their correlation distribution using the Kolmogorov-Smirnov (K-S) test. We search for further correlations using the K-Means and Hierarchical unsupervised classification algorithms. We find no correlation between the intermediate speckle and the message, suggesting a person-in-the-middle attack is not possible. This method is compatible with any digital encryption method and is applicable for codifications in optical wireless communication (OWC).

Unlocking the secrets of the invisible world: incredible deep optical imaging through in-silico clearing

In-silico clearing enables deep optical imaging of biological samples by correcting image blur caused by scattering and aberration. This breakthrough method offers researchers unprecedented insights into three-dimensional biological systems, with enormous potential for advancing biology and medicine to better understand living organisms and human health.

Efficient framework of solving time-gated reflection matrix for imaging through turbid medium

Imaging through turbid medium is a long pursuit in many research fields, such as biomedicine, astronomy and automatic vehicle, in which the reflection matrix-based method is a promising solution. However, the epi-detection geometry suffers from round-trip distortion and it is challenging to isolate the input and output aberrations in non-ideal cases due to system imperfections and measurement noises. Here, we present an efficient framework based on single scattering accumulation together with phase unwrapping that can accurately separate input and output aberrations from the noise-affected reflection matrix. We propose to only correct the output aberration while suppressing the input aberration by incoherent averaging. The proposed method is faster in convergence and more robust against noise, avoiding precise and tedious system adjustments. In both simulations and experiments, we demonstrate the diffraction-limited resolution capability under optical thickness beyond 10 scattering mean free paths, showing the potential of applications in neuroscience and dermatology.

In-silico clearing approach for deep refractive index tomography by partial reconstruction and wave-backpropagation

Refractive index (RI) is considered to be a fundamental physical and biophysical parameter in biological imaging, as it governs light-matter interactions and light propagation while reflecting cellular properties. RI tomography enables volumetric visualization of RI distribution, allowing biologically relevant analysis of a sample. However, multiple scattering (MS) and sample-induced aberration (SIA) caused by the inhomogeneity in RI distribution of a thick sample make its visualization challenging. This paper proposes a deep RI tomographic approach to overcome MS and SIA and allow the enhanced reconstruction of thick samples compared to that enabled by conventional linear-model-based RI tomography. The proposed approach consists of partial RI reconstruction using multiple holograms acquired with angular diversity and their backpropagation using the reconstructed partial RI map, which unambiguously reconstructs the next partial volume. Repeating this operation efficiently reconstructs the entire RI tomogram while suppressing MS and SIA. We visualized a multicellular spheroid of diameter 140 µm within minutes of reconstruction, thereby demonstrating the enhanced deep visualization capability and computational efficiency of the proposed method compared to those of conventional RI tomography. Furthermore, we quantified the high-RI structures and morphological changes inside multicellular spheroids, indicating that the proposed method can retrieve biologically relevant information from the RI distribution. Benefitting from the excellent biological interpretability of RI distributions, the label-free deep visualization capability of the proposed method facilitates a noninvasive understanding of the architecture and time-course morphological changes of thick multicellular specimens.

Ultrasound Matrix Imaging-Part II: The Distortion Matrix for Aberration Correction Over Multiple Isoplanatic Patches

This is the second article in a series of two which report on a matrix approach for ultrasound imaging in heterogeneous media. This article describes the quantification and correction of aberration, i.e. the distortion of an image caused by spatial variations in the medium speed-of-sound. Adaptive focusing can compensate for aberration, but is only effective over a restricted area called the isoplanatic patch. Here, we use an experimentally-recorded matrix of reflected acoustic signals to synthesize a set of virtual transducers. We then examine wave propagation between these virtual transducers and an arbitrary correction plane. Such wave-fronts consist of two components: (i) An ideal geometric wave-front linked to diffraction and the input focusing point, and; (ii) Phase distortions induced by the speed-of-sound variations. These distortions are stored in a so-called distortion matrix, the singular value decomposition of which gives access to an optimized focusing law at any point. We show that, by decoupling the aberrations undergone by the outgoing and incoming waves and applying an iterative strategy, compensation for even high-order and spatially-distributed aberrations can be achieved. After a numerical validation of the process, ultrasound matrix imaging (UMI) is applied to the in-vivo imaging of a gallbladder. A map of isoplanatic modes is retrieved and is shown to be strongly correlated with the arrangement of tissues constituting the medium. The corresponding focusing laws yield an ultrasound image with drastically improved contrast and transverse resolution. UMI thus provides a flexible and powerful route towards computational ultrasound.

Ultrasound Matrix Imaging-Part I: The Focused Reflection Matrix, the F-Factor and the Role of Multiple Scattering

This is the first article in a series of two dealing with a matrix approach for aberration quantification and correction in ultrasound imaging. Advanced synthetic beamforming relies on a double focusing operation at transmission and reception on each point of the medium. Ultrasound matrix imaging (UMI) consists in decoupling the location of these transmitted and received focal spots. The response between those virtual transducers form the so-called focused reflection matrix that actually contains much more information than a confocal ultrasound image. In this paper, a time-frequency analysis of this matrix is performed, which highlights the single and multiple scattering contributions as well as the impact of aberrations in the monochromatic and broadband regimes. Interestingly, this analysis enables the measurement of the incoherent input-output point spread function at any pixel of this image. A fitting process enables the quantification of the single scattering, multiple scattering and noise components in the image. From the single scattering contribution, a focusing criterion is defined, and its evolution used to quantify the amount of aberration throughout the ultrasound image. In contrast to the state-of-the-art coherence factor, this new indicator is robust to multiple scattering and electronic noise, thereby providing a contrasted map of the focusing quality at a much better transverse resolution. After a validation of the proof-of-concept based on time-domain simulations, UMI is applied to the in-vivo study of a human calf. Beyond this specific example, UMI opens a new route for speed-of-sound and scattering quantification in ultrasound imaging.

Coherent enhancement of optical remission in diffusive media

Remitted waves are used for sensing and imaging in diverse diffusive media from the Earth's crust to the human brain. Separating the source and detector increases the penetration depth of light, but the signal strength decreases rapidly, leading to a poor signal-to-noise ratio. Here, we show, experimentally and numerically, that wavefront shaping a laser beam incident on a diffusive sample enables an enhancement of remission by an order of magnitude at depths of up to 10 transport mean free paths. We develop a theoretical model which predicts the maximal remission enhancement. Our analysis reveals a significant improvement in the sensitivity of remitted waves to local changes of absorption deep inside diffusive media. This work illustrates the potential of coherent wavefront control for noninvasive diffuse wave imaging applications, such as diffuse optical tomography and functional near-infrared spectroscopy.

An affordable and easy-to-use tool for automatic fish length and weight estimation in mariculture

Common aquaculture practices involve measuring fish biometrics at different growth stages, which is crucial for feeding regime management and for improving farmed fish welfare. Fish measurements are usually carried out manually on individual fish. However, this process is laborious, time-consuming, and stressful to the fish. Therefore, the development of fast, precise, low cost and indirect measurement would be of great interest to the aquaculture sector. In this study, we explore a promising way to take fish measurements in a non-invasive approach through computer vision. Images captured by a stereoscopic camera are used by Artificial Intelligence algorithms in conjunction with computer vision to automatically obtain an accurate estimation of the characteristics of fish, such as body length and weight. We describe the development of a computer vision system for automated recognition of body traits through image processing and linear models for the measurement of fish length and prediction of body weight. The measurements are obtained through a relatively low-cost prototype consisting of a smart buoy equipped with stereo cameras, tested in a commercial mariculture cage in the Mediterranean Sea. Our findings suggest that this method can successfully estimate fish biometric parameters, with a mean error of ± 1.15 cm.

Through-skull brain imaging in vivo at visible wavelengths via dimensionality reduction adaptive-optical microscopy

Compensation of sample-induced optical aberrations is crucial for visualizing microscopic structures deep within biological tissues. However, strong multiple scattering poses a fundamental limitation for identifying and correcting the tissue-induced aberrations. Here, we introduce a label-free deep-tissue imaging technique termed dimensionality reduction adaptive-optical microscopy (DReAM) to selectively attenuate multiple scattering. We established a theoretical framework in which dimensionality reduction of a time-gated reflection matrix can attenuate uncorrelated multiple scattering while retaining a single-scattering signal with a strong wave correlation, irrespective of sample-induced aberrations. We performed mouse brain imaging in vivo through the intact skull with the probe beam at visible wavelengths. Despite the strong scattering and aberrations, DReAM offered a 17-fold enhancement of single scattering-to-multiple scattering ratio and provided high-contrast images of neural fibers in the brain cortex with the diffraction-limited spatial resolution of 412 nanometers and a 33-fold enhanced Strehl ratio.

Enhance the delivery of light energy ultra-deep into turbid medium by controlling multiple scattering photons to travel in open channels

Multiple light scattering is considered as the major limitation for deep imaging and focusing in turbid media. In this paper, we present an innovative method to overcome this limitation and enhance the delivery of light energy ultra-deep into turbid media with significant improvement in focusing. Our method is based on a wide-field reflection matrix optical coherence tomography (RM-OCT). The time-reversal decomposition of the RM is calibrated with the Tikhonov regularization parameter in order to get more accurate reversal results deep inside the scattering sample. We propose a concept named model energy matrix, which provides a direct mapping of light energy distribution inside the scattering sample. To the best of our knowledge, it is the first time that a method to measure and quantify the distribution of beam intensity inside a scattering sample is demonstrated. By employing the inversion of RM to find the matched wavefront and shaping with a phase-only spatial light modulator, we succeeded in both focusing a beam deep (~9.6 times of scattering mean free path, SMFP) inside the sample and increasing the delivery of light energy by an order of magnitude at an ultra-deep (~14.4 SMFP) position. This technique provides a powerful tool to understand the propagation of photon in a scattering medium and opens a new way to focus light inside biological tissues.

Calibration-free speckle matrix imaging

Unknown speckle patterns can be used to image targets embedded in complex scattering media 100 times faster than previous techniques based on carefully calibrated illuminations.

Multiple-scattering suppressive refractive index tomography for the label-free quantitative assessment of multicellular spheroids

Refractive index (RI) tomography is a quantitative tomographic technique used to visualize the intrinsic contrast of unlabeled biological samples. Conventional RI reconstruction algorithms are based on weak-scattering approximation, such as the Born or Rytov approximation. Although these linear algorithms are computationally efficient, they are invalid when the fields are strongly distorted by multiple scattering (MS) of specimens. Herein, we propose an approach to reconstruct the RI distributions of MS objects even under weak-scattering approximation using an MS-suppressive operation. The operation converts the distorted fields into MS-suppressed fields, where weak-scattering approximation is applicable. Using this approach, we reconstructed a whole multicellular spheroid and successfully visualized its internal subcellular structures. Our work facilitates the realization of RI tomography of MS specimens and label-free quantitative analysis of 3D multicellular specimens.

High-throughput volumetric adaptive optical imaging using compressed time-reversal matrix

Deep-tissue optical imaging suffers from the reduction of resolving power due to tissue-induced optical aberrations and multiple scattering noise. Reflection matrix approaches recording the maps of backscattered waves for all the possible orthogonal input channels have provided formidable solutions for removing severe aberrations and recovering the ideal diffraction-limited spatial resolution without relying on fluorescence labeling and guide stars. However, measuring the full input-output response of the tissue specimen is time-consuming, making the real-time image acquisition difficult. Here, we present the use of a time-reversal matrix, instead of the reflection matrix, for fast high-resolution volumetric imaging of a mouse brain. The time-reversal matrix reduces two-way problem to one-way problem, which effectively relieves the requirement for the coverage of input channels. Using a newly developed aberration correction algorithm designed for the time-reversal matrix, we demonstrated the correction of complex aberrations using as small as 2% of the complete basis while maintaining the image reconstruction fidelity comparable to the fully sampled reflection matrix. Due to nearly 100-fold reduction in the matrix recording time, we could achieve real-time aberration-correction imaging for a field of view of 40 × 40 µm2 (176 × 176 pixels) at a frame rate of 80 Hz. Furthermore, we demonstrated high-throughput volumetric adaptive optical imaging of a mouse brain by recording a volume of 128 × 128 × 125 µm3 (568 × 568 × 125 voxels) in 3.58 s, correcting tissue aberrations at each and every 1 µm depth section, and visualizing myelinated axons with a lateral resolution of 0.45 µm and an axial resolution of 2 µm.

Toward a clinical optoretinogram: a review of noninvasive, optical tests of retinal neural function

The past few years have witnessed rapid development of the optoretinogram-a noninvasive, optical measurement of neural function in the retina, and especially the photoreceptors (Ph). While its recent development has been rapid, it represents the culmination of hundreds of experiments spanning decades. Early work showed measurable and reproducible changes in the optical properties of retinal explants and suspensions of Ph, and uncovered some of the biophysical and biochemical mechanisms underlying them. That work thus provided critical motivation for more recent work based on clinical imaging platforms, whose eventual goal is the improvement of ophthalmic care and streamlining the discovery of novel therapeutics. The first part of this review consists of a selective summary of the early work, and identifies four kinds of stimulus-evoked optical signals that have emerged from it: changes in light scattered from the membranous discs of the Ph's outer segment (OS), changes in light scattered by the front and back boundaries of the OS, rearrangement of scattering material in and near the OS, and changes in the OS length. In the past decade, all four of these signals have continued to be investigated using imaging systems already used in the clinic or intended for clinical and translational use. The second part of this review discusses these imaging modalities, their potential to detect and quantify the signals of interest, and other factors influencing their translational promise. Particular attention is paid to phase-sensitive optical coherence tomography (OCT) with adaptive optics (AO), a method in which both the amplitude and the phase of light reflected from individual Ph is monitored as visible stimuli are delivered to them. The record of the light's phase is decoded to reveal a reproducible pattern of deformation in the OS, while the amplitude reveals changes in scattering and structural rearrangements. The method has been demonstrated in a few labs and has been used to measure responses from both rods and cones. With the ability to detect responses to stimuli isomerizing less than 0.01% of photopigment, this technique may prove to be a quick, noninvasive, and objective way to measure subtle disease-related dysfunction at the cellular level, and thus to provide an entirely new and complementary biomarker for retinal disease and recovery.

Manifestation of aberrations in full-field optical coherence tomography

We report on a theoretical model for image formation in full-field optical coherence tomography (FFOCT). Because the spatial incoherence of the illumination acts as a virtual confocal pinhole in FFOCT, its imaging performance is equivalent to a scanning time-gated coherent confocal microscope. In agreement with optical experiments enabling a precise control of aberrations, FFOCT is shown to have nearly twice the resolution of standard imaging at moderate aberration level. Beyond a rigorous study on the sensitivity of FFOCT with respect to aberrations, this theoretical model paves the way towards an optimized design of adaptive optics and computational tools for high-resolution and deep imaging of biological tissues.

Optical transfer function of time-gated coherent imaging in the presence of a scattering medium

Optical imaging of objects embedded within scattering media such as biological tissues suffers from the loss of resolving power. In our previous work, we proposed an approach called collective accumulation of single scattering (CASS) microscopy that attenuates this detrimental effect of multiple light scattering by combining the time-gated detection and spatial input-output correlation. In the present work, we perform a rigorous theoretical analysis on the effect of multiple light scattering to the optical transfer function of CASS microscopy. In particular, the spatial frequency-dependent signal to noise ratio (SNR) is derived depending on the intensity ratio of the single- and multiple-scattered waves. This allows us to determine the depth-dependent resolving power. We conducted experiments using a Siemens star-like target having various spatial frequency components and supported the theoretical derived SNR spectra. Our study provides a theoretical framework for understanding the effect of multiple light scattering in high-resolution and deep-tissue optical imaging.

Entropy Indicators: An Approach for Low-Speed Bearing Diagnosis

To increase the competitiveness of wind energy, the maintenance costs of offshore floating and fixed wind turbines need to be reduced. One strategy is the enhancement of the condition monitoring techniques for pitch bearings, because their low operational speed and the high loads applied to them make their monitoring challenging. Vibration analysis has been widely used for monitoring the bearing condition with good results obtained for regular bearings, but with difficulties when the operational speed decreases. Therefore, new techniques are required to enhance the capabilities of vibration analysis for bearings under such operational conditions. This study proposes the use of indicators based on entropy for monitoring a low-speed bearing condition. The indicators used are approximate, dispersion, singular value decomposition, and spectral entropy of the permutation entropy. This approach has been tested with vibration signals acquired in a test rig with bearings under different health conditions. The results show that entropy indicators (EIs) can discriminate with higher-accuracy damaged bearings for low-speed bearings compared with the regular indicators. Furthermore, it is shown that the combination of regular and entropy-based indicators can also contribute to a more reliable diagnosis.

Wavefront Shaping Concepts for Application in Optical Coherence Tomography-A Review

Optical coherence tomography (OCT) enables three-dimensional imaging with resolution on the micrometer scale. The technique relies on the time-of-flight gated detection of light scattered from a sample and has received enormous interest in applications as versatile as non-destructive testing, metrology and non-invasive medical diagnostics. However, in strongly scattering media such as biological tissue, the penetration depth and imaging resolution are limited. Combining OCT imaging with wavefront shaping approaches significantly leverages the capabilities of the technique by controlling the scattered light field through manipulation of the field incident on the sample. This article reviews the main concepts developed so far in the field and discusses the latest results achieved with a focus on signal enhancement and imaging.

Laser scanning reflection-matrix microscopy for aberration-free imaging through intact mouse skull

A mouse skull is a barrier for high-resolution optical imaging because its thick and inhomogeneous internal structures induce complex aberrations varying drastically from position to position. Invasive procedures creating either thinned-skull or open-skull windows are often required for the microscopic imaging of brain tissues underneath. Here, we propose a label-free imaging modality termed laser scanning reflection-matrix microscopy for recording the amplitude and phase maps of reflected waves at non-confocal points as well as confocal points. The proposed method enables us to find and computationally correct up to 10,000 angular modes of aberrations varying at every 10 × 10 µm² patch in the sample plane. We realized reflectance imaging of myelinated axons in vivo underneath an intact mouse skull, with an ideal diffraction-limited spatial resolution of 450 nm. Furthermore, we demonstrated through-skull two-photon fluorescence imaging of neuronal dendrites and their spines by physically correcting the aberrations identified from the reflection matrix.