Compressive imaging through a multimode fiber

High-fidelity and high-speed wavefront shaping by leveraging complex media

High-precision light manipulation is crucial for delivering information through complex media. However, existing spatial light modulation devices face a fundamental speed-fidelity tradeoff. Digital micromirror devices have emerged as a promising candidate for high-speed wavefront shaping but at the cost of compromised fidelity due to the limited control degrees of freedom. Here, we leverage the sparse-to-random transformation through complex media to overcome the dimensionality limitation of spatial light modulation devices. We demonstrate that pattern compression by sparsity-constrained wavefront optimization allows sparse and robust wavefront representations in complex media, improving the projection fidelity without sacrificing frame rate, hardware complexity, or optimization time. Our method is generalizable to different pattern types and complex media, supporting consistent performance with up to 89% and 126% improvements in projection accuracy and speckle suppression, respectively. The proposed optimization framework could enable high-fidelity high-speed wavefront shaping through different scattering media and platforms without changes to the existing holographic setups, facilitating a wide range of physics and real-world applications.

Recovery of angular scattering profiles through a flexible multimode fiber

Endoscopic angle-resolved light scattering methods have been developed for early cancer detection but they typically require multi-element coherent fiber optic bundles to recover scattering distributions from tissues. Recent work has focused on using a single multimode fiber (MMF) to measure angle resolved scattering but this approach has practical limitations to overcome before clinical translation. Here we address these limitations by proposing an MMF-based endoscope capable of measuring angular scattering patterns suitable for determining structure. Significantly, this approach implements a spectrally resolved detection scheme to reduce speckle and leverages the azimuthal symmetry of the angular scattering patterns to enable measurements that are robust to fiber bending. This results in a unique method that does not require matrix inversion or machine learning to measure a transmitted scattering distribution. The MMF utilized here is 1000 mm in length with a 200 µm core and is demonstrated to recover angular scattering distributions even with bending displacements of up to 30 cm. This advance has a significant impact on the clinical translation of biomedical endoscopic diagnostic techniques that use angular scattering to determine the size of cell nuclei to detect early cancer.

Exploiting holographically encoded variance to transmit labelled images through a multimode optical fiber

Artificial intelligence has emerged as promising tool to decode an image transmitted through a multimode fiber (MMF) by applying deep learning techniques. By transmitting thousands of images through the MMF, deep neural networks (DNNs) are able to decipher the seemingly random output speckle patterns and unveil the intrinsic input-output relationship. High fidelity reconstruction is obtained for datasets with a large degree of homogeneity, which underutilizes the capacity of the combined MMF-DNN system. Here, we show that holographic modulation can encode an additional layer of variance on the output speckle pattern, improving the overall transmissive capabilities of the system. Operatively, we have implemented this by adding a holographic label to the original dataset and injecting the resulting phase image into the fiber facet through a Fourier transform lens. The resulting speckle pattern dataset can be clustered primarily by holographic label, and can be reconstructed without loss of fidelity. As an application, we describe how color images may be segmented into RGB components and each color component may then be labelled by distinct hologram. A ResUNet architecture was then used to decode each class of speckle patterns and reconstruct the color image without the need for temporal synchronization between sender and receiver.

Optical fiber bundle differential compressive imaging

We present a differential compressive imaging method for an optical fiber bundle (OFB), which provides a solution for an ultrathin bend-resistant endoscope with high resolution. This method uses an OFB and a diffuser to generate speckle illumination patterns. Differential operation is additionally applied to the speckle patterns to produce sensing matrices, by which the correlation between the matrices is greatly reduced from 0.875 to 0.0275, which ensures the high quality of image reconstruction. Pixilation artifacts from the fiber core arrangement are also effectively eliminated with this configuration. We demonstrate high-resolution reconstruction of images of 132 × 132 pixels with a compression rate of 12% using 77 fiber cores, the total diameter of which is only about 91 µm. An experimental verification proves that this method is tolerant to a limited degree of fiber bending, which provides a potential approach for robust high-resolution fiber endoscopy.

Lightweight super-resolution multimode fiber imaging with regularized linear regression

Super-resolution multimode fiber imaging provides the means to image samples quickly with compact and flexible setups finding many applications from biology and medicine to material science and nanolithography. Typically, fiber-based imaging systems suffer from low spatial resolution and long measurement times. State-of-the-art computational approaches can achieve fast super-resolution imaging through a multimode fiber probe but currently rely on either per-sample optimised priors or large data sets with subsequent long training and image reconstruction times. This unfortunately hinders any real-time imaging applications. Here we present an ultimately fast non-iterative algorithm for compressive image reconstruction through a multimode fiber. The proposed approach helps to avoid many constraints by determining the prior of the target distribution from a simulated set and solving the under-determined inverse matrix problem with a mathematical closed-form solution. We have demonstrated theoretical and experimental evidence for enhanced image quality and sub-diffraction spatial resolution of the multimode fiber optical system.

Imaging through a square multimode fiber by scanning focused spots with the memory effect

The existence of a shift-shift memory effect in square waveguides, whereby any translation of the input field induces translations in the output field in four symmetrical directions, has been previously observed by correlation measurements. Here we demonstrate that this memory effect is also observed in real space and can be put to use for imaging purposes. First, a focus is created at the output of a square-core multimode fiber, by wavefront shaping based on feedback from a guide-star. Then, because of the memory effect, four symmetrical spots can be scanned at the fiber output by shifting the wavefront at the fiber input. We demonstrate that this property can be exploited to perform fluorescence imaging through the multimode fiber, without requiring the measurement of a transmission matrix.

Robust real-time imaging through flexible multimode fibers

Conventional endoscopes comprise a bundle of optical fibers, associating one fiber for each pixel in the image. In principle, this can be reduced to a single multimode optical fiber (MMF), the width of a human hair, with one fiber spatial-mode per image pixel. However, images transmitted through a MMF emerge as unrecognizable speckle patterns due to dispersion and coupling between the spatial modes of the fiber. Furthermore, speckle patterns change as the fiber undergoes bending, making the use of MMFs in flexible imaging applications even more complicated. In this paper, we propose a real-time imaging system using flexible MMFs, but which is robust to bending. Our approach does not require access or feedback signal from the distal end of the fiber during imaging. We leverage a variational autoencoder to reconstruct and classify images from the speckles and show that these images can still be recovered when the bend configuration of the fiber is changed to one that was not part of the training set. We utilize a MMF 300 mm long with a 62.5 μm core for imaging [Formula: see text] cm objects placed approximately at 20 cm from the fiber and the system can deal with a change in fiber bend of 50[Formula: see text] and range of movement of 8 cm.

Super-resolution multimode fiber imaging with an untrained neural network

Multimode fiber endoscopes provide extreme miniaturization of imaging components for minimally invasive deep tissue imaging. Typically, such fiber systems suffer from low spatial resolution and long measurement time. Fast super-resolution imaging through a multimode fiber has been achieved by using computational optimization algorithms with hand-picked priors. However, machine learning reconstruction approaches offer the promise of better priors, but require large training datasets and therefore long and unpractical pre-calibration time. Here we report a method of multimode fiber imaging based on unsupervised learning with untrained neural networks. The proposed approach solves the ill-posed inverse problem by not relying on any pre-training process. We have demonstrated both theoretically and experimentally that untrained neural networks enhance the imaging quality and provide sub-diffraction spatial resolution of the multimode fiber imaging system.

Block-based compressed sensing for fast optic fiber bundle imaging with high spatial resolution

The resolution of traditional fiber bundle imaging is usually limited by the density and the diameter of the fiber cores. To improve the resolution, compression sensing was introduced to resolve multiple pixels from a single fiber core, but current methods have the drawbacks of excessive sampling and long reconstruction time. In this paper, we present, what we believe to be, a novel block-based compressed sensing scheme for fast realization of high-resolution optic fiber bundle imaging. In this method, the target image is segmented into multiple small blocks, each of which covers the projection area of one fiber core. All block images are independently and simultaneously sampled and the intensities are recorded by a two-dimensional detector after they are collected and transmitted through corresponding fiber cores. Because the size of sampling patterns and the sampling numbers are greatly reduced, the reconstruction complexity and reconstruction time are also decreased. According to the simulation analysis, our method is 23 times faster than the current compressed sensing optical fiber imaging for reconstructing a fiber image of 128 × 128 pixels, while the sampling number is only 0.39%. Experiment results demonstrate that the method is also effective for reconstructing large target images and the number of sampling does not increase with the size of the image. Our finding may provide a new idea for high-resolution real-time imaging of fiber bundle endoscope.

Swept-source multimode fiber imaging

High-resolution compressive imaging via a flexible multimode fiber is demonstrated using a swept-laser source and wavelength dependent speckle illumination. An in-house built swept-source allowing for independent control of bandwidth and scanning range is used to explore and demonstrate a mechanically scan-free approach for high-resolution imaging through an ultrathin and flexible fiber probe. The computational image reconstruction is shown by utilizing a narrow sweeping bandwidth of [Formula: see text] nm while acquisition time is decreased by 95% compared to conventional raster scanning endoscopy. Demonstrated narrow-band illumination in the visible spectrum is vital for the detection of fluorescence biomarkers in neuroimaging applications. The proposed approach yields device simplicity and flexibility for minimally invasive endoscopy.

Sub-diffraction computational imaging via a flexible multicore-multimode fiber

An ultra-thin multimode fiber is an ideal platform for minimally invasive microscopy with the advantages of a high density of modes, high spatial resolution, and a compact size. In practical applications, the probe needs to be long and flexible, which unfortunately destroys the imaging capabilities of a multimode fiber. In this work, we propose and experimentally demonstrate sub-diffraction imaging through a flexible probe based on a unique multicore-multimode fiber. A multicore part consists of 120 Fermat's spiral distributed single-mode cores. Each of the cores offers stable light delivery to the multimode part, which provides optimal structured light illumination for sub-diffraction imaging. As a result, perturbation-resilient fast sub-diffraction fiber imaging by computational compressive sensing is demonstrated.

Edge detection in single multimode fiber imaging based on deep learning

We propose a new edge detection scheme based on deep learning in single multimode fiber imaging. In this scheme, we creatively design a novel neural network, whose input is a one-dimensional light intensity sequence, and the output is the edge detection result of the target. Different from the traditional scheme, we can directly obtain the edge information of unknown objects by using this neural network without rebuilding the image. Simulation and experimental results show that, compared with the traditional method, this method can get better edge details, especially in the case of low sampling rates. It can increase the structural similarity index of edge detection imaging from 0.38 to 0.62 at the sampling rate of 0.6%. At the same time, the robustness of the method to fiber bending is also proved. This scheme improves the edge detection performance of endoscopic images and provides a promising way for the practical application of multimode fiber endoscopy.

High-speed label-free multimode-fiber-based compressive imaging beyond the diffraction limit

Glass fibers are miniature optical components that serve as ultra-narrow endoscopy probes. Ideally, one would want to perform imaging through a fiber at the highest achievable resolution and speed. State-of-the-art super-resolution techniques have shattered the diffraction limit, but more than twofold improvement requires fluorescent labeling and a long acquisition time. Moreover, it is challenging to implement super-resolution microscopy in a fiber format. Here we present fiber-based label-free video-rate imaging at more than 2-fold higher resolution than the diffraction limit. Our work paves the way to rapid, sub-wavelength endo-microscopy in unlabeled live specimens.

Confocal 3D reflectance imaging through multimode fiber without wavefront shaping

Imaging through optical multimode fiber (MMF) has the potential to enable hair-thin endoscopes that reduce the invasiveness of imaging deep inside tissues and organs. Active wavefront shaping and fluorescent labeling have recently been exploited to overcome modal scrambling and enable MMF imaging. Here, we present a computational approach that circumvents the need for active wavefront control and exogenous fluorophores. We demonstrate the reconstruction of depth-gated confocal images through MMF using a raster-scanned, focused input illumination at the fiber proximal end. We show the compatibility of this approach with quantitative phase, dark-field, and polarimetric imaging. Computational imaging through MMF opens a new pathway for minimally invasive imaging in medical diagnosis and biological investigations.

Ultrafast heterodyne mode imaging and refractive index mapping of a femtosecond laser written multimode waveguide

We demonstrate imaging of individual modes in a femtosecond laser written multimode waveguide by spatial-heterodyne interferometry and decomposition in data post-processing. Despite the spatial and temporal overlap between multiple waveguide modes, we show the extraction of amplitude for each individual mode and their corresponding temporal dynamics. The mode imaging scheme is effective with the presence of intermodal interference and can be prospective for sensing of ultrafast phase and refractive index fluctuations. We also reconstruct the two-dimensional transverse refractive index map of the multimode waveguide leveraging all the imaged modes and substantiate the reconstructed index map by simulation.

Compressive endoscopic imaging with complementary light modulation

We propose an effective endoscopic imaging method utilizing compressive sensing (CS) theory on the basis of complementary light modulation of a spatial light modulator. Both the simulated and the experimental results show that complementary compressive sensing (CCS) always needs less time to obtain better work than conventional CS with normal modulation at the same sampling rate. First, the speed of CCS is at least twice as fast as CS. Second, in comparison with CS, CCS can improve the signal-to-noise ratio of the reconstructed image by 49.7%, which indicates that this method is of great significance to endoscopic applications in terms of image fidelity and denoising performance.

Memory effect assisted imaging through multimode optical fibres

When light propagates through opaque material, the spatial information it holds becomes scrambled, but not necessarily lost. Two classes of techniques have emerged to recover this information: methods relying on optical memory effects, and transmission matrix (TM) approaches. Here we develop a general framework describing the nature of memory effects in structures of arbitrary geometry. We show how this framework, when combined with wavefront shaping driven by feedback from a guide-star, enables estimation of the TM of any such system. This highlights that guide-star assisted imaging is possible regardless of the type of memory effect a scatterer exhibits. We apply this concept to multimode fibres (MMFs) and identify a 'quasi-radial' memory effect. This allows the TM of an MMF to be approximated from only one end - an important step for micro-endoscopy. Our work broadens the applications of memory effects to a range of novel imaging and optical communication scenarios.

Compressively sampling the optical transmission matrix of a multimode fibre

The measurement of the optical transmission matrix (TM) of an opaque material is an advanced form of space-variant aberration correction. Beyond imaging, TM-based methods are emerging in a range of fields, including optical communications, micro-manipulation, and computing. In many cases, the TM is very sensitive to perturbations in the configuration of the scattering medium it represents. Therefore, applications often require an up-to-the-minute characterisation of the fragile TM, typically entailing hundreds to thousands of probe measurements. Here, we explore how these measurement requirements can be relaxed using the framework of compressive sensing, in which the incorporation of prior information enables accurate estimation from fewer measurements than the dimensionality of the TM we aim to reconstruct. Examples of such priors include knowledge of a memory effect linking the input and output fields, an approximate model of the optical system, or a recent but degraded TM measurement. We demonstrate this concept by reconstructing the full-size TM of a multimode fibre supporting 754 modes at compression ratios down to ∼5% with good fidelity. We show that in this case, imaging is still possible using TMs reconstructed at compression ratios down to ∼1% (eight probe measurements). This compressive TM sampling strategy is quite general and may be applied to a variety of other scattering samples, including diffusers, thin layers of tissue, fibre optics of any refractive profile, and reflections from opaque walls. These approaches offer a route towards the measurement of high-dimensional TMs either quickly or with access to limited numbers of measurements.

Comparison of round- and square-core fibers for sensing, imaging, and spectroscopy

Multimode fibers (MMFs) show great promise as miniature probes for sensing, imaging, and spectroscopy applications. Different parameters of the fibers, such as numerical aperture, refractive index profile and length, have been already optimized for better performance. Here we investigate the role of the core shape, in particular for wavefront shaping applications where a focus is formed at the output of the MMF. We demonstrate that in contrast to a conventional round-core MMF, a square-core design does not suffer from focus aberrations. Moreover, we find that how the interference pattern behind a square-core fiber decorrelates with the input frequency is largely independent of the input light coupling. Finally, we demonstrate that a square core shape provides an on-average uniform distribution of the output intensity, free from the input-output correlations seen in round fibers, showing great promise for imaging and spectroscopy applications.

Ultimate resolution limits of speckle-based compressive imaging

Compressive imaging using sparsity constraints is a very promising field of microscopy that provides a dramatic enhancement of the spatial resolution beyond the Abbe diffraction limit. Moreover, it simultaneously overcomes the Nyquist limit by reconstructing an N-pixel image from less than N single-point measurements. Here we present fundamental resolution limits of noiseless compressive imaging via sparsity constraints, speckle illumination and single-pixel detection. We addressed the experimental setup that uses randomly generated speckle patterns (in a scattering media or a multimode fiber). The optimal number of measurements, the ultimate spatial resolution limit and the surprisingly important role of discretization are demonstrated by the theoretical analysis and numerical simulations. We show that, in contrast to conventional microscopy, oversampling may decrease the resolution and reconstruction quality of compressive imaging.

A Pearl Spectrometer

Information recovery from incomplete measurements, typically performed by a numerical means, is beneficial in a variety of classical and quantum signal processing. Random and sparse sampling with nanophotonic and light scattering approaches has received attention to overcome the hardware limitations of conventional spectrometers and hyperspectral imagers but requires high-precision nanofabrications and bulky media. We report a simple spectral information processing scheme in which light transport through an Anderson-localized medium serves as an entropy source for compressive sampling directly in the frequency domain. As implied by the "lustrous" reflection originating from the exquisite multilayered nanostructures, a pearl (or mother-of-pearl) allows us to exploit the spatial and spectral intensity fluctuations originating from strong light localization for extracting salient spectral information with a compact and thin form factor. Pearl-inspired light localization in low-dimensional structures can offer an alternative of spectral information processing by hybridizing digital and physical properties at a material level.

Computational optical imaging with a photonic lantern

The thin and flexible nature of optical fibres often makes them the ideal technology to view biological processes in-vivo, but current microendoscopic approaches are limited in spatial resolution. Here, we demonstrate a route to high resolution microendoscopy using a multicore fibre (MCF) with an adiabatic multimode-to-single-mode “photonic lantern” transition formed at the distal end by tapering. We show that distinct multimode patterns of light can be projected from the output of the lantern by individually exciting the single-mode MCF cores, and that these patterns are highly stable to fibre movement. This capability is then exploited to demonstrate a form of single-pixel imaging, where a single pixel detector is used to detect the fraction of light transmitted through the object for each multimode pattern. A custom computational imaging algorithm we call SARA-COIL is used to reconstruct the object using only the pre-measured multimode patterns themselves and the detector signals.

Here, the authors demonstrate a route to high resolution microendoscopy using a multicore fibre with a photonic lantern. They show that distinct multimode patterns of light can be projected from the output of the lantern by individually exciting the single-mode MCF cores, whose patterns are highly stable to fibre movement.

Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination

We demonstrate reconstruction of angle-resolved optical backscattering after transmission through a multimode fiber. Angle-resolved backscattering is an important tool for particle sizing, and has been developed as a diagnostic modality for detecting epithelial precancer. In this work, we fully characterized the transfer function of a multimode fiber using a plane-wave illumination basis across two dimensions. Once characterized, angle-resolved scattering information which has been scrambled by multimodal propagation can be easily and accurately reconstructed. Our technique was validated using a Mie theory-based inverse light scattering analysis (ILSA) algorithm on polystyrene microsphere phantoms of known sizes. To demonstrate the clinical potential of this approach, nuclear morphology was determined from the reconstructed angular backscattering from MCF-10A human mammary epithelial cell samples and validated against quantitative image analysis (QIA) of fluorescence microscopy images.

Endo-microscopy beyond the Abbe and Nyquist limits

For several centuries, far-field optical microscopy has remained a key instrument in many scientific disciplines, including physical, chemical, and biomedical research. Nonetheless, far-field imaging has many limitations: the spatial resolution is controlled by the diffraction of light, and the imaging speed follows the Nyquist-Shannon sampling theorem. The recent development of super-resolution techniques has pushed the limits of spatial resolution. However, these methods typically require complicated setups and long acquisition times and are still not applicable to deep-tissue bioimaging. Here, we report imaging through an ultra-thin fibre probe with a spatial resolution beyond the Abbe limit and a temporal resolution beyond the Nyquist limit simultaneously in a simple and compact setup. We use the random nature of mode coupling in a multimode fibre, the sparsity constraint and compressive sensing reconstruction. The new approach of super-resolution endo-microscopy does not use any specific properties of the fluorescent label, such as depletion or stochastic activation of the molecular fluorescent state, and therefore can be used for label-free imaging. We demonstrate a spatial resolution more than 2 times better than the diffraction limit and an imaging speed 20 times faster than the Nyquist limit. The proposed approach can significantly expand the realm of the application of nanoscopy for bioimaging.

Averaging speckle patterns to improve the robustness of compressive multimode fiber imaging against fiber bend

Fiber bend is a major challenge of multimode fiber (MMF) imaging. More robustness against fiber bend is demonstrated in compressive MMF imaging using mean speckle patterns captured at multiple potential bending configurations beforehand, rather than sticking to single patterns at initial configuration. Experiments demonstrate an overall quality improvement on recovered images than previous work, which is important for robust endoscopic application.

Quantum key establishment via a multimode fiber

Quantum communication aims to provide absolutely secure transmission of secret information. State-of-the-art methods encode symbols into single photons or coherent light with much less than one photon on average. For long-distance communication, typically a single-mode fiber is used and significant effort has been devoted already to increase the data carrying capacity of a single optical line. Here we propose and demonstrate a fundamentally new concept for remote key establishment. Our method allows high-dimensional alphabets using spatial degrees of freedom by transmitting information through a light-scrambling multimode fiber and exploiting the no-cloning theorem. Eavesdropper attacks can be detected without using randomly switched mutually unbiased bases. We prove the security against a common class of intercept-resend and beam-splitting attacks with single-photon Fock states and with weak coherent light. Since it is optical fiber based, our method allows to naturally extend secure communication to larger distances. We experimentally demonstrate this new type of key exchange method by encoding information into a few-photon light pulse decomposed over guided modes of an easily available multimode fiber.

Robust compressive multimode fiber imaging against bending with enhanced depth of field

Imaging through single multimode fiber prevails its counterpart using single mode fiber bundle on spatial resolution limit and minimum radius. Current multimode fiber imaging suffers from fussy calibration, which can be reduced by recent developed compressive sensing scheme [ [L. V. Amitonova, Opt. Lett. 43, 5427 (2018)]. Experiments demonstrate improvement on depth of field by more than three orders of magnitude, together with robustness against macro fiber bending, which is vital to endoscopic applications.