Ultrathin endoscopes based on multicore fibers and adaptive optics: a status review and perspectives

Neurophotonics beyond the surface: unmasking the brain's complexity exploiting optical scattering

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Improved resolution in fiber bundle inline holographic microscopy using multiple illumination sources

Recent work has shown that high-quality inline holographic microscopy images can be captured through fiber imaging bundles. Speckle patterns arising from modal interference within the bundle cores can be minimized by use of a partially-coherent optical source such as an LED delivered via a multimode fiber. This allows numerical refocusing of holograms from samples at working distances of up to approximately 1 mm from the fiber bundle before the finite coherence begins to degrade the lateral resolution. However, at short working distances the lateral resolution is limited not by coherence, but by sampling effects due to core-to-core spacing in the bundle. In this article we demonstrate that multiple shifted holograms can be combined to improve the resolution by a factor of two. The shifted holograms can be rapidly acquired by sequentially firing LEDs, which are each coupled to their own, mutually offset, illumination fiber. Following a one-time calibration, resolution-enhanced images are created in real-time at an equivalent net frame rate of up to 7.5 Hz. The resolution improvement is demonstrated quantitatively using a resolution target and qualitatively using mounted biological slides. At longer working distances, beyond 0.6 mm, the improvement is reduced as resolution becomes limited by the source spatial and temporal coherence.

Real-timing processing of fiber bundle endomicroscopy images in Python using PyFibreBundle

Fiber imaging bundles allow the transfer of optical images from place-to-place along narrow and flexible conduits. Traditionally used extensively in medical endoscopy, bundles are now finding new applications in endoscopic microscopy and other emerging techniques. PyFibreBundle is an open-source Python package for fast processing of images acquired through imaging bundles. This includes detection and removal of the fiber core pattern by filtering or interpolation, and application of background and flat-field corrections. It also allows images to be stitched together to create mosaics and resolution to be improved by combining multiple shifted images. This paper describes the technical implementation of PyFibreBundle and provides example results from three endomicroscopy imaging systems: color transmission, monochrome transmission, and confocal fluorescence. This allows various processing options to be compared quantitatively and qualitatively, and benchmarking demonstrates that PyFibreBundle can achieve state-of-the-art performance in an open-source package. The paper demonstrates core removal by interpolation and mosaicing at over 100 fps, real-time multi-frame resolution enhancement and the first demonstration of real-time endomicroscopy image processing, including core removal, on a Raspberry Pi single board computer. This demonstrates that PyFibreBundle is potentially a valuable tool for the development of low-cost, high-performance fiber bundle imaging systems.

Imaging through a square multimode fiber by scanning focused spots with the memory effect

The existence of a shift-shift memory effect in square waveguides, whereby any translation of the input field induces translations in the output field in four symmetrical directions, has been previously observed by correlation measurements. Here we demonstrate that this memory effect is also observed in real space and can be put to use for imaging purposes. First, a focus is created at the output of a square-core multimode fiber, by wavefront shaping based on feedback from a guide-star. Then, because of the memory effect, four symmetrical spots can be scanned at the fiber output by shifting the wavefront at the fiber input. We demonstrate that this property can be exploited to perform fluorescence imaging through the multimode fiber, without requiring the measurement of a transmission matrix.

Speckle-correlation imaging through a kaleidoscopic multimode fiber

Speckle-correlation imaging techniques are widely used for noninvasive imaging through complex scattering media. While light propagation through multimode fibers and scattering media share many analogies, reconstructing images through multimode fibers from speckle correlations remains an unsolved challenge. Here, we exploit a kaleidoscopic memory effect emerging in square-core multimode fibers and demonstrate fluorescence imaging with no prior knowledge on the fiber. Experimentally, our approach simply requires to translate random speckle patterns at the input of a square-core fiber and to measure the resulting fluorescence intensity with a bucket detector. The image of the fluorescent object is then reconstructed from the autocorrelation of the measured signal by solving an inverse problem. This strategy does not require the knowledge of the fragile deterministic relation between input and output fields, which makes it promising for the development of flexible minimally invasive endoscopes.

Coupling optimized bending-insensitive multi-core fibers for lensless endoscopy

We report a bending-insensitive multi-core fiber (MCF) for lensless endoscopy imaging with modified fiber geometry that enables optimal light coupling in and out of the individual cores. In a previously reported bending insensitive MCF (twisted MCF), the cores are twisted along the length of the MCF allowing for the development of flexible thin imaging endoscopes with potential applications in dynamic and freely moving experiments. However, for such twisted MCFs the cores are seen to have an optimum coupling angle which is proportional to their radial distance from the center of the MCF. This brings coupling complexity and potentially degrades the endoscope imaging capabilities. In this study, we demonstrate that by introducing a small section (1 cm) at two ends of the MCF, where all the cores are straight and parallel to the optical axis one can rectify the above coupling and output light issues of the twisted MCF, enabling the development of bend-insensitive lensless endoscopes.

Sub-diffraction computational imaging via a flexible multicore-multimode fiber

An ultra-thin multimode fiber is an ideal platform for minimally invasive microscopy with the advantages of a high density of modes, high spatial resolution, and a compact size. In practical applications, the probe needs to be long and flexible, which unfortunately destroys the imaging capabilities of a multimode fiber. In this work, we propose and experimentally demonstrate sub-diffraction imaging through a flexible probe based on a unique multicore-multimode fiber. A multicore part consists of 120 Fermat's spiral distributed single-mode cores. Each of the cores offers stable light delivery to the multimode part, which provides optimal structured light illumination for sub-diffraction imaging. As a result, perturbation-resilient fast sub-diffraction fiber imaging by computational compressive sensing is demonstrated.

Overcoming the field-of-view to diameter trade-off in microendoscopy via computational optrode-array microscopy

High-resolution microscopy of deep tissue with large field-of-view (FOV) is critical for elucidating organization of cellular structures in plant biology. Microscopy with an implanted probe offers an effective solution. However, there exists a fundamental trade-off between the FOV and probe diameter arising from aberrations inherent in conventional imaging optics (typically, FOV < 30% of diameter). Here, we demonstrate the use of microfabricated non-imaging probes (optrodes) that when combined with a trained machine-learning algorithm is able to achieve FOV of 1x to 5x the probe diameter. Further increase in FOV is achieved by using multiple optrodes in parallel. With a 1 × 2 optrode array, we demonstrate imaging of fluorescent beads (including 30 FPS video), stained plant stem sections and stained living stems. Our demonstration lays the foundation for fast, high-resolution microscopy with large FOV in deep tissue via microfabricated non-imaging probes and advanced machine learning.

Video-rate dual-modal photoacoustic and fluorescence imaging through a multimode fibre towards forward-viewing endomicroscopy

Multimode fibres (MMFs) are becoming increasingly attractive in optical endoscopy as they promise to enable unparallelled miniaturisation, spatial resolution and cost. However, high-speed imaging with wavefront shaping has been challenging. Here, we report the development of a video-rate dual-modal photoacoustic (PA) and fluorescence microscopy probe with a high-speed digital micromirror device (DMD) towards forward-viewing endomicroscopy. Optimal DMD patterns were obtained using a real-valued intensity transmission matrix algorithm to raster-scan a 1.5  μ m-diameter focused beam at the distal fibre tip for imaging. The PA imaging speed and spatial resolution were varied from  ∼ 2 to 57 frames per second and from 1.7 to 3  μ m, respectively. Further, high-fidelity PA images of carbon fibres and mouse red blood cells were acquired at unprecedented speed. The capability of dual-modal imaging was demonstrated with phantoms. We anticipate that with further miniaturisation of the ultrasound detector, this probe could be integrated into medical needles to guide minimally invasive procedures.

Miniature 120-beam coherent combiner with 3D-printed optics for multicore fiber-based endoscopy

In this Letter, we report a high-efficiency, miniaturized, ultra-fast coherent beam, combined with 3D-printed micro-optics directly on the tip of a multicore fiber bundle. The highly compact device footprint (180 µm in diameter) facilitates its incorporation into a minimally invasive ultra-thin nonlinear endoscope to perform two-photon imaging.

Optical memory effect in square multimode fibers

We demonstrate experimentally the existence of a translational optical memory effect in square-core multimode fibers. We found that symmetry properties of square-core waveguides lead to speckle patterns shifting along four directions at the fiber output for any given shift direction at the input. A simple theoretical model based on a perfectly reflective square waveguide is introduced to predict and interpret this phenomenon. We report experimental results obtained with 532-nm coherent light propagating through a square-core step-index multimode fiber, demonstrating that this translational memory effect can be observed for shift distances up to typically 10 µm after propagation through several centimeters of fiber.

BPM-Matlab: an open-source optical propagation simulation tool in MATLAB

We present the use of the Douglas-Gunn Alternating Direction Implicit finite difference method for computationally efficient simulation of the electric field propagation through a wide variety of optical fiber geometries. The method can accommodate refractive index profiles of arbitrary shape and is implemented in a tool called BPM-Matlab. We validate BPM-Matlab by comparing it to published experimental, numerical, and theoretical data and to commercially available state-of-the-art software. It is user-friendly, fast, and is available open-source. BPM-Matlab has a broad scope of applications in modeling a variety of optical fibers for diverse fields such as imaging, communication, material processing, and remote sensing.

Inline holographic microscopy through fiber imaging bundles

Fiber imaging bundles are widely used as thin, passive image conduits for miniaturized and endoscopic microscopy, particularly for confocal fluorescence imaging. Holographic microscopy through fiber bundles is more challenging; phase conjugation approaches are complex and require extensive calibration. This paper describes how simple inline holographic microscopy can be performed through an imaging bundle using a partially coherent illumination source from a multimode fiber. The sample is imaged in transmission, with the intensity hologram sampled by the bundle and transmitted to a remote camera. The hologram can then be numerically refocused for volumetric imaging, achieving a resolution of approximately 6 µm over a depth range of 1 mm. The scheme does not require any complex prior calibration and hence is insensitive to bending.

Seeing through multimode fibers with real-valued intensity transmission matrices

Image transmission through multimode optical fibers has been an area of immense interests driven by the demand for miniature endoscopes in biomedicine and higher speed and capacity in telecommunications. Conventionally, a complex-valued transmission matrix is obtained experimentally to link the input and output light fields of a multimode fiber for image retrieval, which complicates the experimental setup and increases the computational complexity. Here, we report a simple and high-speed method for image retrieval based on our demonstration of a pseudo-linearity between the input and output light intensity distributions of multimode fibers. We studied the impact of several key parameters to image retrieval, including image pixel count, fiber core diameter and numerical aperture. We further demonstrated with experiments and numerical simulations that a wide variety of input binary and gray scale images could be faithfully retrieved from the corresponding output speckle patterns. Thus, it promises to be useful for highly miniaturized endoscopy in biomedicine and spatial-mode-division multiplexing in telecommunications.

Ultrathin glass fiber microprobe for electroporation of arbitrary selected cell groups

A new type of ultrathin fiber microprobe for selective electroporation is reported. The microprobe is 10 cm long and has a diameter of 350 µm. This microprobe is a low cost tool, which allows electroporation of an arbitrary selected single cell or groups of cells among population with use of a standard microscope and cell culture plates. The microprobe in its basic form contains two metal microelectrodes made of a silver-copper alloy, running along the fiber, each with a diameter of 23 µm. The probe was tested in vitro on a population of normal and cancer cells. Successful targeted electroporation was observed by means of accumulation of trypan blue (TB) dye marker in the cell. The electroporation phenomenon was also verified with propidium iodide and AnnexinV in fluorescent microscopy.

A minimally invasive lens-free computational microendoscope

Ultra-miniaturized microendoscopes are vital for numerous biomedical applications. Such minimally invasive imagers allow for navigation into hard-to-reach regions and observation of deep brain activity in freely moving animals. Conventional solutions use distal microlenses. However, as lenses become smaller and less invasive, they develop greater aberrations and restricted fields of view. In addition, most of the imagers capable of variable focusing require mechanical actuation of the lens, increasing the distal complexity and weight. Here, we demonstrate a distal lens-free approach to microendoscopy enabled by computational image recovery. Our approach is entirely actuation free and uses a single pseudorandom spatial mask at the distal end of a multicore fiber. Experimentally, this lensless approach increases the space-bandwidth product, i.e., field of view divided by resolution, by threefold over a best-case lens-based system. In addition, the microendoscope demonstrates color resolved imaging and refocusing to 11 distinct depth planes from a single camera frame without any actuated parts.

Nonlinear imaging through a Fermat's golden spiral multicore fiber

We report two-photon lensless imaging through a novel Fermat's golden spiral multicore fiber. This unique layout optimizes the sidelobe levels, field of view, crosstalk, group delay, and mode density to achieve a sidelobe contrast of at least 10.9 dB. We demonstrate experimentally the ability to generate and scan a focal point with femtosecond pulses and perform two-photon imaging.

Bending-induced inter-core group delays in multicore fibers

We examine the impact of fiber bends on ultrashort pulse propagation in a 169-core multicore fiber (MCF) by numerical simulations and experimental measurements. We show that an L-shaped bend (where only one end of the MCF is fixed) induces significant changes in group delays that are a function of core position but linear along the bending axis with a slope directly proportional to the bending angle. For U- and S-shaped bends (where both ends of the MCF are fixed) the induced refractive index and group delay changes are much smaller than the residual, intrinsic inter-core group delay differences of the unbent MCF. We further show that when used for point-scanning lensless endoscopy with ultrashort pulse excitation, bend-induced group delays in the MCF degrade the point-spread function due to spatiotemporal coupling. Our results show that bend-induced effects in MCFs can be parametrized with only two parameters: the angle of the bend axis and the amplitude of the bend. This remains valid for bend amplitudes up to at least 200 degrees.

Ultra-short pulse propagation model for multi-core fibers based on local modes

Multi-core fibers (MCFs) have sparked a new paradigm in optical communications and open new possibilities and applications in experimental physics and other fields of science, such as biological and medical imaging. In many of these cases, ultra-short pulse propagation is revealed as a key factor that enables us to exploit the full potential of this technology. Unfortunately, the propagation of such pulses in real MCFs has not yet been modelled considering polarization effects or typical random medium perturbations, which usually give rise to both longitudinal and temporal birefringent effects. Using the concept of local modes, we develop here an accurate ultra-short pulse propagation model that rigorously accounts for these phenomena in single-mode MCFs. Based on this theory, we demonstrate analytically and numerically the intermodal dispersion between different LP₀₁ polarized core modes induced by these random perturbations when propagating femtosecond pulses in the linear and nonlinear fiber regimes. The ever-decreasing core-to-core distance significantly enhances the intermodal dispersion induced by these birefringent effects, which can become the major physical impairment in the single-mode regime. To demonstrate the power of our model, we give explicit strategies to reduce the impact of this optical impairment by increasing the MCF perturbations.

Learning-based focusing through scattering media

We present a machine-learning-based method for light focusing through scattering media. In this method, the optical process in a scattering medium is computationally inverted based on a nonlinear regression algorithm with a number of training input-output pairs through the medium, and an input optimized for a target output is calculated. We experimentally demonstrate focusing via a process involving randomness due to a scattering medium and nonlinearity due to double modulation with a spatial light modulator. Our approach realizes model-free control of optical fields, where optical processes or models are unknown.