Time-of-flight 3D imaging through multimode optical fibers

Fluorescence lifetime imaging with distance and ranging using a miniaturised SPAD system

In this work we demonstrate a miniaturised imaging system based around a time-gated SPAD array operating in a "chip-on-tip" manner. Two versions of the system are demonstrated, each measuring 23 mm × 23 mm × 28 mm with differing fields of view and working distances. Initial tests demonstrate contrast between materials in widefield fluorescence imaging (WFLIm) mode, with frame rates of > 2 Hz achievable. Following this, WFLIm images of autofluorescence in ovine lung tissue are obtained at frame rates of ~ 1 Hz. Finally, the ability of the second system to perform simultaneous WFLIm and time of flight (aka Flourescence Lifetime Imaging Distance and Ranging, FLImDAR) is also tested. This shows that the system is capable of 4 mm resolution of object separation when tested on 3D printed samples. It is further demonstrated as being able to perform scene reconstruction on autofluorescent lung tissue. This system is, to date, the smallest chip on tip WFLIm system published, and is the first demonstration of the FLImDAR technique in a compact, portable system.

Ultrafast longitudinal imaging of haemodynamics via single-shot volumetric photoacoustic tomography with a single-element detector

Techniques for imaging haemodynamics use ionizing radiation or contrast agents or are limited by imaging depth (within approximately 1 mm), complex and expensive data-acquisition systems, or low imaging speeds, system complexity or cost. Here we show that ultrafast volumetric photoacoustic imaging of haemodynamics in the human body at up to 1 kHz can be achieved using a single laser pulse and a single element functioning as 6,400 virtual detectors. The technique, which does not require recalibration for different objects or during long-term operation, enables the longitudinal volumetric imaging of haemodynamics in vasculature a few millimetres below the skin's surface. We demonstrate this technique in vessels in the feet of healthy human volunteers by capturing haemodynamic changes in response to vascular occlusion. Single-shot volumetric photoacoustic imaging using a single-element detector may facilitate the early detection and monitoring of peripheral vascular diseases and may be advantageous for use in biometrics and point-of-care testing.

Exploiting holographically encoded variance to transmit labelled images through a multimode optical fiber

Artificial intelligence has emerged as promising tool to decode an image transmitted through a multimode fiber (MMF) by applying deep learning techniques. By transmitting thousands of images through the MMF, deep neural networks (DNNs) are able to decipher the seemingly random output speckle patterns and unveil the intrinsic input-output relationship. High fidelity reconstruction is obtained for datasets with a large degree of homogeneity, which underutilizes the capacity of the combined MMF-DNN system. Here, we show that holographic modulation can encode an additional layer of variance on the output speckle pattern, improving the overall transmissive capabilities of the system. Operatively, we have implemented this by adding a holographic label to the original dataset and injecting the resulting phase image into the fiber facet through a Fourier transform lens. The resulting speckle pattern dataset can be clustered primarily by holographic label, and can be reconstructed without loss of fidelity. As an application, we describe how color images may be segmented into RGB components and each color component may then be labelled by distinct hologram. A ResUNet architecture was then used to decode each class of speckle patterns and reconstruct the color image without the need for temporal synchronization between sender and receiver.

High-quality coherent ghost imaging of a transmission target

When the test detector of ghost imaging (GI) is a point-like detector and the detector's transverse size is smaller than the transverse coherence length of the light field at the detection plane, this case is corresponding to coherent GI (CGI) and the imaging result recovered by traditional GI (TGI) reconstruction algorithm is usually bad for a transmission target. Here a CGI scheme of a transmission target is proposed and a corresponding CGI reconstruction algorithm is developed to stably recover the target's image. The validity of the proposed method is verified by both simulation and experiments. Both the simulation and experimental results demonstrate that the target's transmission function can be perfectly reconstructed by CGI. We also show that the imaging quality of CGI with a point-like detector is better than that of TGI with a bucket detector if detection noise exists in the sampling process. Performance comparisons between CGI reconstruction and TGI reconstruction are also discussed.

Structured light enhanced machine learning for fiber bend sensing

The intricate optical distortions that occur when light interacts with complex media, such as few- or multi-mode optical fiber, often appear random in origin and are a fundamental source of error for communication and sensing systems. We propose the use of orbital angular momentum (OAM) feature extraction to mitigate phase-noise and allow for the use of intermodal-coupling as an effective tool for fiber sensing. OAM feature extraction is achieved by passive all-optical OAM demultiplexing, and we demonstrate fiber bend tracking with 94.1% accuracy. Conversely, an accuracy of only 14% was achieved for determining the same bend positions when using a convolutional-neural-network trained with intensity measurements of the output of the fiber. Further, OAM feature extraction used 120 times less information for training compared to intensity image based measurements. This work indicates that structured light enhanced machine learning could be used in a wide range of future sensing technologies.

Combined fluorescence lifetime and surface topographical imaging of biological tissue

In this work a combined fluorescence lifetime and surface topographical imaging system is demonstrated. Based around a 126 × 192 time resolved single photon avalanche diode (SPAD) array operating in time correlated single-photon counting (TCSPC) mode, both the fluorescence lifetime and time of flight (ToF) can be calculated on a pixel by pixel basis. Initial tests on fluorescent samples show it is able to provide 4 mm resolution in distance and 0.4 ns resolution in lifetime. This combined modality has potential biomedical applications such as surgical guidance, endoscopy, and diagnostic imaging. The system is demonstrated on both ovine and human pulmonary tissue samples, where it offers excellent fluorescence lifetime contrast whilst also giving a measure of the distance to the sample surface.

Wavefront shaping improves the transparency of the scattering media: a review

Significance

Wavefront shaping (WFS) can compensate for distortions by optimizing the wavefront of the input light or reversing the transmission matrix of the media. It is a promising field of research. A thorough understanding of principles and developments of WFS is important for optical research.

Aim

To provide insight into WFS for researchers who deal with scattering in biomedicine, imaging, and optical communication, our study summarizes the basic principles and methods of WFS and reviews recent progress.

Approach

The basic principles, methods of WFS, and the latest applications of WFS in focusing, imaging, and multimode fiber (MMF) endoscopy are described. The practical challenges and prospects of future development are also discussed.

Results

Data-driven learning-based methods are opening up new possibilities for WFS. High-resolution imaging through MMFs can support small-diameter endoscopy in the future.

Conclusion

The rapid development of WFS over the past decade has shown that the best solution is not to avoid scattering but to find ways to correct it or even use it. WFS with faster speed, more optical modes, and more modulation degrees of freedom will continue to drive exciting developments in various fields.

Lensless single-fiber ghost imaging

We demonstrate lensless single-fiber ghost imaging, which allows illumination and collection using a single optical fiber without a transmission-type system. Speckle patterns with relative coincidence degrees of 0.14 were formed by image reconstruction using improved differential ghost imaging. Employing fiber with a diameter of 105 µm, we achieved a spatial resolution of 0.05 mm in an observing area of 9m m 2, at a working distance of 10 mm. Compared to a conventional neuroendoscope at a power density of 94m W/c m 2, our imaging could be realized by extremely weak illumination at a laser power density of 0.10m W/c m 2. Using our lensless single-fiber ghost imaging, with 30,000 speckle patterns and implementing a diffuser, we attained an average coincidence degree of 0.45.

Spatially resolved readout of a Fabry-Perot ultrasound sensor interrogated through a multimode optical fiber using wavefront shaping

The spatially resolved interrogation of a Fabry-Perot ultrasound sensor using a laser beam focused through a multimode fiber is demonstrated. To scan the beam across the sensor as required to read it out, optical wavefront shaping was employed to compensate for the scrambling of light in the fiber. By providing a means to map ultrasound through inexpensive, lightweight fibers, this could lead to new ultrasonic and photoacoustic imaging systems, such as endoscopes and flexible handheld probes.

Optimizing the internal phase reference to shape the output of a multimode optical fiber

Pre-shaping light to achieve desired amplitude distributions at the tip of a multimode fiber (MMF) has emerged as a powerful method allowing a wide range of imaging techniques to be implemented at the distal facet. Such techniques rely on measuring the transmission matrix of the optically turbid waveguide which scrambles the coherent input light into an effectively random speckle pattern. Typically, this is done by measuring the interferogram between the output speckle and a reference beam. In recent years, an optical setup where the reference beam passes through the MMF has become an attractive configuration because of the high interferometric stability of the common optical path. However, the merits and drawbacks of an internal reference beam remain controversial. The measurement of the transmission matrix is known to depend on the choice of internal reference and has been reported to result in "blind spots" due to phase singularities of the reference beam. Here, we describe how the focussing efficiency of the calibration can be increased by several percent by optimising the choice of internal reference beam.

Optical single-pixel volumetric imaging by three-dimensional light-field illumination

Three-dimensional single-pixel imaging (3D SPI) has become an attractive imaging modality for both biomedical research and optical sensing. 3D-SPI techniques generally depend on time-of-flight or stereovision principle to extract depth information from backscattered light. However, existing implementations for these two optical schemes are limited to surface mapping of 3D objects at depth resolutions, at best, at the millimeter level. Here, we report 3D light-field illumination single-pixel microscopy (3D-LFI-SPM) that enables volumetric imaging of microscopic objects with a near-diffraction-limit 3D optical resolution. Aimed at 3D space reconstruction, 3D-LFI-SPM optically samples the 3D Fourier spectrum by combining 3D structured light-field illumination with single-element intensity detection. We build a 3D-LFI-SPM prototype that provides an imaging volume of ∼390 × 390 × 3,800 μm3 and achieves 2.7-μm lateral resolution and better than 37-μm axial resolution. Its capability of 3D visualization of label-free optical absorption contrast is demonstrated by imaging single algal cells in vivo. Our approach opens broad perspectives for 3D SPI with potential applications in various fields, such as biomedical functional imaging.

Robust real-time imaging through flexible multimode fibers

Conventional endoscopes comprise a bundle of optical fibers, associating one fiber for each pixel in the image. In principle, this can be reduced to a single multimode optical fiber (MMF), the width of a human hair, with one fiber spatial-mode per image pixel. However, images transmitted through a MMF emerge as unrecognizable speckle patterns due to dispersion and coupling between the spatial modes of the fiber. Furthermore, speckle patterns change as the fiber undergoes bending, making the use of MMFs in flexible imaging applications even more complicated. In this paper, we propose a real-time imaging system using flexible MMFs, but which is robust to bending. Our approach does not require access or feedback signal from the distal end of the fiber during imaging. We leverage a variational autoencoder to reconstruct and classify images from the speckles and show that these images can still be recovered when the bend configuration of the fiber is changed to one that was not part of the training set. We utilize a MMF 300 mm long with a 62.5 μm core for imaging [Formula: see text] cm objects placed approximately at 20 cm from the fiber and the system can deal with a change in fiber bend of 50[Formula: see text] and range of movement of 8 cm.

X-ray-to-visible light-field detection through pixelated colour conversion

Light-field detection measures both the intensity of light rays and their precise direction in free space. However, current light-field detection techniques either require complex microlens arrays or are limited to the ultraviolet-visible light wavelength ranges^1-4. Here we present a robust, scalable method based on lithographically patterned perovskite nanocrystal arrays that can be used to determine radiation vectors from X-rays to visible light (0.002-550 nm). With these multicolour nanocrystal arrays, light rays from specific directions can be converted into pixelated colour outputs with an angular resolution of 0.0018°. We find that three-dimensional light-field detection and spatial positioning of light sources are possible by modifying nanocrystal arrays with specific orientations. We also demonstrate three-dimensional object imaging and visible light and X-ray phase-contrast imaging by combining pixelated nanocrystal arrays with a colour charge-coupled device. The ability to detect light direction beyond optical wavelengths through colour-contrast encoding could enable new applications, for example, in three-dimensional phase-contrast imaging, robotics, virtual reality, tomographic biological imaging and satellite autonomous navigation.

Single-shot 3D photoacoustic tomography using a single-element detector for ultrafast imaging of hemodynamics

Imaging hemodynamics is crucial for the diagnosis, treatment, and prevention of vascular diseases. However, current imaging techniques are limited due to the use of ionizing radiation or contrast agents, short penetration depth, or complex and expensive data acquisition systems. Photoacoustic tomography shows promise as a solution to these issues. However, existing photoacoustic tomography methods collect signals either sequentially or through numerous detector elements, leading to either low imaging speed or high system complexity and cost. To address these issues, here we introduce a method to capture a 3D photoacoustic image of vasculature using a single laser pulse and a single-element detector that functions as 6,400 virtual ones. Our method enables ultrafast volumetric imaging of hemodynamics in the human body at up to 1 kHz and requires only a single calibration for different objects and for long-term operations. We demonstrate 3D imaging of hemodynamics at depth in humans and small animals, capturing the variability in blood flow speeds. This concept can inspire other imaging technologies and find applications such as home-care monitoring, biometrics, point-of-care testing, and wearable monitoring.

Ultra-compact MXene fibers by continuous and controllable synergy of interfacial interactions and thermal drawing-induced stresses

Recent advances in MXene (Ti₃C₂T_x) fibers, prepared from electrically conductive and mechanically strong MXene nanosheets, address the increasing demand of emerging yet promising electrode materials for the development of textile-based devices and beyond. However, to reveal the full potential of MXene fibers, reaching a balance between electrical conductivity and mechanical property is still the fundamental challenge, mainly due to the difficulties to further compact the loose MXene nanosheets. In this work, we demonstrate a continuous and controllable route to fabricate ultra-compact MXene fibers with an in-situ generated protective layer via the synergy of interfacial interactions and thermal drawing-induced stresses. The resulting ultra-compact MXene fibers with high orientation and low porosity exhibit not only excellent tensile strength and ultra-high toughness, but also high electrical conductivity. Then, we construct meter-scale MXene textiles using these ultra-compact fibers to achieve high-performance electromagnetic interference shielding and personal thermal management, accompanied by the high mechanical durability and stability even after multiple washing cycles. The demonstrated generic strategy can be applied to a broad range of nanostructured materials to construct functional fibers for large-scale applications in both space and daily lives.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media. Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution. However, the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications. In addition, the components of an optical system are usually designed and manufactured for a fixed function or performance. Recent advances in wavefront shaping have demonstrated that scattering- or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium. This offers unprecedented opportunities in many applications to achieve controllable optical delivery or detection at depths or dynamically configurable functionalities by using scattering media to substitute conventional optical components. In this article, the recent progress of wavefront shaping in multidisciplinary fields is reviewed, from optical focusing and imaging with scattering media, functionalized devices, modulation of mode coupling, and nonlinearity in multimode fiber to multimode fiber-based applications. Apart from insights into the underlying principles and recent advances in wavefront shaping implementations, practical limitations and roadmap for future development are discussed in depth. Looking back and looking forward, it is believed that wavefront shaping holds a bright future that will open new avenues for noninvasive or minimally invasive optical interactions and arbitrary control inside deep tissues. The high degree of freedom with multiple scattering will also provide unprecedented opportunities to develop novel optical devices based on a single scattering medium (generic or customized) that can outperform traditional optical components.

Fast hyperspectral single-pixel imaging via frequency-division multiplexed illumination

Hyperspectral imaging that detects 3D spectra-spatial information has been used in a wide range of applications. Among reported techniques, multiplexed spectral imaging with a single-pixel detector provides as a photon-efficient and low-cost implementation; however, the previous spectral modulation schemes are mostly complicated and sacrifice the imaging speed. Here, we propose a fast and compact hyperspectral single-pixel imaging technique based on programmable chromatic illumination. A multi-wavelength LED array modulated by independent carriers achieves stable and accurate spectral modulation up to MHz in a frequency-division multiplexed manner, hence allowing the full use of the spatial light modulation speed. Additionally, we propose a multi-channel deep convolutional autoencoder network to reconstruct hyperspectral data from highly-compressed 1D measurement. Experimental reconstructions of 12 spectral channels and 64 × 64 pixels are demonstrated for dynamic imaging at 12 fps image rate. The proposed imaging scheme is highly extensible to a wide spectrum range, and holds potential for portable spectral imagers in low-light or scattering applications.

Holographic Manipulation of Nanostructured Fiber Optics Enables Spatially-Resolved, Reconfigurable Optical Control of Plasmonic Local Field Enhancement and SERS

Integration of plasmonic structures on step-index optical fibers is attracting interest for both applications and fundamental studies. However, the possibility to dynamically control the coupling between the guided light fields and the plasmonic resonances is hindered by the turbidity of light propagation in multimode fibers (MMFs). This pivotal point strongly limits the range of studies that can benefit from nanostructured fiber optics. Fortunately, harnessing the interaction between plasmonic modes on the fiber tip and the full set of guided modes can bring this technology to a next generation progress. Here, the intrinsic wealth of information of guided modes is exploited to spatiotemporally control the plasmonic resonances of the coupled system. This concept is shown by employing dynamic phase modulation to structure both the response of plasmonic MMFs on the plasmonic facet and their response in the corresponding Fourier plane, achieving spatial selective field enhancement and direct control of the probe's work point in the dispersion diagram. Such a conceptual leap would transform the biomedical applications of holographic endoscopic imaging by integrating new sensing and manipulation capabilities.

3D single-pixel imaging at the near-infrared wave band

Focal plane detector array technology in the infrared wave band is expensive or underdeveloped, and the detection efficiency is low, while single-pixel imaging (SPI) offers better performance, such as ultrafast time response and high quantum efficiency in wide wave bands. Therefore, SPI technology can be used for infrared imaging. In this work, a near-infrared raster scan SPI system is proposed. By means of a grating to modulate height information of objects, we can further achieve three-dimensional imaging in the framework of Fourier transform profilometry. The proposed approach is demonstrated with experiments at the wavelength of 1064 nm.

Endoscopic displacement measurement based on fiber optic bundles

In-line monitoring and routine inspection are essential for using and maintaining complex equipment. The simultaneous implementation of visual positioning and displacement measurement allows the accurate acquisition of characteristics, including object dimensions and mechanical vibrations, while rapidly locking the target position. However, the internal structure of equipment is frequently obscured, making direct visual inspection challenging; therefore, flexible and bendable fiber optic-based endoscopes are extremely valuable in harsh conditions. This study enables all-fiber visual displacement measurement using a single-mode fiber and an imaging fiber bundle. Based on optical triangulation and spot centers extraction method from fiber bundle images, 0.07 mm precision at a measurement distance of 40.12 mm is achieved vertically for rough objects. We demonstrate its surface reconstruction and vibration measurement functions. Factors that affect measurement accuracy, such as light source and object roughness, are also discussed.

Time-of-flight 3D imaging through multimode optical fibers

[Figure: see text].