Single-shot ultrafast optical imaging

EventLFM: event camera integrated Fourier light field microscopy for ultrafast 3D imaging

Ultrafast 3D imaging is indispensable for visualizing complex and dynamic biological processes. Conventional scanning-based techniques necessitate an inherent trade-off between acquisition speed and space-bandwidth product (SBP). Emerging single-shot 3D wide-field techniques offer a promising alternative but are bottlenecked by the synchronous readout constraints of conventional CMOS systems, thus restricting data throughput to maintain high SBP at limited frame rates. To address this, we introduce EventLFM, a straightforward and cost-effective system that overcomes these challenges by integrating an event camera with Fourier light field microscopy (LFM), a state-of-the-art single-shot 3D wide-field imaging technique. The event camera operates on a novel asynchronous readout architecture, thereby bypassing the frame rate limitations inherent to conventional CMOS systems. We further develop a simple and robust event-driven LFM reconstruction algorithm that can reliably reconstruct 3D dynamics from the unique spatiotemporal measurements captured by EventLFM. Experimental results demonstrate that EventLFM can robustly reconstruct fast-moving and rapidly blinking 3D fluorescent samples at kHz frame rates. Furthermore, we highlight EventLFM's capability for imaging of blinking neuronal signals in scattering mouse brain tissues and 3D tracking of GFP-labeled neurons in freely moving C. elegans. We believe that the combined ultrafast speed and large 3D SBP offered by EventLFM may open up new possibilities across many biomedical applications.

Post-ensemble generation with Airy beams for spatial and spectral switching in incoherent imaging

Spatial, temporal, and spectral resolutions and field-of-view are important characteristics of any imaging system. In most, if not all, it is impossible to change the above characteristics after recording a digital picture, video, or hologram. In recent years, there have been investigations on the possibilities to change the above characteristics post-recording. In this Letter, for the first time, to the best of our knowledge, we report novel recording and reconstruction methods built upon the principles of coded aperture imaging that allow changing the axial and spectral resolutions post-recording. We named this method-post-ensemble generation with Airy beams for spatial and spectral switching (PEGASASS). In PEGASASS, light from an object point is converted into Airy beams and recorded such that every recording has a unique Airy pattern. An ensemble of Airy patterns is constructed post-recording and the axial and spectral resolutions are tuned by controlling the chaos in the ensemble. The above tunability is achieved without adversely affecting the lateral resolution. Proof-of-concept experimental results of PEGASASS in 3D in both (x,y,z) and (x,y,λ) and 4D in (x,y,z,λ) are presented. We believe that PEGASASS has the potential to revolutionize the field of imaging and holography.

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Unsupervised physics-informed deep learning-based reconstruction for time-resolved imaging by multiplexed ptychography

We explore numerically an unsupervised, physics-informed, deep learning-based reconstruction technique for time-resolved imaging by multiplexed ptychography. In our method, the untrained deep learning model replaces the iterative algorithm's update step, yielding superior reconstructions of multiple dynamic object frames compared to conventional methodologies. More precisely, we demonstrate improvements in image quality and resolution, while reducing sensitivity to the number of recorded frames, the mutual orthogonality of different probe modes, overlap between neighboring probe beams and the cutoff frequency of the ptychographic microscope - properties that are generally of paramount importance for ptychographic reconstruction algorithms.

Swept coded aperture real-time femtophotography

Single-shot real-time femtophotography is indispensable for imaging ultrafast dynamics during their times of occurrence. Despite their advantages over conventional multi-shot approaches, existing techniques confront restricted imaging speed or degraded data quality by the deployed optoelectronic devices and face challenges in the application scope and acquisition accuracy. They are also hindered by the limitations in the acquirable information imposed by the sensing models. Here, we overcome these challenges by developing swept coded aperture real-time femtophotography (SCARF). This computational imaging modality enables all-optical ultrafast sweeping of a static coded aperture during the recording of an ultrafast event, bringing full-sequence encoding of up to 156.3 THz to every pixel on a CCD camera. We demonstrate SCARF's single-shot ultrafast imaging ability at tunable frame rates and spatial scales in both reflection and transmission modes. Using SCARF, we image ultrafast absorption in a semiconductor and ultrafast demagnetization of a metal alloy.

Single-shot ptychographic imaging of non-repetitive ultrafast events

We demonstrate experimentally high-speed ptychographic imaging of non-repetitive complex-valued events. Three time-resolved complex-valued frames are reconstructed from data recorded in a single camera snapshot. The temporal resolution of the microscope is determined by delays between illuminating pulses. The ability to image amplitude and phase of nonrepetitive events with ultrafast temporal resolution will open new opportunities in science and technology.

Tutorial on compressed ultrafast photography

Significance

Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges.

Aim

This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect.

Approach

A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement.

Conclusions

CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

Single-shot optical imaging with spectrum circuit bridging timescales in high-speed photography

Single-shot optical imaging based on ultrashort lasers has revealed nonrepetitive processes in subnanosecond timescales beyond the recording range of conventional high-speed cameras. However, nanosecond photography without sacrificing short exposure time and image quality is still missing because of the gap in recordable timescales between ultrafast optical imaging and high-speed electronic cameras. Here, we demonstrate nanosecond photography and ultrawide time-range high-speed photography using a spectrum circuit that produces interval-tunable pulse trains while keeping short pulse durations. We capture a shock wave propagating through a biological cell with a 1.5-ns frame interval and 44-ps exposure time while suppressing image blur. Furthermore, we observe femtosecond laser processing over multiple timescales (25-ps, 2.0-ns, and 1-ms frame intervals), showing that the plasma generated at the picosecond timescale affects subsequent shock wave formation at the nanosecond timescale. Our technique contributes to accumulating data of various fast processes for analysis and to analyzing multi-timescale phenomena as a series of physical processes.

Flexible and accurate total variation and cascaded denoisers-based image reconstruction algorithm for hyperspectrally compressed ultrafast photography

Hyperspectrally compressed ultrafast photography (HCUP) based on compressed sensing and time- and spectrum-to-space mappings can simultaneously realize the temporal and spectral imaging of non-repeatable or difficult-to-repeat transient events with a passive manner in single exposure. HCUP possesses an incredibly high frame rate of tens of trillions of frames per second and a sequence depth of several hundred, and therefore plays a revolutionary role in single-shot ultrafast optical imaging. However, due to ultra-high data compression ratios induced by the extremely large sequence depth, as well as limited fidelities of traditional algorithms over the image reconstruction process, HCUP suffers from a poor image reconstruction quality and fails to capture fine structures in complex transient scenes. To overcome these restrictions, we report a flexible image reconstruction algorithm based on a total variation (TV) and cascaded denoisers (CD) for HCUP, named the TV-CD algorithm. The TV-CD algorithm applies the TV denoising model cascaded with several advanced deep learning-based denoising models in the iterative plug-and-play alternating direction method of multipliers framework, which not only preserves the image smoothness with TV, but also obtains more priori with CD. Therefore, it solves the common sparsity representation problem in local similarity and motion compensation. Both the simulation and experimental results show that the proposed TV-CD algorithm can effectively improve the image reconstruction accuracy and quality of HCUP, and may further promote the practical applications of HCUP in capturing high-dimensional complex physical, chemical and biological ultrafast dynamic scenes.

Nanoscale chemical imaging with structured X-ray illumination

High-resolution imaging with compositional and chemical sensitivity is crucial for a wide range of scientific and engineering disciplines. Although synchrotron X-ray imaging through spectromicroscopy has been tremendously successful and broadly applied, it encounters challenges in achieving enhanced detection sensitivity, satisfactory spatial resolution, and high experimental throughput simultaneously. In this work, based on structured illumination, we develop a single-pixel X-ray imaging approach coupled with a generative image reconstruction model for mapping the compositional heterogeneity with nanoscale resolvability. This method integrates a full-field transmission X-ray microscope with an X-ray fluorescence detector and eliminates the need for nanoscale X-ray focusing and raster scanning. We experimentally demonstrate the effectiveness of our approach by imaging a battery sample composed of mixed cathode materials and successfully retrieving the compositional variations of the imaged cathode particles. Bridging the gap between structural and chemical characterizations using X-rays, this technique opens up vast opportunities in the fields of biology, environmental, and materials science, especially for radiation-sensitive samples.

Ultrafast pump-probe spectroscopy via chirped-pulse up-conversion with dispersion compensation

In this study, ultrafast transient signals were detected on a single-shot basis using chirped-pulse up-conversion spectroscopy with dispersion compensation. Unlike in the conventional time-encoding technique using chirped pulses, distortion of the ultrafast waveform was reduced by applying dispersion compensation to the chirped probe pulses and using sum-frequency generation with the chirped readout pulses. The method was applied to terahertz time-domain spectroscopy and near-infrared pump-probe spectroscopy, providing ultrafast observations with an improved temporal resolution comparable to the transform-limited pulse durations. Terahertz waveforms, Kerr rotation signals, and phonon-polariton oscillations were measured accurately with no significant waveform distortion, thereby showing the proposed scheme to be promising for single-shot pump-probe spectroscopy in a wide range of spectroscopic applications.

Single-shot videography with multiplex structured illumination using an interferometer

Frequency recognition algorithm for multiple exposures (FRAME) is a high-speed videography technique that exposes a dynamic object to time-varying structured illumination (SI) and captures two-dimensional transients in a single shot. Conventional FRAME requires light splitting to increase the number of frames per shot, thereby resulting in optical loss and a limited number of frames per shot. Here, we propose and demonstrate a novel FRAME method which overcomes these problems by utilizing an interferometer to generate a time-varying SI without light splitting. Combining this method with a pulsed laser enables low-cost, high-speed videography on a variety of timescales from microseconds.

Single-Shot Reconfigurable Femtosecond Imaging of Ultrafast Optical Dynamics

Understanding ultrafast dynamics in the femtosecond timescale plays a pivotal role in fundamental research and technology innovation. Spatiotemporal observation of those events in real-time requires imaging speeds greater than 10¹² frames per second (fps), far beyond the fundamental speed limits of the ubiquitous semiconductor sensor technologies. In addition, a majority of femtosecond events are non-repeatable or difficult-to-repeat since they either work in a highly unstable nonlinear regime or require extreme or rare conditions to initiate. Therefore, the traditional pump-probe imaging approach fails since it heavily depends on precise event repetition. Single-shot ultrafast imaging emerges as the only solution; however, existing techniques cannot reach more than 15×10¹² fps, and they only record an insufficient number of frames. Compressed ultrafast spectral photography (CUSP) is proposed to overcome these limitations. Here, CUSP's full design space is explored by manipulating the ultrashort optical pulse in the active illumination. Via parameter optimization, an extraordinarily fast frame rate of 219×10¹² fps is achieved. This implementation of CUSP is also highly flexible, allowing various combinations of imaging speeds and numbers of frames (several hundred up to 1000) to be readily deployed in diverse scientific studies, such as laser-induced transient birefringence, self-focusing, and filaments in dielectric media.

Unidirectional imaging using deep learning-designed materials

A unidirectional imager would only permit image formation along one direction, from an input field-of-view (FOV) A to an output FOV B, and in the reverse path, B → A, the image formation would be blocked. We report the first demonstration of unidirectional imagers, presenting polarization-insensitive and broadband unidirectional imaging based on successive diffractive layers that are linear and isotropic. After their deep learning-based training, the resulting diffractive layers are fabricated to form a unidirectional imager. Although trained using monochromatic illumination, the diffractive unidirectional imager maintains its functionality over a large spectral band and works under broadband illumination. We experimentally validated this unidirectional imager using terahertz radiation, well matching our numerical results. We also created a wavelength-selective unidirectional imager, where two unidirectional imaging operations, in reverse directions, are multiplexed through different illumination wavelengths. Diffractive unidirectional imaging using structured materials will have numerous applications in, e.g., security, defense, telecommunications, and privacy protection.

Single-Shot Multi-Frame Imaging of Femtosecond Laser-Induced Plasma Propagation

Single-shot ultrafast multi-frame imaging technology plays a crucial role in the observation of laser-induced plasma. However, there are many challenges in the application of laser processing, such as technology fusion and imaging stability. To provide a stable and reliable observation method, we propose an ultrafast single-shot multi-frame imaging technology based on wavelength polarization multiplexing. Through the frequency doubling and birefringence effects of the BBO and the quartz crystal, the 800 nm femtosecond laser pulse was frequency doubled to 400 nm, and a sequence of probe sub-pulses with dual-wavelength and different polarization was generated. The coaxial propagation and framing imaging of multi-frequency pulses provided stable imaging quality and clarity, as well as high temporal/spatial resolution (200 fs and 228 lp/mm). In the experiments involving femtosecond laser-induced plasma propagation, the probe sub-pulses measured their time intervals by capturing the same results. Specifically, the measured time intervals were 200 fs between the same color pulses and 1 ps between the adjacent different. Finally, based on the obtained system time resolution, we observed and revealed the evolution mechanism of femtosecond laser-induced air plasma filaments, the multifilament propagation of femtosecond laser in fused silica, and the influence mechanism of air ionization on laser-induced shock waves.

Single-shot ultrafast terahertz photography

Multidimensional imaging of transient events has proven pivotal in unveiling many fundamental mechanisms in physics, chemistry, and biology. In particular, real-time imaging modalities with ultrahigh temporal resolutions are required for capturing ultrashort events on picosecond timescales. Despite recent approaches witnessing a dramatic boost in high-speed photography, current single-shot ultrafast imaging schemes operate only at conventional optical wavelengths, being suitable solely within an optically-transparent framework. Here, leveraging on the unique penetration capability of terahertz radiation, we demonstrate a single-shot ultrafast terahertz photography system that can capture multiple frames of a complex ultrafast scene in non-transparent media with sub-picosecond temporal resolution. By multiplexing an optical probe beam in both the time and spatial-frequency domains, we encode the terahertz-captured three-dimensional dynamics into distinct spatial-frequency regions of a superimposed optical image, which is then computationally decoded and reconstructed. Our approach opens up the investigation of non-repeatable or destructive events that occur in optically-opaque scenarios.

Augmented light field tomography through parallel spectral encoding

Snapshot recording of transient dynamics in three dimensions (3-D) is highly demanded in both fundamental and applied sciences. Yet it remains challenging for conventional high-speed cameras to address this need due to limited electronic bandwidth and reliance on mechanical scanning. The emergence of light field tomography (LIFT) provides a new solution to these long-standing problems and enables 3-D imaging at an unprecedented frame rate. However, based on sparse-view computed tomography, LIFT can accommodate only a limited number of projections, degrading the resolution in the reconstructed image. To alleviate this problem, we herein present a spectral encoding scheme to significantly increase the number of allowable projections in LIFT while maintaining its snapshot advantage. The resultant system can record 3-D dynamics at a kilohertz volumetric frame rate. Moreover, by using a multichannel compressed sensing algorithm, we improve the image quality with an enhanced spatial resolution and suppressed aliasing artifacts.

High-speed videography of transparent media using illumination-based multiplexed schlieren

Schlieren photography is widely used for visualizing phenomena within transparent media. The technique, which comes in a variety of configurations, is based on detecting or extracting the degree to which light is deflected whilst propagating through a sample. To date, high-speed schlieren videography can only be achieved using high-speed cameras, thus limiting the frame rate of such configurations to the capabilities of the camera. Here we demonstrate, for the first time, optically multiplexed schlieren videography, a concept that allows such hardware limitations to be bypassed, opening up for, in principle, an unlimited frame rate. By illuminating the sample with a rapid burst of uniquely spatially modulated light pulses, a temporally resolved sequence can be captured in a single photograph. The refractive index variations are thereafter measured by quantifying the local phase shift of the superimposed intensity modulations. The presented results demonstrate the ability to acquire a series of images of flame structures at frame rates up to 1 Mfps using a standard 50 fps sCMOS camera.

Simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography

We present a simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography. In this system, we propose to combine a specially designed mini-reflector delay-line array with digital in-line holography. The former including a group of adjustable mini-reflectors can easily generate an array of probe sub-pulses that can be controlled independently in the propagation direction and time delays. The object beams formed by the different sub-pulses will propagate and fall on different recording regions of the image sensor to generate a single-shot spatial-multiplexing in-line hologram. The geometry of the digital in-line holography can simplify the complexity of the system and enable complex amplitude imaging. In addition, the time resolution of this system is limited only by the pulse duration, which allows this system to study the dynamic processes with the femtosecond order. In an experiment about the laser-induced air plasma, our proposed system achieves nine frames sequential holographic images with the frame rate of 7.5 trillion frames per second (Tfps).

A Novel Reconstruction Algorithm with High Performance for Compressed Ultrafast Imaging

Compressed ultrafast photography (CUP) is a type of two-dimensional (2D) imaging technique to observe ultrafast processes. Intelligence reconstruction methods that influence the imaging quality are an essential part of a CUP system. However, existing reconstruction algorithms mostly rely on image priors and complex parameter spaces. Therefore, it usually takes a lot of time to obtain acceptable reconstruction results, which limits the practical application of the CUP. In this paper, we proposed a novel reconstruction algorithm named PnP-FFDNet, which can provide a high quality and high efficiency compared to previous methods. First, we built a forward model of the CUP and three sub-optimization problems were obtained using the alternating direction multiplier method (ADMM), and the closed-form solution of the first sub-optimization problem was derived. Secondly, inspired by the PnP-ADMM framework, we used an advanced denoising algorithm based on a neural network named FFDNet to solve the second sub-optimization problem. On the real CUP data, PSNR and SSIM are improved by an average of 3 dB and 0.06, respectively, compared with traditional algorithms. Both on the benchmark dataset and on the real CUP data, the proposed method reduces the running time by an average of about 96% over state-of-the-art algorithms, and show comparable visual results, but in a much shorter running time.

JR2net: a joint non-linear representation and recovery network for compressive spectral imaging

Deep learning models are state-of-the-art in compressive spectral imaging (CSI) recovery. These methods use a deep neural network (DNN) as an image generator to learn non-linear mapping from compressed measurements to the spectral image. For instance, the deep spectral prior approach uses a convolutional autoencoder (CAE) network in the optimization algorithm to recover the spectral image by using a non-linear representation. However, the CAE training is detached from the recovery problem, which does not guarantee optimal representation of the spectral images for the CSI problem. This work proposes a joint non-linear representation and recovery network (JR2net), linking the representation and recovery task into a single optimization problem. JR2net consists of an optimization-inspired network following an alternating direction method of multipliers (ADMM) formulation that learns a non-linear low-dimensional representation and simultaneously performs the spectral image recovery, trained via the end-to-end approach. Experimental results show the superiority of the proposed method with improvements up to 2.57 dB in peak signal-to-noise ratio (PSNR) and performance around 2000 times faster than state-of-the-art methods.

Weighted multi-scale denoising via adaptive multi-channel fusion for compressed ultrafast photography

Being capable of passively capturing transient scenes occurring in picoseconds and even shorter time with an extremely large sequence depth in a snapshot, compressed ultrafast photography (CUP) has aroused tremendous attention in ultrafast optical imaging. However, the high compression ratio induced by large sequence depth brings the problem of low image quality in image reconstruction, preventing CUP from observing transient scenes with fine spatial information. To overcome these restrictions, we propose an efficient image reconstruction algorithm with multi-scale (MS) weighted denoising based on the plug-and-play (PnP) based alternating direction method of multipliers (ADMM) framework for multi-channel coupled CUP (MC-CUP), named the MCMS-PnP algorithm. By removing non-Gaussian distributed noise using weighted MS denoising during each iteration of the ADMM, and adaptively adjusting the weights via sufficiently exploiting the coupling information among different acquisition channels collected by MC-CUP, a synergistic combination of hardware and algorithm can be realized to significantly improve the quality of image reconstruction. Both simulation and experimental results demonstrate that the proposed adaptive MCMS-PnP algorithm can effectively improve the accuracy and quality of reconstructed images in MC-CUP, and extend the detectable range of CUP to transient scenes with fine structures.

FISI: frequency domain integration sequential imaging at 1.26×1013 frames per second and 108 lines per millimeter

High spatial resolution on the image plane (intrinsic spatial resolution) has always been a problem for ultrafast imaging. Single-shot ultrafast imaging methods can achieve high spatial resolution on the object plane through amplification systems but with low intrinsic spatial resolutions. We present frequency domain integration sequential imaging (FISI), which encodes a transient dynamic by an inversed 4f (IFF) system and decodes it using optical spatial frequencies recognition (OFR), which overcomes the limitation of the spatial frequencies recognition algorithm. In an experiment on the process of an air plasma channel, FISI achieved shadow imaging of the channel with a framing rate of 1.26×10¹³ fps and an intrinsic spatial resolution of 108 lp/mm (the spatial resolution on the image plane). Owing to its excellent framing time and high intrinsic spatial resolution, FISI can probe both repeatable and unrepeatable ultrafast phenomena, such as laser-induced damage, plasma physics, and shockwave interactions in living cells with high quality.

Ultrafast Time-Resolved Pump-Probe Investigation of Nanosecond Extreme Ultraviolet-Light-Induced Damage Dynamics on B4C/Ru Nano-Bilayer Film

An ultrafast time-resolved pump-probe setup with both high temporal and spatial resolution is developed to investigate the transient interaction between a nanosecond extreme ultraviolet (EUV) pulse and matter. By using a delayed femtosecond probe pulse, the pattern evolution of surface modification induced by an EUV pump at a wavelength of 13.5 nm can be imaged at different delay times, which provides deep insight into the EUV-induced damage dynamics and damage mechanisms. As a demonstration, single-shot EUV damage on a B₄C(6.0 nm)/Ru(30.4 nm)/D263 nano-bilayer optical film is studied using this pump-probe method. A recoverable phenomenon is found during the evolution process of the dome-shaped damage region. This is explained by the elastic and plastic deformations resulting from the huge compressive stress difference at the Ru-substrate interface with the help of simulations on the thermal effects and mechanical responses. This damage mechanism is further proven by the complementary experiments at a higher EUV fluence at 13.5 nm.

Time-overlapping structured-light projection: high performance on 3D shape measurement for complex dynamic scenes

High-speed three-dimensional (3D) shape measurement has been continuously researched due to the demand for analyzing dynamic behavior in transient scenes. In this work, a time-overlapping structured-light 3D shape measuring technique is proposed to realize high-speed and high-performance measurement on complex dynamic scenes. Time-overlapping structured-light projection is presented to maximumly reduce the information redundancy in temporal sequences and improve the measuring efficiency; generalized tripartite phase unwrapping (Tri-PU) is used to ensure the measuring robustness; fringe period extension is achieved by improving overlapping rate to further double the encoding fringe periods for higher measuring accuracy. Based on the proposed measuring technique, one new pixel-to-pixel and unambiguous 3D reconstruction result can be updated with three newly required patterns at a reconstruction rate of 3174 fps. Three transient scenes including collapsing wood blocks struck by a flying arrow, free-falling foam snowflakes and flying water balloon towards metal grids were measured to verify the high performance of the proposed method in various complex dynamic scenes.

Attention-based neural network for polarimetric image denoising

In this Letter, we propose an attention-based neural network specially designed for the challenging task of polarimetric image denoising. In particular, the channel attention mechanism is used to effectively extract the features underlying the polarimetric images by rescaling the contributions of channels in the network. In addition, we also design the adaptive polarization loss to make the network focus on the polarization information. Experiments show that our method can well restore the details flooded by serious noise and outperforms previous methods. Moreover, the underlying mechanism of channel attention is revealed visually.

Multi-scale band-limited illumination profilometry for robust three-dimensional surface imaging at video rate

Dynamic three-dimensional (3D) surface imaging by phase-shifting fringe projection profilometry has been widely implemented in diverse applications. However, existing techniques fall short in simultaneously providing the robustness in solving spatially isolated 3D objects, the tolerance of large variation in surface reflectance, and the flexibility of tunable working distances with meter-square-level fields of view (FOVs) at video rate. In this work, we overcome these limitations by developing multi-scale band-limited illumination profilometry (MS-BLIP). Supported by the synergy of dual-level intensity projection, multi-frequency fringe projection, and an iterative method for distortion compensation, MS-BLIP can accurately discern spatially separated 3D objects with highly varying reflectance. MS-BLIP is demonstrated by dynamic 3D imaging of a translating engineered box and a rotating vase. With an FOV of up to 1.7 m × 1.1 m and a working distance of up to 2.8 m, MS-BLIP is applied to capturing full human-body movements at video rate.

Single-shot imaging with multiple frames through delaying optical images

A single-shot imaging system with multiple frames has been developed, which can record sequential multiple frames by delaying multiple optical images with fiber bundles and then capturing the images with a single intensified camera. The observed optical object is imaged through four lenses onto the end faces of four sets of fiber bundles. These fiber bundles with different lengths can provide different delays for delivering optical images, which determine the inter-frame separation times. The optical images exported from the fiber bundles are captured with a single intensified CMOS camera simultaneously. This imaging system has been applied for investigating the dynamic x-ray spot of the rod-pinch diode via a combination of scintillators, which are used to convert x-ray images to optical images. Four sequential x-ray images in a single shot have been obtained, which show the dynamic development of the rod-pinch x-ray spot. The results experimentally reveal the dynamics of the electrons flow bombarding the rod, which roughly agrees with the theoretical modeling of the rod-pinch diode.

Lensless ultrafast optical imaging

Lensless single-shot ultrafast optical imaging is realized by integrating an acousto-optic programmable dispersive filter with spectrally filtered sequentially time all-optical mapping photography, which enables independent control of frame rate, frame intensity, and exposure time with a simple system design.

Acousto-optically driven lensless single-shot ultrafast optical imaging

Driven by many applications in a wide span of scientific fields, a myriad of advanced ultrafast imaging techniques have emerged in the last decade, featuring record-high imaging speeds above a trillion-frame-per-second with long sequence depths. Although bringing remarkable insights into various ultrafast phenomena, their application out of a laboratory environment is however limited in most cases, either by the cost, complexity of the operation or by heavy data processing. We then report a versatile single-shot imaging technique combining sequentially timed all-optical mapping photography (STAMP) with acousto-optics programmable dispersive filtering (AOPDF) and digital in-line holography (DIH). On the one hand, a high degree of simplicity is reached through the AOPDF, which enables full control over the acquisition parameters via an electrically driven phase and amplitude spectro-temporal tailoring of the imaging pulses. Here, contrary to most single-shot techniques, the frame rate, exposure time, and frame intensities can be independently adjusted in a wide range of pulse durations and chirp values without resorting to complex shaping stages, making the system remarkably agile and user-friendly. On the other hand, the use of DIH, which does not require any reference beam, allows to achieve an even higher technical simplicity by allowing its lensless operation but also for reconstructing the object on a wide depth of field, contrary to classical techniques that only provide images in a single plane. The imaging speed of the system as well as its flexibility are demonstrated by visualizing ultrashort events on both the picosecond and nanosecond timescales. The virtues and limitations as well as the potential improvements of this on-demand ultrafast imaging method are critically discussed.

Motion picture of magnified light pulse propagation with extending recordable time of digital light-in-flight holography

We experimentally demonstrate a motion picture imaging technique that can record a magnified image of light pulse propagation with extending the recordable time of digital light-in-flight recording by holography. We constructed an optical system that achieves a recordable time extension and an observation of a magnified image of light pulse propagation. As a result, we experimentally succeeded in recording light pulse propagation with a 7.45 magnification rate with extending the recordable time. The recordable time of the motion picture was 714 fs, which is twice that of the conventional one.

Flatfield ultrafast imaging with single-shot non-synchronous array photography

We present a method for acquiring a sequence of time-resolved images in a single shot, called single-shot non-synchronous array photography (SNAP). In SNAP, a pulsed laser beam is split by a diffractive optical element into an array of angled beamlets whose illumination fronts remain perpendicular to the optical axis. Different time delays are imparted to each beamlet by an echelon, enabling them to probe ultrafast dynamics in rapid succession. The beamlets are imaged onto different regions of a camera by a lenslet array. Because the illumination fronts remain flat (head-on) independently of beamlet angle, the exposure time in SNAP is fundamentally limited only by the laser pulse duration, akin to a "global shutter" in conventional imaging. We demonstrate SNAP by capturing the evolution of a laser induced plasma filament over 20 frames at an average rate of 4.2 trillion frames per second (Tfps) and a peak rate of 5.7 Tfps.

FFT-based simulation of the hologram-recording process for light-in-flight recording by holography

We propose a numerical simulation method of the hologram-recording process for light-in-flight recording by holography (LIF holography) based on fast Fourier transform (FFT) to improve the efficiency of the simulation. Because it is crucial to consider the difference in the optical-path length between the object and reference light pulses, we modify a point-spread function by considering the optical-path lengths of the object and reference light pulses and whether both pulses interfere with each other in LIF holography. The computational time was shortened by 5.5×10⁵ times for the 4,096×4,096 resolution of the hologram using the proposed method. We evaluate the proposed method by calculating the root mean square error (RMSE) of the reconstructed holographic images. The RMSEs were relatively small considering the effect of speckle noise; these results effectively demonstrate the validity of the proposed method. Moreover, we reconstruct the moving pictures of light pulse propagation from the hologram generated by the proposed method. We compare the simulation and experimental results, and succeed in qualitatively demonstrating the validity of the proposed method.

Continuously streaming compressed high-speed photography using time delay integration

An imaging system capable of acquiring high-resolution data at a high speed is in demand. However, the amount of optical information captured by a modern camera is limited by the data transfer bandwidth of electronics, resulting in a reduced spatial and temporal resolution. To overcome this problem, we developed continuously streaming compressed high-speed photography, which can record a dynamic scene with an unprecedented space-bandwidth-time product. By performing compressed imaging in a time-delay-integration manner, we continuously recorded a 0.85 megapixel video at 200 kHz, corresponding to an information flux of 170 gigapixels per second.

Spatiotemporal observation of light propagation in a three-dimensional scattering medium

Spatiotemporal information about light pulse propagation obtained with femtosecond temporal resolution plays an important role in understanding transient phenomena and light–matter interactions. Although ultrafast optical imaging techniques have been developed, it is still difficult to capture light pulse propagation spatiotemporally. Furthermore, imaging through a three-dimensional (3-D) scattering medium is a longstanding challenge due to the optical scattering caused by the interaction between light pulse and a 3-D scattering medium. Here, we propose a technique for ultrafast optical imaging of light pulses propagating inside a 3D scattering medium. We record an image of the light pulse propagation using the ultrashort light pulse even when the interaction between light pulse and a 3-D scattering medium causes the optical scattering. We demonstrated our proposed technique by recording converging, refracted, and diffracted propagating light for 59 ps with femtosecond temporal resolution.

Single-shot imaging of ultrafast all-optical magnetization dynamics with a spatiotemporal resolution

We present a laboratory system for single-shot magneto-optical (MO) imaging of ultrafast magnetization dynamics with less than 8 fs temporal, micrometer spatial resolutions and a MO Faraday's rotation sensitivity of 4 mdeg/μm. We create a stack of MO images repeatedly employing a single pair of pump and defocused probe pulses to induce and visualize MO changes in the sample. Both laser beams are independently wavelength-tunable, allowing for a flexible, resonant adjustable two-color pump and probe scheme. To increase the MO contrast, the probe beam is spatially filtered through a 50 μm aperture. We performed the all-optical switching experiment in Co-doped yttrium iron garnet films (YIG:Co) to demonstrate the capability of the presented method. We determine the spatiotemporal distribution of the effective field of photo-induced anisotropy, driving the all-optical switching of the magnetization in the YIG:Co film without an external magnetic field. Moreover, using this imaging method, we tracked the process of the laser-induced magnetization precession.

Temporal visualization of femtosecond laser pulses with single-edge transport in turbid media via Monte Carlo simulation

We study the propagation of femtosecond laser pulses with a single (front or rear) edge or dual edge through turbid media via Monte Carlo simulation. The results show that both the transmitted pulses spread on the basis of the incident pulse width ${t_{p}} = {{100}}\;{\rm{fs}}$, arising from the scattering effect. Further, the broadening width of the incident laser with a dual-edge pulse is wider than that of the incident laser width a single-edge pulse. The effect of the scattering particles on the front edge and the rear edge of the femtosecond laser can be distinguished in the time domain for femtosecond laser pulses through turbid media with the optical depth (OD) less than 10. In this scattering regime, the front-edge pulse scattered by the particles contributes more to diffused photons, but the effect of the scattering particles on the front edge and the rear edge of the femtosecond laser cannot be discriminated in turbid media with the OD more than 10, where the scattering is dominated by multiple scattering.

Stray light characterization with ultrafast time-of-flight imaging

Understanding stray light (SL) is a crucial aspect in the development of high-end optical instruments, for instance space telescopes. As it drives image quality, SL must be controlled by design and characterized experimentally. However, conventional SL characterization methods are limited as they do not provide information on its origins. The problem is complex due to the diversity of light interaction processes with surfaces, creating various SL contributors. Therefore, when SL level is higher than expected, it can be difficult to determine how to improve the system. We demonstrate a new approach, ultrafast time-of-flight SL characterization, where a pulsed laser source and a streak camera are used to record individually SL contributors which travel with a specific optical path length. Furthermore, the optical path length offers a means of identification to determine its origin. We demonstrate this method in an imaging system, measuring and identifying individual ghosts and scattering components. We then show how it can be used to reverse-engineer the instrument SL origins.

Compressed ultrafast tomographic imaging by passive spatiotemporal projections

Existing streak-camera-based two-dimensional (2D) ultrafast imaging techniques are limited by long acquisition time, the trade-off between spatial and temporal resolutions, and a reduced field of view. They also require additional components, customization, or active illumination. Here we develop compressed ultrafast tomographic imaging (CUTI), which passively records 2D transient events with a standard streak camera. By grafting the concept of computed tomography to the spatiotemporal domain, the operations of temporal shearing and spatiotemporal integration in a streak camera's data acquisition can be equivalently expressed as the spatiotemporal projection of an (x,y,t) datacube from a certain angle. Aided by a new, to the best of our knowledge, compressed-sensing reconstruction algorithm, the 2D transient event can be accurately recovered in a few measurements. CUTI is exhibited as a new imaging mode universally adaptable to most streak cameras. Implemented in an image-converter streak camera, CUTI captures the sequential arrival of two spatially modulated ultrashort ultraviolet laser pulses at 0.5 trillion frames per second. Applied to a rotating-mirror streak camera, CUTI records an amination of fast-bouncing balls at 5,000 frames per second.

Jitter-free 40-fs 375-keV electron pulses directly accelerated by an intense laser beam and their application to direct observation of laser pulse propagation in a vacuum

We report the generation of ultrashort bright electron pulses directly driven by irradiating a solid target with intense femtosecond laser pulses. The duration of electron pulses after compression by a phase rotator composed of permanent magnets was measured as 89 fs via the ponderomotive scattering of electron and laser pulses, which were almost at the compression limit due to the dispersion of the electron optics. The electron pulse compression system consisting of permanent magnets enabled extremely high timing stability between the laser pulse and electron pulse. The long-term RMS arrival time drift was below 14 fs in 4 h, which was limited by the resolution of the current setup. Because there was no time-varying field to generate jitter, the timing jitter was essentially reduced to zero. To demonstrate the capability of the ultrafast electron pulses, we used them to directly visualize laser pulse propagation in a vacuum and perform 2D mapping of the electric fields generated by low-density plasma in real time.

All-optical modulation and detection using a gain medium in a pulse shaper

We demonstrate all-optical modulation and ultrafast detection using an on-resonance optical gain medium, combined with spectral splitting in a Fourier transform pulse shaper. Multiple spectral channels of one optical beam can be independently modulated in time by another beam, allowing high-rate modulation and multiplexing without requiring ultrafast response from the gain medium. For detection of sub-picosecond signals we demonstrate a method of ultrafast signal detection (temporal imaging with no spatial resolution) that utilizes the spatio-temporal tilt of an optical pulse in a pulse shaper. The proposed methods can find applications in optical information technology and ultrafast imaging.

Long sequence single-exposure videography using spatially modulated illumination

Frequency recognition algorithm for multiple exposures (FRAME) is a single-exposure imaging technique that can be used for ultrafast videography, achieved through rapid illumination with spatially modulated laser pulses. To date, both the limit in sequence length as well as the relation between sequence length and image quality are unknown for FRAME imaging. Investigating these questions requires a flexible optical arrangement that has the capability of reaching significantly longer image sequences than currently available solutions. In this paper we present a new type of FRAME setup that fulfills this criteria. The setup relies only on (i) a diffractive optical element, (ii) an imaging lens and (iii) a digital micromirror device to generate a modulated pulse train with sequence lengths ranging from 2 to 1024 image frames. To the best of the authors' knowledge, this is the highest number of temporally resolved frames imaged in a single-exposure.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon's time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.

Sequentially timed all-optical mapping photography boosted by a branched 4f system with a slicing mirror

We present sequentially timed all-optical mapping photography (STAMP) with a slicing mirror in a branched 4f system for an increased number of frames without sacrificing pixel resolution. The branched 4f system spectrally separates the laser light path into multiple paths by the slicing mirror placed in the Fourier plane. Fabricated by an ultra-precision end milling process, the slicing mirror has 18 mirror facets of differing mirror angles. We used the boosted STAMP to observe dynamics of laser ablation with two image sensors which captured 18 subsequent frames at a frame rate of 126 billion frames per second, demonstrating this technique's potential for imaging unexplored ultrafast non-repetitive phenomena.

100,000 frames-per-second compressive imaging with a conventional rolling-shutter camera by random point-spread-function engineering

We demonstrate an approach that allows taking videos at very high frame-rates of over 100,000 frames per second by exploiting the fast sampling rate of the standard rolling-shutter readout mechanism, common to most conventional sensors, and a compressive-sampling acquisition scheme. Our approach is directly applied to a conventional imaging system by the simple addition of a diffuser to the pupil plane that randomly encodes the entire field-of-view to each camera row, while maintaining diffraction-limited resolution. A short video is reconstructed from a single camera frame via a compressed-sensing reconstruction algorithm, exploiting the inherent sparsity of the imaged scene.

Beyond MHz image recordings using LEDs and the FRAME concept

Many important scientific questions in physics, chemistry and biology rely on high-speed optical imaging techniques for their investigations. These techniques are either passive, relying on the rapid readout of photoactive elements, or active, relying on the illumination properties of specially designed pulse trains. Currently, MHz imaging speeds are difficult to realize; passive methods, being dictated by electronics, cause the unification of high spatial resolution with high frame rates to be very challenging, while active methods rely on expensive and complex hardware such as femto- and picosecond laser sources. Here we present an accessible temporally resolved imaging system for shadowgraphy based on multiplexed LED illumination that is capable of producing four images at MHz frame rates. Furthermore as the LEDs are independent of each other, any light burst configuration can be obtained, allowing for instance the simultaneous determination of low- and high speed events in parallel. To the best of the authors' knowledge, this is the fastest high speed imaging system that does not rely on pulsed lasers or fast detectors, in this case reaching up to 4.56 MHz.

Single-shot ultrafast burst imaging using an integral field spectroscope with a microlens array

To fully utilize the functions of a center-wavelength-sweeping pulse train generated by a free-space angular-chirp-enhanced delay optical layout for a probe laser pulse in sequentially timed all-optical mapping photography (STAMP), we introduced an integral field spectroscopy (IFS) method using a microlens array (MLA) to produce hyperspectral images, referring to the technique as lens array (LA)-STAMP. Compared with the previous STAMP utilizing spectral filtering where a bandpass filter generated hyperspectral images, LA-STAMP achieved much higher optical throughput. In a prototype setup, we used a 60×60 MLA and demonstrated single-shot burst imaging of a femtosecond laser-induced ablation process on a glass surface with 300 ps frame intervals in a 1.8 ns time window. Each frame image was constructed by assembling spectrally dispersed 36×36 monochromatic segments distributed by each lenslet on 5×5 pixels of a CCD camera. The spatial resolution was ∼4.4µm, which was determined by the MLA's pitch and the magnification of the microscope lens. We limited the number of frames to seven in this prototype setup, although it can be scaled to ∼24 with a spatial resolution of ∼1µm by designing IFS with a fine pitch MLA.

Extension of time window into nanoseconds in single-shot ultrafast burst imaging by spectrally sweeping pulses

We achieved single-shot 2D-burst imaging with a ∼22ps temporal resolution in a nanosecond time window using sequentially timed all-optical mapping photography with a spectral filtering (SF-STAMP) scheme, where a single snapshot of spectral images measured with a linear frequency chirped laser pulse forms time-resolved snapshots. We combined a pulse-stretching scheme of a free-space angular-chirp-enhanced delay (FACED) composed of a pair of tilted mirrors and a 4f-system. With a 4f-FACED system, we generated collinearly propagating burst laser pulses with a different center wavelength and a tunable time interval and demonstrated single-shot burst imaging with a 303 ps interval in a 1.5 ns time window by an SF-STAMP with spectrally sweeping probe pulses.