Continuously streaming compressed high-speed photography using time delay integration

Tutorial on compressed ultrafast photography

Significance

Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges.

Aim

This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect.

Approach

A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement.

Conclusions

CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

TDI-based continuous window compressed spatio-temporal imaging capable of flexible voxels post-interpretation

To achieve high frame rates and continuous streaming simultaneously, we propose a compressed spatio-temporal imaging framework implemented by combining time-delay-integration sensors and coded exposure. Without additional optical coding elements and subsequent calibration required, this electronic-domain modulation enables a more compact and robust hardware structure, compared to the existing imaging modalities. By exploiting the intra-line charge transfer mechanism, we achieve a super-resolution in both temporal and spatial domains, thus multiplying the frame rate to millions of frames-per-second. In addition, the forward model with post-tunable coefficients, and two reconstruction strategies proposed therefrom, facilitate a flexible voxels post-interpretation. Finally, the effectiveness of the proposed framework is demonstrated by both numerical simulations and proof-of-concept experiments. With the prominent advantages of prolonged time window and flexible voxels post-interpretation, the proposed system will be suitable for imaging random, non-repetitive, or long-term events.

Augmented light field tomography through parallel spectral encoding

Snapshot recording of transient dynamics in three dimensions (3-D) is highly demanded in both fundamental and applied sciences. Yet it remains challenging for conventional high-speed cameras to address this need due to limited electronic bandwidth and reliance on mechanical scanning. The emergence of light field tomography (LIFT) provides a new solution to these long-standing problems and enables 3-D imaging at an unprecedented frame rate. However, based on sparse-view computed tomography, LIFT can accommodate only a limited number of projections, degrading the resolution in the reconstructed image. To alleviate this problem, we herein present a spectral encoding scheme to significantly increase the number of allowable projections in LIFT while maintaining its snapshot advantage. The resultant system can record 3-D dynamics at a kilohertz volumetric frame rate. Moreover, by using a multichannel compressed sensing algorithm, we improve the image quality with an enhanced spatial resolution and suppressed aliasing artifacts.

Simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography

We present a simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography. In this system, we propose to combine a specially designed mini-reflector delay-line array with digital in-line holography. The former including a group of adjustable mini-reflectors can easily generate an array of probe sub-pulses that can be controlled independently in the propagation direction and time delays. The object beams formed by the different sub-pulses will propagate and fall on different recording regions of the image sensor to generate a single-shot spatial-multiplexing in-line hologram. The geometry of the digital in-line holography can simplify the complexity of the system and enable complex amplitude imaging. In addition, the time resolution of this system is limited only by the pulse duration, which allows this system to study the dynamic processes with the femtosecond order. In an experiment about the laser-induced air plasma, our proposed system achieves nine frames sequential holographic images with the frame rate of 7.5 trillion frames per second (Tfps).