Encrypted Three-dimensional Dynamic Imaging using Snapshot Time-of-flight Compressed Ultrafast Photography

Snapshot compressive imaging at 855 million frames per second for aluminium planar wire array Z-pinch

This paper present a novel, integrated compressed ultrafast photography system for comprehensive measurement of the aluminium planar wire array Z-Pinch evolution process. The system incorporates a large array streak camera and embedded encoding to improve the signal-to-noise ratio. Based on the "QiangGuang-I" pulsed power facility, we recorded the complete continuous 2D implosion process of planar wire array Z-Pinch for the first time. Our results contribute valuable understanding of imploding plasma instabilities and offer direction for the optimization of Z-Pinch facilities.

Introducing the Special Issue Honoring Lihong V. Wang, Pioneer in Biomedical Optics

The editorial concludes the JBO Special Issue Honoring Lihong V. Wang, outlining Prof. Wang's salient contributions to advancing the field of biomedical optics.

Tutorial on compressed ultrafast photography

Significance

Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges.

Aim

This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect.

Approach

A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement.

Conclusions

CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

Dual-channel compressed ultrafast photography for Z-pinch dynamic imaging

The compressed ultrafast photography (CUP) can capture non-repetitive time-evolving events at 7 × 10¹³ fps, which is anticipated to find a diverse range of applications in physics, biomedical imaging, and materials science. The feasibility of diagnosing ultrafast phenomenon of Z-pinch by using the CUP has been analyzed in this article. Specifically, a dual-channel CUP design has been adopted for acquiring high quality reconstructed images and the strategies of identical masks, uncorrelated masks, and complementary masks have been compared. Furthermore, the image of the first channel was rotated by 90° to balance the spatial resolution between the sweep direction and the non-sweep direction. Both five synthetic videos and two simulated Z-pinch videos were chosen as the ground truth to validate this approach. The average peak signal to noise ratio of the reconstruction results is 50.55 dB for the self-emission visible light video and 32.53 dB for the laser shadowgraph video with unrelated masks (rotated channel 1). The simulation results show that the time-space-evolving process of plasma distribution can be well retold, and the phenomenon of plasma instability can be accurately diagnosed by the dual-channel CUP with unrelated masks (rotated channel 1). This study may promote the practical applications of the CUP in the field of accelerator physics.

A Novel Reconstruction Algorithm with High Performance for Compressed Ultrafast Imaging

Compressed ultrafast photography (CUP) is a type of two-dimensional (2D) imaging technique to observe ultrafast processes. Intelligence reconstruction methods that influence the imaging quality are an essential part of a CUP system. However, existing reconstruction algorithms mostly rely on image priors and complex parameter spaces. Therefore, it usually takes a lot of time to obtain acceptable reconstruction results, which limits the practical application of the CUP. In this paper, we proposed a novel reconstruction algorithm named PnP-FFDNet, which can provide a high quality and high efficiency compared to previous methods. First, we built a forward model of the CUP and three sub-optimization problems were obtained using the alternating direction multiplier method (ADMM), and the closed-form solution of the first sub-optimization problem was derived. Secondly, inspired by the PnP-ADMM framework, we used an advanced denoising algorithm based on a neural network named FFDNet to solve the second sub-optimization problem. On the real CUP data, PSNR and SSIM are improved by an average of 3 dB and 0.06, respectively, compared with traditional algorithms. Both on the benchmark dataset and on the real CUP data, the proposed method reduces the running time by an average of about 96% over state-of-the-art algorithms, and show comparable visual results, but in a much shorter running time.

Video encryption/compression using compressive coded rotating mirror camera

Compressive coded rotating mirror (CCRM) camera is a novel high-speed imaging system that operates under amplitude optical encoding and frame sweeping modalities in a passive imaging mode that is capable of reconstructing 1400 frames from a single shot image acquisition and achieves the highest compression ratio of 368 compared to the other compressive sensing (CS) based single-shot imaging modalities. The integrated optical encoding and compression adds a strong layer of encryption on the observed data and facilitates the integration of the CCRM camera with the imaging applications that require highly efficient data encryption and compression due to capturing highly sensitive data or limited transmission and storage capacities. CCRM uses amplitude encoding that significantly extends the key space where the probability of having the exact encoder pattern is estimated as [Formula: see text], hence drastically reducing the possibility of data recovery in a brute force manner. Data reconstruction is achieved under CS based algorithms where the obtained amplitude-based pattern from optical encoder operates as the key in the recovery process. Reconstruction on the experimental as well as the synthetic data at various compression ratios demonstrate that the estimated key with less than 95[Formula: see text] matching elements were unable to recover the data where the achieved averaged structural similarity (SSIM) of 0.25 before 95[Formula: see text] encoder similarity and 0.85 SSIM at 100[Formula: see text] encoder similarity demonstrates the high-sensitivity of the proposed optical encryption technique.

Compressed ultrafast tomographic imaging by passive spatiotemporal projections

Existing streak-camera-based two-dimensional (2D) ultrafast imaging techniques are limited by long acquisition time, the trade-off between spatial and temporal resolutions, and a reduced field of view. They also require additional components, customization, or active illumination. Here we develop compressed ultrafast tomographic imaging (CUTI), which passively records 2D transient events with a standard streak camera. By grafting the concept of computed tomography to the spatiotemporal domain, the operations of temporal shearing and spatiotemporal integration in a streak camera's data acquisition can be equivalently expressed as the spatiotemporal projection of an (x,y,t) datacube from a certain angle. Aided by a new, to the best of our knowledge, compressed-sensing reconstruction algorithm, the 2D transient event can be accurately recovered in a few measurements. CUTI is exhibited as a new imaging mode universally adaptable to most streak cameras. Implemented in an image-converter streak camera, CUTI captures the sequential arrival of two spatially modulated ultrashort ultraviolet laser pulses at 0.5 trillion frames per second. Applied to a rotating-mirror streak camera, CUTI records an amination of fast-bouncing balls at 5,000 frames per second.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon's time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.

Deep-learning-based image reconstruction for compressed ultrafast photography

Compressed ultrafast photography (CUP) is a computational optical imaging technique that can capture transient dynamics at an unprecedented speed. Currently, the image reconstruction of CUP relies on iterative algorithms, which are time-consuming and often yield nonoptimal image quality. To solve this problem, we develop a deep-learning-based method for CUP reconstruction that substantially improves the image quality and reconstruction speed. A key innovation toward efficient deep learning reconstruction of a large three-dimensional (3D) event datacube (x,y,t) (x,y, spatial coordinate; t, time) is that we decompose the original datacube into massively parallel two-dimensional (2D) imaging subproblems, which are much simpler to solve by a deep neural network. We validated our approach on simulated and experimental data.

Extension of time window into nanoseconds in single-shot ultrafast burst imaging by spectrally sweeping pulses

We achieved single-shot 2D-burst imaging with a ∼22ps temporal resolution in a nanosecond time window using sequentially timed all-optical mapping photography with a spectral filtering (SF-STAMP) scheme, where a single snapshot of spectral images measured with a linear frequency chirped laser pulse forms time-resolved snapshots. We combined a pulse-stretching scheme of a free-space angular-chirp-enhanced delay (FACED) composed of a pair of tilted mirrors and a 4f-system. With a 4f-FACED system, we generated collinearly propagating burst laser pulses with a different center wavelength and a tunable time interval and demonstrated single-shot burst imaging with a 303 ps interval in a 1.5 ns time window by an SF-STAMP with spectrally sweeping probe pulses.

Picosecond-resolution phase-sensitive imaging of transparent objects in a single shot

With the growing interest in the optical imaging of ultrafast phenomena in transparent objects, from shock wave to neuronal action potentials, high contrast imaging at high frame rates has become desirable. While phase sensitivity provides the contrast, the frame rates and sequence depths are highly limited by the detectors. Here, we present phase-sensitive compressed ultrafast photography (pCUP) for single-shot real-time ultrafast imaging of transparent objects by combining the contrast of dark-field imaging with the speed and the sequence depth of CUP. By imaging the optical Kerr effect and shock wave propagation, we demonstrate that pCUP can image light-speed phase signals in a single shot with up to 350 frames captured at up to 1 trillion frames per second. We expect pCUP to be broadly used for a vast range of fundamental and applied sciences.

Single-shot ultrafast optical imaging

Single-shot ultrafast optical imaging can capture two-dimensional transient scenes in the optical spectral range at ≥100 million frames per second. This rapidly evolving field surpasses conventional pump-probe methods by possessing the real-time imaging capability, which is indispensable for recording non-repeatable and difficult-to-reproduce events and for understanding physical, chemical, and biological mechanisms. In this mini-review, we survey comprehensively the state-of-the-art single-shot ultrafast optical imaging. Based on the illumination requirement, we categorized the field into active-detection and passive-detection domains. Depending on the specific image acquisition and reconstruction strategies, these two categories are further divided into a total of six sub-categories. Under each sub-category, we describe operating principles, present representative cutting-edge techniques with a particular emphasis on their methodology and applications, and discuss their advantages and challenges. Finally, we envision prospects of technical advancement in this field.

Single-shot real-time femtosecond imaging of temporal focusing

While the concept of focusing usually applies to the spatial domain, it is equally applicable to the time domain. Real-time imaging of temporal focusing of single ultrashort laser pulses is of great significance in exploring the physics of the space-time duality and finding diverse applications. The drastic changes in the width and intensity of an ultrashort laser pulse during temporal focusing impose a requirement for femtosecond-level exposure to capture the instantaneous light patterns generated in this exquisite phenomenon. Thus far, established ultrafast imaging techniques either struggle to reach the desired exposure time or require repeatable measurements. We have developed single-shot 10-trillion-frame-per-second compressed ultrafast photography (T-CUP), which passively captures dynamic events with 100-fs frame intervals in a single camera exposure. The synergy between compressed sensing and the Radon transformation empowers T-CUP to significantly reduce the number of projections needed for reconstructing a high-quality three-dimensional spatiotemporal datacube. As the only currently available real-time, passive imaging modality with a femtosecond exposure time, T-CUP was used to record the first-ever movie of non-repeatable temporal focusing of a single ultrashort laser pulse in a dynamic scattering medium. T-CUP's unprecedented ability to clearly reveal the complex evolution in the shape, intensity, and width of a temporally focused pulse in a single measurement paves the way for single-shot characterization of ultrashort pulses, experimental investigation of nonlinear light-matter interactions, and real-time wavefront engineering for deep-tissue light focusing.

Single-shot chemical detection and identification with compressed hyperspectral Raman imaging

Raman imaging is a powerful method to identify and detect chemicals, but the long acquisition time required for full spectroscopic Raman images limits many practical applications. Compressive sensing and compressed ultrafast photography have recently demonstrated the acquisition of multi-dimensional data sets with single-shot detection. In this Letter, we demonstrate the utilization of compressed sensing for single-shot compressed Raman imaging. In particular, we use this technique to demonstrate the identification of two similarly white substances in one image via the recovered two-dimensional array of Raman spectra. This technique can be further extended by coupling the compressed sensing apparatus with a microscope for compressed hyperspectral imaging microscopy.

Volumetric Light-field Encryption at the Microscopic Scale

We report a light-field based method that allows the optical encryption of three-dimensional (3D) volumetric information at the microscopic scale in a single 2D light-field image. The system consists of a microlens array and an array of random phase/amplitude masks. The method utilizes a wave optics model to account for the dominant diffraction effect at this new scale, and the system point-spread function (PSF) serves as the key for encryption and decryption. We successfully developed and demonstrated a deconvolution algorithm to retrieve both spatially multiplexed discrete data and continuous volumetric data from 2D light-field images. Showing that the method is practical for data transmission and storage, we obtained a faithful reconstruction of the 3D volumetric information from a digital copy of the encrypted light-field image. The method represents a new level of optical encryption, paving the way for broad industrial and biomedical applications in processing and securing 3D data at the microscopic scale.

Single-shot real-time video recording of a photonic Mach cone induced by a scattered light pulse

Ultrafast video recording of spatiotemporal light distribution in a scattering medium has a significant impact in biomedicine. Although many simulation tools have been implemented to model light propagation in scattering media, existing experimental instruments still lack sufficient imaging speed to record transient light-scattering events in real time. We report single-shot ultrafast video recording of a light-induced photonic Mach cone propagating in an engineered scattering plate assembly. This dynamic light-scattering event was captured in a single camera exposure by lossless-encoding compressed ultrafast photography at 100 billion frames per second. Our experimental results are in excellent agreement with theoretical predictions by time-resolved Monte Carlo simulation. This technology holds great promise for next-generation biomedical imaging instrumentation.

Glare suppression by coherence gated negation

Imaging of a weak target hidden behind a scattering medium can be significantly confounded by glare. We report a method, termed coherence gated negation (CGN), that uses destructive optical interference to suppress glare and allow improved imaging of a weak target. As a demonstration, we show that by permuting through a set range of amplitude and phase values for a reference beam interfering with the optical field from the glare and target reflection, we can suppress glare by an order of magnitude, even when the optical wavefront is highly disordered. This strategy significantly departs from conventional coherence gating methods in that CGN actively "gates out" the unwanted optical contributions while conventional methods "gate in" the target optical signal. We further show that the CGN method can outperform conventional coherence gating image quality in certain scenarios by more effectively rejecting unwanted optical contributions.

All Photons Imaging Through Volumetric Scattering

Imaging through thick highly scattering media (sample thickness ≫ mean free path) can realize broad applications in biomedical and industrial imaging as well as remote sensing. Here we propose a computational "All Photons Imaging" (API) framework that utilizes time-resolved measurement for imaging through thick volumetric scattering by using both early arrived (non-scattered) and diffused photons. As opposed to other methods which aim to lock on specific photons (coherent, ballistic, acoustically modulated, etc.), this framework aims to use all of the optical signal. Compared to conventional early photon measurements for imaging through a 15 mm tissue phantom, our method shows a two fold improvement in spatial resolution (4db increase in Peak SNR). This all optical, calibration-free framework enables widefield imaging through thick turbid media, and opens new avenues in non-invasive testing, analysis, and diagnosis.

Space- and intensity-constrained reconstruction for compressed ultrafast photography

The single-shot compressed ultrafast photography (CUP) camera is the fastest receive-only camera in the world. In this Letter, we introduce an external CCD camera and a space- and intensity-constrained (SIC) reconstruction algorithm to improve the image quality of CUP. The external CCD camera takes a time-unsheared image of the dynamic scene. Unlike the previously used unconstrained algorithm, the proposed algorithm incorporates both spatial and intensity constraints based on the additional prior information provided by the external CCD camera. First, a spatial mask is extracted from the time-unsheared image to define the zone of action. Next, an intensity threshold is determined based on the similarity between the temporally projected image of the reconstructed datacube and the time-unsheared image. Both simulation and experimental studies show that the SIC reconstruction improves the spatial resolution, contrast, and general quality of the reconstructed image.

Computational multi-depth single-photon imaging

We present an imaging framework that is able to accurately reconstruct multiple depths at individual pixels from single-photon observations. Our active imaging method models the single-photon detection statistics from multiple reflectors within a pixel, and it also exploits the fact that a multi-depth profile at each pixel can be expressed as a sparse signal. We interpret the multi-depth reconstruction problem as a sparse deconvolution problem using single-photon observations, create a convex problem through discretization and relaxation, and use a modified iterative shrinkage-thresholding algorithm to efficiently solve for the optimal multi-depth solution. We experimentally demonstrate that the proposed framework is able to accurately reconstruct the depth features of an object that is behind a partially-reflecting scatterer and 4 m away from the imager with root mean-square error of 11 cm, using only 19 signal photon detections per pixel in the presence of moderate background light. In terms of root mean-square error, this is a factor of 4.2 improvement over the conventional method of Gaussian-mixture fitting for multi-depth recovery.