Single-shot ultrafast imaging attaining 70 trillion frames per second

Single-Shot Intensity- and Phase-Sensitive Compressive Sensing-Based Coherent Modulation Ultrafast Imaging

Ultrafast imaging can capture the dynamic scenes with a nanosecond and even femtosecond temporal resolution. Complementarily, phase imaging can provide the morphology, refractive index, or thickness information that intensity imaging cannot represent. Therefore, it is important to realize the simultaneous ultrafast intensity and phase imaging for achieving as much information as possible in the detection of ultrafast dynamic scenes. Here, we report a single-shot intensity- and phase-sensitive compressive sensing-based coherent modulation ultrafast imaging technique, shortened as CS-CMUI, which integrates coherent modulation imaging, compressive imaging, and streak imaging. We theoretically demonstrate through numerical simulations that CS-CMUI can obtain both the intensity and phase information of the dynamic scenes with ultrahigh fidelity. Furthermore, we experimentally build a CS-CMUI system and successfully measure the intensity and phase evolution of a multimode Q-switched laser pulse and the dynamical behavior of laser ablation on an indium tin oxide thin film. It is anticipated that CS-CMUI enables a profound comprehension of ultrafast phenomena and promotes the advancement of various practical applications, which will have substantial impact on fundamental and applied sciences.

Unsupervised reconstruction with a registered time-unsheared image constraint for compressed ultrafast photography

Compressed ultrafast photography (CUP) is a computational imaging technology capable of capturing transient scenes in picosecond scale with a sequence depth of hundreds of frames. Since the inverse problem of CUP is an ill-posed problem, it is challenging to further improve the reconstruction quality under the condition of high noise level and compression ratio. In addition, there are many articles adding an external charge-coupled device (CCD) camera to the CUP system to form the time-unsheared view because the added constraint can improve the reconstruction quality of images. However, since the images are collected by different cameras, slight affine transformation may have great impacts on the reconstruction quality. Here, we propose an algorithm that combines the time-unsheared image constraint CUP system with unsupervised neural networks. Image registration network is also introduced into the network framework to learn the affine transformation parameters of input images. The proposed algorithm effectively utilizes the implicit image prior in the neural network as well as the extra hardware prior information brought by the time-unsheared view. Combined with image registration network, this joint learning model enables our proposed algorithm to further improve the quality of reconstructed images without training datasets. The simulation and experiment results demonstrate the application prospect of our algorithm in ultrafast event capture.

Short-wave Infrared Photoluminescence Lifetime Mapping of Rare-Earth Doped Nanoparticles Using All-Optical Streak Imaging

The short-wave infrared (SWIR) photoluminescence lifetimes of rare-earth doped nanoparticles (RENPs) have found diverse applications in fundamental and applied research. Despite dazzling progress in the novel design and synthesis of RENPs with attractive optical properties, existing optical systems for SWIR photoluminescence lifetime imaging are still considerably restricted by inefficient photon detection, limited imaging speed, and low sensitivity. To overcome these challenges, SWIR photoluminescence lifetime imaging microscopy using an all-optical streak camera (PLIMASC) is developed. Synergizing scanning optics and a high-sensitivity InGaAs CMOS camera, SWIR-PLIMASC has a 1D imaging speed of up to 138.9 kHz in the spectral range of 900-1700 nm, which quantifies the photoluminescence lifetime of RENPs in a single shot. A 2D photoluminescence lifetime map can be acquired by 1D scanning of the sample. To showcase the power of SWIR-PLIMASC, a series of core-shell RENPs with distinct SWIR photoluminescence lifetimes is synthesized. In particular, using Er3+ -doped RENPs, SWIR-PLIMASC enables multiplexed anti-counterfeiting. Leveraging Ho3+ -doped RENPs as temperature indicators, this system is applied to SWIR photoluminescence lifetime-based thermometry. Opening up a new avenue for efficient SWIR photoluminescence lifetime mapping, this work is envisaged to contribute to advanced materials characterization, information science, and biomedicine.

Swept coded aperture real-time femtophotography

Single-shot real-time femtophotography is indispensable for imaging ultrafast dynamics during their times of occurrence. Despite their advantages over conventional multi-shot approaches, existing techniques confront restricted imaging speed or degraded data quality by the deployed optoelectronic devices and face challenges in the application scope and acquisition accuracy. They are also hindered by the limitations in the acquirable information imposed by the sensing models. Here, we overcome these challenges by developing swept coded aperture real-time femtophotography (SCARF). This computational imaging modality enables all-optical ultrafast sweeping of a static coded aperture during the recording of an ultrafast event, bringing full-sequence encoding of up to 156.3 THz to every pixel on a CCD camera. We demonstrate SCARF's single-shot ultrafast imaging ability at tunable frame rates and spatial scales in both reflection and transmission modes. Using SCARF, we image ultrafast absorption in a semiconductor and ultrafast demagnetization of a metal alloy.

Tutorial on compressed ultrafast photography

Significance

Compressed ultrafast photography (CUP) is currently the world's fastest single-shot imaging technique. Through the integration of compressed sensing and streak imaging, CUP can capture a transient event in a single camera exposure with imaging speeds from thousands to trillions of frames per second, at micrometer-level spatial resolutions, and in broad sensing spectral ranges.

Aim

This tutorial aims to provide a comprehensive review of CUP in its fundamental methods, system implementations, biomedical applications, and prospect.

Approach

A step-by-step guideline to CUP's forward model and representative image reconstruction algorithms is presented with sample codes and illustrations in Matlab and Python. Then, CUP's hardware implementation is described with a focus on the representative techniques, advantages, and limitations of the three key components-the spatial encoder, the temporal shearing unit, and the two-dimensional sensor. Furthermore, four representative biomedical applications enabled by CUP are discussed, followed by the prospect of CUP's technical advancement.

Conclusions

CUP has emerged as a state-of-the-art ultrafast imaging technology. Its advanced imaging ability and versatility contribute to unprecedented observations and new applications in biomedicine. CUP holds great promise in improving technical specifications and facilitating the investigation of biomedical processes.

Introducing the Special Issue Honoring Lihong V. Wang, Pioneer in Biomedical Optics

The editorial concludes the JBO Special Issue Honoring Lihong V. Wang, outlining Prof. Wang's salient contributions to advancing the field of biomedical optics.

Flexible and accurate total variation and cascaded denoisers-based image reconstruction algorithm for hyperspectrally compressed ultrafast photography

Hyperspectrally compressed ultrafast photography (HCUP) based on compressed sensing and time- and spectrum-to-space mappings can simultaneously realize the temporal and spectral imaging of non-repeatable or difficult-to-repeat transient events with a passive manner in single exposure. HCUP possesses an incredibly high frame rate of tens of trillions of frames per second and a sequence depth of several hundred, and therefore plays a revolutionary role in single-shot ultrafast optical imaging. However, due to ultra-high data compression ratios induced by the extremely large sequence depth, as well as limited fidelities of traditional algorithms over the image reconstruction process, HCUP suffers from a poor image reconstruction quality and fails to capture fine structures in complex transient scenes. To overcome these restrictions, we report a flexible image reconstruction algorithm based on a total variation (TV) and cascaded denoisers (CD) for HCUP, named the TV-CD algorithm. The TV-CD algorithm applies the TV denoising model cascaded with several advanced deep learning-based denoising models in the iterative plug-and-play alternating direction method of multipliers framework, which not only preserves the image smoothness with TV, but also obtains more priori with CD. Therefore, it solves the common sparsity representation problem in local similarity and motion compensation. Both the simulation and experimental results show that the proposed TV-CD algorithm can effectively improve the image reconstruction accuracy and quality of HCUP, and may further promote the practical applications of HCUP in capturing high-dimensional complex physical, chemical and biological ultrafast dynamic scenes.

Temporal resolution of ultrafast compressive imaging using a single-chirped optical probe

Ultrafast compressive imaging captures three-dimensional spatiotemporal information of transient events in a single shot. When a single-chirped optical probe is applied, the temporal information is obtained from the probe modulated in amplitude or phase using a direct frequency-time mapping method. Here, we extend the analysis of the temporal resolution of conventional one-dimensional ultrafast measurement techniques such as spectral interferometry to that in three-dimensional ultrafast compressive imaging. In this way, both the amplitude and phase of the probe are necessary for a full Fourier transform method, which obtains temporal information with an improved resolution determined by probe spectral bandwidth. The improved temporal resolution potentially enables ultrafast compressive imaging with an effective imaging speed at the quadrillion-frames-per-second level.

Single-Shot Reconfigurable Femtosecond Imaging of Ultrafast Optical Dynamics

Understanding ultrafast dynamics in the femtosecond timescale plays a pivotal role in fundamental research and technology innovation. Spatiotemporal observation of those events in real-time requires imaging speeds greater than 10¹² frames per second (fps), far beyond the fundamental speed limits of the ubiquitous semiconductor sensor technologies. In addition, a majority of femtosecond events are non-repeatable or difficult-to-repeat since they either work in a highly unstable nonlinear regime or require extreme or rare conditions to initiate. Therefore, the traditional pump-probe imaging approach fails since it heavily depends on precise event repetition. Single-shot ultrafast imaging emerges as the only solution; however, existing techniques cannot reach more than 15×10¹² fps, and they only record an insufficient number of frames. Compressed ultrafast spectral photography (CUSP) is proposed to overcome these limitations. Here, CUSP's full design space is explored by manipulating the ultrashort optical pulse in the active illumination. Via parameter optimization, an extraordinarily fast frame rate of 219×10¹² fps is achieved. This implementation of CUSP is also highly flexible, allowing various combinations of imaging speeds and numbers of frames (several hundred up to 1000) to be readily deployed in diverse scientific studies, such as laser-induced transient birefringence, self-focusing, and filaments in dielectric media.

Dual-channel compressed ultrafast photography for Z-pinch dynamic imaging

The compressed ultrafast photography (CUP) can capture non-repetitive time-evolving events at 7 × 10¹³ fps, which is anticipated to find a diverse range of applications in physics, biomedical imaging, and materials science. The feasibility of diagnosing ultrafast phenomenon of Z-pinch by using the CUP has been analyzed in this article. Specifically, a dual-channel CUP design has been adopted for acquiring high quality reconstructed images and the strategies of identical masks, uncorrelated masks, and complementary masks have been compared. Furthermore, the image of the first channel was rotated by 90° to balance the spatial resolution between the sweep direction and the non-sweep direction. Both five synthetic videos and two simulated Z-pinch videos were chosen as the ground truth to validate this approach. The average peak signal to noise ratio of the reconstruction results is 50.55 dB for the self-emission visible light video and 32.53 dB for the laser shadowgraph video with unrelated masks (rotated channel 1). The simulation results show that the time-space-evolving process of plasma distribution can be well retold, and the phenomenon of plasma instability can be accurately diagnosed by the dual-channel CUP with unrelated masks (rotated channel 1). This study may promote the practical applications of the CUP in the field of accelerator physics.

Single-pulse real-time billion-frames-per-second planar imaging of ultrafast nanoparticle-laser dynamics and temperature in flames

Unburnt hydrocarbon flames produce soot, which is the second biggest contributor to global warming and harmful to human health. The state-of-the-art high-speed imaging techniques, developed to study non-repeatable turbulent flames, are limited to million-frames-per-second imaging rates, falling short in capturing the dynamics of critical species. Unfortunately, these techniques do not provide a complete picture of flame-laser interactions, important for understanding soot formation. Furthermore, thermal effects induced by multiple consecutive pulses modify the optical properties of soot nanoparticles, thus making single-pulse imaging essential. Here, we report single-shot laser-sheet compressed ultrafast photography (LS-CUP) for billion-frames-per-second planar imaging of flame-laser dynamics. We observed laser-induced incandescence, elastic light scattering, and fluorescence of soot precursors - polycyclic aromatic hydrocarbons (PAHs) in real-time using a single nanosecond laser pulse. The spatiotemporal maps of the PAHs emission, soot temperature, primary nanoparticle size, soot aggregate size, and the number of monomers, present strong experimental evidence in support of the theory and modeling of soot inception and growth mechanism in flames. LS-CUP represents a generic and indispensable tool that combines a portfolio of ultrafast combustion diagnostic techniques, covering the entire lifecycle of soot nanoparticles, for probing extremely short-lived (picoseconds to nanoseconds) species in the spatiotemporal domain in non-repeatable turbulent environments. Finally, LS-CUP's unparalleled capability of ultrafast wide-field temperature imaging in real-time is envisioned to unravel mysteries in modern physics such as hot plasma, sonoluminescence, and nuclear fusion.

Augmented light field tomography through parallel spectral encoding

Snapshot recording of transient dynamics in three dimensions (3-D) is highly demanded in both fundamental and applied sciences. Yet it remains challenging for conventional high-speed cameras to address this need due to limited electronic bandwidth and reliance on mechanical scanning. The emergence of light field tomography (LIFT) provides a new solution to these long-standing problems and enables 3-D imaging at an unprecedented frame rate. However, based on sparse-view computed tomography, LIFT can accommodate only a limited number of projections, degrading the resolution in the reconstructed image. To alleviate this problem, we herein present a spectral encoding scheme to significantly increase the number of allowable projections in LIFT while maintaining its snapshot advantage. The resultant system can record 3-D dynamics at a kilohertz volumetric frame rate. Moreover, by using a multichannel compressed sensing algorithm, we improve the image quality with an enhanced spatial resolution and suppressed aliasing artifacts.

Simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography

We present a simple system for realizing single-shot ultrafast sequential imaging based on spatial multiplexing in-line holography. In this system, we propose to combine a specially designed mini-reflector delay-line array with digital in-line holography. The former including a group of adjustable mini-reflectors can easily generate an array of probe sub-pulses that can be controlled independently in the propagation direction and time delays. The object beams formed by the different sub-pulses will propagate and fall on different recording regions of the image sensor to generate a single-shot spatial-multiplexing in-line hologram. The geometry of the digital in-line holography can simplify the complexity of the system and enable complex amplitude imaging. In addition, the time resolution of this system is limited only by the pulse duration, which allows this system to study the dynamic processes with the femtosecond order. In an experiment about the laser-induced air plasma, our proposed system achieves nine frames sequential holographic images with the frame rate of 7.5 trillion frames per second (Tfps).

Ultrafast and hypersensitive phase imaging of propagating internodal current flows in myelinated axons and electromagnetic pulses in dielectrics

Many ultrafast phenomena in biology and physics are fundamental to our scientific understanding but have not yet been visualized owing to the extreme speed and sensitivity requirements in imaging modalities. Two examples are the propagation of passive current flows through myelinated axons and electromagnetic pulses through dielectrics, which are both key to information processing in living organisms and electronic devices. Here, we demonstrate differentially enhanced compressed ultrafast photography (Diff-CUP) to directly visualize propagations of passive current flows at approximately 100 m/s along internodes, i.e., continuous myelinated axons between nodes of Ranvier, from Xenopus laevis sciatic nerves and of electromagnetic pulses at approximately 5 × 107 m/s through lithium niobate. The spatiotemporal dynamics of both propagation processes are consistent with the results from computational models, demonstrating that Diff-CUP can span these two extreme timescales while maintaining high phase sensitivity. With its ultrahigh speed (picosecond resolution), high sensitivity, and noninvasiveness, Diff-CUP provides a powerful tool for investigating ultrafast biological and physical phenomena.

Single-shot compressed optical field topography

Femtosecond lasers are powerful in studying matter's ultrafast dynamics within femtosecond to attosecond time scales. Drawing a three-dimensional (3D) topological map of the optical field of a femtosecond laser pulse including its spatiotemporal amplitude and phase distributions, allows one to predict and understand the underlying physics of light interaction with matter, whose spatially resolved transient dielectric function experiences ultrafast evolution. However, such a task is technically challenging for two reasons: first, one has to capture in single-shot and squeeze the 3D information of an optical field profile into a two-dimensional (2D) detector; second, typical detectors are only sensitive to intensity or amplitude information rather than phase. Here we have demonstrated compressed optical field topography (COFT) drawing a 3D map for an ultrafast optical field in single-shot, by combining the coded aperture snapshot spectral imaging (CASSI) technique with a global 3D phase retrieval procedure. COFT can, in single-shot, fully characterize the spatiotemporal coupling of a femtosecond laser pulse, and live stream the light-speed propagation of an air plasma ionization front, unveiling its potential applications in ultrafast sciences.

FISI: frequency domain integration sequential imaging at 1.26×1013 frames per second and 108 lines per millimeter

High spatial resolution on the image plane (intrinsic spatial resolution) has always been a problem for ultrafast imaging. Single-shot ultrafast imaging methods can achieve high spatial resolution on the object plane through amplification systems but with low intrinsic spatial resolutions. We present frequency domain integration sequential imaging (FISI), which encodes a transient dynamic by an inversed 4f (IFF) system and decodes it using optical spatial frequencies recognition (OFR), which overcomes the limitation of the spatial frequencies recognition algorithm. In an experiment on the process of an air plasma channel, FISI achieved shadow imaging of the channel with a framing rate of 1.26×10¹³ fps and an intrinsic spatial resolution of 108 lp/mm (the spatial resolution on the image plane). Owing to its excellent framing time and high intrinsic spatial resolution, FISI can probe both repeatable and unrepeatable ultrafast phenomena, such as laser-induced damage, plasma physics, and shockwave interactions in living cells with high quality.

Lensless ultrafast optical imaging

Lensless single-shot ultrafast optical imaging is realized by integrating an acousto-optic programmable dispersive filter with spectrally filtered sequentially time all-optical mapping photography, which enables independent control of frame rate, frame intensity, and exposure time with a simple system design.

Acousto-optically driven lensless single-shot ultrafast optical imaging

Driven by many applications in a wide span of scientific fields, a myriad of advanced ultrafast imaging techniques have emerged in the last decade, featuring record-high imaging speeds above a trillion-frame-per-second with long sequence depths. Although bringing remarkable insights into various ultrafast phenomena, their application out of a laboratory environment is however limited in most cases, either by the cost, complexity of the operation or by heavy data processing. We then report a versatile single-shot imaging technique combining sequentially timed all-optical mapping photography (STAMP) with acousto-optics programmable dispersive filtering (AOPDF) and digital in-line holography (DIH). On the one hand, a high degree of simplicity is reached through the AOPDF, which enables full control over the acquisition parameters via an electrically driven phase and amplitude spectro-temporal tailoring of the imaging pulses. Here, contrary to most single-shot techniques, the frame rate, exposure time, and frame intensities can be independently adjusted in a wide range of pulse durations and chirp values without resorting to complex shaping stages, making the system remarkably agile and user-friendly. On the other hand, the use of DIH, which does not require any reference beam, allows to achieve an even higher technical simplicity by allowing its lensless operation but also for reconstructing the object on a wide depth of field, contrary to classical techniques that only provide images in a single plane. The imaging speed of the system as well as its flexibility are demonstrated by visualizing ultrashort events on both the picosecond and nanosecond timescales. The virtues and limitations as well as the potential improvements of this on-demand ultrafast imaging method are critically discussed.

All-fiber high-speed image detection enabled by deep learning

Ultra-high-speed imaging serves as a foundation for modern science. While in biomedicine, optical-fiber-based endoscopy is often required for in vivo applications, the combination of high speed with the fiber endoscopy, which is vital for exploring transient biomedical phenomena, still confronts some challenges. We propose all-fiber imaging at high speeds, which is achieved based on the transformation of two-dimensional spatial information into one-dimensional temporal pulsed streams by leveraging high intermodal dispersion in a multimode fiber. Neural networks are trained to reconstruct images from the temporal waveforms. It can not only detect content-aware images with high quality, but also detect images of different kinds from the training images with slightly reduced quality. The fiber probe can detect micron-scale objects with a high frame rate (15.4 Mfps) and large frame depth (10,000). This scheme combines high speeds with high mechanical flexibility and integration and may stimulate future research exploring various phenomena in vivo.

Single-shot compressive hyperspectral imaging with dispersed and undispersed light using a generally available grating

Commercially available hyperspectral cameras are useful for remote sensing, but in most cases snapshot imaging is difficult due to the need for scanning. The coded aperture snapshot spectral imager (CASSI) has been proposed to simultaneously acquire a target scene's spatial and spectral dimensional data, employing a refractive prism as a disperser. This paper proposes a CASSI-based technique using a generally available diffraction grating of a Ronchi ruling and blazed grating and its improvement using the undispersed zeroth-order light. The feasibility and performance of the proposed technique are experimentally validated, and the grating parameters are identified.

Flatfield ultrafast imaging with single-shot non-synchronous array photography

We present a method for acquiring a sequence of time-resolved images in a single shot, called single-shot non-synchronous array photography (SNAP). In SNAP, a pulsed laser beam is split by a diffractive optical element into an array of angled beamlets whose illumination fronts remain perpendicular to the optical axis. Different time delays are imparted to each beamlet by an echelon, enabling them to probe ultrafast dynamics in rapid succession. The beamlets are imaged onto different regions of a camera by a lenslet array. Because the illumination fronts remain flat (head-on) independently of beamlet angle, the exposure time in SNAP is fundamentally limited only by the laser pulse duration, akin to a "global shutter" in conventional imaging. We demonstrate SNAP by capturing the evolution of a laser induced plasma filament over 20 frames at an average rate of 4.2 trillion frames per second (Tfps) and a peak rate of 5.7 Tfps.

Compressed sensing in fluorescence microscopy

Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.

Continuously streaming compressed high-speed photography using time delay integration

An imaging system capable of acquiring high-resolution data at a high speed is in demand. However, the amount of optical information captured by a modern camera is limited by the data transfer bandwidth of electronics, resulting in a reduced spatial and temporal resolution. To overcome this problem, we developed continuously streaming compressed high-speed photography, which can record a dynamic scene with an unprecedented space-bandwidth-time product. By performing compressed imaging in a time-delay-integration manner, we continuously recorded a 0.85 megapixel video at 200 kHz, corresponding to an information flux of 170 gigapixels per second.

Fast wide-field upconversion luminescence lifetime thermometry enabled by single-shot compressed ultrahigh-speed imaging

Photoluminescence lifetime imaging of upconverting nanoparticles is increasingly featured in recent progress in optical thermometry. Despite remarkable advances in photoluminescent temperature indicators, existing optical instruments lack the ability of wide-field photoluminescence lifetime imaging in real time, thus falling short in dynamic temperature mapping. Here, we report video-rate upconversion temperature sensing in wide field using single-shot photoluminescence lifetime imaging thermometry (SPLIT). Developed from a compressed-sensing ultrahigh-speed imaging paradigm, SPLIT first records wide-field luminescence intensity decay compressively in two views in a single exposure. Then, an algorithm, built upon the plug-and-play alternating direction method of multipliers, is used to reconstruct the video, from which the extracted lifetime distribution is converted to a temperature map. Using the core/shell NaGdF4:Er3+,Yb3+/NaGdF4 upconverting nanoparticles as the lifetime-based temperature indicators, we apply SPLIT in longitudinal wide-field temperature monitoring beneath a thin scattering medium. SPLIT also enables video-rate temperature mapping of a moving biological sample at single-cell resolution.

Two-dimensional time- and space-resolved diagnostic method for an integrated implosion process

To precisely measure and evaluate X-ray generation and evolution in a hohlraum during an implosion process, we present a two-dimensional (2D) time- and space-resolved diagnostic method by combining a compressed ultrafast photography (CUP) system and a simplified version of space-resolving flux detector (SSRFD). Numerical experiment results showed that the reconstruction quality of the conventional CUP significantly improved owing to the addition of the external SSRFD, especially when a coded mask with a large pixel size was used in the CUP. Further, the performance of the CUP cooperation with the SSRFD was better than that of adding an external charge-coupled device or streak camera. Compared with existing ultrafast imaging techniques in laser fusion, the proposed method has a prominent advantage of measuring the 2D evolution of implosion by combining high temporal resolution of streak camera and high spatial resolution of SSRFD; moreover, it can provide guidance for designing diagnostic experiments in laser fusion research.

Sparsity-Based Recovery of Three-Dimensional Photoacoustic Images from Compressed Single-Shot Optical Detection

Photoacoustic (PA) imaging combines optical excitation with ultrasonic detection to achieve high-resolution imaging of biological samples. A high-energy pulsed laser is often used for imaging at multi-centimeter depths in tissue. These lasers typically have a low pulse repetition rate, so to acquire images in real-time, only one pulse of the laser can be used per image. This single pulse necessitates the use of many individual detectors and receive electronics to adequately record the resulting acoustic waves and form an image. Such requirements make many PA imaging systems both costly and complex. This investigation proposes and models a method of volumetric PA imaging using a state-of-the-art compressed sensing approach to achieve real-time acquisition of the initial pressure distribution (IPD) at a reduced level of cost and complexity. In particular, a single exposure of an optical image sensor is used to capture an entire Fabry–Pérot interferometric acoustic sensor. Time resolved encoding as achieved through spatial sweeping with a galvanometer. This optical system further makes use of a random binary mask to set a predetermined subset of pixels to zero, thus enabling recovery of the time-resolved signals. The Two-Step Iterative Shrinking and Thresholding algorithm is used to reconstruct the IPD, harnessing the sparsity naturally occurring in the IPD as well as the additional structure provided by the binary mask. We conduct experiments on simulated data and analyze the performance of our new approach.

Nanosecond resolution photography system for laser-induced cavitation based on PIV dual-head laser and industrial camera

The detailed study of the initial and collapse processes of the laser-induced cavitation requires nanosecond resolution (both nanoseconds exposure and nanoseconds interframe time) of the photography measurement system. The high-speed video cameras are difficult to achieve nanoseconds interval time. The framing and streak cameras are able to reach the nanosecond resolution, but their complex technology and expensive prices make them far from being commercially available. The present study builds a nanosecond resolution photography system based on PIV dual-head laser and conventional industrial camera. The exposure time of the photography system is controlled by the laser pulse width, which is 5 ns. The two heads of the PIV laser are operated independently thus the smallest time interval between two laser pulses can be set to less than 10 ns. A double-pulse per-exposure imaging technique is used to record the information from two laser pulses on single frame on a low-speed industrial camera. The nanosecond resolution photography system was applied to the laser-induced cavitation experiments to verify the reliability of the measurement results. The measurement of the shock wave velocity demonstrates the ability of the system to capture ultrafast phenomena, which reduces from 3611 m/s to approximately 1483 m/s within 400 ns. The experimental results also reveal the asymmetric evolution of laser-induced cavitation bubbles. The major axis of the ellipsoidal bubble has twice reversals along the laser propagation and perpendicular direction from the laser-induced breakdown to the first collapse.

Giga-voxel multidimensional fluorescence imaging combining single-pixel detection and data fusion

Time-resolved fluorescence imaging is a key tool in biomedical applications, as it allows to non-invasively obtain functional and structural information. However, the big amount of collected data introduces challenges in both acquisition speed and processing needs. Here, we introduce a novel technique that allows to acquire a giga-voxel 4D hypercube in a fast manner while measuring only 0.03% of the dataset. The system combines two single-pixel cameras and a conventional 2D array detector working in parallel. Data fusion techniques are introduced to combine the individual 2D and 3D projections acquired by each sensor in the final high-resolution 4D hypercube, which can be used to identify different fluorophore species by their spectral and temporal signatures.

Thresholded single-photon underwater imaging and detection

Optical underwater target imaging and detection have been a tough but significant challenge in deep-sea exploration. Distant reflected signals drown in various underwater noises due to strong absorption and scattering, resulting in degraded image contrast and reduced detection range. Single-photon feature operating at the fundamental limit of the classical electromagnetic waves can broaden the realm of quantum technologies. Here we experimentally demonstrate a thresholded single-photon imaging and detection scheme to extract photon signals from the noisy underwater environment. We reconstruct the images obtained in a high-loss underwater environment by using photon-limited computational algorithms. Furthermore, we achieve a capability of underwater detection down to 0.8 photons per pulse at Jerlov type III water up to 50 meters, which is equivalent to more than 9 attenuation lengths. The results break the limits of classical underwater imaging and detection and may lead to many quantum-enhanced applications, like air-to-sea target tracking and deep-sea optical exploration.

All-optical high spatial-temporal resolution photography with raster principle at 2 trillion frames per second

A novel single-shot ultrafast all-optical photography with raster principle (OPR) that can capture real-time imaging of ultrafast phenomena is proposed and demonstrated. It consists of a sequentially timed module (STM), spectral-shaping module (SSM), and raster framing camera (RFC). STM and SSM are used for linearly encoding frequency-time mapping and system calibration, respectively. The function of the RFC is sampling the target by microlens arrays and framing on the basis of frequency-time-spatial positions conversion. We demonstrated the recording of transient scenes with the spatial resolution of ∼90lp/mm, the frame number of 12 and the frame rate of 2 trillion frames per second (Tfps) in single-shot. Thanks to its high spatial-temporal resolution, high frame rate (maximum up to 10 Tfps or more) and sufficient frame number, our OPR can observe the dynamic processes with complex spatial structure at the atomic time scale (10 fs∼1ps), which is promising for application in plasma physics, shock waves in laser-induced damage, and dynamics of condensed matter materials.

In-situ diagnostic of femtosecond laser probe pulses for high resolution ultrafast imaging

Ultrafast imaging is essential in physics and chemistry to investigate the femtosecond dynamics of nonuniform samples or of phenomena with strong spatial variations. It relies on observing the phenomena induced by an ultrashort laser pump pulse using an ultrashort probe pulse at a later time. Recent years have seen the emergence of very successful ultrafast imaging techniques of single non-reproducible events with extremely high frame rate, based on wavelength or spatial frequency encoding. However, further progress in ultrafast imaging towards high spatial resolution is hampered by the lack of characterization of weak probe beams. For pump-probe experiments realized within solids or liquids, because of the difference in group velocities between pump and probe, the determination of the absolute pump-probe delay depends on the sample position. In addition, pulse-front tilt is a widespread issue, unacceptable for ultrafast imaging, but which is conventionally very difficult to evaluate for the low-intensity probe pulses. Here we show that a pump-induced micro-grating generated from the electronic Kerr effect provides a detailed in-situ characterization of a weak probe pulse. It allows solving the two issues of absolute pump-probe delay determination and pulse-front tilt detection. Our approach is valid whatever the transparent medium with non-negligible Kerr index, whatever the probe pulse polarization and wavelength. Because it is nondestructive and fast to perform, this in-situ probe diagnostic can be repeated to calibrate experimental conditions, particularly in the case where complex wavelength, spatial frequency or polarization encoding is used. We anticipate that this technique will enable previously inaccessible spatiotemporal imaging in a number of fields of ultrafast science at the micro- and nanoscale.

Stray light characterization with ultrafast time-of-flight imaging

Understanding stray light (SL) is a crucial aspect in the development of high-end optical instruments, for instance space telescopes. As it drives image quality, SL must be controlled by design and characterized experimentally. However, conventional SL characterization methods are limited as they do not provide information on its origins. The problem is complex due to the diversity of light interaction processes with surfaces, creating various SL contributors. Therefore, when SL level is higher than expected, it can be difficult to determine how to improve the system. We demonstrate a new approach, ultrafast time-of-flight SL characterization, where a pulsed laser source and a streak camera are used to record individually SL contributors which travel with a specific optical path length. Furthermore, the optical path length offers a means of identification to determine its origin. We demonstrate this method in an imaging system, measuring and identifying individual ghosts and scattering components. We then show how it can be used to reverse-engineer the instrument SL origins.

Dispersed pulses created by aperiodic binary spectral phase jump and applications for pulse shaping

Inspired by pulse-pair generation with periodic phase jump, the generation of dispersed pulses with aperiodic binary spectral phase jump (ABSPJ) is proposed and theoretically investigated. It is presented by the numerical simulations that two dispersed pulses can be generated by ABSPJ of π. The dispersion of one pulse is opposite to the other and can be tuned freely with engineering of the phase jump. The generated dispersed pulse-pair is potentially of great interest for various applications, such as two-dimensional spectroscopy, double pulses laser-wakefield acceleration (LWFA) and chirp management in dual-chirped optical parametric amplification (DC-OPA) system to generate TW single-cycle mid-infrared (MIR) pulses. Furthermore, a pulse shaper configured as a micro-electro-mechanical systems (MEMS) located at the Fourier plane of a 4-f dispersion-free compressor is suggested and the implementation in a high repetition optical parametric chirped pulse amplification (OPCPA) system with picosecond pump has been numerically studied. The simulations showed that MEMS of 900 pixels is enough to pre-compensate TOD of 200000 fs³ for a pulse of 20 fs. Because pixel with only two piston-levels is necessary for such MEMS, the pulse shaper is expected to be compact and reliable.

Highly efficient THz four-wave mixing in doped silicon

Third-order non-linearities are important because they allow control over light pulses in ubiquitous high-quality centro-symmetric materials like silicon and silica. Degenerate four-wave mixing provides a direct measure of the third-order non-linear sheet susceptibility χ(3)L (where L represents the material thickness) as well as technological possibilities such as optically gated detection and emission of photons. Using picosecond pulses from a free electron laser, we show that silicon doped with P or Bi has a value of χ(3)L in the THz domain that is higher than that reported for any other material in any wavelength band. The immediate implication of our results is the efficient generation of intense coherent THz light via upconversion (also a χ(3) process), and they open the door to exploitation of non-degenerate mixing and optical nonlinearities beyond the perturbative regime.

Theoretical investigations of a modified compressed ultrafast photography method suitable for single-shot fluorescence lifetime imaging

A single-shot fluorescence lifetime imaging (FLIM) method based on the compressed ultrafast photography (CUP) is proposed, named space-restricted CUP (srCUP). srCUP is suitable for imaging objects moving slowly (<∼150/Mmm/s, M is the magnification of the objective lens) in the field of view with the intensity changing within nanoseconds in a measurement window around 10 ns. We used synthetic datasets to explore the performances of srCUP compared with CUP and TCUP (a variant of CUP). srCUP not only provides superior reconstruction performances, but its reconstruction speed is also twofold and threefold faster than CUP and TCUP, respectively. The lifetime determination performances were assessed by estimating lifetime components, amplitude- and intensity-weighted average lifetimes (τ_A and τ_I), with the reconstructed scenes using the least squares method based on a bi-exponential model. srCUP has the best accuracy and precision for lifetime determinations with a relative bias less than 7% and a coefficient of variation less than 7% for τ_A, and a relative bias less than 10% and a coefficient of variation less than 11% for τ_I.

Compressive Coded Rotating Mirror Camera for High-Speed Imaging

We develop a novel compressive coded rotating mirror (CCRM) camera to capture events at high frame rates in passive mode with a compact instrument design at a fraction of the cost compared to other high-speed imaging cameras. Operation of the CCRM camera is based on amplitude optical encoding (grey scale) and a continuous frame sweep across a low-cost detector using a motorized rotating mirror system which can achieve single pixel shift between adjacent frames. Amplitude encoding and continuous frame overlapping enable the CCRM camera to achieve a high number of captured frames and high temporal resolution without making sacrifices in the spatial resolution. Two sets of dynamic scenes have been captured at up to a 120 Kfps frame rate in both monochrome and colored scales in the experimental demonstrations. The obtained heavily compressed data from the experiment are reconstructed using the optimization algorithm under the compressive sensing (CS) paradigm and the highest sequence depth of 1400 captured frames in a single exposure has been achieved with the highest compression ratio of 368 compared to other CS-based high-speed imaging technologies. Under similar conditions the CCRM camera is 700× faster than conventional rotating mirror based imaging devices and could reach a frame rate of up to 20 Gfps.

Fluorescent proteins for in vivo imaging, where's the biliverdin?

Noninvasive fluorescent imaging requires far-red and near-infrared fluorescent proteins for deeper imaging. Near-infrared light penetrates biological tissue with blood vessels due to low absorbance, scattering, and reflection of light and has a greater signal-to-noise due to less autofluorescence. Far-red and near-infrared fluorescent proteins absorb light >600 nm to expand the color palette for imaging multiple biosensors and noninvasive in vivo imaging. The ideal fluorescent proteins are bright, photobleach minimally, express well in the desired cells, do not oligomerize, and generate or incorporate exogenous fluorophores efficiently. Coral-derived red fluorescent proteins require oxygen for fluorophore formation and release two hydrogen peroxide molecules. New fluorescent proteins based on phytochrome and phycobiliproteins use biliverdin IXα as fluorophores, do not require oxygen for maturation to image anaerobic organisms and tumor core, and do not generate hydrogen peroxide. The small Ultra-Red Fluorescent Protein (smURFP) was evolved from a cyanobacterial phycobiliprotein to covalently attach biliverdin as an exogenous fluorophore. The small Ultra-Red Fluorescent Protein is biophysically as bright as the enhanced green fluorescent protein, is exceptionally photostable, used for biosensor development, and visible in living mice. Novel applications of smURFP include in vitro protein diagnostics with attomolar (10-18 M) sensitivity, encapsulation in viral particles, and fluorescent protein nanoparticles. However, the availability of biliverdin limits the fluorescence of biliverdin-attaching fluorescent proteins; hence, extra biliverdin is needed to enhance brightness. New methods for improved biliverdin bioavailability are necessary to develop improved bright far-red and near-infrared fluorescent proteins for noninvasive imaging in vivo.

Toward the Super Temporal Resolution Image Sensor with a Germanium Photodiode for Visible Light

The theoretical temporal resolution limit tT of a silicon photodiode (Si PD) is 11.1 ps. We call “super temporal resolution” the temporal resolution that is shorter than that limit. To achieve this resolution, Germanium is selected as a candidate material for the photodiode (Ge PD) for visible light since the absorption coefficient of Ge for the wavelength is several tens of times higher than that of Si, allowing a very thin PD. On the other hand, the saturation drift velocity of electrons in Ge is about 2/3 of that in Si. The ratio suggests an ultra-short propagation time of electrons in the Ge PD. However, the diffusion coefficient of electrons in Ge is four times higher than that of Si. Therefore, Monte Carlo simulations were applied to analyze the temporal resolution of the Ge PD. The estimated theoretical temporal resolution limit is 0.26 ps, while the practical limit is 1.41 ps. To achieve a super temporal resolution better than 11.1 ps, the driver circuit must operate at least 100 GHz. It is thus proposed to develop, at first, a short-wavelength infrared (SWIR) ultra-high-speed image sensor with a thicker and wider Ge PD, and then gradually decrease the size along with the progress of the driver circuits.

Snapshot spectral polarimetric light field imaging using a single detector

In this Letter, we investigate a snapshot spectral-polarimetric-volumetric imaging (SSPVI) system using a single detector. Through compressed acquisition and reconstruction, SSPVI can achieve spectral imaging (x,y,λ), polarization imaging (x,y,ψ,χ), and light field imaging (x,y,θ,φ) simultaneously. The newly discovered performance is showcased by attaining the spectral-polarimetric-volumetric video and different laboratory accuracy experiments. These never-seen-before capacities of the camera open new prospects for many applications, such as biological analysis, object recognition, and remote sensing.

Long sequence single-exposure videography using spatially modulated illumination

Frequency recognition algorithm for multiple exposures (FRAME) is a single-exposure imaging technique that can be used for ultrafast videography, achieved through rapid illumination with spatially modulated laser pulses. To date, both the limit in sequence length as well as the relation between sequence length and image quality are unknown for FRAME imaging. Investigating these questions requires a flexible optical arrangement that has the capability of reaching significantly longer image sequences than currently available solutions. In this paper we present a new type of FRAME setup that fulfills this criteria. The setup relies only on (i) a diffractive optical element, (ii) an imaging lens and (iii) a digital micromirror device to generate a modulated pulse train with sequence lengths ranging from 2 to 1024 image frames. To the best of the authors' knowledge, this is the highest number of temporally resolved frames imaged in a single-exposure.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon's time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.

Beyond MHz image recordings using LEDs and the FRAME concept

Many important scientific questions in physics, chemistry and biology rely on high-speed optical imaging techniques for their investigations. These techniques are either passive, relying on the rapid readout of photoactive elements, or active, relying on the illumination properties of specially designed pulse trains. Currently, MHz imaging speeds are difficult to realize; passive methods, being dictated by electronics, cause the unification of high spatial resolution with high frame rates to be very challenging, while active methods rely on expensive and complex hardware such as femto- and picosecond laser sources. Here we present an accessible temporally resolved imaging system for shadowgraphy based on multiplexed LED illumination that is capable of producing four images at MHz frame rates. Furthermore as the LEDs are independent of each other, any light burst configuration can be obtained, allowing for instance the simultaneous determination of low- and high speed events in parallel. To the best of the authors' knowledge, this is the fastest high speed imaging system that does not rely on pulsed lasers or fast detectors, in this case reaching up to 4.56 MHz.

Single-shot ultrafast burst imaging using an integral field spectroscope with a microlens array

To fully utilize the functions of a center-wavelength-sweeping pulse train generated by a free-space angular-chirp-enhanced delay optical layout for a probe laser pulse in sequentially timed all-optical mapping photography (STAMP), we introduced an integral field spectroscopy (IFS) method using a microlens array (MLA) to produce hyperspectral images, referring to the technique as lens array (LA)-STAMP. Compared with the previous STAMP utilizing spectral filtering where a bandpass filter generated hyperspectral images, LA-STAMP achieved much higher optical throughput. In a prototype setup, we used a 60×60 MLA and demonstrated single-shot burst imaging of a femtosecond laser-induced ablation process on a glass surface with 300 ps frame intervals in a 1.8 ns time window. Each frame image was constructed by assembling spectrally dispersed 36×36 monochromatic segments distributed by each lenslet on 5×5 pixels of a CCD camera. The spatial resolution was ∼4.4µm, which was determined by the MLA's pitch and the magnification of the microscope lens. We limited the number of frames to seven in this prototype setup, although it can be scaled to ∼24 with a spatial resolution of ∼1µm by designing IFS with a fine pitch MLA.