A review of snapshot multidimensional optical imaging: measuring photon tags in parallel

HyperReconNet: Joint Coded Aperture Optimization and Image Reconstruction for Compressive Hyperspectral Imaging

Coded aperture snapshot spectral imaging (CASSI) system encodes the 3D hyperspectral image (HSI) within a single 2D compressive image and then reconstructs the underlying HSI by employing an inverse optimization algorithm, which equips with the distinct advantage of snapshot but usually results in low reconstruction accuracy. To improve the accuracy, existing methods attempt to design either alternative coded apertures or advanced reconstruction methods, but cannot connect these two aspects via a unified framework, which limits the accuracy improvement. In this paper, we propose a convolution neural network (CNN) based endto- end method to boost the accuracy by jointly optimizing the coded aperture and the reconstruction method. On the one hand, based on the nature of CASSI forward model, we design a repeated pattern for the coded aperture, whose entities are learned by acting as the network weights. On the other hand, we conduct the reconstruction through simultaneously exploiting intrinsic properties within HSI - the extensive correlations across the spatial and the spectral dimensions. By leveraging the power of deep learning, the coded aperture design and the image reconstruction are connected and optimized via a unified framework. Experimental results show that our method outperforms the state-of-the-art methods under both comprehensive quantitative metrics and perceptive quality.

Complete plenoptic imaging using a single detector

Multi-dimensional imaging is a powerful technique for many applications, such as biological analysis, remote sensing, and object recognition. Most existing multi-dimensional imaging systems rely on scanning or camera array, which make the system bulky and unstable. To some extent, these problems can be mitigated by employing compressed sensing algorithms. However, they are computationally expensive and highly rely on the ill-posed assumption that the information is sparse in a given domain. Here, we propose a snapshot spectral-volumetric imaging (SSVI) system by introducing the paradigm of light-field imaging into Fourier transform imaging spectroscopy. We demonstrate that SSVI can reconstruct a complete plenoptic function, P(x,y,z,θ,φ,λ,t), of the incoming light rays using a single detector. Compared with other multidimensional imagers, SSVI features prominent advantages in compactness, robustness, and low cost.

Single-shot ultrafast optical imaging

Single-shot ultrafast optical imaging can capture two-dimensional transient scenes in the optical spectral range at ≥100 million frames per second. This rapidly evolving field surpasses conventional pump-probe methods by possessing the real-time imaging capability, which is indispensable for recording non-repeatable and difficult-to-reproduce events and for understanding physical, chemical, and biological mechanisms. In this mini-review, we survey comprehensively the state-of-the-art single-shot ultrafast optical imaging. Based on the illumination requirement, we categorized the field into active-detection and passive-detection domains. Depending on the specific image acquisition and reconstruction strategies, these two categories are further divided into a total of six sub-categories. Under each sub-category, we describe operating principles, present representative cutting-edge techniques with a particular emphasis on their methodology and applications, and discuss their advantages and challenges. Finally, we envision prospects of technical advancement in this field.

A Review of Ghost Imaging via Sparsity Constraints

Different from conventional imaging methods, which are based on the first-order field correlation, ghost imaging (GI) obtains the image information through high-order mutual-correlation of light fields from two paths with an object appearing in only one path. As a new optical imaging technology, GI not only provides us new capabilities beyond the conventional imaging methods, but also gives out a new viewpoint of imaging physical mechanism. It may be applied to many potential applications, such as remote sensing, snap-shot spectral imaging, thermal X-ray diffraction imaging and imaging through scattering media. In this paper, we reviewed mainly our research work of ghost imaging via sparsity constraints (GISC) and discussed the application and theory prospect of GISC concisely.

Hyperspectral ghost imaging camera based on a flat-field grating

A spectral camera based on ghost imaging via sparsity constraints (GISC) acquires a three-dimensional (3D) spatial-spectral data cube of the target through a two-dimensional (2D) detector in a single snapshot. However, the spectral and spatial resolution are interrelated because both of them are modulated by the same spatial random phase modulator. In this paper, we theoretically and experimentally demonstrate a system by equipping the GISC spectral camera with a flat-field grating to disperse the light fields before the spatial random phase modulator, hence consequently decoupling the spatial and spectral resolution. By theoretical derivation of the imaging process we obtain the spectral resolution 1nm and spatial resolution 50μm about the new system which are verified by the experiment. The new system can not only modulate the spatial and spectral resolution separately, but also provide a possibility of optimizing the light field fluctuations of different wavelengths according to the imaging scene.

Bio-inspired imager improves sensitivity in near-infrared fluorescence image-guided surgery

Image-guided surgery can enhance cancer treatment by decreasing, and ideally eliminating, positive tumor margins and iatrogenic damage to healthy tissue. Current state-of-the-art near-infrared fluorescence imaging systems are bulky and costly, lack sensitivity under surgical illumination, and lack co-registration accuracy between multimodal images. As a result, an overwhelming majority of physicians still rely on their unaided eyes and palpation as the primary sensing modalities for distinguishing cancerous from healthy tissue. Here we introduce an innovative design, comprising an artificial multispectral sensor inspired by the Morpho butterfly's compound eye, which can significantly improve image-guided surgery. By monolithically integrating spectral tapetal filters with photodetectors, we have realized a single-chip multispectral imager with 1000 × higher sensitivity and 7 × better spatial co-registration accuracy compared to clinical imaging systems in current use. Preclinical and clinical data demonstrate that this technology seamlessly integrates into the surgical workflow while providing surgeons with real-time information on the location of cancerous tissue and sentinel lymph nodes. Due to its low manufacturing cost, our bio-inspired sensor will provide resource-limited hospitals with much-needed technology to enable more accurate value-based health care.

Multiplexed Optical Imaging of Tumor-Directed Nanoparticles: A Review of Imaging Systems and Approaches

In recent decades, various classes of nanoparticles have been developed for optical imaging of cancers. Many of these nanoparticles are designed to specifically target tumor sites, and specific cancer biomarkers, to facilitate the visualization of tumors. However, one challenge for accurate detection of tumors is that the molecular profiles of most cancers vary greatly between patients as well as spatially and temporally within a single tumor mass. To overcome this challenge, certain nanoparticles and imaging systems have been developed to enable multiplexed imaging of large panels of cancer biomarkers. Multiplexed molecular imaging can potentially enable sensitive tumor detection, precise delineation of tumors during interventional procedures, and the prediction/monitoring of therapy response. In this review, we summarize recent advances in systems that have been developed for the imaging of optical nanoparticles that can be heavily multiplexed, which include surface-enhanced Raman-scattering nanoparticles (SERS NPs) and quantum dots (QDs). In addition to surveying the optical properties of these various types of nanoparticles, and the most-popular multiplexed imaging approaches that have been employed, representative preclinical and clinical imaging studies are also highlighted.

Angle-tolerant linear variable color filter based on a tapered etalon

We propose and fabricate a linear variable color filter (LVCF) that possesses an enhanced angular tolerance in conjunction with a wide linear filtering range (LFR) by taking advantage of an Ag-TiO₂-Ag configuration. The TiO₂ cavity is tapered in thickness along the device so that the resonance wavelength can be continuously tuned according to the position. In addition, the metal-dielectric-metal structure is overlaid with a pre-designed graded anti-reflection coating in SiO₂ to complete the etalon, thereby maximizing the transmission efficiency across the entire device. The tapered dielectric layers in the proposed filter were fabricated via glancing angle deposition without the help of any mask or moving parts. The center wavelength was scanned from 410 nm to 566 nm, resulting in an LFR of 156 nm, and the overall spectra exhibited an approximate peak transmission of 40% and spectral bandwidth of 68 nm. The angular tolerance was as large as 45°, incurring a fractional wavelength shift below 4.2%. The resonance wavelength was verified to be linearly dependent on the position, providing a linearity beyond 99%. The proposed LVCF will thus be actively utilized in a portable micro-spectrometer and spectral scanning device.

High spatial sampling light-guide snapshot spectrometer

A prototype fiber-based imaging spectrometer was developed to provide snapshot hyperspectral imaging tuned for biomedical applications. The system is designed for imaging in the visible spectral range from 400 to 700 nm for compatibility with molecular imaging applications as well as satellite and remote sensing. An 81 × 96 pixel spatial sampling density is achieved by using a custom-made fiber-optic bundle. The design considerations and fabrication aspects of the fiber bundle and imaging spectrometer are described in detail. Through the custom fiber bundle, the image of a scene of interest is collected and divided into discrete spatial groups, with spaces generated in between groups for spectral dispersion. This reorganized image is scaled down by an image taper for compatibility with following optical elements, dispersed by a prism, and is finally acquired by a CCD camera. To obtain an (x, y, λ) datacube from the snapshot measurement, a spectral calibration algorithm is executed for reconstruction of the spatial-spectral signatures of the observed scene. System characterization of throughput, resolution, and crosstalk was performed. Preliminary results illustrating changes in oxygen-saturation in an occluded human finger are presented to demonstrate the system's capabilities.

High resolution snapshot imaging spectrometer using a fusion algorithm based on grouping principal component analysis

We reported a high resolution snapshot imaging spectrometer (HR-SIS) and a fusion algorithm based on the properties of the HR-SIS. The system consists of an imaging branch and a spectral branch. The imaging branch captures a high spatial resolution panchromatic image with 680 × 680 pixels, while the spectral branch acquires a low spatial resolution spectral image with spectral resolution of 250 cm^-1. By using a fusion algorithm base on grouping principal component analysis, the spectral image is highly improved in spatial resolution. Experimental results demonstrated that the performance of the proposed algorithm is competitive with other state-of-the-art algorithms. The computing time for a single frame is less than 1 min with an Intel Core i5-4200H CPU, which can be further reduced by utilizing a graphics processing unit (GPU).