Coded-aperture imaging systems: Past, present and future development – A review

A gamma-ray imaging method for multiple radionuclides and low-activity radioactive sources

Gamma-ray coded-aperture imaging technology has important applications in the fields of nuclear security, isolated source detection, and the decommissioning of nuclear facilities. However, artifacts can reduce the quality of reconstructed images and affect the identification of the intensity and location of radioactive sources. In this paper, a gamma-ray coded-aperture imaging method based on primitive and reversed coded functions (PRCF) was proposed to reduce imaging artifacts. Building on this, the PRCF method was improved by integrating energy spectral information collected by the detector. By selecting energy intervals corresponding to characteristic energies of different radioactive sources for data filtering, the imaging capability of the PRCF method was further enhanced for multiple radioactive sources. Through simulation, the selection range of the correction factor in the PRCF method was determined. Single-source and multi-source imaging experiments were conducted using the self-built coded-aperture imaging system based on a CdZnTe pixel detector, and the selection criteria for the energy interval ranges of different radioactive sources were established. Compared with the conventional maximum likelihood expectation maximization (MLEM) method, the improved PRCF method not only effectively reduced artifacts and enhanced the imaging quality, but also ensured the accuracy of imaging results for multiple radioactive sources. Moreover, through imaging experiments using low-activity 137Cs and high-activity 241Am and 133Ba, it was demonstrated that the PRCF method can achieve low-count imaging in complex environments, providing a solution for imaging low-activity radioactive sources.

Three-Dimensional Scanning Virtual Aperture Imaging with Metasurface

Metasurface-based imaging is attractive due to its low hardware costs and system complexity. However, most of the current metasurface-based imaging systems require stochastic wavefront modulation, complex computational post-processing, and are restricted to 2D imaging. To overcome these limitations, we propose a scanning virtual aperture imaging system. The system first uses a focused beam to achieve near-field focal plane scanning, meanwhile forming a virtual aperture. Secondly, an adapted range migration algorithm (RMA) with a pre-processing step is applied to the virtual aperture to achieve a 3D high-resolution reconstruction. The pre-processing step fully exploits the feature of near-field beamforming that only a time delay is added on the received signal, which introduces ignorable additional calculation time. We build a compact prototype system working at a frequency from 38 to 40 GHz. Both the simulations and the experiments demonstrate that the proposed system can achieve high-quality imaging without complex implementations. Our method can be widely used for single-transceiver coherent systems to significantly improve the imaging depth of field (DOF).

Radiation image reconstruction and uncertainty quantification using a Gaussian process prior

We propose a complete framework for Bayesian image reconstruction and uncertainty quantification based on a Gaussian process prior (GPP) to overcome limitations of maximum likelihood expectation maximization (ML-EM) image reconstruction algorithm. The prior distribution is constructed with a zero-mean Gaussian process (GP) with a choice of a covariance function, and a link function is used to map the Gaussian process to an image. Unlike many other maximum a posteriori approaches, our method offers highly interpretable hyperparamters that are selected automatically with the empirical Bayes method. Furthermore, the GP covariance function can be modified to incorporate a priori structural priors, enabling multi-modality imaging or contextual data fusion. Lastly, we illustrate that our approach lends itself to Bayesian uncertainty quantification techniques, such as the preconditioned Crank-Nicolson method and the Laplace approximation. The proposed framework is general and can be employed in most radiation image reconstruction problems, and we demonstrate it with simulated free-moving single detector radiation source imaging scenarios. We compare the reconstruction results from GPP and ML-EM, and show that the proposed method can significantly improve the image quality over ML-EM, all the while providing greater understanding of the source distribution via the uncertainty quantification capability. Furthermore, significant improvement of the image quality by incorporating a structural prior is illustrated.

On the theory of multi-target coded sources for high-energy, high-resolution, and high-brightness x-ray radiography

X-ray radiography is a ubiquitous diagnostic technique in high energy density (HED) physics, with point projection backlighting commonly used for characterizing static and dynamic objects at high spatial and temporal resolutions. These are typically constrained in attainable resolution by their decrease in brightness, which is a limiting factor for high-Z HED experiments, such as double-shell implosions at the National Ignition Facility (NIF) requiring MeV-scale bremsstrahlung sources at high (<50μm) resolution. Coded source imaging is a technique using multiple point-projection sources to produce multiple overlapping radiographs, which are then decoded as a function of the source positions in a process akin to coded aperture imaging. Here, we discuss a new approach to coded source generation using multiple individual small-diameter wire targets within the footprint of a defocused large-scale a0 ≃ 1 laser to produce an MeV-scale high-resolution bright combined source for x-ray radiography. We outline optimal source designs with NIF-Advanced Radiography Capability as the case study, highlight the need for iterative reconstruction decoding, and discuss the research required to demonstrate a robust physical proof-of-concept.

Roadmap on computational methods in optical imaging and holography [invited]

Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00340-024-08280-3.

Non-invasive estimation of the powder size distribution from a single speckle image

Non-invasive characterization of powders may take one of two approaches: imaging and counting individual particles; or relying on scattered light to estimate the particle size distribution (PSD) of the ensemble. The former approach runs into practical difficulties, as the system must conform to the working distance and other restrictions of the imaging optics. The latter approach requires an inverse map from the speckle autocorrelation to the particle sizes. The principle relies on the pupil function determining the basic sidelobe shape, whereas the particle size spread modulates the sidelobe intensity. We recently showed that it is feasible to invert the speckle autocorrelation and obtain the PSD using a neural network, trained efficiently through a physics-informed semi-generative approach. In this work, we eliminate one of the most time-consuming steps of our previous method by engineering the pupil function. By judiciously blocking portions of the pupil, we sacrifice some photons but in return we achieve much enhanced sidelobes and, hence, higher sensitivity to the change of the size distribution. The result is a 60 × reduction in total acquisition and processing time, or 0.25 seconds per frame in our implementation. Almost real-time operation in our system is not only more appealing toward rapid industrial adoption, it also paves the way for quantitative characterization of complex spatial or temporal dynamics in drying, blending, and other chemical and pharmaceutical manufacturing processes.

Optical experimental results using Singer product apertures

We present the first optical experimental results obtained using the recently developed Singer product apertures. We also show that Fenimore and Cannon's fine sampling and delta decoding techniques can be combined with the fast direct vector decoding algorithm for Singer product apertures. We demonstrate resolutions and decoding speeds comparable to, or better than, those currently reported in the optical literature. Taken together these make possible coded aperture video in the optical domain.

On the symmetry of photon detection arrays: A directionally sensitive 3D model

A detector's ability to obtain the direction of a radioactive source is an invaluable operational asset. A 2D/3D model was developed based on directionally sensitive arrays. The average location of photon interactions within a symmetrical array yields the direction of the source. The model is validated with simulations and laboratory measurements, maximum systematic error being 5-10° at energies >200 keV and approaching zero at lower energies. The symmetry model yields the direction of a shielded source even when no full energy photons could be detected.

Assessment of the axial resolution of a compact gamma camera with coded aperture collimator

PURPOSE

Handheld gamma cameras with coded aperture collimators are under investigation for intraoperative imaging in nuclear medicine. Coded apertures are a promising collimation technique for applications such as lymph node localization due to their high sensitivity and the possibility of 3D imaging. We evaluated the axial resolution and computational performance of two reconstruction methods.

METHODS

An experimental gamma camera was set up consisting of the pixelated semiconductor detector Timepix3 and MURA mask of rank 31 with round holes of 0.08 mm in diameter in a 0.11 mm thick Tungsten sheet. A set of measurements was taken where a point-like gamma source was placed centrally at 21 different positions within the range of 12-100 mm. For each source position, the detector image was reconstructed in 0.5 mm steps around the true source position, resulting in an image stack. The axial resolution was assessed by the full width at half maximum (FWHM) of the contrast-to-noise ratio (CNR) profile along the z-axis of the stack. Two reconstruction methods were compared: MURA Decoding and a 3D maximum likelihood expectation maximization algorithm (3D-MLEM).

RESULTS

While taking 4400 times longer in computation, 3D-MLEM yielded a smaller axial FWHM and a higher CNR. The axial resolution degraded from 5.3 mm and 1.8 mm at 12 mm to 42.2 mm and 13.5 mm at 100 mm for MURA Decoding and 3D-MLEM respectively.

CONCLUSION

Our results show that the coded aperture enables the depth estimation of single point-like sources in the near field. Here, 3D-MLEM offered a better axial resolution but was computationally much slower than MURA Decoding, whose reconstruction time is compatible with real-time imaging.

Intraoperative Gamma Cameras: A Review of Development in the Last Decade and Future Outlook

Portable gamma cameras suitable for intraoperative imaging are in active development and testing. These cameras utilise a range of collimation, detection, and readout architectures, each of which can have significant and interacting impacts on the performance of the system as a whole. In this review, we provide an analysis of intraoperative gamma camera development over the past decade. The designs and performance of 17 imaging systems are compared in depth. We discuss where recent technological developments have had the greatest impact, identify emerging technological and scientific requirements, and predict future research directions. This is a comprehensive review of the current and emerging state-of-the-art as more devices enter clinical practice.

Fast and Accurate Gamma Imaging System Calibration Based on Deep Denoising Networks and Self-Adaptive Data Clustering

Gamma imagers play a key role in both industrial and medical applications. Modern gamma imagers typically employ iterative reconstruction methods in which the system matrix (SM) is a key component to obtain high-quality images. An accurate SM could be acquired from an experimental calibration step with a point source across the FOV, but at a cost of long calibration time to suppress noise, posing challenges to real-world applications. In this work, we propose a time-efficient SM calibration approach for a 4π-view gamma imager with short-time measured SM and deep-learning-based denoising. The key steps include decomposing the SM into multiple detector response function (DRF) images, categorizing DRFs into multiple groups with a self-adaptive K-means clustering method to address sensitivity discrepancy, and independently training separate denoising deep networks for each DRF group. We investigate two denoising networks and compare them against a conventional Gaussian filtering method. The results demonstrate that the denoised SM with deep networks faithfully yields a comparable imaging performance with the long-time measured SM. The SM calibration time is reduced from 1.4 h to 8 min. We conclude that the proposed SM denoising approach is promising and effective in enhancing the productivity of the 4π-view gamma imager, and it is also generally applicable to other imaging systems that require an experimental calibration step.

Resolving hidden pixels beyond the resolution limit of projection imaging by square aperture

Projection imaging has been employed widely in many areas, such as x-ray radiography, due to its penetration power and ballistic geometry of their paths. However, its resolution limit remains a major challenge, caused by the conflict of source intensity and source size associated with image blurriness. A simple yet robust scheme has been proposed here to solve the problem. An unconventional square aperture, rather than the usual circular beam, is constructed, which allows for the straightforward deciphering of a blurred spot, to unravel hundreds originally hidden pixels. With numerical verification and experimental demonstration, our proposal is expected to benefit multiple disciplines, not limited to x-ray imaging.

The atmospheric X-ray imaging spectrometer (AXIS) instrument: Quantifying energetic particle precipitation through bremsstrahlung X-ray imaging

The Atmospheric X-ray Imaging Spectrometer (AXIS) described in this work is a compact, wide field-of-view, hard x-ray imager. The AXIS instrument will fly onboard the Atmospheric Effects of Precipitation through Energetic X-rays (AEPEX) 6U CubeSat mission and will measure bremsstrahlung x-ray photons in the 50-240 keV range with cadmium-zinc-telluride (CZT) detectors using coded aperture optics. AXIS will measure photons generated by energetic particle precipitation for the purpose of determining the spatial scales of precipitation and estimating electron precipitation characteristics. This paper describes the design and testing of the AXIS instrument, including a summary of simulations performed that motivate the shielding, optics, and mechanical design. Testing and characterization is reported that validates the instrument design and shows that the instrument design meets or exceeds the measurement requirements necessary for AEPEX mission success.

An Innovative Concept of a 3D-Coded Aperture Imaging System Proposed for Early Breast Cancer Detection

Coded Aperture (CA) imaging has recently been used in nuclear medicine, but still, there is no commercial SPECT imaging camera based on CA for cancer detection. The literature is rich in examples of using the CA for planar and thin 3D imaging. However, thick 3D reconstruction is still challenging for small lesion detection. This paper presents the results of mosaic modified uniformly redundant array (MURA) mask/antimask CA combined with a maximum-likelihood expectation-maximization (MLEM) algorithm. The MLEM is an iterative algorithm applied to a mosaic MURA CA mask/antimask for 3D anthropomorphic breast phantom reconstruction, slice by slice. The difference between the mask and the antimask suppresses the background noise to enhance the quality of reconstructed images. Furthermore, all reconstructed slices are stacked to form a 3D breast phantom image from single-projection data. The results of phantom reconstruction with 8 mm, 6 mm, 4 mm, and 3 mm lesions are presented. Moreover, the proposed SPECT imaging camera can reconstruct a 3D breast phantom from single-projection data of the patient’s scanning. To assess the quality of lesions in the reconstructed images, the contrast-to-background ratio (CBR), the peak signal-to-noise ratio (PSNR) and mean square error (MSE) were measured.

Nonlinear Reconstruction of Images from Patterns Generated by Deterministic or Random Optical Masks-Concepts and Review of Research

Indirect-imaging methods involve at least two steps, namely optical recording and computational reconstruction. The optical-recording process uses an optical modulator that transforms the light from the object into a typical intensity distribution. This distribution is numerically processed to reconstruct the object's image corresponding to different spatial and spectral dimensions. There have been numerous optical-modulation functions and reconstruction methods developed in the past few years for different applications. In most cases, a compatible pair of the optical-modulation function and reconstruction method gives optimal performance. A new reconstruction method, termed nonlinear reconstruction (NLR), was developed in 2017 to reconstruct the object image in the case of optical-scattering modulators. Over the years, it has been revealed that the NLR can reconstruct an object's image modulated by an axicons, bifocal lenses and even exotic spiral diffractive elements, which generate deterministic optical fields. Apparently, NLR seems to be a universal reconstruction method for indirect imaging. In this review, the performance of NLR isinvestigated for many deterministic and stochastic optical fields. Simulation and experimental results for different cases are presented and discussed.

The cusp plasma imaging detector (CuPID) cubesat observatory: Instrumentation

The Cusp Plasma Imaging Detector (CuPID) CubeSat observatory is a 6U CubeSat designed to observe solar wind charge exchange in magnetospheric cusps to test competing theories of magnetic reconnection at the Earth's magnetopause. The CuPID is equipped with three instruments, namely, a wide field-of-view (4.6° × 4.6°) soft x-ray telescope, a micro-dosimeter suite, and an engineering magnetometer optimized for the science operation. The instrument suite has been tested and calibrated in relevant environments, demonstrating successful design. The testing and calibration of these instruments produced metrics and coefficients that will be used to create the CuPID mission's data product.

Experimental and Geant4 Simulation Study of MURA Mask for Scintimammography

Gamma rays have been extensively investigated for breast imaging using collimators; however, the coded-aperture technique needs to be investigated more. In this paper, we propose an experimental study and Geant4 simulations of MURA mask breast imaging. First, we compare the experimental data against the simulation results carried out using Geant4 (version 10.4) and accreditation phantom. Second, we virtually extend our work by changing the tumor-to-background (TBR) and lesion location parameters. We used 99mTc as a radioactive source. Good agreement has been seen for the benchmark stage, especially in terms of tumor localization. Moreover, the calculated full width at half maximum (FWHM) and contrast for decoded images (having average values of 8 and 3.5 for TBR between 2 and 10) permitted us to conclude that we can accurately localize small lesions up to lower TBR values by following the decoding procedure of deducing the image of a “blank phantom” (phantom with TBR = 1) every time within a matlab-based program. Hence, this work can be considered a continuously added value to previous investigations for scintimammography imaging.

Compact metalens-based integrated imaging devices for near-infrared microscopy

With current trends to progressively miniaturize optical systems, it is now essential to look for alternative methods to control light at extremely small dimensions. Metalenses are composed of subwavelength nanostructures and have an excellent ability to manipulate the polarization, phase, and amplitude of incident light. Although great progress of metalenses has been made, the compact metalens-integrated devices have not been researched adequately. In the study, we present compact imaging devices for near-infrared microscopy, in which a metalens is exploited. The indicators including resolution, magnification, and image quality are investigated via imaging several specimens of intestinal cells to verify the overall performance of the imaging system. The further compact devices, where the metalens is integrated directly on the CMOS imaging sensor, are also researched to detect biomedical issues. This study provides an approach to constructing compact imaging devices based on metalenses for near-infrared microscopy, micro-telecopy, etc., which can promote the miniaturization tending of futural optical systems.

Neural network method for localization of radioactive sources within a partially coded field-of-view in coded-aperture imaging

Coded-aperture imagers typically have a smaller field-of-view (FOV) than in un-collimated gamma imaging systems. However, sources out of the fully coded field-of-view (FCFOV) can cause pseudo hotspots on the wrong side of an image reconstructed using the cross-correlation method. In this work, we propose a neural network method to identify and localize the sources within the partially coded field-of-view (PCFOV). The model was trained using Monte Carlo simulation data and evaluated with both simulation and experimental data. The results showed that the proposed model can identify and localize sources with good classification accuracy, low positioning error, and strong robustness to the statistical noise.

State-of-the-Art Mobile Radiation Detection Systems for Different Scenarios

In the last decade, the development of more compact and lightweight radiation detection systems led to their application in handheld and small unmanned systems, particularly air-based platforms. Examples of improvements are: the use of silicon photomultiplier-based scintillators, new scintillating crystals, compact dual-mode detectors (gamma/neutron), data fusion, mobile sensor networks, cooperative detection and search. Gamma cameras and dual-particle cameras are increasingly being used for source location. This study reviews and discusses the research advancements in the field of gamma-ray and neutron measurements using mobile radiation detection systems since the Fukushima nuclear accident. Four scenarios are considered: radiological and nuclear accidents and emergencies; illicit traffic of special nuclear materials and radioactive materials; nuclear, accelerator, targets, and irradiation facilities; and naturally occurring radioactive materials monitoring-related activities. The work presented in this paper aims to: compile and review information on the radiation detection systems, contextual sensors and platforms used for each scenario; assess their advantages and limitations, looking prospectively to new research and challenges in the field; and support the decision making of national radioprotection agencies and response teams in respect to adequate detection system for each scenario. For that, an extensive literature review was conducted.

Punching holes in light: recent progress in single-shot coded-aperture optical imaging

Single-shot coded-aperture optical imaging physically captures a code-aperture-modulated optical signal in one exposure and then recovers the scene via computational image reconstruction. Recent years have witnessed dazzling advances in various modalities in this hybrid imaging scheme in concomitant technical improvement and widespread applications in physical, chemical and biological sciences. This review comprehensively surveys state-of-the-art single-shot coded-aperture optical imaging. Based on the detected photon tags, this field is divided into six categories: planar imaging, depth imaging, light-field imaging, temporal imaging, spectral imaging, and polarization imaging. In each category, we start with a general description of the available techniques and design principles, then provide two representative examples of active-encoding and passive-encoding approaches, with a particular emphasis on their methodology and applications as well as their advantages and challenges. Finally, we envision prospects for further technical advancement in this field.

Optimizing Coded Aperture Imaging techniques to allow for online tracking of fiducial markers with high-energy scattered radiation from treatment beam

PURPOSE

Real-time visualization of target motion using fiducial markers during radiation therapy treatment will allow for more accurate dose delivery. The purpose of this study was to optimize techniques for online fiducial marker tracking by detecting the scattered treatment beam through coded aperture imaging (CAI). Coded aperture imaging is a novel imaging technique that can allow target tracking in real time during treatment, and do so without adding any additional radiation dose, by making use of the scattered treatment beam radiation.

METHODS

Radiotherapy beams of various energies, incident on phantoms containing gold fiducial markers were modeled using MCNP6.2 Monte Carlo transport code. Orthogonal scatter radiographs were collected through a CAI geometry. After decoding the simulated radiograph data, the centroid location and FWHM/SNR of the fiducial signals were analyzed. The effects of properties related to the CA (rank, pattern, and physical dimensions), detector (dimensions and pixel count), position (CA and phantom), and the incident beam (spectrum and direction) were investigated. These variables were evaluated by quantifying the positional accuracy, resolution, and SNR of the fiducials' signal. The effects of phantom scatter and decoding artifacts were reduced via Fourier filtering to avoid treatment interruption and physical interaction with the coded mask.

RESULTS

The method was able to accurately localize the markers to within 1 pixel of a simulated radiograph. A 10 × 10 × 2 cm tungsten mask was chosen to attenuate >99 % of incident scatter through opaque elements, while minimizing collimation artifacts which arise from vignetting of the coded radiograph. Clear separation of centroids from fiducial signals with 2.5 mm separation was maintained, and initial optimization of parameters has produced an aperture which decodes the location of multiple fiducial markers inside a human phantom properly with a high SNR in the final radiograph image.

CONCLUSION

Current results show a proof of concept for a novel real-time imaging method. Coded aperture imaging is a promising technique for extracting the fiducial scatter signal from a broader Compton-scatter background. These results can be used to further optimize the CAI parameter space and guide fabrication and testing of a clinical device.

Controllable Motion-Blur Effects in Still Images

Motion blur in a photo is the consequence of object motion during the image acquisition. It results in a visible trail along the motion of a recorded object and can be used by photographers to convey a sense of motion. Nevertheless, it is very challenging to acquire this effect as intended and requires much experience from the photographer. To achieve actual control over the motion blur, one could be added in a post process but current solutions require complex manual intervention and can lead to artifacts that mix moving and static objects incorrectly. In this paper, we propose a novel method to add motion blur to a single image that generates the illusion of a photographed motion. Relying on a minimal user input, a filtering process is employed to produce a virtual motion effect. It carefully handles object boundaries to avoid artifacts produced by standard filtering methods. We illustrate the effectiveness of our solution with various complex examples, including multi-directional blur, reflections, multiple objects, and illustrate how several motion-related artistic effects can be achieved. Our post-processing solution is an alternative to capturing the intended real-world motion blur directly and enables fine-grained control of the motion-blur effect.

Speckle-Correlation Scattering Matrix Approaches for Imaging and Sensing through Turbidity

The development of optical and computational techniques has enabled imaging without the need for traditional optical imaging systems. Modern lensless imaging techniques overcome several restrictions imposed by lenses, while preserving or even surpassing the capability of lens-based imaging. However, existing lensless methods often rely on a priori information about objects or imaging conditions. Thus, they are not ideal for general imaging purposes. The recent development of the speckle-correlation scattering matrix (SSM) techniques facilitates new opportunities for lensless imaging and sensing. In this review, we present the fundamentals of SSM methods and highlight recent implementations for holographic imaging, microscopy, optical mode demultiplexing, and quantification of the degree of the coherence of light. We conclude with a discussion of the potential of SSM and future research directions.

Deeply coded aperture for lensless imaging

In this Letter, we present a method for jointly designing a coded aperture and a convolutional neural network for reconstructing an object from a single-shot lensless measurement. The coded aperture and the reconstruction network are connected with a deep learning framework in which the coded aperture is placed as a first convolutional layer. Our co-optimization method was experimentally demonstrated with a fully convolutional network, and its performance was compared to a coded aperture with a modified uniformly redundant array.

Lensless Computational Imaging Technology Using Deep Convolutional Network

Within the framework of Internet of Things or when constrained in limited space, lensless imaging technology provides effective imaging solutions with low cost and reduced size prototypes. In this paper, we proposed a method combining deep learning with lensless coded mask imaging technology. After replacing lenses with the coded mask and using the inverse matrix optimization method to reconstruct the original scene images, we applied FCN-8s, U-Net, and our modified version of U-Net, which is called Dense-U-Net, for post-processing of reconstructed images. The proposed approach showed supreme performance compared to the classical method, where a deep convolutional network leads to critical improvements of the quality of reconstruction.

Unsupervised deep learning for depth estimation with offset pixels

Offset Pixel Aperture (OPA) camera has been recently proposed to estimate disparity of a scene with a single shot. Disparity is obtained in the image by offsetting the pixels by a fixed distance. Previously, correspondence matching schemes have been used for disparity estimation with OPA. To improve disparity estimation we use a data-oriented approach. Specifically, we use unsupervised deep learning to estimate the disparity in OPA images. We propose a simple modification to the training strategy which solves the vanishing gradients problem with the very small baseline of the OPA camera. Training degenerates to poor disparity maps if the OPA images are used directly for left-right consistency check. By using images obtained from displaced cameras at training, accurate disparity maps are obtained. The performance of the OPA camera is significantly improved compared to previously proposed single-shot cameras and unsupervised disparity estimation methods. The approach provides 8 frames per second on a single Nvidia 1080 GPU with 1024×512 OPA images. Unlike conventional approaches, which are evaluated in controlled environments, our paper shows the utility of deep learning for disparity estimation with real life sensors and low quality images. By combining OPA with deep learning, we obtain a small depth sensor capable of providing accurate disparity at usable frame rates. Also the ideas in this work can be used in small-baseline stereo systems for short-range depth estimation and multi-baseline stereo to increase the depth range.

Programmable liquid crystal display based noise reduced dynamic synthetic coded aperture imaging camera (NoRDS-CAIC)

Besides traditional lens-based imaging techniques, coded aperture imaging (CAI) can also provide target images but without using any optical lenses, therefore it is another solution in imaging applications. Most CAI methods reconstruct target image only from a single-shot coded image using a fixed coding mask; however, the collected partial information inevitably deteriorates the reconstruction quality. Though multi-exposure CAI methods are designed, these existed algorithms can hardly improve reconstruction signal-to-noise ratio (SNR) and spatial resolution simultaneously; additionally, dynamic coding mask display still requires expensive devices and complicated systems. In order to reconstruct target image with both enhanced spatial resolution and SNR but using cost-effective devices and a simple system, we design a noise reduced dynamic synthetic coded aperture imaging camera (NoRDS-CAIC) in this paper. The NoRDS-CAIC only consists of a programmable liquid crystal display (LCD) and an image recorder, and both of them are integrated with a three-dimensional printed shell with the compact size of 19 cm × 15 cm × 16 cm and controlled by our designed software to automatically realize coding mask display, coded image recording and target image reconstruction. When using the NoRDS-CAIC, the optimized coding mask is first sent to the programmable LCD and displayed, then the corresponding coded image is automatically captured using the image recorder. Next, cycle the above procedures to capture enough coded images with previously known coding masks and measured point spread functions (PSFs), and the target image can be finally reconstructed using our designed NoRDS-CAIC decoding algorithm, which is shown with better noise suppression capability and higher reconstruction resolution compared to other classical CAI algorithms. According to the experimental verifications, the NoRDS-CAIC can reach the high resolution of 99.2 µm and the high SNR of 19.43 dB, proving that the designed NoRDS-CAIC can be potentially used for lensless imaging in practical applications.

Modal decomposition for the fiber beams with arbitrary degree of coherence based on the Wigner distribution function

Modal decomposition (MD) plays an increasingly important role in characterizing fiber beams. Several promising MD techniques have been proposed in literature, all of which are based on a common hypothesis that the modal field is coherently superposed by transverse modes. However, the partially coherent conditions have to be expected in general. In order to take account of this ordinary case, a novel MD scheme employing the Wigner distribution function (WDF) is introduced, which allows the decomposition of fiber beams without any restrictions regarding their degree of coherence. The four-dimensional (4D) WDF (two spatial and two spatial frequency dimensions) of the 2D fiber beam is reconstructed using the coded aperture technique. Based on the measured WDF and orthogonal property of transverse modes, the modal coefficients as well as the mutual modal degree of coherence will be determined unambiguously. The validity and reliability of the proposed approach are illustrated with numerical examples.

Resolution and signal-to-noise ratio enhancement for synthetic coded aperture imaging via varying pinhole array

In traditional coded aperture imaging (CAI), the main problem with the deconvolution method is that the noise may dominate the decoding process. Focusing on this problem, a synthetic CAI (s-CAI) method is proposed with a potential to improve the signal-to-noise ratio (SNR) and spatial resolution of CAI. A series of raw coded images are recorded while the pinhole array is varied to different structures, and an iterative decoding algorithm is developed to reconstruct the object image by synthetically using all of these pinhole array structures and corresponding decoded images. Because the proposed iterative decoding algorithm has prominent advantage in noise immunization, and more spatial light components are acquired in comparison with common single-exposure CAI, both the image resolution and SNR are remarkably enhanced.