Integral imaging microscopy with enhanced depth-of-field using a spatial multiplexing

High-Quality 3D Visualization System for Light-Field Microscopy with Fine-Scale Shape Measurement through Accurate 3D Surface Data

We propose a light-field microscopy display system that provides improved image quality and realistic three-dimensional (3D) measurement information. Our approach acquires both high-resolution two-dimensional (2D) and light-field images of the specimen sequentially. We put forward a matting Laplacian-based depth estimation algorithm to obtain nearly realistic 3D surface data, allowing the calculation of depth data, which is relatively close to the actual surface, and measurement information from the light-field images of specimens. High-reliability area data of the focus measure map and spatial affinity information of the matting Laplacian are used to estimate nearly realistic depths. This process represents a reference value for the light-field microscopy depth range that was not previously available. A 3D model is regenerated by combining the depth data and the high-resolution 2D image. The element image array is rendered through a simplified direction-reversal calculation method, which depends on user interaction from the 3D model and is displayed on the 3D display device. We confirm that the proposed system increases the accuracy of depth estimation and measurement and improves the quality of visualization and 3D display images.

Depth Estimation for Integral Imaging Microscopy Using a 3D-2D CNN with a Weighted Median Filter

This study proposes a robust depth map framework based on a convolutional neural network (CNN) to calculate disparities using multi-direction epipolar plane images (EPIs). A combination of three-dimensional (3D) and two-dimensional (2D) CNN-based deep learning networks is used to extract the features from each input stream separately. The 3D convolutional blocks are adapted according to the disparity of different directions of epipolar images, and 2D-CNNs are employed to minimize data loss. Finally, the multi-stream networks are merged to restore the depth information. A fully convolutional approach is scalable, which can handle any size of input and is less prone to overfitting. However, there is some noise in the direction of the edge. A weighted median filtering (WMF) is used to acquire the boundary information and improve the accuracy of the results to overcome this issue. Experimental results indicate that the suggested deep learning network architecture outperforms other architectures in terms of depth estimation accuracy.

High-quality 3D display system for an integral imaging microscope using a simplified direction-inversed computation based on user interaction

We propose and implement a high-quality three-dimensional (3D) display system for an integral imaging microscope using a simplified direction-inversed computation method based on user interaction. A model of the specimen is generated from the estimated depth information (via the convolutional neural network-based algorithm), the quality of the model is defined by the high-resolution two-dimensional image. The new elemental image arrays are generated from the models via a simplified direction-inversed computation method according to the user interaction and directly displayed on the display device. A high-quality 3D visualization of the specimen is reconstructed and displayed while the lens array is placed in front of the display device. The user interaction enables more viewpoints of the specimen to be reconstructed by the proposed system, within the basic viewing zone. Remarkable quality improvement is confirmed through quantitative evaluations of the experimental results.

Simplified digital content generation based on an inverse-directed propagation algorithm for holographic stereogram printing

Holographic stereogram (HS) printing requires extensive memory capacity and long computation time during perspective acquisition and implementation of the pixel re-arrangement algorithm. Hogels contain very weak depth information of the object. We propose a HS printing system that uses simplified digital content generation based on the inverse-directed propagation (IDP) algorithm for hogel generation. Specifically, the IDP algorithm generates an array of hogels using a simple process that acquires the full three-dimensional (3D) information of the object, including parallax, depth, color, and shading, via a computer-generated integral imaging technique. This technique requires a short computation time and is capable of accounting for occlusion and accommodation effects of the object points via the IDP algorithm. Parallel computing is utilized to produce a high-resolution hologram based on the properties of independent hogels. To demonstrate the proposed approach, optical experiments are conducted in which the natural 3D visualizations of real and virtual objects are printed on holographic material. Experimental results demonstrate the simplified computation involved in content generation using the proposed IDP-based HS printing system and the improved image quality of the holograms.

Angular spectrum matching for digital holographic microscopy under extremely low light conditions

Digital holographic microscopy (DHM) is a future three-dimensional (3D) microscopy due to its high-resolution and high-precision 3D images. Thus, it is getting attention in bioinformatics, semiconductor defect detection, etc. However, some limitations still exist. Especially, high-speed holographic imaging requires high-power lasers, which are difficult to image on highly absorbent or light-sensitive samples. To overcome these issues, we propose a new, to the best of our knowledge, digital hologram recovery algorithm called angular spectrum matching (ASM), which achieves hologram imitation to recover holograms in digital holography at low light intensities. The hologram used for the background phase comparison is recorded without objects; thus, no power limitation is required. The ASM utilizes this background hologram to recover dark holograms. We present experimental results showing improved DHM numerical reconstructions and recovered holograms under extremely low light conditions.

Roadmap on 3D integral imaging: sensing, processing, and display

This Roadmap article on three-dimensional integral imaging provides an overview of some of the research activities in the field of integral imaging. The article discusses various aspects of the field including sensing of 3D scenes, processing of captured information, and 3D display and visualization of information. The paper consists of a series of 15 sections from the experts presenting various aspects of the field on sensing, processing, displays, augmented reality, microscopy, object recognition, and other applications. Each section represents the vision of its author to describe the progress, potential, vision, and challenging issues in this field.

Fast-switching laterally virtual-moving microlens array for enhancing spatial resolution in light-field imaging system without degradation of angular sampling resolution

We present an electrically controllable fast-switching virtual-moving microlens array (MLA) consisting of a stacked structure of two polarization-dependent microlens arrays (PDMLAs) with optical orthogonality, where the position of the two stacked PDMLAs is shifted by half the elemental pitch in the diagonal direction. By controlling the polarization of the incident light without the physical movement of the molecules comprising the virtual-moving MLA, the periodic sampling position of the MLA can be switched fast using a polarization-switching layer based on a fast-switching liquid crystal cell. Using the fast-switching virtual-moving MLA, the spatial-resolution-enhanced light-field (LF) imaging system was demonstrated without a decrease in the angular sampling resolution as compared to the conventional LF imaging system comprising a passive MLA; two sets of elemental image arrays were captured quickly owing to the short switching time of the virtual-moving MLA of 450 μs. From the two captured sets of the elemental array image, four-times resolution-enhanced reconstruction images of the directional-view and depth-slice images could be obtained.

Three-dimensional imaging system with both improved lateral resolution and depth of field considering non-uniform system parameters

In this paper, we propose a new 3D passive image sensing and visualization technique to improve lateral resolution and depth of field (DoF) of integral imaging simultaneously. There is a resolution trade-off between lateral resolution and DoF in integral imaging. To overcome this issue, a large aperture and a small aperture can be used to record the elemental images to reduce the diffraction effect and extend the DoF, respectively. Therefore, in this paper, we utilize these two pickup concepts with a non-uniform camera array. To show the feasibility of our proposed method, we implement an optical experiment. For comparison in details, we calculate the peak signal-to-noise ratio (PSNR) as the performance metric.

Dual-view integral imaging three-dimensional display using polarized glasses

We propose a dual-view integral imaging (DVII) three-dimensional (3D) display using polarized glasses. The DVII 3D display consists of a display panel, a polarized parallax barrier, a microlens array, and two pairs of polarized glasses. Two kinds of elemental images, which are captured from two different 3D scenes, are alternately arranged on the display panel. The polarized parallax barrier is attached to the display panel and composed of two kinds of units that are also alternately arranged. The polarization directions between adjacent units are perpendicular. The polarization directions of the two pairs of polarized glasses are the same as those of the two kinds of units of the polarized parallax barrier, respectively. The lights emitted from the two kinds of elemental images are modulated by the corresponding polarizer units and microlenses, respectively. Two different 3D images are reconstructed in the viewing zone and separated by using two pairs of polarized glasses. A prototype of the DVII 3D display is developed and two 3D images can be presented simultaneously, verifying the hypothesis.

Three-dimensional visualization system for ophthalmic microscopes using visible light and near-infrared illumination

In this paper, we describe a three-dimensional visualization system for ophthalmic microscopes that is aimed at microsurgery without the eyepieces. A three-dimensional visualization system for ophthalmic microscopes using the mixed illumination, which consists of visible light and near-infrared illumination, is established in order to acquire more exact information of object and reduce the amount of light irradiated to the patients, and its usage in microsurgery without eyepieces is herein described. A custom-designed stereoscopic three-dimensional display which is manufactured for the convenience of the surgeons during the long-time surgery, is connected directly to the camera of the ophthalmic microscope in order to eliminate the discomfort of eyepieces to the surgeon and signal delay between the camera, mounted on the microscope, and display device for surgeon. The main features of the established system are the signal delay-free for surgeon and the low level of illumination for patient. In particular, it could significantly reduce the amount of light irradiated on a patient's eye via NIR illumination. Upon comparison with the conventional system during clinical ophthalmology trials, this system is confirmed to require almost the same operation time and reduced discomfort and eyestrain during long periods of observation.

Enhancement of the depth-of-field of integral imaging microscope by using switchable bifocal liquid-crystalline polymer micro lens array

An integral imaging microscopy (IIM) system with improved depth-of-field (DoF) using a custom-designed bifocal polarization-dependent liquid-crystalline polymer micro lens array (LCP-MLA) is proposed. The implemented MLA has improved electro-optical properties such as a small focal ratio, high fill factor, low driving voltage, and fast switching speed, utilizing a well-aligned reactive mesogen on the imprinted reverse shape of the lens and a polarization switching layer. A bifocal MLA switches its focal length according to the polarization angle and acquires different DoF information of the specimen. After two elemental image arrays are captured, the depth-slices are reconstructed and combined to provide a widened DoF. The fabricated bifocal MLA consists of two identical polarization-dependent LCP-MLAs with 1.6 mm and f/16 focal ratio. Our experimental results confirmed that the proposed system improves the DoF of IIM without the need for mechanical manipulation.

Enhanced depth-of-field of an integral imaging microscope using a bifocal holographic optical element-micro lens array

We propose and implement an integral imaging microscope with extended depth-of-field (DoF) using a bifocal holographic micro lens array (MLA). The properties of the two MLAs are switched via peristrophic multiplexing, where different properties of the MLA are recorded onto the single holographic optical element (HOE). The recorded MLA properties are perpendicular to each other: after the first mode is recorded, the HOE is rotated by 90° clockwise, and the second mode is recorded. The experimental results confirm that the DoF of the integral imaging microscopy system is extended successfully by using the bifocal MLA.

Fabrication of Large-Scale Microlens Arrays Based on Screen Printing for Integral Imaging 3D Display

The low-cost large-scale fabrication of microlens arrays (MLAs) with precise alignment, great uniformity of focusing, and good converging performance are of great importance for integral imaging 3D display. In this work, a simple and effective method for large-scale polymer microlens arrays using screen printing has been successfully presented. The results show that the MLAs possess high-quality surface morphology and excellent optical performances. Furthermore, the microlens' shape and size, i.e., the diameter, the height, and the distance between two adjacent microlenses of the MLAs can be easily controlled by modifying the reflowing time and the size of open apertures of the screen. MLAs with the neighboring microlenses almost tangent can be achieved under suitable size of open apertures of the screen and reflowing time, which can remarkably reduce the color moiré patterns caused by the stray light between the blank areas of the MLAs in the integral imaging 3D display system, exhibiting much better reconstruction performance.