Compact light field photography towards versatile three-dimensional vision

Light-field tomographic fluorescence lifetime imaging microscopy

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging technique that enables the visualization of biological samples at the molecular level by measuring the fluorescence decay rate of fluorescent probes. This provides critical information about molecular interactions, environmental changes, and localization within biological systems. However, creating high-resolution lifetime maps using conventional FLIM systems can be challenging, as it often requires extensive scanning that can significantly lengthen acquisition times. This issue is further compounded in three-dimensional (3D) imaging because it demands additional scanning along the depth axis. To tackle this challenge, we developed a computational imaging technique called light-field tomographic FLIM (LIFT-FLIM). Our approach allows for the acquisition of volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps required compared to conventional point-scanning or line-scanning FLIM imagers. Moreover, LIFT-FLIM enables the measurement of high-dimensional data using low-dimensional detectors, which are typically low cost and feature a higher temporal bandwidth. We demonstrated LIFT-FLIM using a linear single-photon avalanche diode array on various biological systems, showcasing unparalleled single-photon detection sensitivity. Additionally, we expanded the functionality of our method to spectral FLIM and demonstrated its application in high-content multiplexed imaging of lung organoids. LIFT-FLIM has the potential to open up broad avenues in both basic and translational biomedical research.

Design of a Novel Microlens Array and Imaging System for Light Fields

Light field cameras are unsuitable for further acquisition of high-quality images due to their small depth of field, insufficient spatial resolution, and poor imaging quality. To address these issues, we proposed a novel four-focal-square microlens and light field system. A square aspheric microlens array with four orthogonal focal lengths was designed, in which the aperture of a single lens was 100 μm. The square arrangement improves pixel utilization, the four focal lengths increase the depth of field, and the aspheric improves image quality. The simulations demonstrate pixel utilization rates exceeding 90%, depth-of-field ranges 6.57 times that of a single focal length, and image quality is significantly improved. We have provided a potential solution for improving the depth of field and image quality of the light field imaging system.

An Innovative Virtual Reality System for Measuring Refractive Error

In this study, we aimed to validate a novel light field virtual reality (LFVR) system for estimating refractive errors in the human eye. Fifty participants with an average age of 22.12 ± 2.2 years (range 20-30 years) were enrolled. The present study compared spherical equivalent (SE) and focal line measurements (F1 and F2) obtained by the LFVR system with those obtained by established methods, including closed-field and open-field autorefractors, retinoscopy, and subjective refraction. The results showed substantial agreement between the LFVR system and the traditional methods, with intraclass correlation coefficients (ICC) for SE ranging from 82.7% to 86.7% (p < 0.01), and for F1 and F2 from 80.7% to 86.4% (p < 0.01). Intra-repeatability for F1 and F2 demonstrated strong agreement, with ICC values of 88.8% and 97.5%, respectively. These findings suggest that the LFVR system holds potential as a primary tool for refractive error measurement in optical care, offering high agreement and repeatability compared to conventional methods.

Advancements in fluorescence lifetime imaging microscopy Instrumentation: Towards high speed and 3D

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging tool offering molecular specific insights into samples through the measurement of fluorescence decay time, with promising applications in diverse research fields. However, to acquire two-dimensional lifetime images, conventional FLIM relies on extensive scanning in both the spatial and temporal domain, resulting in much slower acquisition rates compared to intensity-based approaches. This problem is further magnified in three-dimensional imaging, as it necessitates additional scanning along the depth axis. Recent advancements have aimed to enhance the speed and three-dimensional imaging capabilities of FLIM. This review explores the progress made in addressing these challenges and discusses potential directions for future developments in FLIM instrumentation.

Discrimination of object information by bat echolocation deciphered from acoustic simulations

High-precision visual sensing has been achieved by combining cameras with deep learning. However, an unresolved challenge involves identifying information that remains elusive for optical sensors, such as occlusion spots hidden behind objects. Compared to light, sound waves have longer wavelengths and can, therefore, collect information on occlusion spots. In this study, we investigated whether bats could perform advanced sound sensing using echolocation to acquire a target's occlusion information. We conducted a two-alternative forced choice test on Pipistrellus abramus with five different targets, including targets with high visual similarity from the front, but different backend geometries, i.e. occlusion spots or textures. Subsequently, the echo impulse responses produced by these targets, which were difficult to obtain with real measurements, were computed using three-dimensional acoustic simulations to provide a detailed analysis consisting of the acoustic cues that the bats obtained through echolocation. Our findings demonstrated that bats could effectively discern differences in target occlusion spot structure and texture through echolocation. Furthermore, the discrimination performance was related to the differences in the logarithmic spectral distortion of the occlusion-related components in the simulated echo impulse responses. This suggested that the bats obtained occlusion information through echolocation, highlighting the advantages of utilizing broadband ultrasound for sensing.

LinoSPAD2: an FPGA-based, hardware-reconfigurable 512×1 single-photon camera system

We report on LinoSPAD2, a single-photon camera system, comprising a 512×1 single-photon avalanche diode (SPAD) front-end and one or two FPGA-based back-ends. Digital signals generated by the SPADs are processed by the FPGA in real time, whereas the FPGA offers full reconfigurability at a very high level of granularity both in time and space domains. The LinoSPAD2 camera system can process 512 SPADs simultaneously through 256 channels, duplicated on each FPGA-based back-end, with a bank of 64 time-to-digital converters (TDCs) operating at 133 MSa/s, whereas each TDC has a time resolution of 20 ps (LSB). To the best of our knowledge, LinoSPAD2 is the first fully reconfigurable SPAD camera system of large format. The SPAD front-end features a pitch of 26.2 μm, a native fill factor of 25.1%, and a microlens array achieving 2.3× concentration factor. At room temperature, the median dark count rate (DCR) is 80 cps at 7 V excess bias, the peak photon detection probability (PDP) is 53% at 520 nm wavelength, and the single-photon timing resolution (SPTR) is 50 ps FWHM. The instrument response function (IRF) is around 100 ps FWHM at system level. The LinoSPAD2 camera system is suitable for numerous applications, including LiDAR imaging, heralded spectroscopy, compressive Raman sensing, and other computational imaging techniques.

Metaverse Wearables for Immersive Digital Healthcare: A Review

The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.

Light-field tomographic fluorescence lifetime imaging microscopy

Fluorescence lifetime imaging microscopy (FLIM) is a powerful imaging technique that enables the visualization of biological samples at the molecular level by measuring the fluorescence decay rate of fluorescent probes. This provides critical information about molecular interactions, environmental changes, and localization within biological systems. However, creating high-resolution lifetime maps using conventional FLIM systems can be challenging, as it often requires extensive scanning that can significantly lengthen acquisition times. This issue is further compounded in three-dimensional (3D) imaging because it demands additional scanning along the depth axis. To tackle this challenge, we developed a novel computational imaging technique called light field tomographic FLIM (LIFT-FLIM). Our approach allows for the acquisition of volumetric fluorescence lifetime images in a highly data-efficient manner, significantly reducing the number of scanning steps required compared to conventional point-scanning or line-scanning FLIM imagers. Moreover, LIFT-FLIM enables the measurement of high-dimensional data using low-dimensional detectors, which are typically low-cost and feature a higher temporal bandwidth. We demonstrated LIFT-FLIM using a linear single-photon avalanche diode array on various biological systems, showcasing unparalleled single-photon detection sensitivity. Additionally, we expanded the functionality of our method to spectral FLIM and demonstrated its application in high-content multiplexed imaging of lung organoids. LIFT-FLIM has the potential to open up new avenues in both basic and translational biomedical research.

Augmented light field tomography through parallel spectral encoding

Snapshot recording of transient dynamics in three dimensions (3-D) is highly demanded in both fundamental and applied sciences. Yet it remains challenging for conventional high-speed cameras to address this need due to limited electronic bandwidth and reliance on mechanical scanning. The emergence of light field tomography (LIFT) provides a new solution to these long-standing problems and enables 3-D imaging at an unprecedented frame rate. However, based on sparse-view computed tomography, LIFT can accommodate only a limited number of projections, degrading the resolution in the reconstructed image. To alleviate this problem, we herein present a spectral encoding scheme to significantly increase the number of allowable projections in LIFT while maintaining its snapshot advantage. The resultant system can record 3-D dynamics at a kilohertz volumetric frame rate. Moreover, by using a multichannel compressed sensing algorithm, we improve the image quality with an enhanced spatial resolution and suppressed aliasing artifacts.