1
|
Liu H, Wang F, Jin Y, Ma X, Li S, Bian Y, Situ G. Learning-based real-time imaging through dynamic scattering media. LIGHT, SCIENCE & APPLICATIONS 2024; 13:194. [PMID: 39152120 PMCID: PMC11329739 DOI: 10.1038/s41377-024-01569-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/28/2024] [Revised: 08/01/2024] [Accepted: 08/05/2024] [Indexed: 08/19/2024]
Abstract
Imaging through dynamic scattering media is one of the most challenging yet fascinating problems in optics, with applications spanning from biological detection to remote sensing. In this study, we propose a comprehensive learning-based technique that facilitates real-time, non-invasive, incoherent imaging of real-world objects through dense and dynamic scattering media. We conduct extensive experiments, demonstrating the capability of our technique to see through turbid water and natural fog. The experimental results indicate that the proposed technique surpasses existing approaches in numerous aspects and holds significant potential for imaging applications across a broad spectrum of disciplines.
Collapse
Affiliation(s)
- Haishan Liu
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, 100049, Beijing, China
| | - Fei Wang
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
| | - Ying Jin
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
| | - Xianzheng Ma
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Siteng Li
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
| | - Yaoming Bian
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
| | - Guohai Situ
- Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China.
- Center of Materials Science and Optoelectronics Engineering, University of Chinese Academy of Sciences, 100049, Beijing, China.
- Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, 310024, China.
| |
Collapse
|
2
|
Wiesel B, Arnon S. Imaging inside highly scattering media using hybrid deep learning and analytical algorithm. JOURNAL OF BIOPHOTONICS 2023; 16:e202300127. [PMID: 37434270 DOI: 10.1002/jbio.202300127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 06/15/2023] [Accepted: 07/10/2023] [Indexed: 07/13/2023]
Abstract
Imaging through highly scattering media is a challenging problem with numerous applications in biomedical and remote-sensing fields. Existing methods that use analytical or deep learning tools are limited by simplified forward models or a requirement for prior physical knowledge, resulting in blurry images or a need for large training databases. To address these limitations, we propose a hybrid scheme called Hybrid-DOT that combines analytically derived image estimates with a deep learning network. Our analysis demonstrates that Hybrid-DOT outperforms a state-of-the-art ToF-DOT algorithm by improving the PSNR ratio by 4.6 dB and reducing the resolution by a factor of 2.5. Furthermore, when compared to a deep learning stand-alone model, Hybrid-DOT achieves a 0.8 dB increase in PSNR, 1.5 times the resolution, and a significant reduction in the required dataset size (factor of 1.6-3). The proposed model remains effective at higher depths, providing similar improvements for up to 160 mean-free paths.
Collapse
Affiliation(s)
- Ben Wiesel
- Ben-Gurion University of the Negev, Department of Electrical and Computer Engineering, Beer-Sheva, Israel
| | - Shlomi Arnon
- Ben-Gurion University of the Negev, Department of Electrical and Computer Engineering, Beer-Sheva, Israel
| |
Collapse
|
3
|
Feng BY, Guo H, Xie M, Boominathan V, Sharma MK, Veeraraghavan A, Metzler CA. NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media. SCIENCE ADVANCES 2023; 9:eadg4671. [PMID: 37379386 DOI: 10.1126/sciadv.adg4671] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 05/23/2023] [Indexed: 06/30/2023]
Abstract
Diffraction-limited optical imaging through scattering media has the potential to transform many applications such as airborne and space-based imaging (through the atmosphere), bioimaging (through skin and human tissue), and fiber-based imaging (through fiber bundles). Existing wavefront shaping methods can image through scattering media and other obscurants by optically correcting wavefront aberrations using high-resolution spatial light modulators-but these methods generally require (i) guidestars, (ii) controlled illumination, (iii) point scanning, and/or (iv) statics scenes and aberrations. We propose neural wavefront shaping (NeuWS), a scanning-free wavefront shaping technique that integrates maximum likelihood estimation, measurement modulation, and neural signal representations to reconstruct diffraction-limited images through strong static and dynamic scattering media without guidestars, sparse targets, controlled illumination, nor specialized image sensors. We experimentally demonstrate guidestar-free, wide field-of-view, high-resolution, diffraction-limited imaging of extended, nonsparse, and static/dynamic scenes captured through static/dynamic aberrations.
Collapse
Affiliation(s)
- Brandon Y Feng
- Department of Computer Science, The University of Maryland, College Park, College Park, MD 20742, USA
| | - Haiyun Guo
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA
| | - Mingyang Xie
- Department of Computer Science, The University of Maryland, College Park, College Park, MD 20742, USA
| | - Vivek Boominathan
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA
| | - Manoj K Sharma
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA
| | - Ashok Veeraraghavan
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA
| | - Christopher A Metzler
- Department of Computer Science, The University of Maryland, College Park, College Park, MD 20742, USA
| |
Collapse
|
4
|
Olivieri L, Peters L, Cecconi V, Cutrona A, Rowley M, Totero Gongora JS, Pasquazi A, Peccianti M. Terahertz Nonlinear Ghost Imaging via Plane Decomposition: Toward Near-Field Micro-Volumetry. ACS PHOTONICS 2023; 10:1726-1734. [PMID: 37363629 PMCID: PMC10288534 DOI: 10.1021/acsphotonics.2c01727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Indexed: 06/28/2023]
Abstract
Terahertz time-domain imaging targets the reconstruction of the full electromagnetic morphology of an object. In this spectral range, the near-field propagation strongly affects the information in the space-time domain in items with microscopic features. While this often represents a challenge, as the information needs to be disentangled to obtain high image fidelity, here, we show that such a phenomenon can enable three-dimensional microscopy. Specifically, we investigate the capability of the time-resolved nonlinear ghost imaging methodology to implement field-sensitive micro-volumetry by plane decomposition. We leverage the temporally resolved, field-sensitive detection to "refocus" an image plane at an arbitrary distance from the source, which defines the near-field condition, and within a microscopic sample. Since space-time coupling rapidly evolves and diffuses within subwavelength length scales, our technique can separate and discriminate the information originating from different planes at different depths. Our approach is particularly suitable for objects with sparse micrometric details. Building upon this principle, we demonstrate complex, time-domain volumetry resolving internal object planes with subwavelength resolution, discussing the range of applicability of our technique.
Collapse
Affiliation(s)
- Luana Olivieri
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Luke Peters
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Vittorio Cecconi
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Antonio Cutrona
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Maxwell Rowley
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Juan Sebastian Totero Gongora
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Alessia Pasquazi
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| | - Marco Peccianti
- Emergent
Photonics Research Centre, Department of Physics, Loughborough University, Loughborough LE11 3TU, UK
- Emergent
Photonics Lab (Epic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, UK
| |
Collapse
|
5
|
Zhao Y, Raghuram A, Wang F, Kim SH, Hielscher A, Robinson JT, Veeraraghavan A. Unrolled-DOT: an interpretable deep network for diffuse optical tomography. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:036002. [PMID: 36908760 PMCID: PMC9995139 DOI: 10.1117/1.jbo.28.3.036002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 02/09/2023] [Indexed: 06/18/2023]
Abstract
SIGNIFICANCE Imaging through scattering media is critical in many biomedical imaging applications, such as breast tumor detection and functional neuroimaging. Time-of-flight diffuse optical tomography (ToF-DOT) is one of the most promising methods for high-resolution imaging through scattering media. ToF-DOT and many traditional DOT methods require an image reconstruction algorithm. Unfortunately, this algorithm often requires long computational runtimes and may produce lower quality reconstructions in the presence of model mismatch or improper hyperparameter tuning. AIM We used a data-driven unrolled network as our ToF-DOT inverse solver. The unrolled network is faster than traditional inverse solvers and achieves higher reconstruction quality by accounting for model mismatch. APPROACH Our model "Unrolled-DOT" uses the learned iterative shrinkage thresholding algorithm. In addition, we incorporate a refinement U-Net and Visual Geometry Group (VGG) perceptual loss to further increase the reconstruction quality. We trained and tested our model on simulated and real-world data and benchmarked against physics-based and learning-based inverse solvers. RESULTS In experiments on real-world data, Unrolled-DOT outperformed learning-based algorithms and achieved over 10× reduction in runtime and mean-squared error, compared to traditional physics-based solvers. CONCLUSION We demonstrated a learning-based ToF-DOT inverse solver that achieves state-of-the-art performance in speed and reconstruction quality, which can aid in future applications for noninvasive biomedical imaging.
Collapse
Affiliation(s)
- Yongyi Zhao
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Ankit Raghuram
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Fay Wang
- Columbia University, Department of Biomedical Engineering, New York, New York, United States
| | - Stephen Hyunkeol Kim
- Columbia University Irvine Medical Center, Department of Radiology, New York, New York, United States
- New York University - Tandon School of Engineering, Department of Biomedical Engineering, New York, New York, United States
| | - Andreas Hielscher
- New York University - Tandon School of Engineering, Department of Biomedical Engineering, New York, New York, United States
| | - Jacob T. Robinson
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Ashok Veeraraghavan
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| |
Collapse
|
6
|
Kumar V, Cecconi V, Peters L, Bertolotti J, Pasquazi A, Totero Gongora JS, Peccianti M. Deterministic Terahertz Wave Control in Scattering Media. ACS PHOTONICS 2022; 9:2634-2642. [PMID: 35996370 PMCID: PMC9389618 DOI: 10.1021/acsphotonics.2c00061] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/20/2023]
Abstract
Scattering-assisted synthesis of broadband optical pulses is recognized to have a cross-disciplinary fundamental and application importance. Achieving full-waveform synthesis generally requires means for assessing the instantaneous electric field, i.e., the absolute electromagnetic phase. These are generally not accessible to established methodologies for scattering-assisted pulse envelope and phase shaping. The lack of field sensitivity also results in complex indirect approaches to evaluate the scattering space-time properties. The terahertz frequency domain potentially offers some distinctive new possibilities, thanks to the availability of methods to perform absolute measurements of the scattered electric field, as opposed to optical intensity-based diagnostics. An interesting conceptual question is whether this additional degree of freedom can lead to different types of methodologies toward wave shaping and direct field-waveform control. In this work, we theoretically investigate a deterministic scheme to achieve broadband, spatiotemporal waveform control of terahertz fields mediated by a scattering medium. Direct field access via time-domain spectroscopy enables a process in which the field and scattering matrix of the medium are assessed with minimal experimental efforts. Then, illumination conditions for an arbitrary targeted output field waveform are deterministically retrieved through numerical inversion. In addition, complete field knowledge enables reconstructing field distributions with complex phase profiles, as in the case of phase-only masks and optical vortices, a significantly challenging task for traditional implementations at optical frequencies based on intensity measurements aided with interferometric techniques.
Collapse
Affiliation(s)
- Vivek Kumar
- Emergent
Photonics Lab (EPic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K.
| | - Vittorio Cecconi
- Emergent
Photonics Lab (EPic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K.
| | - Luke Peters
- Emergent
Photonics Lab (EPic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K.
| | - Jacopo Bertolotti
- Department
of Physics and Astronomy, University of
Exeter, Exeter, Devon EX4 4QL, U.K.
| | - Alessia Pasquazi
- Emergent
Photonics Lab (EPic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K.
| | | | - Marco Peccianti
- Emergent
Photonics Lab (EPic), Department of Physics and Astronomy, University of Sussex, Brighton BN1 9QH, U.K.
| |
Collapse
|
7
|
Xu S, Yang X, Liu W, Jönsson J, Qian R, Konda PC, Zhou KC, Kreiß L, Wang H, Dai Q, Berrocal E, Horstmeyer R. Imaging Dynamics Beneath Turbid Media via Parallelized Single-Photon Detection. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2201885. [PMID: 35748188 PMCID: PMC9404405 DOI: 10.1002/advs.202201885] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Revised: 05/16/2022] [Indexed: 05/05/2023]
Abstract
Noninvasive optical imaging through dynamic scattering media has numerous important biomedical applications but still remains a challenging task. While standard diffuse imaging methods measure optical absorption or fluorescent emission, it is also well-established that the temporal correlation of scattered coherent light diffuses through tissue much like optical intensity. Few works to date, however, have aimed to experimentally measure and process such temporal correlation data to demonstrate deep-tissue video reconstruction of decorrelation dynamics. In this work, a single-photon avalanche diode array camera is utilized to simultaneously monitor the temporal dynamics of speckle fluctuations at the single-photon level from 12 different phantom tissue surface locations delivered via a customized fiber bundle array. Then a deep neural network is applied to convert the acquired single-photon measurements into video of scattering dynamics beneath rapidly decorrelating tissue phantoms. The ability to reconstruct images of transient (0.1-0.4 s) dynamic events occurring up to 8 mm beneath a decorrelating tissue phantom with millimeter-scale resolution is demonstrated, and it is highlighted how the model can flexibly extend to monitor flow speed within buried phantom vessels.
Collapse
Affiliation(s)
- Shiqi Xu
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
| | - Xi Yang
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
| | - Wenhui Liu
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
- Department of AutomationTsinghua UniversityBeijing100084China
| | - Joakim Jönsson
- Division of Combustion PhysicsDepartment of PhysicsLund UniversityLund22100Sweden
| | - Ruobing Qian
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
| | | | - Kevin C. Zhou
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
| | - Lucas Kreiß
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
- Institute of Medical BiotechnologyFriedrich‐Alexander‐University Erlangen‐Nürnberg (FAU)Erlangen91054Germany
| | - Haoqian Wang
- Tsinghua Shenzhen International Graduate SchoolTsinghua UniversityShenzhen518055China
| | - Qionghai Dai
- Department of AutomationTsinghua UniversityBeijing100084China
| | - Edouard Berrocal
- Division of Combustion PhysicsDepartment of PhysicsLund UniversityLund22100Sweden
| | - Roarke Horstmeyer
- Department of Biomedical EngineeringDuke UniversityDurhamNC27708USA
- Department of Electrical and Computer EngineeringDuke UniversityDurhamNC27708USA
- Department of PhysicsDuke UniversityDurhamNC27708USA
| |
Collapse
|
8
|
McShane EP, Chandrasekharan HK, Kufcsák A, Finlayson N, Erdogan AT, Henderson RK, Dhaliwal K, Thomson RR, Tanner MG. High resolution TCSPC imaging of diffuse light with a one-dimensional SPAD array scanning system. OPTICS EXPRESS 2022; 30:27926-27937. [PMID: 36236951 DOI: 10.1364/oe.461334] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 06/22/2022] [Indexed: 06/16/2023]
Abstract
We report a time-correlated single-photon counting (TCSPC) imaging system based on a line-scanning architecture. The system benefits from the high fill-factor, active area, and large dimension of an advanced CMOS single-photon avalanche diode (SPAD) array line-sensor. A two-dimensional image is constructed using a moving mirror to scan the line-sensor field-of-view (FOV) across the target, to enable the efficient acquisition of a two-dimensional 0.26 Mpixel TCSPC image. We demonstrate the capabilities of the system for TCSPC imaging and locating objects obscured in scattering media - specifically to locate a series of discrete point sources of light along an optical fibre submerged in a highly scattering solution. We demonstrate that by selectively imaging using early arriving photons which have undergone less scattering than later arriving photons, our TCSPC imaging system is able to locate the position of discrete point sources of light than a non-time-resolved imaging system.
Collapse
|
9
|
A boundary migration model for imaging within volumetric scattering media. Nat Commun 2022; 13:3234. [PMID: 35680924 PMCID: PMC9184484 DOI: 10.1038/s41467-022-30948-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2021] [Accepted: 05/12/2022] [Indexed: 11/25/2022] Open
Abstract
Effectively imaging within volumetric scattering media is of great importance and challenging especially in macroscopic applications. Recent works have demonstrated the ability to image through scattering media or within the weak volumetric scattering media using spatial distribution or temporal characteristics of the scattered field. Here, we focus on imaging Lambertian objects embedded in highly scattering media, where signal photons are dramatically attenuated during propagation and highly coupled with background photons. We address these challenges by providing a time-to-space boundary migration model (BMM) of the scattered field to convert the scattered measurements in spectral form to the scene information in the temporal domain using all of the optical signals. The experiments are conducted under two typical scattering scenarios: 2D and 3D Lambertian objects embedded in the polyethylene foam and the fog, which demonstrate the effectiveness of the proposed algorithm. It outperforms related works including time gating in terms of reconstruction precision and scattering strength. Even though the proportion of signal photons is only 0.75%, Lambertian objects located at more than 25 transport mean free paths (TMFPs), corresponding to the round-trip scattering length of more than 50 TMFPs, can be reconstructed. Also, the proposed method provides low reconstruction complexity and millisecond-scale runtime, which significantly benefits its application. Imaging in scattering media is challenging due to signal attenuation and strong coupling of scattered and signal photons. The authors present a boundary migration model of the scattered field, converting scattered measurements in spectral form to scene information in temporal domain, and image Lambertian objects in highly scattering media.
Collapse
|
10
|
Optical reciprocity induced wavefront shaping for axial and lateral shifting of focus through a scattering medium. Sci Rep 2022; 12:6387. [PMID: 35430597 PMCID: PMC9013373 DOI: 10.1038/s41598-022-10378-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Accepted: 03/02/2022] [Indexed: 01/02/2023] Open
Abstract
Light propagating along a reversed path experiences the same transmission coefficient as in the forward direction, independent of the path complexity. This is called the optical reciprocity of light, which is valid for not too intense scattering media as well. Hence, by utilizing the reciprocity principle, the proposed novel technique can achieve axially and laterally tunable focus, non-invasively, through a scattering media without a priori knowledge or modeling of its scattering properties. Moreover, the uniqueness of the proposed technique lies in the fact that the illumination and detection are on the same side of the scattering media.
Collapse
|
11
|
Wang H, Qiao H, Lin J, Wu R, Liu Y, Dai Q. Model Study of Transient Imaging With Multi-Frequency Time-of-Flight Sensors. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2021; 43:3523-3539. [PMID: 32191880 DOI: 10.1109/tpami.2020.2981574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
As an emerging imaging modality, transient imaging that records the transient information of light transport has significantly shaped our understanding of scenes. In spite of the great progress made in computer vision and optical imaging fields, commonly used multi-frequency time-of-flight (ToF) sensors are still afflicted with the band-limited modulation frequency and long acquisition process. To overcome such barriers, more effective image-formation schemes and reconstruction algorithms are highly desired. In this paper, we propose a compressive transient imaging model, without any priori knowledge, by constructing a near-tight-frame based representation of the ToF imaging principle. We prove that the compressibility of sensor measurements can be presented in the Fourier domain and held in the frame, and the ToF measurements possess multi-scale characteristics. Solving the inverse problems in transient imaging with our proposed model consists of two major steps, including a compressed-sensing-based approach for full measurement recovery, which essentially reduces the capture time, and a wavelet-based transient image reconstruction framework, which realizes adaptive transient image reconstruction and achieves highly accurate reconstruction results. The compressive transient imaging model is suitable for various existing multi-frequency ToF sensors and requires no hardware modifications. Experimental results using synthetic and real online datasets demonstrate its promising performance.
Collapse
|
12
|
Lai Y, Shang R, Côté CY, Liu X, Laramée A, Légaré F, Luke GP, Liang J. Compressed ultrafast tomographic imaging by passive spatiotemporal projections. OPTICS LETTERS 2021; 46:1788-1791. [PMID: 33793544 PMCID: PMC8050836 DOI: 10.1364/ol.420737] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
Existing streak-camera-based two-dimensional (2D) ultrafast imaging techniques are limited by long acquisition time, the trade-off between spatial and temporal resolutions, and a reduced field of view. They also require additional components, customization, or active illumination. Here we develop compressed ultrafast tomographic imaging (CUTI), which passively records 2D transient events with a standard streak camera. By grafting the concept of computed tomography to the spatiotemporal domain, the operations of temporal shearing and spatiotemporal integration in a streak camera's data acquisition can be equivalently expressed as the spatiotemporal projection of an (x,y,t) datacube from a certain angle. Aided by a new, to the best of our knowledge, compressed-sensing reconstruction algorithm, the 2D transient event can be accurately recovered in a few measurements. CUTI is exhibited as a new imaging mode universally adaptable to most streak cameras. Implemented in an image-converter streak camera, CUTI captures the sequential arrival of two spatially modulated ultrashort ultraviolet laser pulses at 0.5 trillion frames per second. Applied to a rotating-mirror streak camera, CUTI records an amination of fast-bouncing balls at 5,000 frames per second.
Collapse
Affiliation(s)
- Yingming Lai
- Centre Énergie Matériaux Télécommunications, Institut National de la Recherche Scientifique, 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
| | - Ruibo Shang
- Thayer School of Engineering, Dartmouth College, 14 Engineering Drive Hanover, NH 03755, USA
| | - Christian-Yves Côté
- Axis Photonique Inc., 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
| | - Xianglei Liu
- Centre Énergie Matériaux Télécommunications, Institut National de la Recherche Scientifique, 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
| | - Antoine Laramée
- Centre Énergie Matériaux Télécommunications, Institut National de la Recherche Scientifique, 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
| | - François Légaré
- Centre Énergie Matériaux Télécommunications, Institut National de la Recherche Scientifique, 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
| | - Geoffrey P. Luke
- Thayer School of Engineering, Dartmouth College, 14 Engineering Drive Hanover, NH 03755, USA
| | - Jinyang Liang
- Centre Énergie Matériaux Télécommunications, Institut National de la Recherche Scientifique, 1650 boulevard Lionel-Boulet, Varennes, Québec J3X1S2, CANADA
- Corresponding author:
| |
Collapse
|
13
|
Maruca S, Rehain P, Sua YM, Zhu S, Huang Y. Non-invasive single photon imaging through strongly scattering media. OPTICS EXPRESS 2021; 29:9981-9990. [PMID: 33820159 DOI: 10.1364/oe.417299] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 01/28/2021] [Indexed: 06/12/2023]
Abstract
Non-invasive optical imaging through opaque and multi-scattering media remains highly desirable across many application domains. The random scattering and diffusion of light in such media inflict exponential decay and aberration, prohibiting diffraction-limited imaging. By non-interferometric few picoseconds optical gating of backscattered photons, we demonstrate single photon sensitive non-invasive 3D imaging of targets occluded by strongly scattering media with optical thicknesses reaching 9.5ls (19ls round trip). It achieves diffraction-limited imaging of a target placed 130 cm away through the opaque media, with millimeter lateral and depth resolution while requiring only one photon detection out of 50,000 probe pulses. Our single photon sensitive imaging technique does not require wavefront shaping nor computationally-intensive image reconstruction algorithms, promising practical solutions for diffraction-limited imaging through highly opaque and diffusive media with low illumination power.
Collapse
|
14
|
Callenberg C, Lyons A, Brok DD, Fatima A, Turpin A, Zickus V, Machesky L, Whitelaw J, Faccio D, Hullin MB. Super-resolution time-resolved imaging using computational sensor fusion. Sci Rep 2021; 11:1689. [PMID: 33462284 PMCID: PMC7813875 DOI: 10.1038/s41598-021-81159-x] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Accepted: 12/22/2020] [Indexed: 11/29/2022] Open
Abstract
Imaging across both the full transverse spatial and temporal dimensions of a scene with high precision in all three coordinates is key to applications ranging from LIDAR to fluorescence lifetime imaging. However, compromises that sacrifice, for example, spatial resolution at the expense of temporal resolution are often required, in particular when the full 3-dimensional data cube is required in short acquisition times. We introduce a sensor fusion approach that combines data having low-spatial resolution but high temporal precision gathered with a single-photon-avalanche-diode (SPAD) array with data that has high spatial but no temporal resolution, such as that acquired with a standard CMOS camera. Our method, based on blurring the image on the SPAD array and computational sensor fusion, reconstructs time-resolved images at significantly higher spatial resolution than the SPAD input, upsampling numerical data by a factor \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$12 \times 12$$\end{document}12×12, and demonstrating up to \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$4 \times 4$$\end{document}4×4 upsampling of experimental data. We demonstrate the technique for both LIDAR applications and FLIM of fluorescent cancer cells. This technique paves the way to high spatial resolution SPAD imaging or, equivalently, FLIM imaging with conventional microscopes at frame rates accelerated by more than an order of magnitude.
Collapse
Affiliation(s)
- C Callenberg
- Institute of Computer Science, University of Bonn, Bonn, Germany
| | - A Lyons
- School of Physics & Astronomy, University of Glasgow, Glasgow, G12 8QQ, United Kingdom.
| | - D den Brok
- Institute of Computer Science, University of Bonn, Bonn, Germany
| | - A Fatima
- School of Physics & Astronomy, University of Glasgow, Glasgow, G12 8QQ, United Kingdom
| | - A Turpin
- School of Computing Science, University of Glasgow, G12 8LT, Glasgow, United Kingdom
| | - V Zickus
- School of Physics & Astronomy, University of Glasgow, Glasgow, G12 8QQ, United Kingdom
| | - L Machesky
- Cancer Research UK, Beatson Institute, Glasgow, United Kingdom
| | - J Whitelaw
- Cancer Research UK, Beatson Institute, Glasgow, United Kingdom
| | - D Faccio
- School of Physics & Astronomy, University of Glasgow, Glasgow, G12 8QQ, United Kingdom.
| | - M B Hullin
- Institute of Computer Science, University of Bonn, Bonn, Germany.
| |
Collapse
|
15
|
Radford J, Lyons A, Tonolini F, Faccio D. Role of late photons in diffuse optical imaging. OPTICS EXPRESS 2020; 28:29486-29495. [PMID: 33114848 DOI: 10.1364/oe.402503] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2020] [Accepted: 09/08/2020] [Indexed: 06/11/2023]
Abstract
The ability to image through turbid media, such as organic tissues, is a highly attractive prospect for biological and medical imaging. This is challenging, however, due to the highly scattering properties of tissues which scramble the image information. The earliest photons that arrive at the detector are often associated with ballistic transmission, whilst the later photons are associated with complex paths due to multiple independent scattering events and are therefore typically considered to be detrimental to the final image formation process. In this work, we report on the importance of these highly diffuse, "late" photons for computational time-of-flight diffuse optical imaging. In thick scattering materials, >80 transport mean free paths, we provide evidence that including late photons in the inverse retrieval enhances the image reconstruction quality. We also show that the late photons alone have sufficient information to retrieve images of a similar quality to early photon gated data. This result emphasises the importance in the strongly diffusive regime of fully time-resolved imaging techniques.
Collapse
|
16
|
Lindell DB, Wetzstein G. Three-dimensional imaging through scattering media based on confocal diffuse tomography. Nat Commun 2020; 11:4517. [PMID: 32908155 PMCID: PMC7481188 DOI: 10.1038/s41467-020-18346-3] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2020] [Accepted: 08/07/2020] [Indexed: 12/17/2022] Open
Abstract
Optical imaging techniques, such as light detection and ranging (LiDAR), are essential tools in remote sensing, robotic vision, and autonomous driving. However, the presence of scattering places fundamental limits on our ability to image through fog, rain, dust, or the atmosphere. Conventional approaches for imaging through scattering media operate at microscopic scales or require a priori knowledge of the target location for 3D imaging. We introduce a technique that co-designs single-photon avalanche diodes, ultra-fast pulsed lasers, and a new inverse method to capture 3D shape through scattering media. We demonstrate acquisition of shape and position for objects hidden behind a thick diffuser (≈6 transport mean free paths) at macroscopic scales. Our technique, confocal diffuse tomography, may be of considerable value to the aforementioned applications.
Collapse
Affiliation(s)
- David B Lindell
- Department of Electrical Engineering, Stanford University, 350 Jane Stanford Way, Stanford, CA, 94305, USA.
| | - Gordon Wetzstein
- Department of Electrical Engineering, Stanford University, 350 Jane Stanford Way, Stanford, CA, 94305, USA.
| |
Collapse
|
17
|
Hirmiz N, Tsikouras A, Osterlund EJ, Richards M, Andrews DW, Fang Q. Cross-talk reduction in a multiplexed synchroscan streak camera with simultaneous calibration. OPTICS EXPRESS 2019; 27:22602-22614. [PMID: 31510548 DOI: 10.1364/oe.27.022602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/10/2019] [Accepted: 07/14/2019] [Indexed: 06/10/2023]
Abstract
The streak camera is a picosecond resolution photodetector with parallel input capability; however, the degree of multiplexing is limited by crosstalk and temporal uncertainty in the sweeping field. We introduced a fixed time delay between adjacent fibers to reduce crosstalk in the synchroscan mode. The fixed delay and a tunable electronic delay between the input pulse and the synchroscan unit allows robust separation modes between the streaks, while spatial and temporal nonlinearities can be calibrated in. The efficacy of the design is demonstrated through a 100-fold multiplexed confocal fluorescence lifetime imaging microscope in live cells.
Collapse
|
18
|
Okamoto Y, Horisaki R, Tanida J. Noninvasive three-dimensional imaging through scattering media by three-dimensional speckle correlation. OPTICS LETTERS 2019; 44:2526-2529. [PMID: 31090723 DOI: 10.1364/ol.44.002526] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/20/2019] [Accepted: 04/18/2019] [Indexed: 06/09/2023]
Abstract
We present a method for noninvasive three-dimensional imaging through scattering media by using a three-dimensional memory effect in scattering phenomena. In the proposed method, an object in a scattering medium is reconstructed from a three-dimensional autocorrelation of speckle images captured by axially scanning an image sensor, based on a three-dimensional phase retrieval algorithm. We experimentally demonstrated our method with a lensless setup by using a three-dimensionally printed object between diffusers.
Collapse
|
19
|
Faccio D, Velten A. A trillion frames per second: the techniques and applications of light-in-flight photography. REPORTS ON PROGRESS IN PHYSICS. PHYSICAL SOCIETY (GREAT BRITAIN) 2018; 81:105901. [PMID: 29900876 DOI: 10.1088/1361-6633/aacca1] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Cameras capable of capturing videos at a trillion frames per second allow to freeze light in motion, a very counterintuitive capability when related to our everyday experience in which light appears to travel instantaneously. By combining this capability with computational imaging techniques, new imaging opportunities emerge such as 3D imaging of scenes that are hidden behind a corner, the study of relativistic distortion effects, imaging through diffusive media and imaging of ultrafast optical processes such as laser ablation, supercontinuum and plasma generation. We provide an overview of the main techniques that have been developed for ultra-high speed photography with a particular focus on 'light-in-flight' imaging, i.e. applications where the key element is the imaging of light itself at frame rates that allow to freeze its motion and therefore extract information that would otherwise be blurred out and lost.
Collapse
Affiliation(s)
- Daniele Faccio
- School of Physics & Astronomy, University of Glasgow, Glasgow G12 8QQ, United Kingdom
| | | |
Collapse
|
20
|
Angelo JP, Chen SJ, Ochoa M, Sunar U, Gioux S, Intes X. Review of structured light in diffuse optical imaging. JOURNAL OF BIOMEDICAL OPTICS 2018; 24:1-20. [PMID: 30218503 PMCID: PMC6676045 DOI: 10.1117/1.jbo.24.7.071602] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Accepted: 05/31/2018] [Indexed: 05/11/2023]
Abstract
Diffuse optical imaging probes deep living tissue enabling structural, functional, metabolic, and molecular imaging. Recently, due to the availability of spatial light modulators, wide-field quantitative diffuse optical techniques have been implemented, which benefit greatly from structured light methodologies. Such implementations facilitate the quantification and characterization of depth-resolved optical and physiological properties of thick and deep tissue at fast acquisition speeds. We summarize the current state of work and applications in the three main techniques leveraging structured light: spatial frequency-domain imaging, optical tomography, and single-pixel imaging. The theory, measurement, and analysis of spatial frequency-domain imaging are described. Then, advanced theories, processing, and imaging systems are summarized. Preclinical and clinical applications on physiological measurements for guidance and diagnosis are summarized. General theory and method development of tomographic approaches as well as applications including fluorescence molecular tomography are introduced. Lastly, recent developments of single-pixel imaging methodologies and applications are reviewed.
Collapse
Affiliation(s)
- Joseph P. Angelo
- National Institute of Standards and Technology, Sensor Science Division, Gaithersburg, Maryland, United States
- Address all correspondence to: Joseph P. Angelo, E-mail: ; Sez-Jade Chen, E-mail:
| | - Sez-Jade Chen
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
- Address all correspondence to: Joseph P. Angelo, E-mail: ; Sez-Jade Chen, E-mail:
| | - Marien Ochoa
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| | - Ulas Sunar
- Wright State University, Department of Biomedical Industrial and Human Factor Engineering, Dayton, Ohio, United States
| | - Sylvain Gioux
- University of Strasbourg, ICube Laboratory, Strasbourg, France
| | - Xavier Intes
- Rensselaer Polytechnic Institute, Department of Biomedical Engineering, Troy, New York, United States
| |
Collapse
|
21
|
Li X, Hu H, Zhao L, Wang H, Yu Y, Wu L, Liu T. Polarimetric image recovery method combining histogram stretching for underwater imaging. Sci Rep 2018; 8:12430. [PMID: 30127366 PMCID: PMC6102268 DOI: 10.1038/s41598-018-30566-8] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2018] [Accepted: 07/31/2018] [Indexed: 11/09/2022] Open
Abstract
The underwater imaging could be severely degraded by the scattering media because of the backscattered light and signal attenuation, especially in the case of strong scattering for dense turbid medium. In this paper, we propose an improved method for recovering the underwater image combining the histogram stretching and polarimetric recovery in a proper way. In this method, we stretch the histograms of the orthogonal polarization images while maintaining the polarization relation between them, and then, based on the processed orthogonal polarization images, the recovered image with higher quality can be obtained by the traditional polarimetric recovery method. Several groups of experimental results demonstrate that the quality of underwater images can be effectively enhanced by our method, and its performance is better than that of the traditional polarimetric recovery method. In particular, the proposed method is also quite effective in the condition of dense turbid medium.
Collapse
Affiliation(s)
- Xiaobo Li
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China.,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China.,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China.,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China
| | - Haofeng Hu
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China. .,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China. .,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China. .,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China.
| | - Lin Zhao
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China.,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China.,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China.,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China
| | - Hui Wang
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China.,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China.,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China.,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China
| | - Yin Yu
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China.,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China.,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China.,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China
| | - Lan Wu
- State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Zhejiang, 310027, China
| | - Tiegen Liu
- School of Precision Instrument & Opto-electronics Engineering, Tianjin University, Tianjin, 300072, China.,Institute of Optical Fiber Sensing of Tianjin University, Tianjin, 300072, China.,Tianjin Optical Fiber Sensing Engineering Center, Tianjin, 300072, China.,Key Laboratory of Opto-electronics Information Technology, Ministry of Education, Tianjin, 300072, China
| |
Collapse
|
22
|
Altmann Y, McLaughlin S, Padgett MJ, Goyal VK, Hero AO, Faccio D. Quantum-inspired computational imaging. Science 2018; 361:361/6403/eaat2298. [DOI: 10.1126/science.aat2298] [Citation(s) in RCA: 92] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Affiliation(s)
- Yoann Altmann
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, UK
| | - Stephen McLaughlin
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, UK
| | - Miles J. Padgett
- School of Physics and Astronomy, University of Glasgow, Glasgow, UK
| | - Vivek K Goyal
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - Alfred O. Hero
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA
| | - Daniele Faccio
- School of Physics and Astronomy, University of Glasgow, Glasgow, UK
| |
Collapse
|
23
|
Eckert R, Phillips ZF, Waller L. Efficient illumination angle self-calibration in Fourier ptychography. APPLIED OPTICS 2018; 57:5434-5442. [PMID: 30117837 DOI: 10.1364/ao.57.005434] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Fourier ptychography captures intensity images with varying source patterns (illumination angles) in order to computationally reconstruct large space-bandwidth-product images. Accurate knowledge of the illumination angles is necessary for good image quality; hence, calibration methods are crucial, despite often being impractical or slow. Here, we propose a fast, robust, and accurate self-calibration algorithm that uses only experimentally collected data and general knowledge of the illumination setup. First, our algorithm makes a fast direct estimate of the brightfield illumination angles based on image processing. Then, a more computationally intensive spectral correlation method is used inside the iterative solver to further refine the angle estimates of both brightfield and darkfield images. We demonstrate our method for correcting large and small misalignment artifacts in 2D and 3D Fourier ptychography with different source types: an LED array, a galvo-steered laser, and a high-NA quasi-dome LED illuminator.
Collapse
|
24
|
Tanner MG, Choudhary TR, Craven TH, Mills B, Bradley M, Henderson RK, Dhaliwal K, Thomson RR. Ballistic and snake photon imaging for locating optical endomicroscopy fibres. BIOMEDICAL OPTICS EXPRESS 2017; 8:4077-4095. [PMID: 28966848 PMCID: PMC5611924 DOI: 10.1364/boe.8.004077] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2017] [Revised: 07/27/2017] [Accepted: 07/31/2017] [Indexed: 05/10/2023]
Abstract
We demonstrate determination of the location of the distal-end of a fibre-optic device deep in tissue through the imaging of ballistic and snake photons using a time resolved single-photon detector array. The fibre was imaged with centimetre resolution, within clinically relevant settings and models. This technique can overcome the limitations imposed by tissue scattering in optically determining the in vivo location of fibre-optic medical instruments.
Collapse
Affiliation(s)
- M. G. Tanner
- Scottish Universities Physics Alliance (SUPA), Institute of Photonics and Quantum Sciences, School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh EH14 4AS, UK
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
| | - T. R. Choudhary
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
- Institute of Biological Chemistry, Biophysics and Bioengineering, School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh EH14 4AS, UK
| | - T. H. Craven
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
| | - B. Mills
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
| | - M. Bradley
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
- School of Chemistry, University of Edinburgh, Edinburgh, UK
| | - R. K. Henderson
- Institute for Integrated Micro and Nano Systems, School of Engineering, University of Edinburgh, Edinburgh EH9 3FF, UK
| | - K. Dhaliwal
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
| | - R. R. Thomson
- Scottish Universities Physics Alliance (SUPA), Institute of Photonics and Quantum Sciences, School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh EH14 4AS, UK
- EPSRC IRC Hub, MRC Centre for Inflammation Research, Queen’s Medical Research Centre, University of Edinburgh, Edinburgh UK
| |
Collapse
|
25
|
A new Monte Carlo code for light transport in biological tissue. Med Biol Eng Comput 2017; 56:649-655. [PMID: 28849546 DOI: 10.1007/s11517-017-1713-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2016] [Accepted: 08/09/2017] [Indexed: 12/14/2022]
Abstract
The aim of this work was to develop an event-by-event Monte Carlo code for light transport (called MCLTmx) to identify and quantify ballistic, diffuse, and absorbed photons, as well as their interaction coordinates inside the biological tissue. The mean free path length was computed between two interactions for scattering or absorption processes, and if necessary scatter angles were calculated, until the photon disappeared or went out of region of interest. A three-layer array (air-tissue-air) was used, forming a semi-infinite sandwich. The light source was placed at (0,0,0), emitting towards (0,0,1). The input data were: refractive indices, target thickness (0.02, 0.05, 0.1, 0.5, and 1 cm), number of particle histories, and λ from which the code calculated: anisotropy, scattering, and absorption coefficients. Validation presents differences less than 0.1% compared with that reported in the literature. The MCLTmx code discriminates between ballistic and diffuse photons, and inside of biological tissue, it calculates: specular reflection, diffuse reflection, ballistics transmission, diffuse transmission and absorption, and all parameters dependent on wavelength and thickness. The MCLTmx code can be useful for light transport inside any medium by changing the parameters that describe the new medium: anisotropy, dispersion and attenuation coefficients, and refractive indices for specific wavelength.
Collapse
|
26
|
Satat G, Tancik M, Gupta O, Heshmat B, Raskar R. Object classification through scattering media with deep learning on time resolved measurement. OPTICS EXPRESS 2017; 25:17466-17479. [PMID: 28789238 DOI: 10.1364/oe.25.017466] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/12/2017] [Accepted: 06/23/2017] [Indexed: 06/07/2023]
Abstract
We demonstrate an imaging technique that allows identification and classification of objects hidden behind scattering media and is invariant to changes in calibration parameters within a training range. Traditional techniques to image through scattering solve an inverse problem and are limited by the need to tune a forward model with multiple calibration parameters (like camera field of view, illumination position etc.). Instead of tuning a forward model and directly inverting the optical scattering, we use a data driven approach and leverage convolutional neural networks (CNN) to learn a model that is invariant to calibration parameters variations within the training range and nearly invariant beyond that. This effectively allows robust imaging through scattering conditions that is not sensitive to calibration. The CNN is trained with a large synthetic dataset generated with a Monte Carlo (MC) model that contains random realizations of major calibration parameters. The method is evaluated with a time-resolved camera and multiple experimental results are provided including pose estimation of a mannequin hidden behind a paper sheet with 23 correct classifications out of 30 tests in three poses (76.6% accuracy on real-world measurements). This approach paves the way towards real-time practical non line of sight (NLOS) imaging applications.
Collapse
|