1
|
Aziz JAB, Smith-Dryden S, E A Saleh B, Li G. Three-dimensional tomographic reconstruction using Voronoi weighting. OPTICS EXPRESS 2024; 32:20256-20267. [PMID: 38859140 DOI: 10.1364/oe.521968] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Accepted: 04/21/2024] [Indexed: 06/12/2024]
Abstract
Three-dimensional tomographic reconstruction requires careful selection of the illumination angles, often under certain measurement constraints. When the angular distribution must be nonuniform, appropriate selection of the reconstruction weights is necessary. We show that Voronoi weighting can significantly improve the fidelity of optical diffraction tomography.
Collapse
|
2
|
Liu Y, Xiao W, Xiao X, Wang H, Peng R, Feng Y, Zhao Q, Pan F. Dynamic tracking of onion-like carbon nanoparticles in cancer cells using limited-angle holographic tomography with self-supervised learning. BIOMEDICAL OPTICS EXPRESS 2024; 15:3076-3091. [PMID: 38855692 PMCID: PMC11161346 DOI: 10.1364/boe.522563] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 04/03/2024] [Accepted: 04/04/2024] [Indexed: 06/11/2024]
Abstract
This research presents a novel approach for the dynamic monitoring of onion-like carbon nanoparticles inside colorectal cancer cells. Onion-like carbon nanoparticles are widely used in photothermal cancer therapy, and precise 3D tracking of their distribution is crucial. We proposed a limited-angle digital holographic tomography technique with unsupervised learning to achieve rapid and accurate monitoring. A key innovation is our internal learning neural network. This network addresses the information limitations of limited-angle measurements by directly mapping coordinates to measured data and reconstructing phase information at unmeasured angles without external training data. We validated the network using standard SiO2 microspheres. Subsequently, we reconstructed the 3D refractive index of onion-like carbon nanoparticles within cancer cells at various time points. Morphological parameters of the nanoparticles were quantitatively analyzed to understand their temporal evolution, offering initial insights into the underlying mechanisms. This methodology provides a new perspective for efficiently tracking nanoparticles within cancer cells.
Collapse
Affiliation(s)
- Yakun Liu
- Key Laboratory of Precision Opto-mechatronics Technology, School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
| | - Wen Xiao
- Key Laboratory of Precision Opto-mechatronics Technology, School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
| | - Xi Xiao
- Department of Radiation Oncology, Peking University Third Hospital, Beijing 100191, China
| | - Hao Wang
- Department of Radiation Oncology, Peking University Third Hospital, Beijing 100191, China
- Cancer Center, Peking University Third Hospital, Beijing 100191, China
| | - Ran Peng
- Department of Radiation Oncology, Peking University Third Hospital, Beijing 100191, China
| | - Yuchen Feng
- Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Beijing 100190, China
| | - Qi Zhao
- Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Beijing 100190, China
| | - Feng Pan
- Key Laboratory of Precision Opto-mechatronics Technology, School of Instrumentation and Optoelectronic Engineering, Beihang University, Beijing 100191, China
| |
Collapse
|
3
|
Kim H, Jun T, Lee H, Chae BG, Yoon M, Kim C. Deep-learning based 3D birefringence image generation using 2D multi-view holographic images. Sci Rep 2024; 14:9879. [PMID: 38684698 PMCID: PMC11059389 DOI: 10.1038/s41598-024-60023-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 04/17/2024] [Indexed: 05/02/2024] Open
Abstract
Refractive index stands as an inherent characteristic of a material, allowing non-invasive exploration of the three-dimensional (3D) interior of the material. Certain materials with different refractive indices produce a birefringence phenomenon in which incident light is split into two polarization components when it passes through the materials. Representative birefringent materials appear in calcite crystals, liquid crystals (LCs), biological tissues, silk fibers, polymer films, etc. If the internal 3D shape of these materials can be visually expressed through a non-invasive method, it can greatly contribute to the semiconductor, display industry, optical components and devices, and biomedical diagnosis. This paper introduces a novel approach employing deep learning to generate 3D birefringence images using multi-viewed holographic interference images. First, we acquired a set of multi-viewed holographic interference pattern images and a 3D volume image of birefringence directly from a polarizing DTT (dielectric tensor tomography)-based microscope system about each LC droplet sample. The proposed model was trained to generate the 3D volume images of birefringence using the two-dimensional (2D) interference pattern image set. Performance evaluations were conducted against the ground truth images obtained directly from the DTT microscopy. Visualization techniques were applied to describe the refractive index distribution in the generated 3D images of birefringence. The results show the proposed method's efficiency in generating the 3D refractive index distribution from multi-viewed holographic interference images, presenting a novel data-driven alternative to traditional methods from the DTT devices.
Collapse
Affiliation(s)
- Hakdong Kim
- Department of Digital Contents, Sejong University, Seoul, Korea
| | - Taeheul Jun
- Department of Software Convergence, Sejong University, Seoul, Korea
| | - Hyoung Lee
- Communication and Media Research Lab, ETRI, Daejeon, Korea
| | - Byung Gyu Chae
- Communication and Media Research Lab, ETRI, Daejeon, Korea
| | - MinSung Yoon
- Communication and Media Research Lab, ETRI, Daejeon, Korea.
| | - Cheongwon Kim
- Department of Software, Sejong University, Seoul, Korea.
| |
Collapse
|
4
|
Yang Z, Zhang L, Liu T, Wang H, Tang Z, Zhao H, Yuan L, Zhang Z, Liu X. Alternating projection combined with fast gradient projection (FGP-AP) method for intensity-only measurement optical diffraction tomography in LED array microscopy. BIOMEDICAL OPTICS EXPRESS 2024; 15:2524-2542. [PMID: 38633101 PMCID: PMC11019679 DOI: 10.1364/boe.518955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/06/2024] [Accepted: 03/11/2024] [Indexed: 04/19/2024]
Abstract
Optical diffraction tomography (ODT) is a powerful label-free measurement tool that can quantitatively image the three-dimensional (3D) refractive index (RI) distribution of samples. However, the inherent "missing cone problem," limited illumination angles, and dependence on intensity-only measurements in a simplified imaging setup can all lead to insufficient information mapping in the Fourier domain, affecting 3D reconstruction results. In this paper, we propose the alternating projection combined with the fast gradient projection (FGP-AP) method to compensate for the above problem, which effectively reconstructs the 3D RI distribution of samples using intensity-only images captured from LED array microscopy. The FGP-AP method employs the alternating projection (AP) algorithm for gradient descent and the fast gradient projection (FGP) algorithm for regularization constraints. This approach is equivalent to incorporating prior knowledge of sample non-negativity and smoothness into the 3D reconstruction process. Simulations demonstrate that the FGP-AP method improves reconstruction quality compared to the original AP method, particularly in the presence of noise. Experimental results, obtained from mouse kidney cells and label-free blood cells, further affirm the superior 3D imaging efficacy of the FGP-AP method.
Collapse
Affiliation(s)
- Zewen Yang
- State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - Lu Zhang
- State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an 710049, China
- School of Instrument Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
| | - Tong Liu
- State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - Huijun Wang
- State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - Zhiyuan Tang
- Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China
| | - Hong Zhao
- State Key Laboratory for Manufacturing System Engineering, Xi’an Jiaotong University, Xi’an 710049, China
- School of Instrument Science and Technology, Xi’an Jiaotong University, Xi’an 710049, China
| | - Li Yuan
- First Affiliated Hospital, Xi’an Jiaotong University, Xi’an, Shannxi, 710049, China
| | - Zhenxi Zhang
- Key Laboratory of Biomedical Information Engineering of Ministry of Education, Xi’an Jiaotong University, Xi’an 710049, China
| | - Xiaolong Liu
- Mengchao Hepatobiliary Hospital of Fujian Medical University, The United Innovation of Mengchao Hepatobiliary Technology Key Laboratory of Fujian Provincey, Fuzhou 350025, China
| |
Collapse
|
5
|
Hong SJ, Hou JU, Chung MJ, Kang SH, Shim BS, Lee SL, Park DH, Choi A, Oh JY, Lee KJ, Shin E, Cho E, Park SW. Convolutional neural network model for automatic recognition and classification of pancreatic cancer cell based on analysis of lipid droplet on unlabeled sample by 3D optical diffraction tomography. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 246:108041. [PMID: 38325025 DOI: 10.1016/j.cmpb.2024.108041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 01/05/2024] [Accepted: 01/19/2024] [Indexed: 02/09/2024]
Abstract
INTRODUCTION Pancreatic cancer cells generally accumulate large numbers of lipid droplets (LDs), which regulate lipid storage. To promote rapid diagnosis, an automatic pancreatic cancer cell recognition system based on a deep convolutional neural network was proposed in this study using quantitative images of LDs from stain-free cytologic samples by optical diffraction tomography. METHODS We retrieved 3D refractive index tomograms and reconstructed 37 optical images of one cell. From the four cell lines, the obtained fields were separated into training and test datasets with 10,397 and 3,478 images, respectively. Furthermore, we adopted several machine learning techniques based on a single image-based prediction model to improve the performance of the computer-aided diagnostic system. RESULTS Pancreatic cancer cells had a significantly lower total cell volume and dry mass than did normal pancreatic cells and were accompanied by greater numbers of lipid droplets (LDs). When evaluating multitask learning techniques utilizing the EfficientNet-b3 model through confusion matrices, the overall 2-category accuracy for cancer classification reached 96.7 %. Simultaneously, the overall 4-category accuracy for individual cell line classification achieved a high accuracy of 96.2 %. Furthermore, when we added the core techniques one by one, the overall performance of the proposed technique significantly improved, reaching an area under the curve (AUC) of 0.997 and an accuracy of 97.06 %. Finally, the AUC reached 0.998 through the ablation study with the score fusion technique. DISCUSSION Our novel training strategy has significant potential for automating and promoting rapid recognition of pancreatic cancer cells. In the near future, deep learning-embedded medical devices will substitute laborious manual cytopathologic examinations for sustainable economic potential.
Collapse
Affiliation(s)
- Seok Jin Hong
- Department of Otolaryngology-Head and Neck Surgery, Kangbuk Samsung Hospital Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Jong-Uk Hou
- School of Software, Hallym University, Chuncheon, Republic of Korea
| | - Moon Jae Chung
- Division of Gastroenterology, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Sung Hun Kang
- Department of Otolaryngology-Head and Neck Surgery, Kangbuk Samsung Hospital Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
| | - Bo-Seok Shim
- School of Software, Hallym University, Chuncheon, Republic of Korea
| | - Seung-Lee Lee
- School of Software, Hallym University, Chuncheon, Republic of Korea
| | - Da Hae Park
- Division of Gastroenterology, Department of Internal Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, 7, Keunjaebong-gil, Hwaseong-si, Gyeonggi-do 18450, Republic of Korea
| | - Anna Choi
- Division of Gastroenterology, Department of Internal Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, 7, Keunjaebong-gil, Hwaseong-si, Gyeonggi-do 18450, Republic of Korea
| | - Jae Yeon Oh
- Hallym University College of Medicine, Chuncheon, Republic of Korea
| | - Kyong Joo Lee
- Division of Gastroenterology, Department of Internal Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, 7, Keunjaebong-gil, Hwaseong-si, Gyeonggi-do 18450, Republic of Korea
| | - Eun Shin
- Department of Pathology, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, Hwaseong, Republic of Korea
| | - Eunae Cho
- Division of Gastroenterology, Department of Internal Medicine, Chonnam National University Hospital, Gwangju, Republic of Korea
| | - Se Woo Park
- Division of Gastroenterology, Department of Internal Medicine, Hallym University Dongtan Sacred Heart Hospital, Hallym University College of Medicine, 7, Keunjaebong-gil, Hwaseong-si, Gyeonggi-do 18450, Republic of Korea.
| |
Collapse
|
6
|
Haputhanthri U, Herath K, Hettiarachchi R, Kariyawasam H, Ahmad A, Ahluwalia BS, Acharya G, Edussooriya CUS, Wadduwage DN. Towards ultrafast quantitative phase imaging via differentiable microscopy [Invited]. BIOMEDICAL OPTICS EXPRESS 2024; 15:1798-1812. [PMID: 38495703 PMCID: PMC10942716 DOI: 10.1364/boe.504954] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 12/15/2023] [Accepted: 02/09/2024] [Indexed: 03/19/2024]
Abstract
With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. Despite significant advances in fast multiplexed imaging sensors and deep-learning-based inverse solvers, the throughput of QPM is currently limited by the pixel-rate of the image sensors. Complementarily, to improve throughput further, here we propose to acquire images in a compressed form so that more information can be transferred beyond the existing hardware bottleneck of the image sensor. To this end, we present a numerical simulation of a learnable optical compression-decompression framework that learns content-specific features. The proposed differentiable quantitative phase microscopy (∂-QPM) first uses learnable optical processors as image compressors. The intensity representations produced by these optical processors are then captured by the imaging sensor. Finally, a reconstruction network running on a computer decompresses the QPM images post aquisition. In numerical experiments, the proposed system achieves compression of × 64 while maintaining the SSIM of ∼0.90 and PSNR of ∼30 dB on cells. The results demonstrated by our experiments open up a new pathway to QPM systems that may provide unprecedented throughput improvements.
Collapse
Affiliation(s)
- Udith Haputhanthri
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
- Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka
| | - Kithmini Herath
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
- Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka
| | - Ramith Hettiarachchi
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
- Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka
| | - Hasindu Kariyawasam
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
- Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka
| | - Azeem Ahmad
- Department of Physics and Technology, UiT The Arctic University of Norway, Tromsø, 9037, Norway
| | - Balpreet S. Ahluwalia
- Department of Physics and Technology, UiT The Arctic University of Norway, Tromsø, 9037, Norway
| | - Ganesh Acharya
- Division of Obstetrics and Gynecology, Department of Clinical Science, Intervention and Technology, Karolinska Institute, Stockholm, Sweden
| | | | - Dushan N. Wadduwage
- Center for Advanced Imaging, Faculty of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA
| |
Collapse
|
7
|
Verrier N, Debailleul M, Haeberlé O. Recent Advances and Current Trends in Transmission Tomographic Diffraction Microscopy. SENSORS (BASEL, SWITZERLAND) 2024; 24:1594. [PMID: 38475130 DOI: 10.3390/s24051594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Revised: 02/21/2024] [Accepted: 02/27/2024] [Indexed: 03/14/2024]
Abstract
Optical microscopy techniques are among the most used methods in biomedical sample characterization. In their more advanced realization, optical microscopes demonstrate resolution down to the nanometric scale. These methods rely on the use of fluorescent sample labeling in order to break the diffraction limit. However, fluorescent molecules' phototoxicity or photobleaching is not always compatible with the investigated samples. To overcome this limitation, quantitative phase imaging techniques have been proposed. Among these, holographic imaging has demonstrated its ability to image living microscopic samples without staining. However, for a 3D assessment of samples, tomographic acquisitions are needed. Tomographic Diffraction Microscopy (TDM) combines holographic acquisitions with tomographic reconstructions. Relying on a 3D synthetic aperture process, TDM allows for 3D quantitative measurements of the complex refractive index of the investigated sample. Since its initial proposition by Emil Wolf in 1969, the concept of TDM has found a lot of applications and has become one of the hot topics in biomedical imaging. This review focuses on recent achievements in TDM development. Current trends and perspectives of the technique are also discussed.
Collapse
Affiliation(s)
- Nicolas Verrier
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| | - Matthieu Debailleul
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| | - Olivier Haeberlé
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| |
Collapse
|
8
|
Lugo MC, Saito M, Kitamura M, Ide Y, Koide S, Mayama S. Imaging and Quantitative Analysis on the Etching of Diatom Frustules via Digital Holographic Microscopy. ACS Biomater Sci Eng 2024; 10:1106-1111. [PMID: 38154034 DOI: 10.1021/acsbiomaterials.3c01349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2023]
Abstract
Frustules, whose length spans from a few micrometers to more than a hundred micrometers, have been the subject of various modifications to improve their physical properties because of their complex porous silica structure. However, three-dimensional measurements of these changes can be challenging because of the complex 3D architecture and limitations of known methods. In this study, we present a new method that applies digital holographic microscopy (DHM) to analyze controlled etched frustules and observe real-time degradation of frustules at the single-cell level. Frustules obtained from Craspedostauros sp. diatoms were etched in 1 N NaOH for 5 min at 25 and 60 °C, respectively, and the frustule's valve was analyzed using DHM. DHM uses a combination of holography and tomography to reconstruct a 3D refractive index image of the frustule. Measurements of the width, volume, and surface area are achieved. Results showed that at 60 °C of etching, a significant difference with the unetched frustule was observed for all measurements but with high fluctuation values. Finally, real-time observation of the degradation of the frustule is observed when immersed in a high concentration of NaOH. This is the first time the real-time etching of the frustule is observed at the single-cell level. This research provides an easy estimation of the 3D measurements of frustules that may provide new fundamental information and applications.
Collapse
Affiliation(s)
- Maria Christine Lugo
- Department of Physics, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku, Tokyo 162-8601, Japan
| | - Makoto Saito
- Department of Physics, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku, Tokyo 162-8601, Japan
| | - Masaki Kitamura
- Department of Physics, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku, Tokyo 162-8601, Japan
| | - Yuki Ide
- Department of Physics, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku, Tokyo 162-8601, Japan
| | - Shinji Koide
- Department of Physics, Tokyo University of Science, 1-3 Kagurazaka, Shinjuku, Tokyo 162-8601, Japan
| | - Shigeki Mayama
- Tokyo Diatomology Lab, 2-3-2 Nukuikitamachi, Koganei, Tokyo 184-0015, Japan
| |
Collapse
|
9
|
Dunn KJ, Matlock A, Funkenbusch G, Yaqoob Z, So PTC, Berger AJ. Optical diffraction tomography for assessing single cell models in angular light scattering. BIOMEDICAL OPTICS EXPRESS 2024; 15:973-990. [PMID: 38404316 PMCID: PMC10890861 DOI: 10.1364/boe.512149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 01/09/2024] [Accepted: 01/09/2024] [Indexed: 02/27/2024]
Abstract
Angularly resolved light scattering (ALS) has become a useful tool for assessing the size and refractive index of biological scatterers at cellular and organelle length scales. Sizing organelle populations with ALS relies on Mie scattering theory models, which require significant assumptions about the object, including spherical scatterers and a homogeneous medium. These assumptions may incur greater error at the single cell level, where there are fewer scatterers to be averaged over. We investigate the validity of these assumptions using 3D refractive index (RI) tomograms measured via optical diffraction tomography (ODT). We compute the angular scattering on digitally manipulated tomograms with increasingly strong model assumptions, including RI-matched immersion media, homogeneous cytosol, and spherical organelles. We also compare the tomogram-computed angular scattering to experimental measurements of angular scattering from the same cells to ensure that the ODT-based approach accurately models angular scattering. We show that enforced RI-matching with the immersion medium and a homogeneous cytosol significantly affects the angular scattering intensity shape, suggesting that these assumptions can reduce the accuracy of size distribution estimates.
Collapse
Affiliation(s)
- Kaitlin J. Dunn
- The Institute of Optics, University of Rochester, Rochester, NY, USA
| | - Alex Matlock
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | - Zahid Yaqoob
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Peter T. C. So
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Andrew J. Berger
- The Institute of Optics, University of Rochester, Rochester, NY, USA
| |
Collapse
|
10
|
Sun J, Yang B, Koukourakis N, Guck J, Czarske JW. AI-driven projection tomography with multicore fibre-optic cell rotation. Nat Commun 2024; 15:147. [PMID: 38167247 PMCID: PMC10762230 DOI: 10.1038/s41467-023-44280-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 12/06/2023] [Indexed: 01/05/2024] Open
Abstract
Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.
Collapse
Affiliation(s)
- Jiawei Sun
- Shanghai Artificial Intelligence Laboratory, Longwen Road 129, Xuhui District, 200232, Shanghai, China.
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany.
| | - Bin Yang
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany
| | - Nektarios Koukourakis
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany
| | - Jochen Guck
- Max Planck Institute for the Science of Light & Max Planck-Zentrum für Physik und Medizin, 91058, Erlangen, Germany
| | - Juergen W Czarske
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany.
- Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany.
- Institute of Applied Physics, TU Dresden, Dresden, Germany.
| |
Collapse
|
11
|
Liu R, Wen K, Li J, Ma Y, Zheng J, An S, Min J, Zalevsky Z, Yao B, Gao P. Multi-harmonic structured illumination-based optical diffraction tomography. APPLIED OPTICS 2023; 62:9199-9206. [PMID: 38108690 DOI: 10.1364/ao.508138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 11/08/2023] [Indexed: 12/19/2023]
Abstract
Imaging speed and spatial resolution are key factors in optical diffraction tomography (ODT), while they are mutually exclusive in 3D refractive index imaging. This paper presents a multi-harmonic structured illumination-based optical diffraction tomography (MHSI-ODT) to acquire 3D refractive index (RI) maps of transparent samples. MHSI-ODT utilizes a digital micromirror device (DMD) to generate structured illumination containing multiple harmonics. For each structured illumination orientation, four spherical spectral crowns are solved from five phase-shifted holograms, meaning that the acquisition of each spectral crown costs 1.25 raw images. Compared to conventional SI-ODT, which retrieves two spectral crowns from three phase-shifted raw images, MHSI-ODT enhances the imaging speed by 16.7% in 3D RI imaging. Meanwhile, MHSI-ODT exploits both the 1st-order and the 2nd-order harmonics; therefore, it has a better intensity utilization of structured illumination. We demonstrated the performance of MHSI-ODT by rendering the 3D RI distributions of 5 µm polystyrene (PS) microspheres and biological samples.
Collapse
|
12
|
Park J, Bai B, Ryu D, Liu T, Lee C, Luo Y, Lee MJ, Huang L, Shin J, Zhang Y, Ryu D, Li Y, Kim G, Min HS, Ozcan A, Park Y. Artificial intelligence-enabled quantitative phase imaging methods for life sciences. Nat Methods 2023; 20:1645-1660. [PMID: 37872244 DOI: 10.1038/s41592-023-02041-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2023] [Accepted: 09/11/2023] [Indexed: 10/25/2023]
Abstract
Quantitative phase imaging, integrated with artificial intelligence, allows for the rapid and label-free investigation of the physiology and pathology of biological systems. This review presents the principles of various two-dimensional and three-dimensional label-free phase imaging techniques that exploit refractive index as an intrinsic optical imaging contrast. In particular, we discuss artificial intelligence-based analysis methodologies for biomedical studies including image enhancement, segmentation of cellular or subcellular structures, classification of types of biological samples and image translation to furnish subcellular and histochemical information from label-free phase images. We also discuss the advantages and challenges of artificial intelligence-enabled quantitative phase imaging analyses, summarize recent notable applications in the life sciences, and cover the potential of this field for basic and industrial research in the life sciences.
Collapse
Affiliation(s)
- Juyeon Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - Bijie Bai
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
- Bioengineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | - DongHun Ryu
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Tairan Liu
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | - Chungha Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - Yi Luo
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | - Mahn Jae Lee
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
- Graduate School of Medical Science and Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Luzhe Huang
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | - Jeongwon Shin
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
| | - Yijie Zhang
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | | | - Yuzhu Li
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA
| | - Geon Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | | | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, Los Angeles, CA, USA.
- Bioengineering Department, University of California, Los Angeles, Los Angeles, CA, USA.
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea.
- Tomocube, Daejeon, Republic of Korea.
| |
Collapse
|
13
|
Lee M, Kunzi M, Neurohr G, Lee SS, Park Y. Hybrid machine-learning framework for volumetric segmentation and quantification of vacuoles in individual yeast cells using holotomography. BIOMEDICAL OPTICS EXPRESS 2023; 14:4567-4578. [PMID: 37791265 PMCID: PMC10545186 DOI: 10.1364/boe.498475] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 07/23/2023] [Accepted: 07/31/2023] [Indexed: 10/05/2023]
Abstract
The precise, quantitative evaluation of intracellular organelles in three-dimensional (3D) imaging data poses a significant challenge due to the inherent constraints of traditional microscopy techniques, the requirements of the use of exogenous labeling agents, and existing computational methods. To counter these challenges, we present a hybrid machine-learning framework exploiting correlative imaging of 3D quantitative phase imaging with 3D fluorescence imaging of labeled cells. The algorithm, which synergistically integrates a random-forest classifier with a deep neural network, is trained using the correlative imaging data set, and the trained network is then applied to 3D quantitative phase imaging of cell data. We applied this method to live budding yeast cells. The results revealed precise segmentation of vacuoles inside individual yeast cells, and also provided quantitative evaluations of biophysical parameters, including volumes, concentration, and dry masses of automatically segmented vacuoles.
Collapse
Affiliation(s)
- Moosung Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Current affiliation: Institute for Functional Matter and Quantum Technologies, Universität Stuttgart, 70569 Stuttgart, Germany
| | - Marina Kunzi
- Institute for Biochemistry, Department of Biology, ETH Zürich, 8093 Zürich, Switzerland
- Bringing Materials to Life Initiative, ETH Zürich, Zürich, Switzerland
| | - Gabriel Neurohr
- Institute for Biochemistry, Department of Biology, ETH Zürich, 8093 Zürich, Switzerland
- Bringing Materials to Life Initiative, ETH Zürich, Zürich, Switzerland
| | - Sung Sik Lee
- Institute for Biochemistry, Department of Biology, ETH Zürich, 8093 Zürich, Switzerland
- Bringing Materials to Life Initiative, ETH Zürich, Zürich, Switzerland
- ScopeM (Scientific Center of Optical and Electron Microscopy), ETH Zürich, 8093, Zurich, Switzerland
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Tomocube Inc., Daejeon 34051, Republic of Korea
| |
Collapse
|
14
|
Nyakuchena M, Juntunen C, Shea P, Sung Y. Refractive index dispersion measurement in the short-wave infrared range using synthetic phase microscopy. Phys Chem Chem Phys 2023; 25:23141-23149. [PMID: 37603384 DOI: 10.1039/d3cp03158f] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/22/2023]
Abstract
Refractive index is an optical property explored in the light scattering measurement of micro- and nano-particles as well as in label-free imaging of cells and tissues. Because the refractive index value is a major input to the characterization and quantification of the analyzed specimens, various methods have been developed targeting at different sample types. In this paper, we demonstrate a technique for the refractive index measurement of homogeneous microspheres and liquids in the short-wave infrared (SWIR) range. We use synthetic phase microscopy (SPM), which records a scattering-corrected projection of the 3D refractive index distribution, in combination with a least-squares fitting to a theoretical model of a sphere. Using the method, we determine the refractive index dispersion of two polymer microspheres (polymethyl methacrylate and polystyrene), two glass microspheres (silica and soda lime), and three microscopy mounting media (glycerol, FluorSave, and Eukitt) in the SWIR range of 1100-1650 nm.
Collapse
Affiliation(s)
- Melisa Nyakuchena
- University of Wisconsin-Milwaukee, 3200 N Cramer St, Milwaukee, WI, USA.
| | - Cory Juntunen
- University of Wisconsin-Milwaukee, 3200 N Cramer St, Milwaukee, WI, USA.
| | - Peter Shea
- University of Wisconsin-Milwaukee, 3200 N Cramer St, Milwaukee, WI, USA.
| | - Yongjin Sung
- University of Wisconsin-Milwaukee, 3200 N Cramer St, Milwaukee, WI, USA.
| |
Collapse
|
15
|
Kang S, Zhou R, Brelen M, Mak HK, Lin Y, So PTC, Yaqoob Z. Mapping nanoscale topographic features in thick tissues with speckle diffraction tomography. LIGHT, SCIENCE & APPLICATIONS 2023; 12:200. [PMID: 37607903 PMCID: PMC10444882 DOI: 10.1038/s41377-023-01240-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 07/11/2023] [Accepted: 07/19/2023] [Indexed: 08/24/2023]
Abstract
Resolving three-dimensional morphological features in thick specimens remains a significant challenge for label-free imaging. We report a new speckle diffraction tomography (SDT) approach that can image thick biological specimens with ~500 nm lateral resolution and ~1 μm axial resolution in a reflection geometry. In SDT, multiple-scattering background is rejected through spatiotemporal gating provided by dynamic speckle-field interferometry, while depth-resolved refractive index maps are reconstructed by developing a comprehensive inverse-scattering model that also considers specimen-induced aberrations. Benefiting from the high-resolution and full-field quantitative imaging capabilities of SDT, we successfully imaged red blood cells and quantified their membrane fluctuations behind a turbid medium with a thickness of 2.8 scattering mean-free paths. Most importantly, we performed volumetric imaging of cornea inside an ex vivo rat eye and quantified its optical properties, including the mapping of nanoscale topographic features of Dua's and Descemet's membranes that had not been previously visualized.
Collapse
Affiliation(s)
- Sungsam Kang
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Renjie Zhou
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| | - Marten Brelen
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Heather K Mak
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Yuechuan Lin
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Peter T C So
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Zahid Yaqoob
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
- Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA.
| |
Collapse
|
16
|
Zhao J, Jiang L, Matlock A, Xu Y, Zhu J, Zhu H, Tian L, Wolozin B, Cheng JX. Mid-infrared chemical imaging of intracellular tau fibrils using fluorescence-guided computational photothermal microscopy. LIGHT, SCIENCE & APPLICATIONS 2023; 12:147. [PMID: 37322011 PMCID: PMC10272128 DOI: 10.1038/s41377-023-01191-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 05/18/2023] [Accepted: 05/21/2023] [Indexed: 06/17/2023]
Abstract
Amyloid proteins are associated with a broad spectrum of neurodegenerative diseases. However, it remains a grand challenge to extract molecular structure information from intracellular amyloid proteins in their native cellular environment. To address this challenge, we developed a computational chemical microscope integrating 3D mid-infrared photothermal imaging with fluorescence imaging, termed Fluorescence-guided Bond-Selective Intensity Diffraction Tomography (FBS-IDT). Based on a low-cost and simple optical design, FBS-IDT enables chemical-specific volumetric imaging and 3D site-specific mid-IR fingerprint spectroscopic analysis of tau fibrils, an important type of amyloid protein aggregates, in their intracellular environment. Label-free volumetric chemical imaging of human cells with/without seeded tau fibrils is demonstrated to show the potential correlation between lipid accumulation and tau aggregate formation. Depth-resolved mid-infrared fingerprint spectroscopy is performed to reveal the protein secondary structure of the intracellular tau fibrils. 3D visualization of the β-sheet for tau fibril structure is achieved.
Collapse
Affiliation(s)
- Jian Zhao
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA.
- The Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA, 02142, USA.
| | - Lulu Jiang
- Department of Pharmacology and Experimental Therapeutics, Boston University School of Medicine, Boston, MA, 02118, USA
| | - Alex Matlock
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02142, USA
| | - Yihong Xu
- Department of Physics, Boston University, Boston, MA, 02215, USA
| | - Jiabei Zhu
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA
| | - Hongbo Zhu
- State Key Laboratory of Luminescence and Applications, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 130033, Changchun, China
| | - Lei Tian
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA
- Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA
| | - Benjamin Wolozin
- Department of Pharmacology and Experimental Therapeutics, Boston University School of Medicine, Boston, MA, 02118, USA
| | - Ji-Xin Cheng
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, 02215, USA.
- Department of Physics, Boston University, Boston, MA, 02215, USA.
- Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA.
- Photonics Center, Boston University, Boston, MA, 02215, USA.
| |
Collapse
|
17
|
Barsanti L, Birindelli L, Sbrana F, Lombardi G, Gualtieri P. Advanced Microscopy Techniques for Molecular Biophysics. Int J Mol Sci 2023; 24:9973. [PMID: 37373120 DOI: 10.3390/ijms24129973] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 05/31/2023] [Accepted: 06/05/2023] [Indexed: 06/29/2023] Open
Abstract
Though microscopy is most often intended as a technique for providing qualitative assessment of cellular and subcellular properties, when coupled with other instruments such as wavelength selectors, lasers, photoelectric devices and computers, it can perform a wide variety of quantitative measurements, which are demanding in establishing relationships between the properties and structures of biological material in all their spatial and temporal complexities. These combinations of instruments are a powerful approach to improve non-destructive investigations of cellular and subcellular properties (both physical and chemical) at a macromolecular scale resolution. Since many subcellular compartments in living cells are characterized by structurally organized molecules, this review deals with three advanced microscopy techniques well-suited for these kind of investigations, i.e., microspectrophotometry (MSP), super-resolution localization microscopy (SRLM) and holotomographic microscopy (HTM). These techniques can achieve an insight view into the role intracellular molecular organizations such as photoreceptive and photosynthetic structures and lipid bodies play in many cellular processes as well as their biophysical properties. Microspectrophotometry uses a set-up based on the combination of a wide-field microscope and a polychromator, which allows the measurement of spectroscopic features such as absorption spectra. Super resolution localization microscopy combines dedicated optics and sophisticated software algorithms to overcome the diffraction limit of light and allow the visualization of subcellular structures and dynamics in greater detail with respect to conventional optical microscopy. Holotomographic microscopy combines holography and tomography techniques into a single microscopy set-up, and allows 3D reconstruction by means of the phase separation of biomolecule condensates. This review is organized in sections, which for each technique describe some general aspects, a peculiar theoretical aspect, a specific experimental configuration and examples of applications (fish and algae photoreceptors, single labeled proteins and endocellular aggregates of lipids).
Collapse
Affiliation(s)
- Laura Barsanti
- Istituto di Biofisica, CNR, Via Moruzzi 1, 56124 Pisa, Italy
| | | | | | - Giovanni Lombardi
- Istituto di Scienza e Tecnologia dell'Informazione, CNR, Via Moruzzi 1, 56124 Pisa, Italy
| | - Paolo Gualtieri
- Istituto di Biofisica, CNR, Via Moruzzi 1, 56124 Pisa, Italy
| |
Collapse
|
18
|
Pan Y, Smith ZJ, Chu K. Image reconstruction for low cost spatial light interference microscopy with fixed and arbitrary phase modulation. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2023; 40:1155-1164. [PMID: 37706768 DOI: 10.1364/josaa.485557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 04/06/2023] [Indexed: 09/15/2023]
Abstract
During the past decade, spatial light interference microscopy (SLIM) has undergone rapid development, evidenced by its broadening applications in biology and medicine. However, the need for an expensive spatial light modulator (SLM) may limit its adoption, and the requirement for multiple images per plane limits its speed in volumetric imaging. Here we propose to address these issues by replacing the SLM with a mask fabricated from a low cost optical density (OD) filter, and recover high contrast images computationally rather than through phase-shifting. This is done using a specially constructed Wiener filter to recover the object scattering potential. A crucial part of the Wiener filter is estimating the arbitrary phase introduced by the OD filter. Our results demonstrate that not only were we able to estimate the OD filter's phase modulation in situ, but also the contrast of the reconstructed images is greatly improved. Comparisons with other related methods are also performed, with the conclusion that the combination of an inexpensive OD mask and modified Wiener filtering leads to results that are closest to the traditional SLIM setup. Thus, we have demonstrated the feasibility of a low cost, high speed SLIM system utilizing computational phase reconstruction, paving the way for wider adoption of high resolution phase microscopy.
Collapse
|
19
|
Moser S, Jesacher A, Ritsch-Marte M. Efficient and accurate intensity diffraction tomography of multiple-scattering samples. OPTICS EXPRESS 2023; 31:18274-18289. [PMID: 37381541 DOI: 10.1364/oe.486296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/25/2023] [Indexed: 06/30/2023]
Abstract
Optical Diffraction Tomography (ODT) is a label-free method to quantitatively estimate the 3D refractive index (RI) distributions of microscopic samples. Recently, significant efforts were directed towards methods to model multiple-scattering objects. The fidelity of reconstructions rely on accurately modelling light-matter interactions, but the efficient simulation of light propagation through high-RI structures over a large range of illumination angles is still challenging. Here we present a solution dealing with these problems, proposing a method that allows one to efficiently model the tomographic image formation for strongly scattering objects illuminated over a wide range of angles. Instead of propagating tilted plane waves we apply rotations on the illuminated object and optical field and formulate a new and robust multi-slice model suitable for high-RI contrast structures. We test reconstructions made by our approach against simulations and experiments, using rigorous solutions to Maxwell's equations as ground truth. We find the proposed method to produce reconstructions of higher fidelity compared to conventional multi-slice methods, especially for the challenging case of strongly scattering samples where conventional reconstruction methods fail.
Collapse
|
20
|
Kim T, Yoo J, Do S, Hwang DS, Park Y, Shin Y. RNA-mediated demixing transition of low-density condensates. Nat Commun 2023; 14:2425. [PMID: 37105967 PMCID: PMC10140143 DOI: 10.1038/s41467-023-38118-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2021] [Accepted: 04/17/2023] [Indexed: 04/29/2023] Open
Abstract
Biomolecular condensates play a key role in organizing cellular reactions by concentrating a specific set of biomolecules. However, whether condensate formation is accompanied by an increase in the total mass concentration within condensates or by the demixing of already highly crowded intracellular components remains elusive. Here, using refractive index imaging, we quantify the mass density of several condensates, including nucleoli, heterochromatin, nuclear speckles, and stress granules. Surprisingly, the latter two condensates exhibit low densities with a total mass concentration similar to the surrounding cyto- or nucleoplasm. Low-density condensates display higher permeability to cellular protein probes. We find that RNA tunes the biomolecular density of condensates. Moreover, intracellular structures such as mitochondria heavily influence the way phase separation proceeds, impacting the localization, morphology, and growth of condensates. These findings favor a model where segregative phase separation driven by non-associative or repulsive molecular interactions together with RNA-mediated selective association of specific components can give rise to low-density condensates in the crowded cellular environment.
Collapse
Affiliation(s)
- Taehyun Kim
- Department of Mechanical Engineering, Seoul National University, Seoul, 08826, Republic of Korea
| | - Jaeyoon Yoo
- Department of Mechanical Engineering, Seoul National University, Seoul, 08826, Republic of Korea
| | - Sungho Do
- Department of Mechanical Engineering, Seoul National University, Seoul, 08826, Republic of Korea
| | - Dong Soo Hwang
- Division of Environmental Science and Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, Daejeon, 34141, Republic of Korea
- Tomocube Inc, Daejeon, 34109, South Korea
| | - Yongdae Shin
- Department of Mechanical Engineering, Seoul National University, Seoul, 08826, Republic of Korea.
- Interdisciplinary Program in Bioengineering, Seoul National University, Seoul, 08826, Korea.
| |
Collapse
|
21
|
Lee C, Hugonnet H, Park J, Lee MJ, Park W, Park Y. Single-shot refractive index slice imaging using spectrally multiplexed optical transfer function reshaping. OPTICS EXPRESS 2023; 31:13806-13816. [PMID: 37157259 DOI: 10.1364/oe.485559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
The refractive index (RI) of cells and tissues is crucial in pathophysiology as a noninvasive and quantitative imaging contrast. Although its measurements have been demonstrated using three-dimensional quantitative phase imaging methods, these methods often require bulky interferometric setups or multiple measurements, which limits the measurement sensitivity and speed. Here, we present a single-shot RI imaging method that visualizes the RI of the in-focus region of a sample. By exploiting spectral multiplexing and optical transfer function engineering, three color-coded intensity images of a sample with three optimized illuminations were simultaneously obtained in a single-shot measurement. The measured intensity images were then deconvoluted to obtain the RI image of the in-focus slice of the sample. As a proof of concept, a setup was built using Fresnel lenses and a liquid-crystal display. For validation purposes, we measured microspheres of known RI and cross-validated the results with simulated results. Various static and highly dynamic biological cells were imaged to demonstrate that the proposed method can conduct single-shot RI slice imaging of biological samples with subcellular resolution.
Collapse
|
22
|
Rajora S, Butola M, Khare K. 3D reconstruction of unstained weakly scattering cells from a single defocused hologram. APPLIED OPTICS 2023; 62:D146-D156. [PMID: 37132780 DOI: 10.1364/ao.478351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
We investigate the problem of 3D complex field reconstruction corresponding to unstained red blood cells (RBCs) with a single defocused off-axis digital hologram. The main challenge in this problem is the localization of cells to the correct axial range. While investigating the volume recovery problem for a continuous phase object like the RBC, we observe an interesting feature of the backpropagated field that it does not show a clear focusing effect. Therefore, sparsity enforcement within the iterative optimization framework using a single hologram data frame cannot effectively restrict the reconstruction to the true object volume. For phase objects, it is known that the amplitude contrast of the backpropagated object field at the focus plane is minimum. We use this information available in the recovered object field in the hologram plane to device depth-dependent weights that are proportional to the inverse of amplitude contrast. This weight function is employed in the iterative steps of the optimization algorithm to assist the object volume localization. The overall reconstruction process is performed using the mean gradient descent (MGD) framework. Experimental illustrations of 3D volume reconstruction of the healthy as well as malaria-infected RBCs are presented. A test sample of polystyrene microsphere bead is also used to validate the axial localization capability of the proposed iterative technique. The proposed methodology is simple to implement experimentally and provides an approximate tomographic solution, which is axially restricted and consistent with the object field data.
Collapse
|
23
|
Ong JJY, Oh J, Yong Ang X, Naidu R, Chu TTT, Hyoung Im J, Manzoor U, Kha Nguyen T, Na SW, Han ET, Davis C, Sun Park W, Chun W, Jun H, Jin Lee S, Na S, Chan JKY, Park Y, Russell B, Chandramohanadas R, Han JH. Optical diffraction tomography and image reconstruction to measure host cell alterations caused by divergent Plasmodium species. SPECTROCHIMICA ACTA. PART A, MOLECULAR AND BIOMOLECULAR SPECTROSCOPY 2023; 286:122026. [PMID: 36395614 DOI: 10.1016/j.saa.2022.122026] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 09/29/2022] [Accepted: 10/18/2022] [Indexed: 06/16/2023]
Abstract
Malaria is a life-threatening infectious disease caused by parasites of the genus Plasmodium. Understanding the biological features of various parasite forms is important for the optical diagnosis and defining pathological states, which are often constrained by the lack of ambient visualization approaches. Here, we employ a label-free tomographic technique to visualize the host red blood cell (RBC) remodeling process and quantify changes in biochemical properties arising from parasitization. Through this, we provide a quantitative body of information pertaining to the influence of host cell environment on growth, survival, and replication of P. falciparum and P. vivax in their respective host cells: mature erythrocytes and young reticulocytes. These exquisite three-dimensional measurements of infected red cells demonstrats the potential of evolving 3D imaging to advance our understanding of Plasmodium biology and host-parasite interactions.
Collapse
Affiliation(s)
- Jessica J Y Ong
- Department of Microbiology and Immunology, University of Otago, Dunedin 9054, New Zealand
| | - Jeonghun Oh
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea; KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Xiang Yong Ang
- Department of Microbiology and Immunology, National University of Singapore, Singapore
| | - Renugah Naidu
- Pillar of Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore
| | - Trang T T Chu
- Pillar of Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore
| | - Jae Hyoung Im
- Department of Infectious Disease, Inha University School of Medicine, Incheon 22212, Republic of Korea
| | - Umar Manzoor
- Department of Medical Environmental Biology and Tropical Medicine, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Tuyet Kha Nguyen
- Department of Medical Environmental Biology and Tropical Medicine, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Seok-Won Na
- Department of Medical Environmental Biology and Tropical Medicine, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Eun-Taek Han
- Department of Medical Environmental Biology and Tropical Medicine, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Christeen Davis
- DBT Rajiv Gandhi Centre for Biotechnology, Thiruvananthapuram, Kerala, India; Manipal Academy of Higher Education, Manipal, Karnataka 576104, India
| | - Won Sun Park
- Department of Physiology, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Wanjoo Chun
- Department of Pharmacology, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea
| | - Hojong Jun
- Department of Tropical Medicine, Inha University College of Medicine, Incheon 22212, Republic of Korea
| | - Se Jin Lee
- Department of Obstetrics and Gynecology, Kangwon National University Hospital, Kangwon National University School of Medicine, Chuncheon 24341, South Korea
| | - Sunghun Na
- Department of Obstetrics and Gynecology, Kangwon National University Hospital, Kangwon National University School of Medicine, Chuncheon 24341, South Korea
| | - Jerry K Y Chan
- KK Womens' and Childrens' Hospital, Singapore; Academic Clinical Program in Obstetrics and Gynaecology, Duke-NUS Medical School, 169857, Singapore
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea; KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea; Tomocube Inc, Daejeon 34109, Republic of Korea
| | - Bruce Russell
- Department of Microbiology and Immunology, University of Otago, Dunedin 9054, New Zealand
| | - Rajesh Chandramohanadas
- Department of Microbiology and Immunology, National University of Singapore, Singapore; Pillar of Engineering Product Development, Singapore University of Technology and Design, Singapore, Singapore; DBT Rajiv Gandhi Centre for Biotechnology, Thiruvananthapuram, Kerala, India.
| | - Jin-Hee Han
- Department of Microbiology and Immunology, University of Otago, Dunedin 9054, New Zealand; Department of Medical Environmental Biology and Tropical Medicine, School of Medicine, Kangwon National University, Chuncheon 24341, Republic of Korea.
| |
Collapse
|
24
|
Hugonnet H, Shin S, Park Y. Regularization of dielectric tensor tomography. OPTICS EXPRESS 2023; 31:3774-3783. [PMID: 36785362 DOI: 10.1364/oe.478260] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Accepted: 01/02/2023] [Indexed: 06/18/2023]
Abstract
Dielectric tensor tomography reconstructs the three-dimensional dielectric tensors of microscopic objects and provides information about the crystalline structure orientations and principal refractive indices. Because dielectric tensor tomography is based on transmission measurement, it suffers from the missing cone problem, which causes poor axial resolution, underestimation of the refractive index, and halo artifacts. In this study, we study the application of total variation and positive semi-definiteness regularization to three-dimensional tensor distributions. In particular, we demonstrate the reduction of artifacts when applied to dielectric tensor tomography.
Collapse
|
25
|
On the inverse problem in optical coherence tomography. Sci Rep 2023; 13:1507. [PMID: 36707545 PMCID: PMC9883281 DOI: 10.1038/s41598-023-28366-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Accepted: 01/17/2023] [Indexed: 01/28/2023] Open
Abstract
We examine the inverse problem of retrieving sample refractive index information in the context of optical coherence tomography. Using two separate approaches, we discuss the limitations of the inverse problem which lead to it being ill-posed, primarily as a consequence of the limited viewing angles available in the reflection geometry. This is first considered from the theoretical point of view of diffraction tomography under a weak scattering approximation. We then investigate the full non-linear inverse problem using a variational approach. This presents another illustration of the non-uniqueness of the solution, and shows that even the non-linear (strongly scattering) scenario suffers a similar fate as the linear problem, with the observable spatial Fourier components of the sample occupying a limited support. Through examples we demonstrate how the solutions to the inverse problem compare when using the variational and diffraction-tomography approaches.
Collapse
|
26
|
Shin J, Kim G, Park J, Lee M, Park Y. Long-term label-free assessments of individual bacteria using three-dimensional quantitative phase imaging and hydrogel-based immobilization. Sci Rep 2023; 13:46. [PMID: 36593327 PMCID: PMC9806822 DOI: 10.1038/s41598-022-27158-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 12/27/2022] [Indexed: 01/03/2023] Open
Abstract
Three-dimensional (3D) quantitative phase imaging (QPI) enables long-term label-free tomographic imaging and quantitative analysis of live individual bacteria. However, the Brownian motion or motility of bacteria in a liquid medium produces motion artifacts during 3D measurements and hinders precise cell imaging and analysis. Meanwhile, existing cell immobilization methods produce noisy backgrounds and even alter cellular physiology. Here, we introduce a protocol that utilizes hydrogels for high-quality 3D QPI of live bacteria maintaining bacterial physiology. We demonstrate long-term high-resolution quantitative imaging and analysis of individual bacteria, including measuring the biophysical parameters of bacteria and responses to antibiotic treatments.
Collapse
Affiliation(s)
- Jeongwon Shin
- grid.37172.300000 0001 2292 0500Department of Biological Sciences, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea
| | - Geon Kim
- grid.37172.300000 0001 2292 0500Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea ,grid.37172.300000 0001 2292 0500KAIST Institute for Health Science and Technology, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea
| | - Jinho Park
- grid.37172.300000 0001 2292 0500Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea
| | - Moosung Lee
- grid.37172.300000 0001 2292 0500Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea ,grid.37172.300000 0001 2292 0500KAIST Institute for Health Science and Technology, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea
| | - YongKeun Park
- grid.37172.300000 0001 2292 0500Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea ,grid.37172.300000 0001 2292 0500KAIST Institute for Health Science and Technology, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141 South Korea ,Tomocube Inc., Daejeon, 34051 South Korea
| |
Collapse
|
27
|
Yu W, Li X, Wang B, Qu J, Liu L. Optical diffraction tomography of second-order nonlinear structures in weak scattering media: theoretical analysis and experimental consideration. OPTICS EXPRESS 2022; 30:45724-45737. [PMID: 36522971 DOI: 10.1364/oe.472637] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/07/2022] [Accepted: 11/10/2022] [Indexed: 06/17/2023]
Abstract
Computed tomography (CT) allows for high lateral and axial resolution imaging of the endogenous structure of matter thanks to its large spatial frequency support and has been realized in X-ray and linear optical domain known as optical diffraction tomography (ODT). Here, we present the theoretical basis and experimental considerations for ODT of second-order nonlinear structures in weak scattering media. We have derived the relation between second harmonic wave and the anisotropic nonlinear tensor in spatial frequency domain under first-order Born approximation. Our results show that, under a plane wave illumination, the two dimensional (2D) spatial spectra of generated second harmonic complex field relates to the inverse lattice of nonlinear structure on Ewald sphere shells. The centers of the Ewald spheres are determined by 2 times wavevector of the incident fundamental wave and the radii are determined by the modulus of the second harmonic wavevector. More importantly, it shows that the 2D spatial spectra is a superposition of the Ewald spheres of different components of the anisotropic nonlinear tensor. We propose to solve the inverse problem by controlling the polarizations of the fundamental and second harmonic signal. We tested the feasibility of the proposed method using a numerical phantom and make some discussions on practical implementations, including angular scanning schemes, polarization detection and illumination profile for optimizing reconstruction region. Possessing high resolution, wide-field imaging and polarization-sensitive property, we believe that the proposed scheme would have important applications in nonlinear microscopy.
Collapse
|
28
|
Lee D, Lee M, Kwak H, Kim YS, Shim J, Jung JH, Park WS, Park JH, Lee S, Park Y. High-fidelity optical diffraction tomography of live organisms using iodixanol refractive index matching. BIOMEDICAL OPTICS EXPRESS 2022; 13:6404-6415. [PMID: 36589574 PMCID: PMC9774853 DOI: 10.1364/boe.465066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 10/25/2022] [Accepted: 10/25/2022] [Indexed: 06/17/2023]
Abstract
Optical diffraction tomography (ODT) enables the three-dimensional (3D) refractive index (RI) reconstruction. However, when the RI difference between a sample and a medium increases, the effects of light scattering become significant, preventing the acquisition of high-quality and accurate RI reconstructions. Herein, we present a method for high-fidelity ODT by introducing non-toxic RI matching media. Optimally reducing the RI contrast enhances the fidelity and accuracy of 3D RI reconstruction, enabling visualization of the morphology and intra-organization of live biological samples without producing toxic effects. We validate our method using various biological organisms, including C. albicans and C. elegans.
Collapse
Affiliation(s)
- Dohyeon Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Moosung Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Haechan Kwak
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Young Seo Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Jaehyu Shim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Jik Han Jung
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - Wei-sun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Ji-Ho Park
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - Sumin Lee
- Tomocube Inc., Daejeon 34109, Republic of Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Tomocube Inc., Daejeon 34109, Republic of Korea
| |
Collapse
|
29
|
Krauze W, Kuś A, Ziemczonok M, Haimowitz M, Chowdhury S, Kujawińska M. 3D scattering microphantom sample to assess quantitative accuracy in tomographic phase microscopy techniques. Sci Rep 2022; 12:19586. [PMID: 36380058 PMCID: PMC9666505 DOI: 10.1038/s41598-022-24193-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 11/11/2022] [Indexed: 11/16/2022] Open
Abstract
In this paper we present a structurally-complex biomimetic scattering structure, fabricated with two-photon polymerization, and utilize this object in order to benchmark a computational imaging system. The phantom allows to tailor the scattering by modifying its degrees of freedom i.e. refractive index contrast and scattering layer dimensions and incorporates a 3D imaging quality test, representing a single cell within tissue. While the sample may be used with multiple 3D microscopy techniques, we demonstrate the impact of scattering on three tomographic phase microscopy (TPM) reconstruction methods. One of these methods assumes the sample to be weak-scattering, while the other two take multiple scattering into account. The study is performed at two wavelengths (visible and near-infrared), which serve as a scaling factor for the scattering phenomenon. We find that changing the wavelength from visible into near-infrared impacts the applicability of TPM reconstruction methods. As a result of reduced scattering in near-infrared region, the multiple-scattering-oriented techniques perform in fact worse than a method aimed for weak-scattering samples. This implies a necessity of selecting proper approach depending on sample's scattering characteristics even in case of subtle changes in the object-light interaction.
Collapse
Affiliation(s)
- Wojciech Krauze
- grid.1035.70000000099214842Institute of Micromechanics and Photonics, Warsaw University of Technology, Boboli 8 street, Warsaw, 02-525 Poland
| | - Arkadiusz Kuś
- grid.1035.70000000099214842Institute of Micromechanics and Photonics, Warsaw University of Technology, Boboli 8 street, Warsaw, 02-525 Poland
| | - Michał Ziemczonok
- grid.1035.70000000099214842Institute of Micromechanics and Photonics, Warsaw University of Technology, Boboli 8 street, Warsaw, 02-525 Poland
| | - Max Haimowitz
- grid.89336.370000 0004 1936 9924Department of Electrical Engineering, University of Texas at Austin, 2501 Speedway, Austin, TX 78712 USA
| | - Shwetadwip Chowdhury
- grid.89336.370000 0004 1936 9924Department of Electrical Engineering, University of Texas at Austin, 2501 Speedway, Austin, TX 78712 USA
| | - Małgorzata Kujawińska
- grid.1035.70000000099214842Institute of Micromechanics and Photonics, Warsaw University of Technology, Boboli 8 street, Warsaw, 02-525 Poland
| |
Collapse
|
30
|
Ossowski P, Kuś A, Krauze W, Tamborski S, Ziemczonok M, Kuźbicki Ł, Szkulmowski M, Kujawińska M. Near-infrared, wavelength, and illumination scanning holographic tomography. BIOMEDICAL OPTICS EXPRESS 2022; 13:5971-5988. [PMID: 36733741 PMCID: PMC9872886 DOI: 10.1364/boe.468046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 08/19/2022] [Accepted: 08/24/2022] [Indexed: 06/18/2023]
Abstract
We present a holographic tomography technique in which the projections are acquired using both wavelength and illumination scanning in the near-infrared region. We show how to process the acquired data to obtain correct values of three-dimensional refractive index distributions in both single-wavelength and multi-wavelength data acquisition schemes and how to properly account for the dispersion of the sample. We perform numerical and experimental comparisons of different illumination scenarios to determine the most efficient measurement protocol. We show that the multi-wavelength protocol is advantageous in terms of signal-to-noise ratio and contrast-to-noise ratio over single-wavelength protocols, even for the same number of projections used for reconstructions. Finally, we show that this approach is suitable for providing high-quality refractive index distributions of relatively thick colon cancer samples.
Collapse
Affiliation(s)
- Paweł Ossowski
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Arkadiusz Kuś
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Wojciech Krauze
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Szymon Tamborski
- Institute of Physics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland
| | - Michał Ziemczonok
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Łukasz Kuźbicki
- Institute of Biology, Faculty of Biological and Veterinary Sciences, Nicolaus Copernicus University in Toruń, Lwowska 1, 87-100 Toruń, Poland
| | - Maciej Szkulmowski
- Institute of Physics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland
| | - Małgorzata Kujawińska
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| |
Collapse
|
31
|
Tang M, He H, Yu L. Real-time 3D imaging of ocean algae with crosstalk suppressed single-shot digital holographic microscopy. BIOMEDICAL OPTICS EXPRESS 2022; 13:4455-4467. [PMID: 36032587 PMCID: PMC9408253 DOI: 10.1364/boe.463678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 07/21/2022] [Accepted: 07/27/2022] [Indexed: 06/15/2023]
Abstract
Digital holographic microscopy (DHM) has the potential to reconstruct the 3D shape of volumetric samples from a single-shot hologram in a label-free and noninvasive manner. However, the holographic reconstruction is significantly compromised by the out-of-focus image resulting from the crosstalk between refocused planes, leading to the low fidelity of the results. In this paper, we propose a crosstalk suppression algorithm-assisted 3D imaging method combined with a home built DHM system to achieve accurate 3D imaging of ocean algae using only a single hologram. As a key step in the algorithm, a hybrid edge detection strategy using gradient-based and deep learning-based methods is proposed to offer accurate boundary information for the downstream processing. With this information, the crosstalk of each refocused plane can be estimated with adjacent refocused planes. Empowered by this method, we demonstrated successful 3D imaging of six kinds of ocean algae that agree well with the ground truth; we further demonstrated that this method could achieve real-time 3D imaging of the quick swimming ocean algae in the water environment. To our knowledge, this is the first time single-shot DHM is reported in 3D imaging of ocean algae, paving the way for on-site monitoring of the ocean algae.
Collapse
Affiliation(s)
- Ming Tang
- School of Advanced Manufacturing, Nanchang University, Nanchang, Jiangxi 330031, China
| | - Hao He
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, Guangdong 518055, China
| | - Longkun Yu
- School of Advanced Manufacturing, Nanchang University, Nanchang, Jiangxi 330031, China
| |
Collapse
|
32
|
Kim G, Ahn D, Kang M, Park J, Ryu D, Jo Y, Song J, Ryu JS, Choi G, Chung HJ, Kim K, Chung DR, Yoo IY, Huh HJ, Min HS, Lee NY, Park Y. Rapid species identification of pathogenic bacteria from a minute quantity exploiting three-dimensional quantitative phase imaging and artificial neural network. LIGHT, SCIENCE & APPLICATIONS 2022; 11:190. [PMID: 35739098 PMCID: PMC9226356 DOI: 10.1038/s41377-022-00881-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/26/2021] [Revised: 06/03/2022] [Accepted: 06/09/2022] [Indexed: 05/14/2023]
Abstract
The healthcare industry is in dire need of rapid microbial identification techniques for treating microbial infections. Microbial infections are a major healthcare issue worldwide, as these widespread diseases often develop into deadly symptoms. While studies have shown that an early appropriate antibiotic treatment significantly reduces the mortality of an infection, this effective treatment is difficult to practice. The main obstacle to early appropriate antibiotic treatments is the long turnaround time of the routine microbial identification, which includes time-consuming sample growth. Here, we propose a microscopy-based framework that identifies the pathogen from single to few cells. Our framework obtains and exploits the morphology of the limited sample by incorporating three-dimensional quantitative phase imaging and an artificial neural network. We demonstrate the identification of 19 bacterial species that cause bloodstream infections, achieving an accuracy of 82.5% from an individual bacterial cell or cluster. This performance, comparable to that of the gold standard mass spectroscopy under a sufficient amount of sample, underpins the effectiveness of our framework in clinical applications. Furthermore, our accuracy increases with multiple measurements, reaching 99.9% with seven different measurements of cells or clusters. We believe that our framework can serve as a beneficial advisory tool for clinicians during the initial treatment of infections.
Collapse
Affiliation(s)
- Geon Kim
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea
| | - Daewoong Ahn
- Tomocube Inc., Daejeon, 34109, Republic of Korea
| | - Minhee Kang
- Smart Healthcare & Device Research Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, 06351, Republic of Korea
| | - Jinho Park
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea
| | - DongHun Ryu
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea
| | - YoungJu Jo
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea
- Tomocube Inc., Daejeon, 34109, Republic of Korea
- Department of Applied Physics, Stanford University, Stanford, CA, 94305, USA
| | - Jinyeop Song
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea
- Department of Physics, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Jea Sung Ryu
- Graduate School of Nanoscience and Technology, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
| | - Gunho Choi
- Tomocube Inc., Daejeon, 34109, Republic of Korea
| | - Hyun Jung Chung
- Graduate School of Nanoscience and Technology, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
- Department of Biological Sciences, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea
| | - Kyuseok Kim
- Department of Emergency Medicine, Bundang CHA Hospital, Seongnam-si, Gyeonggi-Do, 13496, Korea
| | - Doo Ryeon Chung
- Division of Infectious Diseases, Department of Internal Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, 06351, Republic of Korea
| | - In Young Yoo
- Department of Laboratory Medicine, Seoul St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, 06591, Republic of Korea
| | - Hee Jae Huh
- Department of Laboratory Medicine and Genetics, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, 06351, Republic of Korea
| | | | - Nam Yong Lee
- Department of Laboratory Medicine and Genetics, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, 06351, Republic of Korea.
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea.
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, Republic of Korea.
- Tomocube Inc., Daejeon, 34109, Republic of Korea.
| |
Collapse
|
33
|
Lipke W, Winnik J, Trusiak M. Numerical analysis of the effect of reduced temporal coherence in quantitative phase microscopy and tomography. OPTICS EXPRESS 2022; 30:21241-21257. [PMID: 36224847 DOI: 10.1364/oe.458167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 04/20/2022] [Indexed: 06/16/2023]
Abstract
We present the numerical analysis of the effect of the temporarily partially coherent illumination on the phase measurement accuracy in digital holography microscopy (DHM) and optical diffraction tomography (ODT), as reconstruction algorithms tend to assume purely monochromatic conditions. In the regime of reduced temporal coherence, we simulate the hologram formation in two different optical setups, representing classical off-axis two-beam and grating common-path configurations. We consider two ODT variants: with sample rotation and angle-scanning of illumination. Besides the coherence degree of illumination, our simulation considers the influence of the sample normal dispersion, shape of the light spectrum, and optical parameters of the imaging setup. As reconstruction algorithms we employ Fourier hologram method and first-order Rytov approximation with direct inversion and nonnegativity constraints. Quantitative evaluation of the measurement results deviations introduced by the mentioned error sources is comprehensively analyzed, for the first time to the best of our knowledge. Obtained outcomes indicate low final DHM/ODT reconstruction errors for the grating-assisted common-path configuration. Nevertheless, dispersion and asymmetric spectrum introduce non-negligible overestimated refractive index values and noise, and should be thus carefully considered within experimental frameworks.
Collapse
|
34
|
Li J, Zhou N, Sun J, Zhou S, Bai Z, Lu L, Chen Q, Zuo C. Transport of intensity diffraction tomography with non-interferometric synthetic aperture for three-dimensional label-free microscopy. LIGHT, SCIENCE & APPLICATIONS 2022; 11:154. [PMID: 35650186 PMCID: PMC9160286 DOI: 10.1038/s41377-022-00815-7] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 04/12/2022] [Accepted: 04/24/2022] [Indexed: 05/05/2023]
Abstract
We present a new label-free three-dimensional (3D) microscopy technique, termed transport of intensity diffraction tomography with non-interferometric synthetic aperture (TIDT-NSA). Without resorting to interferometric detection, TIDT-NSA retrieves the 3D refractive index (RI) distribution of biological specimens from 3D intensity-only measurements at various illumination angles, allowing incoherent-diffraction-limited quantitative 3D phase-contrast imaging. The unique combination of z-scanning the sample with illumination angle diversity in TIDT-NSA provides strong defocus phase contrast and better optical sectioning capabilities suitable for high-resolution tomography of thick biological samples. Based on an off-the-shelf bright-field microscope with a programmable light-emitting-diode (LED) illumination source, TIDT-NSA achieves an imaging resolution of 206 nm laterally and 520 nm axially with a high-NA oil immersion objective. We validate the 3D RI tomographic imaging performance on various unlabeled fixed and live samples, including human breast cancer cell lines MCF-7, human hepatocyte carcinoma cell lines HepG2, mouse macrophage cell lines RAW 264.7, Caenorhabditis elegans (C. elegans), and live Henrietta Lacks (HeLa) cells. These results establish TIDT-NSA as a new non-interferometric approach to optical diffraction tomography and 3D label-free microscopy, permitting quantitative characterization of cell morphology and time-dependent subcellular changes for widespread biological and medical applications.
Collapse
Affiliation(s)
- Jiaji Li
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Ning Zhou
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Jiasong Sun
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Shun Zhou
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Zhidong Bai
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Linpeng Lu
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China
| | - Qian Chen
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China.
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China.
| | - Chao Zuo
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, No. 200 Xiaolingwei Street, Nanjing, Jiangsu Province, 210094, China.
- Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China.
- Smart Computational Imaging Laboratory (SCILab), Nanjing University of Science and Technology, Nanjing, Jiangsu Province, 210094, China.
| |
Collapse
|
35
|
Hugonnet H, Lee MJ, Park YK. Quantitative phase and refractive index imaging of 3D objects via optical transfer function reshaping. OPTICS EXPRESS 2022; 30:13802-13809. [PMID: 35472985 DOI: 10.1364/oe.454533] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Accepted: 03/19/2022] [Indexed: 06/14/2023]
Abstract
Deconvolution phase microscopy enables high-contrast visualization of transparent samples through reconstructions of their transmitted phases or refractive indexes. Herein, we propose a method to extend 2D deconvolution phase microscopy to thick 3D samples. The refractive index distribution of a sample can be obtained at a specific axial plane by measuring only four intensity images obtained under optimized illumination patterns. Also, the optical phase delay of a sample can be measured using different illumination patterns.
Collapse
|
36
|
Pan Y, Guo S, Smith ZJ, Chu K. Simultaneous 3D deconvolution and halo removal for spatial light interference microscopy through a two-edge apodized Wiener filter. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:287-296. [PMID: 35200967 DOI: 10.1364/josaa.444764] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Accepted: 12/24/2021] [Indexed: 06/14/2023]
Abstract
As one of the most sensitive quantitative phase microscopy techniques, spatial light interference microscopy (SLIM) has undergone rapid development in the past decade and has seen wide application in both basic science and clinical studies. However, as with any other traditional microscope, the axial resolution is the worst among the three dimensions. This leads to lower contrast in the thicker regions of cell samples. Another common foe in the phase contrast image is the halo artifact, which can block underlying structures, in particular when high resolution is desired. Current solutions focus on either halo removal or contrast enhancement alone, and thus need two processing steps to create both high contrast and halo-free phase images. Further, raw images often suffer from artifacts that are both bright and slowly varying, dubbed here as cloud-like artifacts. After deconvolution, these cloud-like artifacts often dominate the image and obscure high-frequency information, which is typically of greatest interest. In this paper, we first analyzed the unique characteristics of the phase transfer function associated with SLIM to find the root of the cloud-like artifacts and halo artifacts. Then we designed a two-edge apodized deconvolution scheme as a counter measure. We show that even with a simple Wiener filter, the two-edge apodization (TEA) can effectively improve the contrast while suppressing the halo and cloud-like artifacts. Our algorithm, named TEA-Weiner, is non-iterative and thus can be implemented in real time. For low-contrast structures inside the cell such as the endoplasmic reticulum (ER), where ringing artifacts are more likely, we show that two-edge apodization can be combined with additional constraints such as total variation so that their contrast can be enhanced simultaneously with other bright structures inside the cell. Comparing our method with other state-of-the-art algorithms, our method has two advantages: First, deconvolution and halo removal are accomplished simultaneously; second, the image quality is highest using TEA-Weiner filtering.
Collapse
|
37
|
Denneulin L, Momey F, Brault D, Debailleul M, Taddese AM, Verrier N, Haeberlé O. GSURE criterion for unsupervised regularized reconstruction in tomographic diffractive microscopy. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2022; 39:A52-A61. [PMID: 35200955 DOI: 10.1364/josaa.444890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Accepted: 12/17/2021] [Indexed: 06/14/2023]
Abstract
We propose an unsupervised regularized inversion method for reconstruction of the 3D refractive index map of a sample in tomographic diffractive microscopy. It is based on the minimization of the generalized Stein's unbiased risk estimator (GSURE) to automatically estimate optimal values for the hyperparameters of one or several regularization terms (sparsity, edge-preserving smoothness, total variation). We evaluate the performance of our approach on simulated and experimental limited-view data. Our results show that GSURE is an efficient criterion to find suitable regularization weights, which is a critical task, particularly in the context of reducing the amount of required data to allow faster yet efficient acquisitions and reconstructions.
Collapse
|
38
|
Yasuhiko O, Takeuchi K, Yamada H, Ueda Y. Multiple-scattering suppressive refractive index tomography for the label-free quantitative assessment of multicellular spheroids. BIOMEDICAL OPTICS EXPRESS 2022; 13:962-979. [PMID: 35284178 PMCID: PMC8884216 DOI: 10.1364/boe.446622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 01/14/2022] [Accepted: 01/14/2022] [Indexed: 05/13/2023]
Abstract
Refractive index (RI) tomography is a quantitative tomographic technique used to visualize the intrinsic contrast of unlabeled biological samples. Conventional RI reconstruction algorithms are based on weak-scattering approximation, such as the Born or Rytov approximation. Although these linear algorithms are computationally efficient, they are invalid when the fields are strongly distorted by multiple scattering (MS) of specimens. Herein, we propose an approach to reconstruct the RI distributions of MS objects even under weak-scattering approximation using an MS-suppressive operation. The operation converts the distorted fields into MS-suppressed fields, where weak-scattering approximation is applicable. Using this approach, we reconstructed a whole multicellular spheroid and successfully visualized its internal subcellular structures. Our work facilitates the realization of RI tomography of MS specimens and label-free quantitative analysis of 3D multicellular specimens.
Collapse
Affiliation(s)
- Osamu Yasuhiko
- Central Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu, Shizuoka 434-8601, Japan
| | - Kozo Takeuchi
- Central Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu, Shizuoka 434-8601, Japan
| | - Hidenao Yamada
- Central Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu, Shizuoka 434-8601, Japan
| | - Yukio Ueda
- Central Research Laboratory, Hamamatsu Photonics K.K., 5000 Hirakuchi, Hamakita-ku, Hamamatsu, Shizuoka 434-8601, Japan
| |
Collapse
|
39
|
Jo Y, Cho H, Park WS, Kim G, Ryu D, Kim YS, Lee M, Park S, Lee MJ, Joo H, Jo H, Lee S, Lee S, Min HS, Heo WD, Park Y. Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning. Nat Cell Biol 2021; 23:1329-1337. [PMID: 34876684 DOI: 10.1038/s41556-021-00802-x] [Citation(s) in RCA: 28] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2021] [Accepted: 10/25/2021] [Indexed: 02/07/2023]
Abstract
Simultaneous imaging of various facets of intact biological systems across multiple spatiotemporal scales is a long-standing goal in biology and medicine, for which progress is hindered by limits of conventional imaging modalities. Here we propose using the refractive index (RI), an intrinsic quantity governing light-matter interaction, as a means for such measurement. We show that major endogenous subcellular structures, which are conventionally accessed via exogenous fluorescence labelling, are encoded in three-dimensional (3D) RI tomograms. We decode this information in a data-driven manner, with a deep learning-based model that infers multiple 3D fluorescence tomograms from RI measurements of the corresponding subcellular targets, thereby achieving multiplexed microtomography. This approach, called RI2FL for refractive index to fluorescence, inherits the advantages of both high-specificity fluorescence imaging and label-free RI imaging. Importantly, full 3D modelling of absolute and unbiased RI improves generalization, such that the approach is applicable to a broad range of new samples without retraining to facilitate immediate applicability. The performance, reliability and scalability of this technology are extensively characterized, and its various applications within single-cell profiling at unprecedented scales (which can generate new experimentally testable hypotheses) are demonstrated.
Collapse
Affiliation(s)
- YoungJu Jo
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea.,Tomocube, Daejeon, Republic of Korea.,Departments of Applied Physics and of Biology, Stanford University, Stanford, CA, USA
| | | | - Wei Sun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - Geon Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - DongHun Ryu
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - Young Seo Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea.,Graduate School of Medial Science and Engineering, KAIST, Daejeon, Republic of Korea
| | - Moosung Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea
| | - Sangwoo Park
- Gwangju Center, Korea Basic Science Institute (KBSI), Gwangju, Republic of Korea
| | - Mahn Jae Lee
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea.,Graduate School of Medial Science and Engineering, KAIST, Daejeon, Republic of Korea
| | | | | | - Seongsoo Lee
- Gwangju Center, Korea Basic Science Institute (KBSI), Gwangju, Republic of Korea
| | - Sumin Lee
- Tomocube, Daejeon, Republic of Korea
| | | | - Won Do Heo
- Department of Biological Sciences, KAIST, Daejeon, Republic of Korea. .,KAIST Institute for the BioCentury, KAIST, Daejeon, Republic of Korea.
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea. .,KAIST Institute for Health Science and Technology, KAIST, Daejeon, Republic of Korea. .,Tomocube, Daejeon, Republic of Korea.
| |
Collapse
|
40
|
Roadmap on Digital Holography-Based Quantitative Phase Imaging. J Imaging 2021; 7:jimaging7120252. [PMID: 34940719 PMCID: PMC8703719 DOI: 10.3390/jimaging7120252] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 11/11/2021] [Accepted: 11/15/2021] [Indexed: 12/02/2022] Open
Abstract
Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly discusses the present and future perspectives of 2D and 3D QPI research based on digital holographic microscopy, holographic tomography, and their applications.
Collapse
|
41
|
Lee AJ, Yoon D, Han S, Hugonnet H, Park W, Park JK, Nam Y, Park Y. Label-free monitoring of 3D cortical neuronal growth in vitro using optical diffraction tomography. BIOMEDICAL OPTICS EXPRESS 2021; 12:6928-6939. [PMID: 34858689 PMCID: PMC8606138 DOI: 10.1364/boe.439404] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/02/2021] [Revised: 09/17/2021] [Accepted: 09/18/2021] [Indexed: 05/10/2023]
Abstract
The highly complex central nervous systems of mammals are often studied using three-dimensional (3D) in vitro primary neuronal cultures. A coupled confocal microscopy and immunofluorescence labeling are widely utilized for visualizing the 3D structures of neurons. However, this requires fixation of the neurons and is not suitable for monitoring an identical sample at multiple time points. Thus, we propose a label-free monitoring method for 3D neuronal growth based on refractive index tomograms obtained by optical diffraction tomography. The 3D morphology of the neurons was clearly visualized, and the developmental processes of neurite outgrowth in 3D spaces were analyzed for individual neurons.
Collapse
Affiliation(s)
- Ariel J Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Current Affiliation: Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
- Contributed equally
| | - DongJo Yoon
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
- Contributed equally
| | - SeungYun Han
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Current Affiliation: Department of Applied Physics, Yale University, New Haven, CT 06520, USA
- Contributed equally
| | - Herve Hugonnet
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - WeiSun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Je-Kyun Park
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - YoonKey Nam
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Tomocube Inc., Daejeon, Republic of Korea
| |
Collapse
|
42
|
Long JM, Chun JY, Gaylord TK. ADMM approach for efficient iterative tomographic deconvolution reconstruction of 3D quantitative phase images. APPLIED OPTICS 2021; 60:8485-8492. [PMID: 34612951 DOI: 10.1364/ao.433999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Accepted: 08/20/2021] [Indexed: 06/13/2023]
Abstract
Tomographic deconvolution phase microscopy (TDPM) is a promising approach for 3D quantitative imaging of phase objects such as biological cells and optical fibers. In the present work, the alternating direction method of multipliers (ADMM) is applied to TDPM to shorten its image acquisition and processing times while simultaneously improving its accuracy. ADMM-TDPM is used to optimize the image fidelity by minimizing Gaussian noise and by using total variation regularization with the constraints of nonnegativity and known zeros. ADMM-TDPM can reconstruct phase objects that are shift variant in three spatial dimensions. ADMM-TDPM achieves speedups of 5x in image acquisition time and greater than 10x in image processing time with accompanying higher accuracy compared to TDPM.
Collapse
|
43
|
Delestre B, Talbi M, Abad A, Brunel M. Tomography of irregular rough particles using the error-reduction algorithm with multi-views interferometric particle imaging. JOURNAL OF THE OPTICAL SOCIETY OF AMERICA. A, OPTICS, IMAGE SCIENCE, AND VISION 2021; 38:1237-1247. [PMID: 34613319 DOI: 10.1364/josaa.423742] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Accepted: 06/24/2021] [Indexed: 06/13/2023]
Abstract
This work reports the 3D reconstruction of a particle from a set of three simulated interferometric images of this particle (from three perpendicular angles of view). The reconstruction of each view from its corresponding interferometric pattern uses the error-reduction (ER) algorithm. The 3D reconstruction enables an estimation of the volume of the particle. The method is tested on a dendrite-like particle. An experimental demonstration of the technique is done using a digital micromirror device (DMD) that generates the interferometric images of "programmable" rough particles.
Collapse
|
44
|
Chen X, Kandel ME, Popescu G. Spatial light interference microscopy: principle and applications to biomedicine. ADVANCES IN OPTICS AND PHOTONICS 2021; 13:353-425. [PMID: 35494404 PMCID: PMC9048520 DOI: 10.1364/aop.417837] [Citation(s) in RCA: 27] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
In this paper, we review spatial light interference microscopy (SLIM), a common-path, phase-shifting interferometer, built onto a phase-contrast microscope, with white-light illumination. As one of the most sensitive quantitative phase imaging (QPI) methods, SLIM allows for speckle-free phase reconstruction with sub-nanometer path-length stability. We first review image formation in QPI, scattering, and full-field methods. Then, we outline SLIM imaging from theory and instrumentation to diffraction tomography. Zernike's phase-contrast microscopy, phase retrieval in SLIM, and halo removal algorithms are discussed. Next, we discuss the requirements for operation, with a focus on software developed in-house for SLIM that enables high-throughput acquisition, whole slide scanning, mosaic tile registration, and imaging with a color camera. We introduce two methods for solving the inverse problem using SLIM, white-light tomography, and Wolf phase tomography. Lastly, we review the applications of SLIM in basic science and clinical studies. SLIM can study cell dynamics, cell growth and proliferation, cell migration, mass transport, etc. In clinical settings, SLIM can assist with cancer studies, reproductive technology, blood testing, etc. Finally, we review an emerging trend, where SLIM imaging in conjunction with artificial intelligence brings computational specificity and, in turn, offers new solutions to outstanding challenges in cell biology and pathology.
Collapse
|
45
|
Sun J, Koukourakis N, Guck J, Czarske JW. Rapid computational cell-rotation around arbitrary axes in 3D with multi-core fiber. BIOMEDICAL OPTICS EXPRESS 2021; 12:3423-3437. [PMID: 34221669 PMCID: PMC8221929 DOI: 10.1364/boe.423035] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Revised: 05/08/2021] [Accepted: 05/10/2021] [Indexed: 05/25/2023]
Abstract
Optical trapping is a vital tool in biology, allowing precise optical manipulation of nanoparticles, micro-robots, and cells. Due to the low risk of photodamage and high trap stiffness, fiber-based dual-beam traps are widely used for optical manipulation of large cells. Besides trapping, advanced applications like 3D refractive index tomography need a rotation of cells, which requires precise control of the forces, for example, the acting-point of the forces and the intensities in the region of interest (ROI). A precise rotation of large cells in 3D about arbitrary axes has not been reported yet in dual-beam traps. We introduce a novel dual-beam optical trap in which a multi-core fiber (MCF) is transformed to a phased array, using wavefront shaping and computationally programmable light. The light-field distribution in the trapping region is holographically controlled within 0.1 s, which determines the orientation and the rotation axis of the cell with small retardation. We demonstrate real-time controlled rotation of HL60 cells about all 3D axes with a very high degree of freedom by holographic controlled light through an MCF with a resolution close to the diffraction limit. For the first time, the orientation of the cell can be precisely controlled about all 3D axes in a dual-beam trap. MCFs provide much higher flexibility beyond the bulky optics, enabling lab-on-a-chip applications and can be easily integrated for applications like contactless cell surgery, refractive index tomography, cell-elasticity measurement, which require precise 3D manipulation of cells.
Collapse
Affiliation(s)
- Jiawei Sun
- Laboratory of Measurement and Sensor System Technique, TU Dresden, Helmholtzstrasse 18, 01069 Dresden, Germany
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany
| | - Nektarios Koukourakis
- Laboratory of Measurement and Sensor System Technique, TU Dresden, Helmholtzstrasse 18, 01069 Dresden, Germany
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany
| | - Jochen Guck
- Max Planck Institute for the Science of Light, Staudtstrasse 2, 91058 Erlangen, Germany
- Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany
| | - Jürgen W. Czarske
- Laboratory of Measurement and Sensor System Technique, TU Dresden, Helmholtzstrasse 18, 01069 Dresden, Germany
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Dresden, Germany
- Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany
| |
Collapse
|
46
|
Wang H, Tahir W, Zhu J, Tian L. Large-scale holographic particle 3D imaging with the beam propagation model. OPTICS EXPRESS 2021; 29:17159-17172. [PMID: 34154264 DOI: 10.1364/oe.424752] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2021] [Accepted: 05/06/2021] [Indexed: 06/13/2023]
Abstract
We develop a novel algorithm for large-scale holographic reconstruction of 3D particle fields. Our method is based on a multiple-scattering beam propagation method (BPM) combined with sparse regularization that enables recovering dense 3D particles of high refractive index contrast from a single hologram. We show that the BPM-computed hologram generates intensity statistics closely matching with the experimental measurements and provides up to 9× higher accuracy than the single-scattering model. To solve the inverse problem, we devise a computationally efficient algorithm, which reduces the computation time by two orders of magnitude as compared to the state-of-the-art multiple-scattering based technique. We demonstrate the superior reconstruction accuracy in both simulations and experiments under different scattering strengths. We show that the BPM reconstruction significantly outperforms the single-scattering method in particular for deep imaging depths and high particle densities.
Collapse
|
47
|
Lee M, Kim K, Oh J, Park Y. Isotropically resolved label-free tomographic imaging based on tomographic moulds for optical trapping. LIGHT, SCIENCE & APPLICATIONS 2021; 10:102. [PMID: 33994544 PMCID: PMC8126562 DOI: 10.1038/s41377-021-00535-4] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/22/2020] [Revised: 04/06/2021] [Accepted: 04/14/2021] [Indexed: 05/13/2023]
Abstract
A major challenge in three-dimensional (3D) microscopy is to obtain accurate spatial information while simultaneously keeping the microscopic samples in their native states. In conventional 3D microscopy, axial resolution is inferior to spatial resolution due to the inaccessibility to side scattering signals. In this study, we demonstrate the isotropic microtomography of free-floating samples by optically rotating a sample. Contrary to previous approaches using optical tweezers with multiple foci which are only applicable to simple shapes, we exploited 3D structured light traps that can stably rotate freestanding complex-shaped microscopic specimens, and side scattering information is measured at various sample orientations to achieve isotropic resolution. The proposed method yields an isotropic resolution of 230 nm and captures structural details of colloidal multimers and live red blood cells, which are inaccessible using conventional tomographic microscopy. We envision that the proposed approach can be deployed for solving diverse imaging problems that are beyond the examples shown here.
Collapse
Affiliation(s)
- Moosung Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, South Korea
| | - Kyoohyun Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea
- Max Planck Institute for the Science of Light & Max-Planck-Zentrum für Physik und Medizin, 91058, Erlangen, Germany
| | - Jeonghun Oh
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, South Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, 34141, South Korea.
- KAIST Institute for Health Science and Technology, KAIST, Daejeon, 34141, South Korea.
- Tomocube Inc., Daejeon, 34109, South Korea.
| |
Collapse
|
48
|
Ryu D, Ryu D, Baek Y, Cho H, Kim G, Kim YS, Lee Y, Kim Y, Ye JC, Min HS, Park Y. DeepRegularizer: Rapid Resolution Enhancement of Tomographic Imaging Using Deep Learning. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:1508-1518. [PMID: 33566760 DOI: 10.1109/tmi.2021.3058373] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.
Collapse
|
49
|
Lu CW, Belashov AV, Zhikhoreva AA, Semenova IV, Cheng CJ, Su LY, Wu CH. Application of digital holographic tomography in antitumor effect of cantharides complex on 4T1 breast cancer cells. APPLIED OPTICS 2021; 60:3365-3373. [PMID: 33983241 DOI: 10.1364/ao.416943] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Accepted: 03/22/2021] [Indexed: 06/12/2023]
Abstract
The study focuses on a methodology providing noninvasive monitoring and evaluation of the antitumor effect of traditional Chinese medicine, cantharides complex (canth), on 4T1 breast tumor cells. Digital holographic tomography (DHT) and developed data post-processing algorithms were used for quantitative estimation of changes in optical and morphological parameters of cells. We calculated and compared data on the refractive index, thickness, and projected area of 4T1 breast tumor cells in control untreated specimens and those treated with doxorubicin hydrochloride (DOX), canth, and their combinations. Post-treatment changes in cellular morphology recorded by DHT demonstrated that the two drugs led to noticeably different morphological changes in cells that can be presumably associated with different pathways of their death, apoptosis, or necrosis. The effect of combined treatment with these two drugs strongly depended on their relative concentrations and could lead to changes characteristic either for DOX or for canth; however, being more profound than those obtained when using each drug solely. The results obtained by DHT are in a good correspondence with commonly used cell viability analysis and immunofluorescent analysis of changes in cellular cytoskeleton.
Collapse
|
50
|
Kang I, Goy A, Barbastathis G. Dynamical machine learning volumetric reconstruction of objects' interiors from limited angular views. LIGHT, SCIENCE & APPLICATIONS 2021; 10:74. [PMID: 33828073 PMCID: PMC8027224 DOI: 10.1038/s41377-021-00512-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/17/2020] [Revised: 03/04/2021] [Accepted: 03/13/2021] [Indexed: 05/26/2023]
Abstract
Limited-angle tomography of an interior volume is a challenging, highly ill-posed problem with practical implications in medical and biological imaging, manufacturing, automation, and environmental and food security. Regularizing priors are necessary to reduce artifacts by improving the condition of such problems. Recently, it was shown that one effective way to learn the priors for strongly scattering yet highly structured 3D objects, e.g. layered and Manhattan, is by a static neural network [Goy et al. Proc. Natl. Acad. Sci. 116, 19848-19856 (2019)]. Here, we present a radically different approach where the collection of raw images from multiple angles is viewed analogously to a dynamical system driven by the object-dependent forward scattering operator. The sequence index in the angle of illumination plays the role of discrete time in the dynamical system analogy. Thus, the imaging problem turns into a problem of nonlinear system identification, which also suggests dynamical learning as a better fit to regularize the reconstructions. We devised a Recurrent Neural Network (RNN) architecture with a novel Separable-Convolution Gated Recurrent Unit (SC-GRU) as the fundamental building block. Through a comprehensive comparison of several quantitative metrics, we show that the dynamic method is suitable for a generic interior-volumetric reconstruction under a limited-angle scheme. We show that this approach accurately reconstructs volume interiors under two conditions: weak scattering, when the Radon transform approximation is applicable and the forward operator well defined; and strong scattering, which is nonlinear with respect to the 3D refractive index distribution and includes uncertainty in the forward operator.
Collapse
Affiliation(s)
- Iksung Kang
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA, USA.
| | - Alexandre Goy
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Omnisens SA, Morges, 1110, Switzerland
| | - George Barbastathis
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Singapore-MIT Alliance for Research and Technology (SMART) Centre, 1 Create Way, Singapore, 117543, Singapore
| |
Collapse
|