1
|
Bresci A, Kobayashi-Kirschvink KJ, Cerullo G, Vanna R, So PTC, Polli D, Kang JW. Label-free morpho-molecular phenotyping of living cancer cells by combined Raman spectroscopy and phase tomography. Commun Biol 2024; 7:785. [PMID: 38951178 PMCID: PMC11217291 DOI: 10.1038/s42003-024-06496-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Accepted: 06/23/2024] [Indexed: 07/03/2024] Open
Abstract
Accurate, rapid and non-invasive cancer cell phenotyping is a pressing concern across the life sciences, as standard immuno-chemical imaging and omics require extended sample manipulation. Here we combine Raman micro-spectroscopy and phase tomography to achieve label-free morpho-molecular profiling of human colon cancer cells, following the adenoma, carcinoma, and metastasis disease progression, in living and unperturbed conditions. We describe how to decode and interpret quantitative chemical and co-registered morphological cell traits from Raman fingerprint spectra and refractive index tomograms. Our multimodal imaging strategy rapidly distinguishes cancer phenotypes, limiting observations to a low number of pristine cells in culture. This synergistic dataset allows us to study independent or correlated information in spectral and tomographic maps, and how it benefits cell type inference. This method is a valuable asset in biomedical research, particularly when biological material is in short supply, and it holds the potential for non-invasive monitoring of cancer progression in living organisms.
Collapse
Affiliation(s)
- Arianna Bresci
- G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
- Department of Physics, Politecnico di Milano, Milan, 20133, Italy.
| | - Koseki J Kobayashi-Kirschvink
- G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Klarman Cell Observatory, Broad Institute of MIT and Harvard, Cambridge, MA, 02142, USA
| | - Giulio Cerullo
- Department of Physics, Politecnico di Milano, Milan, 20133, Italy
- CNR-Institute for Photonics and Nanotechnologies (CNR-IFN), Milan, 20133, Italy
| | - Renzo Vanna
- CNR-Institute for Photonics and Nanotechnologies (CNR-IFN), Milan, 20133, Italy
| | - Peter T C So
- G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Dario Polli
- Department of Physics, Politecnico di Milano, Milan, 20133, Italy.
- CNR-Institute for Photonics and Nanotechnologies (CNR-IFN), Milan, 20133, Italy.
| | - Jeon Woong Kang
- G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
| |
Collapse
|
2
|
Liu Y, Uttam S. Perspective on quantitative phase imaging to improve precision cancer medicine. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S22705. [PMID: 38584967 PMCID: PMC10996848 DOI: 10.1117/1.jbo.29.s2.s22705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Revised: 03/03/2024] [Accepted: 03/15/2024] [Indexed: 04/09/2024]
Abstract
Significance Quantitative phase imaging (QPI) offers a label-free approach to non-invasively characterize cellular processes by exploiting their refractive index based intrinsic contrast. QPI captures this contrast by translating refractive index associated phase shifts into intensity-based quantifiable data with nanoscale sensitivity. It holds significant potential for advancing precision cancer medicine by providing quantitative characterization of the biophysical properties of cells and tissue in their natural states. Aim This perspective aims to discuss the potential of QPI to increase our understanding of cancer development and its response to therapeutics. It also explores new developments in QPI methods towards advancing personalized cancer therapy and early detection. Approach We begin by detailing the technical advancements of QPI, examining its implementations across transmission and reflection geometries and phase retrieval methods, both interferometric and non-interferometric. The focus then shifts to QPI's applications in cancer research, including dynamic cell mass imaging for drug response assessment, cancer risk stratification, and in-vivo tissue imaging. Results QPI has emerged as a crucial tool in precision cancer medicine, offering insights into tumor biology and treatment efficacy. Its sensitivity to detecting nanoscale changes holds promise for enhancing cancer diagnostics, risk assessment, and prognostication. The future of QPI is envisioned in its integration with artificial intelligence, morpho-dynamics, and spatial biology, broadening its impact in cancer research. Conclusions QPI presents significant potential in advancing precision cancer medicine and redefining our approach to cancer diagnosis, monitoring, and treatment. Future directions include harnessing high-throughput dynamic imaging, 3D QPI for realistic tumor models, and combining artificial intelligence with multi-omics data to extend QPI's capabilities. As a result, QPI stands at the forefront of cancer research and clinical application in cancer care.
Collapse
Affiliation(s)
- Yang Liu
- University of Illinois Urbana-Champaign, Beckman Institute for Advanced Science and Technology, Cancer Center at Illinois, Department of Bioengineering, Department of Electrical and Computer Engineering, Urbana, Illinois, United States
- University of Pittsburgh, Departments of Medicine and Bioengineering, Pittsburgh, Pennsylvania, United States
| | - Shikhar Uttam
- University of Pittsburgh, Department of Computational and Systems Biology, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
3
|
Chung Y, Hugonnet H, Hong SM, Park Y. Fourier space aberration correction for high resolution refractive index imaging using incoherent light. OPTICS EXPRESS 2024; 32:18790-18799. [PMID: 38859028 DOI: 10.1364/oe.518479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/10/2024] [Accepted: 04/23/2024] [Indexed: 06/12/2024]
Abstract
An aberration correction method is introduced for 3D phase deconvolution microscopy. Our technique capitalizes on multiple illumination patterns to iteratively extract Fourier space aberrations, utilizing the overlapping information inherent in these patterns. By refining the point spread function based on the retrieved aberration data, we significantly improve the precision of refractive index deconvolution. We validate the effectiveness of our method on both synthetic and biological three-dimensional samples, achieving notable enhancements in resolution and measurement accuracy. The method's reliability in aberration retrieval is further confirmed through controlled experiments with intentionally induced spherical aberrations, underscoring its potential for wide-ranging applications in microscopy and biomedicine.
Collapse
|
4
|
Zhang J, Sarollahi M, Luckhart S, Harrison MJ, Vasdekis AE. Quantitative phase imaging by gradient retardance optical microscopy. Sci Rep 2024; 14:9754. [PMID: 38679622 PMCID: PMC11056386 DOI: 10.1038/s41598-024-60057-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/29/2023] [Accepted: 04/18/2024] [Indexed: 05/01/2024] Open
Abstract
Quantitative phase imaging (QPI) has become a vital tool in bioimaging, offering precise measurements of wavefront distortion and, thus, of key cellular metabolism metrics, such as dry mass and density. However, only a few QPI applications have been demonstrated in optically thick specimens, where scattering increases background and reduces contrast. Building upon the concept of structured illumination interferometry, we introduce Gradient Retardance Optical Microscopy (GROM) for QPI of both thin and thick samples. GROM transforms any standard Differential Interference Contrast (DIC) microscope into a QPI platform by incorporating a liquid crystal retarder into the illumination path, enabling independent phase-shifting of the DIC microscope's sheared beams. GROM greatly simplifies related configurations, reduces costs, and eradicates energy losses in parallel imaging modalities, such as fluorescence. We successfully tested GROM on a diverse range of specimens, from microbes and red blood cells to optically thick (~ 300 μm) plant roots without fixation or clearing.
Collapse
Affiliation(s)
- Jinming Zhang
- Department of Physics, University of Idaho, 875 Perimeter Drive, Moscow, ID, 83844, USA
| | - Mirsaeid Sarollahi
- Department of Physics, University of Idaho, 875 Perimeter Drive, Moscow, ID, 83844, USA
| | - Shirley Luckhart
- Department of Entomology, Plant Pathology and Nematology, University of Idaho, 875 Perimeter Drive, Moscow, ID, 83844, USA
- Department of Biological Sciences, University of Idaho, 875 Perimeter Drive, Moscow, ID, 83844, USA
| | | | - Andreas E Vasdekis
- Department of Physics, University of Idaho, 875 Perimeter Drive, Moscow, ID, 83844, USA.
| |
Collapse
|
5
|
Verrier N, Debailleul M, Haeberlé O. Recent Advances and Current Trends in Transmission Tomographic Diffraction Microscopy. SENSORS (BASEL, SWITZERLAND) 2024; 24:1594. [PMID: 38475130 DOI: 10.3390/s24051594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Revised: 02/21/2024] [Accepted: 02/27/2024] [Indexed: 03/14/2024]
Abstract
Optical microscopy techniques are among the most used methods in biomedical sample characterization. In their more advanced realization, optical microscopes demonstrate resolution down to the nanometric scale. These methods rely on the use of fluorescent sample labeling in order to break the diffraction limit. However, fluorescent molecules' phototoxicity or photobleaching is not always compatible with the investigated samples. To overcome this limitation, quantitative phase imaging techniques have been proposed. Among these, holographic imaging has demonstrated its ability to image living microscopic samples without staining. However, for a 3D assessment of samples, tomographic acquisitions are needed. Tomographic Diffraction Microscopy (TDM) combines holographic acquisitions with tomographic reconstructions. Relying on a 3D synthetic aperture process, TDM allows for 3D quantitative measurements of the complex refractive index of the investigated sample. Since its initial proposition by Emil Wolf in 1969, the concept of TDM has found a lot of applications and has become one of the hot topics in biomedical imaging. This review focuses on recent achievements in TDM development. Current trends and perspectives of the technique are also discussed.
Collapse
Affiliation(s)
- Nicolas Verrier
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| | - Matthieu Debailleul
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| | - Olivier Haeberlé
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, IUT Mulhouse, 61 rue Albert Camus, 68093 Mulhouse, France
| |
Collapse
|
6
|
Pirone D, Bianco V, Miccio L, Memmolo P, Psaltis D, Ferraro P. Beyond fluorescence: advances in computational label-free full specificity in 3D quantitative phase microscopy. Curr Opin Biotechnol 2024; 85:103054. [PMID: 38142647 DOI: 10.1016/j.copbio.2023.103054] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Revised: 11/23/2023] [Accepted: 11/28/2023] [Indexed: 12/26/2023]
Abstract
Despite remarkable progresses in quantitative phase imaging (QPI) microscopes, their wide acceptance is limited due to the lack of specificity compared with the well-established fluorescence microscopy. In fact, the absence of fluorescent tag prevents to identify subcellular structures in single cells, making challenging the interpretation of label-free 2D and 3D phase-contrast data. Great effort has been made by many groups worldwide to address and overcome such limitation. Different computational methods have been proposed and many more are currently under investigation to achieve label-free microscopic imaging at single-cell level to recognize and quantify different subcellular compartments. This route promises to bridge the gap between QPI and FM for real-world applications.
Collapse
Affiliation(s)
- Daniele Pirone
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Vittorio Bianco
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Lisa Miccio
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Pasquale Memmolo
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Demetri Psaltis
- EPFL, Ecole Polytechnique Fédérale de Lausanne, Optics Laboratory, CH-1015 Lausanne, Switzerland
| | - Pietro Ferraro
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| |
Collapse
|
7
|
Dunn KJ, Matlock A, Funkenbusch G, Yaqoob Z, So PTC, Berger AJ. Optical diffraction tomography for assessing single cell models in angular light scattering. BIOMEDICAL OPTICS EXPRESS 2024; 15:973-990. [PMID: 38404316 PMCID: PMC10890861 DOI: 10.1364/boe.512149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 01/09/2024] [Accepted: 01/09/2024] [Indexed: 02/27/2024]
Abstract
Angularly resolved light scattering (ALS) has become a useful tool for assessing the size and refractive index of biological scatterers at cellular and organelle length scales. Sizing organelle populations with ALS relies on Mie scattering theory models, which require significant assumptions about the object, including spherical scatterers and a homogeneous medium. These assumptions may incur greater error at the single cell level, where there are fewer scatterers to be averaged over. We investigate the validity of these assumptions using 3D refractive index (RI) tomograms measured via optical diffraction tomography (ODT). We compute the angular scattering on digitally manipulated tomograms with increasingly strong model assumptions, including RI-matched immersion media, homogeneous cytosol, and spherical organelles. We also compare the tomogram-computed angular scattering to experimental measurements of angular scattering from the same cells to ensure that the ODT-based approach accurately models angular scattering. We show that enforced RI-matching with the immersion medium and a homogeneous cytosol significantly affects the angular scattering intensity shape, suggesting that these assumptions can reduce the accuracy of size distribution estimates.
Collapse
Affiliation(s)
- Kaitlin J. Dunn
- The Institute of Optics, University of Rochester, Rochester, NY, USA
| | - Alex Matlock
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | - Zahid Yaqoob
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biomedical Engineering, Boston University, Boston, MA, USA
| | - Peter T. C. So
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Andrew J. Berger
- The Institute of Optics, University of Rochester, Rochester, NY, USA
| |
Collapse
|
8
|
Sun J, Yang B, Koukourakis N, Guck J, Czarske JW. AI-driven projection tomography with multicore fibre-optic cell rotation. Nat Commun 2024; 15:147. [PMID: 38167247 PMCID: PMC10762230 DOI: 10.1038/s41467-023-44280-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Accepted: 12/06/2023] [Indexed: 01/05/2024] Open
Abstract
Optical tomography has emerged as a non-invasive imaging method, providing three-dimensional insights into subcellular structures and thereby enabling a deeper understanding of cellular functions, interactions, and processes. Conventional optical tomography methods are constrained by a limited illumination scanning range, leading to anisotropic resolution and incomplete imaging of cellular structures. To overcome this problem, we employ a compact multi-core fibre-optic cell rotator system that facilitates precise optical manipulation of cells within a microfluidic chip, achieving full-angle projection tomography with isotropic resolution. Moreover, we demonstrate an AI-driven tomographic reconstruction workflow, which can be a paradigm shift from conventional computational methods, often demanding manual processing, to a fully autonomous process. The performance of the proposed cell rotation tomography approach is validated through the three-dimensional reconstruction of cell phantoms and HL60 human cancer cells. The versatility of this learning-based tomographic reconstruction workflow paves the way for its broad application across diverse tomographic imaging modalities, including but not limited to flow cytometry tomography and acoustic rotation tomography. Therefore, this AI-driven approach can propel advancements in cell biology, aiding in the inception of pioneering therapeutics, and augmenting early-stage cancer diagnostics.
Collapse
Affiliation(s)
- Jiawei Sun
- Shanghai Artificial Intelligence Laboratory, Longwen Road 129, Xuhui District, 200232, Shanghai, China.
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany.
| | - Bin Yang
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany
| | - Nektarios Koukourakis
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany
| | - Jochen Guck
- Max Planck Institute for the Science of Light & Max Planck-Zentrum für Physik und Medizin, 91058, Erlangen, Germany
| | - Juergen W Czarske
- Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden, Helmholtzstrasse 18, 01069, Dresden, Germany.
- Laboratory of Measurement and Sensor System Technique (MST), TU Dresden, Dresden, Germany.
- Cluster of Excellence Physics of Life, TU Dresden, Dresden, Germany.
- Institute of Applied Physics, TU Dresden, Dresden, Germany.
| |
Collapse
|
9
|
Lee MJ, Kim B, Lee D, Kim G, Chung Y, Shin HS, Choi S, Park Y. Enhanced functionalities of immune cells separated by a microfluidic lattice: assessment based on holotomography. BIOMEDICAL OPTICS EXPRESS 2023; 14:6127-6137. [PMID: 38420329 PMCID: PMC10898572 DOI: 10.1364/boe.503957] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/28/2023] [Revised: 10/17/2023] [Accepted: 10/17/2023] [Indexed: 03/02/2024]
Abstract
The isolation of white blood cells (WBCs) from whole blood constitutes a pivotal process for immunological studies, diagnosis of hematologic disorders, and the facilitation of immunotherapy. Despite the ubiquity of density gradient centrifugation in WBC isolation, its influence on WBC functionality remains inadequately understood. This research employs holotomography to explore the effects of two distinct WBC separation techniques, namely conventional centrifugation and microfluidic separation, on the functionality of the isolated cells. We utilize three-dimensional refractive index distribution and time-lapse dynamics to analyze individual WBCs in-depth, focusing on their morphology, motility, and phagocytic capabilities. Our observations highlight that centrifugal processes negatively impact WBC motility and phagocytic capacity, whereas microfluidic separation yields a more favorable outcome in preserving WBC functionality. These findings emphasize the potential of microfluidic separation techniques as a viable alternative to traditional centrifugation for WBC isolation, potentially enabling more precise analyses in immunology research and improving the accuracy of hematologic disorder diagnoses.
Collapse
Affiliation(s)
- Mahn Jae Lee
- Graduate School of Medical Science and Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Byungyeon Kim
- Department of Electronic Engineering, Hanyang University, Seoul 04763, Republic of Korea
| | - Dohyeon Lee
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Physics, KAIST, Daejeon 34141, Republic of Korea
| | - Geon Kim
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Physics, KAIST, Daejeon 34141, Republic of Korea
| | - Yoonjae Chung
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Physics, KAIST, Daejeon 34141, Republic of Korea
| | - Hee Sik Shin
- Department of Electronic Engineering, Hanyang University, Seoul 04763, Republic of Korea
| | - Sungyoung Choi
- Department of Electronic Engineering, Hanyang University, Seoul 04763, Republic of Korea
- Department of Biomedical Engineering, Hanyang University, Seoul 04763, Republic of Korea
| | - YongKeun Park
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Electronic Engineering, Hanyang University, Seoul 04763, Republic of Korea
- Tomocube Inc., Daejeon 34109, Republic of Korea
| |
Collapse
|
10
|
Astratov VN, Sahel YB, Eldar YC, Huang L, Ozcan A, Zheludev N, Zhao J, Burns Z, Liu Z, Narimanov E, Goswami N, Popescu G, Pfitzner E, Kukura P, Hsiao YT, Hsieh CL, Abbey B, Diaspro A, LeGratiet A, Bianchini P, Shaked NT, Simon B, Verrier N, Debailleul M, Haeberlé O, Wang S, Liu M, Bai Y, Cheng JX, Kariman BS, Fujita K, Sinvani M, Zalevsky Z, Li X, Huang GJ, Chu SW, Tzang O, Hershkovitz D, Cheshnovsky O, Huttunen MJ, Stanciu SG, Smolyaninova VN, Smolyaninov II, Leonhardt U, Sahebdivan S, Wang Z, Luk’yanchuk B, Wu L, Maslov AV, Jin B, Simovski CR, Perrin S, Montgomery P, Lecler S. Roadmap on Label-Free Super-Resolution Imaging. LASER & PHOTONICS REVIEWS 2023; 17:2200029. [PMID: 38883699 PMCID: PMC11178318 DOI: 10.1002/lpor.202200029] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Indexed: 06/18/2024]
Abstract
Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles which need to be overcome to break the classical diffraction limit of the LFSR imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability which are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.
Collapse
Affiliation(s)
- Vasily N. Astratov
- Department of Physics and Optical Science, University of North Carolina at Charlotte, Charlotte, North Carolina 28223-0001, USA
| | - Yair Ben Sahel
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot 7610001, Israel
| | - Yonina C. Eldar
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot 7610001, Israel
| | - Luzhe Huang
- Electrical and Computer Engineering Department, University of California, Los Angeles, California 90095, USA
- Bioengineering Department, University of California, Los Angeles, California 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California 90095, USA
| | - Aydogan Ozcan
- Electrical and Computer Engineering Department, University of California, Los Angeles, California 90095, USA
- Bioengineering Department, University of California, Los Angeles, California 90095, USA
- California Nano Systems Institute (CNSI), University of California, Los Angeles, California 90095, USA
- David Geffen School of Medicine, University of California, Los Angeles, California 90095, USA
| | - Nikolay Zheludev
- Optoelectronics Research Centre, University of Southampton, Southampton, SO17 1BJ, UK
- Centre for Disruptive Photonic Technologies, The Photonics Institute, School of Physical and Mathematical Sciences, Nanyang Technological University, 637371, Singapore
| | - Junxiang Zhao
- Department of Electrical and Computer Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| | - Zachary Burns
- Department of Electrical and Computer Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| | - Zhaowei Liu
- Department of Electrical and Computer Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
- Material Science and Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, California 92093, USA
| | - Evgenii Narimanov
- School of Electrical Engineering, and Birck Nanotechnology Center, Purdue University, West Lafayette, Indiana 47907, USA
| | - Neha Goswami
- Quantitative Light Imaging Laboratory, Beckman Institute of Advanced Science and Technology, University of Illinois at Urbana-Champaign, Illinois 61801, USA
| | - Gabriel Popescu
- Quantitative Light Imaging Laboratory, Beckman Institute of Advanced Science and Technology, University of Illinois at Urbana-Champaign, Illinois 61801, USA
| | - Emanuel Pfitzner
- Department of Chemistry, University of Oxford, Oxford OX1 3QZ, United Kingdom
| | - Philipp Kukura
- Department of Chemistry, University of Oxford, Oxford OX1 3QZ, United Kingdom
| | - Yi-Teng Hsiao
- Institute of Atomic and Molecular Sciences (IAMS), Academia Sinica 1, Roosevelt Rd. Sec. 4, Taipei 10617 Taiwan
| | - Chia-Lung Hsieh
- Institute of Atomic and Molecular Sciences (IAMS), Academia Sinica 1, Roosevelt Rd. Sec. 4, Taipei 10617 Taiwan
| | - Brian Abbey
- Australian Research Council Centre of Excellence for Advanced Molecular Imaging, La Trobe University, Melbourne, Victoria, Australia
- Department of Chemistry and Physics, La Trobe Institute for Molecular Science (LIMS), La Trobe University, Melbourne, Victoria, Australia
| | - Alberto Diaspro
- Optical Nanoscopy and NIC@IIT, CHT, Istituto Italiano di Tecnologia, Via Enrico Melen 83B, 16152 Genoa, Italy
- DIFILAB, Department of Physics, University of Genoa, Via Dodecaneso 33, 16146 Genoa, Italy
| | - Aymeric LeGratiet
- Optical Nanoscopy and NIC@IIT, CHT, Istituto Italiano di Tecnologia, Via Enrico Melen 83B, 16152 Genoa, Italy
- Université de Rennes, CNRS, Institut FOTON - UMR 6082, F-22305 Lannion, France
| | - Paolo Bianchini
- Optical Nanoscopy and NIC@IIT, CHT, Istituto Italiano di Tecnologia, Via Enrico Melen 83B, 16152 Genoa, Italy
- DIFILAB, Department of Physics, University of Genoa, Via Dodecaneso 33, 16146 Genoa, Italy
| | - Natan T. Shaked
- Tel Aviv University, Faculty of Engineering, Department of Biomedical Engineering, Tel Aviv 6997801, Israel
| | - Bertrand Simon
- LP2N, Institut d’Optique Graduate School, CNRS UMR 5298, Université de Bordeaux, Talence France
| | - Nicolas Verrier
- IRIMAS UR UHA 7499, Université de Haute-Alsace, Mulhouse, France
| | | | - Olivier Haeberlé
- IRIMAS UR UHA 7499, Université de Haute-Alsace, Mulhouse, France
| | - Sheng Wang
- School of Physics and Technology, Wuhan University, China
- Wuhan Institute of Quantum Technology, China
| | - Mengkun Liu
- Department of Physics and Astronomy, Stony Brook University, USA
- National Synchrotron Light Source II, Brookhaven National Laboratory, USA
| | - Yeran Bai
- Boston University Photonics Center, Boston, MA 02215, USA
| | - Ji-Xin Cheng
- Boston University Photonics Center, Boston, MA 02215, USA
| | - Behjat S. Kariman
- Optical Nanoscopy and NIC@IIT, CHT, Istituto Italiano di Tecnologia, Via Enrico Melen 83B, 16152 Genoa, Italy
- DIFILAB, Department of Physics, University of Genoa, Via Dodecaneso 33, 16146 Genoa, Italy
| | - Katsumasa Fujita
- Department of Applied Physics and the Advanced Photonics and Biosensing Open Innovation Laboratory (AIST); and the Transdimensional Life Imaging Division, Institute for Open and Transdisciplinary Research Initiatives, Osaka University, Osaka, Japan
| | - Moshe Sinvani
- Faculty of Engineering and the Nano-Technology Center, Bar-Ilan University, Ramat Gan, 52900 Israel
| | - Zeev Zalevsky
- Faculty of Engineering and the Nano-Technology Center, Bar-Ilan University, Ramat Gan, 52900 Israel
| | - Xiangping Li
- Guangdong Provincial Key Laboratory of Optical Fiber Sensing and Communications, Institute of Photonics Technology, Jinan University, Guangzhou 510632, China
| | - Guan-Jie Huang
- Department of Physics and Molecular Imaging Center, National Taiwan University, Taipei 10617, Taiwan
- Brain Research Center, National Tsing Hua University, Hsinchu 30013, Taiwan
| | - Shi-Wei Chu
- Department of Physics and Molecular Imaging Center, National Taiwan University, Taipei 10617, Taiwan
- Brain Research Center, National Tsing Hua University, Hsinchu 30013, Taiwan
| | - Omer Tzang
- School of Chemistry, The Sackler faculty of Exact Sciences, and the Center for Light matter Interactions, and the Tel Aviv University Center for Nanoscience and Nanotechnology, Tel Aviv 69978, Israel
| | - Dror Hershkovitz
- School of Chemistry, The Sackler faculty of Exact Sciences, and the Center for Light matter Interactions, and the Tel Aviv University Center for Nanoscience and Nanotechnology, Tel Aviv 69978, Israel
| | - Ori Cheshnovsky
- School of Chemistry, The Sackler faculty of Exact Sciences, and the Center for Light matter Interactions, and the Tel Aviv University Center for Nanoscience and Nanotechnology, Tel Aviv 69978, Israel
| | - Mikko J. Huttunen
- Laboratory of Photonics, Physics Unit, Tampere University, FI-33014, Tampere, Finland
| | - Stefan G. Stanciu
- Center for Microscopy – Microanalysis and Information Processing, Politehnica University of Bucharest, 313 Splaiul Independentei, 060042, Bucharest, Romania
| | - Vera N. Smolyaninova
- Department of Physics Astronomy and Geosciences, Towson University, 8000 York Rd., Towson, MD 21252, USA
| | - Igor I. Smolyaninov
- Department of Electrical and Computer Engineering, University of Maryland, College Park, MD 20742, USA
| | - Ulf Leonhardt
- Weizmann Institute of Science, Rehovot 7610001, Israel
| | - Sahar Sahebdivan
- EMTensor GmbH, TechGate, Donau-City-Strasse 1, 1220 Wien, Austria
| | - Zengbo Wang
- School of Computer Science and Electronic Engineering, Bangor University, Bangor, LL57 1UT, United Kingdom
| | - Boris Luk’yanchuk
- Faculty of Physics, Lomonosov Moscow State University, Moscow 119991, Russia
| | - Limin Wu
- Department of Materials Science and State Key Laboratory of Molecular Engineering of Polymers, Fudan University, Shanghai 200433, China
| | - Alexey V. Maslov
- Department of Radiophysics, University of Nizhny Novgorod, Nizhny Novgorod, 603022, Russia
| | - Boya Jin
- Department of Physics and Optical Science, University of North Carolina at Charlotte, Charlotte, North Carolina 28223-0001, USA
| | - Constantin R. Simovski
- Department of Electronics and Nano-Engineering, Aalto University, FI-00076, Espoo, Finland
- Faculty of Physics and Engineering, ITMO University, 199034, St-Petersburg, Russia
| | - Stephane Perrin
- ICube Research Institute, University of Strasbourg - CNRS - INSA de Strasbourg, 300 Bd. Sébastien Brant, 67412 Illkirch, France
| | - Paul Montgomery
- ICube Research Institute, University of Strasbourg - CNRS - INSA de Strasbourg, 300 Bd. Sébastien Brant, 67412 Illkirch, France
| | - Sylvain Lecler
- ICube Research Institute, University of Strasbourg - CNRS - INSA de Strasbourg, 300 Bd. Sébastien Brant, 67412 Illkirch, France
| |
Collapse
|
11
|
Shaked NT, Boppart SA, Wang LV, Popp J. Label-free biomedical optical imaging. NATURE PHOTONICS 2023; 17:1031-1041. [PMID: 38523771 PMCID: PMC10956740 DOI: 10.1038/s41566-023-01299-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 08/22/2023] [Indexed: 03/22/2024]
Abstract
Label-free optical imaging employs natural and nondestructive approaches for the visualisation of biomedical samples for both biological assays and clinical diagnosis. Currently, this field revolves around multiple broad technology-oriented communities, each with a specific focus on a particular modality despite the existence of shared challenges and applications. As a result, biologists or clinical researchers who require label-free imaging are often not aware of the most appropriate modality to use. This manuscript presents a comprehensive review of and comparison among different label-free imaging modalities and discusses common challenges and applications. We expect this review to facilitate collaborative interactions between imaging communities, push the field forward and foster technological advancements, biophysical discoveries, as well as clinical detection, diagnosis, and monitoring of disease.
Collapse
Affiliation(s)
- Natan T Shaked
- Department of Biomedical Engineering, Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel
| | - Stephen A Boppart
- Beckman Institute for Advanced Science and Technology, Department of Electrical and Computer Engineering,; Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, IL, USA
| | - Lihong V Wang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, Pasadena, CA, USA
| | - Jürgen Popp
- Leibniz Institute of Photonic Technology, Member of Leibniz Health Technologies, Member of the Leibniz Centre for Photonics in Infection Research, Jena, Germany; Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich Schiller University Jena, Jena, Germany
| |
Collapse
|
12
|
Bianco V, D'Agostino M, Pirone D, Giugliano G, Mosca N, Di Summa M, Scerra G, Memmolo P, Miccio L, Russo T, Stella E, Ferraro P. Label-Free Intracellular Multi-Specificity in Yeast Cells by Phase-Contrast Tomographic Flow Cytometry. SMALL METHODS 2023; 7:e2300447. [PMID: 37670547 DOI: 10.1002/smtd.202300447] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/04/2023] [Revised: 08/14/2023] [Indexed: 09/07/2023]
Abstract
In-flow phase-contrast tomography provides a 3D refractive index of label-free cells in cytometry systems. Its major limitation, as with any quantitative phase imaging approach, is the lack of specificity compared to fluorescence microscopy, thus restraining its huge potentialities in single-cell analysis and diagnostics. Remarkable results in introducing specificity are obtained through artificial intelligence (AI), but only for adherent cells. However, accessing the 3D fluorescence ground truth and obtaining accurate voxel-level co-registration of image pairs for AI training is not viable for high-throughput cytometry. The recent statistical inference approach is a significant step forward for label-free specificity but remains limited to cells' nuclei. Here, a generalized computational strategy based on a self-consistent statistical inference to achieve intracellular multi-specificity is shown. Various subcellular compartments (i.e., nuclei, cytoplasmic vacuoles, the peri-vacuolar membrane area, cytoplasm, vacuole-nucleus contact site) can be identified and characterized quantitatively at different phases of the cells life cycle by using yeast cells as a biological model. Moreover, for the first time, virtual reality is introduced for handling the information content of multi-specificity in single cells. Full fruition is proofed for exploring and interacting with 3D quantitative biophysical parameters of the identified compartments on demand, thus opening the route to a metaverse for 3D microscopy.
Collapse
Affiliation(s)
- Vittorio Bianco
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| | - Massimo D'Agostino
- Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Via S. Pansini 5, Naples, 80131, Italy
| | - Daniele Pirone
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| | - Giusy Giugliano
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| | - Nicola Mosca
- Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Via Amendola 122/D-O, Bari, 70125, Italy
| | - Maria Di Summa
- Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Via Amendola 122/D-O, Bari, 70125, Italy
| | - Gianluca Scerra
- Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Via S. Pansini 5, Naples, 80131, Italy
| | - Pasquale Memmolo
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| | - Lisa Miccio
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| | - Tommaso Russo
- Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Via S. Pansini 5, Naples, 80131, Italy
| | - Ettore Stella
- Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing, National Research Council of Italy, Via Amendola 122/D-O, Bari, 70125, Italy
| | - Pietro Ferraro
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, Pozzuoli, Napoli, 80078, Italy
| |
Collapse
|
13
|
Mazur M, Krauze W. Volumetric segmentation of biological cells and subcellular structures for optical diffraction tomography images. BIOMEDICAL OPTICS EXPRESS 2023; 14:5022-5035. [PMID: 37854559 PMCID: PMC10581803 DOI: 10.1364/boe.498275] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Revised: 07/27/2023] [Accepted: 07/31/2023] [Indexed: 10/20/2023]
Abstract
Three-dimensional, quantitative imaging of biological cells and their internal structures performed by optical diffraction tomography (ODT) is an important part of biomedical research. However, conducting quantitative analysis of ODT images requires performing 3D segmentation with high accuracy, often unattainable with available segmentation methods. Therefore, in this work, we present a new semi-automatic method, called ODT-SAS, which combines several non-machine-learning techniques to segment cells and 2 types of their organelles: nucleoli and lipid structures (LS). ODT-SAS has been compared with Cellpose and slice-by-slice manual segmentation, respectively, in cell segmentation and organelles segmentation. The comparison shows superiority of ODT-SAS over Cellpose and reveals the potential of our technique in detecting cells, nucleoli and LS.
Collapse
Affiliation(s)
- Martyna Mazur
- Warsaw University of Technology, 8 Boboli Str., Warsaw, 02-525, Poland
| | - Wojciech Krauze
- Warsaw University of Technology, 8 Boboli Str., Warsaw, 02-525, Poland
| |
Collapse
|
14
|
Kang S, Zhou R, Brelen M, Mak HK, Lin Y, So PTC, Yaqoob Z. Mapping nanoscale topographic features in thick tissues with speckle diffraction tomography. LIGHT, SCIENCE & APPLICATIONS 2023; 12:200. [PMID: 37607903 PMCID: PMC10444882 DOI: 10.1038/s41377-023-01240-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 07/11/2023] [Accepted: 07/19/2023] [Indexed: 08/24/2023]
Abstract
Resolving three-dimensional morphological features in thick specimens remains a significant challenge for label-free imaging. We report a new speckle diffraction tomography (SDT) approach that can image thick biological specimens with ~500 nm lateral resolution and ~1 μm axial resolution in a reflection geometry. In SDT, multiple-scattering background is rejected through spatiotemporal gating provided by dynamic speckle-field interferometry, while depth-resolved refractive index maps are reconstructed by developing a comprehensive inverse-scattering model that also considers specimen-induced aberrations. Benefiting from the high-resolution and full-field quantitative imaging capabilities of SDT, we successfully imaged red blood cells and quantified their membrane fluctuations behind a turbid medium with a thickness of 2.8 scattering mean-free paths. Most importantly, we performed volumetric imaging of cornea inside an ex vivo rat eye and quantified its optical properties, including the mapping of nanoscale topographic features of Dua's and Descemet's membranes that had not been previously visualized.
Collapse
Affiliation(s)
- Sungsam Kang
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Renjie Zhou
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China.
| | - Marten Brelen
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Heather K Mak
- Department of Ophthalmology and Visual Sciences, The Chinese University of Hong Kong, Hong Kong, China
| | - Yuechuan Lin
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Peter T C So
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
- Department of Biological Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Zahid Yaqoob
- Laser Biomedical Research Center, G. R. Harrison Spectroscopy Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
- Department of Biomedical Engineering, Boston University, Boston, MA, 02215, USA.
| |
Collapse
|
15
|
Bazow B, Phan T, Raub CB, Nehmetallah G. Three-dimensional refractive index estimation based on deep-inverse non-interferometric optical diffraction tomography (ODT-Deep). OPTICS EXPRESS 2023; 31:28382-28399. [PMID: 37710893 DOI: 10.1364/oe.491707] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/28/2023] [Accepted: 07/27/2023] [Indexed: 09/16/2023]
Abstract
Optical diffraction tomography (ODT) solves an inverse scattering problem to obtain label-free, 3D refractive index (RI) estimation of biological specimens. This work demonstrates 3D RI retrieval methods suitable for partially-coherent ODT systems supported by intensity-only measurements consisting of axial and angular illumination scanning. This framework allows for access to 3D quantitative RI contrast using a simplified non-interferometric technique. We consider a traditional iterative tomographic solver based on a multiple in-plane representation of the optical scattering process and gradient descent optimization adapted for focus-scanning systems, as well as an approach that relies solely on 3D convolutional neural networks (CNNs) to invert the scattering process. The approaches are validated using simulations of the 3D scattering potential for weak phase 3D biological samples.
Collapse
|
16
|
Gontarz M, Dutta V, Kujawińska M, Krauze W. Phase unwrapping using deep learning in holographic tomography. OPTICS EXPRESS 2023; 31:18964-18992. [PMID: 37381325 DOI: 10.1364/oe.486984] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 03/29/2023] [Indexed: 06/30/2023]
Abstract
Holographic tomography (HT) is a measurement technique that generates phase images, often containing high noise levels and irregularities. Due to the nature of phase retrieval algorithms within the HT data processing, the phase has to be unwrapped before tomographic reconstruction. Conventional algorithms lack noise robustness, reliability, speed, and possible automation. In order to address these problems, this work proposes a convolutional neural network based pipeline consisting of two steps: denoising and unwrapping. Both steps are carried out under the umbrella of a U-Net architecture; however, unwrapping is aided by introducing Attention Gates (AG) and Residual Blocks (RB) to the architecture. Through the experiments, the proposed pipeline makes possible the phase unwrapping of highly irregular, noisy, and complex experimental phase images captured in HT. This work proposes phase unwrapping carried out by segmentation with a U-Net network, that is aided by a pre-processing denoising step. It also discusses the implementation of the AGs and RBs in an ablation study. What is more, this is the first deep learning based solution that is trained solely on real images acquired with HT.
Collapse
|
17
|
Moser S, Jesacher A, Ritsch-Marte M. Efficient and accurate intensity diffraction tomography of multiple-scattering samples. OPTICS EXPRESS 2023; 31:18274-18289. [PMID: 37381541 DOI: 10.1364/oe.486296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/25/2023] [Indexed: 06/30/2023]
Abstract
Optical Diffraction Tomography (ODT) is a label-free method to quantitatively estimate the 3D refractive index (RI) distributions of microscopic samples. Recently, significant efforts were directed towards methods to model multiple-scattering objects. The fidelity of reconstructions rely on accurately modelling light-matter interactions, but the efficient simulation of light propagation through high-RI structures over a large range of illumination angles is still challenging. Here we present a solution dealing with these problems, proposing a method that allows one to efficiently model the tomographic image formation for strongly scattering objects illuminated over a wide range of angles. Instead of propagating tilted plane waves we apply rotations on the illuminated object and optical field and formulate a new and robust multi-slice model suitable for high-RI contrast structures. We test reconstructions made by our approach against simulations and experiments, using rigorous solutions to Maxwell's equations as ground truth. We find the proposed method to produce reconstructions of higher fidelity compared to conventional multi-slice methods, especially for the challenging case of strongly scattering samples where conventional reconstruction methods fail.
Collapse
|
18
|
Pirone D, Montella A, Sirico DG, Mugnano M, Villone MM, Bianco V, Miccio L, Porcelli AM, Kurelac I, Capasso M, Iolascon A, Maffettone PL, Memmolo P, Ferraro P. Label-free liquid biopsy through the identification of tumor cells by machine learning-powered tomographic phase imaging flow cytometry. Sci Rep 2023; 13:6042. [PMID: 37055398 PMCID: PMC10101968 DOI: 10.1038/s41598-023-32110-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/21/2023] [Indexed: 04/15/2023] Open
Abstract
Image-based identification of circulating tumor cells in microfluidic cytometry condition is one of the most challenging perspectives in the Liquid Biopsy scenario. Here we show a machine learning-powered tomographic phase imaging flow cytometry system capable to provide high-throughput 3D phase-contrast tomograms of each single cell. In fact, we show that discrimination of tumor cells against white blood cells is potentially achievable with the aid of artificial intelligence in a label-free flow-cyto-tomography method. We propose a hierarchical machine learning decision-maker, working on a set of features calculated from the 3D tomograms of the cells' refractive index. We prove that 3D morphological features are adequately distinctive to identify tumor cells versus the white blood cell background in the first stage and, moreover, in recognizing the tumor type at the second decision step. Proof-of-concept experiments are shown, in which two different tumor cell lines, namely neuroblastoma cancer cells and ovarian cancer cells, are used against monocytes. The reported results allow claiming the identification of tumor cells with a success rate higher than 97% and with an accuracy over 97% in discriminating between the two cancer cell types, thus opening in a near future the route to a new Liquid Biopsy tool for detecting and classifying circulating tumor cells in blood by stain-free method.
Collapse
Affiliation(s)
- Daniele Pirone
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy
| | - Annalaura Montella
- CEINGE Advanced Biotechnologies, Naples, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Naples, Italy
| | - Daniele G Sirico
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy
| | - Martina Mugnano
- Department of Chemical, Materials and Production Engineering, DICMaPI, University of Naples "Federico II", Piazzale Tecchio 80, 80125, Naples, Italy
| | - Massimiliano M Villone
- Department of Chemical, Materials and Production Engineering, DICMaPI, University of Naples "Federico II", Piazzale Tecchio 80, 80125, Naples, Italy
| | - Vittorio Bianco
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy
| | - Lisa Miccio
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy
| | - Anna Maria Porcelli
- Department of Pharmacy and Biotechnology (FABIT), University of Bologna, Bologna, Italy
- Interdepartmental Centre for Industrial Research 'Scienze Della Vita e Tecnologie per La Salute', University of Bologna, Bologna, Italy
- Centre for Applied Biomedical Research (CRBA), University of Bologna, Bologna, Italy
| | - Ivana Kurelac
- Centre for Applied Biomedical Research (CRBA), University of Bologna, Bologna, Italy
- DIMEC, Department of Medical and Surgical Sciences, Centro di Studio e Ricerca Sulle Neoplasie (CSR) Ginecologiche, Alma Mater Studiorum-University of Bologna, 40138, Bologna, Italy
| | - Mario Capasso
- CEINGE Advanced Biotechnologies, Naples, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Naples, Italy
| | - Achille Iolascon
- CEINGE Advanced Biotechnologies, Naples, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples "Federico II", Naples, Italy
| | - Pier Luca Maffettone
- Department of Chemical, Materials and Production Engineering, DICMaPI, University of Naples "Federico II", Piazzale Tecchio 80, 80125, Naples, Italy
| | - Pasquale Memmolo
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy.
| | - Pietro Ferraro
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "Eduardo Caianiello", Via Campi Flegrei 34, 80078, Pozzuoli, Naples, Italy.
| |
Collapse
|
19
|
Ledwig P, Robles FE. Partially coherent broadband 3D optical transfer functions with arbitrary temporal and angular power spectra. APL PHOTONICS 2023; 8:041301. [PMID: 37038474 PMCID: PMC10080387 DOI: 10.1063/5.0123206] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 03/10/2023] [Indexed: 06/19/2023]
Abstract
Optical diffraction tomography is a powerful technique to produce 3D volumetric images of biological samples using contrast produced by variations in the index of refraction in an unlabeled specimen. While this is typically performed with coherent illumination from a variety of angles, interest has grown in partially coherent methods due to the simplicity of the illumination and the computation-free axial sectioning provided by the coherence window of the source. However, such methods rely on the symmetry or discretization of a source to facilitate quantitative analysis and are unable to efficiently handle arbitrary illumination that may vary asymmetrically in angle and continuously in the spectrum, such as diffusely scattered or thermal sources. A general broadband theory may expand the scope of illumination methods available for quantitative analysis, as partially coherent sources are commonly available and may benefit from the effects of spatial and temporal incoherence. In this work, we investigate partially coherent tomographic phase microscopy from arbitrary sources regardless of angular distribution and spectrum by unifying the effects of spatial and temporal coherence into a single formulation. This approach further yields a method for efficient computation of the overall systems' optical transfer function, which scales with O(n 3), down from O(mn 4) for existing convolutional methods, where n 3 is the number of spatial voxels in 3D space and m is the number of discrete wavelengths in the illumination spectrum. This work has important implications for enabling partially coherent 3D quantitative phase microscopy and refractive index tomography in virtually any transmission or epi-illumination microscope.
Collapse
|
20
|
Rajora S, Butola M, Khare K. 3D reconstruction of unstained weakly scattering cells from a single defocused hologram. APPLIED OPTICS 2023; 62:D146-D156. [PMID: 37132780 DOI: 10.1364/ao.478351] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
We investigate the problem of 3D complex field reconstruction corresponding to unstained red blood cells (RBCs) with a single defocused off-axis digital hologram. The main challenge in this problem is the localization of cells to the correct axial range. While investigating the volume recovery problem for a continuous phase object like the RBC, we observe an interesting feature of the backpropagated field that it does not show a clear focusing effect. Therefore, sparsity enforcement within the iterative optimization framework using a single hologram data frame cannot effectively restrict the reconstruction to the true object volume. For phase objects, it is known that the amplitude contrast of the backpropagated object field at the focus plane is minimum. We use this information available in the recovered object field in the hologram plane to device depth-dependent weights that are proportional to the inverse of amplitude contrast. This weight function is employed in the iterative steps of the optimization algorithm to assist the object volume localization. The overall reconstruction process is performed using the mean gradient descent (MGD) framework. Experimental illustrations of 3D volume reconstruction of the healthy as well as malaria-infected RBCs are presented. A test sample of polystyrene microsphere bead is also used to validate the axial localization capability of the proposed iterative technique. The proposed methodology is simple to implement experimentally and provides an approximate tomographic solution, which is axially restricted and consistent with the object field data.
Collapse
|
21
|
Taddese AM, Lo M, Verrier N, Debailleul M, Haeberlé O. Jones tomographic diffractive microscopy with a polarized array sensor. OPTICS EXPRESS 2023; 31:9034-9051. [PMID: 36860005 DOI: 10.1364/oe.483050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 02/05/2023] [Indexed: 06/18/2023]
Abstract
Tomographic diffractive microscopy (TDM) based on scalar light-field approximation is widely implemented. Samples exhibiting anisotropic structures, however, necessitate accounting for the vectorial nature of light, leading to 3-D quantitative polarimetric imaging. In this work, we have developed a high-numerical aperture (at both illumination and detection) Jones TDM system, with detection multiplexing via a polarized array sensor (PAS), for imaging optically birefringent samples at high resolution. The method is first studied through image simulations. To validate our setup, an experiment using a sample containing both birefringent and non-birefringent objects is performed. Araneus diadematus spider silk fiber and Pinna nobilis oyster shell crystals are finally studied, allowing us to assess both birefringence and fast-axis orientation maps.
Collapse
|
22
|
Verrier N, Taddese AM, Abbessi R, Debailleul M, Haeberlé O. 3D differential interference contrast microscopy using polarisation-sensitive tomographic diffraction microscopy. J Microsc 2023; 289:128-133. [PMID: 36408663 PMCID: PMC10107843 DOI: 10.1111/jmi.13160] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 10/18/2022] [Accepted: 11/14/2022] [Indexed: 11/22/2022]
Abstract
Tomographic diffraction microscopy (TDM) is a generalisation of digital holographic microscopy (DHM), for which the illumination angle onto the sample is fully controlled, which has become a tool of choice for 3D, high-resolution imaging of unlabelled samples. TDM makes it possible to obtain the optical field in both amplitude and phase for each illumination angle. Proper information reallocation eventually allows for 3D reconstruction of the complex refractive index map. On the other hand, polarisation array sensors (PAS) paves new way for TDM, as vectorial information assessment about the investigated sample. In this contribution, we show an alternative use of this polarisation information based on the phase sensitive nature of TDM. Here, we demonstrated that TDM coupled with PAS can lead to a 3D differential interference contrast (DIC) microscope with almost no experimental configuration modification.
Collapse
Affiliation(s)
- Nicolas Verrier
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, Mulhouse Cedex, France
| | - Asemare Mengistie Taddese
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, Mulhouse Cedex, France
| | - Riadh Abbessi
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, Mulhouse Cedex, France
| | - Matthieu Debailleul
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, Mulhouse Cedex, France
| | - Olivier Haeberlé
- Institut Recherche en Informatique, Mathématiques, Automatique et Signal (IRIMAS UR UHA 7499), Université de Haute-Alsace, Mulhouse Cedex, France
| |
Collapse
|
23
|
Matlock A, Zhu J, Tian L. Multiple-scattering simulator-trained neural network for intensity diffraction tomography. OPTICS EXPRESS 2023; 31:4094-4107. [PMID: 36785385 DOI: 10.1364/oe.477396] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 12/29/2022] [Indexed: 06/18/2023]
Abstract
Recovering 3D phase features of complex biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. Here, we overcome this challenge using an approximant-guided deep learning framework in a high-speed intensity diffraction tomography system. Applying a physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework on experimental measurements of weakly scattering epithelial buccal cells and strongly scattering C. elegans worms. We benchmark the network's performance against a state-of-the-art multiple-scattering model-based iterative reconstruction algorithm. We highlight the network's robustness by reconstructing dynamic samples from a living worm video. We further emphasize the network's generalization capabilities by recovering algae samples imaged from different experimental setups. To assess the prediction quality, we develop a quantitative evaluation metric to show that our predictions are consistent with both multiple-scattering physics and experimental measurements.
Collapse
|
24
|
He Y, Zhou N, Ziemczonok M, Wang Y, Lei L, Duan L, Zhou R. Standardizing image assessment in optical diffraction tomography. OPTICS LETTERS 2023; 48:395-398. [PMID: 36638466 DOI: 10.1364/ol.478554] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 12/03/2022] [Indexed: 06/17/2023]
Abstract
Optical diffraction tomography (ODT) has gradually become a popular label-free imaging technique that offers diffraction-limited resolution by mapping an object's three-dimensional (3D) refractive index (RI) distribution. However, there is a lack of comprehensive quantitative image assessment metrics in ODT for studying how various experimental conditions influence image quality, and subsequently optimizing the experimental conditions. In this Letter, we propose to standardize the image assessment in ODT by proposing a set of metrics, including signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and structural distinguishability (SD). To test the feasibility of the metrics, we performed experiments on angle-scanning ODT by varying the number of illumination angles, RI contrast of samples, sample feature sizes, and sample types (e.g., standard polystyrene beads and 3D printed structures) and evaluated the RI tomograms with SNR, CNR, and SD. We further quantitatively studied how image quality can be improved, and tested the image assessment metrics on subcellular structures of living cells. We envision the proposed image assessment metrics may greatly benefit end-users for assessing the RI tomograms, as well as experimentalists for optimizing ODT instruments.
Collapse
|
25
|
Lee D, Lee M, Kwak H, Kim YS, Shim J, Jung JH, Park WS, Park JH, Lee S, Park Y. High-fidelity optical diffraction tomography of live organisms using iodixanol refractive index matching. BIOMEDICAL OPTICS EXPRESS 2022; 13:6404-6415. [PMID: 36589574 PMCID: PMC9774853 DOI: 10.1364/boe.465066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 10/25/2022] [Accepted: 10/25/2022] [Indexed: 06/17/2023]
Abstract
Optical diffraction tomography (ODT) enables the three-dimensional (3D) refractive index (RI) reconstruction. However, when the RI difference between a sample and a medium increases, the effects of light scattering become significant, preventing the acquisition of high-quality and accurate RI reconstructions. Herein, we present a method for high-fidelity ODT by introducing non-toxic RI matching media. Optimally reducing the RI contrast enhances the fidelity and accuracy of 3D RI reconstruction, enabling visualization of the morphology and intra-organization of live biological samples without producing toxic effects. We validate our method using various biological organisms, including C. albicans and C. elegans.
Collapse
Affiliation(s)
- Dohyeon Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Moosung Lee
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Haechan Kwak
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Young Seo Kim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Jaehyu Shim
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Jik Han Jung
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - Wei-sun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
| | - Ji-Ho Park
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Department of Bio and Brain Engineering, KAIST, Daejeon 34141, Republic of Korea
| | - Sumin Lee
- Tomocube Inc., Daejeon 34109, Republic of Korea
| | - YongKeun Park
- Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea
- KAIST Institute for Health Science and Technology, KAIST, Daejeon 34141, Republic of Korea
- Tomocube Inc., Daejeon 34109, Republic of Korea
| |
Collapse
|
26
|
Pirone D, Lim J, Merola F, Miccio L, Mugnano M, Bianco V, Cimmino F, Visconte F, Montella A, Capasso M, Iolascon A, Memmolo P, Psaltis D, Ferraro P. Stain-free identification of cell nuclei using tomographic phase microscopy in flow cytometry. NATURE PHOTONICS 2022; 16:851-859. [PMID: 36451849 PMCID: PMC7613862 DOI: 10.1038/s41566-022-01096-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Quantitative Phase Imaging (QPI) has gained popularity in bioimaging because it can avoid the need for cell staining, which in some cases is difficult or impossible. However, as a result, QPI does not provide labelling of various specific intracellular structures. Here we show a novel computational segmentation method based on statistical inference that makes it possible for QPI techniques to identify the cell nucleus. We demonstrate the approach with refractive index tomograms of stain-free cells reconstructed through the tomographic phase microscopy in flow cytometry mode. In particular, by means of numerical simulations and two cancer cell lines, we demonstrate that the nucleus can be accurately distinguished within the stain-free tomograms. We show that our experimental results are consistent with confocal fluorescence microscopy (FM) data and microfluidic cytofluorimeter outputs. This is a significant step towards extracting specific three-dimensional intracellular structures directly from the phase-contrast data in a typical flow cytometry configuration.
Collapse
Affiliation(s)
- Daniele Pirone
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
- DIETI, Department of Electrical Engineering and Information Technologies, University of Naples “Federico II”, Via Claudio 21, 80125 Napoli, Italy
| | - Joowon Lim
- EPFL, Ecole Polytechnique Fédérale de Lausanne, Optics Laboratory, CH-1015 Lausanne, Switzerland
| | - Francesco Merola
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Lisa Miccio
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Martina Mugnano
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Vittorio Bianco
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Flora Cimmino
- CEINGE - Advanced Biotechnologies, Via Gaetano Salvatore 486, 80131 Napoli, Italy
| | - Feliciano Visconte
- CEINGE - Advanced Biotechnologies, Via Gaetano Salvatore 486, 80131 Napoli, Italy
| | - Annalaura Montella
- CEINGE - Advanced Biotechnologies, Via Gaetano Salvatore 486, 80131 Napoli, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples “Federico II”, Via Pansini 5, 80131 Napoli, Italy
| | - Mario Capasso
- CEINGE - Advanced Biotechnologies, Via Gaetano Salvatore 486, 80131 Napoli, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples “Federico II”, Via Pansini 5, 80131 Napoli, Italy
| | - Achille Iolascon
- CEINGE - Advanced Biotechnologies, Via Gaetano Salvatore 486, 80131 Napoli, Italy
- DMMBM, Department of Molecular Medicine and Medical Biotechnology, University of Naples “Federico II”, Via Pansini 5, 80131 Napoli, Italy
| | - Pasquale Memmolo
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| | - Demetri Psaltis
- EPFL, Ecole Polytechnique Fédérale de Lausanne, Optics Laboratory, CH-1015 Lausanne, Switzerland
| | - Pietro Ferraro
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems “E. Caianiello”, Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy
| |
Collapse
|
27
|
Fanous MJ, He S, Sengupta S, Tangella K, Sobh N, Anastasio MA, Popescu G. White blood cell detection, classification and analysis using phase imaging with computational specificity (PICS). Sci Rep 2022; 12:20043. [PMID: 36414631 PMCID: PMC9681839 DOI: 10.1038/s41598-022-21250-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Accepted: 09/26/2022] [Indexed: 11/23/2022] Open
Abstract
Treatment of blood smears with Wright's stain is one of the most helpful tools in detecting white blood cell abnormalities. However, to diagnose leukocyte disorders, a clinical pathologist must perform a tedious, manual process of locating and identifying individual cells. Furthermore, the staining procedure requires considerable preparation time and clinical infrastructure, which is incompatible with point-of-care diagnosis. Thus, rapid and automated evaluations of unlabeled blood smears are highly desirable. In this study, we used color spatial light interference microcopy (cSLIM), a highly sensitive quantitative phase imaging (QPI) technique, coupled with deep learning tools, to localize, classify and segment white blood cells (WBCs) in blood smears. The concept of combining QPI label-free data with AI for the purpose of extracting cellular specificity has recently been introduced in the context of fluorescence imaging as phase imaging with computational specificity (PICS). We employed AI models to first translate SLIM images into brightfield micrographs, then ran parallel tasks of locating and labelling cells using EfficientNet, which is an object detection model. Next, WBC binary masks were created using U-net, a convolutional neural network that performs precise segmentation. After training on digitally stained brightfield images of blood smears with WBCs, we achieved a mean average precision of 75% for localizing and classifying neutrophils, eosinophils, lymphocytes, and monocytes, and an average pixel-wise majority-voting F1 score of 80% for determining the cell class from semantic segmentation maps. Therefore, PICS renders and analyzes synthetically stained blood smears rapidly, at a reduced cost of sample preparation, providing quantitative clinical information.
Collapse
Affiliation(s)
- Michae J. Fanous
- grid.35403.310000 0004 1936 9991Quantitative Light Imaging Laboratory, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA ,grid.35403.310000 0004 1936 9991Department of Bioengineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA
| | - Shenghua He
- grid.4367.60000 0001 2355 7002Department of Computer Science and Engineering, Washington University in St. Louis, 1 Brookings Drive, St. Louis, MO 63130 USA
| | - Sourya Sengupta
- grid.35403.310000 0004 1936 9991Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA
| | | | - Nahil Sobh
- grid.35403.310000 0004 1936 9991NCSA Center for Artificial Intelligence Innovation, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA
| | - Mark A. Anastasio
- grid.35403.310000 0004 1936 9991Department of Bioengineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA ,grid.35403.310000 0004 1936 9991Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA ,grid.35403.310000 0004 1936 9991Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA
| | - Gabriel Popescu
- grid.35403.310000 0004 1936 9991Quantitative Light Imaging Laboratory, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL 61801 USA ,grid.35403.310000 0004 1936 9991Department of Bioengineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA ,grid.35403.310000 0004 1936 9991Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, 306 N. Wright Street, Urbana, IL 61801 USA
| |
Collapse
|
28
|
Ossowski P, Kuś A, Krauze W, Tamborski S, Ziemczonok M, Kuźbicki Ł, Szkulmowski M, Kujawińska M. Near-infrared, wavelength, and illumination scanning holographic tomography. BIOMEDICAL OPTICS EXPRESS 2022; 13:5971-5988. [PMID: 36733741 PMCID: PMC9872886 DOI: 10.1364/boe.468046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 08/19/2022] [Accepted: 08/24/2022] [Indexed: 06/18/2023]
Abstract
We present a holographic tomography technique in which the projections are acquired using both wavelength and illumination scanning in the near-infrared region. We show how to process the acquired data to obtain correct values of three-dimensional refractive index distributions in both single-wavelength and multi-wavelength data acquisition schemes and how to properly account for the dispersion of the sample. We perform numerical and experimental comparisons of different illumination scenarios to determine the most efficient measurement protocol. We show that the multi-wavelength protocol is advantageous in terms of signal-to-noise ratio and contrast-to-noise ratio over single-wavelength protocols, even for the same number of projections used for reconstructions. Finally, we show that this approach is suitable for providing high-quality refractive index distributions of relatively thick colon cancer samples.
Collapse
Affiliation(s)
- Paweł Ossowski
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Arkadiusz Kuś
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Wojciech Krauze
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Szymon Tamborski
- Institute of Physics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland
| | - Michał Ziemczonok
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| | - Łukasz Kuźbicki
- Institute of Biology, Faculty of Biological and Veterinary Sciences, Nicolaus Copernicus University in Toruń, Lwowska 1, 87-100 Toruń, Poland
| | - Maciej Szkulmowski
- Institute of Physics, Faculty of Physics, Astronomy and Informatics, Nicolaus Copernicus University in Toruń, Grudziadzka 5, 87-100 Toruń, Poland
| | - Małgorzata Kujawińska
- Warsaw University of Technology, Institute of Micromechanics and Photonics, Boboli 8 street, Warsaw, 02-525, Poland
| |
Collapse
|
29
|
Jiang Y, Li H, Pang Y, Ling J, Wang H, Yang Y, Li X, Tian Y, Wang X. Cell image reconstruction using digital holography with an improved GS algorithm. Front Physiol 2022; 13:1040777. [DOI: 10.3389/fphys.2022.1040777] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 10/17/2022] [Indexed: 11/13/2022] Open
Abstract
Digital holography is an effective technology in image reconstruction as amplitude and phase information of cells can be acquired without any staining. In this paper, we propose a holographic technique with an improved Gerchberg-Saxton (GS) algorithm to reconstruct cell imaging based on phase reconstruction information. Comparative experiments are conducted on four specific models to investigate the effectiveness of the proposed method. The morphological parameters (such as shape, volume, and sphericity) of abnormal erythrocytes can be obtained by reconstructing cell hologram of urinary sediment. Notably, abnormal red blood cells can also be detected in mussy circumstances by the proposed method, owing to the significantly biophysical contrast (refractive index distribution and mass density) between two different cells. Therefore, this proposed method has a broad application prospect in cell image reconstruction and cell dynamic detection.
Collapse
|
30
|
Chen C, Gu Y, Xiao Z, Wang H, He X, Jiang Z, Kong Y, Liu C, Xue L, Vargas J, Wang S. Automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. Anal Chim Acta 2022; 1229:340401. [PMID: 36156229 DOI: 10.1016/j.aca.2022.340401] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2022] [Revised: 08/27/2022] [Accepted: 09/11/2022] [Indexed: 11/01/2022]
Abstract
Whole blood cell analysis is widely used in medical applications since its results are indicators for diagnosing a series of diseases. In this work, we report automatic whole blood cell analysis from blood smear using label-free multi-modal imaging with deep neural networks. First, a commercial microscope equipped with our developed Phase Real-time Microscope Camera (PhaseRMiC) obtains both bright-field and quantitative phase images. Then, these images are automatically processed by our designed blood smear recognition networks (BSRNet) that recognize erythrocytes, leukocytes and platelets. Finally, blood cell parameters such as counts, shapes and volumes can be extracted according to both quantitative phase images and automatic recognition results. The proposed whole blood cell analysis technique provides high-quality blood cell images and supports accurate blood cell recognition and analysis. Moreover, this approach requires rather simple and cost-effective setups as well as easy and rapid sample preparations. Therefore, this proposed method has great potential application in blood testing aiming at disease diagnostics.
Collapse
Affiliation(s)
- Chao Chen
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Yuanjie Gu
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Zhibo Xiao
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Hailun Wang
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Xiaoliang He
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Zhilong Jiang
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Yan Kong
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China
| | - Cheng Liu
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China; Shanghai Institute of Optics and Fine Mechanics, Chinese Academy of Sciences, Shanghai, 201800, China
| | - Liang Xue
- College of Electronics and Information Engineering, Shanghai University of Electric Power, Shanghai, 200090, China.
| | - Javier Vargas
- Applied Optics Complutense Group, Optics Department, Universidad Complutense de Madrid, Facultad de CC. Físicas, Plaza de Ciencias, 1, 28040, Madrid, Spain
| | - Shouyu Wang
- Computational Optics Laboratory, School of Science, Jiangnan University, Wuxi, Jiangsu, 214122, China; OptiX+ Laboratory, Wuxi, Jiangsu, China.
| |
Collapse
|
31
|
Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields. NAT MACH INTELL 2022. [DOI: 10.1038/s42256-022-00530-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|
32
|
Zhu J, Wang H, Tian L. High-fidelity intensity diffraction tomography with a non-paraxial multiple-scattering model. OPTICS EXPRESS 2022; 30:32808-32821. [PMID: 36242335 DOI: 10.1364/oe.469503] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 08/09/2022] [Indexed: 06/16/2023]
Abstract
We propose a novel intensity diffraction tomography (IDT) reconstruction algorithm based on the split-step non-paraxial (SSNP) model for recovering the 3D refractive index (RI) distribution of multiple-scattering biological samples. High-quality IDT reconstruction requires high-angle illumination to encode both low- and high- spatial frequency information of the 3D biological sample. We show that our SSNP model can more accurately compute multiple scattering from high-angle illumination compared to paraxial approximation-based multiple-scattering models. We apply this SSNP model to both sequential and multiplexed IDT techniques. We develop a unified reconstruction algorithm for both IDT modalities that is highly computationally efficient and is implemented by a modular automatic differentiation framework. We demonstrate the capability of our reconstruction algorithm on both weakly scattering buccal epithelial cells and strongly scattering live C. elegans worms and live C. elegans embryos.
Collapse
|
33
|
Běhal J, Borrelli F, Mugnano M, Bianco V, Capozzoli A, Curcio C, Liseno A, Miccio L, Memmolo P, Ferraro P. Developing a Reliable Holographic Flow Cyto-Tomography Apparatus by Optimizing the Experimental Layout and Computational Processing. Cells 2022; 11:cells11162591. [PMID: 36010667 PMCID: PMC9406712 DOI: 10.3390/cells11162591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 08/17/2022] [Accepted: 08/18/2022] [Indexed: 11/16/2022] Open
Abstract
Digital Holographic Tomography (DHT) has recently been established as a means of retrieving the 3D refractive index mapping of single cells. To make DHT a viable system, it is necessary to develop a reliable and robust holographic apparatus in order that such technology can be utilized outside of specialized optics laboratories and operated in the in-flow modality. In this paper, we propose a quasi-common-path lateral-shearing holographic optical set-up to be used, for the first time, for DHT in a flow-cytometer modality. The proposed solution is able to withstand environmental vibrations that can severely affect the interference process. Furthermore, we have scaled down the system while ensuring that a full 360° rotation of the cells occurs in the field-of-view, in order to retrieve 3D phase-contrast tomograms of single cells flowing along a microfluidic channel. This was achieved by setting the camera sensor at 45° with respect to the microfluidic direction. Additional optimizations were made to the computational elements to ensure the reliable retrieval of 3D refractive index distributions by demonstrating an effective method of tomographic reconstruction, based on high-order total variation. The results were first demonstrated using realistic 3D numerical phantom cells to assess the performance of the proposed high-order total variation method in comparison with the gold-standard algorithm for tomographic reconstructions: namely, filtered back projection. Then, the proposed DHT system and the processing pipeline were experimentally validated for monocytes and mouse embryonic fibroblast NIH-3T3 cells lines. Moreover, the repeatability of these tomographic measurements was also investigated by recording the same cell multiple times and quantifying the ability to provide reliable and comparable tomographic reconstructions, as confirmed by a correlation coefficient greater than 95%. The reported results represent various steps forward in several key aspects of in-flow DHT, thus paving the way for its use in real-world applications.
Collapse
Affiliation(s)
- Jaromír Běhal
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
| | - Francesca Borrelli
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell’Informazione, Università di Napoli Federico II, 80125 Napoli, Italy
| | - Martina Mugnano
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
| | - Vittorio Bianco
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
| | - Amedeo Capozzoli
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell’Informazione, Università di Napoli Federico II, 80125 Napoli, Italy
| | - Claudio Curcio
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell’Informazione, Università di Napoli Federico II, 80125 Napoli, Italy
| | - Angelo Liseno
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell’Informazione, Università di Napoli Federico II, 80125 Napoli, Italy
| | - Lisa Miccio
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
| | - Pasquale Memmolo
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
- Correspondence:
| | - Pietro Ferraro
- Institute of Applied Sciences and Intelligent Systems, Italian National Research Council (CNR-ISASI), 80078 Pozzuoli, Italy
| |
Collapse
|
34
|
Neurons: The Interplay between Cytoskeleton, Ion Channels/Transporters and Mitochondria. Cells 2022; 11:cells11162499. [PMID: 36010576 PMCID: PMC9406945 DOI: 10.3390/cells11162499] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Revised: 08/06/2022] [Accepted: 08/09/2022] [Indexed: 11/17/2022] Open
Abstract
Neurons are permanent cells whose key feature is information transmission via chemical and electrical signals. Therefore, a finely tuned homeostasis is necessary to maintain function and preserve neuronal lifelong survival. The cytoskeleton, and in particular microtubules, are far from being inert actors in the maintenance of this complex cellular equilibrium, and they participate in the mobilization of molecular cargos and organelles, thus influencing neuronal migration, neuritis growth and synaptic transmission. Notably, alterations of cytoskeletal dynamics have been linked to alterations of neuronal excitability. In this review, we discuss the characteristics of the neuronal cytoskeleton and provide insights into alterations of this component leading to human diseases, addressing how these might affect excitability/synaptic activity, as well as neuronal functioning. We also provide an overview of the microscopic approaches to visualize and assess the cytoskeleton, with a specific focus on mitochondrial trafficking.
Collapse
|
35
|
Wang Y, Liu M, Wei Q, Wu W, He Y, Gao J, Zhou R, Jiang L, Qu J, Xia J. Phase-Separated Multienzyme Compartmentalization for Terpene Biosynthesis in a Prokaryote. Angew Chem Int Ed Engl 2022; 61:e202203909. [PMID: 35562330 DOI: 10.1002/anie.202203909] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Indexed: 01/18/2023]
Abstract
Liquid-liquid phase separation (LLPS) forms biomolecular condensates or coacervates in cells. Metabolic enzymes can form phase-separated subcellular compartments that enrich enzymes, cofactors, and substrates. Herein, we report the construction of synthetic multienzyme condensates that catalyze the biosynthesis of a terpene, α-farnesene, in the prokaryote E. coli. RGGRGG derived from LAF-1 was used as the scaffold protein to form the condensates by LLPS. Multienzyme condensates were then formed by assembling two enzymes Idi and IspA through an RIAD/RIDD interaction. Multienzyme condensates constructed inside E. coli cells compartmentalized the cytosolic space into regions of high and low enzyme density and led to a significant enhancement of α-farnesene production. This work demonstrates LLPS-driven compartmentalization of the cytosolic space of prokaryotic cells, condensation of a biosynthetic pathway, and enhancement of the biosynthesis of α-farnesene.
Collapse
Affiliation(s)
- Yue Wang
- Department of Chemistry and Center for Cell & Developmental Biology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Min Liu
- Department of Chemistry and Center for Cell & Developmental Biology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Qixin Wei
- Department of Chemistry and Center for Cell & Developmental Biology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Wanjie Wu
- Departments of Electronic and Computer Engineering, Center of Systems Biology and Human Health, School of Science and Institute for Advanced Study, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR, China
| | - Yanping He
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Jiayang Gao
- Center for Cell & Developmental Biology, School of Life Sciences, State Key Laboratory of Agrobiotechnology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Renjie Zhou
- Department of Biomedical Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Liwen Jiang
- Center for Cell & Developmental Biology, School of Life Sciences, State Key Laboratory of Agrobiotechnology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| | - Jianan Qu
- Departments of Electronic and Computer Engineering, Center of Systems Biology and Human Health, School of Science and Institute for Advanced Study, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR, China
| | - Jiang Xia
- Department of Chemistry and Center for Cell & Developmental Biology, The Chinese University of Hong Kong, Shatin, Hong Kong SAR, China
| |
Collapse
|
36
|
Lipke W, Winnik J, Trusiak M. Numerical analysis of the effect of reduced temporal coherence in quantitative phase microscopy and tomography. OPTICS EXPRESS 2022; 30:21241-21257. [PMID: 36224847 DOI: 10.1364/oe.458167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 04/20/2022] [Indexed: 06/16/2023]
Abstract
We present the numerical analysis of the effect of the temporarily partially coherent illumination on the phase measurement accuracy in digital holography microscopy (DHM) and optical diffraction tomography (ODT), as reconstruction algorithms tend to assume purely monochromatic conditions. In the regime of reduced temporal coherence, we simulate the hologram formation in two different optical setups, representing classical off-axis two-beam and grating common-path configurations. We consider two ODT variants: with sample rotation and angle-scanning of illumination. Besides the coherence degree of illumination, our simulation considers the influence of the sample normal dispersion, shape of the light spectrum, and optical parameters of the imaging setup. As reconstruction algorithms we employ Fourier hologram method and first-order Rytov approximation with direct inversion and nonnegativity constraints. Quantitative evaluation of the measurement results deviations introduced by the mentioned error sources is comprehensively analyzed, for the first time to the best of our knowledge. Obtained outcomes indicate low final DHM/ODT reconstruction errors for the grating-assisted common-path configuration. Nevertheless, dispersion and asymmetric spectrum introduce non-negligible overestimated refractive index values and noise, and should be thus carefully considered within experimental frameworks.
Collapse
|
37
|
Hu J, Li S, Xie H, Shen Y. Multi-slice ptychographic imaging with multistage coarse-to-fine reconstruction. OPTICS EXPRESS 2022; 30:21211-21229. [PMID: 36224845 DOI: 10.1364/oe.457945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 05/17/2022] [Indexed: 06/16/2023]
Abstract
The ability to image 3D samples with optical sectioning is essential for the study of tomographic morphology in material and biological sciences. However, it is often hampered by limitations of acquisition speed and equipment complexity when performing 3D volumetric imaging. Here, we propose, to the best of our knowledge, a new method for 3D reconstruction from a minimum of four intensity-only measurements. The complementary structured patterns provided by the digital micromirror device (DMD) irradiate the outermost layer of the sample to generate the corresponding diffraction intensities for recording, which enables rapid scanning of loaded patterns for fast acquisition. Our multistage reconstruction algorithm first extracts the overall coarse-grained information, and then iteratively optimizes the information of different layers to obtain fine features, thereby achieving high-resolution 3D tomography. The high-fidelity reconstruction in experiments on two-slice resolution targets, unstained Polyrhachis vicina Roger and freely moving C. elegans proves the robustness of the method. Compared with traditional 3D reconstruction methods such as interferometry-based methods or Fourier ptychographic tomography (FPT), our method increases the reconstruction speed by at least 10 times and is suitable for label-free dynamic imaging in multiple-scattering samples. Such 3D reconstruction suggests potential applications in a wide range of fields.
Collapse
|
38
|
Wang Y, Liu M, Wei Q, Wu W, He Y, Gao J, Zhou R, Jiang L, Qu J, Xia J. Phase‐Separated Multienzyme Compartmentalization for Terpene Biosynthesis in a Prokaryote. Angew Chem Int Ed Engl 2022. [DOI: 10.1002/ange.202203909] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Affiliation(s)
- Yue Wang
- Chinese University of Hong Kong Shaw College: The Chinese University of Hong Kong Chemistry HONG KONG
| | - Min Liu
- The Chinese University of Hong Kong Chemistry HONG KONG
| | - Qixin Wei
- The Chinese University of Hong Kong Chemistry HONG KONG
| | - Wanjie Wu
- Hong Kong University of Science and Technology School of Engineering Engineering HONG KONG
| | - Yanping He
- The Chinese University of Hong Kong Department of Biomedical Engineering HONG KONG
| | - Jiayang Gao
- The Chinese University of Hong Kong School of Life Sciences HONG KONG
| | - Renjie Zhou
- The Chinese University of Hong Kong Department of Biomedical Engineering HONG KONG
| | - Liwen Jiang
- The Chinese University of Hong Kong School of Life Sciences HONG KONG
| | - Jianan Qu
- Hong Kong University of Science and Technology School of Engineering Engineering HONG KONG
| | - Jiang Xia
- The Chinese University of Hong Kong Department of Chemistry SC G59, Department of ChemistryThe Chinese University of Hong Kong 00000 Shatin HONG KONG
| |
Collapse
|
39
|
Liao J, Wang Y, Xue S, Sun Y, Xu Y. Decoupling of cell refractive index and thickness under three-wavelengthphase imaging. INT J PATTERN RECOGN 2022. [DOI: 10.1142/s0218001422540155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
40
|
Pirone D, Sirico D, Miccio L, Bianco V, Mugnano M, Ferraro P, Memmolo P. Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning. LAB ON A CHIP 2022; 22:793-804. [PMID: 35076055 DOI: 10.1039/d1lc01087e] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Tomographic flow cytometry by digital holography is an emerging imaging modality capable of collecting multiple views of moving and rotating cells with the aim of recovering their refractive index distribution in 3D. Although this modality allows us to access high-resolution imaging with high-throughput, the huge amount of time-lapse holographic images to be processed (hundreds of digital holograms per cell) constitutes the actual bottleneck. This prevents the system from being suitable for lab-on-a-chip platforms in real-world applications, where fast analysis of measured data is mandatory. Here we demonstrate a significant speeding-up reconstruction of phase-contrast tomograms by introducing in the processing pipeline a multi-scale fully-convolutional context aggregation network. Although it was originally developed in the context of semantic image analysis, we demonstrate for the first time that it can be successfully adapted to a holographic lab-on-chip platform for achieving 3D tomograms through a faster computational process. We trained the network with input-output image pairs to reproduce the end-to-end holographic reconstruction process, i.e. recovering quantitative phase maps (QPMs) of single cells from their digital holograms. Then, the sequence of QPMs of the same rotating cell is used to perform the tomographic reconstruction. The proposed approach significantly reduces the computational time for retrieving tomograms, thus making them available in a few seconds instead of tens of minutes, while essentially preserving the high-content information of tomographic data. Moreover, we have accomplished a compact deep convolutional neural network parameterization that can fit into on-chip SRAM and a small memory footprint, thus demonstrating its possible exploitation to provide onboard computations for lab-on-chip devices with low processing hardware resources.
Collapse
Affiliation(s)
- Daniele Pirone
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
- DIETI, Department of Electrical Engineering and Information Technologies, University of Naples "Federico II", via Claudio 21, 80125 Napoli, Italy
| | - Daniele Sirico
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| | - Lisa Miccio
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| | - Vittorio Bianco
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| | - Martina Mugnano
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| | - Pietro Ferraro
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| | - Pasquale Memmolo
- CNR-ISASI, Institute of Applied Sciences and Intelligent Systems "E. Caianiello", Via Campi Flegrei 34, 80078 Pozzuoli, Napoli, Italy.
| |
Collapse
|
41
|
Bazow B, Phan T, Raub CB, Nehmetallah G. Computational multi-wavelength phase synthesis using convolutional neural networks [Invited]. APPLIED OPTICS 2022; 61:B132-B146. [PMID: 35201134 DOI: 10.1364/ao.439323] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 11/13/2021] [Indexed: 05/22/2023]
Abstract
Multi-wavelength digital holographic microscopy (MWDHM) provides indirect measurements of the refractive index for non-dispersive samples. Successive-shot MWDHM is not appropriate for dynamic samples and single-shot MWDHM significantly increases the complexity of the optical setup due to the need for multiple lasers or a wavelength tunable source. Here we consider deep learning convolutional neural networks for computational phase synthesis to obtain high-speed simultaneous phase estimates on different wavelengths and thus single-shot estimates of the integral refractive index without increased experimental complexity. This novel, to the best of our knowledge, computational concept is validated using cell phantoms consisting of internal refractive index variations representing cytoplasm and membrane-bound organelles, respectively, and a simulation of a realistic holographic recording process. Specifically, in this work we employed data-driven computational techniques to perform accurate dual-wavelength hologram synthesis (hologram-to-hologram prediction), dual-wavelength phase synthesis (unwrapped phase-to-phase prediction), direct phase-to-index prediction using a single wavelength, hologram-to-phase prediction, and 2D phase unwrapping with sharp discontinuities (wrapped-to-unwrapped phase prediction).
Collapse
|
42
|
Vaughan L, Zamyadi A, Ajjampur S, Almutaram H, Freguia S. A review of microscopic cell imaging and neural network recognition for synergistic cyanobacteria identification and enumeration. ANAL SCI 2022; 38:261-279. [PMID: 35286640 PMCID: PMC8938360 DOI: 10.1007/s44211-021-00013-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Accepted: 10/07/2021] [Indexed: 11/30/2022]
Abstract
Real-time cyanobacteria/algal monitoring is a valuable tool for early detection of harmful algal blooms, water treatment efficacy evaluation, and assists tailored water quality risk assessments by considering taxonomy and cell counts. This review evaluates and proposes a synergistic approach using neural network image recognition and microscopic imaging devices by first evaluating published literature for both imaging microscopes and image recognition. Quantitative phase imaging was considered the most promising of the investigated imaging techniques due to the provision of enhanced information relative to alternatives. This information provides significant value to image recognition neural networks, such as the convolutional neural networks discussed within this review. Considering published literature, a cyanobacteria monitoring system and corresponding image processing workflow using in situ sample collection buoys and on-shore sample processing was proposed. This system can be implemented using commercially available equipment to facilitate accurate, real-time water quality monitoring.
Collapse
Affiliation(s)
- Liam Vaughan
- Water Research Australia, Level 2, 250 Victoria Square, Adelaide, SA, 5000, Australia.
| | - Arash Zamyadi
- Water Research Australia, Level 2, 250 Victoria Square, Adelaide, SA, 5000, Australia
- Department of Chemical Engineering, The University of Melbourne, Parkville, VIC, 3010, Australia
| | - Suraj Ajjampur
- Water Research Australia, Level 2, 250 Victoria Square, Adelaide, SA, 5000, Australia
| | - Husein Almutaram
- Department of Civil and Mineral Engineering, University of Toronto, Toronto, ON, M5S 1A4, Canada
| | - Stefano Freguia
- Department of Chemical Engineering, The University of Melbourne, Parkville, VIC, 3010, Australia
| |
Collapse
|
43
|
Muhamad RK, Stępień P, Kujawińska M, Schelkens P. Off-axis image plane hologram compression in holographic tomography - metrological assessment. OPTICS EXPRESS 2022; 30:4261-4273. [PMID: 35209666 DOI: 10.1364/oe.449932] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 12/28/2021] [Indexed: 06/14/2023]
Abstract
In this paper, we present a novel study on the impact of lossy data compression on the metrological properties of holographic tomography reconstruction of the refractive index (RI). We use a spatial bandwidth-optimized compression procedure that leverages the properties of image plane off-axis holograms and standardized compression codecs, both widely applied in research and industry. The compression procedure is tested at multiple bitrates, for four different objects and against three reconstruction algorithms. The metrological evaluation is primarily done by comparison to the reconstruction from original data using the root-mean-squared error (RMSE). We show that due to differences between objects and different noise sensitivities of the reconstruction algorithms, the rate-distortion behaviour varies, but in most cases allows for the compression below 1 bit per pixel, while maintaining an RI RMSE less than 10-4.
Collapse
|
44
|
Roadmap on Digital Holography-Based Quantitative Phase Imaging. J Imaging 2021; 7:jimaging7120252. [PMID: 34940719 PMCID: PMC8703719 DOI: 10.3390/jimaging7120252] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 11/11/2021] [Accepted: 11/15/2021] [Indexed: 12/02/2022] Open
Abstract
Quantitative Phase Imaging (QPI) provides unique means for the imaging of biological or technical microstructures, merging beneficial features identified with microscopy, interferometry, holography, and numerical computations. This roadmap article reviews several digital holography-based QPI approaches developed by prominent research groups. It also briefly discusses the present and future perspectives of 2D and 3D QPI research based on digital holographic microscopy, holographic tomography, and their applications.
Collapse
|
45
|
Barré N, Shivaraman R, Ackermann L, Moser S, Schmidt M, Salter P, Booth M, Jesacher A. Tomographic refractive index profiling of direct laser written waveguides. OPTICS EXPRESS 2021; 29:35414-35425. [PMID: 34808976 DOI: 10.1364/oe.434846] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
The fabrication of complex integrated photonic devices via direct laser writing is a powerful and rapidly developing technology. However, the approach is still facing several challenges. One of them is the reliable quantitative characterization of refractive index (RI) changes induced upon laser exposure. To this end, we develop a tomographic reconstruction algorithm following a modern optimization approach, relying on accelerated proximal gradient descent, based on intensity images only. Very recently, such algorithms have become the state of the art in the community of bioimaging, but have never been applied to direct laser written structures such as waveguides. We adapt the algorithm to our concern of characterizing these translation-invariant structures and extend it in order to jointly estimate the aberrations introduced by the imaging system. We show that a correct estimation of these aberrations is necessary to make use of data recorded at larger angles and that it can increase the fidelity of the reconstructed RI profiles. Moreover, we present a method allowing to cross-validate the RI reconstructions by comparing en-face widefield images of thin waveguide sections with matching simulations based on the retrieved RI profile.
Collapse
|
46
|
Ayoub AB, Psaltis D. High speed, complex wavefront shaping using the digital micro-mirror device. Sci Rep 2021; 11:18837. [PMID: 34552161 PMCID: PMC8458445 DOI: 10.1038/s41598-021-98430-w] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Accepted: 09/07/2021] [Indexed: 11/26/2022] Open
Abstract
Digital micro-mirror devices (DMDs) have been deployed in many optical applications. As compared to spatial light modulators (SLMs), they are characterized by their much faster refresh rates (full-frame refresh rates up to 32 kHz for binary patterns) compared to 120 Hz for most liquid crystal SLMs. DMDs however can only display binary, unipolar patterns and utilize temporal modulation to represent with excellent accuracy multiple gray-levels in display applications. We used the built-in time domain dynamic range representation of the DMD to project 8-bit complex-fields. With this method, we demonstrated 8-bit complex field modulation with a frame time of 38.4 ms (around 0.15 s for the entire complex-field). We performed phase conjugation by compensating the distortions incurred due to propagation through free-space and a scattering medium. For faster modulation speed, an electro-optic modulator was used in synchronization with the DMD in an amplitude modulation mode to create grayscale patterns with frame rate ~ 833 Hz with display time of only 1.2 ms instead of 38.4 ms for time multiplexing gaining a speed up by a factor of 32.
Collapse
Affiliation(s)
- Ahmed B Ayoub
- Optics Laboratory, Ecole Polytechnique Federale de Lausanne (EPFL), 1015, Lausanne, Vaud, Switzerland.
| | - Demetri Psaltis
- Optics Laboratory, Ecole Polytechnique Federale de Lausanne (EPFL), 1015, Lausanne, Vaud, Switzerland
| |
Collapse
|
47
|
Taddese AM, Verrier N, Debailleul M, Courbot JB, Haeberlé O. Optimizing sample illumination scanning for reflection and 4Pi tomographic diffractive microscopy. APPLIED OPTICS 2021; 60:7745-7753. [PMID: 34613246 DOI: 10.1364/ao.435721] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2021] [Accepted: 08/02/2021] [Indexed: 06/13/2023]
Abstract
Tomographic diffractive microscopy (TDM) is increasingly gaining attention, owing to its high-resolution, label-free imaging capability. Fast acquisitions necessitate limiting the number of holograms to be recorded. Reconstructions then rely on optimal Fourier space filling to retain image quality and resolution, that is, they rely on optimal scanning of the tomographic illuminations. In this work, we theoretically study reflection TDM, and then the 4Pi TDM, a combination of transmission and reflection systems. Image simulations are conducted to determine optimal angular sweeping. We found that three-dimensional uniform scanning fills Fourier space the best for both reflection and 4Pi configurations, providing a better refractive index estimation for the observed sample.
Collapse
|
48
|
Yang F, Pham TA, Brandenberg N, Lutolf MP, Ma J, Unser M. Robust Phase Unwrapping via Deep Image Prior for Quantitative Phase Imaging. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:7025-7037. [PMID: 34329165 DOI: 10.1109/tip.2021.3099956] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Quantitative phase imaging (QPI) is an emerging label-free technique that produces images containing morphological and dynamical information without contrast agents. Unfortunately, the phase is wrapped in most imaging system. Phase unwrapping is the computational process that recovers a more informative image. It is particularly challenging with thick and complex samples such as organoids. Recent works that rely on supervised training show that deep learning is a powerful method to unwrap the phase; however, supervised approaches require large and representative datasets which are difficult to obtain for complex biological samples. Inspired by the concept of deep image priors, we propose a deep-learning-based method that does not need any training set. Our framework relies on an untrained convolutional neural network to accurately unwrap the phase while ensuring the consistency of the measurements. We experimentally demonstrate that the proposed method faithfully recovers the phase of complex samples on both real and simulated data. Our work paves the way to reliable phase imaging of thick and complex samples with QPI.
Collapse
|
49
|
Géloën A, Isaieva K, Isaiev M, Levinson O, Berger E, Lysenko V. Intracellular Detection and Localization of Nanoparticles by Refractive Index Measurement. SENSORS 2021; 21:s21155001. [PMID: 34372238 PMCID: PMC8347443 DOI: 10.3390/s21155001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 07/19/2021] [Accepted: 07/20/2021] [Indexed: 01/24/2023]
Abstract
The measuring of nanoparticle toxicity faces an important limitation since it is based on metrics exposure, the concentration at which cells are exposed instead the true concentration inside the cells. In vitro studies of nanomaterials would benefit from the direct measuring of the true intracellular dose of nanoparticles. The objective of the present study was to state whether the intracellular detection of nanodiamonds is possible by measuring the refractive index. Based on optical diffraction tomography of treated live cells, the results show that unlabeled nanoparticles can be detected and localized inside cells. The results were confirmed by fluorescence measurements. Optical diffraction tomography paves the way to measuring the true intracellular concentrations and the localization of nanoparticles which will improve the dose-response paradigm of pharmacology and toxicology in the field of nanomaterials.
Collapse
Affiliation(s)
- Alain Géloën
- UMR Ecologie Microbienne Lyon (LEM), CNRS 5557, INRAE 1418, Claude Bernard University of Lyon, VetAgro Sup, Research Team “Bacterial Opportunistic Pathogens and Environment” (BPOE), University of Lyon, F-69622 Villeurbanne, France;
- Correspondence:
| | - Karyna Isaieva
- IADI, INSERM, Université de Lorraine, F-54000 Nancy, France;
| | - Mykola Isaiev
- LEMTA, CNRS, Université de Lorraine, F-54000 Nancy, France;
| | - Olga Levinson
- Ray Techniques LTD, P.O. Box 39162, Jerusalem 9139101, Israel;
| | - Emmanuelle Berger
- UMR Ecologie Microbienne Lyon (LEM), CNRS 5557, INRAE 1418, Claude Bernard University of Lyon, VetAgro Sup, Research Team “Bacterial Opportunistic Pathogens and Environment” (BPOE), University of Lyon, F-69622 Villeurbanne, France;
| | - Vladimir Lysenko
- Light Matter Institute, UMR-5306, Claude Bernard University of Lyon, 2 rue Victor Grignard, F-69622 Villeurbanne, France;
| |
Collapse
|
50
|
Fanous M, Shi C, Caputo MP, Rund LA, Johnson RW, Das T, Kuchan MJ, Sobh N, Popescu G. Label-free screening of brain tissue myelin content using phase imaging with computational specificity (PICS). APL PHOTONICS 2021; 6:076103. [PMID: 34291159 PMCID: PMC8278825 DOI: 10.1063/5.0050889] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Accepted: 06/24/2021] [Indexed: 05/03/2023]
Abstract
Inadequate myelination in the central nervous system is associated with neurodevelopmental complications. Thus, quantitative, high spatial resolution measurements of myelin levels are highly desirable. We used spatial light interference microcopy (SLIM), a highly sensitive quantitative phase imaging (QPI) technique, to correlate the dry mass content of myelin in piglet brain tissue with dietary changes and gestational size. We combined SLIM micrographs with an artificial intelligence (AI) classifying model that allows us to discern subtle disparities in myelin distributions with high accuracy. This concept of combining QPI label-free data with AI for the purpose of extracting molecular specificity has recently been introduced by our laboratory as phase imaging with computational specificity. Training on 8000 SLIM images of piglet brain tissue with the 71-layer transfer learning model Xception, we created a two-parameter classification to differentiate gestational size and diet type with an accuracy of 82% and 80%, respectively. To our knowledge, this type of evaluation is impossible to perform by an expert pathologist or other techniques.
Collapse
Affiliation(s)
| | - Chuqiao Shi
- Quantitative Light Imaging Laboratory, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | - Megan P. Caputo
- Division of Nutritional Sciences, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | - Laurie A. Rund
- Laboratory of Integrative Immunology & Behavior, Department of Animal Sciences, University of Illinois at Urbana-Champaign, Urbana, Illinois 61801, USA
| | | | - Tapas Das
- Abbott Nutrition, Discovery Research, Columbus, Ohio 43219, USA
| | - Matthew J. Kuchan
- Abbott Nutrition, Strategic Research, 3300 Stelzer Road, Columbus, Ohio 43219, USA
| | | | | |
Collapse
|