1
|
Nakatani Y, Gaumer S, Shechtman Y, Gustavsson AK. Long axial-range double-helix point spread functions for 3D volumetric super-resolution imaging. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.31.605907. [PMID: 39131321 PMCID: PMC11312577 DOI: 10.1101/2024.07.31.605907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/13/2024]
Abstract
Single-molecule localization microscopy (SMLM) is a powerful tool for observing structures beyond the diffraction limit of light. Combining SMLM with engineered point spread functions (PSFs) enables 3D imaging over an extended axial range, as has been demonstrated for super-resolution imaging of various cellular structures. However, super-resolving structures in 3D in thick samples, such as whole mammalian cells, remains challenging as it typically requires acquisition and post-processing stitching of multiple slices to cover the entire sample volume or more complex analysis of the data. Here, we demonstrate how the imaging and analysis workflows can be simplified by 3D single-molecule super-resolution imaging with long axial-range double-helix (DH)-PSFs. First, we experimentally benchmark the localization precisions of short- and long axial-range DH-PSFs at different signal-to-background ratios by imaging of fluorescent beads. The performance of the DH-PSFs in terms of achievable resolution and imaging speed was then quantified for 3D single-molecule super-resolution imaging of mammalian cells by DNA-PAINT imaging of the nuclear lamina protein lamin B1 in U-2 OS cells. Furthermore, we demonstrate how the use of a deep learning-based algorithm allows the localization of dense emitters, drastically improving the achievable imaging speed and resolution. Our data demonstrate that using long axial-range DH-PSFs offers stitching-free, 3D super-resolution imaging of whole mammalian cells, simplifying the experimental and analysis procedures for obtaining volumetric nanoscale structural information.
Collapse
Affiliation(s)
- Yuya Nakatani
- Department of Chemistry, Rice University, 6100 Main St, Houston, TX 77005, USA
| | - Scott Gaumer
- Double Helix Optics Inc, 3415 Colorado Ave, Boulder, CO 80303, USA
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion, 32000 Haifa, Israel
- Department of Mechanical Engineering, University of Texas at Austin, Dean Keeton St, TX 78705, USA
| | - Anna-Karin Gustavsson
- Department of Chemistry, Rice University, 6100 Main St, Houston, TX 77005, USA
- Department of Biosciences, Rice University, 6100 Main St, Houston, TX, USA
- Department of Electrical and Computer Engineering, Rice University, 6100 Main St, Houston, TX 77005, USA
- Smalley-Curl Institute, Rice University, 6100 Main St, Houston, TX 77005, USA
- Center for Nanoscale Imaging Sciences, Rice University, 6100 Main St, Houston, TX 77005, USA
- Department of Cancer Biology, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX 77030, USA
| |
Collapse
|
2
|
Liu J, Li Y, Chen T, Zhang F, Xu F. Machine Learning for Single-Molecule Localization Microscopy: From Data Analysis to Quantification. Anal Chem 2024; 96:11103-11114. [PMID: 38946062 DOI: 10.1021/acs.analchem.3c05857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/02/2024]
Abstract
Single-molecule localization microscopy (SMLM) is a versatile tool for realizing nanoscale imaging with visible light and providing unprecedented opportunities to observe bioprocesses. The integration of machine learning with SMLM enhances data analysis by improving efficiency and accuracy. This tutorial aims to provide a comprehensive overview of the data analysis process and theoretical aspects of SMLM, while also highlighting the typical applications of machine learning in this field. By leveraging advanced analytical techniques, SMLM is becoming a powerful quantitative analysis tool for biological research.
Collapse
Affiliation(s)
- Jianli Liu
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China
| | - Yumian Li
- Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing 100081, China
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Tailong Chen
- Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing 100081, China
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Fa Zhang
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China
| | - Fan Xu
- Advanced Research Institute of Multidisciplinary Science, Beijing Institute of Technology, Beijing 100081, China
- School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| |
Collapse
|
3
|
Li C, Rai MR, Cai Y, Ghashghaei HT, Greenbaum A. Intelligent Beam Optimization for Light-Sheet Fluorescence Microscopy through Deep Learning. INTELLIGENT COMPUTING (WASHINGTON, D.C.) 2024; 3:0095. [PMID: 39099879 PMCID: PMC11298055 DOI: 10.34133/icomputing.0095] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 04/22/2024] [Indexed: 08/06/2024]
Abstract
Light-sheet fluorescence microscopy (LSFM) provides the benefit of optical sectioning coupled with rapid acquisition times, enabling high-resolution 3-dimensional imaging of large tissue-cleared samples. Inherent to LSFM, the quality of the imaging heavily relies on the characteristics of the illumination beam, which only illuminates a thin section of the sample. Therefore, substantial efforts are dedicated to identifying slender, nondiffracting beam profiles that yield uniform and high-contrast images. An ongoing debate concerns the identification of optimal illumination beams for different samples: Gaussian, Bessel, Airy patterns, and/or others. However, comparisons among different beam profiles are challenging as their optimization objectives are often different. Given that our large imaging datasets (approximately 0.5 TB of images per sample) are already analyzed using deep learning models, we envisioned a different approach to the problem by designing an illumination beam tailored to boost the performance of the deep learning model. We hypothesized that integrating the physical LSFM illumination model (after passing it through a variable phase mask) into the training of a cell detection network would achieve this goal. Here, we report that joint optimization continuously updates the phase mask and results in improved image quality for better cell detection. The efficacy of our method is demonstrated through both simulations and experiments that reveal substantial enhancements in imaging quality compared to the traditional Gaussian light sheet. We discuss how designing microscopy systems through a computational approach provides novel insights for advancing optical design that relies on deep learning models for the analysis of imaging datasets.
Collapse
Affiliation(s)
- Chen Li
- Joint Department of Biomedical Engineering, North Carolina State University and University of North Carolina at Chapel Hill, Raleigh, NC 27695, USA
- Comparative Medicine Institute, North Carolina State University, Raleigh, NC 27695, USA
| | - Mani Ratnam Rai
- Joint Department of Biomedical Engineering, North Carolina State University and University of North Carolina at Chapel Hill, Raleigh, NC 27695, USA
- Comparative Medicine Institute, North Carolina State University, Raleigh, NC 27695, USA
| | - Yuheng Cai
- Joint Department of Biomedical Engineering, North Carolina State University and University of North Carolina at Chapel Hill, Raleigh, NC 27695, USA
- Comparative Medicine Institute, North Carolina State University, Raleigh, NC 27695, USA
| | - H. Troy Ghashghaei
- Comparative Medicine Institute, North Carolina State University, Raleigh, NC 27695, USA
- Department of Molecular Biomedical Sciences, North Carolina State University, Raleigh, NC 27695, USA
| | - Alon Greenbaum
- Joint Department of Biomedical Engineering, North Carolina State University and University of North Carolina at Chapel Hill, Raleigh, NC 27695, USA
- Comparative Medicine Institute, North Carolina State University, Raleigh, NC 27695, USA
- Bioinformatics Research Center, North Carolina State University, Raleigh, NC 27695, USA
| |
Collapse
|
4
|
Opatovski N, Nehme E, Zoref N, Barzilai I, Orange Kedem R, Ferdman B, Keselman P, Alalouf O, Shechtman Y. Depth-enhanced high-throughput microscopy by compact PSF engineering. Nat Commun 2024; 15:4861. [PMID: 38849376 PMCID: PMC11161645 DOI: 10.1038/s41467-024-48502-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 05/03/2024] [Indexed: 06/09/2024] Open
Abstract
High-throughput microscopy is vital for screening applications, where three-dimensional (3D) cellular models play a key role. However, due to defocus susceptibility, current 3D high-throughput microscopes require axial scanning, which lowers throughput and increases photobleaching and photodamage. Point spread function (PSF) engineering is an optical method that enables various 3D imaging capabilities, yet it has not been implemented in high-throughput microscopy due to the cumbersome optical extension it typically requires. Here we demonstrate compact PSF engineering in the objective lens, which allows us to enhance the imaging depth of field and, combined with deep learning, recover 3D information using single snapshots. Beyond the applications shown here, this work showcases the usefulness of high-throughput microscopy in obtaining training data for deep learning-based algorithms, applicable to a variety of microscopy modalities.
Collapse
Affiliation(s)
- Nadav Opatovski
- Russell Berrie Nanotechnology Institute, Technion - Israel Institute of Technology, Haifa, Israel
| | - Elias Nehme
- Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
- Department of Electrical and Computer Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Noam Zoref
- Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Ilana Barzilai
- Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Reut Orange Kedem
- Russell Berrie Nanotechnology Institute, Technion - Israel Institute of Technology, Haifa, Israel
| | - Boris Ferdman
- Russell Berrie Nanotechnology Institute, Technion - Israel Institute of Technology, Haifa, Israel
| | - Paul Keselman
- Sartorius Stedim North America Inc., Bohemia, NY, USA
| | - Onit Alalouf
- Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel
| | - Yoav Shechtman
- Russell Berrie Nanotechnology Institute, Technion - Israel Institute of Technology, Haifa, Israel.
- Department of Biomedical Engineering, Technion - Israel Institute of Technology, Haifa, Israel.
- Department of Mechanical Engineering, University of Texas at Austin, Austin, TX, USA.
| |
Collapse
|
5
|
Liu S, Chen J, Hellgoth J, Müller LR, Ferdman B, Karras C, Xiao D, Lidke KA, Heintzmann R, Shechtman Y, Li Y, Ries J. Universal inverse modeling of point spread functions for SMLM localization and microscope characterization. Nat Methods 2024; 21:1082-1093. [PMID: 38831208 DOI: 10.1038/s41592-024-02282-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 04/16/2024] [Indexed: 06/05/2024]
Abstract
The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single-molecule localization, aberration correction and deconvolution. Here we present universal inverse modeling of point spread functions (uiPSF), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single-molecule localization microscopy (SMLM). Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system- or sample-specific characteristics, for example, the bead size, field- and depth- dependent aberrations, and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single-molecule blinking data.
Collapse
Affiliation(s)
- Sheng Liu
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Jianwei Chen
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
- Collaboration for joint PhD degree between Southern University of Science and Technology and Harbin Institute of Technology, Harbin, China
| | - Jonas Hellgoth
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Faculty of Biosciences, Collaboration for joint PhD degree from EMBL and Heidelberg University, Heidelberg, Germany
| | - Lucas-Raphael Müller
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Tübingen, Germany
| | - Boris Ferdman
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, Israel
| | - Christian Karras
- Leibniz Institute of Photonic Technology, Jena, Germany
- JENOPTIK Optical Systems, Jena, Germany
| | - Dafei Xiao
- Russell Berrie Nanotechnology Institute, Technion-Israel Institute of Technology, Haifa, Israel
| | - Keith A Lidke
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Rainer Heintzmann
- Leibniz Institute of Photonic Technology, Jena, Germany
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University Jena, Jena, Germany
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion-Israel Institute of Technology, Haifa, Israel
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Yiming Li
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China.
| | - Jonas Ries
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany.
- Max Perutz Labs, Vienna Biocenter Campus, Vienna, Austria.
- Department of Structural and Computational Biology, Center for Molecular Biology, University of Vienna, Vienna, Austria.
- Faculty of Physics, University of Vienna, Vienna, Austria.
| |
Collapse
|
6
|
Hayward-Lara G, Fischer MD, Mir M. Dynamic microenvironments shape nuclear organization and gene expression. Curr Opin Genet Dev 2024; 86:102177. [PMID: 38461773 PMCID: PMC11162947 DOI: 10.1016/j.gde.2024.102177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 02/11/2024] [Accepted: 02/14/2024] [Indexed: 03/12/2024]
Abstract
Live imaging has revealed that the regulation of gene expression is largely driven by transient interactions. For example, many regulatory proteins bind chromatin for just seconds, and loop-like genomic contacts are rare and last only minutes. These discoveries have been difficult to reconcile with our canonical models that are predicated on stable and hierarchical interactions. Proteomic microenvironments that concentrate nuclear factors may explain how brief interactions can still mediate gene regulation by creating conditions where reactions occur more frequently. Here, we summarize new imaging technologies and recent discoveries implicating microenvironments as a potential driver of nuclear function. Finally, we propose that key properties of proteomic microenvironments, such as their size, enrichment, and lifetimes, are directly linked to regulatory function.
Collapse
Affiliation(s)
- Gabriela Hayward-Lara
- Department of Cell and Developmental Biology, Perelman School of Medicine, University of Pennsylvania. Philadelphia, PA 19104
- Center for Computational and Genomic Medicine, Children’s Hospital of Philadelphia. Philadelphia, PA 19104
- Developmental, Stem Cell, and Regenerative Biology Graduate Group, Perelman School of Medicine, University of Pennsylvania. Philadelphia, PA 19104
| | - Matthew D. Fischer
- Department of Cell and Developmental Biology, Perelman School of Medicine, University of Pennsylvania. Philadelphia, PA 19104
- Center for Computational and Genomic Medicine, Children’s Hospital of Philadelphia. Philadelphia, PA 19104
| | - Mustafa Mir
- Department of Cell and Developmental Biology, Perelman School of Medicine, University of Pennsylvania. Philadelphia, PA 19104
- Center for Computational and Genomic Medicine, Children’s Hospital of Philadelphia. Philadelphia, PA 19104
- Epigenetics Institute, Perelman School of Medicine, University of Pennsylvania. Philadelphia, PA 19104
- Howard Hughes Medical Institute, Children’s Hospital of Philadelphia. Philadelphia, PA 19104
| |
Collapse
|
7
|
Seong B, Kim W, Kim Y, Hyun KA, Jung HI, Lee JS, Yoo J, Joo C. E2E-BPF microscope: extended depth-of-field microscopy using learning-based implementation of binary phase filter and image deconvolution. LIGHT, SCIENCE & APPLICATIONS 2023; 12:269. [PMID: 37953314 PMCID: PMC10641084 DOI: 10.1038/s41377-023-01300-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 09/09/2023] [Accepted: 10/07/2023] [Indexed: 11/14/2023]
Abstract
Several image-based biomedical diagnoses require high-resolution imaging capabilities at large spatial scales. However, conventional microscopes exhibit an inherent trade-off between depth-of-field (DoF) and spatial resolution, and thus require objects to be refocused at each lateral location, which is time consuming. Here, we present a computational imaging platform, termed E2E-BPF microscope, which enables large-area, high-resolution imaging of large-scale objects without serial refocusing. This method involves a physics-incorporated, deep-learned design of binary phase filter (BPF) and jointly optimized deconvolution neural network, which altogether produces high-resolution, high-contrast images over extended depth ranges. We demonstrate the method through numerical simulations and experiments with fluorescently labeled beads, cells and tissue section, and present high-resolution imaging capability over a 15.5-fold larger DoF than the conventional microscope. Our method provides highly effective and scalable strategy for DoF-extended optical imaging system, and is expected to find numerous applications in rapid image-based diagnosis, optical vision, and metrology.
Collapse
Affiliation(s)
- Baekcheon Seong
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
| | - Woovin Kim
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
| | - Younghun Kim
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
| | - Kyung-A Hyun
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
| | - Hyo-Il Jung
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
- The DABOM Inc, Seoul, 03722, Republic of Korea
| | - Jong-Seok Lee
- School of Integrated Technology, Yonsei University, Incheon, 21983, Republic of Korea
| | - Jeonghoon Yoo
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea
| | - Chulmin Joo
- Department of Mechanical Engineering, Yonsei University, Seoul, 03722, Republic of Korea.
| |
Collapse
|
8
|
Liu S, Chen J, Hellgoth J, Müller LR, Ferdman B, Karras C, Xiao D, Lidke KA, Heintzmann R, Shechtman Y, Li Y, Ries J. Universal inverse modelling of point spread functions for SMLM localization and microscope characterization. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.26.564064. [PMID: 37961269 PMCID: PMC10634843 DOI: 10.1101/2023.10.26.564064] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
The point spread function (PSF) of a microscope describes the image of a point emitter. Knowing the accurate PSF model is essential for various imaging tasks, including single molecule localization, aberration correction and deconvolution. Here we present uiPSF (universal inverse modelling of Point Spread Functions), a toolbox to infer accurate PSF models from microscopy data, using either image stacks of fluorescent beads or directly images of blinking fluorophores, the raw data in single molecule localization microscopy (SMLM). The resulting PSF model enables accurate 3D super-resolution imaging using SMLM. Additionally, uiPSF can be used to characterize and optimize a microscope system by quantifying the aberrations, including field-dependent aberrations, and resolutions. Our modular framework is applicable to a variety of microscope modalities and the PSF model incorporates system or sample specific characteristics, e.g., the bead size, depth dependent aberrations and transformations among channels. We demonstrate its application in single or multiple channels or large field-of-view SMLM systems, 4Pi-SMLM, and lattice light-sheet microscopes using either bead data or single molecule blinking data.
Collapse
Affiliation(s)
- Sheng Liu
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Jianwei Chen
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
- Collaboration for joint PhD degree between Southern University of Science and Technology and Harbin Institute of Technology, Harbin, 150001, China
| | - Jonas Hellgoth
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Lucas-Raphael Müller
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
| | - Boris Ferdman
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
| | - Christian Karras
- Leibniz Institute of Photonic Technology, Albert-Einstein-Straße 9, 07745, Jena, Germany
- Currently at JENOPTIK Optical Systems GmbH, Jena, Germany
| | - Dafei Xiao
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
| | - Keith A. Lidke
- Department of Physics and Astronomy, University of New Mexico, Albuquerque, NM, USA
| | - Rainer Heintzmann
- Institute of Physical Chemistry and Abbe Center of Photonics, Friedrich-Schiller-University Jena, Jena, Germany
- Leibniz Institute of Photonic Technology, Albert-Einstein-Straße 9, 07745, Jena, Germany
| | - Yoav Shechtman
- Department of Biomedical Engineering, Technion–Israel Institute of Technology, Haifa, Israel
- Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Yiming Li
- Department of Biomedical Engineering, Guangdong Provincial Key Laboratory of Advanced Biomaterials, Southern University of Science and Technology, Shenzhen, China
| | - Jonas Ries
- European Molecular Biology Laboratory, Cell Biology and Biophysics, Heidelberg, Germany
- Max Perutz Labs, Vienna Biocenter Campus (VBC), Dr.-Bohr-Gasse 9, 1030, Vienna, Austria
- University of Vienna, Center for Molecular Biology, Department of Structural and Computational Biology, Dr.-Bohr-Gasse 9, 1030, Vienna, Austria
- University of Vienna, Faculty of Physics, Boltzmanngasse 5, 1090 Vienna, Austria
| |
Collapse
|
9
|
Burgers TCQ, Vlijm R. Fluorescence-based super-resolution-microscopy strategies for chromatin studies. Chromosoma 2023:10.1007/s00412-023-00792-9. [PMID: 37000292 PMCID: PMC10356683 DOI: 10.1007/s00412-023-00792-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Revised: 02/28/2023] [Accepted: 03/16/2023] [Indexed: 04/01/2023]
Abstract
Super-resolution microscopy (SRM) is a prime tool to study chromatin organisation at near biomolecular resolution in the native cellular environment. With fluorescent labels DNA, chromatin-associated proteins and specific epigenetic states can be identified with high molecular specificity. The aim of this review is to introduce the field of diffraction-unlimited SRM to enable an informed selection of the most suitable SRM method for a specific chromatin-related research question. We will explain both diffraction-unlimited approaches (coordinate-targeted and stochastic-localisation-based) and list their characteristic spatio-temporal resolutions, live-cell compatibility, image-processing, and ability for multi-colour imaging. As the increase in resolution, compared to, e.g. confocal microscopy, leads to a central role of the sample quality, important considerations for sample preparation and concrete examples of labelling strategies applicable to chromatin research are discussed. To illustrate how SRM-based methods can significantly improve our understanding of chromatin functioning, and to serve as an inspiring starting point for future work, we conclude with examples of recent applications of SRM in chromatin research.
Collapse
Affiliation(s)
- Thomas C Q Burgers
- Molecular Biophysics, Zernike Institute for Advanced Materials, Rijksuniversiteit Groningen, Groningen, the Netherlands
| | - Rifka Vlijm
- Molecular Biophysics, Zernike Institute for Advanced Materials, Rijksuniversiteit Groningen, Groningen, the Netherlands.
| |
Collapse
|
10
|
Wang F, Lai J, Liu H, Zhao M, Zhang Y, Xu J, Yu Y, Wang C. Double helix point spread function with variable spacing for precise 3D particle localization. OPTICS EXPRESS 2023; 31:11680-11694. [PMID: 37155797 DOI: 10.1364/oe.482390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
To extend the axial depth of nanoscale 3D-localization microscopy, we propose here a splicing-type vortex singularities (SVS) phase mask, which has been meticulously optimized with a Fresnel approximation imaging inverse operation. The optimized SVS DH-PSF has proven to have high transfer function efficiency with adjustable performance in its axial range. The axial position of the particle was computed by using both the main lobes' spacing and the rotation angle, an improvement of the localization precision of the particle. Concretely, the proposed optimized SVS DH-PSF, with a smaller spatial extent, can effectively reduce the overlap of nanoparticle images and realize the 3D localization of multiple nanoparticles with small spacing, with respect to PSFs for large axial 3D localization. Finally, we successfully conducted extensive experiments on 3D localization for tracking dense nanoparticles at 8µm depth with a numerical aperture of 1.4, demonstrating its great potential.
Collapse
|
11
|
Field-dependent deep learning enables high-throughput whole-cell 3D super-resolution imaging. Nat Methods 2023; 20:459-468. [PMID: 36823335 DOI: 10.1038/s41592-023-01775-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 01/09/2023] [Indexed: 02/25/2023]
Abstract
Single-molecule localization microscopy in a typical wide-field setup has been widely used for investigating subcellular structures with super resolution; however, field-dependent aberrations restrict the field of view (FOV) to only tens of micrometers. Here, we present a deep-learning method for precise localization of spatially variant point emitters (FD-DeepLoc) over a large FOV covering the full chip of a modern sCMOS camera. Using a graphic processing unit-based vectorial point spread function (PSF) fitter, we can fast and accurately model the spatially variant PSF of a high numerical aperture objective in the entire FOV. Combined with deformable mirror-based optimal PSF engineering, we demonstrate high-accuracy three-dimensional single-molecule localization microscopy over a volume of ~180 × 180 × 5 μm3, allowing us to image mitochondria and nuclear pore complexes in entire cells in a single imaging cycle without hardware scanning; a 100-fold increase in throughput compared to the state of the art.
Collapse
|
12
|
Villegas Burgos CM, Xiong P, Qiu L, Zhu Y, Vamivakas AN. Co-designed metaoptoelectronic deep learning. OPTICS EXPRESS 2023; 31:6453-6463. [PMID: 36823900 DOI: 10.1364/oe.479038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 12/13/2022] [Indexed: 06/18/2023]
Abstract
A metaoptical system is co-designed with electronic hardware to implement deep learning image recognition. The optical convolution block includes a reflective metasurface to perform one layer of a deep neural network. The optical and digital components are jointly optimized to perform an image classification task attaining 65% accuracy, which is close to the 66% accuracy of a fully-digital network where the optical block is replaced by a digital convolution layer.
Collapse
|
13
|
Zhang O, Guo Z, He Y, Wu T, Vahey MD, Lew MD. Six-Dimensional Single-Molecule Imaging with Isotropic Resolution using a Multi-View Reflector Microscope. NATURE PHOTONICS 2023; 17:179-186. [PMID: 36968242 PMCID: PMC10035538 DOI: 10.1038/s41566-022-01116-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Accepted: 10/20/2022] [Indexed: 05/31/2023]
Abstract
Imaging both the positions and orientations of single fluorophores, termed single-molecule orientation-localisation microscopy, is a powerful tool to study biochemical processes. However, the limited photon budget associated with single-molecule fluorescence makes high-dimensional imaging with isotropic, nanoscale spatial resolution a formidable challenge. Here, we realise a radially and azimuthally polarized multi-view reflector (raMVR) microscope for the imaging of the 3D positions and 3D orientations of single molecules, with precision of 10.9 nm and 2.0° over a 1.5 μm depth range. The raMVR microscope achieves 6D super-resolution imaging of Nile red (NR) molecules transiently bound to lipid-coated spheres, accurately resolving their spherical morphology despite refractive-index mismatch. By observing the rotational dynamics of NR, raMVR images also resolve the infiltration of lipid membranes by amyloid-beta oligomers without covalent labelling. Finally, we demonstrate 6D imaging of cell membranes, where the orientations of specific fluorophores reveal heterogeneity in membrane fluidity. With its nearly isotropic 3D spatial resolution and orientation measurement precision, we expect the raMVR microscope to enable 6D imaging of molecular dynamics within biological and chemical systems with exceptional detail.
Collapse
Affiliation(s)
- Oumeng Zhang
- Department of Electrical and Systems Engineering
| | | | | | - Tingting Wu
- Department of Electrical and Systems Engineering
| | - Michael D. Vahey
- Department of Biomedical Engineering
- Center for Biomolecular Condensates
| | - Matthew D. Lew
- Department of Electrical and Systems Engineering
- Center for Biomolecular Condensates
- Institute of Materials Science and Engineering, Washington University in St. Louis, Missouri 63130, USA
| |
Collapse
|
14
|
Opatovski N, Xiao D, Harari G, Shechtman Y. Monocular kilometer-scale passive ranging by point-spread function engineering. OPTICS EXPRESS 2022; 30:37925-37937. [PMID: 36258371 DOI: 10.1364/oe.472150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 09/05/2022] [Indexed: 06/16/2023]
Abstract
Standard imaging systems are designed for 2D representation of objects, while information about the third dimension remains implicit, as imaging-based distance estimation is a difficult challenge. Existing long-range distance estimation technologies mostly rely on active emission of signal, which as a subsystem, constitutes a significant portion of the complexity, size and cost of the active-ranging apparatus. Despite the appeal of alleviating the requirement for signal-emission, passive distance estimation methods are essentially nonexistent for ranges greater than a few hundreds of meters. Here, we present monocular long-range, telescope-based passive ranging, realized by integration of point-spread-function engineering into a telescope, extending the scale of point-spread-function engineering-based ranging to distances where it has never been tested before. We provide experimental demonstrations of the optical system in a variety of challenging imaging scenarios, including adversarial weather conditions, dynamic targets and scenes of diversified textures, at distances extending beyond 1.7 km. We conclude with brief quantification of the effect of atmospheric turbulence on estimation precision, which becomes a significant error source in long-range optical imaging.
Collapse
|
15
|
Jusuf JM, Lew MD. Towards optimal point spread function design for resolving closely spaced emitters in three dimensions. OPTICS EXPRESS 2022; 30:37154-37174. [PMID: 36258632 DOI: 10.1364/oe.472067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 09/12/2022] [Indexed: 06/16/2023]
Abstract
The past decade has brought many innovations in optical design for 3D super-resolution imaging of point-like emitters, but these methods often focus on single-emitter localization precision as a performance metric. Here, we propose a simple heuristic for designing a point spread function (PSF) that allows for precise measurement of the distance between two emitters. We discover that there are two types of PSFs that achieve high performance for resolving emitters in 3D, as quantified by the Cramér-Rao bounds for estimating the separation between two closely spaced emitters. One PSF is very similar to the existing Tetrapod PSFs; the other is a rotating single-spot PSF, which we call the crescent PSF. The latter exhibits excellent performance for localizing single emitters throughout a 1-µm focal volume (localization precisions of 7.3 nm in x, 7.7 nm in y, and 18.3 nm in z using 1000 detected photons), and it distinguishes between one and two closely spaced emitters with superior accuracy (25-53% lower error rates than the best-performing Tetrapod PSF, averaged throughout a 1-µm focal volume). Our study provides additional insights into optimal strategies for encoding 3D spatial information into optical PSFs.
Collapse
|
16
|
Ferdman B, Saguy A, Xiao D, Shechtman Y. Diffractive optical system design by cascaded propagation. OPTICS EXPRESS 2022; 30:27509-27530. [PMID: 36236921 DOI: 10.1364/oe.465230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Accepted: 06/30/2022] [Indexed: 06/16/2023]
Abstract
Modern design of complex optical systems relies heavily on computational tools. These frequently use geometrical optics as well as Fourier optics. Fourier optics is typically used for designing thin diffractive elements, placed in the system's aperture, generating a shift-invariant Point Spread Function (PSF). A major bottleneck in applying Fourier Optics in many cases of interest, e.g. when dealing with multiple, or out-of-aperture elements, comes from numerical complexity. In this work, we propose and implement an efficient and differentiable propagation model based on the Collins integral, which enables the optimization of diffractive optical systems with unprecedented design freedom using backpropagation. We demonstrate the applicability of our method, numerically and experimentally, by engineering shift-variant PSFs via thin plate elements placed in arbitrary planes inside complex imaging systems, performing cascaded optimization of multiple planes, and designing optimal machine-vision systems by deep learning.
Collapse
|
17
|
Li Y, Shi W, Liu S, Cavka I, Wu YL, Matti U, Wu D, Koehler S, Ries J. Global fitting for high-accuracy multi-channel single-molecule localization. Nat Commun 2022; 13:3133. [PMID: 35668089 PMCID: PMC9170706 DOI: 10.1038/s41467-022-30719-4] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 05/16/2022] [Indexed: 11/09/2022] Open
Abstract
Multi-channel detection in single-molecule localization microscopy greatly increases information content for various biological applications. Here, we present globLoc, a graphics processing unit based global fitting algorithm with flexible PSF modeling and parameter sharing, to extract maximum information from multi-channel single molecule data. As signals in multi-channel data are highly correlated, globLoc links parameters such as 3D coordinates or photon counts across channels, improving localization precision and robustness. We show, both in simulations and experiments, that global fitting can substantially improve the 3D localization precision for biplane and 4Pi single-molecule localization microscopy and color assignment for ratiometric multicolor imaging.
Collapse
Affiliation(s)
- Yiming Li
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, 518055, China.
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany.
| | - Wei Shi
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Sheng Liu
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany
| | - Ivana Cavka
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany
- Collaboration for joint PhD degree between EMBL and Heidelberg University, Faculty of Biosciences, Heidelberg, Germany
| | - Yu-Le Wu
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany
- Collaboration for joint PhD degree between EMBL and Heidelberg University, Faculty of Biosciences, Heidelberg, Germany
| | - Ulf Matti
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany
| | - Decheng Wu
- Department of Biomedical Engineering, Southern University of Science and Technology, Shenzhen, 518055, China
| | - Simone Koehler
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany
| | - Jonas Ries
- European Molecular Biology Laboratory, Cell Biology and Biophysics, 69117, Heidelberg, Germany.
| |
Collapse
|
18
|
Fontbonne A, Sauer H, Goudail F. Comparison of methods for end-to-end co-optimization of optical systems and image processing with commercial lens design software. OPTICS EXPRESS 2022; 30:13556-13571. [PMID: 35472965 DOI: 10.1364/oe.455669] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Accepted: 03/24/2022] [Indexed: 06/14/2023]
Abstract
We compare three different methods to co-optimize hybrid optical/digital imaging systems with a commercial lens design software: conventional optimization based on spot diagram minimization, optimization of a surrogate criterion based on a priori equalization of modulation transfer functions (MTFs), and minimization of the mean square error (MSE) between the ideal sharp image and the image restored by a unique deconvolution filter. To implement the latter method, we integrate - for the first time to our knowledge - MSE optimization to the software Synopsys CodeV. Taking as an application example the design of a Cooke triplet having good image quality everywhere in the field of view (FoV), we show that it is possible, by leveraging deconvolution during the optimization process, to adapt the spatial distribution of imaging performance to a prescribed goal. We also demonstrate the superiority of MSE co-optimization over the other methods, both in terms of quantitative and visual image quality.
Collapse
|
19
|
Ikoma H, Kudo T, Peng Y, Broxton M, Wetzstein G. Deep learning multi-shot 3D localization microscopy using hybrid optical-electronic computing. OPTICS LETTERS 2021; 46:6023-6026. [PMID: 34913909 DOI: 10.1364/ol.441743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Accepted: 11/03/2021] [Indexed: 06/14/2023]
Abstract
Current 3D localization microscopy approaches are fundamentally limited in their ability to image thick, densely labeled specimens. Here, we introduce a hybrid optical-electronic computing approach that jointly optimizes an optical encoder (a set of multiple, simultaneously imaged 3D point spread functions) and an electronic decoder (a neural-network-based localization algorithm) to optimize 3D localization performance under these conditions. With extensive simulations and biological experiments, we demonstrate that our deep-learning-based microscope achieves significantly higher 3D localization accuracy than existing approaches, especially in challenging scenarios with high molecular density over large depth ranges.
Collapse
|
20
|
Ding T, Lew MD. Single-Molecule Localization Microscopy of 3D Orientation and Anisotropic Wobble Using a Polarized Vortex Point Spread Function. J Phys Chem B 2021; 125:12718-12729. [PMID: 34766758 PMCID: PMC8662813 DOI: 10.1021/acs.jpcb.1c08073] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
Abstract
Within condensed matter, single fluorophores are sensitive probes of their chemical environments, but it is difficult to use their limited photon budget to image precisely their positions, 3D orientations, and rotational diffusion simultaneously. We demonstrate the polarized vortex point spread function (PSF) for measuring these parameters, including characterizing the anisotropy of a molecule's wobble, simultaneously from a single image. Even when imaging dim emitters (∼500 photons detected), the polarized vortex PSF can obtain 12 nm localization precision, 4°-8° orientation precision, and 26° wobble precision. We use the vortex PSF to measure the emission anisotropy of fluorescent beads, the wobble dynamics of Nile red (NR) within supported lipid bilayers, and the distinct orientation signatures of NR in contact with amyloid-beta fibrils, oligomers, and tangles. The unparalleled sensitivity of the vortex PSF transforms single-molecule microscopes into nanoscale orientation imaging spectrometers, where the orientations and wobbles of individual probes reveal structures and organization of soft matter that are nearly impossible to perceive by using molecular positions alone.
Collapse
Affiliation(s)
- Tianben Ding
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, Missouri 63130, United States
- Center for Science and Engineering of Living Systems, Washington University in St. Louis, St. Louis, Missouri 63130, United States
| | - Matthew D Lew
- Department of Electrical and Systems Engineering, Washington University in St. Louis, St. Louis, Missouri 63130, United States
- Center for Science and Engineering of Living Systems, Washington University in St. Louis, St. Louis, Missouri 63130, United States
- Institute of Materials Science and Engineering, Washington University in St. Louis, St. Louis, Missouri 63130, United States
| |
Collapse
|