1
|
Xu F, Wu Z, Tan C, Liao Y, Wang Z, Chen K, Pan A. Fourier Ptychographic Microscopy 10 Years on: A Review. Cells 2024; 13:324. [PMID: 38391937 PMCID: PMC10887115 DOI: 10.3390/cells13040324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 01/31/2024] [Accepted: 02/08/2024] [Indexed: 02/24/2024] Open
Abstract
Fourier ptychographic microscopy (FPM) emerged as a prominent imaging technique in 2013, attracting significant interest due to its remarkable features such as precise phase retrieval, expansive field of view (FOV), and superior resolution. Over the past decade, FPM has become an essential tool in microscopy, with applications in metrology, scientific research, biomedicine, and inspection. This achievement arises from its ability to effectively address the persistent challenge of achieving a trade-off between FOV and resolution in imaging systems. It has a wide range of applications, including label-free imaging, drug screening, and digital pathology. In this comprehensive review, we present a concise overview of the fundamental principles of FPM and compare it with similar imaging techniques. In addition, we present a study on achieving colorization of restored photographs and enhancing the speed of FPM. Subsequently, we showcase several FPM applications utilizing the previously described technologies, with a specific focus on digital pathology, drug screening, and three-dimensional imaging. We thoroughly examine the benefits and challenges associated with integrating deep learning and FPM. To summarize, we express our own viewpoints on the technological progress of FPM and explore prospective avenues for its future developments.
Collapse
Affiliation(s)
- Fannuo Xu
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Zipei Wu
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- School of Physics and Optoelectronic Engineering, Shenzhen University, Shenzhen 518060, China
| | - Chao Tan
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- School of Electronics and Information Engineering, Sichuan University, Chengdu 610065, China
| | - Yizheng Liao
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Zhiping Wang
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- School of Physical Science and Technology, Lanzhou University, Lanzhou 730000, China
| | - Keru Chen
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- School of Automation Science and Engineering, Xi’an Jiaotong University, Xi’an 710049, China
| | - An Pan
- State Key Laboratory of Transient Optics and Photonics, Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi’an 710119, China; (F.X.); (Z.W.); (C.T.); (Y.L.); (Z.W.); (K.C.)
- University of Chinese Academy of Sciences, Beijing 100049, China
| |
Collapse
|
2
|
Wu R, Luo Z, Liu M, Zhang H, Zhen J, Yan L, Luo J, Wu Y. Fast Fourier ptychographic quantitative phase microscopy for in vitro label-free imaging. BIOMEDICAL OPTICS EXPRESS 2024; 15:95-113. [PMID: 38223174 PMCID: PMC10783909 DOI: 10.1364/boe.505267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 11/18/2023] [Accepted: 11/18/2023] [Indexed: 01/16/2024]
Abstract
Quantitative phase microscopy (QPM) is indispensable in biomedical research due to its advantages in unlabeled transparent sample thickness quantification and obtaining refractive index information. Fourier ptychographic microscopy (FPM) is among the most promising QPM methods, incorporating multi-angle illumination and iterative phase recovery for high-resolution quantitative phase imaging (QPI) of large cell populations over a wide field of-view (FOV) in a single pass. However, FPM is limited by data redundancy and sequential acquisition strategies, resulting in low imaging efficiency, which in turn limits its real-time application in in vitro label-free imaging. Here, we report a fast QPM based on Fourier ptychography (FQP-FPM), which uses an optimized annular downsampling and parallel acquisition strategy to minimize the amount of data required in the front end and reduce the iteration time of the back-end algorithm (3.3% and 4.4% of conventional FPM, respectively). Theoretical and data redundancy analyses show that FQP-FPM can realize high-throughput quantitative phase reconstruction at thrice the resolution of the coherent diffraction limit by acquiring only ten raw images, providing a precondition for in vitro label-free real-time imaging. The FQP-FPM application was validated for various in vitro label-free live-cell imaging. Cell morphology and subcellular phenomena in different periods were observed with a synthetic aperture of 0.75 NA at a 10× FOV, demonstrating its advantages and application potential for fast high-throughput QPI.
Collapse
Affiliation(s)
- Ruofei Wu
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Zicong Luo
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Mingdi Liu
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Haiqi Zhang
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Junrui Zhen
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Lisong Yan
- School of Optical and Electronic Information, Huazhong University of Science and Technology, Wuhan 430074, China
| | - Jiaxiong Luo
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
| | - Yanxiong Wu
- School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528000, China
- Ji Hua Laboratory, Foshan, Guangdong 528200, China
| |
Collapse
|
3
|
Wang N, Zhang C, Wei X, Yan T, Zhou W, Zhang J, Kang H, Yuan Z, Chen X. Harnessing the power of optical microscopy for visualization and analysis of histopathological images. BIOMEDICAL OPTICS EXPRESS 2023; 14:5451-5465. [PMID: 37854561 PMCID: PMC10581782 DOI: 10.1364/boe.501893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 08/29/2023] [Accepted: 09/01/2023] [Indexed: 10/20/2023]
Abstract
Histopathology is the foundation and gold standard for identifying diseases, and precise quantification of histopathological images can provide the pathologist with objective clues to make a more convincing diagnosis. Optical microscopy (OM), an important branch of optical imaging technology that provides high-resolution images of tissue cytology and structural morphology, has been used in the diagnosis of histopathology and evolved into a new disciplinary direction of optical microscopic histopathology (OMH). There are a number of ex-vivo studies providing applicability of different OMH approaches, and a transfer of these techniques toward in vivo diagnosis is currently in progress. Furthermore, combined with advanced artificial intelligence algorithms, OMH allows for improved diagnostic reliability and convenience due to the complementarity of retrieval information. In this review, we cover recent advances in OMH, including the exploration of new techniques in OMH as well as their applications, and look ahead to new challenges in OMH. These typical application examples well demonstrate the application potential and clinical value of OMH techniques in histopathological diagnosis.
Collapse
Affiliation(s)
- Nan Wang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Chang Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Xinyu Wei
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
| | - Tianyu Yan
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Wangting Zhou
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Jiaojiao Zhang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Huan Kang
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
| | - Zhen Yuan
- Faculty of Health Sciences, University of Macau, Macau, 999078, China
| | - Xueli Chen
- Center for Biomedical-photonics and Molecular Imaging, Xi’an Key Laboratory of Intelligent Sensing and Regulation of Trans-Scale Life Information, School of Life Science and Technology, Xidian University, Xi’an, Shaanxi 710126, China
- Engineering Research Center of Molecular and Neuro Imaging, Ministry of Education, Xi’an, Shaanxi 710126, China
- Inovation Center for Advanced Medical Imaging and Intelligent Medicine, Guangzhou Institute of Technology, Xidian University, Guangzhou, Guangdong 510555, China
| |
Collapse
|
4
|
Jiang S, Song P, Wang T, Yang L, Wang R, Guo C, Feng B, Maiden A, Zheng G. Spatial- and Fourier-domain ptychography for high-throughput bio-imaging. Nat Protoc 2023:10.1038/s41596-023-00829-4. [PMID: 37248392 DOI: 10.1038/s41596-023-00829-4] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Accepted: 03/03/2023] [Indexed: 05/31/2023]
Abstract
First envisioned for determining crystalline structures, ptychography has become a useful imaging tool for microscopists. However, ptychography remains underused by biomedical researchers due to its limited resolution and throughput in the visible light regime. Recent developments of spatial- and Fourier-domain ptychography have successfully addressed these issues and now offer the potential for high-resolution, high-throughput optical imaging with minimal hardware modifications to existing microscopy setups, often providing an excellent trade-off between resolution and field of view inherent to conventional imaging systems, giving biomedical researchers the best of both worlds. Here, we provide extensive information to enable the implementation of ptychography by biomedical researchers in the visible light regime. We first discuss the intrinsic connections between spatial-domain coded ptychography and Fourier ptychography. A step-by-step guide then provides the user instructions for developing both systems with practical examples. In the spatial-domain implementation, we explain how a large-scale, high-performance blood-cell lens can be made at negligible expense. In the Fourier-domain implementation, we explain how adding a low-cost light source to a regular microscope can improve the resolution beyond the limit of the objective lens. The turnkey operation of these setups is suitable for use by professional research laboratories, as well as citizen scientists. Users with basic experience in optics and programming can build the setups within a week. The do-it-yourself nature of the setups also allows these procedures to be implemented in laboratory courses related to Fourier optics, biomedical instrumentation, digital image processing, robotics and capstone projects.
Collapse
Affiliation(s)
- Shaowei Jiang
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Pengming Song
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Tianbo Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Liming Yang
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Ruihai Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Chengfei Guo
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
- Hangzhou Institute of Technology, Xidian University, Hangzhou, China
| | - Bin Feng
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA
| | - Andrew Maiden
- Department of Electronic and Electrical Engineering, University of Sheffield, Sheffield, UK
- Diamond Light Source, Harwell Science and Innovation Campus, Chilton, UK
| | - Guoan Zheng
- Department of Biomedical Engineering, University of Connecticut, Storrs, USA.
| |
Collapse
|
5
|
Guo C, Jiang S, Yang L, Song P, Pirhanov A, Wang R, Wang T, Shao X, Wu Q, Cho YK, Zheng G. Depth-multiplexed ptychographic microscopy for high-throughput imaging of stacked bio-specimens on a chip. Biosens Bioelectron 2023; 224:115049. [PMID: 36623342 DOI: 10.1016/j.bios.2022.115049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Revised: 12/15/2022] [Accepted: 12/26/2022] [Indexed: 01/01/2023]
Abstract
Imaging a large number of bio-specimens at high speed is essential for many biomedical applications. The common strategy is to place specimens at different lateral positions and image them sequentially. Here we report a new on-chip imaging strategy, termed depth-multiplexed ptychographic microscopy (DPM), for parallel imaging and sensing at high speed. Different from the common strategy, DPM stacks multiple specimens in the axial direction and images the entire z-stack all at once. In our prototype platform, we modify a low-cost car mirror for programmable steering of the incident laser beam. A blood-coated image sensor is then placed underneath the stacked sample for acquiring the resulting diffraction patterns. With the captured images, we perform blind recovery of the incident beam angle and model different layers of the stacked sample as different coded surfaces for object reconstruction. For in vitro experiment, we demonstrate time-lapse cell culture monitoring by imaging 3 stacked microfluidic channels on the coded sensor. For high-throughput cytometric analysis, we image 5 stacked brain sections with a 205-mm2 field of view in ∼50 s. Cytometric analysis is also performed to quantify the cellular proliferation biomarkers on the slides. The DPM approach adds a new degree of freedom for data multiplexing in microscopy, enabling parallel imaging of multiple specimens using a single detector. The demonstrated 6-mm depth of field is among the longest ones in microscopy imaging. The novel depth-multiplexed configuration also complements the miniaturization provided by microfluidics devices, offering a solution for on-chip sensing and imaging with efficient sample handling.
Collapse
Affiliation(s)
- Chengfei Guo
- Hangzhou Institute of Technology, Xidian University, Hangzhou, 311231, China; Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Shaowei Jiang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Liming Yang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Pengming Song
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Azady Pirhanov
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Ruihai Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Tianbo Wang
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Xiaopeng Shao
- Hangzhou Institute of Technology, Xidian University, Hangzhou, 311231, China
| | - Qian Wu
- Department of Pathology and Laboratory Medicine, University of Connecticut Health Centre, Farmington, CT, 06030, USA
| | - Yong Ku Cho
- Department of Chemical & Biomolecular Engineering, University of Connecticut, Storrs, CT, 06269, USA
| | - Guoan Zheng
- Department of Biomedical Engineering, University of Connecticut, Storrs, CT, 06269, USA.
| |
Collapse
|
6
|
Valentino M, Bianco V, Miccio L, Memmolo P, Brancato V, Libretti P, Gambacorta M, Salvatore M, Ferraro P. Beyond conventional microscopy: Observing kidney tissues by means of fourier ptychography. Front Physiol 2023; 14:1120099. [PMID: 36860516 PMCID: PMC9968938 DOI: 10.3389/fphys.2023.1120099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Accepted: 02/01/2023] [Indexed: 02/17/2023] Open
Abstract
Kidney microscopy is a mainstay in studying the morphological structure, physiology and pathology of kidney tissues, as histology provides important results for a reliable diagnosis. A microscopy modality providing at same time high-resolution images and a wide field of view could be very useful for analyzing the whole architecture and the functioning of the renal tissue. Recently, Fourier Ptychography (FP) has been proofed to yield images of biology samples such as tissues and in vitro cells while providing high resolution and large field of view, thus making it a unique and attractive opportunity for histopathology. Moreover, FP offers tissue imaging with high contrast assuring visualization of small desirable features, although with a stain-free mode that avoids any chemical process in histopathology. Here we report an experimental measuring campaign for creating the first comprehensive and extensive collection of images of kidney tissues captured by this FP microscope. We show that FP microscopy unlocks a new opportunity for the physicians to observe and judge renal tissue slides through the novel FP quantitative phase-contrast microscopy. Phase-contrast images of kidney tissue are analyzed by comparing them with the corresponding renal images taken under a conventional bright-field microscope both for stained and unstained tissue samples of different thicknesses. In depth discussion on the advantages and limitations of this new stain-free microscopy modality is reported, showing its usefulness over the classical light microscopy and opening a potential route for using FP in clinical practice for histopathology of kidney.
Collapse
Affiliation(s)
- Marika Valentino
- National Research Council (CNR) of Italy, Institute of Applied Sciences and Intelligent Systems (ISASI), Pozzuoli, Italy,Department of Electric and Information Technologies Engineering, University of Naples “Federico II”, Naples, Italy
| | - Vittorio Bianco
- National Research Council (CNR) of Italy, Institute of Applied Sciences and Intelligent Systems (ISASI), Pozzuoli, Italy,*Correspondence: Vittorio Bianco, ; Marcello Gambacorta,
| | - Lisa Miccio
- National Research Council (CNR) of Italy, Institute of Applied Sciences and Intelligent Systems (ISASI), Pozzuoli, Italy
| | - Pasquale Memmolo
- National Research Council (CNR) of Italy, Institute of Applied Sciences and Intelligent Systems (ISASI), Pozzuoli, Italy
| | | | | | - Marcello Gambacorta
- IRCCS SYNLAB SDN, Naples, Italy,*Correspondence: Vittorio Bianco, ; Marcello Gambacorta,
| | | | - Pietro Ferraro
- National Research Council (CNR) of Italy, Institute of Applied Sciences and Intelligent Systems (ISASI), Pozzuoli, Italy
| |
Collapse
|