1
|
Kim J, Kim G, Li L, Zhang P, Kim JY, Kim Y, Kim HH, Wang LV, Lee S, Kim C. Deep learning acceleration of multiscale superresolution localization photoacoustic imaging. LIGHT, SCIENCE & APPLICATIONS 2022; 11:131. [PMID: 35545614 PMCID: PMC9095876 DOI: 10.1038/s41377-022-00820-w] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Revised: 04/24/2022] [Accepted: 04/26/2022] [Indexed: 05/02/2023]
Abstract
A superresolution imaging approach that localizes very small targets, such as red blood cells or droplets of injected photoacoustic dye, has significantly improved spatial resolution in various biological and medical imaging modalities. However, this superior spatial resolution is achieved by sacrificing temporal resolution because many raw image frames, each containing the localization target, must be superimposed to form a sufficiently sampled high-density superresolution image. Here, we demonstrate a computational strategy based on deep neural networks (DNNs) to reconstruct high-density superresolution images from far fewer raw image frames. The localization strategy can be applied for both 3D label-free localization optical-resolution photoacoustic microscopy (OR-PAM) and 2D labeled localization photoacoustic computed tomography (PACT). For the former, the required number of raw volumetric frames is reduced from tens to fewer than ten. For the latter, the required number of raw 2D frames is reduced by 12 fold. Therefore, our proposed method has simultaneously improved temporal (via the DNN) and spatial (via the localization method) resolutions in both label-free microscopy and labeled tomography. Deep-learning powered localization PA imaging can potentially provide a practical tool in preclinical and clinical studies requiring fast temporal and fine spatial resolutions.
Collapse
Affiliation(s)
- Jongbeom Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea
| | - Gyuwon Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea
| | - Lei Li
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, 1200 E. California Blvd., MC 138-78, Pasadena, CA, 91125, USA
| | - Pengfei Zhang
- School of Precision Instruments and Optoelectronics Engineering, Tianjin University, 92 Weijin Road, Nankai District, Tianjin, 300072, China
| | - Jin Young Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea
- Opticho, 532, CHANGeUP GROUND, 87 Cheongam-ro, Nam-gu, Pohang, Gyeongsangbuk, 37673, Republic of Korea
| | - Yeonggeun Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea
| | - Hyung Ham Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea
| | - Lihong V Wang
- Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Department of Electrical Engineering, California Institute of Technology, 1200 E. California Blvd., MC 138-78, Pasadena, CA, 91125, USA.
| | - Seungchul Lee
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea.
| | - Chulhong Kim
- Departments of Electrical Engineering, Mechanical Engineering, Convergence IT Engineering, and Interdisciplinary Bioscience and Bioengineering, Graduate School of Artificial Intelligence, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Nam-gu, Pohang, Gyeongbuk, 37673, Republic of Korea.
- Opticho, 532, CHANGeUP GROUND, 87 Cheongam-ro, Nam-gu, Pohang, Gyeongsangbuk, 37673, Republic of Korea.
| |
Collapse
|
2
|
Cho S, Baik J, Managuli R, Kim C. 3D PHOVIS: 3D photoacoustic visualization studio. PHOTOACOUSTICS 2020; 18:100168. [PMID: 32211292 PMCID: PMC7082691 DOI: 10.1016/j.pacs.2020.100168] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 01/30/2020] [Accepted: 02/11/2020] [Indexed: 05/05/2023]
Abstract
Photoacoustic (PA) imaging (or optoacoustic imaging) is a novel biomedical imaging method in biological and medical research. This modality performs morphological, functional, and molecular imaging with and without labels in both microscopic and deep tissue imaging domains. A variety of innovations have enhanced 3D PA imaging performance and thus has opened new opportunities in preclinical and clinical imaging. However, the 3D visualization tools for PA images remains a challenge. There are several commercially available software packages to visualize the generated 3D PA images. They are generally expensive, and their features are not optimized for 3D visualization of PA images. Here, we demonstrate a specialized 3D visualization software package, namely 3D Photoacoustic Visualization Studio (3D PHOVIS), specifically targeting photoacoustic data, image, and visualization processes. To support the research environment for visualization and fast processing, we incorporated 3D PHOVIS onto the MATLAB with graphical user interface and developed multi-core graphics processing unit modules for fast processing. The 3D PHOVIS includes following modules: (1) a mosaic volume generator, (2) a scan converter for optical scanning photoacoustic microscopy, (3) a skin profile estimator and depth encoder, (4) a multiplanar viewer with a navigation map, and (5) a volume renderer with a movie maker. This paper discusses the algorithms present in the software package and demonstrates their functions. In addition, the applicability of this software to ultrasound imaging and optical coherence tomography is also investigated. User manuals and application files for 3D PHOVIS are available for free on the website (www.boa-lab.com). Core functions of 3D PHOVIS are developed as a result of a summer class at POSTECH, "High-Performance Algorithm in CPU/GPU/DSP, and Computer Architecture." We believe our 3D PHOVIS provides a unique tool to PA imaging researchers, expedites its growth, and attracts broad interests in a wide range of studies.
Collapse
Affiliation(s)
- Seonghee Cho
- School of Interdisciplinary Bioscience and Bioengineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
| | - Jinwoo Baik
- Department of Creative IT Engineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
| | - Ravi Managuli
- Department of Bioengineering, University of Washington, Seattle, WA, 98195, USA
- Hitachi Healthcare America, Twinsburg, OH, 44087, USA
| | - Chulhong Kim
- Departments of Creative IT Engineering, Mechanical Engineering, Electrical Engineering, and School of Interdisciplinary Bioscience and Bioengineering, Pohang University of Science and Technology (POSTECH), Pohang, 37673, Republic of Korea
- Corresponding author.
| |
Collapse
|