1
|
Han B, Chang Y, Tan RR, Han C. Evaluating deep learning techniques for identifying tongue features in subthreshold depression: a prospective observational study. Front Psychiatry 2024; 15:1361177. [PMID: 39176227 PMCID: PMC11338782 DOI: 10.3389/fpsyt.2024.1361177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Accepted: 07/15/2024] [Indexed: 08/24/2024] Open
Abstract
Objective This study aims to evaluate the potential of using tongue image features as non-invasive biomarkers for diagnosing subthreshold depression and to assess the correlation between these features and acupuncture treatment outcomes using advanced deep learning models. Methods We employed five advanced deep learning models-DenseNet169, MobileNetV3Small, SEResNet101, SqueezeNet, and VGG19_bn-to analyze tongue image features in individuals with subthreshold depression. These models were assessed based on accuracy, precision, recall, and F1 score. Additionally, we investigated the relationship between the best-performing model's predictions and the success of acupuncture treatment using Pearson's correlation coefficient. Results Among the models, SEResNet101 emerged as the most effective, achieving an impressive 98.5% accuracy and an F1 score of 0.97. A significant positive correlation was found between its predictions and the alleviation of depressive symptoms following acupuncture (Pearson's correlation coefficient = 0.72, p<0.001). Conclusion The findings suggest that the SEResNet101 model is highly accurate and reliable for identifying tongue image features in subthreshold depression. It also appears promising for assessing the impact of acupuncture treatment. This study contributes novel insights and approaches to the auxiliary diagnosis and treatment evaluation of subthreshold depression.
Collapse
Affiliation(s)
- Bo Han
- Department of Rehabilitation, Daqing Longnan Hospital, Daqing, China
| | - Yue Chang
- Department of Pharmacy, Baoan Central Hospital of Shenzhen, Shenzhen, China
| | - Rui-rui Tan
- Changchun University of Chinese Medicine, Changchun, China
| | - Chao Han
- Department of Acupuncture, Shenzhen Bao’an Authentic Traditional Chinese Medicine (TCM) Therapy Hospital, Shenzhen, China
| |
Collapse
|
2
|
Hong X, Tang F, Wang L, Chen J. Unsupervised deep learning enables real-time image registration of fast-scanning optical-resolution photoacoustic microscopy. PHOTOACOUSTICS 2024; 38:100632. [PMID: 39100197 PMCID: PMC11296048 DOI: 10.1016/j.pacs.2024.100632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/06/2024] [Revised: 06/16/2024] [Accepted: 07/02/2024] [Indexed: 08/06/2024]
Abstract
A fast scanner of optical-resolution photoacoustic microscopy is inherently vulnerable to perturbation, leading to severe image distortion and significant misalignment among multiple 2D or 3D images. Restoration and registration of these images is critical for accurately quantifying dynamic information in long-term imaging. However, traditional registration algorithms face a great challenge in computational throughput. Here, we develop an unsupervised deep learning based registration network to achieve real-time image restoration and registration. This method can correct artifacts from B-scan distortion and remove misalignment among adjacent and repetitive images in real time. Compared with conventional intensity based registration algorithms, the throughput of the developed algorithm is improved by 50 times. After training, the new deep learning method performs better than conventional feature based image registration algorithms. The results show that the proposed method can accurately restore and register the images of fast-scanning photoacoustic microscopy in real time, offering a powerful tool to extract dynamic vascular structural and functional information.
Collapse
Affiliation(s)
- Xiaobin Hong
- School of Mechanical & Automotive Engineering, South China University of Technology, Guangzhou, Guangdong, PR China
| | - Furong Tang
- School of Mechanical & Automotive Engineering, South China University of Technology, Guangzhou, Guangdong, PR China
| | - Lidai Wang
- Department of Biomedical Engineering, City University of Hong Kong, 83 Tat Chee Ave, Kowloon, Hong Kong Special Administrative Region of China
| | - Jiangbo Chen
- School of Mechanical & Automotive Engineering, South China University of Technology, Guangzhou, Guangdong, PR China
| |
Collapse
|
3
|
Feng T, Li J, Xie W, Cheng Q, Ta D. Adaptively multi-scale microstructure characterization of cancellous bone via Photoacoustic signal decomposition. ULTRASONICS 2024; 144:107407. [PMID: 39173274 DOI: 10.1016/j.ultras.2024.107407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/10/2023] [Revised: 07/09/2024] [Accepted: 07/15/2024] [Indexed: 08/24/2024]
Abstract
Osteoporosis is a systemic disease with a high incidence in the elderly and seriously affects the quality of life of patients. Photoacoustic (PA) technology, which combines the advantages of light and ultrasound, can provide information about the physiological structure and chemical information of biological tissues in a non-invasive and non-radiative way. Due to the complex structural characteristics of bone tissue, PA signals generated by bone tissue are non-stationary and nonlinear. However, conventional PA signal processing methods are not effective for non-stationary signal processing. In this study, an empirical mode decomposition (EMD)-based Hilbert-Huang transform (HHT) PA signal analysis method, called HHT PA signal analysis (HPSA), was developed to assess the microstructure information of bone tissue, which is closely related to bone health. The feasibility of the HPSA method in bone health assessment was proven by numerical simulation and experimental studies on animal samples with different bone volume/total volume (BV/TV) and bone mineral densities. First, based on adaptive EMD, the different modes correlated with multi-scale information were mined from the PA signal, the correlations between different intrinsic mode function (IMF) modes and BV/TVs were analyzed, and the optimal mode for more efficient PA time-frequency analysis was selected. Second, multi-wavelength HPSA was used to assess the changes in the chemical components of the bone tissue. The results demonstrate that the HPSA method can distinguish bones with different BV/TVs and microstructure conditions adaptively with high efficiency. They further emphasize the potential of PA techniques in characterizing biological tissues in bones for early and rapid detection of bone diseases.
Collapse
Affiliation(s)
- Ting Feng
- Institute of Biomedical Engineering & Technology, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China.
| | - Jieshu Li
- School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
| | - Weiya Xie
- Institute of Acoustics, School of Physics Science and Engineering, Tongji University, Shanghai 200092, China; Qingdao innovation and development base, Harbin Engineering University, Qingdao 266000, China
| | - Qian Cheng
- Institute of Acoustics, School of Physics Science and Engineering, Tongji University, Shanghai 200092, China.
| | - Dean Ta
- Institute of Biomedical Engineering & Technology, Academy for Engineering and Technology, Fudan University, Shanghai 200433, China; Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai 200433, China.
| |
Collapse
|
4
|
Lan H, Huang L, Wei X, Li Z, Lv J, Ma C, Nie L, Luo J. Masked cross-domain self-supervised deep learning framework for photoacoustic computed tomography reconstruction. Neural Netw 2024; 179:106515. [PMID: 39032393 DOI: 10.1016/j.neunet.2024.106515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2023] [Revised: 06/24/2024] [Accepted: 07/05/2024] [Indexed: 07/23/2024]
Abstract
Accurate image reconstruction is crucial for photoacoustic (PA) computed tomography (PACT). Recently, deep learning has been used to reconstruct PA images with a supervised scheme, which requires high-quality images as ground truth labels. However, practical implementations encounter inevitable trade-offs between cost and performance due to the expensive nature of employing additional channels for accessing more measurements. Here, we propose a masked cross-domain self-supervised (CDSS) reconstruction strategy to overcome the lack of ground truth labels from limited PA measurements. We implement the self-supervised reconstruction in a model-based form. Simultaneously, we take advantage of self-supervision to enforce the consistency of measurements and images across three partitions of the measured PA data, achieved by randomly masking different channels. Our findings indicate that dynamically masking a substantial proportion of channels, such as 80%, yields meaningful self-supervisors in both the image and signal domains. Consequently, this approach reduces the multiplicity of pseudo solutions and enables efficient image reconstruction using fewer PA measurements, ultimately minimizing reconstruction error. Experimental results on in-vivo PACT dataset of mice demonstrate the potential of our self-supervised framework. Moreover, our method exhibits impressive performance, achieving a structural similarity index (SSIM) of 0.87 in an extreme sparse case utilizing only 13 channels, which outperforms the performance of the supervised scheme with 16 channels (0.77 SSIM). Adding to its advantages, our method can be deployed on different trainable models in an end-to-end manner, further enhancing its versatility and applicability.
Collapse
Affiliation(s)
- Hengrong Lan
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Lijie Huang
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Xingyue Wei
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Zhiqiang Li
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Jing Lv
- Medical Research Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Southern Medical University, Guangzhou 510080, China
| | - Cheng Ma
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Liming Nie
- Medical Research Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Southern Medical University, Guangzhou 510080, China
| | - Jianwen Luo
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China.
| |
Collapse
|
5
|
Zhang S, Miao J, Li LS. Challenges and advances in two-dimensional photoacoustic computed tomography: a review. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:070901. [PMID: 39006312 PMCID: PMC11245175 DOI: 10.1117/1.jbo.29.7.070901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2023] [Revised: 06/18/2024] [Accepted: 06/19/2024] [Indexed: 07/16/2024]
Abstract
Significance Photoacoustic computed tomography (PACT), a hybrid imaging modality combining optical excitation with acoustic detection, has rapidly emerged as a prominent biomedical imaging technique. Aim We review the challenges and advances of PACT, including (1) limited view, (2) anisotropy resolution, (3) spatial aliasing, (4) acoustic heterogeneity (speed of sound mismatch), and (5) fluence correction of spectral unmixing. Approach We performed a comprehensive literature review to summarize the key challenges in PACT toward practical applications and discuss various solutions. Results There is a wide range of contributions from both industry and academic spaces. Various approaches, including emerging deep learning methods, are proposed to improve the performance of PACT further. Conclusions We outline contemporary technologies aimed at tackling the challenges in PACT applications.
Collapse
Affiliation(s)
- Shunyao Zhang
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Jingyi Miao
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| | - Lei S. Li
- Rice University, Department of Electrical and Computer Engineering, Houston, Texas, United States
| |
Collapse
|
6
|
Mitra M, Haworth A, Gaddale P, Badran F, Lagno N, Pameijer C, Aziz F, Kothapalli SR. Multiwavelength laser diode based portable photoacoustic and ultrasound imaging system for point of care applications. JOURNAL OF BIOPHOTONICS 2024; 17:e202400058. [PMID: 38695390 PMCID: PMC11239316 DOI: 10.1002/jbio.202400058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 04/14/2024] [Accepted: 04/19/2024] [Indexed: 07/13/2024]
Abstract
Vascular diseases are a leading cause of death and disability worldwide. Despite having precursor conditions like peripheral arterial disease (PAD), they are often only diagnosed after the onset of stroke or heart attack. Low-cost, portable, noninvasive, point-of-care (POC), label-free assessment of deep vascular function benefits PAD diagnosis, especially in resource poor settings of the world. Doppler ultrasound-based blood flow measurements can diagnose PAD, albeit with limited sensitivity and specificity. To overcome this, here, we propose the first-of-its-kind dual-modality photoacoustic-and-ultrasound (PAUS) imaging system that integrates a multiwavelength pulsed laser diode (PLD) with a compact ultrasound data acquisition unit. The mesoscopic imaging depth of the portable PLD-PAUS system was validated using tissue phantoms, and its multispectral photoacoustic imaging capabilities were validated using an atherosclerosis-mimicking phantom. Furthermore, we demonstrated high-contrast volumetric in vivo photoacoustic imaging of rodent abdominal vasculature and quantified vessel reactivity due to hypercapnia stimulation. The multiparametric functional and molecular imaging capabilities of the PLD-PAUS system holds promise for POC applications.
Collapse
Affiliation(s)
- Mahaan Mitra
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
| | - Abigail Haworth
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
| | - Prameth Gaddale
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
| | - Faisal Badran
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
| | - Nuzhath Lagno
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
| | - Colette Pameijer
- Divison of Vascular Surgery, Department of Surgery, The Pennsylvania State University, College of Medicine, Hershey, PA 17033, USA
| | - Faisal Aziz
- Divison of Vascular Surgery, Department of Surgery, The Pennsylvania State University, College of Medicine, Hershey, PA 17033, USA
| | - Sri-Rajasekhar Kothapalli
- Department of Biomedical Engineering, The Pennsylvania State University, University Park, PA 16802, USA
- Center for Neural Engineering, The Pennsylvania State University, University Park, PA 16802, USA
- Penn State Cancer Institute, The Pennsylvania State University, College of Medicine, Hershey, PA 17033, USA
- Graduate Program in Acoustics, The Pennsylvania State University, University Park, PA 16802, USA
| |
Collapse
|
7
|
Paul A, Mallidi S. U-Net enhanced real-time LED-based photoacoustic imaging. JOURNAL OF BIOPHOTONICS 2024; 17:e202300465. [PMID: 38622811 PMCID: PMC11164633 DOI: 10.1002/jbio.202300465] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 02/18/2024] [Accepted: 03/17/2024] [Indexed: 04/17/2024]
Abstract
Photoacoustic (PA) imaging is hybrid imaging modality with good optical contrast and spatial resolution. Portable, cost-effective, smaller footprint light emitting diodes (LEDs) are rapidly becoming important PA optical sources. However, the key challenge faced by the LED-based systems is the low light fluence that is generally compensated by high frame averaging, consequently reducing acquisition frame-rate. In this study, we present a simple deep learning U-Net framework that enhances the signal-to-noise ratio (SNR) and contrast of PA image obtained by averaging low number of frames. The SNR increased by approximately four-fold for both in-class in vitro phantoms (4.39 ± 2.55) and out-of-class in vivo models (4.27 ± 0.87). We also demonstrate the noise invariancy of the network and discuss the downsides (blurry outcome and failure to reduce the salt & pepper noise). Overall, the developed U-Net framework can provide a real-time image enhancement platform for clinically translatable low-cost and low-energy light source-based PA imaging systems.
Collapse
Affiliation(s)
- Avijit Paul
- Department of Biomedical Engineering, Tufts University, Medford, MA, USA
| | | |
Collapse
|
8
|
Poimala J, Cox B, Hauptmann A. Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography. PHOTOACOUSTICS 2024; 37:100597. [PMID: 38425677 PMCID: PMC10901832 DOI: 10.1016/j.pacs.2024.100597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/15/2023] [Accepted: 02/16/2024] [Indexed: 03/02/2024]
Abstract
Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.
Collapse
Affiliation(s)
- Jenni Poimala
- Research Unit of Mathematical Sciences, University of Oulu, Finland
| | - Ben Cox
- Department of Medical Physics and Biomedical Engineering, University College London, UK
| | - Andreas Hauptmann
- Research Unit of Mathematical Sciences, University of Oulu, Finland
- Department of Computer Science, University College London, UK
| |
Collapse
|
9
|
Tajaldeen A, Alrashidi M, Alsaadi MJ, Alghamdi SS, Alshammari H, Alsleem H, Jafer M, Aljondi R, Alqahtani S, Alotaibi A, Alzandi AM, Alahmari AM. Photoacoustic imaging in prostate cancer: A new paradigm for diagnosis and management. Photodiagnosis Photodyn Ther 2024; 47:104225. [PMID: 38821240 DOI: 10.1016/j.pdpdt.2024.104225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Revised: 05/24/2024] [Accepted: 05/28/2024] [Indexed: 06/02/2024]
Abstract
The global health issue of prostate cancer (PCa) requires better diagnosis and treatment. Photoacoustic imaging (PAI) may change PCa management. This review examines PAI's principles, diagnostic role, and therapeutic guidance. PAI uses optical light excitation and ultrasonic detection for high-resolution functional and molecular imaging. PAI uses endogenous and exogenous contrast agents to distinguish cancerous and benign prostate tissues with greater sensitivity and specificity than PSA testing and TRUS-guided biopsy. In addition to diagnosing, PAI can guide and monitor PCa therapy. Its real-time imaging allows precise biopsies and brachytherapy seed placement. Photoacoustic temperature imaging allows non-invasive monitoring of thermal therapies like cryotherapy, improving treatment precision and success. Transurethral illumination probes, innovative contrast agents, integration with other imaging modalities, and machine learning analysis are being developed to overcome depth and data complexity restrictions. PAI could become an essential tool for PCa diagnosis and therapeutic guidance as the field advances.
Collapse
Affiliation(s)
- Abdulrahman Tajaldeen
- Department of Radiologic Technology, College of Applied Medical Sciences, University of Jeddah, Jeddah 21959, Saudi Arabia.
| | - Muteb Alrashidi
- Department of Radiological Sciences, College of Applied Medical Sciences, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
| | - Mohamed J Alsaadi
- Radiology and Medical Imaging Department, College of Applied Medical Sciences, Prince Sattam Bin Abdulaziz University, Al-Kharj 11942, Saudi Arabia
| | - Salem Saeed Alghamdi
- Department of Radiologic Technology, College of Applied Medical Sciences, University of Jeddah, Jeddah 21959, Saudi Arabia
| | - Hamed Alshammari
- Department of Radiological Sciences, College of Applied Medical Sciences, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
| | - Haney Alsleem
- Department of Radiological Sciences, College of Applied Medical Sciences, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
| | - Mustafa Jafer
- Department of Radiologic Technology, College of Applied Medical Sciences, University of Jeddah, Jeddah 21959, Saudi Arabia
| | - Rowa Aljondi
- Department of Radiologic Technology, College of Applied Medical Sciences, University of Jeddah, Jeddah 21959, Saudi Arabia
| | - Saeed Alqahtani
- Radiological Sciences Department, College of Applied Medical Sciences, Najran University, Najran, Saudi Arabia
| | - Awatif Alotaibi
- Department of Radiological Sciences, College of Applied Medical Sciences, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
| | - Abdulrahman M Alzandi
- Department of Radiological Sciences, College of Applied Medical Sciences, Imam Abdulrahman Bin Faisal University, Dammam 34212, Saudi Arabia
| | | |
Collapse
|
10
|
Ma Y, Zhou W, Ma R, Wang E, Yang S, Tang Y, Zhang XP, Guan X. DOVE: Doodled vessel enhancement for photoacoustic angiography super resolution. Med Image Anal 2024; 94:103106. [PMID: 38387244 DOI: 10.1016/j.media.2024.103106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2023] [Revised: 12/12/2023] [Accepted: 02/08/2024] [Indexed: 02/24/2024]
Abstract
Deep-learning-based super-resolution photoacoustic angiography (PAA) has emerged as a valuable tool for enhancing the resolution of blood vessel images and aiding in disease diagnosis. However, due to the scarcity of training samples, PAA super-resolution models do not generalize well, especially in the challenging in-vivo imaging of organs with deep tissue penetration. Furthermore, prolonged exposure to high laser intensity during the image acquisition process can lead to tissue damage and secondary infections. To address these challenges, we propose an approach doodled vessel enhancement (DOVE) that utilizes hand-drawn doodles to train a PAA super-resolution model. With a training dataset consisting of only 32 real PAA images, we construct a diffusion model that interprets hand-drawn doodles as low-resolution images. DOVE enables us to generate a large number of realistic PAA images, achieving a 49.375% fool rate, even among experts in photoacoustic imaging. Subsequently, we employ these generated images to train a self-similarity-based model for super-resolution. During cross-domain tests, our method, trained solely on generated images, achieves a structural similarity value of 0.8591, surpassing the scores of all other models trained with real high-resolution images. DOVE successfully overcomes the limitation of insufficient training samples and unlocks the clinic application potential of super-resolution-based biomedical imaging.
Collapse
Affiliation(s)
- Yuanzheng Ma
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China; Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Wangting Zhou
- Engineering Research Center of Molecular & Neuro Imaging of the Ministry of Education, Xidian University, Xi'an, Shaanxi 710126, China
| | - Rui Ma
- MOE Key Laboratory of Laser Life Science & Institute of Laser Life Science, College of Biophotonics, South China Normal University, Guangzhou, 510631, China
| | - Erqi Wang
- MOE Key Laboratory of Laser Life Science & Institute of Laser Life Science, College of Biophotonics, South China Normal University, Guangzhou, 510631, China
| | - Sihua Yang
- MOE Key Laboratory of Laser Life Science & Institute of Laser Life Science, College of Biophotonics, South China Normal University, Guangzhou, 510631, China.
| | - Yansong Tang
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China; Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Xiao-Ping Zhang
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China; Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China
| | - Xun Guan
- Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China; Institute of Data and Information, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, 518055, China.
| |
Collapse
|
11
|
Yamakawa M, Shiina T. Artifact reduction in photoacoustic images by generating virtual dense array sensor from hemispheric sparse array sensor using deep learning. J Med Ultrason (2001) 2024; 51:169-183. [PMID: 38480548 PMCID: PMC11098876 DOI: 10.1007/s10396-024-01413-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/30/2024] [Indexed: 05/19/2024]
Abstract
PURPOSE Vascular distribution is important information for diagnosing diseases and supporting surgery. Photoacoustic imaging is a technology that can image blood vessels noninvasively and with high resolution. In photoacoustic imaging, a hemispherical array sensor is especially suitable for measuring blood vessels running in various directions. However, as a hemispherical array sensor, a sparse array sensor is often used due to technical and cost issues, which causes artifacts in photoacoustic images. Therefore, in this study, we reduce these artifacts using deep learning technology to generate signals of virtual dense array sensors. METHODS Generating 2D virtual array sensor signals using a 3D convolutional neural network (CNN) requires huge computational costs and is impractical. Therefore, we installed virtual sensors between the real sensors along the spiral pattern in three different directions and used a 2D CNN to generate signals of the virtual sensors in each direction. Then we reconstructed a photoacoustic image using the signals from both the real sensors and the virtual sensors. RESULTS We evaluated the proposed method using simulation data and human palm measurement data. We found that these artifacts were significantly reduced in the images reconstructed using the proposed method, while the artifacts were strong in the images obtained only from the real sensor signals. CONCLUSION Using the proposed method, we were able to significantly reduce artifacts, and as a result, it became possible to recognize deep blood vessels. In addition, the processing time of the proposed method was sufficiently applicable to clinical measurement.
Collapse
Affiliation(s)
- Makoto Yamakawa
- SIT Research Laboratories, Shibaura Institute of Technology, 3-7-5 Toyosu, Koto-ku, Tokyo, 135-8548, Japan.
| | - Tsuyoshi Shiina
- SIT Research Laboratories, Shibaura Institute of Technology, 3-7-5 Toyosu, Koto-ku, Tokyo, 135-8548, Japan
| |
Collapse
|
12
|
Pérez-Pacheco A, Ramírez-Chavarría RG, Quispe-Siccha RM, Colín-García MP. Dynamic modeling of photoacoustic sensor data to classify human blood samples. Med Biol Eng Comput 2024; 62:389-403. [PMID: 37880558 PMCID: PMC10794472 DOI: 10.1007/s11517-023-02939-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 09/14/2023] [Indexed: 10/27/2023]
Abstract
The photoacoustic effect is an attractive tool for diagnosis in several biomedical applications. Analyzing photoacoustic signals, however, is challenging to provide qualitative results in an automated way. In this work, we introduce a dynamic modeling scheme of photoacoustic sensor data to classify blood samples according to their physiological status. Thirty-five whole human blood samples were studied with a state-space model estimated by a subspace method. Furthermore, the samples are classified using the model parameters and the linear discriminant analysis algorithm. The classification performance is compared with time- and frequency-domain features and an autoregressive-moving-average model. As a result, the proposed analysis can predict five blood classes: healthy women and men, microcytic and macrocytic anemia, and leukemia. Our findings indicate that the proposed method outperforms conventional signal processing techniques to analyze photoacoustic data for medical diagnosis. Hence, the method is a promising tool in point-of-care devices to detect hematological diseases in clinical scenarios.
Collapse
Affiliation(s)
- Argelia Pérez-Pacheco
- Unidad de Investigación y Desarrollo Tecnológico (UIDT), Hospital General de México "Dr. Eduardo Liceaga", Dr. Balmis 148, 06720, Cuauhtémoc, Doctores, Ciudad de México, México.
| | - Roberto G Ramírez-Chavarría
- Instituto de Ingeniería, Universidad Nacional Autónoma de México, Av. Universidad 3000, 04510, Ciudad Universitaria, Coyoacán, Ciudad de México, México.
| | - Rosa M Quispe-Siccha
- Unidad de Investigación y Desarrollo Tecnológico (UIDT), Hospital General de México "Dr. Eduardo Liceaga", Dr. Balmis 148, 06720, Cuauhtémoc, Doctores, Ciudad de México, México
| | - Marco P Colín-García
- Programa de Maestría y Doctorado en Ingeniería, Universidad Nacional Autónoma de México, Av. Universidad 3000, 04510, Ciudad Universitaria, Coyoacán, Ciudad de México, México
| |
Collapse
|
13
|
Cho SW, Nguyen VT, DiSpirito A, Yang J, Kim CS, Yao J. Sounding out the dynamics: a concise review of high-speed photoacoustic microscopy. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11521. [PMID: 38323297 PMCID: PMC10846286 DOI: 10.1117/1.jbo.29.s1.s11521] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 12/15/2023] [Accepted: 01/11/2024] [Indexed: 02/08/2024]
Abstract
Significance Photoacoustic microscopy (PAM) offers advantages in high-resolution and high-contrast imaging of biomedical chromophores. The speed of imaging is critical for leveraging these benefits in both preclinical and clinical settings. Ongoing technological innovations have substantially boosted PAM's imaging speed, enabling real-time monitoring of dynamic biological processes. Aim This concise review synthesizes historical context and current advancements in high-speed PAM, with an emphasis on developments enabled by ultrafast lasers, scanning mechanisms, and advanced imaging processing methods. Approach We examine cutting-edge innovations across multiple facets of PAM, including light sources, scanning and detection systems, and computational techniques and explore their representative applications in biomedical research. Results This work delineates the challenges that persist in achieving optimal high-speed PAM performance and forecasts its prospective impact on biomedical imaging. Conclusions Recognizing the current limitations, breaking through the drawbacks, and adopting the optimal combination of each technology will lead to the realization of ultimate high-speed PAM for both fundamental research and clinical translation.
Collapse
Affiliation(s)
- Soon-Woo Cho
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
- Pusan National University, Engineering Research Center for Color-Modulated Extra-Sensory Perception Technology, Busan, Republic of Korea
| | - Van Tu Nguyen
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Anthony DiSpirito
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Joseph Yang
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| | - Chang-Seok Kim
- Pusan National University, Engineering Research Center for Color-Modulated Extra-Sensory Perception Technology, Busan, Republic of Korea
| | - Junjie Yao
- Duke University, Department of Biomedical Engineering, Durham, North Carolina, United States
| |
Collapse
|
14
|
Tarvainen T, Cox B. Quantitative photoacoustic tomography: modeling and inverse problems. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11509. [PMID: 38125717 PMCID: PMC10731766 DOI: 10.1117/1.jbo.29.s1.s11509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 11/19/2023] [Accepted: 11/28/2023] [Indexed: 12/23/2023]
Abstract
Significance Quantitative photoacoustic tomography (QPAT) exploits the photoacoustic effect with the aim of estimating images of clinically relevant quantities related to the tissue's optical absorption. The technique has two aspects: an acoustic part, where the initial acoustic pressure distribution is estimated from measured photoacoustic time-series, and an optical part, where the distributions of the optical parameters are estimated from the initial pressure. Aim Our study is focused on the optical part. In particular, computational modeling of light propagation (forward problem) and numerical solution methodologies of the image reconstruction (inverse problem) are discussed. Approach The commonly used mathematical models of how light and sound propagate in biological tissue are reviewed. A short overview of how the acoustic inverse problem is usually treated is given. The optical inverse problem and methods for its solution are reviewed. In addition, some limitations of real-life measurements and their effect on the inverse problems are discussed. Results An overview of QPAT with a focus on the optical part was given. Computational modeling and inverse problems of QPAT were addressed, and some key challenges were discussed. Furthermore, the developments for tackling these problems were reviewed. Although modeling of light transport is well-understood and there is a well-developed framework of inverse mathematics for approaching the inverse problem of QPAT, there are still challenges in taking these methodologies to practice. Conclusions Modeling and inverse problems of QPAT together were discussed. The scope was limited to the optical part, and the acoustic aspects were discussed only to the extent that they relate to the optical aspect.
Collapse
Affiliation(s)
- Tanja Tarvainen
- University of Eastern Finland, Department of Technical Physics, Kuopio, Finland
| | - Ben Cox
- University College London, Department of Medical Physics and Biomedical Engineering, London, United Kingdom
| |
Collapse
|
15
|
Liu H, Wang M, Ji F, Jiang Y, Yang M. Mini review of photoacoustic clinical imaging: a noninvasive tool for disease diagnosis and treatment evaluation. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11522. [PMID: 38230369 PMCID: PMC10790789 DOI: 10.1117/1.jbo.29.s1.s11522] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Revised: 12/05/2023] [Accepted: 12/14/2023] [Indexed: 01/18/2024]
Abstract
Significance Photoacoustic (PA) imaging is an imaging modality that integrates anatomical, functional, metabolic, and histologic insights. It has been a hot topic of medical research and draws extensive attention. Aim This review aims to explore the applications of PA clinical imaging in human diseases, highlighting recent advancements. Approach A systemic survey of the literature concerning the clinical utility of PA imaging was conducted, with a particular focus on its application in tumors, autoimmune diseases, inflammatory conditions, and endocrine disorders. Results PA imaging is emerging as a valuable tool for human disease investigation. Information provided by PA imaging can be used for diagnosis, grading, and prognosis in multiple types of tumors including breast tumors, ovarian neoplasms, thyroid nodules, and cutaneous malignancies. PA imaging facilitates the monitoring of disease activity in autoimmune and inflammatory diseases such as rheumatoid arthritis, systemic sclerosis, arteritis, and inflammatory bowel disease by capturing dynamic functional alterations. Furthermore, its unique capability of visualizing vascular structure and oxygenation levels aids in assessing diabetes mellitus comorbidities and thyroid function. Conclusions Despite extant challenges, PA imaging offers a promising noninvasive tool for precision disease diagnosis, long-term evaluation, and prognosis anticipation, making it a potentially significant imaging modality for clinical practice.
Collapse
Affiliation(s)
- Huazhen Liu
- Chinese Academy of Medical Sciences and Peking Union Medical College, Peking Union Medical College Hospital, Department of Ultrasound, Beijing, China
| | - Ming Wang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Peking Union Medical College Hospital, Department of Ultrasound, Beijing, China
| | - Fei Ji
- Chinese Academy of Medical Sciences and Peking Union Medical College, Peking Union Medical College Hospital, Department of Ultrasound, Beijing, China
| | - Yuxin Jiang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Peking Union Medical College Hospital, Department of Ultrasound, Beijing, China
| | - Meng Yang
- Chinese Academy of Medical Sciences and Peking Union Medical College, Peking Union Medical College Hospital, Department of Ultrasound, Beijing, China
| |
Collapse
|
16
|
Kim M, Pelivanov I, O'Donnell M. Review of Deep Learning Approaches for Interleaved Photoacoustic and Ultrasound (PAUS) Imaging. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:1591-1606. [PMID: 37910419 PMCID: PMC10788151 DOI: 10.1109/tuffc.2023.3329119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/03/2023]
Abstract
Photoacoustic (PA) imaging provides optical contrast at relatively large depths within the human body, compared to other optical methods, at ultrasound (US) spatial resolution. By integrating real-time PA and US (PAUS) modalities, PAUS imaging has the potential to become a routine clinical modality bringing the molecular sensitivity of optics to medical US imaging. For applications where the full capabilities of clinical US scanners must be maintained in PAUS, conventional limited view and bandwidth transducers must be used. This approach, however, cannot provide high-quality maps of PA sources, especially vascular structures. Deep learning (DL) using data-driven modeling with minimal human design has been very effective in medical imaging, medical data analysis, and disease diagnosis, and has the potential to overcome many of the technical limitations of current PAUS imaging systems. The primary purpose of this article is to summarize the background and current status of DL applications in PAUS imaging. It also looks beyond current approaches to identify remaining challenges and opportunities for robust translation of PAUS technologies to the clinic.
Collapse
|
17
|
Gao Y, Feng T, Qiu H, Gu Y, Chen Q, Zuo C, Ma H. 4D spectral-spatial computational photoacoustic dermoscopy. PHOTOACOUSTICS 2023; 34:100572. [PMID: 38058749 PMCID: PMC10696115 DOI: 10.1016/j.pacs.2023.100572] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 10/16/2023] [Accepted: 11/09/2023] [Indexed: 12/08/2023]
Abstract
Photoacoustic dermoscopy (PAD) is an emerging non-invasive imaging technology aids in the diagnosis of dermatological conditions by obtaining optical absorption information of skin tissues. Despite advances in PAD, it remains unclear how to obtain quantitative accuracy of the reconstructed PAD images according to the optical and acoustic properties of multilayered skin, the wavelength and distribution of excitation light, and the detection performance of ultrasound transducers. In this work, a computing method of four-dimensional (4D) spectral-spatial imaging for PAD is developed to enable quantitative analysis and optimization of structural and functional imaging of skin. This method takes the optical and acoustic properties of heterogeneous skin tissues into account, which can be used to correct the optical field of excitation light, detectable ultrasonic field, and provide accurate single-spectrum analysis or multi-spectral imaging solutions of PAD for multilayered skin tissues. A series of experiments were performed, and simulation datasets obtained from the computational model were used to train neural networks to further improve the imaging quality of the PAD system. All the results demonstrated the method could contribute to the development and optimization of clinical PADs by datasets with multiple variable parameters, and provide clinical predictability of photoacoustic (PA) data for human skin.
Collapse
Affiliation(s)
- Yang Gao
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Smart Computational Imaging Laboratory (SCILab), Nanjing 210094, China
- Smart Computational Imaging Research Institute (SCIRI) of Nanjing University of Science and Technology, Nanjing 210094, China
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing 210094, China
| | - Ting Feng
- Fudan University, Academy for Engineering and Technology, Shanghai 200433, China
| | - Haixia Qiu
- First Medical Center of PLA General Hospital, Beijing 100853, China
| | - Ying Gu
- First Medical Center of PLA General Hospital, Beijing 100853, China
| | - Qian Chen
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Smart Computational Imaging Laboratory (SCILab), Nanjing 210094, China
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing 210094, China
| | - Chao Zuo
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Smart Computational Imaging Laboratory (SCILab), Nanjing 210094, China
- Smart Computational Imaging Research Institute (SCIRI) of Nanjing University of Science and Technology, Nanjing 210094, China
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing 210094, China
| | - Haigang Ma
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Smart Computational Imaging Laboratory (SCILab), Nanjing 210094, China
- Smart Computational Imaging Research Institute (SCIRI) of Nanjing University of Science and Technology, Nanjing 210094, China
- Nanjing University of Science and Technology, School of Electronic and Optical Engineering, Jiangsu Key Laboratory of Spectral Imaging & Intelligent Sense, Nanjing 210094, China
| |
Collapse
|
18
|
Mandal T, Mishra SR, Singh V. Comprehensive advances in the synthesis, fluorescence mechanism and multifunctional applications of red-emitting carbon nanomaterials. NANOSCALE ADVANCES 2023; 5:5717-5765. [PMID: 37881704 PMCID: PMC10597556 DOI: 10.1039/d3na00447c] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 09/12/2023] [Indexed: 10/27/2023]
Abstract
Red emitting fluorescent carbon nanomaterials have drawn significant scientific interest in recent years due to their high quantum yield, water-dispersibility, photostability, biocompatibility, ease of surface functionalization, low cost and eco-friendliness. The red emissive characteristics of fluorescent carbon nanomaterials generally depend on the carbon source, reaction time, synthetic approach/methodology, surface functional groups, average size, and other reaction environments, which directly or indirectly help to achieve red emission. The importance of several factors to achieve red fluorescent carbon nanomaterials is highlighted in this review. Numerous plausible theories have been explained in detail to understand the origin of red fluorescence and tunable emission in these carbon-based nanostructures. The above advantages and fluorescence in the red region make them a potential candidate for multifunctional applications in various current fields. Therefore, this review focused on the recent advances in the synthesis approach, mechanism of fluorescence, and electronic and optical properties of red-emitting fluorescent carbon nanomaterials. This review also explains the several innovative applications of red-emitting fluorescent carbon nanomaterials such as biomedicine, light-emitting devices, sensing, photocatalysis, energy, anticounterfeiting, fluorescent silk, artificial photosynthesis, etc. It is hoped that by choosing appropriate methods, the present review can inspire and guide future research on the design of red emissive fluorescent carbon nanomaterials for potential advancements in multifunctional applications.
Collapse
Affiliation(s)
- Tuhin Mandal
- Environment Emission and CRM Section, CSIR-Central Institute of Mining and Fuel Research Dhanbad Jharkhand 828108 India
- Academy of Scientific and Innovative Research (AcSIR) Ghaziabad 201 002 India
| | - Shiv Rag Mishra
- Environment Emission and CRM Section, CSIR-Central Institute of Mining and Fuel Research Dhanbad Jharkhand 828108 India
- Academy of Scientific and Innovative Research (AcSIR) Ghaziabad 201 002 India
| | - Vikram Singh
- Environment Emission and CRM Section, CSIR-Central Institute of Mining and Fuel Research Dhanbad Jharkhand 828108 India
- Academy of Scientific and Innovative Research (AcSIR) Ghaziabad 201 002 India
| |
Collapse
|
19
|
Zheng W, Zhang H, Huang C, Shijo V, Xu C, Xu W, Xia J. Deep Learning Enhanced Volumetric Photoacoustic Imaging of Vasculature in Human. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2023; 10:e2301277. [PMID: 37530209 PMCID: PMC10582405 DOI: 10.1002/advs.202301277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 06/26/2023] [Indexed: 08/03/2023]
Abstract
The development of high-performance imaging processing algorithms is a core area of photoacoustic tomography. While various deep learning based image processing techniques have been developed in the area, their applications in 3D imaging are still limited due to challenges in computational cost and memory allocation. To address those limitations, this work implements a 3D fully-dense (3DFD) U-net to linear array based photoacoustic tomography and utilizes volumetric simulation and mixed precision training to increase efficiency and training size. Through numerical simulation, phantom imaging, and in vivo experiments, this work demonstrates that the trained network restores the true object size, reduces the noise level and artifacts, improves the contrast at deep regions, and reveals vessels subject to limited view distortion. With these enhancements, 3DFD U-net successfully produces clear 3D vascular images of the palm, arms, breasts, and feet of human subjects. These enhanced vascular images offer improved capabilities for biometric identification, foot ulcer evaluation, and breast cancer imaging. These results indicate that the new algorithm will have a significant impact on preclinical and clinical photoacoustic tomography.
Collapse
Affiliation(s)
- Wenhan Zheng
- Department of Biomedical EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Huijuan Zhang
- Department of Biomedical EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Chuqin Huang
- Department of Biomedical EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Varun Shijo
- Department of Biomedical EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
- Department of Computer Science and EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Chenhan Xu
- Department of Computer Science and EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Wenyao Xu
- Department of Computer Science and EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| | - Jun Xia
- Department of Biomedical EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
- Department of Computer Science and EngineeringUniversity at BuffaloThe State University of New YorkBuffaloNew YorkNY14260USA
| |
Collapse
|
20
|
Ramírez-Chavarría RG, Pérez-Pacheco A, Terán E, Quispe-Siccha RM. Study of Polyvinyl Alcohol Hydrogels Applying Physical-Mechanical Methods and Dynamic Models of Photoacoustic Signals. Gels 2023; 9:727. [PMID: 37754408 PMCID: PMC10529391 DOI: 10.3390/gels9090727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Revised: 08/23/2023] [Accepted: 08/29/2023] [Indexed: 09/28/2023] Open
Abstract
This study aims to analyze the physical-mechanical properties and dynamic models of tissue-simulating hydrogels, specifically the photoacoustic (PA) response signals, by varying the concentrations of polyvinyl alcohol (PVA) and molecular weight (MW). A state-space model (SSM) is proposed to study the PVA hydrogels to retrieve the PA-related signal's damping ratio and natural frequency. Nine box-shaped PVA hydrogels containing saline solution were used, with five concentrations of PVA (7, 9, 12, 15, 20%) for MW1 and four for MW2. The results indicated that the concentration of PVA and MW played an important role in the PA wave's amplitude, arrival time, and speed of sound over the hydrogels. The SSM parameters showed that increasing PVA and MW concentrations improved the hydrogels' ability to absorb and transfer energy under the PA effect. These parameters were also found to be correlated with density and modulus of elasticity. Additionally, the concentrations of PVA and MW affected the absorption and optical scattering coefficients. The physical-mechanical properties, including porosity, density, and modulus of elasticity, improved as the concentration of PVA and MW increased. The ultimate goal of this study is to develop hydrogels as phantoms that can be used for tissue simulation and imaging.
Collapse
Affiliation(s)
| | - Argelia Pérez-Pacheco
- Research and Technological Development Unit, Research Department, General Hospital of Mexico “Dr. Eduardo Liceaga”, Mexico City 06726, Mexico;
| | - Emiliano Terán
- Faculty of Physical-Mathematical Sciences, Autonomous University of Sinaloa, Culiacan, Sinaloa 80040, Mexico;
| | - Rosa M. Quispe-Siccha
- Research and Technological Development Unit, Research Department, General Hospital of Mexico “Dr. Eduardo Liceaga”, Mexico City 06726, Mexico;
| |
Collapse
|
21
|
Le TD, Min JJ, Lee C. Enhanced resolution and sensitivity acoustic-resolution photoacoustic microscopy with semi/unsupervised GANs. Sci Rep 2023; 13:13423. [PMID: 37591911 PMCID: PMC10435476 DOI: 10.1038/s41598-023-40583-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 08/13/2023] [Indexed: 08/19/2023] Open
Abstract
Acoustic-resolution photoacoustic microscopy (AR-PAM) enables visualization of biological tissues at depths of several millimeters with superior optical absorption contrast. However, the lateral resolution and sensitivity of AR-PAM are generally lower than those of optical-resolution PAM (OR-PAM) owing to the intrinsic physical acoustic focusing mechanism. Here, we demonstrate a computational strategy with two generative adversarial networks (GANs) to perform semi/unsupervised reconstruction with high resolution and sensitivity in AR-PAM by maintaining its imaging capability at enhanced depths. The b-scan PAM images were prepared as paired (for semi-supervised conditional GAN) and unpaired (for unsupervised CycleGAN) groups for label-free reconstructed AR-PAM b-scan image generation and training. The semi/unsupervised GANs successfully improved resolution and sensitivity in a phantom and in vivo mouse ear test with ground truth. We also confirmed that GANs could enhance resolution and sensitivity of deep tissues without the ground truth.
Collapse
Affiliation(s)
- Thanh Dat Le
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju, 61186, Korea
| | - Jung-Joon Min
- Department of Nuclear Medicine, Chonnam National University Medical School and Hwasun Hospital, 264, Seoyang-ro, Hwasun-eup, Hwasun-gun, 58128, Jeollanam-do, Korea
| | - Changho Lee
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju, 61186, Korea.
- Department of Nuclear Medicine, Chonnam National University Medical School and Hwasun Hospital, 264, Seoyang-ro, Hwasun-eup, Hwasun-gun, 58128, Jeollanam-do, Korea.
| |
Collapse
|
22
|
Rix T, Dreher KK, Nölke JH, Schellenberg M, Tizabi MD, Seitel A, Maier-Hein L. Efficient Photoacoustic Image Synthesis with Deep Learning. SENSORS (BASEL, SWITZERLAND) 2023; 23:7085. [PMID: 37631628 PMCID: PMC10457787 DOI: 10.3390/s23167085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 07/25/2023] [Accepted: 08/07/2023] [Indexed: 08/27/2023]
Abstract
Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.
Collapse
Affiliation(s)
- Tom Rix
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany
| | - Kris K. Dreher
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- Faculty of Physics and Astronomy, Heidelberg University, 69120 Heidelberg, Germany
| | - Jan-Hinrich Nölke
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany
| | - Melanie Schellenberg
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany
- HIDSS4Health—Helmholtz Information and Data Science School for Health, 69120 Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany
| | - Minu D. Tizabi
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany
| | - Alexander Seitel
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 223, 69120 Heidelberg, Germany
- Faculty of Mathematics and Computer Sciences, Heidelberg University, 69120 Heidelberg, Germany
- HIDSS4Health—Helmholtz Information and Data Science School for Health, 69120 Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, 69120 Heidelberg, Germany
- Medical Faculty, Heidelberg University, 69120 Heidelberg, Germany
| |
Collapse
|
23
|
Gao R, Chen T, Ren Y, Liu L, Chen N, Wong KK, Song L, Ma X, Liu C. Restoring the imaging quality of circular transducer array-based PACT using synthetic aperture focusing technique integrated with 2nd-derivative-based back projection scheme. PHOTOACOUSTICS 2023; 32:100537. [PMID: 37559663 PMCID: PMC10407438 DOI: 10.1016/j.pacs.2023.100537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 07/16/2023] [Accepted: 07/19/2023] [Indexed: 08/11/2023]
Abstract
Circular-array-based photoacoustic computed tomography (CA-PACT) is a promising imaging tool owing to its broad acoustic detection coverage and fidelity. However, CA-PACT suffers from poor image quality outside the focal zone along both elevational and lateral dimensions. To address this challenge, we proposed a novel reconstruction strategy by integrating the synthetic aperture focusing technique (SAFT) with the 2nd derivative-based back projection (2nd D-BP) algorithm to restore the image quality outside the focal zone along both the elevational and lateral axes. The proposed solution is a two-phase reconstruction scheme. In the first phase, with the assistance of an acoustic lens, we designed a circular array-based SAFT algorithm to restore the resolution and SNR along the elevational axis. The acoustic lens pushes the boundary of the upper limit of the SAFT scheme to achieve enhanced elevational resolution. In the second phase, we proposed a 2nd D-BP scheme to improve the lateral resolution and suppress noises in 3D imaging results. The 2nd D-BP strategy enhances the image quality along the lateral dimension by up-converting the high spatial frequencies of the object's absorption pattern. We validated the effectiveness of the proposed strategy using both phantoms and in vivo human experiments. The experimental results demonstrated superior image quality (7-fold enhancement in elevational resolution, 3-fold enhancement in lateral resolution, and an 11-dB increase in SNR). This strategy provides a new paradigm in the PACT system as it significantly enhances the spatial resolution and imaging contrast in both the elevational and lateral dimensions while maintaining a large focal zone.
Collapse
Affiliation(s)
- Rongkang Gao
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Tao Chen
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Yaguang Ren
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Liangjian Liu
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Ningbo Chen
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- The University of Hong Kong, Department of Electrical and Electronic Engineering, Hong Kong China
| | - Kenneth K.Y. Wong
- The University of Hong Kong, Department of Electrical and Electronic Engineering, Hong Kong China
| | - Liang Song
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Xiaohui Ma
- The first medical center of Chinese PLA General Hospital, the Department of Vascular and Endovascular Surgery, Beijing, China
| | - Chengbo Liu
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| |
Collapse
|
24
|
Han S, Kye H, Kim CS, Kim TK, Yoo J, Kim J. Automated Laser-Fiber Coupling Module for Optical-Resolution Photoacoustic Microscopy. SENSORS (BASEL, SWITZERLAND) 2023; 23:6643. [PMID: 37514935 PMCID: PMC10384817 DOI: 10.3390/s23146643] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 07/11/2023] [Accepted: 07/14/2023] [Indexed: 07/30/2023]
Abstract
Photoacoustic imaging has emerged as a promising biomedical imaging technique that enables visualization of the optical absorption characteristics of biological tissues in vivo. Among the different photoacoustic imaging system configurations, optical-resolution photoacoustic microscopy stands out by providing high spatial resolution using a tightly focused laser beam, which is typically transmitted through optical fibers. Achieving high-quality images depends significantly on optical fluence, which is directly proportional to the signal-to-noise ratio. Hence, optimizing the laser-fiber coupling is critical. Conventional coupling systems require manual adjustment of the optical path to direct the laser beam into the fiber, which is a repetitive and time-consuming process. In this study, we propose an automated laser-fiber coupling module that optimizes laser delivery and minimizes the need for manual intervention. By incorporating a motor-mounted mirror holder and proportional derivative control, we successfully achieved efficient and robust laser delivery. The performance of the proposed system was evaluated using a leaf-skeleton phantom in vitro and a human finger in vivo, resulting in high-quality photoacoustic images. This innovation has the potential to significantly enhance the quality and efficiency of optical-resolution photoacoustic microscopy.
Collapse
Affiliation(s)
- Seongyi Han
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Hyunjun Kye
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Chang-Seok Kim
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Tae-Kyoung Kim
- Department of Electronic Engineering, Gachon University, Seongnam 13120, Republic of Korea
| | - Jinwoo Yoo
- Department of Automobile and IT Convergence, Kookmin University, Seoul 02707, Republic of Korea
| | - Jeesu Kim
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| |
Collapse
|
25
|
Zhang Z, Jin H, Zhang W, Lu W, Zheng Z, Sharma A, Pramanik M, Zheng Y. Adaptive enhancement of acoustic resolution photoacoustic microscopy imaging via deep CNN prior. PHOTOACOUSTICS 2023; 30:100484. [PMID: 37095888 PMCID: PMC10121479 DOI: 10.1016/j.pacs.2023.100484] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Accepted: 03/29/2023] [Indexed: 05/03/2023]
Abstract
Acoustic resolution photoacoustic microscopy (AR-PAM) is a promising medical imaging modality that can be employed for deep bio-tissue imaging. However, its relatively low imaging resolution has greatly hindered its wide applications. Previous model-based or learning-based PAM enhancement algorithms either require design of complex handcrafted prior to achieve good performance or lack the interpretability and flexibility that can adapt to different degradation models. However, the degradation model of AR-PAM imaging is subject to both imaging depth and center frequency of ultrasound transducer, which varies in different imaging conditions and cannot be handled by a single neural network model. To address this limitation, an algorithm integrating both learning-based and model-based method is proposed here so that a single framework can deal with various distortion functions adaptively. The vasculature image statistics is implicitly learned by a deep convolutional neural network, which served as plug and play (PnP) prior. The trained network can be directly plugged into the model-based optimization framework for iterative AR-PAM image enhancement, which fitted for different degradation mechanisms. Based on physical model, the point spread function (PSF) kernels for various AR-PAM imaging situations are derived and used for the enhancement of simulation and in vivo AR-PAM images, which collectively proved the effectiveness of proposed method. Quantitatively, the PSNR and SSIM values have all achieve best performance with the proposed algorithm in all three simulation scenarios; The SNR and CNR values have also significantly raised from 6.34 and 5.79 to 35.37 and 29.66 respectively in an in vivo testing result with the proposed algorithm.
Collapse
Affiliation(s)
- Zhengyuan Zhang
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Haoran Jin
- Zhejiang University, College of Mechanical Engineering, The State Key Laboratory of Fluid Power and Mechatronic Systems, Hangzhou 310027, China
| | - Wenwen Zhang
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Wenhao Lu
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Zesheng Zheng
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
| | - Arunima Sharma
- Johns Hopkins University, Electrical and Computer Engineering, Baltimore, MD 21218, USA
| | - Manojit Pramanik
- Iowa State University, Department of Electrical and Computer Engineering, Ames, Iowa, USA
| | - Yuanjin Zheng
- Nanyang Technological University, School of Electrical and Electronic Engineering, 639798, Singapore
- Corresponding author.
| |
Collapse
|
26
|
Wang R, Zhu J, Xia J, Yao J, Shi J, Li C. Photoacoustic imaging with limited sampling: a review of machine learning approaches. BIOMEDICAL OPTICS EXPRESS 2023; 14:1777-1799. [PMID: 37078052 PMCID: PMC10110324 DOI: 10.1364/boe.483081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 03/03/2023] [Accepted: 03/17/2023] [Indexed: 05/03/2023]
Abstract
Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.
Collapse
Affiliation(s)
- Ruofan Wang
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jing Zhu
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jun Xia
- Department of Biomedical Engineering, University at Buffalo, The State University of New York, Buffalo, NY 14260, USA
| | - Junjie Yao
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Junhui Shi
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Chiye Li
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| |
Collapse
|
27
|
Cox B, Anastasio M. Photoacoustics special issue 'Photoacoustic image reconstruction: Theory and practice'. PHOTOACOUSTICS 2023; 30:100461. [PMID: 37125256 PMCID: PMC10140389 DOI: 10.1016/j.pacs.2023.100461] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
|
28
|
Han S, Ninjbadgar T, Kang M, Kim C, Kim J. Recent Advances in Photoacoustic Agents for Theranostic Applications. NANOMATERIALS (BASEL, SWITZERLAND) 2023; 13:695. [PMID: 36839061 PMCID: PMC9964871 DOI: 10.3390/nano13040695] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Revised: 02/07/2023] [Accepted: 02/08/2023] [Indexed: 06/18/2023]
Abstract
Photoacoustic agents are widely used in various theranostic applications. By evaluating the biodistribution obtained from photoacoustic images, the effectiveness of theranostic agents in terms of their delivery efficiency and treatment responses can be analyzed. Through this study, we evaluate and summarize the recent advances in photoacoustic-guided phototherapy, particularly in photothermal and photodynamic therapy. This overview can guide the future directions for theranostic development. Because of the recent applications of photoacoustic imaging in clinical trials, theranostic agents with photoacoustic monitoring have the potential to be translated into the clinical world.
Collapse
Affiliation(s)
- Seongyi Han
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Tsedendamba Ninjbadgar
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Mijeong Kang
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| | - Chulhong Kim
- Departments of Convergence IT Engineering, Mechanical Engineering, and Electrical Engineering, School of Interdisciplinary Bioscience and Bioengineering, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
| | - Jeesu Kim
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Republic of Korea
| |
Collapse
|
29
|
Lan H, Yang C, Gao F. A jointed feature fusion framework for photoacoustic image reconstruction. PHOTOACOUSTICS 2023; 29:100442. [PMID: 36589516 PMCID: PMC9798177 DOI: 10.1016/j.pacs.2022.100442] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 12/19/2022] [Indexed: 06/17/2023]
Abstract
The standard reconstruction of Photoacoustic (PA) computed tomography (PACT) image could cause the artifacts due to interferences or ill-posed setup. Recently, deep learning has been used to reconstruct the PA image with ill-posed conditions. In this paper, we propose a jointed feature fusion framework (JEFF-Net) based on deep learning to reconstruct the PA image using limited-view data. The cross-domain features from limited-view position-wise data and the reconstructed image are fused by a backtracked supervision. A quarter position-wise data (32 channels) is fed into model, which outputs another 3-quarters-view data (96 channels). Moreover, two novel losses are designed to restrain the artifacts by sufficiently manipulating superposed data. The experimental results have demonstrated the superior performance and quantitative evaluations show that our proposed method outperformed the ground-truth in some metrics by 135% (SSIM for simulation) and 40% (gCNR for in-vivo) improvement.
Collapse
Affiliation(s)
- Hengrong Lan
- Hybrid Imaging System Laboratory, Shanghai Engineering Research Center of Intelligent Vision and Imaging, School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing 100084, China
| | - Changchun Yang
- Hybrid Imaging System Laboratory, Shanghai Engineering Research Center of Intelligent Vision and Imaging, School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
| | - Fei Gao
- Hybrid Imaging System Laboratory, Shanghai Engineering Research Center of Intelligent Vision and Imaging, School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China
- Shanghai Clinical Research and Trial Center, Shanghai 201210, China
| |
Collapse
|
30
|
Huen I, Zhang R, Bi R, Li X, Moothanchery M, Olivo M. An Investigation of Signal Preprocessing for Photoacoustic Tomography. SENSORS (BASEL, SWITZERLAND) 2023; 23:510. [PMID: 36617107 PMCID: PMC9823775 DOI: 10.3390/s23010510] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 12/07/2022] [Accepted: 12/21/2022] [Indexed: 06/17/2023]
Abstract
Photoacoustic tomography (PAT) is increasingly being used for high-resolution biological imaging at depth. Signal-to-noise ratios and resolution are the main factors that determine image quality. Various reconstruction algorithms have been proposed and applied to reduce noise and enhance resolution, but the efficacy of signal preprocessing methods which also affect image quality, are seldom discussed. We, therefore, compared common preprocessing techniques, namely bandpass filters, wavelet denoising, empirical mode decomposition, and singular value decomposition. Each was compared with and without accounting for sensor directivity. The denoising performance was evaluated with the contrast-to-noise ratio (CNR), and the resolution was calculated as the full width at half maximum (FWHM) in both the lateral and axial directions. In the phantom experiment, counting in directivity was found to significantly reduce noise, outperforming other methods. Irrespective of directivity, the best performing methods for denoising were bandpass, unfiltered, SVD, wavelet, and EMD, in that order. Only bandpass filtering consistently yielded improvements. Significant improvements in the lateral resolution were observed using directivity in two out of three acquisitions. This study investigated the advantages and disadvantages of different preprocessing methods and may help to determine better practices in PAT reconstruction.
Collapse
|
31
|
Schellenberg M, Gröhl J, Dreher KK, Nölke JH, Holzwarth N, Tizabi MD, Seitel A, Maier-Hein L. Photoacoustic image synthesis with generative adversarial networks. PHOTOACOUSTICS 2022; 28:100402. [PMID: 36281320 PMCID: PMC9587371 DOI: 10.1016/j.pacs.2022.100402] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 09/03/2022] [Accepted: 09/08/2022] [Indexed: 06/16/2023]
Abstract
Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).
Collapse
Affiliation(s)
- Melanie Schellenberg
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg, Germany
| | - Janek Gröhl
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Cancer Research UK Cambridge Institute, University of Cambridge, Robinson Way, Cambridge, CB2 0RE, UK
| | - Kris K. Dreher
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany
| | - Jan-Hinrich Nölke
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Niklas Holzwarth
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
| | - Minu D. Tizabi
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Alexander Seitel
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Lena Maier-Hein
- Intelligent Medical Systems (IMSY), German Cancer Research Center (DKFZ), Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg, Germany
- Medical Faculty, Heidelberg University, Heidelberg, Germany
- HIP Applied Computer Vision Lab, Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
32
|
Fang Z, Gao F, Jin H, Liu S, Wang W, Zhang R, Zheng Z, Xiao X, Tang K, Lou L, Tang KT, Chen J, Zheng Y. A Review of Emerging Electromagnetic-Acoustic Sensing Techniques for Healthcare Monitoring. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2022; 16:1075-1094. [PMID: 36459601 DOI: 10.1109/tbcas.2022.3226290] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Conventional electromagnetic (EM) sensing techniques such as radar and LiDAR are widely used for remote sensing, vehicle applications, weather monitoring, and clinical monitoring. Acoustic techniques such as sonar and ultrasound sensors are also used for consumer applications, such as ranging and in vivo medical/healthcare applications. It has been of long-term interest to doctors and clinical practitioners to realize continuous healthcare monitoring in hospitals and/or homes. Physiological and biopotential signals in real-time serve as important health indicators to predict and prevent serious illness. Emerging electromagnetic-acoustic (EMA) sensing techniques synergistically combine the merits of EM sensing with acoustic imaging to achieve comprehensive detection of physiological and biopotential signals. Further, EMA enables complementary fusion sensing for challenging healthcare settings, such as real-world long-term monitoring of treatment effects at home or in remote environments. This article reviews various examples of EMA sensing instruments, including implementation, performance, and application from the perspectives of circuits to systems. The novel and significant applications to healthcare are discussed. Three types of EMA sensors are presented: (1) Chip-based radar sensors for health status monitoring, (2) Thermo-acoustic sensing instruments for biomedical applications, and (3) Photoacoustic (PA) sensing and imaging systems, including dedicated reconstruction algorithms were reviewed from time-domain, frequency-domain, time-reversal, and model-based solutions. The future of EMA techniques for continuous healthcare with enhanced accuracy supported by artificial intelligence (AI) is also presented.
Collapse
|
33
|
Guezzi N, Lee C, Le TD, Seong H, Choi KH, Min JJ, Yu J. Multistage adaptive noise reduction technique for optical resolution photoacoustic microscopy. JOURNAL OF BIOPHOTONICS 2022; 15:e202200164. [PMID: 36053943 DOI: 10.1002/jbio.202200164] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 07/22/2022] [Accepted: 07/25/2022] [Indexed: 06/15/2023]
Abstract
Photoacoustic microscopy has received great attention due to the benefits of the optical resolution contrast as well as its superior spatial resolution and relatively deep depth. Like other imaging modalities, photoacoustic images suffer from noise, and filtering techniques are required to remove them. To overcome the noise, we proposed a combination of filters, including an adaptive median filter, an effective filter for impulsive noise, and a nonlocal means filter, an effective filter for background noise, for noise removal and image quality enhancement. Our proposed method enhanced the signal-to-noise ratio by 16 dB in an in vivo study compared to the traditional image reconstruction approach and preserved the image detail with minimal blurring, which usually occurs when filtering. These experimental results verified that the proposed adaptive multistage denoising techniques could effectively improve image quality under noisy data acquisition conditions, providing a strong foundation for photoacoustic microscopy with limited laser power.
Collapse
Affiliation(s)
- Nizar Guezzi
- Department of Robotics and Mechatronics Engineering, DGIST, Daegu, South Korea
- DGIST Robotics Research Center, DGIST, Daegu, South Korea
| | - Changho Lee
- Department of Nuclear Medicine, Chonnam National University Medical School & Hwasun Hospital, Hwasun, Jeollanamado, South Korea
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju, South Korea
| | - Thanh Dat Le
- Department of Artificial Intelligence Convergence, Chonnam National University, Gwangju, South Korea
| | - Hyojin Seong
- Department of Robotics and Mechatronics Engineering, DGIST, Daegu, South Korea
- DGIST Robotics Research Center, DGIST, Daegu, South Korea
| | - Kang-Ho Choi
- Department of Neurology, Chonnam National University Hospital, Gwangju, South Korea
| | - Jung-Joon Min
- Department of Nuclear Medicine, Chonnam National University Medical School & Hwasun Hospital, Hwasun, Jeollanamado, South Korea
| | - Jaesok Yu
- Department of Robotics and Mechatronics Engineering, DGIST, Daegu, South Korea
- DGIST Robotics Research Center, DGIST, Daegu, South Korea
- The Interdisciplinary Studies of Artificial Intelligence, DGIST, Daegu, South Korea
| |
Collapse
|
34
|
Yoon C, Lee C, Shin K, Kim C. Motion Compensation for 3D Multispectral Handheld Photoacoustic Imaging. BIOSENSORS 2022; 12:1092. [PMID: 36551059 PMCID: PMC9775698 DOI: 10.3390/bios12121092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 11/26/2022] [Accepted: 11/28/2022] [Indexed: 06/17/2023]
Abstract
Three-dimensional (3D) handheld photoacoustic (PA) and ultrasound (US) imaging performed using mechanical scanning are more useful than conventional 2D PA/US imaging for obtaining local volumetric information and reducing operator dependence. In particular, 3D multispectral PA imaging can capture vital functional information, such as hemoglobin concentrations and hemoglobin oxygen saturation (sO2), of epidermal, hemorrhagic, ischemic, and cancerous diseases. However, the accuracy of PA morphology and physiological parameters is hampered by motion artifacts during image acquisition. The aim of this paper is to apply appropriate correction to remove the effect of such motion artifacts. We propose a new motion compensation method that corrects PA images in both axial and lateral directions based on structural US information. 3D PA/US imaging experiments are performed on a tissue-mimicking phantom and a human wrist to verify the effects of the proposed motion compensation mechanism and the consequent spectral unmixing results. The structural motions and sO2 values are confirmed to be successfully corrected by comparing the motion-compensated images with the original images. The proposed method is expected to be useful in various clinical PA imaging applications (e.g., breast cancer, thyroid cancer, and carotid artery disease) that are susceptible to motion contamination during multispectral PA image analysis.
Collapse
Affiliation(s)
- Chiho Yoon
- Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
| | - Changyeop Lee
- Department of Mechanical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
| | | | - Chulhong Kim
- Departments of Electrical Engineering, Convergence IT Engineering, and Mechanical Engineering, Medical Science and Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, Republic of Korea
| |
Collapse
|
35
|
Kasinathan P, Prabha R, Sabeenian RS, Baskar K, Ramkumar A, Alemayehu S. Development of Deep Learning Technique of Features for the Analysis of Clinical Images Integrated with CANN. BIOMED RESEARCH INTERNATIONAL 2022; 2022:2742274. [PMID: 36277892 PMCID: PMC9586784 DOI: 10.1155/2022/2742274] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 09/30/2022] [Indexed: 11/23/2022]
Abstract
Computer tomography is an extensively used method for the detection of the disease in the subjects. Basically, computer-aided tomography depending on the artificial intelligence reveals its significance in smart health care monitoring system. Owing to its security and the private issue, analyzing the computed tomography dataset has become a tedious process. This study puts forward the convolutional autoencrypted deep learning neural network to assist unsupervised learning technique. By carrying out various experiments, our proposed method produces better results comparative to other traditional methods, which efficaciously solves the issues related to the artificial image description. Hence, the convolutional autoencoder is widely used in measuring the lumps in the bronchi. With the unsupervised machine learning, the extracted features are used for various applications.
Collapse
Affiliation(s)
- Prabakaran Kasinathan
- Department of Computer Science and Engineering, School of Computing, Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology, Chennai, 600062 Tamil Nadu, India
| | - R. Prabha
- Department of Electronics and Communication Engineering, Sri Sai Ram Institute of Technology, Chennai, 600044 Tamil Nadu, India
| | - R. S. Sabeenian
- Department of Electronics and Communication Engineering, Sona College of Technology, Salem, 636005 Tamil Nadu, India
| | - K. Baskar
- Department of Artificial Intelligence and Data Science, Kongunadu College of Engineering and Technology, Trichy, 621215 Tamil Nadu, India
| | - A. Ramkumar
- Department of Electrical and Electronics Engineering, Kalasalingam Academy of Research and Education, Krishnankoil, Tamil Nadu, India
| | - Samson Alemayehu
- Department of Electrical and Computer Engineering, Faculty of Electrical and Biomedical Engineering, Institute of Technology, Hawassa University, Ethiopia
| |
Collapse
|
36
|
Zheng W, Zhang H, Huang C, McQuillan K, Li H, Xu W, Xia J. Deep-E Enhanced Photoacoustic Tomography Using Three-Dimensional Reconstruction for High-Quality Vascular Imaging. SENSORS (BASEL, SWITZERLAND) 2022; 22:7725. [PMID: 36298076 PMCID: PMC9606963 DOI: 10.3390/s22207725] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Revised: 09/30/2022] [Accepted: 10/09/2022] [Indexed: 06/01/2023]
Abstract
Linear-array-based photoacoustic computed tomography (PACT) has been widely used in vascular imaging due to its low cost and high compatibility with current ultrasound systems. However, linear-array transducers have inherent limitations for three-dimensional imaging due to the poor elevation resolution. In this study, we introduced a deep learning-assisted data process algorithm to enhance the image quality in linear-array-based PACT. Compared to our earlier study where training was performed on 2D reconstructed data, here, we utilized 2D and 3D reconstructed data to train the two networks separately. We then fused the image data from both 2D and 3D training to get features from both algorithms. The numerical and in vivo validations indicate that our approach can improve elevation resolution, recover the true size of the object, and enhance deep vessels. Our deep learning-assisted approach can be applied to translational imaging applications that require detailed visualization of vascular features.
Collapse
Affiliation(s)
- Wenhan Zheng
- Department of Biomedical Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Huijuan Zhang
- Department of Biomedical Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Chuqin Huang
- Department of Biomedical Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Kaylin McQuillan
- Department of Biomedical Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Huining Li
- Department of Computer Science and Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Wenyao Xu
- Department of Computer Science and Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| | - Jun Xia
- Department of Biomedical Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
- Department of Computer Science and Engineering, University at Buffalo North Campus, Buffalo, NY 14260, USA
| |
Collapse
|
37
|
Mirg S, Turner KL, Chen H, Drew PJ, Kothapalli SR. Photoacoustic imaging for microcirculation. Microcirculation 2022; 29:e12776. [PMID: 35793421 PMCID: PMC9870710 DOI: 10.1111/micc.12776] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 06/13/2022] [Accepted: 06/28/2022] [Indexed: 01/26/2023]
Abstract
Microcirculation facilitates the blood-tissue exchange of nutrients and regulates blood perfusion. It is, therefore, essential in maintaining tissue health. Aberrations in microcirculation are potentially indicative of underlying cardiovascular and metabolic pathologies. Thus, quantitative information about it is of great clinical relevance. Photoacoustic imaging (PAI) is a capable technique that relies on the generation of imaging contrast via the absorption of light and can image at micron-scale resolution. PAI is especially desirable to map microvasculature as hemoglobin strongly absorbs light and can generate a photoacoustic signal. This paper reviews the current state of the art for imaging microvascular networks using photoacoustic imaging. We further describe how quantitative information about blood dynamics such as the total hemoglobin concentration, oxygen saturation, and blood flow rate is obtained using PAI. We also discuss its importance in understanding key pathophysiological processes in neurovascular, cardiovascular, ophthalmic, and cancer research fields. We then discuss the current challenges and limitations of PAI and the approaches that can help overcome these limitations. Finally, we provide the reader with an overview of future trends in the field of PAI for imaging microcirculation.
Collapse
Affiliation(s)
- Shubham Mirg
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Kevin L. Turner
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Haoyang Chen
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Patrick J. Drew
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA 16802, USA,Department of Neurosurgery, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Sri-Rajasekhar Kothapalli
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Penn State Cancer Institute, Pennsylvania State University, Hershey, PA 17033, USA,Graduate Program in Acoustics, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA,Corresponding author: Sri-Rajasekhar Kothapalli, 325 CBE Building, State College, PA, 16802, USA,
| |
Collapse
|
38
|
Liu S, Zhang R, Han T, Pan Y, Zhang G, Long X, Zhao C, Wang M, Li X, Yang F, Sang Y, Zhu L, He X, Li J, Zhang Y, Li C, Jiang Y, Yang M. Validation of photoacoustic/ultrasound dual imaging in evaluating blood oxygen saturation. BIOMEDICAL OPTICS EXPRESS 2022; 13:5551-5570. [PMID: 36425613 PMCID: PMC9664893 DOI: 10.1364/boe.469747] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 09/16/2022] [Accepted: 09/16/2022] [Indexed: 06/16/2023]
Abstract
Photoacoustic imaging (PAI) was performed to evaluate oxygen saturation (sO2) of blood-mimicking phantoms, femoral arteries in beagles, and radial arteries in humans at various sO2 plateaus. The accuracy (root mean square error, RMSE) of PAI sO2 compared with reference sO2 was calculated. In blood-mimicking phantoms, PAI achieved an accuracy of 1.49% and a mean absolute error (MAE) of 1.09% within 25 mm depth, and good linearity (R = 0.968; p < 0.001) was obtained between PAI sO2 and reference sO2. In canine femoral arteries, PAI achieved an accuracy of 2.16% and an MAE of 1.58% within 8 mm depth (R = 0.965; p < 0.001). In human radial arteries, PAI achieved an accuracy of 3.97% and an MAE of 3.28% in depth from 4 to 14 mm (R = 0.892; p < 0.001). For PAI sO2 evaluation at different depths in healthy volunteers, the RMSE accuracy of PAI sO2 increased from 2.66% to 24.96% with depth increasing from 4 to 14 mm. Through the multiscale method, we confirmed the feasibility of the hand-held photoacoustic/ultrasound (PA/US) in evaluating sO2. These results demonstrate the potential clinical value of PAI in evaluating blood sO2. Consequently, protocols for verifying the feasibility of medical devices based on PAI may be established.
Collapse
Affiliation(s)
- Sirui Liu
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
- The authors contributed equally to this manuscript
| | - Rui Zhang
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
- The authors contributed equally to this manuscript
| | - Tao Han
- Department of Biomedical Engineering, College of Engineering, Peking University, Beijing, China
| | - Yinhao Pan
- Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Guangjie Zhang
- Department of Biomedical Engineering, College of Engineering, Peking University, Beijing, China
| | - Xing Long
- Department of Biomedical Engineering, College of Engineering, Peking University, Beijing, China
| | - Chenyang Zhao
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| | - Ming Wang
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| | - Xuelan Li
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| | - Fang Yang
- Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Yuchao Sang
- Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Lei Zhu
- Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Xujin He
- Mindray Bio-Medical Electronics Co., Ltd., Shenzhen, China
| | - Jianchu Li
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| | - Yewei Zhang
- Hepatopancreatobiliary Center, The Second Affiliated Hospital of Nanjing Medical University, Nanjing, China
| | - Changhui Li
- Department of Biomedical Engineering, College of Engineering, Peking University, Beijing, China
| | - Yuxin Jiang
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| | - Meng Yang
- Department of Ultrasound, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Science and Peking Union Medical College, Beijing, China
| |
Collapse
|
39
|
Xiao J, Jiang J, Zhang J, Wang Y, Wang B. Acoustic-resolution-based spectroscopic photoacoustic endoscopy towards molecular imaging in deep tissues. OPTICS EXPRESS 2022; 30:35014-35028. [PMID: 36242503 DOI: 10.1364/oe.469550] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 08/22/2022] [Indexed: 06/16/2023]
Abstract
Due to many technical difficulties, the study of molecular photoacoustic endoscopic (PAE) imaging in deep tissues is limited. In this work, we have set up a multimodal acoustic-resolution-based PAE (AR-PAE) system to image the rabbit rectum and preliminarily explored the potential of molecular PAE for deep-seated targets in proof-of-concept. We developed an improved back-projection (IBP) algorithm for focused detection over the centimeter-scale imaging depth. We also developed a deep-learning-based algorithm to remove the electrical noise from the step motor to prevent data averaging for reduced scanning time. We injected a dose of indocyanine green (ICG) near the rabbit rectum and compared 2D and 3D photoacoustic/ultrasound (PA/US) images at different wavelengths. We proposed incorporating a small camera to guide the slow PA/US endoscopic scan. Results show that this system has achieved a lateral resolution of about 0.77/0.65 mm for PA/US images with a signal-to-noise ratio (SNR) of 25/38 dB at an imaging depth of 1.4 cm. We found that the rectum wall and the ICG can be well distinguished spectroscopically. Results also show that the PA images at 532 nm have higher signal intensity and reflection artifacts from pelvic tendons and bones than those at longer wavelengths such as 800 nm. The proposed methods and the intuitive findings in this work may guide and promote the development of high-penetration molecular PAE.
Collapse
|
40
|
Shahid H, Khalid A, Yue Y, Liu X, Ta D. Feasibility of a Generative Adversarial Network for Artifact Removal in Experimental Photoacoustic Imaging. ULTRASOUND IN MEDICINE & BIOLOGY 2022; 48:1628-1643. [PMID: 35660105 DOI: 10.1016/j.ultrasmedbio.2022.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 03/06/2022] [Accepted: 04/16/2022] [Indexed: 06/15/2023]
Abstract
Photoacoustic tomography (PAT) reconstruction is an expeditiously growing interest among biomedical researchers because of its possible transition from laboratory to clinical pre-eminence. Nonetheless, the PAT inverse problem is yet to achieve an optimal solution in rapid and precise reconstruction under practical constraints. Precisely, the sparse sampling problem and random noise are the main impediments to attaining accuracy but in support of rapid PAT reconstruction. The limitations are associated with acquiring undersampled artifacts that deteriorate the optimality of the reconstruction task. Therefore, the former achievements of fast image formation limit the modality for clinical settings. Delving into the problem, here we explore a deep learning-based generative adversarial network (GAN) to improve the image quality by denoising and removing these artifacts. The specially designed attributes and unique manner of optimizing the problem, such as incorporating the data set limitations and providing stable training performance, constitute the main motivation behind the employment of GAN. Moreover, exploitation of the U-net variant as a generator network offers robust performance in terms of quality and computational cost, which is further validated with the detailed quantitative and qualitative analysis. The quantitatively evaluated structured similarity indexing method = 0.980 ± 0.043 and peak signal-to-noise ratio = 31 ± 0.002 dB state that the proposed solution provides the high-resolution image at the output, even training with a low-quality data set.
Collapse
Affiliation(s)
- Husnain Shahid
- Center for Biomedical Engineering, Fudan University, China
| | - Adnan Khalid
- School of Information and Communication Engineering, Tianjin University, China
| | - Yaoting Yue
- Center for Biomedical Engineering, Fudan University, China
| | - Xin Liu
- Academy for Engineering and Technology, Fudan University, Shanghai, China.
| | - Dean Ta
- Center for Biomedical Engineering, Fudan University, China; Academy for Engineering and Technology, Fudan University, Shanghai, China.
| |
Collapse
|
41
|
Meng J, Zhang X, Liu L, Zeng S, Fang C, Liu C. Depth-extended acoustic-resolution photoacoustic microscopy based on a two-stage deep learning network. BIOMEDICAL OPTICS EXPRESS 2022; 13:4386-4397. [PMID: 36032586 PMCID: PMC9408237 DOI: 10.1364/boe.461183] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 06/25/2022] [Accepted: 07/17/2022] [Indexed: 06/15/2023]
Abstract
Acoustic resolution photoacoustic microscopy (AR-PAM) is a major modality of photoacoustic imaging. It can non-invasively provide high-resolution morphological and functional information about biological tissues. However, the image quality of AR-PAM degrades rapidly when the targets move far away from the focus. Although some works have been conducted to extend the high-resolution imaging depth of AR-PAM, most of them have a small focal point requirement, which is generally not satisfied in a regular AR-PAM system. Therefore, we propose a two-stage deep learning (DL) reconstruction strategy for AR-PAM to recover high-resolution photoacoustic images at different out-of-focus depths adaptively. The residual U-Net with attention gate was developed to implement the image reconstruction. We carried out phantom and in vivo experiments to optimize the proposed DL network and verify the performance of the proposed reconstruction method. Experimental results demonstrated that our approach extends the depth-of-focus of AR-PAM from 1mm to 3mm under the 4 mJ/cm2 light energy used in the imaging system. In addition, the imaging resolution of the region 2 mm far away from the focus can be improved, similar to the in-focus area. The proposed method effectively improves the imaging ability of AR-PAM and thus could be used in various biomedical studies needing deeper depth.
Collapse
Affiliation(s)
- Jing Meng
- School of Computer, Qufu Normal University, Rizhao 276826, China
- These authors contributed equally to this work
| | - Xueting Zhang
- School of Computer, Qufu Normal University, Rizhao 276826, China
- These authors contributed equally to this work
| | - Liangjian Liu
- Institute of Biomedical and Health Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- These authors contributed equally to this work
| | - Silue Zeng
- Institute of Biomedical and Health Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Department of Hepatobiliary Surgery I, Zhujiang Hospital, Southern Medical University, Guangzhou 510280, China
| | - Chihua Fang
- Department of Hepatobiliary Surgery I, Zhujiang Hospital, Southern Medical University, Guangzhou 510280, China
| | - Chengbo Liu
- Institute of Biomedical and Health Engineering, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| |
Collapse
|
42
|
Kye H, Song Y, Ninjbadgar T, Kim C, Kim J. Whole-Body Photoacoustic Imaging Techniques for Preclinical Small Animal Studies. SENSORS (BASEL, SWITZERLAND) 2022; 22:5130. [PMID: 35890810 PMCID: PMC9318812 DOI: 10.3390/s22145130] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/15/2022] [Revised: 07/06/2022] [Accepted: 07/07/2022] [Indexed: 06/15/2023]
Abstract
Photoacoustic imaging is a hybrid imaging technique that has received considerable attention in biomedical studies. In contrast to pure optical imaging techniques, photoacoustic imaging enables the visualization of optical absorption properties at deeper imaging depths. In preclinical small animal studies, photoacoustic imaging is widely used to visualize biodistribution at the molecular level. Monitoring the whole-body distribution of chromophores in small animals is a key method used in preclinical research, including drug-delivery monitoring, treatment assessment, contrast-enhanced tumor imaging, and gastrointestinal tracking. In this review, photoacoustic systems for the whole-body imaging of small animals are explored and summarized. The configurations of the systems vary with the scanning methods and geometries of the ultrasound transducers. The future direction of research is also discussed with regard to achieving a deeper imaging depth and faster imaging speed, which are the main factors that an imaging system should realize to broaden its application in biomedical studies.
Collapse
Affiliation(s)
- Hyunjun Kye
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Korea; (H.K.); (Y.S.); (T.N.)
| | - Yuon Song
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Korea; (H.K.); (Y.S.); (T.N.)
| | - Tsedendamba Ninjbadgar
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Korea; (H.K.); (Y.S.); (T.N.)
| | - Chulhong Kim
- Departments of Convergence IT Engineering, Mechanical Engineering, and Electrical Engineering, School of Interdisciplinary Bioscience and Bioengineering, Medical Device Innovation Center, Pohang University of Science and Technology (POSTECH), Pohang 37673, Korea
| | - Jeesu Kim
- Departments of Cogno-Mechatronics Engineering and Optics & Mechatronics Engineering, Pusan National University, Busan 46241, Korea; (H.K.); (Y.S.); (T.N.)
| |
Collapse
|
43
|
Rajendran P, Pramanik M. High frame rate (∼3 Hz) circular photoacoustic tomography using single-element ultrasound transducer aided with deep learning. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:066005. [PMID: 36452448 PMCID: PMC9209813 DOI: 10.1117/1.jbo.27.6.066005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Accepted: 06/01/2022] [Indexed: 05/29/2023]
Abstract
SIGNIFICANCE In circular scanning photoacoustic tomography (PAT), it takes several minutes to generate an image of acceptable quality, especially with a single-element ultrasound transducer (UST). The imaging speed can be enhanced by faster scanning (with high repetition rate light sources) and using multiple-USTs. However, artifacts arising from the sparse signal acquisition and low signal-to-noise ratio at higher scanning speeds limit the imaging speed. Thus, there is a need to improve the imaging speed of the PAT systems without hampering the quality of the PAT image. AIM To improve the frame rate (or imaging speed) of the PAT system by using deep learning (DL). APPROACH For improving the frame rate (or imaging speed) of the PAT system, we propose a novel U-Net-based DL framework to reconstruct PAT images from fast scanning data. RESULTS The efficiency of the network was evaluated on both single- and multiple-UST-based PAT systems. Both phantom and in vivo imaging demonstrate that the network can improve the imaging frame rate by approximately sixfold in single-UST-based PAT systems and by approximately twofold in multi-UST-based PAT systems. CONCLUSIONS We proposed an innovative method to improve the frame rate (or imaging speed) by using DL and with this method, the fastest frame rate of ∼ 3 Hz imaging is achieved without hampering the quality of the reconstructed image.
Collapse
Affiliation(s)
| | - Manojit Pramanik
- Nanyang Technological University, School of Chemical and Biomedical Engineering, Singapore
| |
Collapse
|
44
|
Gao R, Xue Q, Ren Y, Zhang H, Song L, Liu C. Achieving depth-independent lateral resolution in AR-PAM using the synthetic-aperture focusing technique. PHOTOACOUSTICS 2022; 26:100328. [PMID: 35242539 PMCID: PMC8861412 DOI: 10.1016/j.pacs.2021.100328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Revised: 12/12/2021] [Accepted: 12/23/2021] [Indexed: 05/02/2023]
Abstract
Acoustic-resolution photoacoustic microscopy (AR-PAM) is a promising imaging modality that renders images with ultrasound resolution and extends the imaging depth beyond the optical ballistic regime. To achieve a high lateral resolution, a large numerical aperture (NA) of a focused transducer is usually applied for AR-PAM. However, AR-PAM fails to hold its performance in the out-of-focus region. The lateral resolution and signal-to-noise ratio (SNR) degrade substantially, thereby leading to a significantly deteriorated image quality outside the focal area. Based on the concept of the synthetic-aperture focusing technique (SAFT), various strategies have been developed to address this challenge. These include 1D-SAFT, 2D-SAFT, adaptive-SAFT, spatial impulse response (SIR)-based schemes, and delay-multiply-and-sum (DMAS) strategies. These techniques have shown progress in achieving depth-independent lateral resolution, while several challenges remain. This review aims to introduce these developments in SAFT-based approaches, highlight their fundamental mechanisms, underline the advantages and limitations of each approach, and discuss the outlook of the remaining challenges for future advances.
Collapse
Affiliation(s)
- Rongkang Gao
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Qiang Xue
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- School of Medicine, Southern University of Science and Technology, Shenzhen 518055, China
- Department of Ultrasound, First Affiliated Hospital of Southern University of Science and Technology, The Shenzhen Medical Ultrasound Engineering Center, Shenzhen People's Hospital, Shenzhen 518020, China
| | - Yaguang Ren
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Hai Zhang
- Department of Ultrasound, First Affiliated Hospital of Southern University of Science and Technology, The Shenzhen Medical Ultrasound Engineering Center, Shenzhen People's Hospital, Shenzhen 518020, China
- Department of Ultrasound, The Second Clinical College of Jinan University, Shenzhen People's Hospital, Shenzhen 518020, China
| | - Liang Song
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
| | - Chengbo Liu
- Research Laboratory for Biomedical Optics and Molecular Imaging, CAS Key Laboratory of Health Informatics, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
- Corresponding author.
| |
Collapse
|
45
|
Deep-Learning-Based Algorithm for the Removal of Electromagnetic Interference Noise in Photoacoustic Endoscopic Image Processing. SENSORS 2022; 22:s22103961. [PMID: 35632370 PMCID: PMC9147354 DOI: 10.3390/s22103961] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 05/18/2022] [Accepted: 05/21/2022] [Indexed: 12/10/2022]
Abstract
Despite all the expectations for photoacoustic endoscopy (PAE), there are still several technical issues that must be resolved before the technique can be successfully translated into clinics. Among these, electromagnetic interference (EMI) noise, in addition to the limited signal-to-noise ratio (SNR), have hindered the rapid development of related technologies. Unlike endoscopic ultrasound, in which the SNR can be increased by simply applying a higher pulsing voltage, there is a fundamental limitation in leveraging the SNR of PAE signals because they are mostly determined by the optical pulse energy applied, which must be within the safety limits. Moreover, a typical PAE hardware situation requires a wide separation between the ultrasonic sensor and the amplifier, meaning that it is not easy to build an ideal PAE system that would be unaffected by EMI noise. With the intention of expediting the progress of related research, in this study, we investigated the feasibility of deep-learning-based EMI noise removal involved in PAE image processing. In particular, we selected four fully convolutional neural network architectures, U-Net, Segnet, FCN-16s, and FCN-8s, and observed that a modified U-Net architecture outperformed the other architectures in the EMI noise removal. Classical filter methods were also compared to confirm the superiority of the deep-learning-based approach. Still, it was by the U-Net architecture that we were able to successfully produce a denoised 3D vasculature map that could even depict the mesh-like capillary networks distributed in the wall of a rat colorectum. As the development of a low-cost laser diode or LED-based photoacoustic tomography (PAT) system is now emerging as one of the important topics in PAT, we expect that the presented AI strategy for the removal of EMI noise could be broadly applicable to many areas of PAT, in which the ability to apply a hardware-based prevention method is limited and thus EMI noise appears more prominently due to poor SNR.
Collapse
|
46
|
Althobaiti MM, Ashour AA, Alhindi NA, Althobaiti A, Mansour RF, Gupta D, Khanna A. Deep Transfer Learning-Based Breast Cancer Detection and Classification Model Using Photoacoustic Multimodal Images. BIOMED RESEARCH INTERNATIONAL 2022; 2022:3714422. [PMID: 35572730 PMCID: PMC9098312 DOI: 10.1155/2022/3714422] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/13/2022] [Revised: 03/29/2022] [Accepted: 04/07/2022] [Indexed: 01/28/2023]
Abstract
The rapid development of technologies in biomedical research has enriched and broadened the range of medical equipment. Magnetic resonance imaging, ultrasonic imaging, and optical imaging have been discovered by diverse research communities to design multimodal systems, which is essential for biomedical applications. One of the important tools is photoacoustic multimodal imaging (PAMI) which combines the concepts of optics and ultrasonic systems. At the same time, earlier detection of breast cancer becomes essential to reduce mortality. The recent advancements of deep learning (DL) models enable detection and classification the breast cancer using biomedical images. This article introduces a novel social engineering optimization with deep transfer learning-based breast cancer detection and classification (SEODTL-BDC) model using PAI. The intention of the SEODTL-BDC technique is to detect and categorize the presence of breast cancer using ultrasound images. Primarily, bilateral filtering (BF) is applied as an image preprocessing technique to remove noise. Besides, a lightweight LEDNet model is employed for the segmentation of biomedical images. In addition, residual network (ResNet-18) model can be utilized as a feature extractor. Finally, SEO with recurrent neural network (RNN) model, named SEO-RNN classifier, is applied to allot proper class labels to the biomedical images. The performance validation of the SEODTL-BDC technique is carried out using benchmark dataset and the experimental outcomes pointed out the supremacy of the SEODTL-BDC approach over the existing methods.
Collapse
Affiliation(s)
- Maha M. Althobaiti
- Department of Computer Science, College of Computing and Information Technology, Taif University, P.O.Box 11099, Taif 21944, Saudi Arabia
| | - Amal Adnan Ashour
- Department of Oral & Maxillofacial Surgery and Diagnostic Sciences, Faculty of Dentistry, Taif University, P.O. Box 11099, Taif 21944, Saudi Arabia
| | - Nada A. Alhindi
- Oral Diagnostic Sciences Department, Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Asim Althobaiti
- Regional Laboratory and Blood Bank, Taif Health, Taif, Saudi Arabia
| | - Romany F. Mansour
- Department of Mathematics, Faculty of Science, New Valley University, El-Kharga 72511, Egypt
| | - Deepak Gupta
- Department of Computer Science & Engineering, Maharaja Agrasen Institute of Technology, Delhi, India
| | - Ashish Khanna
- Department of Computer Science & Engineering, Maharaja Agrasen Institute of Technology, Delhi, India
| |
Collapse
|
47
|
Zhang H, Bo W, Wang D, DiSpirito A, Huang C, Nyayapathi N, Zheng E, Vu T, Gong Y, Yao J, Xu W, Xia J. Deep-E: A Fully-Dense Neural Network for Improving the Elevation Resolution in Linear-Array-Based Photoacoustic Tomography. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1279-1288. [PMID: 34928793 PMCID: PMC9161237 DOI: 10.1109/tmi.2021.3137060] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/02/2023]
Abstract
Linear-array-based photoacoustic tomography has shown broad applications in biomedical research and preclinical imaging. However, the elevational resolution of a linear array is fundamentally limited due to the weak cylindrical focus of the transducer element. While several methods have been proposed to address this issue, they have all handled the problem in a less time-efficient way. In this work, we propose to improve the elevational resolution of a linear array through Deep-E, a fully dense neural network based on U-net. Deep-E exhibits high computational efficiency by converting the three-dimensional problem into a two-dimension problem: it focused on training a model to enhance the resolution along elevational direction by only using the 2D slices in the axial and elevational plane and thereby reducing the computational burden in simulation and training. We demonstrated the efficacy of Deep-E using various datasets, including simulation, phantom, and human subject results. We found that Deep-E could improve elevational resolution by at least four times and recover the object's true size. We envision that Deep-E will have a significant impact in linear-array-based photoacoustic imaging studies by providing high-speed and high-resolution image enhancement.
Collapse
|
48
|
Zheng S, Meng Q, Wang XY. Quantitative endoscopic photoacoustic tomography using a convolutional neural network. APPLIED OPTICS 2022; 61:2574-2581. [PMID: 35471325 DOI: 10.1364/ao.441250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/24/2021] [Accepted: 02/23/2022] [Indexed: 06/14/2023]
Abstract
Endoscopic photoacoustic tomography (EPAT) is a catheter-based hybrid imaging modality capable of providing structural and functional information of biological luminal structures, such as coronary arterial vessels and the digestive tract. The recovery of the optical properties of the imaged tissue from acoustic measurements achieved by optical inversion is essential for implementing quantitative EPAT (qEPAT). In this paper, a convolutional neural network (CNN) based on deep gradient descent is developed for qEPAT. The network enables the reconstruction of images representing the spatially varying absorption coefficient in cross-sections of the tubular structures from limited measurement data. The forward operator reflecting the mapping from the absorption coefficient to the optical deposition due to pulsed irradiation is embedded into the network training. The network parameters are optimized layer by layer through the deep gradient descent mechanism using the numerically simulated data. The operation processes of the forward operator and its adjoint operator are separated from the network training. The trained network outputs an image representing the distribution of absorption coefficients by inputting an image that represents the optical deposition. The method has been tested with computer-generated phantoms mimicking coronary arterial vessels containing various tissue types. Results suggest that the structural similarity of the images reconstructed by our method is increased by about 10% in comparison with the non-learning method based on error minimization in the case of the same measuring view.
Collapse
|
49
|
Gröhl J, Dreher KK, Schellenberg M, Rix T, Holzwarth N, Vieten P, Ayala L, Bohndiek SE, Seitel A, Maier-Hein L. SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-210395SSR. [PMID: 35380031 PMCID: PMC8978263 DOI: 10.1117/1.jbo.27.8.083010] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 02/28/2022] [Indexed: 05/09/2023]
Abstract
SIGNIFICANCE Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings. AIM To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards. APPROACH SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models. RESULTS To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations. CONCLUSIONS SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.
Collapse
Affiliation(s)
- Janek Gröhl
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Kris K. Dreher
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Melanie Schellenberg
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg, Germany
| | - Tom Rix
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
| | - Niklas Holzwarth
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Patricia Vieten
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Leonardo Ayala
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| | - Sarah E. Bohndiek
- University of Cambridge, Cancer Research UK Cambridge Institute, Robinson Way, Cambridge, United Kingdom
- University of Cambridge, Department of Physics, Cambridge, United Kingdom
| | - Alexander Seitel
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Lena Maier-Hein
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| |
Collapse
|
50
|
Lan H, Gong J, Gao F. Deep learning adapted acceleration for limited-view photoacoustic image reconstruction. OPTICS LETTERS 2022; 47:1911-1914. [PMID: 35363767 DOI: 10.1364/ol.450860] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 03/07/2022] [Indexed: 06/14/2023]
Abstract
The limited-view issue can cause a low-quality image in photoacoustic (PA) computed tomography due to the limitation of geometric condition. The model-based method is used to resolve this problem, which contains different regularization. To adapt fast and high-quality reconstruction of limited-view PA data, in this Letter, a model-based method that combines the mathematical variational model with deep learning is proposed to speed up and regularize the unrolled procedure of reconstruction. A deep neural network is designed to adapt the step of the gradient updated term of data consistency in the gradient descent procedure, which can obtain a high-quality PA image with only a few iterations. A comparison of different model-based methods shows that our proposed scheme has superior performances (over 0.05 for SSIM) with the same iteration (three times) steps. Finally, we find that our method obtains superior results (0.94 value of SSIM for in vivo) with a high robustness and accelerated reconstruction.
Collapse
|