1
|
Kim M, Pelivanov I, O'Donnell M. Review of Deep Learning Approaches for Interleaved Photoacoustic and Ultrasound (PAUS) Imaging. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:1591-1606. [PMID: 37910419 PMCID: PMC10788151 DOI: 10.1109/tuffc.2023.3329119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/03/2023]
Abstract
Photoacoustic (PA) imaging provides optical contrast at relatively large depths within the human body, compared to other optical methods, at ultrasound (US) spatial resolution. By integrating real-time PA and US (PAUS) modalities, PAUS imaging has the potential to become a routine clinical modality bringing the molecular sensitivity of optics to medical US imaging. For applications where the full capabilities of clinical US scanners must be maintained in PAUS, conventional limited view and bandwidth transducers must be used. This approach, however, cannot provide high-quality maps of PA sources, especially vascular structures. Deep learning (DL) using data-driven modeling with minimal human design has been very effective in medical imaging, medical data analysis, and disease diagnosis, and has the potential to overcome many of the technical limitations of current PAUS imaging systems. The primary purpose of this article is to summarize the background and current status of DL applications in PAUS imaging. It also looks beyond current approaches to identify remaining challenges and opportunities for robust translation of PAUS technologies to the clinic.
Collapse
|
2
|
Xu M, Chen Z, Zheng J, Zhao Q, Yuan Z. Artificial Intelligence-Aided Optical Imaging for Cancer Theranostics. Semin Cancer Biol 2023:S1044-579X(23)00094-9. [PMID: 37302519 DOI: 10.1016/j.semcancer.2023.06.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2022] [Revised: 06/08/2023] [Accepted: 06/08/2023] [Indexed: 06/13/2023]
Abstract
The use of artificial intelligence (AI) to assist biomedical imaging have demonstrated its high accuracy and high efficiency in medical decision-making for individualized cancer medicine. In particular, optical imaging methods are able to visualize both the structural and functional information of tumors tissues with high contrast, low cost, and noninvasive property. However, no systematic work has been performed to inspect the recent advances on AI-aided optical imaging for cancer theranostics. In this review, we demonstrated how AI can guide optical imaging methods to improve the accuracy on tumor detection, automated analysis and prediction of its histopathological section, its monitoring during treatment, and its prognosis by using computer vision, deep learning and natural language processing. By contrast, the optical imaging techniques involved mainly consisted of various tomography and microscopy imaging methods such as optical endoscopy imaging, optical coherence tomography, photoacoustic imaging, diffuse optical tomography, optical microscopy imaging, Raman imaging, and fluorescent imaging. Meanwhile, existing problems, possible challenges and future prospects for AI-aided optical imaging protocol for cancer theranostics were also discussed. It is expected that the present work can open a new avenue for precision oncology by using AI and optical imaging tools.
Collapse
Affiliation(s)
- Mengze Xu
- Center for Cognition and Neuroergonomics, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Zhuhai, China; Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China
| | - Zhiyi Chen
- Institute of Medical Imaging, Hengyang Medical School, University of South China, Hengyang, China
| | - Junxiao Zheng
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China
| | - Qi Zhao
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China
| | - Zhen Yuan
- Cancer Center, Faculty of Health Sciences, University of Macau, Macau SAR, China; Centre for Cognitive and Brain Sciences, University of Macau, Macau SAR, China.
| |
Collapse
|
3
|
Salih AK, ALWAN AH, Opulencia MJC, Uinarni H, Khamidova FM, Atiyah MS, Awadh SA, Hammid AT, Arzehgar Z. Evaluation of Cholesterol Thickness of Blood Vessels Using Photoacoustic Technology. BIOMED RESEARCH INTERNATIONAL 2023; 2023:2721427. [PMID: 37090193 PMCID: PMC10115531 DOI: 10.1155/2023/2721427] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/17/2022] [Revised: 06/05/2022] [Accepted: 06/24/2022] [Indexed: 04/25/2023]
Abstract
One of the primary indicators of plaque vulnerability is the lipid composition of atherosclerotic plaques. Therefore, the medical industry requires a method to evaluate necrotic nuclei in atherosclerosis imaging with sensitivity. In this regard, photoacoustic imaging is a plaque detection method that provides chemical information on lipids and cholesterol thickness in the arterial walls of the patient. This aspect aims to increase the low-frequency axial resolution by developing a new photoacoustic-based system. A photoacoustic system has been developed to detect the cholesterol thickness of the blood vessels to observe the progression of plaque in the heart's blood vessels. The application of the coherent photoacoustic discontinuous correlation tomography technique, which is based on a novel signal processing, significantly increased the cholesterol oleate's sensitivity to plaque necrosis. By enhancing the quality of thickness detection, the system for measuring the thickness of cholesterol in blood vessels has been reduced to approximately 23 microns. The results show that the phase spectrum peaked at 100 Hz at 58.66 degrees, and at 400 Hz, the phase spectrum was 46.37 degrees. The minimum amplitude is 1.95 at 100 Hz and 17.67 at 400 Hz. In conclusion, it can be stated that photoacoustic imaging as a method based on new technologies is of great importance in medical research, which is based on the use of nonionizing radiation to perform diagnostic processes and measure different types of body tissues.
Collapse
Affiliation(s)
| | - Ala Hadi ALWAN
- Ibn Al-Bitar Specialized Center for Cardiac Surgery, Baghdad, Iraq
| | | | - Herlina Uinarni
- Atma Jaya Catholic University of Indonesia, Jakarta, Indonesia
- Pantai Indah Kapuk Hospital, North Jakarta, Indonesia
| | - Firuza M. Khamidova
- Department of Ophthalmology, Samarkand State Medical Institute, Samarkand, Uzbekistan
- Tashkent State Dental Institute, Tashkent, Uzbekistan
| | | | | | | | - Zeinab Arzehgar
- Department of Chemistry, Payame Noor University, Tehran, Iran
| |
Collapse
|
4
|
Wang R, Zhu J, Xia J, Yao J, Shi J, Li C. Photoacoustic imaging with limited sampling: a review of machine learning approaches. BIOMEDICAL OPTICS EXPRESS 2023; 14:1777-1799. [PMID: 37078052 PMCID: PMC10110324 DOI: 10.1364/boe.483081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 03/03/2023] [Accepted: 03/17/2023] [Indexed: 05/03/2023]
Abstract
Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.
Collapse
Affiliation(s)
- Ruofan Wang
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jing Zhu
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jun Xia
- Department of Biomedical Engineering, University at Buffalo, The State University of New York, Buffalo, NY 14260, USA
| | - Junjie Yao
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Junhui Shi
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Chiye Li
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| |
Collapse
|
5
|
Gholampour A, Muller JW, Cano C, van Sambeek MRHM, Lopata R, Schwab HM, Wu M. Multiperspective Photoacoustic Imaging Using Spatially Diverse CMUTs. IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS, AND FREQUENCY CONTROL 2023; 70:16-24. [PMID: 36350862 DOI: 10.1109/tuffc.2022.3220999] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Photoacoustic imaging (PAI) is a promising technique to assess different constituents in tissue. In PAI, the propagating waves are low-amplitude, isotropic, and broadband. A common approach in PAI is the use of a single linear or curved piezoelectric transducer array to perform both PA and ultrasound imaging. These systems provide freedom, agility, and versatility for performing imaging, but have limited field of view (FOV) and directivity that degrade the final image quality. Capacitive micromachined ultrasonic transducers (CMUTs) have a great potential to be used for PAI since they provide larger bandwidth and better cost efficiency. In this study, to improve the FOV, resolution, and contrast, we propose a multiperspective PAI (MP-PAI) approach using multiple CMUTs on a flexible array with shared channels. The designed array was used to perform MP-PAI in an in vitro experiment using a plaque mimicking phantom where the images were compounded both incoherently and coherently. The MP-PAI approach showed a significant improvement in overall image quality. Using only three CMUTs led to about 20% increase in generalized-contrast-to-noise ratio (gCNR), 2-dB improvement in peak signal-to-noise ratio (PSNR), and double the structural coverage in comparison to a single CMUT setup. In numerical studies, the MP-PAI was thoroughly evaluated for both the coherent and incoherent compounding methods. The assessments showed that the image quality further improved for increased number of transducers and angular coverage. For 15 transducers, the improvement for resolution and contrast could be up to three times the amount in a single-perspective image. Nonetheless, the most prominent improvement of MP-PAI was its ability to resolve the structural information of the phantoms.
Collapse
|
6
|
Barbosa RCS, Mendes PM. A Comprehensive Review on Photoacoustic-Based Devices for Biomedical Applications. SENSORS (BASEL, SWITZERLAND) 2022; 22:9541. [PMID: 36502258 PMCID: PMC9736954 DOI: 10.3390/s22239541] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Revised: 12/02/2022] [Accepted: 12/03/2022] [Indexed: 06/17/2023]
Abstract
The photoacoustic effect is an emerging technology that has sparked significant interest in the research field since an acoustic wave can be produced simply by the incidence of light on a material or tissue. This phenomenon has been extensively investigated, not only to perform photoacoustic imaging but also to develop highly miniaturized ultrasound probes that can provide biologically meaningful information. Therefore, this review aims to outline the materials and their fabrication process that can be employed as photoacoustic targets, both biological and non-biological, and report the main components' features to achieve a certain performance. When designing a device, it is of utmost importance to model it at an early stage for a deeper understanding and to ease the optimization process. As such, throughout this article, the different methods already implemented to model the photoacoustic effect are introduced, as well as the advantages and drawbacks inherent in each approach. However, some remaining challenges are still faced when developing such a system regarding its fabrication, modeling, and characterization, which are also discussed.
Collapse
|
7
|
Biswas D, Roy S, Vasudevan S. Biomedical Application of Photoacoustics: A Plethora of Opportunities. MICROMACHINES 2022; 13:1900. [PMID: 36363921 PMCID: PMC9692656 DOI: 10.3390/mi13111900] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 10/19/2022] [Accepted: 10/29/2022] [Indexed: 06/16/2023]
Abstract
The photoacoustic (PA) technique is a non-invasive, non-ionizing hybrid technique that exploits laser irradiation for sample excitation and acquires an ultrasound signal generated due to thermoelastic expansion of the sample. Being a hybrid technique, PA possesses the inherent advantages of conventional optical (high resolution) and ultrasonic (high depth of penetration in biological tissue) techniques and eliminates some of the major limitations of these conventional techniques. Hence, PA has been employed for different biomedical applications. In this review, we first discuss the basic physics of PA. Then, we discuss different aspects of PA techniques, which includes PA imaging and also PA frequency spectral analysis. The theory of PA signal generation, detection and analysis is also detailed in this work. Later, we also discuss the major biomedical application area of PA technique.
Collapse
Affiliation(s)
- Deblina Biswas
- School of Bioengineering and Food Technology, Shoolini University, Solan 173229, HP, India
| | - Swarup Roy
- School of Bioengineering and Food Technology, Shoolini University, Solan 173229, HP, India
| | - Srivathsan Vasudevan
- Discipline of Electrical Engineering, Indian Institute of Technology Indore, Khandwa Road, Simrol 453552, MP, India
| |
Collapse
|
8
|
Wen Y, Guo D, Zhang J, Liu X, Liu T, Li L, Jiang S, Wu D, Jiang H. Clinical photoacoustic/ultrasound dual-modal imaging: Current status and future trends. Front Physiol 2022; 13:1036621. [PMID: 36388111 PMCID: PMC9651137 DOI: 10.3389/fphys.2022.1036621] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Accepted: 10/05/2022] [Indexed: 08/24/2023] Open
Abstract
Photoacoustic tomography (PAT) is an emerging biomedical imaging modality that combines optical and ultrasonic imaging, providing overlapping fields of view. This hybrid approach allows for a natural integration of PAT and ultrasound (US) imaging in a single platform. Due to the similarities in signal acquisition and processing, the combination of PAT and US imaging creates a new hybrid imaging for novel clinical applications. Over the recent years, particular attention is paid to the development of PAT/US dual-modal systems highlighting mutual benefits in clinical cases, with an aim of substantially improving the specificity and sensitivity for diagnosis of diseases. The demonstrated feasibility and accuracy in these efforts open an avenue of translating PAT/US imaging to practical clinical applications. In this review, the current PAT/US dual-modal imaging systems are discussed in detail, and their promising clinical applications are presented and compared systematically. Finally, this review describes the potential impacts of these combined systems in the coming future.
Collapse
Affiliation(s)
- Yanting Wen
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
- School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Dan Guo
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
| | - Jing Zhang
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
- School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Xiaotian Liu
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
| | - Ting Liu
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
| | - Lu Li
- Department of Ultrasound Imaging, The Fifth People’s Hospital of Chengdu, Chengdu, China
| | - Shixie Jiang
- Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, United States
| | - Dan Wu
- School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
| | - Huabei Jiang
- Department of Medical Engineering, University of South Florida, Tampa, FL, United States
| |
Collapse
|
9
|
Mirg S, Turner KL, Chen H, Drew PJ, Kothapalli SR. Photoacoustic imaging for microcirculation. Microcirculation 2022; 29:e12776. [PMID: 35793421 PMCID: PMC9870710 DOI: 10.1111/micc.12776] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2021] [Revised: 06/13/2022] [Accepted: 06/28/2022] [Indexed: 01/26/2023]
Abstract
Microcirculation facilitates the blood-tissue exchange of nutrients and regulates blood perfusion. It is, therefore, essential in maintaining tissue health. Aberrations in microcirculation are potentially indicative of underlying cardiovascular and metabolic pathologies. Thus, quantitative information about it is of great clinical relevance. Photoacoustic imaging (PAI) is a capable technique that relies on the generation of imaging contrast via the absorption of light and can image at micron-scale resolution. PAI is especially desirable to map microvasculature as hemoglobin strongly absorbs light and can generate a photoacoustic signal. This paper reviews the current state of the art for imaging microvascular networks using photoacoustic imaging. We further describe how quantitative information about blood dynamics such as the total hemoglobin concentration, oxygen saturation, and blood flow rate is obtained using PAI. We also discuss its importance in understanding key pathophysiological processes in neurovascular, cardiovascular, ophthalmic, and cancer research fields. We then discuss the current challenges and limitations of PAI and the approaches that can help overcome these limitations. Finally, we provide the reader with an overview of future trends in the field of PAI for imaging microcirculation.
Collapse
Affiliation(s)
- Shubham Mirg
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Kevin L. Turner
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Haoyang Chen
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Patrick J. Drew
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Department of Engineering Science and Mechanics, Pennsylvania State University, University Park, PA 16802, USA,Department of Neurosurgery, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA
| | - Sri-Rajasekhar Kothapalli
- Department of Biomedical Engineering, Pennsylvania State University, University Park, PA 16802, USA,Penn State Cancer Institute, Pennsylvania State University, Hershey, PA 17033, USA,Graduate Program in Acoustics, Pennsylvania State University, University Park, PA 16802, USA,Center for Neural Engineering, Pennsylvania State University, University Park, PA 16802, USA,Corresponding author: Sri-Rajasekhar Kothapalli, 325 CBE Building, State College, PA, 16802, USA,
| |
Collapse
|
10
|
Muller JW, Arabul MÜ, Schwab HM, Rutten MCM, van Sambeek MRHM, Wu M, Lopata RGP. Modeling toolchain for realistic simulation of photoacoustic data acquisition. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:096005. [PMID: 36104838 PMCID: PMC9470848 DOI: 10.1117/1.jbo.27.9.096005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/07/2022] [Accepted: 08/31/2022] [Indexed: 06/15/2023]
Abstract
SIGNIFICANCE Physics-based simulations of photoacoustic (PA) signals are used to validate new methods, to characterize PA setups and to generate training datasets for machine learning. However, a thoroughly validated PA simulation toolchain that can simulate realistic images is still lacking. AIM A quantitative toolchain was developed to model PA image acquisition in complex tissues, by simulating both the optical fluence and the acoustic wave propagation. APPROACH Sampling techniques were developed to decrease artifacts in acoustic simulations. The performance of the simulations was analyzed by measuring the point spread function (PSF) and using a rotatable three-channel phantom, filled with cholesterol, a human carotid plaque sample, and porcine blood. Ex vivo human plaque samples were simulated to validate the methods in more complex tissues. RESULTS The sampling techniques could enhance the quality of the simulated PA images effectively. The resolution and intensity of the PSF in the turbid medium matched the experimental data well. Overall, the appearance, signal-to-noise ratio and speckle of the images could be simulated accurately. CONCLUSIONS A PA toolchain was developed and validated, and the results indicate a great potential of PA simulations in more complex and heterogeneous media.
Collapse
Affiliation(s)
- Jan-Willem Muller
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
- Catharina Hospital, Department of Vascular Surgery, Eindhoven, The Netherlands
| | - Mustafa Ü. Arabul
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Hans-Martin Schwab
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Marcel C. M. Rutten
- Cardiovascular Biomechanics Group, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Marc R. H. M. van Sambeek
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
- Catharina Hospital, Department of Vascular Surgery, Eindhoven, The Netherlands
| | - Min Wu
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| | - Richard G. P. Lopata
- Eindhoven University of Technology, Photoacoustics and Ultrasound Laboratory Eindhoven, Department of Biomedical Engineering, Eindhoven, The Netherlands
| |
Collapse
|
11
|
Zhang Y, Wang L. Video-rate full-ring ultrasound and photoacoustic computed tomography with real-time sound speed optimization. BIOMEDICAL OPTICS EXPRESS 2022; 13:4398-4413. [PMID: 36032563 PMCID: PMC9408242 DOI: 10.1364/boe.464360] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 07/13/2022] [Accepted: 07/17/2022] [Indexed: 06/15/2023]
Abstract
Full-ring dual-modal ultrasound and photoacoustic imaging provide complementary contrasts, high spatial resolution, full view angle and are more desirable in pre-clinical and clinical applications. However, two long-standing challenges exist in achieving high-quality video-rate dual-modal imaging. One is the increased data processing burden from the dense acquisition. Another one is the object-dependent speed of sound variation, which may cause blurry, splitting artifacts, and low imaging contrast. Here, we develop a video-rate full-ring ultrasound and photoacoustic computed tomography (VF-USPACT) with real-time optimization of the speed of sound. We improve the imaging speed by selective and parallel image reconstruction. We determine the optimal sound speed via co-registered ultrasound imaging. Equipped with a 256-channel ultrasound array, the dual-modal system can optimize the sound speed and reconstruct dual-modal images at 10 Hz in real-time. The optimized sound speed can effectively enhance the imaging quality under various sample sizes, types, or physiological states. In animal and human imaging, the system shows co-registered dual contrasts, high spatial resolution (140 µm), single-pulse photoacoustic imaging (< 50 µs), deep penetration (> 20 mm), full view, and adaptive sound speed correction. We believe VF-USPACT can advance many real-time biomedical imaging applications, such as vascular disease diagnosing, cancer screening, or neuroimaging.
Collapse
Affiliation(s)
- Yachao Zhang
- Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Lidai Wang
- Department of Biomedical Engineering, City University of Hong Kong, Hong Kong, 999077, China
- City University of Hong Kong Shenzhen Research Institute, Shen Zhen, 518057, China
| |
Collapse
|
12
|
Gröhl J, Dreher KK, Schellenberg M, Rix T, Holzwarth N, Vieten P, Ayala L, Bohndiek SE, Seitel A, Maier-Hein L. SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics. JOURNAL OF BIOMEDICAL OPTICS 2022; 27:JBO-210395SSR. [PMID: 35380031 PMCID: PMC8978263 DOI: 10.1117/1.jbo.27.8.083010] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Accepted: 02/28/2022] [Indexed: 05/09/2023]
Abstract
SIGNIFICANCE Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings. AIM To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards. APPROACH SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models. RESULTS To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations. CONCLUSIONS SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.
Collapse
Affiliation(s)
- Janek Gröhl
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Kris K. Dreher
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Melanie Schellenberg
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
- HIDSS4Health - Helmholtz Information and Data Science School for Health, Heidelberg, Germany
| | - Tom Rix
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
| | - Niklas Holzwarth
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Patricia Vieten
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Physics and Astronomy, Heidelberg, Germany
| | - Leonardo Ayala
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| | - Sarah E. Bohndiek
- University of Cambridge, Cancer Research UK Cambridge Institute, Robinson Way, Cambridge, United Kingdom
- University of Cambridge, Department of Physics, Cambridge, United Kingdom
| | - Alexander Seitel
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Lena Maier-Hein
- German Cancer Research Center (DKFZ), Division of Intelligent Medical Systems, Heidelberg, Germany
- Heidelberg University, Faculty of Mathematics and Computer Science, Heidelberg, Germany
- Heidelberg University, Medical Faculty, Heidelberg, Germany
| |
Collapse
|
13
|
Yazdani A, Agrawal S, Johnstonbaugh K, Kothapalli SR, Monga V. Simultaneous Denoising and Localization Network for Photoacoustic Target Localization. IEEE TRANSACTIONS ON MEDICAL IMAGING 2021; 40:2367-2379. [PMID: 33939612 PMCID: PMC8526152 DOI: 10.1109/tmi.2021.3077187] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
A significant research problem of recent interest is the localization of targets like vessels, surgical needles, and tumors in photoacoustic (PA) images.To achieve accurate localization, a high photoacoustic signal-to-noise ratio (SNR) is required. However, this is not guaranteed for deep targets, as optical scattering causes an exponential decay in optical fluence with respect to tissue depth. To address this, we develop a novel deep learning method designed to explicitly exhibit robustness to noise present in photoacoustic radio-frequency (RF) data. More precisely, we describe and evaluate a deep neural network architecture consisting of a shared encoder and two parallel decoders. One decoder extracts the target coordinates from the input RF data while the other boosts the SNR and estimates clean RF data. The joint optimization of the shared encoder and dual decoders lends significant noise robustness to the features extracted by the encoder, which in turn enables the network to contain detailed information about deep targets that may be obscured by noise. Additional custom layers and newly proposed regularizers in the training loss function (designed based on observed RF data signal and noise behavior) serve to increase the SNR in the cleaned RF output and improve model performance. To account for depth-dependent strong optical scattering, our network was trained with simulated photoacoustic datasets of targets embedded at different depths inside tissue media of different scattering levels. The network trained on this novel dataset accurately locates targets in experimental PA data that is clinically relevant with respect to the localization of vessels, needles, or brachytherapy seeds. We verify the merits of the proposed architecture by outperforming the state of the art on both simulated and experimental datasets.
Collapse
|