1
|
Lan H, Huang L, Wei X, Li Z, Lv J, Ma C, Nie L, Luo J. Masked cross-domain self-supervised deep learning framework for photoacoustic computed tomography reconstruction. Neural Netw 2024; 179:106515. [PMID: 39032393 DOI: 10.1016/j.neunet.2024.106515] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2023] [Revised: 06/24/2024] [Accepted: 07/05/2024] [Indexed: 07/23/2024]
Abstract
Accurate image reconstruction is crucial for photoacoustic (PA) computed tomography (PACT). Recently, deep learning has been used to reconstruct PA images with a supervised scheme, which requires high-quality images as ground truth labels. However, practical implementations encounter inevitable trade-offs between cost and performance due to the expensive nature of employing additional channels for accessing more measurements. Here, we propose a masked cross-domain self-supervised (CDSS) reconstruction strategy to overcome the lack of ground truth labels from limited PA measurements. We implement the self-supervised reconstruction in a model-based form. Simultaneously, we take advantage of self-supervision to enforce the consistency of measurements and images across three partitions of the measured PA data, achieved by randomly masking different channels. Our findings indicate that dynamically masking a substantial proportion of channels, such as 80%, yields meaningful self-supervisors in both the image and signal domains. Consequently, this approach reduces the multiplicity of pseudo solutions and enables efficient image reconstruction using fewer PA measurements, ultimately minimizing reconstruction error. Experimental results on in-vivo PACT dataset of mice demonstrate the potential of our self-supervised framework. Moreover, our method exhibits impressive performance, achieving a structural similarity index (SSIM) of 0.87 in an extreme sparse case utilizing only 13 channels, which outperforms the performance of the supervised scheme with 16 channels (0.77 SSIM). Adding to its advantages, our method can be deployed on different trainable models in an end-to-end manner, further enhancing its versatility and applicability.
Collapse
Affiliation(s)
- Hengrong Lan
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Lijie Huang
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Xingyue Wei
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Zhiqiang Li
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China
| | - Jing Lv
- Medical Research Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Southern Medical University, Guangzhou 510080, China
| | - Cheng Ma
- Department of Electronic Engineering, Tsinghua University, Beijing 100084, China
| | - Liming Nie
- Medical Research Institute, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Southern Medical University, Guangzhou 510080, China
| | - Jianwen Luo
- School of Biomedical Engineering, Tsinghua University, Beijing 100084, China.
| |
Collapse
|
2
|
Jiang D, Zhu L, Tong S, Shen Y, Gao F, Gao F. Photoacoustic imaging plus X: a review. JOURNAL OF BIOMEDICAL OPTICS 2024; 29:S11513. [PMID: 38156064 PMCID: PMC10753847 DOI: 10.1117/1.jbo.29.s1.s11513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 11/14/2023] [Accepted: 12/11/2023] [Indexed: 12/30/2023]
Abstract
Significance Photoacoustic (PA) imaging (PAI) represents an emerging modality within the realm of biomedical imaging technology. It seamlessly blends the wealth of optical contrast with the remarkable depth of penetration offered by ultrasound. These distinctive features of PAI hold tremendous potential for various applications, including early cancer detection, functional imaging, hybrid imaging, monitoring ablation therapy, and providing guidance during surgical procedures. The synergy between PAI and other cutting-edge technologies not only enhances its capabilities but also propels it toward broader clinical applicability. Aim The integration of PAI with advanced technology for PA signal detection, signal processing, image reconstruction, hybrid imaging, and clinical applications has significantly bolstered the capabilities of PAI. This review endeavor contributes to a deeper comprehension of how the synergy between PAI and other advanced technologies can lead to improved applications. Approach An examination of the evolving research frontiers in PAI, integrated with other advanced technologies, reveals six key categories named "PAI plus X." These categories encompass a range of topics, including but not limited to PAI plus treatment, PAI plus circuits design, PAI plus accurate positioning system, PAI plus fast scanning systems, PAI plus ultrasound sensors, PAI plus advanced laser sources, PAI plus deep learning, and PAI plus other imaging modalities. Results After conducting a comprehensive review of the existing literature and research on PAI integrated with other technologies, various proposals have emerged to advance the development of PAI plus X. These proposals aim to enhance system hardware, improve imaging quality, and address clinical challenges effectively. Conclusions The progression of innovative and sophisticated approaches within each category of PAI plus X is positioned to drive significant advancements in both the development of PAI technology and its clinical applications. Furthermore, PAI not only has the potential to integrate with the above-mentioned technologies but also to broaden its applications even further.
Collapse
Affiliation(s)
- Daohuai Jiang
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
- Fujian Normal University, College of Photonic and Electronic Engineering, Fuzhou, China
| | - Luyao Zhu
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Shangqing Tong
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Yuting Shen
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Feng Gao
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
| | - Fei Gao
- ShanghaiTech University, School of Information Science and Technology, Shanghai, China
- Shanghai Engineering Research Center of Energy Efficient and Custom AI IC, Shanghai, China
- Shanghai Clinical Research and Trial Center, Shanghai, China
| |
Collapse
|
3
|
Yan B, Song B, Mu G, Fan Y, Zhao Y. Compressed single-shot 3D photoacoustic imaging with a single-element transducer. PHOTOACOUSTICS 2023; 34:100570. [PMID: 38027529 PMCID: PMC10661598 DOI: 10.1016/j.pacs.2023.100570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 10/14/2023] [Accepted: 11/06/2023] [Indexed: 12/01/2023]
Abstract
Three-dimensional (3D) photoacoustic imaging (PAI) can provide rich information content and has gained increasingly more attention in various biomedical applications. However, current 3D PAI methods either involves pointwise scanning of the 3D volume using a single-element transducer, which can be time-consuming, or requires an array of transducers, which is known to be complex and expensive. By utilizing a 3D encoder and compressed sensing techniques, we develop a new imaging modality that is capable of single-shot 3D PAI using a single-element transducer. The proposed method is validated with phantom study, which demonstrates single-shot 3D imaging of different objects and 3D tracking of a moving object. After one-time calibration, while the system could perform single-shot 3D imaging for different objects, the calibration could remain effective over 7 days, which is highly beneficial for practical translation. Overall, the experimental results showcase the potential of this technique for both scientific research and clinical applications.
Collapse
Affiliation(s)
- Bingbao Yan
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Bowen Song
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Gen Mu
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Yubo Fan
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| | - Yanyu Zhao
- Beijing Advanced Innovation Center for Biomedical Engineering, Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, School of Engineering Medicine, Beihang University, Beijing 100191, China
| |
Collapse
|
4
|
Khan S, Vasudevan S. Biomedical instrumentation of photoacoustic imaging and quantitative sensing for clinical applications. THE REVIEW OF SCIENTIFIC INSTRUMENTS 2023; 94:091502. [PMID: 37747328 DOI: 10.1063/5.0151882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2023] [Accepted: 09/02/2023] [Indexed: 09/26/2023]
Abstract
Photoacoustic (PA) imaging has been well researched over the last couple of decades and has found many applications in biomedical engineering. This has evinced interest among many scientists in developing this as a compact instrument for biomedical diagnosis. This review discusses various instrumentation developments for PA experimental setups and their applications in the biomedical diagnostic field. It also covers the PA spectral response or PA sensing technique, which uses the spectral information of the PA signal and performs sensing to deliver a fast, cost-effective, and compact screening tool instead of imaging. Primarily, this review provides an overview of PA imaging concepts and the development of hardware instrumentation systems in both the excitation and acquisition stages of this technique. Later, the paper discusses PA sensing, the quantitative spectral parameter extraction from the PA spectrum, and the correlation study of the spectral parameters with the physical parameters of the tissue. This PA sensing technique was used to diagnose various diseases, such as thyroid nodules, breast cancer, renal disorders, and zoonotic diseases, based on the mechanical and biological characteristics of the tissues. The paper culminates with a discussion section that provides future developments that are necessary to take this technique into clinical applications as a quantitative PA imaging technique.
Collapse
Affiliation(s)
- S Khan
- Department of Electrical Engineering, Indian Institute of Technology, Indore 453552, India
| | - S Vasudevan
- Department of Electrical Engineering, Indian Institute of Technology, Indore 453552, India
| |
Collapse
|
5
|
Wang R, Zhu J, Xia J, Yao J, Shi J, Li C. Photoacoustic imaging with limited sampling: a review of machine learning approaches. BIOMEDICAL OPTICS EXPRESS 2023; 14:1777-1799. [PMID: 37078052 PMCID: PMC10110324 DOI: 10.1364/boe.483081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/09/2022] [Revised: 03/03/2023] [Accepted: 03/17/2023] [Indexed: 05/03/2023]
Abstract
Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.
Collapse
Affiliation(s)
- Ruofan Wang
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jing Zhu
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Jun Xia
- Department of Biomedical Engineering, University at Buffalo, The State University of New York, Buffalo, NY 14260, USA
| | - Junjie Yao
- Department of Biomedical Engineering, Duke University, Durham, NC 27708, USA
| | - Junhui Shi
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| | - Chiye Li
- Research Center for Humanoid Sensing, Zhejiang Lab, Hangzhou, 311100, China
| |
Collapse
|
6
|
Zhao Y, Wang LV. Single-shot photoacoustic imaging with single-element transducer through a spatiotemporal encoder. JOURNAL OF BIOMEDICAL OPTICS 2023; 28:046004. [PMID: 37065647 PMCID: PMC10098145 DOI: 10.1117/1.jbo.28.4.046004] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 03/21/2023] [Indexed: 05/18/2023]
Abstract
Significance Current photoacoustic (PA) imaging modalities typically require either serial detection with a single-element transducer or parallel detections with an ultrasonic array, indicating a dilemma between system cost and imaging throughput. PA topography through ergodic relay (PATER) was recently developed to address this bottleneck. However, PATER requires object-specific calibration due to varied boundary condition and must be recalibrated through pointwise scanning for each object before measurements, which is time-consuming and severely limits practical application. Aim We aim to develop a new single-shot PA imaging technique that only requires a one-time calibration for imaging different objects using a single-element transducer. Approach We develop an imaging method, PA imaging through a spatiotemporal encoder (PAISE), to address the above issue. The spatial information is effectively coded into unique temporal features by the spatiotemporal encoder, which allows for compressive image reconstruction. An ultrasonic waveguide is proposed as a critical element to guide the PA waves from the object into the prism, which effectively accounts for the varied boundary condition of different objects. We further add irregular-shaped edges on the prism to introduce randomized internal reflections and further facilitate the scrambling of acoustic waves. Results The proposed technique is validated through comprehensive numerical simulations and experiments, and it is demonstrated that PAISE can successfully overcome the changed boundary condition and can image different samples given a single calibration. Conclusions The proposed PAISE technique is capable of single-shot widefield PA imaging with a single-element transducer and does not require sample-specific calibration, which successfully overcomes the major limitation of previous PATER technology.
Collapse
Affiliation(s)
- Yanyu Zhao
- California Institute of Technology, Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Pasadena, California, United States
| | - Lihong V. Wang
- California Institute of Technology, Caltech Optical Imaging Laboratory, Andrew and Peggy Cherng Department of Medical Engineering, Pasadena, California, United States
- Address all correspondence to Lihong V. Wang,
| |
Collapse
|
7
|
Fang Z, Gao F, Jin H, Liu S, Wang W, Zhang R, Zheng Z, Xiao X, Tang K, Lou L, Tang KT, Chen J, Zheng Y. A Review of Emerging Electromagnetic-Acoustic Sensing Techniques for Healthcare Monitoring. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2022; 16:1075-1094. [PMID: 36459601 DOI: 10.1109/tbcas.2022.3226290] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
Conventional electromagnetic (EM) sensing techniques such as radar and LiDAR are widely used for remote sensing, vehicle applications, weather monitoring, and clinical monitoring. Acoustic techniques such as sonar and ultrasound sensors are also used for consumer applications, such as ranging and in vivo medical/healthcare applications. It has been of long-term interest to doctors and clinical practitioners to realize continuous healthcare monitoring in hospitals and/or homes. Physiological and biopotential signals in real-time serve as important health indicators to predict and prevent serious illness. Emerging electromagnetic-acoustic (EMA) sensing techniques synergistically combine the merits of EM sensing with acoustic imaging to achieve comprehensive detection of physiological and biopotential signals. Further, EMA enables complementary fusion sensing for challenging healthcare settings, such as real-world long-term monitoring of treatment effects at home or in remote environments. This article reviews various examples of EMA sensing instruments, including implementation, performance, and application from the perspectives of circuits to systems. The novel and significant applications to healthcare are discussed. Three types of EMA sensors are presented: (1) Chip-based radar sensors for health status monitoring, (2) Thermo-acoustic sensing instruments for biomedical applications, and (3) Photoacoustic (PA) sensing and imaging systems, including dedicated reconstruction algorithms were reviewed from time-domain, frequency-domain, time-reversal, and model-based solutions. The future of EMA techniques for continuous healthcare with enhanced accuracy supported by artificial intelligence (AI) is also presented.
Collapse
|
8
|
Shi M, Zhao T, West SJ, Desjardins AE, Vercauteren T, Xia W. Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets. PHOTOACOUSTICS 2022; 26:100351. [PMID: 35495095 PMCID: PMC9048160 DOI: 10.1016/j.pacs.2022.100351] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/05/2022] [Revised: 03/29/2022] [Accepted: 03/30/2022] [Indexed: 06/14/2023]
Abstract
Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging.
Collapse
Affiliation(s)
- Mengjie Shi
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, United Kingdom
| | - Tianrui Zhao
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, United Kingdom
| | - Simeon J. West
- Department of Anaesthesia, University College Hospital, London NW1 2BU, United Kingdom
| | - Adrien E. Desjardins
- Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London, London W1 W 7TY, United Kingdom
- Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, United Kingdom
| | - Tom Vercauteren
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, United Kingdom
| | - Wenfeng Xia
- School of Biomedical Engineering and Imaging Sciences, King’s College London, London SE1 7EH, United Kingdom
| |
Collapse
|