A generative adversarial network for artifact removal in photoacoustic computed tomography with a linear-array transducer

Unsupervised adversarial neural network for enhancing vasculature in photoacoustic tomography images using optical coherence tomography angiography

Photoacoustic tomography (PAT) is a powerful imaging modality for visualizing tissue physiology and exogenous contrast agents. However, PAT faces challenges in visualizing deep-seated vascular structures due to light scattering, absorption, and reduced signal intensity with depth. Optical coherence tomography angiography (OCTA) offers high-contrast visualization of vasculature networks, yet its imaging depth is limited to a millimeter scale. Herein, we propose OCPA-Net, a novel unsupervised deep learning method that utilizes the rich vascular feature of OCTA to enhance PAT images. Trained on unpaired OCTA and PAT images, OCPA-Net incorporates a vessel-aware attention module to enhance deep-seated vessel details captured from OCTA. It leverages a domain-adversarial loss function to enforce structural consistency and a novel identity invariant loss to mitigate excessive image content generation. We validate the structural fidelity of OCPA-Net on simulation experiments, and then demonstrate its vascular enhancement performance on in vivo imaging experiments of tumor-bearing mice and contrast-enhanced pregnant mice. The results show the promise of our method for comprehensive vessel-related image analysis in preclinical research applications.

Unsupervised deep learning enables real-time image registration of fast-scanning optical-resolution photoacoustic microscopy

A fast scanner of optical-resolution photoacoustic microscopy is inherently vulnerable to perturbation, leading to severe image distortion and significant misalignment among multiple 2D or 3D images. Restoration and registration of these images is critical for accurately quantifying dynamic information in long-term imaging. However, traditional registration algorithms face a great challenge in computational throughput. Here, we develop an unsupervised deep learning based registration network to achieve real-time image restoration and registration. This method can correct artifacts from B-scan distortion and remove misalignment among adjacent and repetitive images in real time. Compared with conventional intensity based registration algorithms, the throughput of the developed algorithm is improved by 50 times. After training, the new deep learning method performs better than conventional feature based image registration algorithms. The results show that the proposed method can accurately restore and register the images of fast-scanning photoacoustic microscopy in real time, offering a powerful tool to extract dynamic vascular structural and functional information.

Image restoration for ring-array photoacoustic tomography system based on blind spatially rotational deconvolution

Ring-array photoacoustic tomography (PAT) system has been widely used in noninvasive biomedical imaging. However, the reconstructed image usually suffers from spatially rotational blur and streak artifacts due to the non-ideal imaging conditions. To improve the reconstructed image towards higher quality, we propose a concept of spatially rotational convolution to formulate the image blur process, then we build a regularized restoration problem model accordingly and design an alternating minimization algorithm which is called blind spatially rotational deconvolution to achieve the restored image. Besides, we also present an image preprocessing method based on the proposed algorithm to remove the streak artifacts. We take experiments on phantoms and in vivo biological tissues for evaluation, the results show that our approach can significantly enhance the resolution of the image obtained from ring-array PAT system and remove the streak artifacts effectively.

Unsupervised disentanglement strategy for mitigating artifact in photoacoustic tomography under extremely sparse view

Traditional methods under sparse view for reconstruction of photoacoustic tomography (PAT) often result in significant artifacts. Here, a novel image to image transformation method based on unsupervised learning artifact disentanglement network (ADN), named PAT-ADN, was proposed to address the issue. This network is equipped with specialized encoders and decoders that are responsible for encoding and decoding the artifacts and content components of unpaired images, respectively. The performance of the proposed PAT-ADN was evaluated using circular phantom data and the animal in vivo experimental data. The results demonstrate that PAT-ADN exhibits excellent performance in effectively removing artifacts. In particular, under extremely sparse view (e.g., 16 projections), structural similarity index and peak signal-to-noise ratio are improved by ∼188 % and ∼85 % in in vivo experimental data using the proposed method compared to traditional reconstruction methods. PAT-ADN improves the imaging performance of PAT, opening up possibilities for its application in multiple domains.

Challenges and advances in two-dimensional photoacoustic computed tomography: a review

Significance

Photoacoustic computed tomography (PACT), a hybrid imaging modality combining optical excitation with acoustic detection, has rapidly emerged as a prominent biomedical imaging technique.

Aim

We review the challenges and advances of PACT, including (1) limited view, (2) anisotropy resolution, (3) spatial aliasing, (4) acoustic heterogeneity (speed of sound mismatch), and (5) fluence correction of spectral unmixing.

Approach

We performed a comprehensive literature review to summarize the key challenges in PACT toward practical applications and discuss various solutions.

Results

There is a wide range of contributions from both industry and academic spaces. Various approaches, including emerging deep learning methods, are proposed to improve the performance of PACT further.

Conclusions

We outline contemporary technologies aimed at tackling the challenges in PACT applications.

U-Net enhanced real-time LED-based photoacoustic imaging

Photoacoustic (PA) imaging is hybrid imaging modality with good optical contrast and spatial resolution. Portable, cost-effective, smaller footprint light emitting diodes (LEDs) are rapidly becoming important PA optical sources. However, the key challenge faced by the LED-based systems is the low light fluence that is generally compensated by high frame averaging, consequently reducing acquisition frame-rate. In this study, we present a simple deep learning U-Net framework that enhances the signal-to-noise ratio (SNR) and contrast of PA image obtained by averaging low number of frames. The SNR increased by approximately four-fold for both in-class in vitro phantoms (4.39 ± 2.55) and out-of-class in vivo models (4.27 ± 0.87). We also demonstrate the noise invariancy of the network and discuss the downsides (blurry outcome and failure to reduce the salt & pepper noise). Overall, the developed U-Net framework can provide a real-time image enhancement platform for clinically translatable low-cost and low-energy light source-based PA imaging systems.

Compensating unknown speed of sound in learned fast 3D limited-view photoacoustic tomography

Real-time applications in three-dimensional photoacoustic tomography from planar sensors rely on fast reconstruction algorithms that assume the speed of sound (SoS) in the tissue is homogeneous. Moreover, the reconstruction quality depends on the correct choice for the constant SoS. In this study, we discuss the possibility of ameliorating the problem of unknown or heterogeneous SoS distributions by using learned reconstruction methods. This can be done by modelling the uncertainties in the training data. In addition, a correction term can be included in the learned reconstruction method. We investigate the influence of both and while a learned correction component can improve reconstruction quality further, we show that a careful choice of uncertainties in the training data is the primary factor to overcome unknown SoS. We support our findings with simulated and in vivo measurements in 3D.

Photoacoustic imaging of squirrel monkey cortical responses induced by peripheral mechanical stimulation

Non-human primates (NHPs) are crucial models for studies of neuronal activity. Emerging photoacoustic imaging modalities offer excellent tools for studying NHP brains with high sensitivity and high spatial resolution. In this research, a photoacoustic microscopy (PAM) device was used to provide a label-free quantitative characterization of cerebral hemodynamic changes due to peripheral mechanical stimulation. A 5 × 5 mm area within the somatosensory cortex region of an adult squirrel monkey was imaged. A deep, fully connected neural network was characterized and applied to the PAM images of the cortex to enhance the vessel structures after mechanical stimulation on the forelimb digits. The quality of the PAM images was improved significantly with a neural network while preserving the hemodynamic responses. The functional responses to the mechanical stimulation were characterized based on the improved PAM images. This study demonstrates capability of PAM combined with machine learning for functional imaging of the NHP brain.

Signal domain adaptation network for limited-view optoacoustic tomography

Optoacoustic (OA) imaging is based on optical excitation of biological tissues with nanosecond-duration laser pulses and detection of ultrasound (US) waves generated by thermoelastic expansion following light absorption. The image quality and fidelity of OA images critically depend on the extent of tomographic coverage provided by the US detector arrays. However, full tomographic coverage is not always possible due to experimental constraints. One major challenge concerns an efficient integration between OA and pulse-echo US measurements using the same transducer array. A common approach toward the hybridization consists in using standard linear transducer arrays, which readily results in arc-type artifacts and distorted shapes in OA images due to the limited angular coverage. Deep learning methods have been proposed to mitigate limited-view artifacts in OA reconstructions by mapping artifactual to artifact-free (ground truth) images. However, acquisition of ground truth data with full angular coverage is not always possible, particularly when using handheld probes in a clinical setting. Deep learning methods operating in the image domain are then commonly based on networks trained on simulated data. This approach is yet incapable of transferring the learned features between two domains, which results in poor performance on experimental data. Here, we propose a signal domain adaptation network (SDAN) consisting of i) a domain adaptation network to reduce the domain gap between simulated and experimental signals and ii) a sides prediction network to complement the missing signals in limited-view OA datasets acquired from a human forearm by means of a handheld linear transducer array. The proposed method showed improved performance in reducing limited-view artifacts without the need for ground truth signals from full tomographic acquisitions.

Adaptive machine learning method for photoacoustic computed tomography based on sparse array sensor data

BACKGROUND AND OBJECTIVE

Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms.

METHODS

We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs.

RESULTS

The proposed method achieved superior performance as demonstrated by simulation, phantom experiments, and in vivo experiment results. Compared with linear interpolation, XGBoost, CAE, and U-net, the simulation results show that the SSIM value is increased by 12.83%, 6.78%, 21.46%, and 12.33%. Moreover, the median R2 is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data.

CONCLUSIONS

This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.

Photoacoustic imaging on its way toward clinical utility: a tutorial review focusing on practical application in medicine

Significance

Photoacoustic imaging (PAI) enables the visualization of optical contrast with ultrasonic imaging. It is a field of intense research, with great promise for clinical application. Understanding the principles of PAI is important for engineering research and image interpretation.

Aim

In this tutorial review, we lay out the imaging physics, instrumentation requirements, standardization, and some practical examples for (junior) researchers, who have an interest in developing PAI systems and applications for clinical translation or applying PAI in clinical research.

Approach

We discuss PAI principles and implementation in a shared context, emphasizing technical solutions that are amenable to broad clinical deployment, considering factors such as robustness, mobility, and cost in addition to image quality and quantification.

Results

Photoacoustics, capitalizing on endogenous contrast or administered contrast agents that are approved for human use, yields highly informative images in clinical settings, which can support diagnosis and interventions in the future.

Conclusion

PAI offers unique image contrast that has been demonstrated in a broad set of clinical scenarios. The transition of PAI from a "nice-to-have" to a "need-to-have" modality will require dedicated clinical studies that evaluate therapeutic decision-making based on PAI and consideration of the actual value for patients and clinicians, compared with the associated cost.

Review of Deep Learning Approaches for Interleaved Photoacoustic and Ultrasound (PAUS) Imaging

Photoacoustic (PA) imaging provides optical contrast at relatively large depths within the human body, compared to other optical methods, at ultrasound (US) spatial resolution. By integrating real-time PA and US (PAUS) modalities, PAUS imaging has the potential to become a routine clinical modality bringing the molecular sensitivity of optics to medical US imaging. For applications where the full capabilities of clinical US scanners must be maintained in PAUS, conventional limited view and bandwidth transducers must be used. This approach, however, cannot provide high-quality maps of PA sources, especially vascular structures. Deep learning (DL) using data-driven modeling with minimal human design has been very effective in medical imaging, medical data analysis, and disease diagnosis, and has the potential to overcome many of the technical limitations of current PAUS imaging systems. The primary purpose of this article is to summarize the background and current status of DL applications in PAUS imaging. It also looks beyond current approaches to identify remaining challenges and opportunities for robust translation of PAUS technologies to the clinic.

Multikernel positional embedding convolutional neural network for photoacoustic reconstruction with sparse data

Photoacoustic imaging (PAI) is an emerging noninvasive imaging modality that merges the high contrast of optical imaging with the high resolution of ultrasonic imaging. Low-quality photoacoustic reconstruction with sparse data due to sparse spatial sampling and limited view detection is a major obstacle to the popularization of PAI for medical applications. Deep learning has been considered as the best solution to this problem in the past decade. In this paper, we propose what we believe to be a novel architecture, named DPM-UNet, which consists of the U-Net backbone with additional position embedding block and two multi-kernel-size convolution blocks, a dilated dense block and dilated multi-kernel-size convolution block. Our method was experimentally validated with both simulated data and in vivo data, achieving a SSIM of 0.9824 and a PSNR of 33.2744 dB. Furthermore, the reconstructed images of our proposed method were compared with those obtained by other advanced methods. The results have shown that our proposed DPM-UNet has a great advantage in PAI over other methods with respect to the imaging effect and memory consumption.

Noise-insensitive defocused signal and resolution enhancement for optical-resolution photoacoustic microscopy via deep learning

Optical-resolution photoacoustic microscopy suffers from narrow depth of field and a significant deterioration in defocused signal intensity and spatial resolution. Here, a method based on deep learning was proposed to enhance the defocused resolution and signal-to-noise ratio. A virtual optical-resolution photoacoustic microscopy based on k-wave was used to obtain the datasets of deep learning with different noise levels. A fully dense U-Net was trained with randomly distributed sources to improve the quality of photoacoustic images. The results show that the PSNR of defocused signal was enhanced by more than 1.2 times. An over 2.6-fold enhancement in lateral resolution and an over 3.4-fold enhancement in axial resolution of defocused regions were achieved. The large volumetric and high-resolution imaging of blood vessels further verified that the proposed method can effectively overcome the deterioration of the signal and the spatial resolution due to the narrow depth of field of optical-resolution photoacoustic microscopy.

Deep Learning Enhanced Volumetric Photoacoustic Imaging of Vasculature in Human

The development of high-performance imaging processing algorithms is a core area of photoacoustic tomography. While various deep learning based image processing techniques have been developed in the area, their applications in 3D imaging are still limited due to challenges in computational cost and memory allocation. To address those limitations, this work implements a 3D fully-dense (3DFD) U-net to linear array based photoacoustic tomography and utilizes volumetric simulation and mixed precision training to increase efficiency and training size. Through numerical simulation, phantom imaging, and in vivo experiments, this work demonstrates that the trained network restores the true object size, reduces the noise level and artifacts, improves the contrast at deep regions, and reveals vessels subject to limited view distortion. With these enhancements, 3DFD U-net successfully produces clear 3D vascular images of the palm, arms, breasts, and feet of human subjects. These enhanced vascular images offer improved capabilities for biometric identification, foot ulcer evaluation, and breast cancer imaging. These results indicate that the new algorithm will have a significant impact on preclinical and clinical photoacoustic tomography.

Enhanced resolution and sensitivity acoustic-resolution photoacoustic microscopy with semi/unsupervised GANs

Acoustic-resolution photoacoustic microscopy (AR-PAM) enables visualization of biological tissues at depths of several millimeters with superior optical absorption contrast. However, the lateral resolution and sensitivity of AR-PAM are generally lower than those of optical-resolution PAM (OR-PAM) owing to the intrinsic physical acoustic focusing mechanism. Here, we demonstrate a computational strategy with two generative adversarial networks (GANs) to perform semi/unsupervised reconstruction with high resolution and sensitivity in AR-PAM by maintaining its imaging capability at enhanced depths. The b-scan PAM images were prepared as paired (for semi-supervised conditional GAN) and unpaired (for unsupervised CycleGAN) groups for label-free reconstructed AR-PAM b-scan image generation and training. The semi/unsupervised GANs successfully improved resolution and sensitivity in a phantom and in vivo mouse ear test with ground truth. We also confirmed that GANs could enhance resolution and sensitivity of deep tissues without the ground truth.

Restoring the imaging quality of circular transducer array-based PACT using synthetic aperture focusing technique integrated with 2nd-derivative-based back projection scheme

Circular-array-based photoacoustic computed tomography (CA-PACT) is a promising imaging tool owing to its broad acoustic detection coverage and fidelity. However, CA-PACT suffers from poor image quality outside the focal zone along both elevational and lateral dimensions. To address this challenge, we proposed a novel reconstruction strategy by integrating the synthetic aperture focusing technique (SAFT) with the 2nd derivative-based back projection (2nd D-BP) algorithm to restore the image quality outside the focal zone along both the elevational and lateral axes. The proposed solution is a two-phase reconstruction scheme. In the first phase, with the assistance of an acoustic lens, we designed a circular array-based SAFT algorithm to restore the resolution and SNR along the elevational axis. The acoustic lens pushes the boundary of the upper limit of the SAFT scheme to achieve enhanced elevational resolution. In the second phase, we proposed a 2nd D-BP scheme to improve the lateral resolution and suppress noises in 3D imaging results. The 2nd D-BP strategy enhances the image quality along the lateral dimension by up-converting the high spatial frequencies of the object's absorption pattern. We validated the effectiveness of the proposed strategy using both phantoms and in vivo human experiments. The experimental results demonstrated superior image quality (7-fold enhancement in elevational resolution, 3-fold enhancement in lateral resolution, and an 11-dB increase in SNR). This strategy provides a new paradigm in the PACT system as it significantly enhances the spatial resolution and imaging contrast in both the elevational and lateral dimensions while maintaining a large focal zone.

Integrated Photoacoustic, Ultrasound, and Angiographic Tomography (PAUSAT) for NonInvasive Whole-Brain Imaging of Ischemic Stroke

Presented here is an experimental ischemic stroke study using our newly developed noninvasive imaging system that integrates three acoustic-based imaging technologies: photoacoustic, ultrasound, and angiographic tomography (PAUSAT). Combining these three modalities helps acquire multi-spectral photoacoustic tomography (PAT) of the brain blood oxygenation, high-frequency ultrasound imaging of the brain tissue, and acoustic angiography of the cerebral blood perfusion. The multi-modal imaging platform allows the study of cerebral perfusion and oxygenation changes in the whole mouse brain after stroke. Two commonly used ischemic stroke models were evaluated: the permanent middle cerebral artery occlusion (pMCAO) model and the photothrombotic (PT) model. PAUSAT was used to image the same mouse brains before and after a stroke and quantitatively analyze both stroke models. This imaging system was able to clearly show the brain vascular changes after ischemic stroke, including significantly reduced blood perfusion and oxygenation in the stroke infarct region (ipsilateral) compared to the uninjured tissue (contralateral). The results were confirmed by both laser speckle contrast imaging and triphenyltetrazolium chloride (TTC) staining. Furthermore, stroke infarct volume in both stroke models was measured and validated by TTC staining as the ground truth. Through this study, we have demonstrated that PAUSAT can be a powerful tool in noninvasive and longitudinal preclinical studies of ischemic stroke.

Performance evaluation of mesoscopic photoacoustic imaging

Photoacoustic mesoscopy visualises vascular architecture at high-resolution up to ~3 mm depth. Despite promise in preclinical and clinical imaging studies, with applications in oncology and dermatology, the accuracy and precision of photoacoustic mesoscopy is not well established. Here, we evaluate a commercial photoacoustic mesoscopy system for imaging vascular structures. Typical artefact types are first highlighted and limitations due to non-isotropic illumination and detection are evaluated with respect to rotation, angularity, and depth of the target. Then, using tailored phantoms and mouse models, we investigate system precision, showing coefficients of variation (COV) between repeated scans [short term (1 h): COV= 1.2%; long term (25 days): COV= 9.6%], from target repositioning (without: COV=1.2%, with: COV=4.1%), or from varying in vivo user experience (experienced: COV=15.9%, unexperienced: COV=20.2%). Our findings show robustness of the technique, but also underscore general challenges of limited-view photoacoustic systems in accurately imaging vessel-like structures, thereby guiding users when interpreting biologically-relevant information.

Adaptive enhancement of acoustic resolution photoacoustic microscopy imaging via deep CNN prior

Acoustic resolution photoacoustic microscopy (AR-PAM) is a promising medical imaging modality that can be employed for deep bio-tissue imaging. However, its relatively low imaging resolution has greatly hindered its wide applications. Previous model-based or learning-based PAM enhancement algorithms either require design of complex handcrafted prior to achieve good performance or lack the interpretability and flexibility that can adapt to different degradation models. However, the degradation model of AR-PAM imaging is subject to both imaging depth and center frequency of ultrasound transducer, which varies in different imaging conditions and cannot be handled by a single neural network model. To address this limitation, an algorithm integrating both learning-based and model-based method is proposed here so that a single framework can deal with various distortion functions adaptively. The vasculature image statistics is implicitly learned by a deep convolutional neural network, which served as plug and play (PnP) prior. The trained network can be directly plugged into the model-based optimization framework for iterative AR-PAM image enhancement, which fitted for different degradation mechanisms. Based on physical model, the point spread function (PSF) kernels for various AR-PAM imaging situations are derived and used for the enhancement of simulation and in vivo AR-PAM images, which collectively proved the effectiveness of proposed method. Quantitatively, the PSNR and SSIM values have all achieve best performance with the proposed algorithm in all three simulation scenarios; The SNR and CNR values have also significantly raised from 6.34 and 5.79 to 35.37 and 29.66 respectively in an in vivo testing result with the proposed algorithm.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

A jointed feature fusion framework for photoacoustic image reconstruction

The standard reconstruction of Photoacoustic (PA) computed tomography (PACT) image could cause the artifacts due to interferences or ill-posed setup. Recently, deep learning has been used to reconstruct the PA image with ill-posed conditions. In this paper, we propose a jointed feature fusion framework (JEFF-Net) based on deep learning to reconstruct the PA image using limited-view data. The cross-domain features from limited-view position-wise data and the reconstructed image are fused by a backtracked supervision. A quarter position-wise data (32 channels) is fed into model, which outputs another 3-quarters-view data (96 channels). Moreover, two novel losses are designed to restrain the artifacts by sufficiently manipulating superposed data. The experimental results have demonstrated the superior performance and quantitative evaluations show that our proposed method outperformed the ground-truth in some metrics by 135% (SSIM for simulation) and 40% (gCNR for in-vivo) improvement.

Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging

Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in image synthesis, Generative Adversarial Networks (GANs), and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.

Three-dimensional non-invasive brain imaging of ischemic stroke by integrated photoacoustic, ultrasound and angiographic tomography (PAUSAT)

We present an ischemic stroke study using our newly-developed PAUSAT system that integrates photoacoustic computed tomography (PACT), high-frequency ultrasound imaging, and acoustic angiographic tomography. PAUSAT is capable of three-dimensional (3D) imaging of the brain morphology, blood perfusion, and blood oxygenation. Using PAUSAT, we studied the hemodynamic changes in the whole mouse brain induced by two common ischemic stroke models: the permanent middle cerebral artery occlusion (pMCAO) model and the photothrombotic (PT) model. We imaged the same mouse brains before and after stroke, and quantitatively compared the two stroke models. We observed clear hemodynamic changes after ischemic stroke, including reduced blood perfusion and oxygenation. Such changes were spatially heterogenous. We also quantified the tissue infarct volume in both stroke models. The PAUSAT measurements were validated by laser speckle imaging and histology. Our results have collectively demonstrated that PAUSAT can be a valuable tool for non-invasive longitudinal studies of neurological diseases at the whole-brain scale.

A Deep Learning Method for Motion Artifact Correction in Intravascular Photoacoustic Image Sequence

In vivo application of intravascular photoacoustic (IVPA) imaging for coronary arteries is hampered by motion artifacts associated with the cardiac cycle. Gating is a common strategy to mitigate motion artifacts. However, a large amount of diagnostically valuable information might be lost due to one frame per cycle. In this work, we present a deep learning-based method for directly correcting motion artifacts in non-gated IVPA pullback sequences. The raw signal frames are classified into dynamic and static frames by clustering. Then, a neural network named Motion Artifact Correction (MAC)-Net is designed to correct motion in dynamic frames. Given the lack of the ground truth information on the underlying dynamics of coronary arteries, we trained and tested the network using a computer-generated dataset. Based on the results, it has been observed that the trained network can directly correct motion in successive frames while preserving the original structures without discarding any frames. The improvement in the visual effect of the longitudinal view has been demonstrated based on quantitative evaluation of the inter-frame dissimilarity. The comparison results validated the motion-suppression ability of our method comparable to gating and image registration-based non-learning methods, while maintaining the integrity of the pullbacks without image preprocessing. Experimental results from in vivo intravascular ultrasound and optical coherence tomography pullbacks validated the feasibility of our method in the in vivo intracoronary imaging scenario.

Sound out the impaired perfusion: Photoacoustic imaging in preclinical ischemic stroke

Acoustically detecting the optical absorption contrast, photoacoustic imaging (PAI) is a highly versatile imaging modality that can provide anatomical, functional, molecular, and metabolic information of biological tissues. PAI is highly scalable and can probe the same biological process at various length scales ranging from single cells (microscopic) to the whole organ (macroscopic). Using hemoglobin as the endogenous contrast, PAI is capable of label-free imaging of blood vessels in the brain and mapping hemodynamic functions such as blood oxygenation and blood flow. These imaging merits make PAI a great tool for studying ischemic stroke, particularly for probing into hemodynamic changes and impaired cerebral blood perfusion as a consequence of stroke. In this narrative review, we aim to summarize the scientific progresses in the past decade by using PAI to monitor cerebral blood vessel impairment and restoration after ischemic stroke, mostly in the preclinical setting. We also outline and discuss the major technological barriers and challenges that need to be overcome so that PAI can play a more significant role in preclinical stroke research, and more importantly, accelerate its translation to be a useful clinical diagnosis and management tool for human strokes.

Deep and Domain Transfer Learning Aided Photoacoustic Microscopy: Acoustic Resolution to Optical Resolution

Acoustic resolution photoacoustic micros- copy (AR-PAM) can achieve deeper imaging depth in biological tissue, with the sacrifice of imaging resolution compared with optical resolution photoacoustic microscopy (OR-PAM). Here we aim to enhance the AR-PAM image quality towards OR-PAM image, which specifically includes the enhancement of imaging resolution, restoration of micro-vasculatures, and reduction of artifacts. To address this issue, a network (MultiResU-Net) is first trained as generative model with simulated AR-OR image pairs, which are synthesized with physical transducer model. Moderate enhancement results can already be obtained when applying this model to in vivo AR imaging data. Nevertheless, the perceptual quality is unsatisfactory due to domain shift. Further, domain transfer learning technique under generative adversarial network (GAN) framework is proposed to drive the enhanced image's manifold towards that of real OR image. In this way, perceptually convincing AR to OR enhancement result is obtained, which can also be supported by quantitative analysis. Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values are significantly increased from 14.74 dB to 19.01 dB and from 0.1974 to 0.2937, respectively, validating the improvement of reconstruction correctness and overall perceptual quality. The proposed algorithm has also been validated across different imaging depths with experiments conducted in both shallow and deep tissue. The above AR to OR domain transfer learning with GAN (AODTL-GAN) framework has enabled the enhancement target with limited amount of matched in vivo AR-OR imaging data.

Feasibility of a Generative Adversarial Network for Artifact Removal in Experimental Photoacoustic Imaging

Photoacoustic tomography (PAT) reconstruction is an expeditiously growing interest among biomedical researchers because of its possible transition from laboratory to clinical pre-eminence. Nonetheless, the PAT inverse problem is yet to achieve an optimal solution in rapid and precise reconstruction under practical constraints. Precisely, the sparse sampling problem and random noise are the main impediments to attaining accuracy but in support of rapid PAT reconstruction. The limitations are associated with acquiring undersampled artifacts that deteriorate the optimality of the reconstruction task. Therefore, the former achievements of fast image formation limit the modality for clinical settings. Delving into the problem, here we explore a deep learning-based generative adversarial network (GAN) to improve the image quality by denoising and removing these artifacts. The specially designed attributes and unique manner of optimizing the problem, such as incorporating the data set limitations and providing stable training performance, constitute the main motivation behind the employment of GAN. Moreover, exploitation of the U-net variant as a generator network offers robust performance in terms of quality and computational cost, which is further validated with the detailed quantitative and qualitative analysis. The quantitatively evaluated structured similarity indexing method = 0.980 ± 0.043 and peak signal-to-noise ratio = 31 ± 0.002 dB state that the proposed solution provides the high-resolution image at the output, even training with a low-quality data set.

Deep Learning-Based Photoacoustic Imaging of Vascular Network Through Thick Porous Media

Photoacoustic imaging is a promising approach used to realize in vivo transcranial cerebral vascular imaging. However, the strong attenuation and distortion of the photoacoustic wave caused by the thick porous skull greatly affect the imaging quality. In this study, we developed a convolutional neural network based on U-Net to extract the effective photoacoustic information hidden in the speckle patterns obtained from vascular network images datasets under porous media. Our simulation and experimental results show that the proposed neural network can learn the mapping relationship between the speckle pattern and the target, and extract the photoacoustic signals of the vessels submerged in noise to reconstruct high-quality images of the vessels with a sharp outline and a clean background. Compared with the traditional photoacoustic reconstruction methods, the proposed deep learning-based reconstruction algorithm has a better performance with a lower mean absolute error, higher structural similarity, and higher peak signal-to-noise ratio of reconstructed images. In conclusion, the proposed neural network can effectively extract valid information from highly blurred speckle patterns for the rapid reconstruction of target images, which offers promising applications in transcranial photoacoustic imaging.

Deep-E: A Fully-Dense Neural Network for Improving the Elevation Resolution in Linear-Array-Based Photoacoustic Tomography

Linear-array-based photoacoustic tomography has shown broad applications in biomedical research and preclinical imaging. However, the elevational resolution of a linear array is fundamentally limited due to the weak cylindrical focus of the transducer element. While several methods have been proposed to address this issue, they have all handled the problem in a less time-efficient way. In this work, we propose to improve the elevational resolution of a linear array through Deep-E, a fully dense neural network based on U-net. Deep-E exhibits high computational efficiency by converting the three-dimensional problem into a two-dimension problem: it focused on training a model to enhance the resolution along elevational direction by only using the 2D slices in the axial and elevational plane and thereby reducing the computational burden in simulation and training. We demonstrated the efficacy of Deep-E using various datasets, including simulation, phantom, and human subject results. We found that Deep-E could improve elevational resolution by at least four times and recover the object's true size. We envision that Deep-E will have a significant impact in linear-array-based photoacoustic imaging studies by providing high-speed and high-resolution image enhancement.

Image reconstruction for endoscopic photoacoustic tomography including effects of detector responses

In photoacoustic tomography (PAT), conventional image reconstruction methods are generally based on the assumption of an ideal point-like ultrasonic detector. This assumption is appropriate when the receiving surface of the detector is sufficiently small and/or the distance between the imaged object and the detector is large enough. However, it does not hold in endoscopic applications of PAT. In this study, we propose a model-based image reconstruction method for endoscopic photoacoustic tomography (EPAT), considering the effect of detector responses on image quality. We construct a forward model to physically describe the imaging process of EPAT, including the generation of the initial pressure due to optical absorption and thermoelastic expansion, the propagation of photoacoustic waves in tissues, and the acoustic measurement. The model outputs the theoretical sampling voltage signal, which is the response of the ultrasonic detector to the acoustic pressure reaching its receiving surface. The images representing the distribution map of the optical absorption energy density on cross-sections of the imaged luminal structures are reconstructed from the sampling voltage signals output by the detector through iterative inversion of the forward model. Compared with the conventional approaches based on back-projection and other imaging models, our method improved the quality and spatial resolution of the resulting images.

High-resolution photoacoustic microscopy with deep penetration through learning

Optical-resolution photoacoustic microscopy (OR-PAM) enjoys superior spatial resolution and has received intense attention in recent years. The application, however, has been limited to shallow depths because of strong scattering of light in biological tissues. In this work, we propose to achieve deep-penetrating OR-PAM performance by using deep learning enabled image transformation on blurry living mouse vascular images that were acquired with an acoustic-resolution photoacoustic microscopy (AR-PAM) setup. A generative adversarial network (GAN) was trained in this study and improved the imaging lateral resolution of AR-PAM from 54.0 µm to 5.1 µm, comparable to that of a typical OR-PAM (4.7 µm). The feasibility of the network was evaluated with living mouse ear data, producing superior microvasculature images that outperforms blind deconvolution. The generalization of the network was validated with in vivo mouse brain data. Moreover, it was shown experimentally that the deep-learning method can retain high resolution at tissue depths beyond one optical transport mean free path. Whilst it can be further improved, the proposed method provides new horizons to expand the scope of OR-PAM towards deep-tissue imaging and wide applications in biomedicine.

An Intelligent Music Production Technology Based on Generation Confrontation Mechanism

In recent years, with the development of deep neural network becoming more and more mature, especially after the proposal of generative confrontation mechanism, academia has made many achievements in the research of image, video and text generation. Therefore, scholars began to use similar attempts in the research of music generation. Therefore, based on the existing theoretical technology and research work, this paper studies music production, and then proposes an intelligent music production technology based on generation confrontation mechanism to enrich the research in the field of computer music generation. This paper takes the music generation method based on generation countermeasure mechanism as the research topic, and mainly studies the following: after studying the existing music generation model based on generation countermeasure network, a time structure model for maintaining music coherence is proposed. In music generation, avoid manual input and ensure the interdependence between tracks. At the same time, this paper studies and implements the generation method of discrete music events based on multi track, including multi track correlation model and discrete processing. The lakh MIDI data set is studied. On this basis, the lakh MIDI is pre-processed to obtain the LMD piano roll data set, which is used in the music generation experiment of MCT-GAN. When studying the multi track music generation based on generation countermeasure network, this paper studies and analyzes three models, and puts forward the multi track music generation method based on CT-GAN, which mainly improves the existing music generation model based on GAN. Finally, the generation results of MCT-GAN are compared with those of Muse-GAN, so as to reflect the improvement effect of MCT-GAN. Select 20 auditees to listen to the generated music and real music and distinguish them. Finally, analyze them according to the evaluation results. After evaluation, it is concluded that the research effect of multi track music generation based on CT-GAN is improved.

Denoising of pre-beamformed photoacoustic data using generative adversarial networks

We have trained generative adversarial networks (GANs) to mimic both the effect of temporal averaging and of singular value decomposition (SVD) denoising. This effectively removes noise and acquisition artifacts and improves signal-to-noise ratio (SNR) in both the radio-frequency (RF) data and in the corresponding photoacoustic reconstructions. The method allows a single frame acquisition instead of averaging multiple frames, reducing scan time and total laser dose significantly. We have tested this method on experimental data, and quantified the improvement over using either SVD denoising or frame averaging individually for both the RF data and the reconstructed images. We achieve a mean squared error (MSE) of 0.05%, structural similarity index measure (SSIM) of 0.78, and a feature similarity index measure (FSIM) of 0.85 compared to our ground-truth RF results. In the subsequent reconstructions using the denoised data we achieve a MSE of 0.05%, SSIM of 0.80, and a FSIM of 0.80 compared to our ground-truth reconstructions.

Artifact removal in photoacoustic tomography with an unsupervised method

Photoacoustic tomography (PAT) is an emerging biomedical imaging technology that can realize high contrast imaging with a penetration depth of the acoustic. Recently, deep learning (DL) methods have also been successfully applied to PAT for improving the image reconstruction quality. However, the current DL-based PAT methods are implemented by the supervised learning strategy, and the imaging performance is dependent on the available ground-truth data. To overcome the limitation, this work introduces a new image domain transformation method based on cyclic generative adversarial network (CycleGAN), termed as PA-GAN, which is used to remove artifacts in PAT images caused by the use of the limited-view measurement data in an unsupervised learning way. A series of data from phantom and in vivo experiments are used to evaluate the performance of the proposed PA-GAN. The experimental results show that PA-GAN provides a good performance in removing artifacts existing in photoacoustic tomographic images. In particular, when dealing with extremely sparse measurement data (e.g., 8 projections in circle phantom experiments), higher imaging performance is achieved by the proposed unsupervised PA-GAN, with an improvement of ∼14% in structural similarity (SSIM) and ∼66% in peak signal to noise ratio (PSNR), compared with the supervised-learning U-Net method. With an increasing number of projections (e.g., 128 projections), U-Net, especially FD U-Net, shows a slight improvement in artifact removal capability, in terms of SSIM and PSNR. Furthermore, the computational time obtained by PA-GAN and U-Net is similar (∼60 ms/frame), once the network is trained. More importantly, PA-GAN is more flexible than U-Net that allows the model to be effectively trained with unpaired data. As a result, PA-GAN makes it possible to implement PAT with higher flexibility without compromising imaging performance.

Comparing Deep Learning Frameworks for Photoacoustic Tomography Image Reconstruction

Conventional reconstruction methods for photoacoustic images are not suitable for the scenario of sparse sensing and geometrical limitation. To overcome these challenges and enhance the quality of reconstruction, several learning-based methods have recently been introduced for photoacoustic tomography reconstruction. The goal of this study is to compare and systematically evaluate the recently proposed learning-based methods and modified networks for photoacoustic image reconstruction. Specifically, learning-based post-processing methods and model-based learned iterative reconstruction methods are investigated. In addition to comparing the differences inherently brought by the models, we also study the impact of different inputs on the reconstruction effect. Our results demonstrate that the reconstruction performance mainly stems from the effective amount of information carried by the input. The inherent difference of the models based on the learning-based post-processing method does not provide a significant difference in photoacoustic image reconstruction. Furthermore, the results indicate that the model-based learned iterative reconstruction method outperforms all other learning-based post-processing methods in terms of generalizability and robustness.

Photoacoustic computed tomography of mechanical HIFU-induced vascular injury

Mechanical high-intensity focused ultrasound (HIFU) has been used for cancer treatment and drug delivery. Existing monitoring methods for mechanical HIFU therapies such as MRI and ultrasound imaging often suffer from high cost, poor spatial-temporal resolution, and/or low sensitivity to tissue's hemodynamic changes. Evaluating vascular injury during mechanical HIFU treatment, therefore, remains challenging. Photoacoustic computed tomography (PACT) is a promising tool to meet this need. Intrinsically sensitive to optical absorption, PACT provides high-resolution imaging of blood vessels using hemoglobin as the endogenous contrast. In this study, we have developed an integrated HIFU-PACT system for detecting vascular rupture in mechanical HIFU treatment. We have demonstrated singular value decomposition for enhancing hemorrhage detection. We have validated the HIFU-PACT performance on phantoms and in vivo animal tumor models. We expect that PACT-HIFU will find practical applications in oncology research using small animal models.

Investigation of photoacoustic tomography reconstruction with a limited view from linear array

SIGNIFICANCE

As linear array transducers are widely used in clinical ultrasound imaging, photoacoustic tomography (PAT) with linear arrays is similarly suitable for clinical applications. However, due to the limited-view problem, a linear array has limited performance and leads to artifacts and blurring, which has hindered its broader application. There is a need to address the limited-view problem in PAT imaging with linear arrays.

AIM

We investigate potential approaches for improving PAT reconstruction from linear array, by optimizing the detection geometry and implementing iterative reconstruction.

APPROACH

PAT imaging with a single-array, dual-probe configurations in parallel-shape and L-shape, and square-shape configuration are compared in simulations and phantom experiments. An iterative model-based algorithm based on the variance-reduced stochastic gradient descent (VR-SGD) method is implemented. The optimum configuration found in simulation is validated on phantom experiments.

RESULTS

PAT imaging with dual-probe detection and VR-SGD algorithm is found to improve the limited-view problem compared to a single probe and provide comparable performance as full-view geometry in simulation. This configuration is validated in experiments where more complete structure is obtained with reduced artifacts compared with a single array.

CONCLUSIONS

PAT with dual-probe detection and iterative reconstruction is a promising solution to the limited-view problem of linear arrays.

Binary photoacoustic tomography for improved vasculature imaging

SIGNIFICANCE

The proposed binary tomography approach was able to recover the vasculature structures accurately, which could potentially enable the utilization of binary tomography algorithm in scenarios such as therapy monitoring and hemorrhage detection in different organs.

AIM

Photoacoustic tomography (PAT) involves reconstruction of vascular networks having direct implications in cancer research, cardiovascular studies, and neuroimaging. Various methods have been proposed for recovering vascular networks in photoacoustic imaging; however, most methods are two-step (image reconstruction and image segmentation) in nature. We propose a binary PAT approach wherein direct reconstruction of vascular network from the acquired photoacoustic sinogram data is plausible.

APPROACH

Binary tomography approach relies on solving a dual-optimization problem to reconstruct images with every pixel resulting in a binary outcome (i.e., either background or the absorber). Further, the binary tomography approach was compared against backprojection, Tikhonov regularization, and sparse recovery-based schemes.

RESULTS

Numerical simulations, physical phantom experiment, and in-vivo rat brain vasculature data were used to compare the performance of different algorithms. The results indicate that the binary tomography approach improved the vasculature recovery by 10% using in-silico data with respect to the Dice similarity coefficient against the other reconstruction methods.

CONCLUSION

The proposed algorithm demonstrates superior vasculature recovery with limited data both visually and based on quantitative image metrics.

Deep Learning in Biomedical Optics

This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. Lasers Surg. Med. © 2021 Wiley Periodicals LLC.

Subaperture Processing-Based Adaptive Beamforming for Photoacoustic Imaging

Delay-and-sum (DAS) beamformers, when applied to photoacoustic (PA) image reconstruction, produce strong sidelobes due to the absence of transmit focusing. Consequently, DAS PA images are often severely degraded by strong off-axis clutter. For preclinical in vivo cardiac PA imaging, the presence of these noise artifacts hampers the detectability and interpretation of PA signals from the myocardial wall, crucial for studying blood-dominated cardiac pathological information and to complement functional information derived from ultrasound imaging. In this article, we present PA subaperture processing (PSAP), an adaptive beamforming method, to mitigate these image degrading effects. In PSAP, a pair of DAS reconstructed images is formed by splitting the received channel data into two complementary nonoverlapping subapertures. Then, a weighting matrix is derived by analyzing the correlation between subaperture beamformed images and multiplied with the full-aperture DAS PA image to reduce sidelobes and incoherent clutter. We validated PSAP using numerical simulation studies using point target, diffuse inclusion and microvasculature imaging, and in vivo feasibility studies on five healthy murine models. Qualitative and quantitative analysis demonstrate improvements in PAI image quality with PSAP compared to DAS and coherence factor weighted DAS (DAS _CF ). PSAP demonstrated improved target detectability with a higher generalized contrast-to-noise (gCNR) ratio in vasculature simulations where PSAP produces 19.61% and 19.53% higher gCNRs than DAS and DAS _CF , respectively. Furthermore, PSAP provided higher image contrast quantified using contrast ratio (CR) (e.g., PSAP produces 89.26% and 11.90% higher CR than DAS and DAS _CF in vasculature simulations) and improved clutter suppression.

Photoacoustic computed tomography for functional human brain imaging [Invited]

The successes of magnetic resonance imaging and modern optical imaging of human brain function have stimulated the development of complementary modalities that offer molecular specificity, fine spatiotemporal resolution, and sufficient penetration simultaneously. By virtue of its rich optical contrast, acoustic resolution, and imaging depth far beyond the optical transport mean free path (∼1 mm in biological tissues), photoacoustic computed tomography (PACT) offers a promising complementary modality. In this article, PACT for functional human brain imaging is reviewed in its hardware, reconstruction algorithms, in vivo demonstration, and potential roadmap.

Deep image prior for undersampling high-speed photoacoustic microscopy

Photoacoustic microscopy (PAM) is an emerging imaging method combining light and sound. However, limited by the laser's repetition rate, state-of-the-art high-speed PAM technology often sacrifices spatial sampling density (i.e., undersampling) for increased imaging speed over a large field-of-view. Deep learning (DL) methods have recently been used to improve sparsely sampled PAM images; however, these methods often require time-consuming pre-training and large training dataset with ground truth. Here, we propose the use of deep image prior (DIP) to improve the image quality of undersampled PAM images. Unlike other DL approaches, DIP requires neither pre-training nor fully-sampled ground truth, enabling its flexible and fast implementation on various imaging targets. Our results have demonstrated substantial improvement in PAM images with as few as 1.4 % of the fully sampled pixels on high-speed PAM. Our approach outperforms interpolation, is competitive with pre-trained supervised DL method, and is readily translated to other high-speed, undersampling imaging modalities.

Sounding out the hidden data: A concise review of deep learning in photoacoustic imaging

The rapidly evolving field of photoacoustic tomography utilizes endogenous chromophores to extract both functional and structural information from deep within tissues. It is this power to perform precise quantitative measurements in vivo-with endogenous or exogenous contrast-that makes photoacoustic tomography highly promising for clinical translation in functional brain imaging, early cancer detection, real-time surgical guidance, and the visualization of dynamic drug responses. Considering photoacoustic tomography has benefited from numerous engineering innovations, it is of no surprise that many of photoacoustic tomography's current cutting-edge developments incorporate advances from the equally novel field of artificial intelligence. More specifically, alongside the growth and prevalence of graphical processing unit capabilities within recent years has emerged an offshoot of artificial intelligence known as deep learning. Rooted in the solid foundation of signal processing, deep learning typically utilizes a method of optimization known as gradient descent to minimize a loss function and update model parameters. There are already a number of innovative efforts in photoacoustic tomography utilizing deep learning techniques for a variety of purposes, including resolution enhancement, reconstruction artifact removal, undersampling correction, and improved quantification. Most of these efforts have proven to be highly promising in addressing long-standing technical obstacles where traditional solutions either completely fail or make only incremental progress. This concise review focuses on the history of applied artificial intelligence in photoacoustic tomography, presents recent advances at this multifaceted intersection of fields, and outlines the most exciting advances that will likely propagate into promising future innovations.

Perspective on fast-evolving photoacoustic tomography

SIGNIFICANCE

Acoustically detecting the rich optical absorption contrast in biological tissues, photoacoustic tomography (PAT) seamlessly bridges the functional and molecular sensitivity of optical excitation with the deep penetration and high scalability of ultrasound detection. As a result of continuous technological innovations and commercial development, PAT has been playing an increasingly important role in life sciences and patient care, including functional brain imaging, smart drug delivery, early cancer diagnosis, and interventional therapy guidance.

AIM

Built on our 2016 tutorial article that focused on the principles and implementations of PAT, this perspective aims to provide an update on the exciting technical advances in PAT.

APPROACH

This perspective focuses on the recent PAT innovations in volumetric deep-tissue imaging, high-speed wide-field microscopic imaging, high-sensitivity optical ultrasound detection, and machine-learning enhanced image reconstruction and data processing. Representative applications are introduced to demonstrate these enabling technical breakthroughs in biomedical research.

CONCLUSIONS

We conclude the perspective by discussing the future development of PAT technologies.

Deep learning for biomedical photoacoustic imaging: A review

Photoacoustic imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.

Deep learning in photoacoustic imaging: a review

SIGNIFICANCE

Photoacoustic (PA) imaging can provide structural, functional, and molecular information for preclinical and clinical studies. For PA imaging (PAI), non-ideal signal detection deteriorates image quality, and quantitative PAI (QPAI) remains challenging due to the unknown light fluence spectra in deep tissue. In recent years, deep learning (DL) has shown outstanding performance when implemented in PAI, with applications in image reconstruction, quantification, and understanding.

AIM

We provide (i) a comprehensive overview of the DL techniques that have been applied in PAI, (ii) references for designing DL models for various PAI tasks, and (iii) a summary of the future challenges and opportunities.

APPROACH

Papers published before November 2020 in the area of applying DL in PAI were reviewed. We categorized them into three types: image understanding, reconstruction of the initial pressure distribution, and QPAI.

RESULTS

When applied in PAI, DL can effectively process images, improve reconstruction quality, fuse information, and assist quantitative analysis.

CONCLUSION

DL has become a powerful tool in PAI. With the development of DL theory and technology, it will continue to boost the performance and facilitate the clinical translation of PAI.

Photoacoustic-guided surgery from head to toe [Invited]

Photoacoustic imaging-the combination of optics and acoustics to visualize differences in optical absorption - has recently demonstrated strong viability as a promising method to provide critical guidance of multiple surgeries and procedures. Benefits include its potential to assist with tumor resection, identify hemorrhaged and ablated tissue, visualize metal implants (e.g., needle tips, tool tips, brachytherapy seeds), track catheter tips, and avoid accidental injury to critical subsurface anatomy (e.g., major vessels and nerves hidden by tissue during surgery). These benefits are significant because they reduce surgical error, associated surgery-related complications (e.g., cancer recurrence, paralysis, excessive bleeding), and accidental patient death in the operating room. This invited review covers multiple aspects of the use of photoacoustic imaging to guide both surgical and related non-surgical interventions. Applicable organ systems span structures within the head to contents of the toes, with an eye toward surgical and interventional translation for the benefit of patients and for use in operating rooms and interventional suites worldwide. We additionally include a critical discussion of complete systems and tools needed to maximize the success of surgical and interventional applications of photoacoustic-based technology, spanning light delivery, acoustic detection, and robotic methods. Multiple enabling hardware and software integration components are also discussed, concluding with a summary and future outlook based on the current state of technological developments, recent achievements, and possible new directions.

Development of a digital breast phantom for photoacoustic computed tomography

Photoacoustic (PA) imaging provides morphological and functional information about angiogenesis and thus is potentially suitable for breast cancer diagnosis. However, the development of PA breast imaging has been hindered by inadequate patients and a lack of ground truth images. Here, we report a digital breast phantom with realistic acoustic and optical properties, with which a digital PA-ultrasound imaging pipeline is developed to create a diverse pool of virtual patients with three types of masses: ductal carcinoma in situ, invasive breast cancer, and fibroadenoma. The experimental results demonstrate that our model is realistic, flexible, and can be potentially useful for accelerating the development of PA breast imaging technology.