Comparing Deep Learning Frameworks for Photoacoustic Tomography Image Reconstruction

MSD-Net: Multi-scale dense convolutional neural network for photoacoustic image reconstruction with sparse data

Photoacoustic imaging (PAI) is an emerging hybrid imaging technology that combines the advantages of optical and ultrasound imaging. Despite its excellent imaging capabilities, PAI still faces numerous challenges in clinical applications, particularly sparse spatial sampling and limited view detection. These limitations often result in severe streak artifacts and blurring when using standard methods to reconstruct images from incomplete data. In this work, we propose an improved convolutional neural network (CNN) architecture, called multi-scale dense UNet (MSD-Net), to correct artifacts in 2D photoacoustic tomography (PAT). MSD-Net exploits the advantages of multi-scale information fusion and dense connections to improve the performance of CNN. Experimental validation with both simulated and in vivo datasets demonstrates that our method achieves better reconstructions with improved speed.

Trends and developments in 3D photoacoustic imaging systems: A review of recent progress

Photoacoustic imaging (PAI) is a non-invasive diagnostic imaging technique that utilizes the photoacoustic effect by combining optical and ultrasound imaging systems. The development of PAI is mostly centered on the generation of a high-quality 3D reconstruction system for more optimal and accurate identification of tissue abnormalities. This literature study was conducted to analyze the 3D image reconstruction in PAI over 2017-2024. In this review, the collected articles in 3D photoacoustic imaging were categorized based on the approach, design, and purpose of each study. Firstly, the approaches of the studies were classified into three groups: experimental studies, numerical simulation, and numerical simulation with experimental validation. Secondly, the design of the study was assessed based on the photoacoustic modality, laser type, and sensing mechanism. Thirdly, the purpose of the collected studies was summarized into seven subsections, including image quality improvement, frame rate improvement, image segmentation, system integration, inter-systems comparisons, improving computational efficiency, and portable system development. The results of this review revealed that the 3D PAI systems have been developed by various research groups, suggesting the investigation of numerous biological objects. Therefore, 3D PAI has the potential to contribute a wide range of novel biological imaging systems that support real-time biomedical imaging in the future.

Extractor-attention-predictor network for quantitative photoacoustic tomography

Quantitative photoacoustic tomography (qPAT) holds great potential in estimating chromophore concentrations, whereas the involved optical inverse problem, aiming to recover absorption coefficient distributions from photoacoustic images, remains challenging. To address this problem, we propose an extractor-attention-predictor network architecture (EAPNet), which employs a contracting-expanding structure to capture contextual information alongside a multilayer perceptron to enhance nonlinear modeling capability. A spatial attention module is introduced to facilitate the utilization of important information. We also use a balanced loss function to prevent network parameter updates from being biased towards specific regions. Our method obtains satisfactory quantitative metrics in simulated and real-world validations. Moreover, it demonstrates superior robustness to target properties and yields reliable results for targets with small size, deep location, or relatively low absorption intensity, indicating its broader applicability. The EAPNet, compared to the conventional UNet, exhibits improved efficiency, which significantly enhances performance while maintaining similar network size and computational complexity.

Unsupervised denoising of photoacoustic images based on the Noise2Noise network

In this study, we implemented an unsupervised deep learning method, the Noise2Noise network, for the improvement of linear-array-based photoacoustic (PA) imaging. Unlike supervised learning, which requires a noise-free ground truth, the Noise2Noise network can learn noise patterns from a pair of noisy images. This is particularly important for in vivo PA imaging, where the ground truth is not available. In this study, we developed a method to generate noise pairs from a single set of PA images and verified our approach through simulation and experimental studies. Our results reveal that the method can effectively remove noise, improve signal-to-noise ratio, and enhance vascular structures at deeper depths. The denoised images show clear and detailed vascular structure at different depths, providing valuable insights for preclinical research and potential clinical applications.

Dual-modal Photoacoustic and Ultrasound Imaging: from preclinical to clinical applications

Photoacoustic imaging is a novel biomedical imaging modality that has emerged over the recent decades. Due to the conversion of optical energy into the acoustic wave, photoacoustic imaging offers high-resolution imaging in depth beyond the optical diffusion limit. Photoacoustic imaging is frequently used in conjunction with ultrasound as a hybrid modality. The combination enables the acquisition of both optical and acoustic contrasts of tissue, providing functional, structural, molecular, and vascular information within the same field of view. In this review, we first described the principles of various photoacoustic and ultrasound imaging techniques and then classified the dual-modal imaging systems based on their preclinical and clinical imaging applications. The advantages of dual-modal imaging were thoroughly analyzed. Finally, the review ends with a critical discussion of existing developments and a look toward the future.

A Systematic Literature Review of 3D Deep Learning Techniques in Computed Tomography Reconstruction

Computed tomography (CT) is used in a wide range of medical imaging diagnoses. However, the reconstruction of CT images from raw projection data is inherently complex and is subject to artifacts and noise, which compromises image quality and accuracy. In order to address these challenges, deep learning developments have the potential to improve the reconstruction of computed tomography images. In this regard, our research aim is to determine the techniques that are used for 3D deep learning in CT reconstruction and to identify the training and validation datasets that are accessible. This research was performed on five databases. After a careful assessment of each record based on the objective and scope of the study, we selected 60 research articles for this review. This systematic literature review revealed that convolutional neural networks (CNNs), 3D convolutional neural networks (3D CNNs), and deep learning reconstruction (DLR) were the most suitable deep learning algorithms for CT reconstruction. Additionally, two major datasets appropriate for training and developing deep learning systems were identified: 2016 NIH-AAPM-Mayo and MSCT. These datasets are important resources for the creation and assessment of CT reconstruction models. According to the results, 3D deep learning may increase the effectiveness of CT image reconstruction, boost image quality, and lower radiation exposure. By using these deep learning approaches, CT image reconstruction may be made more precise and effective, improving patient outcomes, diagnostic accuracy, and healthcare system productivity.

Adaptive machine learning method for photoacoustic computed tomography based on sparse array sensor data

BACKGROUND AND OBJECTIVE

Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms.

METHODS

We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs.

RESULTS

The proposed method achieved superior performance as demonstrated by simulation, phantom experiments, and in vivo experiment results. Compared with linear interpolation, XGBoost, CAE, and U-net, the simulation results show that the SSIM value is increased by 12.83%, 6.78%, 21.46%, and 12.33%. Moreover, the median R2 is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data.

CONCLUSIONS

This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

Application of photoacoustic computed tomography in biomedical imaging: A literature review

Photoacoustic computed tomography (PACT) is a hybrid imaging modality that combines optical excitation and acoustic detection techniques. It obtains high-resolution deep-tissue images based on the deep penetration of light, the anisotropy of light absorption in objects, and the photoacoustic effect. Hence, PACT shows great potential in biomedical sample imaging. Recently, due to its advantages of high sensitivity to optical absorption and wide scalability of spatial resolution with the desired imaging depth, PACT has received increasing attention in preclinical and clinical practice. To date, there has been a proliferation of PACT systems designed for specific biomedical imaging applications, from small animals to human organs, from ex vivo to in vivo real-time imaging, and from simple structural imaging to functional and molecular imaging with external contrast agents. Therefore, it is of great importance to summarize the previous applications of PACT systems in biomedical imaging and clinical practice. In this review, we searched for studies related to PACT imaging of biomedical tissues and samples over the past two decades; divided the studies into two categories, PACT imaging of preclinical animals and PACT imaging of human organs and body parts; and discussed the significance of the studies. Finally, we pointed out the future directions of PACT in biomedical applications. With the development of exogenous contrast agents and advances of imaging technique, in the future, PACT will enable biomedical imaging from organs to whole bodies, from superficial vasculature to internal organs, from anatomy to functions, and will play an increasingly important role in biomedical research and clinical practice.

Photoacoustic imaging of hidden dental caries using visible-light diode laser

BACKGROUND

Hidden caries is a type of tooth decay that is difficult to identify through visual diagnosis because teeth with hidden caries appear normal on the tooth surface but are damaged underneath.

METHODS

A photoacoustic imaging system based on visible light using a diode laser with a wavelength of 532 nm was developed to detect hidden caries in teeth.

RESULTS

The results indicate that the average of acoustic intensity level for healthy teeth is -74.2 ± 0.1 dB, and the average of acoustic intensity range for teeth with hidden caries is -81.2 ± 0.5 dB. The intensity level for the caries area varies depending on the severity of caries.

CONCLUSION

Based on the acoustic intensity level measured by the interaction of teeth with laser light, the photoacoustic imaging system in the study can accurately detect the presence of hidden caries and recognize the difference between caries teeth and healthy teeth. This research can be developed into a prototype of a simple device that makes it easy to operate in dental practice.

Fast iterative reconstruction for photoacoustic tomography using learned physical model: Theoretical validation

Iterative reconstruction has demonstrated superior performance in medical imaging under compressed, sparse, and limited-view sensing scenarios. However, iterative reconstruction algorithms are slow to converge and rely heavily on hand-crafted parameters to achieve good performance. Many iterations are usually required to reconstruct a high-quality image, which is computationally expensive due to repeated evaluations of the physical model. While learned iterative reconstruction approaches such as model-based learning (MBLr) can reduce the number of iterations through convolutional neural networks, it still requires repeated evaluations of the physical models at each iteration. Therefore, the goal of this study is to develop a Fast Iterative Reconstruction (FIRe) algorithm that incorporates a learned physical model into the learned iterative reconstruction scheme to further reduce the reconstruction time while maintaining robust reconstruction performance. We also propose an efficient training scheme for FIRe, which releases the enormous memory footprint required by learned iterative reconstruction methods through the concept of recursive training. The results of our proposed method demonstrate comparable reconstruction performance to learned iterative reconstruction methods with a 9x reduction in computation time and a 620x reduction in computation time compared to variational reconstruction.

Alignment-invariant signal reality reconstruction in hyperspectral imaging using a deep convolutional neural network architecture

The energy resolution in hyperspectral imaging techniques has always been an important matter in data interpretation. In many cases, spectral information is distorted by elements such as instruments' broad optical transfer function, and electronic high frequency noises. In the past decades, advances in artificial intelligence methods have provided robust tools to better study sophisticated system artifacts in spectral data and take steps towards removing these artifacts from the experimentally obtained data. This study evaluates the capability of a recently developed deep convolutional neural network script, EELSpecNet, in restoring the reality of a spectral data. The particular strength of the deep neural networks is to remove multiple instrumental artifacts such as random energy jitters of the source, signal convolution by the optical transfer function and high frequency noise at once using a single training data set. Here, EELSpecNet performance in reducing noise, and restoring the original reality of the spectra is evaluated for near zero-loss electron energy loss spectroscopy signals in Scanning Transmission Electron Microscopy. EELSpecNet demonstrates to be more efficient and more robust than the currently widely used Bayesian statistical method, even in harsh conditions (e.g. high signal broadening, intense high frequency noise).

Deep learning methods hold promise for light fluence compensation in three-dimensional optoacoustic imaging

SIGNIFICANCE

Quantitative optoacoustic imaging (QOAI) continues to be a challenge due to the influence of nonlinear optical fluence distribution, which distorts the optoacoustic image representation. Nonlinear optical fluence correction in OA imaging is highly ill-posed, leading to the inaccurate recovery of optical absorption maps. This work aims to recover the optical absorption maps using deep learning (DL) approach by correcting for the fluence effect.

AIM

Different DL models were compared and investigated to enable optical absorption coefficient recovery at a particular wavelength in a nonhomogeneous foreground and background medium.

APPROACH

Data-driven models were trained with two-dimensional (2D) Blood vessel and three-dimensional (3D) numerical breast phantom with highly heterogeneous/realistic structures to correct for the nonlinear optical fluence distribution. The trained DL models such as U-Net, Fully Dense (FD) U-Net, Y-Net, FD Y-Net, Deep residual U-Net (Deep ResU-Net), and generative adversarial network (GAN) were tested to evaluate the performance of optical absorption coefficient recovery (or fluence compensation) with in-silico and in-vivo datasets.

RESULTS

The results indicated that FD U-Net-based deconvolution improves by about 10% over reconstructed optoacoustic images in terms of peak-signal-to-noise ratio. Further, it was observed that DL models can indeed highlight deep-seated structures with higher contrast due to fluence compensation. Importantly, the DL models were found to be about 17 times faster than solving diffusion equation for fluence correction.

CONCLUSIONS

The DL methods were able to compensate for nonlinear optical fluence distribution more effectively and improve the optoacoustic image quality.

Feasibility of a Generative Adversarial Network for Artifact Removal in Experimental Photoacoustic Imaging

Photoacoustic tomography (PAT) reconstruction is an expeditiously growing interest among biomedical researchers because of its possible transition from laboratory to clinical pre-eminence. Nonetheless, the PAT inverse problem is yet to achieve an optimal solution in rapid and precise reconstruction under practical constraints. Precisely, the sparse sampling problem and random noise are the main impediments to attaining accuracy but in support of rapid PAT reconstruction. The limitations are associated with acquiring undersampled artifacts that deteriorate the optimality of the reconstruction task. Therefore, the former achievements of fast image formation limit the modality for clinical settings. Delving into the problem, here we explore a deep learning-based generative adversarial network (GAN) to improve the image quality by denoising and removing these artifacts. The specially designed attributes and unique manner of optimizing the problem, such as incorporating the data set limitations and providing stable training performance, constitute the main motivation behind the employment of GAN. Moreover, exploitation of the U-net variant as a generator network offers robust performance in terms of quality and computational cost, which is further validated with the detailed quantitative and qualitative analysis. The quantitatively evaluated structured similarity indexing method = 0.980 ± 0.043 and peak signal-to-noise ratio = 31 ± 0.002 dB state that the proposed solution provides the high-resolution image at the output, even training with a low-quality data set.

Spectral crosstalk in photoacoustic computed tomography

Multispectral photoacoustic (PA) imaging faces two major challenges: the spectral coloring effect, which has been studied extensively as an optical inversion problem, and the spectral crosstalk, which is basically a result of non-ideal acoustic inversion. So far, there is no systematic work to analyze the spectral crosstalk because acoustic inversion and spectroscopic measurement are always treated as decoupled. In this work, we theorize and demonstrate through a series of simulations and experiments how imperfect acoustic inversion induces inaccurate PA spectrum measurement. We provide detailed analysis to elucidate how different factors, including limited bandwidth, limited view, light attenuation, out-of-plane signal, and image reconstruction schemes, conspire to render the measured PA spectrum inaccurate. We found that the model-based reconstruction outperforms universal back-projection in suppressing the spectral crosstalk in some cases.

Applying Deep Learning-Based Human Motion Recognition System in Sports Competition

The exploration here intends to compensate for the traditional human motion recognition (HMR) systems' poor performance on large-scale datasets and micromotions. To this end, improvement is designed for the HMR in sports competition based on the deep learning (DL) algorithm. First, the background and research status of HMR are introduced. Then, a new HMR algorithm is proposed based on kernel extreme learning machine (KELM) multidimensional feature fusion (MFF). Afterward, a simulation experiment is designed to evaluate the performance of the proposed KELM-MFF-based HMR algorithm. The results showed that the recognition rate of the proposed KELM-MFF-based HMR is higher than other algorithms. The recognition rate at 10 video frame sampling points is ranked from high to low: the proposed KELM-MFF-based HMR, support vector machine (SVM)-MFF-based HMR, convolutional neural network (CNN) + optical flow (CNN-T)-based HMR, improved dense trajectory (IDT)-based HMR, converse3D (C3D)-based HMR, and CNN-based HMR. Meanwhile, the feature recognition rate of the proposed KELM-MFF-based HMR for the color dimension is higher than the time dimension, by up to 24%. Besides, the proposed KELM-MFF-based HMR algorithm's recognition rate is 92.4% under early feature fusion and 92.1% under late feature fusion, higher than 91.8 and 90.5% of the SVM-MFF-based HMR. Finally, the proposed KELM-MFF-based HMR algorithm takes 30 and 15 s for training and testing. Therefore, the algorithm designed here can be used to deal with large-scale datasets and capture and recognize micromotions. The research content provides a reference for applying extreme learning machine algorithms in sports competitions.

Deep-E: A Fully-Dense Neural Network for Improving the Elevation Resolution in Linear-Array-Based Photoacoustic Tomography

Linear-array-based photoacoustic tomography has shown broad applications in biomedical research and preclinical imaging. However, the elevational resolution of a linear array is fundamentally limited due to the weak cylindrical focus of the transducer element. While several methods have been proposed to address this issue, they have all handled the problem in a less time-efficient way. In this work, we propose to improve the elevational resolution of a linear array through Deep-E, a fully dense neural network based on U-net. Deep-E exhibits high computational efficiency by converting the three-dimensional problem into a two-dimension problem: it focused on training a model to enhance the resolution along elevational direction by only using the 2D slices in the axial and elevational plane and thereby reducing the computational burden in simulation and training. We demonstrated the efficacy of Deep-E using various datasets, including simulation, phantom, and human subject results. We found that Deep-E could improve elevational resolution by at least four times and recover the object's true size. We envision that Deep-E will have a significant impact in linear-array-based photoacoustic imaging studies by providing high-speed and high-resolution image enhancement.

Advanced Ultrasound and Photoacoustic Imaging in Cardiology

Cardiovascular diseases (CVDs) remain the leading cause of death worldwide. An effective management and treatment of CVDs highly relies on accurate diagnosis of the disease. As the most common imaging technique for clinical diagnosis of the CVDs, US imaging has been intensively explored. Especially with the introduction of deep learning (DL) techniques, US imaging has advanced tremendously in recent years. Photoacoustic imaging (PAI) is one of the most promising new imaging methods in addition to the existing clinical imaging methods. It can characterize different tissue compositions based on optical absorption contrast and thus can assess the functionality of the tissue. This paper reviews some major technological developments in both US (combined with deep learning techniques) and PA imaging in the application of diagnosis of CVDs.

Photoacoustic computed tomography for functional human brain imaging [Invited]

The successes of magnetic resonance imaging and modern optical imaging of human brain function have stimulated the development of complementary modalities that offer molecular specificity, fine spatiotemporal resolution, and sufficient penetration simultaneously. By virtue of its rich optical contrast, acoustic resolution, and imaging depth far beyond the optical transport mean free path (∼1 mm in biological tissues), photoacoustic computed tomography (PACT) offers a promising complementary modality. In this article, PACT for functional human brain imaging is reviewed in its hardware, reconstruction algorithms, in vivo demonstration, and potential roadmap.