Cross-sectional photoacoustic tomography image reconstruction with a multi-curve integration model

Image reconstruction of multispectral sparse sampling photoacoustic tomography based on deep algorithm unrolling

Photoacoustic tomography (PAT), as a novel medical imaging technology, provides structural, functional, and metabolism information of biological tissue in vivo. Sparse Sampling PAT, or SS-PAT, generates images with a smaller number of detectors, yet its image reconstruction is inherently ill-posed. Model-based methods are the state-of-the-art method for SS-PAT image reconstruction, but they require design of complex handcrafted prior. Owing to their ability to derive robust prior from labeled datasets, deep-learning-based methods have achieved great success in solving inverse problems, yet their interpretability is poor. Herein, we propose a novel SS-PAT image reconstruction method based on deep algorithm unrolling (DAU), which integrates the advantages of model-based and deep-learning-based methods. We firstly provide a thorough analysis of DAU for PAT reconstruction. Then, in order to incorporate the structural prior constraint, we propose a nested DAU framework based on plug-and-play Alternating Direction Method of Multipliers (PnP-ADMM) to deal with the sparse sampling problem. Experimental results on numerical simulation, in vivo animal imaging, and multispectral un-mixing demonstrate that the proposed DAU image reconstruction framework outperforms state-of-the-art model-based and deep-learning-based methods.

Model-based quantitative photoacoustic tomography with directional total variation

In photoacoustic tomography (PAT), acoustic inversion aims to recover the spatial distribution of light energy deposition within the imaging object from the signals captured by detectors. To achieve quantitative imaging, optical inversion is further employed to derive absorption coefficient (AC) images. However, limitations such as restricted detection angles and inherent noise lead to substantial artifacts and degradation in the quality of PAT images, consequently affecting the accuracy of optical inversion results. In this study, we propose a directional total variation constrained optical inversion model to reconstruct the AC image. By incorporating anatomy prior information into the optical inversion process, our method can effectively suppress artifacts in AC images while maintaining structural integrity. Simulation, phantom, and in vivo experimental results demonstrate that our method significantly improves the reconstructed AC image quality. Our method provides a reliable foundation for achieving high-quality quantitative PAT imaging.

Deep learning acceleration of iterative model-based light fluence correction for photoacoustic tomography

Photoacoustic tomography (PAT) is a promising imaging technique that can visualize the distribution of chromophores within biological tissue. However, the accuracy of PAT imaging is compromised by light fluence (LF), which hinders the quantification of light absorbers. Currently, model-based iterative methods are used for LF correction, but they require extensive computational resources due to repeated LF estimation based on differential light transport models. To improve LF correction efficiency, we propose to use Fourier neural operator (FNO), a neural network specially designed for estimating partial differential equations, to learn the forward projection of light transport in PAT. Trained using paired finite-element-based LF simulation data, our FNO model replaces the traditional computational heavy LF estimator during iterative correction, such that the correction procedure is considerably accelerated. Simulation and experimental results demonstrate that our method achieves comparable LF correction quality to traditional iterative methods while reducing the correction time by over 30 times.

Adaptive machine learning method for photoacoustic computed tomography based on sparse array sensor data

BACKGROUND AND OBJECTIVE

Photoacoustic computed tomography (PACT) is a non-invasive biomedical imaging technology that has developed rapidly in recent decades, especially has shown potential for small animal studies and early diagnosis of human diseases. To obtain high-quality images, the photoacoustic imaging system needs a high-element-density detector array. However, in practical applications, due to the cost limitation, manufacturing technology, and the system requirement in miniaturization and robustness, it is challenging to achieve sufficient elements and high-quality reconstructed images, which may even suffer from artifacts. Different from the latest machine learning methods based on removing distortions and artifacts to recover high-quality images, this paper proposes an adaptive machine learning method to firstly predict and complement the photoacoustic sensor channel data from sparse array sampling and then reconstruct images through conventional reconstruction algorithms.

METHODS

We develop an adaptive machine learning method to predict and complement the photoacoustic sensor channel data. The model consists of XGBoost and a neural network named SS-net. To handle data sets of different sizes and improve the generalization, a tunable parameter is used to control the weights of XGBoost and SS-net outputs.

RESULTS

The proposed method achieved superior performance as demonstrated by simulation, phantom experiments, and in vivo experiment results. Compared with linear interpolation, XGBoost, CAE, and U-net, the simulation results show that the SSIM value is increased by 12.83%, 6.78%, 21.46%, and 12.33%. Moreover, the median R2 is increased by 34.4%, 8.1%, 28.6%, and 84.1% with the in vivo data.

CONCLUSIONS

This model provides a framework to predict the missed photoacoustic sensor data on a sparse ring-shaped array for PACT imaging and has achieved considerable improvements in reconstructing the objects. Compared with linear interpolation and other deep learning methods qualitatively and quantitatively, our proposed methods can effectively suppress artifacts and improve image quality. The advantage of our methods is that there is no need for preparing a large number of images as the training dataset, and the data for training is directly from the sensors. It has the potential to be applied to a wide range of photoacoustic imaging detector arrays for low-cost and user-friendly clinical applications.

Learning spatially variant degradation for unsupervised blind photoacoustic tomography image restoration

Photoacoustic tomography (PAT) images contain inherent distortions due to the imaging system and heterogeneous tissue properties. Improving image quality requires the removal of these system distortions. While model-based approaches and data-driven techniques have been proposed for PAT image restoration, achieving accurate and robust image recovery remains challenging. Recently, deep-learning-based image deconvolution approaches have shown promise for image recovery. However, PAT imaging presents unique challenges, including spatially varying resolution and the absence of ground truth data. Consequently, there is a pressing need for a novel learning strategy specifically tailored for PAT imaging. Herein, we propose a configurable network model named Deep hybrid Image-PSF Prior (DIPP) that builds upon the physical image degradation model of PAT. DIPP is an unsupervised and deeply learned network model that aims to extract the ideal PAT image from complex system degradation. Our DIPP framework captures the degraded information solely from the acquired PAT image, without relying on ground truth or labeled data for network training. Additionally, we can incorporate the experimentally measured Point Spread Functions (PSFs) of the specific PAT system as a reference to further enhance performance. To evaluate the algorithm's effectiveness in addressing multiple degradations in PAT, we conduct extensive experiments using simulation images, publicly available datasets, phantom images, and in vivo small animal imaging data. Comparative analyses with classical analytical methods and state-of-the-art deep learning models demonstrate that our DIPP approach achieves significantly improved restoration results in terms of image details and contrast.

In vivo co-registered hybrid-contrast imaging by successive photoacoustic tomography and magnetic resonance imaging

Magnetic resonance imaging (MRI) and photoacoustic tomography (PAT) offer two distinct image contrasts. To integrate these two modalities, we present a comprehensive hardware-software solution for the successive acquisition and co-registration of PAT and MRI images in in vivo animal studies. Based on commercial PAT and MRI scanners, our solution includes a 3D-printed dual-modality imaging bed, a 3-D spatial image co-registration algorithm with dual-modality markers, and a robust modality switching protocol for in vivo imaging studies. Using the proposed solution, we successfully demonstrated co-registered hybrid-contrast PAT-MRI imaging that simultaneously displays multi-scale anatomical, functional and molecular characteristics on healthy and cancerous living mice. Week-long longitudinal dual-modality imaging of tumor development reveals information on size, border, vascular pattern, blood oxygenation, and molecular probe metabolism of the tumor micro-environment at the same time. The proposed methodology holds promise for a wide range of pre-clinical research applications that benefit from the PAT-MRI dual-modality image contrast.

Pixel-wise reconstruction of tissue absorption coefficients in photoacoustic tomography using a non-segmentation iterative method

In Photoacoustic Tomography (PAT), the acquired image represents a light energy deposition map of the imaging object. For quantitative imaging, the PAT image is converted into an absorption coefficient ( μ a ) map by dividing the light fluence (LF). Previous methods usually assume a uniform tissue μ a distribution, and consequently degrade the LF correction results. Here, we propose a simple method to reconstruct the pixel-wise μ a map. Our method is based on a non-segmentation-based iterative algorithm, which alternately optimizes the LF distribution and the μ a map. Using simulation data, as well as phantom and animal data, we implemented our algorithm and compared it to segmentation-based correction methods. The results show that our method can obtain accurate estimation of the LF distribution and therefore improve the image quality and feature visibility of the μ a map. Our method may facilitate efficient calculation of the concentration distributions of endogenous and exogenous agents in vivo.

MRI Information-Based Correction and Restoration of Photoacoustic Tomography

As an emerging molecular imaging modality, Photoacoustic Tomography (PAT) is capable of mapping tissue physiological metabolism and exogenous contrast agent information with high specificity. Due to its ultrasonic detection mechanism, the precise localization of targeted lesions has long been a challenge for PAT imaging. The poor soft-tissue contrast of the PAT image makes this process difficult and inaccurate. To meet this challenge, in this study, we first make use of the rich and clear structural information brought about by another advanced imaging modality, Magnetic Resonance Imaging (MRI), to assist organ segmentation and correct for the light fluence attenuation of PAT. We demonstrate improved feature visibility and enhanced localization of endogenous and exogenous agents in the fluence corrected PAT images. Compared with PAT-based methods, the contrast-to-noise ratio (CNR) of our MRI-assisted method increases by 29.1% in live animal experiments. Furthermore, we show that the co-registered MRI image can also be incorporated into PAT image restoration, and achieves improved anatomical landscape and soft-tissue contrast (CNR increased by 25.36%) while preserving similar spatial resolution. This PAT-MRI combination provides excellent structural, functional and molecular images of the subject, and may enable more comprehensive analysis of various preclinical research applications.

Multispectral interlaced sparse sampling photoacoustic tomography based on directional total variation

BACKGROUND AND OBJECTIVE

Photoacoustic tomography (PAT) is capable of obtaining cross-sectional images of small animals that represent the optical absorption of biological tissues. The multispectral Interlaced Sparse Sampling PAT, or ISS-PAT, is a previously proposed PAT imaging method that offered high quality images with much sparser transducer angular coverage. Although it provides superior imaging performance, the original ISS-PAT method suffered from a heavy computation burden, which hinders its practical application.

METHODS

Here, we propose a new regularization scheme based on the directional total variation (dTV) for ISS-PAT. This method efficiently imposes the structural information by considering both the edge position and direction information of the anatomical prior image in ISS-PAT. It does not require image segmentation, and can be conveniently solved by a modified alternating direction of multipliers (ADMM) algorithm.

RESULTS

We perform simulation, tissue mimicking phantom and in vivo small animal experiments to evaluate the proposed scheme. The reconstructed PAT images showed image quality and spectral un-mixing accuracy close to those obtained by non-local means based ISS-PAT, but with much shorter image reconstruction time. For a 1/6 sparse sampling rate, the average efficiency improvement is nearly 16-folds.

CONCLUSIONS

The experimental results demonstrate the feasibility of the dTV regularization scheme for ISS-PAT. Its efficient image reconstruction performance facilitates the potential of the hardware realization and practical applications of the ISS-PAT.

Automatic 3-D segmentation and volumetric light fluence correction for photoacoustic tomography based on optimal 3-D graph search

Preclinical imaging with photoacoustic tomography (PAT) has attracted wide attention in recent years since it is capable of providing molecular contrast with deep imaging depth. The automatic extraction and segmentation of the animal in PAT images is crucial for improving image analysis efficiency and enabling advanced image post-processing, such as light fluence (LF) correction for quantitative PAT imaging. Previous automatic segmentation methods are mostly two-dimensional approaches, which failed to conserve the 3-D surface continuity because the image slices were processed separately. This discontinuity problem further hampers LF correction, which, ideally, should be carried out in 3-D due to spatially diffused illumination. Here, to solve these problems, we propose a volumetric auto-segmentation method for small animal PAT imaging based on the 3-D optimal graph search (3-D GS) algorithm. The 3-D GS algorithm takes into account the relation among image slices by constructing a 3-D node-weighted directed graph, and thus ensures surface continuity. In view of the characteristics of PAT images, we improve the original 3-D GS algorithm on graph construction, solution guidance and cost assignment, such that the accuracy and smoothness of the segmented animal surface were guaranteed. We tested the performance of the proposed method by conducting in vivo nude mice imaging experiments with a commercial preclinical cross-sectional PAT system. The results showed that our method successfully retained the continuous global surface structure of the whole 3-D animal body, as well as smooth local subcutaneous tumor boundaries at different development stages. Moreover, based on the 3-D segmentation result, we were able to simulate volumetric LF distribution of the entire animal body and obtained LF corrected PAT images with enhanced structural visibility and uniform image intensity.