Context encoding enables machine learning-based quantitative photoacoustics

Unsupervised disentanglement strategy for mitigating artifact in photoacoustic tomography under extremely sparse view

Traditional methods under sparse view for reconstruction of photoacoustic tomography (PAT) often result in significant artifacts. Here, a novel image to image transformation method based on unsupervised learning artifact disentanglement network (ADN), named PAT-ADN, was proposed to address the issue. This network is equipped with specialized encoders and decoders that are responsible for encoding and decoding the artifacts and content components of unpaired images, respectively. The performance of the proposed PAT-ADN was evaluated using circular phantom data and the animal in vivo experimental data. The results demonstrate that PAT-ADN exhibits excellent performance in effectively removing artifacts. In particular, under extremely sparse view (e.g., 16 projections), structural similarity index and peak signal-to-noise ratio are improved by ∼188 % and ∼85 % in in vivo experimental data using the proposed method compared to traditional reconstruction methods. PAT-ADN improves the imaging performance of PAT, opening up possibilities for its application in multiple domains.

Deep learning acceleration of iterative model-based light fluence correction for photoacoustic tomography

Photoacoustic tomography (PAT) is a promising imaging technique that can visualize the distribution of chromophores within biological tissue. However, the accuracy of PAT imaging is compromised by light fluence (LF), which hinders the quantification of light absorbers. Currently, model-based iterative methods are used for LF correction, but they require extensive computational resources due to repeated LF estimation based on differential light transport models. To improve LF correction efficiency, we propose to use Fourier neural operator (FNO), a neural network specially designed for estimating partial differential equations, to learn the forward projection of light transport in PAT. Trained using paired finite-element-based LF simulation data, our FNO model replaces the traditional computational heavy LF estimator during iterative correction, such that the correction procedure is considerably accelerated. Simulation and experimental results demonstrate that our method achieves comparable LF correction quality to traditional iterative methods while reducing the correction time by over 30 times.

Oxygenation heterogeneity facilitates spatiotemporal flow pattern visualization inside human blood vessels using photoacoustic computed tomography

Hemodynamics can be explored through various biomedical imaging techniques. However, observing transient spatiotemporal variations in the saturation of oxygen (sO2) within human blood vessels proves challenging with conventional methods. In this study, we employed photoacoustic computed tomography (PACT) to reconstruct the evolving spatiotemporal patterns in a human vein. Through analysis of the multi-wavelength photoacoustic (PA) spectrum, we illustrated the dynamic distribution within blood vessels. Additionally, we computationally rendered the dynamic process of venous blood flowing into the major vein and entering a branching vessel. Notably, we successfully recovered, in real time, the parabolic wavefront profile of laminar flow inside a deep vein in vivo-a first-time achievement. While the study is preliminary, the demonstrated capability of dynamic sO2 imaging holds promise for new applications in biology and medicine.

Ultrasound-guided Photoacoustic image Annotation Toolkit in MATLAB (PHANTOM) for preclinical applications

Depth-dependent fluence-compensation in photoacoustic (PA) imaging is paramount for accurate quantification of chromophores from deep tissues. Here we present a user-friendly toolkit named PHANTOM (PHotoacoustic ANnotation TOolkit for MATLAB) that includes a graphical interface and assists in the segmentation of ultrasound-guided PA images. We modelled the light source configuration with Monte Carlo eXtreme and utilized 3D segmented tissues from ultrasound to generate fluence maps to depth compensate PA images. The methodology was used to analyze PA images of phantoms with varying blood oxygenation and results were validated with oxygen electrode measurements. Two preclinical models, a subcutaneous tumor and a calcified placenta, were imaged and fluence-compensated using the PHANTOM toolkit and the results were verified with immunohistochemistry. The PHANTOM toolkit provides scripts and auxiliary functions to enable biomedical researchers not specialized in optical imaging to apply fluence correction to PA images, enhancing accessibility of quantitative PAI for researchers in various fields.

Efficient Photoacoustic Image Synthesis with Deep Learning

Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.

Photoacoustic imaging with limited sampling: a review of machine learning approaches

Photoacoustic imaging combines high optical absorption contrast and deep acoustic penetration, and can reveal structural, molecular, and functional information about biological tissue non-invasively. Due to practical restrictions, photoacoustic imaging systems often face various challenges, such as complex system configuration, long imaging time, and/or less-than-ideal image quality, which collectively hinder their clinical application. Machine learning has been applied to improve photoacoustic imaging and mitigate the otherwise strict requirements in system setup and data acquisition. In contrast to the previous reviews of learned methods in photoacoustic computed tomography (PACT), this review focuses on the application of machine learning approaches to address the limited spatial sampling problems in photoacoustic imaging, specifically the limited view and undersampling issues. We summarize the relevant PACT works based on their training data, workflow, and model architecture. Notably, we also introduce the recent limited sampling works on the other major implementation of photoacoustic imaging, i.e., photoacoustic microscopy (PAM). With machine learning-based processing, photoacoustic imaging can achieve improved image quality with modest spatial sampling, presenting great potential for low-cost and user-friendly clinical applications.

Optical Encoding Model Based on Orbital Angular Momentum Powered by Machine Learning

Based on orbital angular momentum (OAM) properties of Laguerre-Gaussian beams LG(p,ℓ), a robust optical encoding model for efficient data transmission applications is designed. This paper presents an optical encoding model based on an intensity profile generated by a coherent superposition of two OAM-carrying Laguerre-Gaussian modes and a machine learning detection method. In the encoding process, the intensity profile for data encoding is generated based on the selection of p and ℓ indices, while the decoding process is performed using a support vector machine (SVM) algorithm. Two different decoding models based on an SVM algorithm are tested to verify the robustness of the optical encoding model, finding a BER =10-9 for 10.2 dB of signal-to-noise ratio in one of the SVM models.

Deep and Domain Transfer Learning Aided Photoacoustic Microscopy: Acoustic Resolution to Optical Resolution

Acoustic resolution photoacoustic micros- copy (AR-PAM) can achieve deeper imaging depth in biological tissue, with the sacrifice of imaging resolution compared with optical resolution photoacoustic microscopy (OR-PAM). Here we aim to enhance the AR-PAM image quality towards OR-PAM image, which specifically includes the enhancement of imaging resolution, restoration of micro-vasculatures, and reduction of artifacts. To address this issue, a network (MultiResU-Net) is first trained as generative model with simulated AR-OR image pairs, which are synthesized with physical transducer model. Moderate enhancement results can already be obtained when applying this model to in vivo AR imaging data. Nevertheless, the perceptual quality is unsatisfactory due to domain shift. Further, domain transfer learning technique under generative adversarial network (GAN) framework is proposed to drive the enhanced image's manifold towards that of real OR image. In this way, perceptually convincing AR to OR enhancement result is obtained, which can also be supported by quantitative analysis. Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values are significantly increased from 14.74 dB to 19.01 dB and from 0.1974 to 0.2937, respectively, validating the improvement of reconstruction correctness and overall perceptual quality. The proposed algorithm has also been validated across different imaging depths with experiments conducted in both shallow and deep tissue. The above AR to OR domain transfer learning with GAN (AODTL-GAN) framework has enabled the enhancement target with limited amount of matched in vivo AR-OR imaging data.

Spectral crosstalk in photoacoustic computed tomography

Multispectral photoacoustic (PA) imaging faces two major challenges: the spectral coloring effect, which has been studied extensively as an optical inversion problem, and the spectral crosstalk, which is basically a result of non-ideal acoustic inversion. So far, there is no systematic work to analyze the spectral crosstalk because acoustic inversion and spectroscopic measurement are always treated as decoupled. In this work, we theorize and demonstrate through a series of simulations and experiments how imperfect acoustic inversion induces inaccurate PA spectrum measurement. We provide detailed analysis to elucidate how different factors, including limited bandwidth, limited view, light attenuation, out-of-plane signal, and image reconstruction schemes, conspire to render the measured PA spectrum inaccurate. We found that the model-based reconstruction outperforms universal back-projection in suppressing the spectral crosstalk in some cases.

Quantitative endoscopic photoacoustic tomography using a convolutional neural network

Endoscopic photoacoustic tomography (EPAT) is a catheter-based hybrid imaging modality capable of providing structural and functional information of biological luminal structures, such as coronary arterial vessels and the digestive tract. The recovery of the optical properties of the imaged tissue from acoustic measurements achieved by optical inversion is essential for implementing quantitative EPAT (qEPAT). In this paper, a convolutional neural network (CNN) based on deep gradient descent is developed for qEPAT. The network enables the reconstruction of images representing the spatially varying absorption coefficient in cross-sections of the tubular structures from limited measurement data. The forward operator reflecting the mapping from the absorption coefficient to the optical deposition due to pulsed irradiation is embedded into the network training. The network parameters are optimized layer by layer through the deep gradient descent mechanism using the numerically simulated data. The operation processes of the forward operator and its adjoint operator are separated from the network training. The trained network outputs an image representing the distribution of absorption coefficients by inputting an image that represents the optical deposition. The method has been tested with computer-generated phantoms mimicking coronary arterial vessels containing various tissue types. Results suggest that the structural similarity of the images reconstructed by our method is increased by about 10% in comparison with the non-learning method based on error minimization in the case of the same measuring view.

SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics

SIGNIFICANCE

Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings.

AIM

To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards.

APPROACH

SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models.

RESULTS

To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations.

CONCLUSIONS

SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.

Artifact removal in photoacoustic tomography with an unsupervised method

Photoacoustic tomography (PAT) is an emerging biomedical imaging technology that can realize high contrast imaging with a penetration depth of the acoustic. Recently, deep learning (DL) methods have also been successfully applied to PAT for improving the image reconstruction quality. However, the current DL-based PAT methods are implemented by the supervised learning strategy, and the imaging performance is dependent on the available ground-truth data. To overcome the limitation, this work introduces a new image domain transformation method based on cyclic generative adversarial network (CycleGAN), termed as PA-GAN, which is used to remove artifacts in PAT images caused by the use of the limited-view measurement data in an unsupervised learning way. A series of data from phantom and in vivo experiments are used to evaluate the performance of the proposed PA-GAN. The experimental results show that PA-GAN provides a good performance in removing artifacts existing in photoacoustic tomographic images. In particular, when dealing with extremely sparse measurement data (e.g., 8 projections in circle phantom experiments), higher imaging performance is achieved by the proposed unsupervised PA-GAN, with an improvement of ∼14% in structural similarity (SSIM) and ∼66% in peak signal to noise ratio (PSNR), compared with the supervised-learning U-Net method. With an increasing number of projections (e.g., 128 projections), U-Net, especially FD U-Net, shows a slight improvement in artifact removal capability, in terms of SSIM and PSNR. Furthermore, the computational time obtained by PA-GAN and U-Net is similar (∼60 ms/frame), once the network is trained. More importantly, PA-GAN is more flexible than U-Net that allows the model to be effectively trained with unpaired data. As a result, PA-GAN makes it possible to implement PAT with higher flexibility without compromising imaging performance.

Multiple illumination learned spectral decoloring for quantitative optoacoustic oximetry imaging

SIGNIFICANCE

Quantitative measurement of blood oxygen saturation (sO2) with optoacoustic (OA) imaging is one of the most sought after goals of quantitative OA imaging research due to its wide range of biomedical applications.

AIM

A method for accurate and applicable real-time quantification of local sO2 with OA imaging.

APPROACH

We combine multiple illumination (MI) sensing with learned spectral decoloring (LSD). We train LSD feedforward neural networks and random forests on Monte Carlo simulations of spectrally colored absorbed energy spectra, to apply the trained models to real OA measurements. We validate our combined MI-LSD method on a highly reliable, reproducible, and easily scalable phantom model, based on copper and nickel sulfate solutions.

RESULTS

With this sulfate model, we see a consistently high estimation accuracy using MI-LSD, with median absolute estimation errors of 2.5 to 4.5 percentage points. We further find fewer outliers in MI-LSD estimates compared with LSD. Random forest regressors outperform previously reported neural network approaches.

CONCLUSIONS

Random forest-based MI-LSD is a promising method for accurate quantitative OA oximetry imaging.

Deep learning for biomedical photoacoustic imaging: A review

Photoacoustic imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.

Heart, Eye, and Artificial Intelligence: A Review

Heart disease continues to be the leading cause of death in the USA. Deep learning-based artificial intelligence (AI) methods have become increasingly common in studying the various factors involved in cardiovascular disease. The usage of retinal scanning techniques to diagnose retinal diseases, such as diabetic retinopathy, age-related macular degeneration, glaucoma and others, using fundus photographs and optical coherence tomography angiography (OCTA) has been extensively documented. Researchers are now looking to combine the power of AI with the non-invasive ease of retinal scanning to examine the workings of the heart and predict changes in the macrovasculature based on microvascular features and function. In this review, we summarize the current state of the field in using retinal imaging to diagnose cardiovascular issues and other diseases.

Role of blood oxygenation saturation in ovarian cancer diagnosis using multi-spectral photoacoustic tomography

In photoacoustic tomography (PAT), a tunable laser typically illuminates the tissue at multiple wavelengths, and the received photoacoustic waves are used to form functional images of relative total haemoglobin (rHbT) and blood oxygenation saturation (%sO2 ). Due to measurement errors, the estimation of these parameters can be challenging, especially in clinical studies. In this study, we use a multi-pixel method to smooth the measurements before calculating rHbT and %sO2 . We first perform phantom studies using blood tubes of calibrated %sO2 to evaluate the accuracy of our %sO2 estimation. We conclude by presenting diagnostic results from PAT of 33 patients with 51 ovarian masses imaged by our co-registered PAT and ultrasound system. The ovarian masses were divided into malignant and benign/normal groups. Functional maps of rHbT and %sO2 and their histograms as well as spectral features were calculated using the PAT data from all ovaries in these two groups. Support vector machine models were trained on different combinations of the significant features. The area under ROC (AUC) of 0.93 (0.95%CI: 0.90-0.96) on the testing data set was achieved by combining mean %sO2 , a spectral feature, and the score of the study radiologist.

Deep learning in photoacoustic imaging: a review

SIGNIFICANCE

Photoacoustic (PA) imaging can provide structural, functional, and molecular information for preclinical and clinical studies. For PA imaging (PAI), non-ideal signal detection deteriorates image quality, and quantitative PAI (QPAI) remains challenging due to the unknown light fluence spectra in deep tissue. In recent years, deep learning (DL) has shown outstanding performance when implemented in PAI, with applications in image reconstruction, quantification, and understanding.

AIM

We provide (i) a comprehensive overview of the DL techniques that have been applied in PAI, (ii) references for designing DL models for various PAI tasks, and (iii) a summary of the future challenges and opportunities.

APPROACH

Papers published before November 2020 in the area of applying DL in PAI were reviewed. We categorized them into three types: image understanding, reconstruction of the initial pressure distribution, and QPAI.

RESULTS

When applied in PAI, DL can effectively process images, improve reconstruction quality, fuse information, and assist quantitative analysis.

CONCLUSION

DL has become a powerful tool in PAI. With the development of DL theory and technology, it will continue to boost the performance and facilitate the clinical translation of PAI.

Method for parametric imaging of attenuation by intravascular optical coherence tomography

Catheter-based intravascular optical coherence tomography (IVOCT) is a powerful imaging modality for visualization of atherosclerosis with high resolution. Quantitative characterization of various tissue types by attenuation coefficient (AC) extraction has been proven to be a potentially significant application of OCT attenuation imaging. However, existing methods for AC extraction from OCT suffer from the challenge of variability in complex tissue types in IVOCT pullback data such as healthy vessel wall, mixed atherosclerotic plaques, plaques with a single component and stent struts, etc. This challenge leads to the ineffectiveness in the tissue differentiation by AC representation based on single scattering model of OCT signal. In this paper, we propose a novel method based on multiple scattering model for parametric imaging of optical attenuation by AC retrieval from IVOCT images conventionally acquired during cardiac catheterization. The OCT signal characterized by the AC is physically modeled by Monte Carlo simulation. Then, the pixel-wise AC retrieval is achieved by iteratively minimizing an error function regarding the modeled and measured backscattered signal. This method provides a general scheme for AC extraction from IVOCT without the premise of complete attenuation of the incident light through the imaging depths. Results of computer-simulated and clinical images demonstrate that the method can avoid overestimation at the end of the depth profile in comparison with the approaches based on the depth-resolved (DR) model. The energy error depth and structural similarity are improved by about 30% and 10% respectively compared with DR. It provides a useful way to differentiate and characterize arterial tissue types in IVOCT images.

Learned spectral decoloring enables photoacoustic oximetry

The ability of photoacoustic imaging to measure functional tissue properties, such as blood oxygenation sO[Formula: see text], enables a wide variety of possible applications. sO[Formula: see text] can be computed from the ratio of oxyhemoglobin HbO[Formula: see text] and deoxyhemoglobin Hb, which can be distuinguished by multispectral photoacoustic imaging due to their distinct wavelength-dependent absorption. However, current methods for estimating sO[Formula: see text] yield inaccurate results in realistic settings, due to the unknown and wavelength-dependent influence of the light fluence on the signal. In this work, we propose learned spectral decoloring to enable blood oxygenation measurements to be inferred from multispectral photoacoustic imaging. The method computes sO[Formula: see text] pixel-wise, directly from initial pressure spectra [Formula: see text], which represent initial pressure values at a fixed spatial location [Formula: see text] over all recorded wavelengths [Formula: see text]. The method is compared to linear unmixing approaches, as well as pO[Formula: see text] and blood gas analysis reference measurements. Experimental results suggest that the proposed method is able to obtain sO[Formula: see text] estimates from multispectral photoacoustic measurements in silico, in vitro, and in vivo.

Compensating for visibility artefacts in photoacoustic imaging with a deep learning approach providing prediction uncertainties

Conventional photoacoustic imaging may suffer from the limited view and bandwidth of ultrasound transducers. A deep learning approach is proposed to handle these problems and is demonstrated both in simulations and in experiments on a multi-scale model of leaf skeleton. We employed an experimental approach to build the training and the test sets using photographs of the samples as ground truth images. Reconstructions produced by the neural network show a greatly improved image quality as compared to conventional approaches. In addition, this work aimed at quantifying the reliability of the neural network predictions. To achieve this, the dropout Monte-Carlo procedure is applied to estimate a pixel-wise degree of confidence on each predicted picture. Last, we address the possibility to use transfer learning with simulated data in order to drastically limit the size of the experimental dataset.

Real-time interleaved spectroscopic photoacoustic and ultrasound (PAUS) scanning with simultaneous fluence compensation and motion correction

For over two decades photoacoustic imaging has been tested clinically, but successful human trials have been limited. To enable quantitative clinical spectroscopy, the fundamental issues of wavelength-dependent fluence variations and inter-wavelength motion must be overcome. Here we propose a real-time, spectroscopic photoacoustic/ultrasound (PAUS) imaging approach using a compact, 1-kHz rate wavelength-tunable laser. Instead of illuminating tissue over a large area, the fiber-optic delivery system surrounding an US array sequentially scans a narrow laser beam, with partial PA image reconstruction for each laser pulse. The final image is then formed by coherently summing partial images. This scheme enables (i) automatic compensation for wavelength-dependent fluence variations in spectroscopic PA imaging and (ii) motion correction of spectroscopic PA frames using US speckle tracking in real-time systems. The 50-Hz video rate PAUS system is demonstrated in vivo using a murine model of labelled drug delivery.

Oxygen Saturation Imaging Using LED-Based Photoacoustic System

Oxygen saturation imaging has potential in several preclinical and clinical applications. Dual-wavelength LED array-based photoacoustic oxygen saturation imaging can be an affordable solution in this case. For the translation of this technology, there is a need to improve its accuracy and validate it against ground truth methods. We propose a fluence compensated oxygen saturation imaging method, utilizing structural information from the ultrasound image, and prior knowledge of the optical properties of the tissue with a Monte-Carlo based light propagation model for the dual-wavelength LED array configuration. We then validate the proposed method with oximeter measurements in tissue-mimicking phantoms. Further, we demonstrate in vivo imaging on small animal and a human subject. We conclude that the proposed oxygen saturation imaging can be used to image tissue at a depth of 6–8 mm in both preclinical and clinical applications.

A Deep Learning Approach to Photoacoustic Wavefront Localization in Deep-Tissue Medium

Optical photons undergo strong scattering when propagating beyond 1-mm deep inside biological tissue. Finding the origin of these diffused optical wavefronts is a challenging task. Breaking through the optical diffusion limit, photoacoustic (PA) imaging (PAI) provides high-resolution and label-free images of human vasculature with high contrast due to the optical absorption of hemoglobin. In real-time PAI, an ultrasound transducer array detects PA signals, and B-mode images are formed by delay-and-sum or frequency-domain beamforming. Fundamentally, the strength of a PA signal is proportional to the local optical fluence, which decreases with the increasing depth due to depth-dependent optical attenuation. This limits the visibility of deep-tissue vasculature or other light-absorbing PA targets. To address this practical challenge, an encoder-decoder convolutional neural network architecture was constructed with custom modules and trained to identify the origin of the PA wavefronts inside an optically scattering deep-tissue medium. A comprehensive ablation study provides strong evidence that each module improves the localization accuracy. The network was trained on model-based simulated PA signals produced by 16 240 blood-vessel targets subjected to both optical scattering and Gaussian noise. Test results on 4600 simulated and five experimental PA signals collected under various scattering conditions show that the network can localize the targets with a mean error less than 30 microns (standard deviation 20.9 microns) for targets below 40-mm imaging depth and 1.06 mm (standard deviation 2.68 mm) for targets at a depth between 40 and 60 mm. The proposed work has broad applications such as diffused optical wavefront shaping, circulating melanoma cell detection, and real-time vascular surgeries (e.g., deep-vein thrombosis).

Deep Learning-Based Spectral Unmixing for Optoacoustic Imaging of Tissue Oxygen Saturation

Label free imaging of oxygenation distribution in tissues is highly desired in numerous biomedical applications, but is still elusive, in particular in sub-epidermal measurements. Eigenspectra multispectral optoacoustic tomography (eMSOT) and its Bayesian-based implementation have been introduced to offer accurate label-free blood oxygen saturation (sO2) maps in tissues. The method uses the eigenspectra model of light fluence in tissue to account for the spectral changes due to the wavelength dependent attenuation of light with tissue depth. eMSOT relies on the solution of an inverse problem bounded by a number of ad hoc hand-engineered constraints. Despite the quantitative advantage offered by eMSOT, both the non-convex nature of the optimization problem and the possible sub-optimality of the constraints may lead to reduced accuracy. We present herein a neural network architecture that is able to learn how to solve the inverse problem of eMSOT by directly regressing from a set of input spectra to the desired fluence values. The architecture is composed of a combination of recurrent and convolutional layers and uses both spectral and spatial features for inference. We train an ensemble of such networks using solely simulated data and demonstrate how this approach can improve the accuracy of sO2 computation over the original eMSOT, not only in simulations but also in experimental datasets obtained from blood phantoms and small animals (mice) in vivo. The use of a deep-learning approach in optoacoustic sO2 imaging is confirmed herein for the first time on ground truth sO2 values experimentally obtained in vivo and ex vivo.

End-to-end Res-Unet based reconstruction algorithm for photoacoustic imaging

Recently, deep neural networks have attracted great attention in photoacoustic imaging (PAI). In PAI, reconstructing the initial pressure distribution from acquired photoacoustic (PA) signals is a typically inverse problem. In this paper, an end-to-end Unet with residual blocks (Res-Unet) is designed and trained to solve the inverse problem in PAI. The performance of the proposed algorithm is explored and analyzed by comparing a recent model-resolution-based regularization algorithm (MRR) with numerical and physical phantom experiments. The improvement obtained in the reconstructed images was more than 95% in pearson correlation and 39% in peak signal-to-noise ratio in comparison to the MRR. The Res-Unet also achieved superior performance over the state-of-the-art Unet++ architecture by more than 18% in PSNR in simulation experiments.

Correction of wavelength-dependent laser fluence in swept-beam spectroscopic photoacoustic imaging with a hand-held probe

Recently, we demonstrated an integrated photoacoustic (PA) and ultrasound (PAUS) system using a kHz-rate wavelength-tunable laser and a swept-beam delivery approach. It irradiates a medium using a narrow laser beam swept at high repetition rate (∼1 kHz) over the desired imaging area, in contrast to the conventional PA approach using broad-beam illumination at a low repetition rate (10-50 Hz). Here, we present a method to correct the wavelength-dependent fluence distribution and demonstrate its performance in phantom studies using a conventional limited view/bandwidth hand-held US probe. We adopted analytic fluence models, extending diffusion theory for the case of a pencil beam obliquely incident on an optically homogenous turbid medium, and developed a robust method to estimate fluence attenuation in the medium using PA measurements acquired from multiple fiber-irradiation positions swept at a kHz rate. We conducted comprehensive simulation tests and phantom studies using well-known contrast-agents to validate the reliability of the fluence model and its spectral corrections.

Evaluation of Fluence Correction Algorithms in Multispectral Photoacoustic Imaging

Multispectral photoacoustic imaging (MPAI) is a promising emerging diagnostic technology, but fluence artifacts can degrade device performance. Our goal was to develop well-validated phantom-based test methods for evaluating and comparing MPAI fluence correction algorithms, including a heuristic diffusion approximation, Monte Carlo simulations, and an algorithm we developed based on novel application of the diffusion dipole model (DDM). Phantoms simulated a range of breast-mimicking optical properties and contained channels filled with chromophore solutions (ink, hemoglobin, or copper sulfate) or connected to a previously developed blood flow circuit providing tunable oxygen saturation (SO₂). The DDM algorithm achieved similar spectral recovery and SO₂ measurement accuracy to Monte Carlo-based corrections with lower computational cost, potentially providing an accurate, real-time correction approach. Algorithms were sensitive to optical property uncertainty, but error was minimized by matching phantom albedo. The developed test methods may provide a foundation for standardized assessment of MPAI fluence correction algorithm performance.

Toward accurate quantitative photoacoustic imaging: learning vascular blood oxygen saturation in three dimensions

SIGNIFICANCE

Two-dimensional (2-D) fully convolutional neural networks have been shown capable of producing maps of sO2 from 2-D simulated images of simple tissue models. However, their potential to produce accurate estimates in vivo is uncertain as they are limited by the 2-D nature of the training data when the problem is inherently three-dimensional (3-D), and they have not been tested with realistic images.

AIM

To demonstrate the capability of deep neural networks to process whole 3-D images and output 3-D maps of vascular sO2 from realistic tissue models/images.

APPROACH

Two separate fully convolutional neural networks were trained to produce 3-D maps of vascular blood oxygen saturation and vessel positions from multiwavelength simulated images of tissue models.

RESULTS

The mean of the absolute difference between the true mean vessel sO2 and the network output for 40 examples was 4.4% and the standard deviation was 4.5%.

CONCLUSIONS

3-D fully convolutional networks were shown capable of producing accurate sO2 maps using the full extent of spatial information contained within 3-D images generated under conditions mimicking real imaging scenarios. We demonstrate that networks can cope with some of the confounding effects present in real images such as limited-view artifacts and have the potential to produce accurate estimates in vivo.

Quantitative PA tomography of high resolution 3-D images: Experimental validation in a tissue phantom

Quantitative photoacoustic tomography aims to recover the spatial distribution of absolute chromophore concentrations and their ratios from deep tissue, high-resolution images. In this study, a model-based inversion scheme based on a Monte-Carlo light transport model is experimentally validated on 3-D multispectral images of a tissue phantom acquired using an all-optical scanner with a planar detection geometry. A calibrated absorber allowed scaling of the measured data during the inversion, while an acoustic correction method was employed to compensate the effects of limited view detection. Chromophore- and fluence-dependent step sizes and Adam optimization were implemented to achieve rapid convergence. High resolution 3-D maps of absolute concentrations and their ratios were recovered with high accuracy. Potential applications of this method include quantitative functional and molecular photoacoustic tomography of deep tissue in preclinical and clinical studies.

Listening to tissues with new light: recent technological advances in photoacoustic imaging

Photoacoustic tomography (PAT), or optoacoustic tomography, has achieved remarkable progress in the past decade, benefiting from the joint developments in optics, acoustics, chemistry, computing and mathematics. Unlike pure optical or ultrasound imaging, PAT can provide unique optical absorption contrast as well as widely scalable spatial resolution, penetration depth and imaging speed. Moreover, PAT has inherent sensitivity to tissue's functional, molecular, and metabolic state. With these merits, PAT has been applied in a wide range of life science disciplines, and has enabled biomedical research unattainable by other imaging methods. This Review article aims at introducing state-of-the-art PAT technologies and their representative applications. The focus is on recent technological breakthroughs in structural, functional, molecular PAT, including super-resolution imaging, real-time small-animal whole-body imaging, and high-sensitivity functional/molecular imaging. We also discuss the remaining challenges in PAT and envisioned opportunities.

Three-dimensional quantitative photoacoustic tomography using an adjoint radiance Monte Carlo model and gradient descent

Quantitative photoacoustic tomography aims to recover maps of the local concentrations of tissue chromophores from multispectral images. While model-based inversion schemes are promising approaches, major challenges to their practical implementation include the unknown fluence distribution and the scale of the inverse problem. We describe an inversion scheme based on a radiance Monte Carlo model and an adjoint-assisted gradient optimization that incorporates fluence-dependent step sizes and adaptive moment estimation. The inversion is shown to recover absolute chromophore concentrations, blood oxygen saturation, and the Grüneisen parameter from in silico three-dimensional phantom images for different radiance approximations. The scattering coefficient is assumed to be homogeneous and known a priori.

Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks

PURPOSE

Optical imaging is evolving as a key technique for advanced sensing in the operating room. Recent research has shown that machine learning algorithms can be used to address the inverse problem of converting pixel-wise multispectral reflectance measurements to underlying tissue parameters, such as oxygenation. Assessment of the specific hardware used in conjunction with such algorithms, however, has not properly addressed the possibility that the problem may be ill-posed.

METHODS

We present a novel approach to the assessment of optical imaging modalities, which is sensitive to the different types of uncertainties that may occur when inferring tissue parameters. Based on the concept of invertible neural networks, our framework goes beyond point estimates and maps each multispectral measurement to a full posterior probability distribution which is capable of representing ambiguity in the solution via multiple modes. Performance metrics for a hardware setup can then be computed from the characteristics of the posteriors.

RESULTS

Application of the assessment framework to the specific use case of camera selection for physiological parameter estimation yields the following insights: (1) estimation of tissue oxygenation from multispectral images is a well-posed problem, while (2) blood volume fraction may not be recovered without ambiguity. (3) In general, ambiguity may be reduced by increasing the number of spectral bands in the camera.

CONCLUSION

Our method could help to optimize optical camera design in an application-specific manner.

Confidence Estimation for Machine Learning-Based Quantitative Photoacoustics

In medical applications, the accuracy and robustness of imaging methods are of crucial importance to ensure optimal patient care. While photoacoustic imaging (PAI) is an emerging modality with promising clinical applicability, state-of-the-art approaches to quantitative photoacoustic imaging (qPAI), which aim to solve the ill-posed inverse problem of recovering optical absorption from the measurements obtained, currently cannot comply with these high standards. This can be attributed to the fact that existing methods often rely on several simplifying a priori assumptions of the underlying physical tissue properties or cannot deal with realistic noise levels. In this manuscript, we address this issue with a new method for estimating an indicator of the uncertainty of an estimated optical property. Specifically, our method uses a deep learning model to compute error estimates for optical parameter estimations of a qPAI algorithm. Functional tissue parameters, such as blood oxygen saturation, are usually derived by averaging over entire signal intensity-based regions of interest (ROIs). Therefore, we propose to reduce the systematic error of the ROI samples by additionally discarding those pixels for which our method estimates a high error and thus a low confidence. In silico experiments show an improvement in the accuracy of optical absorption quantification when applying our method to refine the ROI, and it might thus become a valuable tool for increasing the robustness of qPAI methods.

Signed Real-Time Delay Multiply and Sum Beamforming for Multispectral Photoacoustic Imaging

Reconstruction of photoacoustic (PA) images acquired with clinical ultrasound transducers is usually performed using the Delay and Sum (DAS) beamforming algorithm. Recently, a variant of DAS, referred to as Delay Multiply and Sum (DMAS) beamforming has been shown to provide increased contrast, signal-to-noise ratio (SNR) and resolution in PA imaging. The main reasons for the use of DAS beamforming in photoacoustics are its simple implementation, real-time capability, and the linearity of the beamformed image to the PA signal. This is crucial for the identification of different chromophores in multispectral PA applications. In contrast, current DMAS implementations are not responsive to the full spectrum of sound frequencies from a photoacoustic source and have not been shown to provide a reconstruction linear to the PA signal. Furthermore, due to its increased computational complexity, DMAS has not been shown yet to work in real-time. Here, we present an open-source real-time variant of the DMAS algorithm, signed DMAS (sDMAS), that ensures linearity in the original PA signal response while providing the increased image quality of DMAS. We show the applicability of sDMAS for multispectral PA applications, in vitro and in vivo. The sDMAS and reference DAS algorithms were integrated in the open-source Medical Imaging Interaction Toolkit (MITK) and are available as real-time capable implementations.

Photoacoustic tomography of blood oxygenation: A mini review

Photoacoustic tomography (PAT) is a hybrid imaging modality that combines rich contrast of optical excitation and deep penetration of ultrasound detection. With its unique optical absorption contrast mechanism, PAT is inherently sensitive to the functional and molecular information of biological tissues, and thus has been widely used in preclinical and clinical studies. Among many functional capabilities of PAT, measuring blood oxygenation is arguably one of the most important applications, and has been widely performed in photoacoustic studies of brain functions, tumor hypoxia, wound healing, and cancer therapy. Yet, the complex optical conditions of biological tissues, especially the strong wavelength-dependent optical attenuation, have long hurdled the PAT measurement of blood oxygenation at depths beyond a few millimeters. A variety of PAT methods have been developed to improve the accuracy of blood oxygenation measurement, using novel laser illumination schemes, oxygen-sensitive fluorescent dyes, comprehensive mathematic models, or prior information provided by complementary imaging modalities. These novel methods have made exciting progress, while several challenges remain. This concise review aims to introduce the recent developments in photoacoustic blood oxygenation measurement, compare each method's advantages and limitations, highlight their representative applications, and discuss the remaining challenges for future advances.