Learned spectral decoloring enables photoacoustic oximetry

Extractor-attention-predictor network for quantitative photoacoustic tomography

Quantitative photoacoustic tomography (qPAT) holds great potential in estimating chromophore concentrations, whereas the involved optical inverse problem, aiming to recover absorption coefficient distributions from photoacoustic images, remains challenging. To address this problem, we propose an extractor-attention-predictor network architecture (EAPNet), which employs a contracting-expanding structure to capture contextual information alongside a multilayer perceptron to enhance nonlinear modeling capability. A spatial attention module is introduced to facilitate the utilization of important information. We also use a balanced loss function to prevent network parameter updates from being biased towards specific regions. Our method obtains satisfactory quantitative metrics in simulated and real-world validations. Moreover, it demonstrates superior robustness to target properties and yields reliable results for targets with small size, deep location, or relatively low absorption intensity, indicating its broader applicability. The EAPNet, compared to the conventional UNet, exhibits improved efficiency, which significantly enhances performance while maintaining similar network size and computational complexity.

[The digital operating room : Chances and risks of artificial intelligence]

At the central workplace of the surgeon the digitalization of the operating room has particular consequences for the surgical work. Starting with intraoperative cross-sectional imaging and sonography, through functional imaging, minimally invasive and robot-assisted surgery up to digital surgical and anesthesiological documentation, the vast majority of operating rooms are now at least partially digitalized. The increasing digitalization of the whole process chain enables not only for the collection but also the analysis of big data. Current research focuses on artificial intelligence for the analysis of intraoperative data as the prerequisite for assistance systems that support surgical decision making or warn of risks; however, these technologies raise new ethical questions for the surgical community that affect the core of surgical work.

Depth-Specific Hypoxic Responses to Spreading Depolarizations in Gyrencephalic Swine Cortex Unveiled by Photoacoustic Imaging

Spreading depolarizations (SDs) are a marker of brain injury and have a causative effect on ischemic lesion progression. The hemodynamic responses elicited by SDs are contingent upon the metabolic integrity of the affected tissue, with vasoconstrictive reactions leading to pronounced hypoxia often indicating poor outcomes. The stratification of hemodynamic responses within different cortical layers remains poorly characterized. This pilot study sought to elucidate the depth-specific hemodynamic changes in response to SDs within the gray matter of the gyrencephalic swine brain. Employing a potassium chloride-induced SD model, we utilized multispectral photoacoustic imaging (PAI) to estimate regional cerebral oxygen saturation (rcSO2%) changes consequent to potassium chloride-induced SDs. Regions of interest were demarcated at three cortical depths covering up to 4 mm. Electrocorticography (ECoG) strips were placed to validate the presence of SDs. Through PAI, we detected 12 distinct rcSO2% responses, which corresponded with SDs detected in ECoG. Notably, a higher frequency of hypoxic responses was observed in the deeper cortical layers compared to superficial layers, where hyperoxic and mixed responses predominated (p < 0.001). This data provides novel insights into the differential oxygenation patterns across cortical layers in response to SDs, underlining the complexity of cerebral hemodynamics post-injury.

Multi-Scale Volumetric Dynamic Optoacoustic and Laser Ultrasound (OPLUS) Imaging Enabled by Semi-Transparent Optical Guidance

Major biological discoveries are made by interrogating living organisms with light. However, the limited penetration of un-scattered photons within biological tissues limits the depth range covered by optical methods. Deep-tissue imaging is achieved by combining light and ultrasound. Optoacoustic imaging exploits the optical generation of ultrasound to render high-resolution images at depths unattainable with optical microscopy. Recently, laser ultrasound has been suggested as a means of generating broadband acoustic waves for high-resolution pulse-echo ultrasound imaging. Herein, an approach is proposed to simultaneously interrogate biological tissues with light and ultrasound based on layer-by-layer coating of silica optical fibers with a controlled degree of transparency. The time separation between optoacoustic and ultrasound signals collected with a custom-made spherical array transducer is exploited for simultaneous 3D optoacoustic and laser ultrasound (OPLUS) imaging with a single laser pulse. OPLUS is shown to enable large-scale anatomical characterization of tissues along with functional multi-spectral imaging of chromophores and assessment of cardiac dynamics at ultrafast rates only limited by the pulse repetition frequency of the laser. The suggested approach provides a flexible and scalable means for developing a new generation of systems synergistically combining the powerful capabilities of optoacoustics and ultrasound imaging in biology and medicine.

Multispectral optoacoustic tomography enables assessment of disease activity in paediatric inflammatory bowel disease

Multispectral optoacoustic tomography (MSOT) allows non-invasive molecular disease activity assessment in adults with inflammatory bowel disease (IBD). In this prospective pilot-study, we investigated, whether increased levels of MSOT haemoglobin parameters corresponded to inflammatory activity in paediatric IBD patients, too. 23 children with suspected IBD underwent MSOT of the terminal ileum and sigmoid colon with standard validation (e.g. endoscopy). In Crohn`s disease (CD) and ulcerative colitis (UC) patients with endoscopically confirmed disease activity, MSOT total haemoglobin (HbT) signals were increased in the terminal ileum of CD (72.1 ± 13.0 a.u. vs. 32.9 ± 15.4 a.u., p = 0.0049) and in the sigmoid colon of UC patients (62.9 ± 13.8 a.u. vs. 35.1 ± 16.3 a.u., p = 0.0311) as compared to controls, respectively. Furthermore, MSOT haemoglobin parameters correlated well with standard disease activity assessment (e.g. SES-CD and MSOT HbT (rs =0.69, p = 0.0075). Summarizing, MSOT is a novel technology for non-invasive molecular disease activity assessment in paediatric patients with inflammatory bowel disease.

Effects of skin tone on photoacoustic imaging and oximetry

Significance

Photoacoustic imaging (PAI) provides contrast based on the concentration of optical absorbers in tissue, enabling the assessment of functional physiological parameters such as blood oxygen saturation (sO 2 ). Recent evidence suggests that variation in melanin levels in the epidermis leads to measurement biases in optical technologies, which could potentially limit the application of these biomarkers in diverse populations.

Aim

To examine the effects of skin melanin pigmentation on PAI and oximetry.

Approach

We evaluated the effects of skin tone in PAI using a computational skin model, two-layer melanin-containing tissue-mimicking phantoms, and mice of a consistent genetic background with varying pigmentations. The computational skin model was validated by simulating the diffuse reflectance spectrum using the adding-doubling method, allowing us to assign our simulation parameters to approximate Fitzpatrick skin types. Monte Carlo simulations and acoustic simulations were run to obtain idealized photoacoustic images of our skin model. Photoacoustic images of the phantoms and mice were acquired using a commercial instrument. Reconstructed images were processed with linear spectral unmixing to estimate blood oxygenation. Linear unmixing results were compared with a learned unmixing approach based on gradient-boosted regression.

Results

Our computational skin model was consistent with representative literature for in vivo skin reflectance measurements. We observed consistent spectral coloring effects across all model systems, with an overestimation of sO 2 and more image artifacts observed with increasing melanin concentration. The learned unmixing approach reduced the measurement bias, but predictions made at lower blood sO 2 still suffered from a skin tone-dependent effect.

Conclusion

PAI demonstrates measurement bias, including an overestimation of blood sO 2 , in higher Fitzpatrick skin types. Future research should aim to characterize this effect in humans to ensure equitable application of the technology.

Quantitative photoacoustic tomography: modeling and inverse problems

Significance

Quantitative photoacoustic tomography (QPAT) exploits the photoacoustic effect with the aim of estimating images of clinically relevant quantities related to the tissue's optical absorption. The technique has two aspects: an acoustic part, where the initial acoustic pressure distribution is estimated from measured photoacoustic time-series, and an optical part, where the distributions of the optical parameters are estimated from the initial pressure.

Aim

Our study is focused on the optical part. In particular, computational modeling of light propagation (forward problem) and numerical solution methodologies of the image reconstruction (inverse problem) are discussed.

Approach

The commonly used mathematical models of how light and sound propagate in biological tissue are reviewed. A short overview of how the acoustic inverse problem is usually treated is given. The optical inverse problem and methods for its solution are reviewed. In addition, some limitations of real-life measurements and their effect on the inverse problems are discussed.

Results

An overview of QPAT with a focus on the optical part was given. Computational modeling and inverse problems of QPAT were addressed, and some key challenges were discussed. Furthermore, the developments for tackling these problems were reviewed. Although modeling of light transport is well-understood and there is a well-developed framework of inverse mathematics for approaching the inverse problem of QPAT, there are still challenges in taking these methodologies to practice.

Conclusions

Modeling and inverse problems of QPAT together were discussed. The scope was limited to the optical part, and the acoustic aspects were discussed only to the extent that they relate to the optical aspect.

Ultrasound-guided Photoacoustic image Annotation Toolkit in MATLAB (PHANTOM) for preclinical applications

Depth-dependent fluence-compensation in photoacoustic (PA) imaging is paramount for accurate quantification of chromophores from deep tissues. Here we present a user-friendly toolkit named PHANTOM (PHotoacoustic ANnotation TOolkit for MATLAB) that includes a graphical interface and assists in the segmentation of ultrasound-guided PA images. We modelled the light source configuration with Monte Carlo eXtreme and utilized 3D segmented tissues from ultrasound to generate fluence maps to depth compensate PA images. The methodology was used to analyze PA images of phantoms with varying blood oxygenation and results were validated with oxygen electrode measurements. Two preclinical models, a subcutaneous tumor and a calcified placenta, were imaged and fluence-compensated using the PHANTOM toolkit and the results were verified with immunohistochemistry. The PHANTOM toolkit provides scripts and auxiliary functions to enable biomedical researchers not specialized in optical imaging to apply fluence correction to PA images, enhancing accessibility of quantitative PAI for researchers in various fields.

Optical Breast Imaging: A Review of Physical Principles, Technologies, and Clinical Applications

Optical imaging involves the propagation of light through tissue. Current optical breast imaging technologies, including diffuse optical spectroscopy, diffuse optical tomography, and photoacoustic imaging, capitalize on the selective absorption of light in the near-infrared spectrum by deoxygenated and oxygenated hemoglobin. They provide information on the morphological and functional characteristics of different tissues based on their varied interactions with light, including physiologic information on lesion vascular content and anatomic information on tissue vascularity. Fluorescent contrast agents, such as indocyanine green, are used to visualize specific tissues, molecules, or proteins depending on how and where the agent accumulates. In this review, we describe the physical principles, spectrum of technologies, and clinical applications of the most common optical systems currently being used or developed for breast imaging. Most notably, US co-registered photoacoustic imaging and US-guided diffuse optical tomography have demonstrated efficacy in differentiating benign from malignant breast masses, thereby improving the specificity of diagnostic imaging. Diffuse optical tomography and diffuse optical spectroscopy have shown promise in assessing treatment response to preoperative systemic therapy, and photoacoustic imaging and diffuse optical tomography may help predict tumor phenotype. Lastly, fluorescent imaging using indocyanine green dye performs comparably to radioisotope mapping of sentinel lymph nodes and appears to improve the outcomes of autologous tissue flap breast reconstruction.

A review of a strategic roadmapping exercise to advance clinical translation of photoacoustic imaging: From current barriers to future adoption

Photoacoustic imaging (PAI), also referred to as optoacoustic imaging, has shown promise in early-stage clinical trials in a range of applications from inflammatory diseases to cancer. While the first PAI systems have recently received regulatory approvals, successful adoption of PAI technology into healthcare systems for clinical decision making must still overcome a range of barriers, from education and training to data acquisition and interpretation. The International Photoacoustic Standardisation Consortium (IPASC) undertook an community exercise in 2022 to identify and understand these barriers, then develop a roadmap of strategic plans to address them. Here, we outline the nature and scope of the barriers that were identified, along with short-, medium- and long-term community efforts required to overcome them, both within and beyond the IPASC group.

Multispectral optoacoustic tomography of the human intestine - temporal precision and the influence of postprandial gastrointestinal blood flow

Multispectral optoacoustic tomography (MSOT) holds great promise as a non-invasive diagnostic tool for inflammatory bowel diseases. Yet, reliability and the impact of physiological processes during fasting and after food intake on optoacoustic signals have not been studied. In the present investigator initiated trial (NCT05160077) the intestines of ten healthy subjects were examined by MSOT at eight timepoints on two days, one fasting and one after food intake. While within-timepoint and within-day reproducibility were good for single wavelength 800 nm and total hemoglobin (ICC 0.722-0.956), between-day reproducibility was inferior (ICC -0.137 to 0.438). However, temporal variability was smaller than variation between individuals (coefficients of variation 8.9%-33.7% vs. 17.0%-48.5%). After food intake and consecutive increased intestinal circulation, indicated by reduced resistance index of simultaneous Doppler ultrasound, optoacoustic signals did not alter significantly. In summary, this study demonstrates high reliability and temporal stability of MSOT for imaging the human intestine during fasting and after food intake.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Photoacoustic image synthesis with generative adversarial networks

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).

Dual-wavelength photoacoustic atlas method to estimate fractional methylene blue and hemoglobin contents

SIGNIFICANCE

Methylene blue (MB) is an exogenous contrast agent that has the potential to assist with visualization and penetration challenges in photoacoustic imaging. However, monitoring the local concentration between MB and endogenous chromophores is critical for avoiding unnecessary MB accumulations that could lead to adverse effects such as hemolysis when exposed to increased dose and photodamage when exposed to high laser energies.

AIM

We developed a modified version of a previously proposed acoustic-based atlas method to estimate concentration levels from a mixture of two photoacoustic-sensitive materials after two laser wavelength emissions.

APPROACH

Photoacoustic data were acquired from mixtures of 100-μM MB and either human or porcine blood (Hb) injected in a plastisol phantom, using laser wavelengths of 710 and 870 nm. An algorithm to perform linear regression of the acoustic frequency response from an atlas composed of pure concentrations was designed to assess the concentration levels from photoacoustic samples obtained from 11 known MB/Hb volume mixtures. The mean absolute error (MAE), coefficient of determination (i.e., R2), and Spearman's correlation coefficient (i.e., ρ) between the estimated results and ground-truth labels were calculated to assess the algorithm performance, linearity, and monotonicity, respectively.

RESULTS

The overall MAE, R2, and ρ were 12.68%, 0.80, and 0.89, respectively, for the human Hb dataset and 9.92%, 0.86, and 0.93, respectively, for the porcine Hb dataset. In addition, a similarly linear relationship was observed between the acoustic frequency response at 2.3 MHz and 870-nm laser wavelength and the ground-truth concentrations, with R2 and | ρ | values of 0.76 and 0.88, respectively.

CONCLUSIONS

Contrast agent concentration monitoring is feasible with the proposed approach. The potential for minimal data acquisition times with only two wavelength emissions is advantageous toward real-time implementation in the operating room.

An optoacoustic imaging feature set to characterise blood vessels surrounding benign and malignant breast lesions

Combining optoacoustic (OA) imaging with ultrasound (US) enables visualisation of functional blood vasculature in breast lesions by OA to be overlaid with the morphological information of US. Here, we develop a simple OA feature set to differentiate benign and malignant breast lesions. 94 female patients with benign, indeterminate or suspicious lesions were recruited and underwent OA-US. An OA-US imaging feature set was developed using images from the first 38 patients, which contained 14 malignant and 8 benign solid lesions. Two independent radiologists blindly scored the OA-US images of a further 56 patients, which included 31 malignant and 13 benign solid lesions, with a sensitivity of 96.8% and specificity of 84.6%. Our findings indicate that OA-US can reveal vascular patterns of breast lesions that indicate malignancy using a simple feature set based on single wavelength OA data, which is therefore amenable to application in low resource settings for breast cancer management.

Noninvasive hemoglobin sensing and imaging: optical tools for disease diagnosis

SIGNIFICANCE

Measurement and imaging of hemoglobin oxygenation are used extensively in the detection and diagnosis of disease; however, the applied instruments vary widely in their depth of imaging, spatiotemporal resolution, sensitivity, accuracy, complexity, physical size, and cost. The wide variation in available instrumentation can make it challenging for end users to select the appropriate tools for their application and to understand the relative limitations of different methods.

AIM

We aim to provide a systematic overview of the field of hemoglobin imaging and sensing.

APPROACH

We reviewed the sensing and imaging methods used to analyze hemoglobin oxygenation, including pulse oximetry, spectral reflectance imaging, diffuse optical imaging, spectroscopic optical coherence tomography, photoacoustic imaging, and diffuse correlation spectroscopy.

RESULTS

We compared and contrasted the ability of different methods to determine hemoglobin biomarkers such as oxygenation while considering factors that influence their practical application.

CONCLUSIONS

We highlight key limitations in the current state-of-the-art and make suggestions for routes to advance the clinical use and interpretation of hemoglobin oxygenation information.

3D Ultrasound-Guided Photoacoustic Imaging to Monitor the Effects of Suboptimal Tyrosine Kinase Inhibitor Therapy in Pancreatic Tumors

Pancreatic cancer is a disease with an incredibly poor survival rate. As only about 20% of patients are eligible for surgical resection, neoadjuvant treatments that can relieve symptoms and shrink tumors for surgical resection become critical. Many forms of treatments rely on increased vulnerability of cancerous cells, but tumors or regions within the tumors that may be hypoxic could be drug resistant. Particularly for neoadjuvant therapies such as the tyrosine kinase inhibitors utilized to shrink tumors, it is critical to monitor changes in vascular function and hypoxia to predict treatment efficacy. Current clinical imaging modalities used to obtain structural and functional information regarding hypoxia or oxygen saturation (StO2) do not provide sufficient depth penetration or require the use of exogenous contrast agents. Recently, ultrasound-guided photoacoustic imaging (US-PAI) has garnered significant popularity, as it can noninvasively provide multiparametric information on tumor vasculature and function without the need for contrast agents. Here, we built upon existing literature on US-PAI and demonstrate the importance of changes in StO2 values to predict treatment response, particularly tumor growth rate, when the outcomes are suboptimal. Specifically, we image xenograft mouse models of pancreatic adenocarcinoma treated with suboptimal doses of a tyrosine kinase inhibitor cabozantinib. We utilize the US-PAI data to develop a multivariate regression model that demonstrates that a therapy-induced reduction in tumor growth rate can be predicted with 100% positive predictive power and a moderate (58.33%) negative predictive power when a combination of pretreatment tumor volume and changes in StO2 values pretreatment and immediately posttreatment was employed. Overall, our study indicates that US-PAI has the potential to provide label-free surrogate imaging biomarkers that can predict tumor growth rate in suboptimal therapy.

High-resolution label-free mapping of murine kidney vasculature by raster-scanning optoacoustic mesoscopy: an ex vivo study

BACKGROUND

Chronic kidney disease (CKD) is a global burden affecting both children and adults. Novel imaging modalities hold great promise to visualize and quantify structural, functional, and molecular organ damage. The aim of the study was to visualize and quantify murine renal vasculature using label-free raster scanning optoacoustic mesoscopy (RSOM) in explanted organs from mice with renal injury.

MATERIAL AND METHODS

For the experiments, freshly bisected kidneys of alpha 8 integrin knock-out (KO) and wildtype mice (WT) were used. A total of n=7 female (n=4 KO, n=3 WT) and n=6 male animals (n=2 KO, n=4 WT) aged 6 weeks were examined with RSOM optoacoustic imaging systems (RSOM Explorer P50 at SWL 532nm and/or ms-P50 imaging system at 532 nm, 555 nm, 579 nm, and 606 nm). Images were reconstructed using a dedicated software, analyzed for size and vascular area and compared to standard histologic sections.

RESULTS

RSOM enabled mapping of murine kidney size and vascular area, revealing differences between kidney sizes of male (m) and female (f) mice (merged frequencies (MF) f vs. m: 52.42±6.24 mm² vs. 69.18±15.96 mm², p=0.0156) and absolute vascular area (MF f vs. m: 35.67±4.22 mm² vs. 49.07±13.48 mm², p=0.0036). Without respect to sex, the absolute kidney area was found to be smaller in knock-out (KO) than in wildtype (WT) mice (WT vs. KO: MF: p=0.0255) and showed a similar trend for the relative vessel area (WT vs. KO: MF p=0.0031). Also the absolute vessel areas of KO compared to WT were found significantly different (MF p=0.0089). A significant decrease in absolute vessel area was found in KO compared to WT male mice (MF WT vs. KO: 54.37±9.35 mm² vs. 34.93±13.82 mm², p=0.0232). In addition, multispectral RSOM allowed visualization of oxygenated and deoxygenated parenchymal regions by spectral unmixing.

CONCLUSION

This study demonstrates the capability of RSOM for label-free visualization of differences in vascular morphology in ex vivo murine renal tissue at high resolution. Due to its scalability optoacoustic imaging provides an emerging modality with potential for further preclinical and clinical imaging applications.

Semantic segmentation of multispectral photoacoustic images using deep learning

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Superpixel spectral unmixing framework for the volumetric assessment of tissue chromophores: A photoacoustic data-driven approach

The assessment of tissue chromophores at a volumetric scale is vital for an improved diagnosis and treatment of a large number of diseases. Spectral photoacoustic imaging (sPAI) co-registered with high-resolution ultrasound (US) is an innovative technology that has a great potential for clinical translation as it can assess the volumetric distribution of the tissue components. Conventionally, to detect and separate the chromophores from sPAI, an input of the expected tissue absorption spectra is required. However, in pathological conditions, the prediction of the absorption spectra is difficult as it can change with respect to the physiological state. Besides, this conventional approach can also be hampered due to spectral coloring, which is a prominent distortion effect that induces spectral changes at depth. Here, we are proposing a novel data-driven framework that can overcome all these limitations and provide an improved assessment of the tissue chromophores. We have developed a superpixel spectral unmixing (SPAX) approach that can detect the most and less prominent absorber spectra and their volumetric distribution without any user interactions. Within the SPAX framework, we have also implemented an advanced spectral coloring compensation approach by utilizing US image segmentation and Monte Carlo simulations, based on a predefined library of optical properties. The framework has been tested on tissue-mimicking phantoms and also on healthy animals. The obtained results show enhanced specificity and sensitivity for the detection of tissue chromophores. To our knowledge, this is a unique framework that accounts for the spectral coloring and provides automated detection of tissue spectral signatures at a volumetric scale, which can open many possibilities for translational research.

Machine learning enabled multiple illumination quantitative optoacoustic oximetry imaging in humans

Optoacoustic (OA) imaging is a promising modality for quantifying blood oxygen saturation (sO₂) in various biomedical applications - in diagnosis, monitoring of organ function, or even tumor treatment planning. We present an accurate and practically feasible real-time capable method for quantitative imaging of sO₂ based on combining multispectral (MS) and multiple illumination (MI) OA imaging with learned spectral decoloring (LSD). For this purpose we developed a hybrid real-time MI MS OA imaging setup with ultrasound (US) imaging capability; we trained gradient boosting machines on MI spectrally colored absorbed energy spectra generated by generic Monte Carlo simulations and used the trained models to estimate sO₂ on real OA measurements. We validated MI-LSD in silico and on in vivo image sequences of radial arteries and accompanying veins of five healthy human volunteers. We compared the performance of the method to prior LSD work and conventional linear unmixing. MI-LSD provided highly accurate results in silico and consistently plausible results in vivo. This preliminary study shows a potentially high applicability of quantitative OA oximetry imaging, using our method.

Functional photoacoustic microscopy of hemodynamics: a review

Functional blood imaging can reflect tissue metabolism and organ viability, which is important for life science and biomedical studies. However, conventional imaging modalities either cannot provide sufficient contrast or cannot support simultaneous multi-functional imaging for hemodynamics. Photoacoustic imaging, as a hybrid imaging modality, can provide sufficient optical contrast and high spatial resolution, making it a powerful tool for in vivo vascular imaging. By using the optical-acoustic confocal alignment, photoacoustic imaging can even provide subcellular insight, referred as optical-resolution photoacoustic microscopy (OR-PAM). Based on a multi-wavelength laser source and developed the calculation methods, OR-PAM can provide multi-functional hemodynamic microscopic imaging of the total hemoglobin concentration (C_Hb), oxygen saturation (sO₂), blood flow (BF), partial oxygen pressure (pO₂), oxygen extraction fraction, and metabolic rate of oxygen (MRO₂). This concise review aims to systematically introduce the principles and methods to acquire various functional parameters for hemodynamics by photoacoustic microscopy in recent studies, with characteristics and advantages comparison, typical biomedical applications introduction, and future outlook discussion.

Quantitative endoscopic photoacoustic tomography using a convolutional neural network

Endoscopic photoacoustic tomography (EPAT) is a catheter-based hybrid imaging modality capable of providing structural and functional information of biological luminal structures, such as coronary arterial vessels and the digestive tract. The recovery of the optical properties of the imaged tissue from acoustic measurements achieved by optical inversion is essential for implementing quantitative EPAT (qEPAT). In this paper, a convolutional neural network (CNN) based on deep gradient descent is developed for qEPAT. The network enables the reconstruction of images representing the spatially varying absorption coefficient in cross-sections of the tubular structures from limited measurement data. The forward operator reflecting the mapping from the absorption coefficient to the optical deposition due to pulsed irradiation is embedded into the network training. The network parameters are optimized layer by layer through the deep gradient descent mechanism using the numerically simulated data. The operation processes of the forward operator and its adjoint operator are separated from the network training. The trained network outputs an image representing the distribution of absorption coefficients by inputting an image that represents the optical deposition. The method has been tested with computer-generated phantoms mimicking coronary arterial vessels containing various tissue types. Results suggest that the structural similarity of the images reconstructed by our method is increased by about 10% in comparison with the non-learning method based on error minimization in the case of the same measuring view.

SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics

SIGNIFICANCE

Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings.

AIM

To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards.

APPROACH

SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models.

RESULTS

To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations.

CONCLUSIONS

SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.

Segmentation and Quantitative Analysis of Photoacoustic Imaging: A Review

Photoacoustic imaging is an emerging biomedical imaging technique that combines optical contrast and ultrasound resolution to create unprecedented light absorption contrast in deep tissue. Thanks to its fusional imaging advantages, photoacoustic imaging can provide multiple structural and functional insights into biological tissues such as blood vasculatures and tumors and monitor the kinetic movements of hemoglobin and lipids. To better visualize and analyze the regions of interest, segmentation and quantitative analyses were used to extract several biological factors, such as the intensity level changes, diameter, and tortuosity of the tissues. Over the past 10 years, classical segmentation methods and advances in deep learning approaches have been utilized in research investigations. In this review, we provide a comprehensive review of segmentation and quantitative methods that have been developed to process photoacoustic imaging in preclinical and clinical experiments. We focus on the parametric reliability of quantitative analysis for semantic and instance-level segmentation. We also introduce the similarities and alternatives of deep learning models in qualitative measurements using classical segmentation methods for photoacoustic imaging.

Model-based optical and acoustical compensation for photoacoustic tomography of heterogeneous mediums

Photoacoustic tomography (PAT) is a non-invasive, high-resolution imaging modality, capable of providing functional and molecular information of various pathologies, such as cancer. One limitation of PAT is the depth and wavelength dependent optical fluence, which results in reduced PA signal amplitude from deeper tissue regions. These factors can therefore introduce errors into quantitative measurements such as oxygen saturation (sO₂) or the localization and concentration of various chromophores. The variation in the speed-of-sound between different tissues can also lead to distortions in object location and shape. Compensating for these effects allows PAT to be used more quantitatively. We have developed a proof-of-concept algorithm capable of compensating for the heterogeneity in speed-of-sound and depth dependent optical fluence. Speed-of-sound correction was done by using a straight ray-based algorithm for calculating the family of iso-time-of-flight contours between the transducers and every pixel in the imaging grid, while fluence compensation was done by utilizing the graphics processing unit (GPU) accelerated software MCXCL for Monte Carlo modeling of optical fluence variation. This algorithm was tested on a polyvinyl chloride plastisol (PVCP) phantom, which contained cyst mimics and blood inclusions to test the algorithm under relatively heterogeneous conditions. Our results indicate that our PAT algorithm can compensate for the speed-of-sound variation and depth dependent fluence effects within a heterogeneous phantom. The results of this study will pave the way for further development and evaluation of the proposed method in more complex in-vitro and ex-vivo phantoms, as well as compensating for the wavelength-dependent optical fluence in spectroscopic PAT.

Multiple illumination learned spectral decoloring for quantitative optoacoustic oximetry imaging

SIGNIFICANCE

Quantitative measurement of blood oxygen saturation (sO2) with optoacoustic (OA) imaging is one of the most sought after goals of quantitative OA imaging research due to its wide range of biomedical applications.

AIM

A method for accurate and applicable real-time quantification of local sO2 with OA imaging.

APPROACH

We combine multiple illumination (MI) sensing with learned spectral decoloring (LSD). We train LSD feedforward neural networks and random forests on Monte Carlo simulations of spectrally colored absorbed energy spectra, to apply the trained models to real OA measurements. We validate our combined MI-LSD method on a highly reliable, reproducible, and easily scalable phantom model, based on copper and nickel sulfate solutions.

RESULTS

With this sulfate model, we see a consistently high estimation accuracy using MI-LSD, with median absolute estimation errors of 2.5 to 4.5 percentage points. We further find fewer outliers in MI-LSD estimates compared with LSD. Random forest regressors outperform previously reported neural network approaches.

CONCLUSIONS

Random forest-based MI-LSD is a promising method for accurate quantitative OA oximetry imaging.

Perspective on fast-evolving photoacoustic tomography

SIGNIFICANCE

Acoustically detecting the rich optical absorption contrast in biological tissues, photoacoustic tomography (PAT) seamlessly bridges the functional and molecular sensitivity of optical excitation with the deep penetration and high scalability of ultrasound detection. As a result of continuous technological innovations and commercial development, PAT has been playing an increasingly important role in life sciences and patient care, including functional brain imaging, smart drug delivery, early cancer diagnosis, and interventional therapy guidance.

AIM

Built on our 2016 tutorial article that focused on the principles and implementations of PAT, this perspective aims to provide an update on the exciting technical advances in PAT.

APPROACH

This perspective focuses on the recent PAT innovations in volumetric deep-tissue imaging, high-speed wide-field microscopic imaging, high-sensitivity optical ultrasound detection, and machine-learning enhanced image reconstruction and data processing. Representative applications are introduced to demonstrate these enabling technical breakthroughs in biomedical research.

CONCLUSIONS

We conclude the perspective by discussing the future development of PAT technologies.

In Vitro and In Vivo Multispectral Photoacoustic Imaging for the Evaluation of Chromophore Concentration

Multispectral photoacoustic imaging is a powerful noninvasive medical imaging technique that provides access to functional information. In this study, a set of methods is proposed and validated, with experimental multispectral photoacoustic images used to estimate the concentration of chromophores. The unmixing techniques used in this paper consist of two steps: (1) automatic extraction of the reference spectrum of each pure chromophore; and (2) abundance calculation of each pure chromophore from the estimated reference spectra. The compared strategies bring positivity and sum-to-one constraints, from the hyperspectral remote sensing field to multispectral photoacoustic, to evaluate chromophore concentration. Particularly, the study extracts the endmembers and compares the algorithms from the hyperspectral remote sensing domain and a dedicated algorithm for segmentation of multispectral photoacoustic data to this end. First, these strategies are tested with dilution and mixing of chromophores on colored 4% agar phantom data. Then, some preliminary in vivo experiments are performed. These consist of estimations of the oxygen saturation rate (sO2) in mouse tumors. This article proposes then a proof-of-concept of the interest to bring hyperspectral remote sensing algorithms to multispectral photoacoustic imaging for the estimation of chromophore concentration.