Motion clustering for deblurring multispectral optoacoustic tomography images of the mouse heart

Correction of high-rate motion for photoacoustic microscopy by orthogonal cross-correlation

Photoacoustic imaging is a promising technology for in vivo imaging. However, its imaging performance can be hampered by motion artifacts, especially when dealing with high-rate motion. In this paper, we propose an orthogonal motion correction method that utilizes cross-correlation along orthogonal scan directions to extract accurate motion displacements from the photoacoustic data. The extracted displacements are then applied to remove artifacts and compensate for motion-induced distortions. Phantom experiments demonstrate that the proposed method can extract the motion information and the structural similarity index measurement after correction is increased by 26.5% and 11.2% compared to no correction and the previous correction method. Then the effectiveness of our method is evaluated in vivo imaging of a mouse brain. Our method shows a stable and effective performance under high-rate motion. The high accuracy of the motion correction method makes it valuable in improving the accuracy of photoacoustic imaging.

A Deep Learning Method for Motion Artifact Correction in Intravascular Photoacoustic Image Sequence

In vivo application of intravascular photoacoustic (IVPA) imaging for coronary arteries is hampered by motion artifacts associated with the cardiac cycle. Gating is a common strategy to mitigate motion artifacts. However, a large amount of diagnostically valuable information might be lost due to one frame per cycle. In this work, we present a deep learning-based method for directly correcting motion artifacts in non-gated IVPA pullback sequences. The raw signal frames are classified into dynamic and static frames by clustering. Then, a neural network named Motion Artifact Correction (MAC)-Net is designed to correct motion in dynamic frames. Given the lack of the ground truth information on the underlying dynamics of coronary arteries, we trained and tested the network using a computer-generated dataset. Based on the results, it has been observed that the trained network can directly correct motion in successive frames while preserving the original structures without discarding any frames. The improvement in the visual effect of the longitudinal view has been demonstrated based on quantitative evaluation of the inter-frame dissimilarity. The comparison results validated the motion-suppression ability of our method comparable to gating and image registration-based non-learning methods, while maintaining the integrity of the pullbacks without image preprocessing. Experimental results from in vivo intravascular ultrasound and optical coherence tomography pullbacks validated the feasibility of our method in the in vivo intracoronary imaging scenario.

Photoacoustic imaging for microcirculation

Microcirculation facilitates the blood-tissue exchange of nutrients and regulates blood perfusion. It is, therefore, essential in maintaining tissue health. Aberrations in microcirculation are potentially indicative of underlying cardiovascular and metabolic pathologies. Thus, quantitative information about it is of great clinical relevance. Photoacoustic imaging (PAI) is a capable technique that relies on the generation of imaging contrast via the absorption of light and can image at micron-scale resolution. PAI is especially desirable to map microvasculature as hemoglobin strongly absorbs light and can generate a photoacoustic signal. This paper reviews the current state of the art for imaging microvascular networks using photoacoustic imaging. We further describe how quantitative information about blood dynamics such as the total hemoglobin concentration, oxygen saturation, and blood flow rate is obtained using PAI. We also discuss its importance in understanding key pathophysiological processes in neurovascular, cardiovascular, ophthalmic, and cancer research fields. We then discuss the current challenges and limitations of PAI and the approaches that can help overcome these limitations. Finally, we provide the reader with an overview of future trends in the field of PAI for imaging microcirculation.

Suppression of motion artifacts in intravascular photoacoustic image sequences

Intravascular photoacoustic (IVPA) imaging is an image-based imaging modality for the assessment of atherosclerotic plaques. Successful application of IVPA for in vivo coronary arterial imaging requires one overcomes the challenge of motion artifacts associated with the cardiac cycle. We propose a method for correcting artifacts owing to cardiac motion, which are observed in sequential IVPA images acquired by the continuous pullback of the imaging catheter. This method groups raw photoacoustic signals into subsets corresponding to similar phases in the cardiac cycles. Thereafter, the sequential images are reconstructed, by representing the initial pressure distribution on the vascular cross-sections based on the clustered frames of signals by time reversal. Results of simulation data demonstrate the efficacy of this method in suppressing motion artifacts. Qualitative and quantitative evaluations of the method indicate an enhancement of the image quality. Comparison results reveal that this method is computationally efficient in motion correction compared with the image-based gating.

Deep learning in photoacoustic imaging: a review

SIGNIFICANCE

Photoacoustic (PA) imaging can provide structural, functional, and molecular information for preclinical and clinical studies. For PA imaging (PAI), non-ideal signal detection deteriorates image quality, and quantitative PAI (QPAI) remains challenging due to the unknown light fluence spectra in deep tissue. In recent years, deep learning (DL) has shown outstanding performance when implemented in PAI, with applications in image reconstruction, quantification, and understanding.

AIM

We provide (i) a comprehensive overview of the DL techniques that have been applied in PAI, (ii) references for designing DL models for various PAI tasks, and (iii) a summary of the future challenges and opportunities.

APPROACH

Papers published before November 2020 in the area of applying DL in PAI were reviewed. We categorized them into three types: image understanding, reconstruction of the initial pressure distribution, and QPAI.

RESULTS

When applied in PAI, DL can effectively process images, improve reconstruction quality, fuse information, and assist quantitative analysis.

CONCLUSION

DL has become a powerful tool in PAI. With the development of DL theory and technology, it will continue to boost the performance and facilitate the clinical translation of PAI.

Evaluating online filtering algorithms to enhance dynamic multispectral optoacoustic tomography

Multispectral optoacoustic tomography (MSOT) is an emerging imaging modality, which is able to capture data at high spatiotemporal resolution using rapid tuning of the excitation laser wavelength. However, owing to the necessity of imaging one wavelength at a time to the exclusion of others, forming a complete multispectral image requires multiple excitations over time, which may introduce aliasing due to underlying spectral dynamics or noise in the data. In order to mitigate this limitation, we have applied kinematic α and α β filters to multispectral time series, providing an estimate of the underlying multispectral image at every point in time throughout data acquisition. We demonstrate the efficacy of these methods in suppressing the inter-frame noise present in dynamic multispectral image time courses using a multispectral Shepp-Logan phantom and mice bearing distinct renal cell carcinoma tumors. The gains in signal to noise ratio provided by these filters enable higher-fidelity downstream analysis such as spectral unmixing and improved hypothesis testing in quantifying the onset of signal changes during an oxygen gas challenge.

Dual-comb photoacoustic spectroscopy

Spectrally resolved photoacoustic imaging is promising for label-free imaging in optically scattering materials. However, this technique often requires acquisition of a separate image at each wavelength of interest. This reduces imaging speeds and causes errors if the sample changes in time between images acquired at different wavelengths. We demonstrate a solution to this problem by using dual-comb spectroscopy for photoacoustic measurements. This approach enables a photoacoustic measurement at thousands of wavelengths simultaneously. In this technique, two optical-frequency combs are interfered on a sample and the resulting pressure wave is measured with an ultrasound transducer. This acoustic signal is processed in the frequency-domain to obtain an optical absorption spectrum. For a proof-of-concept demonstration, we measure photoacoustic signals from polymer films. The absorption spectra obtained from these measurements agree with those measured using a spectrophotometer. Improving the signal-to-noise ratio of the dual-comb photoacoustic spectrometer could enable high-speed spectrally resolved photoacoustic imaging.

Spatial Compounding of Volumetric Data Enables Freehand Optoacoustic Angiography of Large-Scale Vascular Networks

Optoacoustic tomography systems have attained unprecedented volumetric imaging speeds, thus enabling insights into rapid biological dynamics and marking a milestone in the clinical translation of this modality. Fast imaging performance often comes at the cost of limited field-of-view, which may hinder potential applications looking at larger tissue volumes. The imaged field-of-view can potentially be expanded via scanning and using additional hardware to track the position of the imaging probe. However, this approach turns impractical for high-resolution volumetric scans performed in a freehand mode along arbitrary trajectories. We have developed an accurate framework for spatial compounding of time-lapse optoacoustic data. The method exploits the frequency-domain properties of vascular networks in optoacoustic images and estimates the relative motion and orientation of the imaging probe. This allows rapidly combining sequential volumetric frames into large area scans without additional tracking hardware. The approach is universally applicable for compounding volumetric data acquired with calibrated scanning systems but also in a freehand mode with up to six degrees of freedom. Robust performance is demonstrated for whole-body mouse imaging with spiral volumetric optoacoustic tomography and for freehand visualization of vascular networks in humans using volumetric imaging probes. The newly introduced capability for angiographic observations at multiple spatial and temporal scales is expected to greatly facilitate the use of optoacoustic imaging technology in pre-clinical research and clinical diagnostics. The technique can equally benefit other biomedical imaging modalities, such as scanning fluorescence microscopy, optical coherence tomography or ultrasonography, thus optimizing their trade-offs between fast imaging performance and field-of-view.

Deep-learning-based motion-correction algorithm in optical resolution photoacoustic microscopy

In this study, we propose a deep-learning-based method to correct motion artifacts in optical resolution photoacoustic microscopy (OR-PAM). The method is a convolutional neural network that establishes an end-to-end map from input raw data with motion artifacts to output corrected images. First, we performed simulation studies to evaluate the feasibility and effectiveness of the proposed method. Second, we employed this method to process images of rat brain vessels with multiple motion artifacts to evaluate its performance for in vivo applications. The results demonstrate that this method works well for both large blood vessels and capillary networks. In comparison with traditional methods, the proposed method in this study can be easily modified to satisfy different scenarios of motion corrections in OR-PAM by revising the training sets.

Motion Correction in Optical Resolution Photoacoustic Microscopy

In this paper, we are proposing a novel motion correction algorithm for high-resolution OR-PAM imaging. Our algorithm combines a modified demons-based tracking approach with a newly developed multi-scale vascular feature matching method to track motion between adjacent B-scan images without needing any reference object. We first applied this algorithm to correct motion artifacts within one three-dimensional (3D) data segment of rat iris obtained with OR-PAM imaging. We then extended the application of this algorithm to correct motions to obtain vasculature imaging in the whole mouse back. In here, we stitched five adjacent 3D data segments (large field-of-view) obtained while changing the focus of OR-PAM differently for each subarea. The results showed that the motion artifacts of both large blood vessels and microvessels could be accurately corrected in both cases. Compared to the manually stitching method and the traditional SIFT algorithm, the algorithm proposed in this paper has better performance in stitching adjacent data segments. The high accuracy of the motion correction algorithm makes it valuable in OR-PAM for high-resolution imaging of large animals and for quantitative functional imaging.

Self-Gated Respiratory Motion Rejection for Optoacoustic Tomography

Respiratory motion in living organisms is known to result in image blurring and loss of resolution, chiefly due to the lengthy acquisition times of the corresponding image acquisition methods. Optoacoustic tomography can effectively eliminate in vivo motion artifacts due to its inherent capacity for collecting image data from the entire imaged region following a single nanoseconds-duration laser pulse. However, multi-frame image analysis is often essential in applications relying on spectroscopic data acquisition or for scanning-based systems. Thereby, efficient methods to correct for image distortions due to motion are imperative. Herein, we demonstrate that efficient motion rejection in optoacoustic tomography can readily be accomplished by frame clustering during image acquisition, thus averting excessive data acquisition and post-processing. The algorithm’s efficiency for two- and three-dimensional imaging was validated with experimental whole-body mouse data acquired by spiral volumetric optoacoustic tomography (SVOT) and full-ring cross-sectional imaging scanners.

Cardiovascular optoacoustics: From mice to men - A review

Imaging has become an indispensable tool in the research and clinical management of cardiovascular disease (CVD). An array of imaging technologies is considered for CVD diagnostics and therapeutic assessment, ranging from ultrasonography, X-ray computed tomography and magnetic resonance imaging to nuclear and optical imaging methods. Each method has different operational characteristics and assesses different aspects of CVD pathophysiology; nevertheless, more information is desirable for achieving a comprehensive view of the disease. Optoacoustic (photoacoustic) imaging is an emerging modality promising to offer novel information on CVD parameters by allowing high-resolution imaging of optical contrast several centimeters deep inside tissue. Implemented with illumination at several wavelengths, multi-spectral optoacoustic tomography (MSOT) in particular, is sensitive to oxygenated and deoxygenated hemoglobin, water and lipids allowing imaging of the vasculature, tissue oxygen saturation and metabolic or inflammatory parameters. Progress with fast-tuning lasers, parallel detection and advanced image reconstruction and data-processing algorithms have recently transformed optoacoustics from a laboratory tool to a promising modality for small animal and clinical imaging. We review progress with optoacoustic CVD imaging, highlight the research and diagnostic potential and current applications and discuss the advantages, limitations and possibilities for integration into clinical routine.

Emerging Technologies to Image Tissue Metabolism

Due to the implication of altered metabolism in a large spectrum of tissue function and disease, assessment of metabolic processes becomes essential in managing health. In this regard, imaging can play a critical role in allowing observation of biochemical and physiological processes. Nuclear imaging methods, in particular positron emission tomography, have been widely employed for imaging metabolism but are mainly limited by the use of ionizing radiation and the sensing of only one parameter at each scanning session. Observations in healthy individuals or longitudinal studies of disease could markedly benefit from non-ionizing, multi-parameter imaging methods. We therefore focus this review on progress with the non-ionizing radiation methods of MRI, hyperpolarized magnetic resonance and magnetic resonance spectroscopy, chemical exchange saturation transfer, and emerging optoacoustic (photoacoustic) imaging. We also briefly discuss the role of nuclear and optical imaging methods for research and clinical protocols.

Applying dynamic contrast enhanced MSOT imaging to intratumoral pharmacokinetic modeling

Examining the dynamics of an agent in the tumor microenvironment can offer critical insights to the influx rate and accumulation of the agent. Intratumoral kinetic characterization in the in vivo setting can further elicudate distribution patterns and tumor microenvironment. Dynamic contrast-enhanced Multispectral Optoacoustic Tomographic imaging (DCE-MSOT) acquires serial MSOT images with the administration of an exogenous contrast agent over time. We tracked the dynamics of a tumor-targeted contrast agent, HypoxiSense 680 (HS680), in breast xenograft mouse models using MSOT. Arterial input function (AIF) approach with MSOT imaging allowed for tracking HS680 dynamics within the mouse. The optoacoustic signal for HS680 was quantified using the ROI function in the ViewMSOT software. A two-compartment pharmacokinetics (PK) model constructed in MATLAB to fit rate parameters. The contrast influx (k_in) and outflux (k_out) rate constants predicted are k_in = 1.96 × 10^-2 s^-1 and k_out = 9.5 × 10^-3 s^-1 (R = 0.9945).

Tomographic breathing detection: a method to noninvasively assess in situ respiratory dynamics

Physiological monitoring is a critical aspect of in vivo experimentation, particularly imaging studies. Physiological monitoring facilitates gated acquisition of imaging data and more robust experimental interpretation but has historically required additional instrumentation that may be cumbersome. As frame rates have increased, imaging methods have been able to capture ever more rapid dynamics, passing the Nyquist sampling rate of most physiological processes and allowing the capture of motion, such as breathing. With this transition, image artifacts have also changed their nature; rather than intraframe motion causing blurring and deteriorating resolution, interframe motion does not affect individual frames and may be recovered as useful information from an image time series. We demonstrate a method that takes advantage of interframe movement for detection of gross physiological motion in real-time image sequences. We further demonstrate the ability of the method, dubbed tomographic breathing detection to quantify the dynamics of respiration, allowing the capture of respiratory information pertinent to anesthetic depth monitoring. Our example uses multispectral optoacoustic tomography, but it will be widely relevant to other technologies.

Breakthroughs in modern cancer therapy and elusive cardiotoxicity: Critical research-practice gaps, challenges, and insights

To date, five cancer treatment modalities have been defined. The three traditional modalities of cancer treatment are surgery, radiotherapy, and conventional chemotherapy, and the two modern modalities include molecularly targeted therapy (the fourth modality) and immunotherapy (the fifth modality). The cardiotoxicity associated with conventional chemotherapy and radiotherapy is well known. Similar adverse cardiac events are resurging with the fourth modality. Aside from the conventional and newer targeted agents, even the most newly developed, immune-based therapeutic modalities of anticancer treatment (the fifth modality), e.g., immune checkpoint inhibitors and chimeric antigen receptor (CAR) T-cell therapy, have unfortunately led to potentially lethal cardiotoxicity in patients. Cardiac complications represent unresolved and potentially life-threatening conditions in cancer survivors, while effective clinical management remains quite challenging. As a consequence, morbidity and mortality related to cardiac complications now threaten to offset some favorable benefits of modern cancer treatments in cancer-related survival, regardless of the oncologic prognosis. This review focuses on identifying critical research-practice gaps, addressing real-world challenges and pinpointing real-time insights in general terms under the context of clinical cardiotoxicity induced by the fourth and fifth modalities of cancer treatment. The information ranges from basic science to clinical management in the field of cardio-oncology and crosses the interface between oncology and onco-pharmacology. The complexity of the ongoing clinical problem is addressed at different levels. A better understanding of these research-practice gaps may advance research initiatives on the development of mechanism-based diagnoses and treatments for the effective clinical management of cardiotoxicity.

Multispectral Optoacoustic Tomography of Brown Adipose Tissue

MSOT has revolutionized biomedical imaging because it allows anatomical, functional, and molecular imaging of deep tissues in vivo in an entirely noninvasive, label-free, and real-time manner. This imaging modality works by pulsing light onto tissue, triggering the production of acoustic waves, which can be collected and reconstructed to provide high-resolution images of features as deep as several centimeters below the body surface. Advances in hardware and software continue to bring MSOT closer to clinical translation. Most recently, a clinical handheld MSOT system has been used to image brown fat tissue (BAT) and its metabolic activity by directly resolving the spectral signatures of hemoglobin and lipids. This opens up new possibilities for studying BAT physiology and its role in metabolic disease without the need to inject animals or humans with contrast agents. In this chapter, we overview how MSOT works and how it has been implemented in preclinical and clinical contexts. We focus on our recent work using MSOT to image BAT in resting and activated states both in mice and humans.

Motion correction in optoacoustic mesoscopy

Raster-scan optoacoustic mesoscopy (RSOM), also termed photoacoustic mesoscopy, offers novel insights into vascular morphology and pathophysiological biomarkers of skin inflammation in vivo at depths unattainable by other optical imaging methods. Using ultra-wideband detection and focused ultrasound transducers, RSOM can achieve axial resolution of 4 micron and lateral resolution of 20 micron to depths of several millimeters. However, motion effects may deteriorate performance and reduce the effective resolution. To provide high-quality optoacoustic images in clinical measurements, we developed a motion correction algorithm for RSOM. The algorithm is based on observing disruptions of the ultrasound wave front generated by the vertical movement of the melanin layer at the skin surface. From the disrupted skin surface, a smooth synthetic surface is generated, and the offset between the two surfaces is used to correct for the relative position of the ultrasound detector. We test the algorithm in measurements of healthy and psoriatic human skin and achieve effective resolution up to 5-fold higher than before correction. We discuss the performance of the correction algorithm and its implications in the context of multispectral mesoscopy.

Future of the Renal Biopsy: Time to Change the Conventional Modality Using Nanotechnology

At the present time, imaging guided renal biopsy is used to provide diagnoses in most types of primary and secondary renal diseases. It has been claimed that renal biopsy can provide a link between diagnosis of renal disease and its pathological conditions. However, sometimes there is a considerable mismatch between patient renal outcome and pathological findings in renal biopsy. This is the time to address some new diagnostic methods to resolve the insufficiency of conventional percutaneous guided renal biopsy. Nanotechnology is still in its infancy in renal imaging; however, it seems that it is the next step in renal biopsy, providing solutions to the limitations of conventional modalities.

Self-assembled nanomaterials for photoacoustic imaging

In recent years, extensive endeavors have been paid to construct functional self-assembled nanomaterials for various applications such as catalysis, separation, energy and biomedicines. To date, different strategies have been developed for preparing nanomaterials with diversified structures and functionalities via fine tuning of self-assembled building blocks. In terms of biomedical applications, bioimaging technologies are urgently calling for high-efficient probes/contrast agents for high-performance bioimaging. Photoacoustic (PA) imaging is an emerging whole-body imaging modality offering high spatial resolution, deep penetration and high contrast in vivo. The self-assembled nanomaterials show high stability in vivo, specific tolerance to sterilization and prolonged half-life stability and desirable targeting properties, which is a kind of promising PA contrast agents for biomedical imaging. Herein, we focus on summarizing recent advances in smart self-assembled nanomaterials with NIR absorption as PA contrast agents for biomedical imaging. According to the preparation strategy of the contrast agents, the self-assembled nanomaterials are categorized into two groups, i.e., the ex situ and in situ self-assembled nanomaterials. The driving forces, assembly modes and regulation of PA properties of self-assembled nanomaterials and their applications for long-term imaging, enzyme activity detection and aggregation-induced retention (AIR) effect for diagnosis and therapy are emphasized. Finally, we conclude with an outlook towards future developments of self-assembled nanomaterials for PA imaging.

siRNA liposome-gold nanorod vectors for multispectral optoacoustic tomography theranostics

Therapeutic applications of gene silencing using siRNA have seen increasing interest over the past decade. The optimization of the delivery and biodistribution of siRNA using liposome-gold nanorod (AuNRs) nanoscale carriers can greatly benefit from adept imaging methods that can visualize the time-resolved delivery performance of such vectors. In this work, we describe the effect of AuNR length incorporated with liposomes and show their complexation with siRNA as a novel gene delivery vehicle. We demonstrate the application of multispectral optoacoustic tomography (MSOT) to longitudinally visualize the localisation of siRNA carrying liposome-AuNR hybrids within tumors. Combination of in vivo MSOT with ex vivo fluorescence cryo-slice imaging offers further insight into the siRNA transport and activity obtained.

Retrospective respiration-gated whole-body photoacoustic computed tomography of mice

Photoacoustic tomography (PAT) is an emerging technique that has a great potential for preclinical whole-body imaging. To date, most whole-body PAT systems require multiple laser shots to generate one cross-sectional image, yielding a frame rate of <1  Hz. Because a mouse breathes at up to 3 Hz, without proper gating mechanisms, acquired images are susceptible to motion artifacts. Here, we introduce, for the first time to our knowledge, retrospective respiratory gating for whole-body photoacoustic computed tomography. This new method involves simultaneous capturing of the animal's respiratory waveform during photoacoustic data acquisition. The recorded photoacoustic signals are sorted and clustered according to the respiratory phase, and an image of the animal at each respiratory phase is reconstructed subsequently from the corresponding cluster. The new method was tested in a ring-shaped confocal photoacoustic computed tomography system with a hardware-limited frame rate of 0.625 Hz. After respiratory gating, we observed sharper vascular and anatomical images at different positions of the animal body. The entire breathing cycle can also be visualized at 20  frames/cycle.

Real-time handheld multispectral optoacoustic imaging

Multispectral optoacoustic tomography (MSOT) of functional and molecular contrast has the potential to find broad deployment in clinical practice. We have developed the first handheld MSOT imaging device with fast wavelength tuning achieving a frame rate of 50 Hz. In this Letter, we demonstrate its clinical potential by dynamically resolving multiple disease-relevant tissue chromophores, including oxy-/deoxyhemoglobin, and melanin, in human volunteers.

Multispectral optoacoustic tomography of myocardial infarction

OBJECTIVES

To investigate the feasibility of a high resolution optical imaging strategy for myocardial infarction.

BACKGROUND

Near-infrared approaches to imaging cardiovascular disease enable visualization of disease-associated biological processes in vivo. However, even at the scale of small animals, the strong scattering of light prevents high resolution imaging after the first 1-2 mm of tissue, leading to degraded signal localization.

METHODS

Multispectral optoacoustic tomography (MSOT) was used to non-invasively image myocardial infarction (MI) in a murine model of coronary artery ligation at resolutions not possible with current deep-tissue optical imaging methods. Post-MI imaging was based on resolving the spectral absorption signature of a dendritic polyglycerol sulfate-based (dPGS) near-infrared imaging agent targeted to P- and L-selectin.

RESULTS

In vivo imaging succeeded in detection of the agent in the injured myocardium after intravenous injection. The high anatomic resolution (<200 μm) achieved by the described method allowed signals originating in the infarcted heart to be distinguished from uptake in adjacent regions. Histological analysis found dPGS signal in infarcted areas, originating from leukocytes and endothelial cells.

CONCLUSIONS

MSOT imaging of myocardial infarction provides non-invasive visualization of optical contrast with a high spatial resolution that is not degraded by the scattering of light.