Real-time Triple-modal Photoacoustic, Ultrasound, and Magnetic Resonance Fusion Imaging of Humans

F-DARTS: Foveated Differentiable Architecture Search Based Multimodal Medical Image Fusion

Multimodal medical image fusion (MMIF) is highly significant in such fields as disease diagnosis and treatment. The traditional MMIF methods are difficult to provide satisfactory fusion accuracy and robustness due to the influence of such possible human-crafted components as image transform and fusion strategies. Existing deep learning based fusion methods are generally difficult to ensure image fusion effect due to the adoption of a human-designed network structure and a relatively simple loss function and the ignorance of human visual characteristics during weight learning. To address these issues, we have presented the foveated differentiable architecture search (F-DARTS) based unsupervised MMIF method. In this method, the foveation operator is introduced into the weight learning process to fully explore human visual characteristics for the effective image fusion. Meanwhile, a distinctive unsupervised loss function is designed for network training by integrating mutual information, sum of the correlations of differences, structural similarity and edge preservation value. Based on the presented foveation operator and loss function, an end-to-end encoder-decoder network architecture will be searched using the F-DARTS to produce the fused image. Experimental results on three multimodal medical image datasets demonstrate that the F-DARTS performs better than several traditional and deep learning based fusion methods by providing visually superior fused results and better objective evaluation metrics.

Review on ultrasound-guided photoacoustic imaging for complementary analyses of biological systems in vivo

Photoacoustic imaging has been developed as a new biomedical molecular imaging modality. Due to its similarity to conventional ultrasound imaging in terms of signal detection and image generation, dual-modal photoacoustic and ultrasound imaging has been applied to visualize physiological and morphological information in biological systems in vivo. By complementing each other, dual-modal photoacoustic and ultrasound imaging showed synergistic advances in photoacoustic imaging with the guidance of ultrasound images. In this review, we introduce our recent progresses in dual-modal photoacoustic and ultrasound imaging systems at various scales of study, from preclinical small animals to clinical humans. A summary of the works reveals various strategies for combining the structural information of ultrasound images with the molecular information of photoacoustic images.

Ultrasound fusion imaging for improving diagnostic and therapeutic strategies of focal liver lesions: A preliminary study

PURPOSE

To assess the effect of ultrasound (US) fusion imaging on the clinical diagnostic and therapeutic strategies of focal liver lesions, which are difficult to detect or diagnose by conventional US.

METHODS

From November 2019 to June 2022, 71 patients with invisible or undiagnosed focal liver lesions who underwent fusion imaging combining US with CT or MR were included in this retrospective study. The reasons for US fusion imaging were as follows: (1) lesions that were undetectable or inconspicuous on B-mode US; (2) post-ablation lesions that could not be assessed accurately by B-mode US; (3) to evaluate whether the lesions detected by B-mode US that were consistent with those presented on MRI/CT images.

RESULTS

Of the 71 cases, 43 cases were single lesions, and 28 cases were multiple lesions. Among the 46 cases which were invisible on conventional US, the display rate of lesions using US-CT/MRI fusion imaging was 30.8%, and that combined with CEUS was 76.9%. US-guided biopsy was performed in 30 patients after the detection and localization determined by fusion imaging, with a positive rate of 73.3%. Six patients with recurrence after ablation therapy were all detected and located accurately after fusion imaging, and 4 of them successfully underwent ablation therapy again.

CONCLUSION

Fusion imaging contributes to the understanding of the anatomical relationship between lesion location and blood vessels. Additionally, fusion imaging can improve the diagnostic confidence, be helpful to guide interventional operations, and hence be conducive to clinical therapeutic strategies.

Clinical photoacoustic/ultrasound dual-modal imaging: Current status and future trends

Photoacoustic tomography (PAT) is an emerging biomedical imaging modality that combines optical and ultrasonic imaging, providing overlapping fields of view. This hybrid approach allows for a natural integration of PAT and ultrasound (US) imaging in a single platform. Due to the similarities in signal acquisition and processing, the combination of PAT and US imaging creates a new hybrid imaging for novel clinical applications. Over the recent years, particular attention is paid to the development of PAT/US dual-modal systems highlighting mutual benefits in clinical cases, with an aim of substantially improving the specificity and sensitivity for diagnosis of diseases. The demonstrated feasibility and accuracy in these efforts open an avenue of translating PAT/US imaging to practical clinical applications. In this review, the current PAT/US dual-modal imaging systems are discussed in detail, and their promising clinical applications are presented and compared systematically. Finally, this review describes the potential impacts of these combined systems in the coming future.

Photoactivated Nanohybrid for Dual-Nuclei MR/US/PA Multimodal-Guided Photothermal Therapy

Nanohybrids have gained immense popularity for the diagnosis and chemotherapy of lung cancer for their excellent biocompatibility, biodegradability, and targeting ability. However, most of them suffer from limited imaging information, low tumor-to-background ratios, and multidrug resistance, limiting their potential clinical application. Herein, we engineered a photoresponsive nanohybrid by assembling polypyrrole@bovine serum albumin (PPy@BSA) encapsulating perfluoropentane (PFP)/¹²⁹Xe for selective magnetic resonance (MR)/ultrasonic (US)/photoacoustic (PA) trimodal imaging and photothermal therapy of lung cancer, overcoming these drawbacks of single imaging modality and chemotherapy. The nanohybrid exhibited superior US, PA, and MR multimodal imaging performance for lung cancer detection. The high sensitivity of the nanohybrid to near-infrared light (NIR) resulted in a rapid increase in temperature in a low-intensity laser state, which initiated the phase transition of liquid PFP into the gas. The ultrasound signal inside the tumor, which is almost zero initially, is dramatically increased. Beyond this, it led to the complete depression of ¹⁹F/¹²⁹Xe Hyper-CEST (chemical exchange saturation transfer) MRI during laser irradiation, which can precisely locate lung cancer. In vitro and in vivo results of the nanohybrid exhibited a successful therapeutic effect on lung cancer. Under the guidance of imaging results, a sound effect of photothermal therapy (PTT) for lung cancer was achieved. We expect this nanohybrid and photosensitive behavior will be helpful as fundamental tools to decipher lung cancer in an earlier stage through trimodality imaging methods.

Video-rate full-ring ultrasound and photoacoustic computed tomography with real-time sound speed optimization

Full-ring dual-modal ultrasound and photoacoustic imaging provide complementary contrasts, high spatial resolution, full view angle and are more desirable in pre-clinical and clinical applications. However, two long-standing challenges exist in achieving high-quality video-rate dual-modal imaging. One is the increased data processing burden from the dense acquisition. Another one is the object-dependent speed of sound variation, which may cause blurry, splitting artifacts, and low imaging contrast. Here, we develop a video-rate full-ring ultrasound and photoacoustic computed tomography (VF-USPACT) with real-time optimization of the speed of sound. We improve the imaging speed by selective and parallel image reconstruction. We determine the optimal sound speed via co-registered ultrasound imaging. Equipped with a 256-channel ultrasound array, the dual-modal system can optimize the sound speed and reconstruct dual-modal images at 10 Hz in real-time. The optimized sound speed can effectively enhance the imaging quality under various sample sizes, types, or physiological states. In animal and human imaging, the system shows co-registered dual contrasts, high spatial resolution (140 µm), single-pulse photoacoustic imaging (< 50 µs), deep penetration (> 20 mm), full view, and adaptive sound speed correction. We believe VF-USPACT can advance many real-time biomedical imaging applications, such as vascular disease diagnosing, cancer screening, or neuroimaging.

Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets

Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging.

Design Principles Governing the Development of Theranostic Anticancer Agents and Their Nanoformulations with Photoacoustic Properties

The unmet need to develop novel approaches for cancer diagnosis and treatment has led to the evolution of theranostic agents, which usually include, in addition to the anticancer drug, an imaging agent based mostly on fluorescent agents. Over the past few years, a non-invasive photoacoustic imaging modality has been effectively integrated into theranostic agents. Herein, we shed light on the design principles governing the development of theranostic agents with photoacoustic properties, which can be formulated into nanocarriers to enhance their potency. Specifically, we provide an extensive analysis of their individual constituents including the imaging dyes, drugs, linkers, targeting moieties, and their formulation into nanocarriers. Along these lines, we present numerous relevant paradigms. Finally, we discuss the clinical relevance of the specific strategy, as also the limitations and future perspectives, and through this review, we envisage paving the way for the development of theranostic agents endowed with photoacoustic properties as effective anticancer medicines.

An Excitation-Reception Collinear Probe for Ultrasonic, Photoacoustic, and Thermoacoustic Tri-Modal Volumetric Imaging

Imaging systems that integrate multiple modalities can reveal complementary anatomic and functional information as they exploit different contrast mechanisms, which have shown great application potential and advantages in preclinical studies. A portable and easy-to-use imaging probe will be more conducive to transfer to clinical practice. Here, we present a tri-modal ultrasonic (US), photoacoustic (PA), and thermoacoustic (TA) imaging system with an excitation-reception collinear probe. The acoustic field, light field, and electric field of the probe were designed to be coaxial, realizing homogeneous illumination and high-sensitivity detection at the same detection position. US images can provide detailed information about structures, PA images can delineate the morphology of blood vessels in tissues, and TA images can reveal dielectric properties of the tissues. Moreover, phantoms and in vivo human finger experiments were performed by the tri-modal imaging system to demonstrate its performance. The results show that the tri-modal imaging system with the proposed probe has the ability to detect small breast tumors with a radius of only 2.5 mm and visualize the anatomical structure of the finger in three dimensions. Our work confirms that the tri-modal imaging system equipped with a collinear probe can be applied to a variety of different scenarios, which lays a solid foundation for the application of the tri-modality system in clinical trials.

Multi-modality photoacoustic/ultrasound imaging based on a commercial ultrasound platform

Multimodal imaging takes advantage of each modality and has become a recent trend in the field of biomedical imaging. In this Letter, we develop and evaluate an integrated multi-modality imaging system combining photoacoustic computed tomography, optical resolution photoacoustic microscopy, brightness mode, and power Doppler ultrasound imaging on a commercial ultrasonographic platform. Using different imaging modalities enables the hybrid system to recover dense vascular networks and hemodynamic and morphological variations in both superficial and deep tissues. To evaluate the performance and illustrate the advantages of this system, we carried out both phantom and in vivo experiments. In addition to the complementary tissue information offered by different imaging modalities, the use of a commercial ultrasound platform shows the feasibility of the proposed method for future clinical translation.

A Mixed-Signal Chip-Based Configurable Coherent Photoacoustic-Radar Sensing Platform for In Vivo Temperature Monitoring and Vital Signs Detection

For precise health status monitoring and accurate disease diagnostics in the current COVID-19 pandemic, it is essential to detect various kinds of target signals robustly under high noise and strong interferences. Moreover, the health monitoring system is preferred to be realized in a small form factor for convenient mass deployments. A CMOS-integrated coherent sensing platform is proposed to achieve the goal, which synergetically leverages quadrature coherent photoacoustic (PA) detection and coherent radar sensing for achieving universal healthcare. By utilizing configurable mixed-signal quadrature coherent PA detection, high sensitivity and enhanced specificity can be achieved. In-phase (I) and quadrature (Q) templates are specifically designed to accurately sense and precisely reconstruct the target PA signals in a coherent mode. By mixed-signal implementation leveraging an FPGA to generate template waveforms adaptively, accurate tracking and precise reconstruction on the target PA signal can be attained based on the early-late tracking principle. The multiplication between the received PA signal and the templates is implemented efficiently in analog-domain by the Gilbert cell on-chip. In vivo blood temperature monitoring was realized based on the integrated PA sensing platform fabricated in a 65-nm CMOS process. With an integrated radar sensor deployed in the indoor scenario, noncontact monitoring on respiration and heartbeat rates can be attained based on electromagnetic (EM) sensing. By complementary usage of PA-EM sensing mechanisms, comprehensive health status monitoring and precise remote disease diagnostics can be achieved for the currentglobal COVID-19 pandemic and the future pervasive healthcare in the Internet of Everything (IoE) era.

Deep learning enabled real-time photoacoustic tomography system via single data acquisition channel

Photoacoustic computed tomography (PACT) combines the optical contrast of optical imaging and the penetrability of sonography. In this work, we develop a novel PACT system to provide real-time imaging, which is achieved by a 120-elements ultrasound array only using a single data acquisition (DAQ) channel. To reduce the channel number of DAQ, we superimpose 30 nearby channels' signals together in the analog domain, and shrinking to 4 channels of data (120/30 = 4). Furthermore, a four-to-one delay-line module is designed to combine these four channels' data into one channel before entering the single-channel DAQ, followed by decoupling the signals after data acquisition. To reconstruct the image from four superimposed 30-channels' PA signals, we train a dedicated deep learning model to reconstruct the final PA image. In this paper, we present the preliminary results of phantom and in-vivo experiments, which manifests its robust real-time imaging performance. The significance of this novel PACT system is that it dramatically reduces the cost of multi-channel DAQ module (from 120 channels to 1 channel), paving the way to a portable, low-cost and real-time PACT system.

Using non-invasive neuroimaging to enhance the care, well-being and experimental outcomes of laboratory non-human primates (monkeys)

Over the past 10-20 years, neuroscience witnessed an explosion in the use of non-invasive imaging methods, particularly magnetic resonance imaging (MRI), to study brain structure and function. Simultaneously, with access to MRI in many research institutions, MRI has become an indispensable tool for researchers and veterinarians to guide improvements in surgical procedures and implants and thus, experimental as well as clinical outcomes, given that access to MRI also allows for improved diagnosis and monitoring for brain disease. As part of the PRIMEatE Data Exchange, we gathered expert scientists, veterinarians, and clinicians who treat humans, to provide an overview of the use of non-invasive imaging tools, primarily MRI, to enhance experimental and welfare outcomes for laboratory non-human primates engaged in neuroscientific experiments. We aimed to provide guidance for other researchers, scientists and veterinarians in the use of this powerful imaging technology as well as to foster a larger conversation and community of scientists and veterinarians with a shared goal of improving the well-being and experimental outcomes for laboratory animals.

Video-Rate Ring-Array Ultrasound and Photoacoustic Tomography

Ultrasonography and photoacoustic tomography provide complementary contrasts in preclinical studies, disease diagnoses, and imaging-guided interventional procedures. Here, we present a video-rate (20 Hz) dual-modality ultrasound and photoacoustic tomographic platform that has a high resolution, rich contrasts, deep penetration, and wide field of view. A three-quarter ring-array ultrasonic transducer is used for both ultrasound and photoacoustic imaging. Plane-wave transmission/receiving approach is used for ultrasound imaging, which improves the imaging speed by nearly two folds and reduces the RF data size compared with the sequential single-channel scanning approach. GPU-based image reconstruction is developed to advance computational speed. We demonstrate fast dual-modality imaging in phantom, mouse, and human finger joint experiments. The results show respiration motion, heart beating, and detailed features in the mouse internal organs. To our knowledge, this is the first report on fast plane-wave ultrasound imaging and single-shot photoacoustic computed tomography in a ring-array system.

Medical Imaging of Microrobots: Toward In Vivo Applications

Medical microrobots (MRs) have been demonstrated for a variety of non-invasive biomedical applications, such as tissue engineering, drug delivery, and assisted fertilization, among others. However, most of these demonstrations have been carried out in in vitro settings and under optical microscopy, being significantly different from the clinical practice. Thus, medical imaging techniques are required for localizing and tracking such tiny therapeutic machines when used in medical-relevant applications. This review aims at analyzing the state of the art of microrobots imaging by critically discussing the potentialities and limitations of the techniques employed in this field. Moreover, the physics and the working principle behind each analyzed imaging strategy, the spatiotemporal resolution, and the penetration depth are thoroughly discussed. The paper deals with the suitability of each imaging technique for tracking single or swarms of MRs and discusses the scenarios where contrast or imaging agent's inclusion is required, either to absorb, emit, or reflect a determined physical signal detected by an external system. Finally, the review highlights the existing challenges and perspective solutions which could be promising for future in vivo applications.

In Vivo Dual-Modal Photoacoustic and Ultrasound Imaging of Sentinel Lymph Nodes Using a Solid-State Dye Laser System

Photoacoustic imaging (PAI) is being actively investigated as a non-invasive and non-radioactive imaging technique for sentinel lymph node (SLN) biopsy. By taking advantage of optical and ultrasound imaging, PAI probes SLNs non-invasively with methylene blue (MB) in both live animals and breast cancer patients. However, these PAI systems have limitations for widespread use in clinics and commercial marketplaces because the lasers used by the PAI systems, e.g., tunable liquid dye laser systems and optical parametric oscillator (OPO) lasers, are bulky in size, not economical, and use risky flammable and toxic liquid dyes. To overcome these limitations, we are proposing a novel dual-modal photoacoustic and ultrasound imaging system based on a solid-state dye laser (SD-PAUSI), which is compact, convenient, and carries far less risk of flammability and toxicity. Using a solid-state dye handpiece that generates 650-nm wavelength, we successfully imaged the MB tube positioned deeply (~3.9 cm) in chicken breast tissue. The SLNs were also photoacoustically detected in the in vivo rats beneath a 2.2-cm-thick layer of chicken breast, which is deeper than the typical depth of SLNs in humans (1.2 ± 0.5 cm). Furthermore, we showed the multispectral capability of the PAI by switching the dye handpiece, in which the MB-dyed SLN was selectively highlighted from the surrounding vasculature. These results demonstrated the great potential of the SD-PAUSI as an easy but effective modality for SLN detection.

Three-dimensional clinical handheld photoacoustic/ultrasound scanner

Clinical 2D photoacoustic (PA) imaging can be easily implemented in a traditional ultrasound (US) system. However, 3D PA imaging is still preferable because 2D B-mode PA/US imaging suffers from low reproducibility and high-operator dependency. Here, we demonstrate a compact clinical handheld 3D PA/US scanner using an 1D linear array US transducer combined with a mechanical scanning stage working via a Scotch yoke mechanism. The entire scanner measures just 100 × 80 × 100 mm3 and weighs only 950 g, so it can easily be operated by hand. Blood vessels and hemoglobin oxygen saturation images of different parts of the human body (e.g., neck, wrist, thigh, and instep) have been successfully acquired. The system can potentially be used for clinical applications in fields such as oncology, dermatology, nephrology, and internal medicine.

Towards clinical photoacoustic and ultrasound imaging: Probe improvement and real-time graphical user interface

Photoacoustic imaging is a non-invasive and non-ionizing biomedical technique that has been investigated widely for various clinical applications. By taking the advantages of conventional ultrasound imaging, hand-held operation with a linear array transducer should be favorable for successful clinical translation of photoacoustic imaging. In this paper, we present new key updates contributed to the previously developed real-time clinical photoacoustic and ultrasound imaging system for improving the clinical usability of the system. We developed a seamless image optimization platform, designed a real-time parameter control software with a user-friendly graphical user interface, performed Monte Carlo simulation of the optical fluence in the imaging plane, and optimized the geometry of the imaging probe. The updated system allows optimizing of all imaging parameters while continuously acquiring the photoacoustic and ultrasound images in real-time. The updated system has great potential to be used in a variety of clinical applications such as assessing the malignancy of thyroid cancer, breast cancer, and melanoma.

Impact statement

Photoacoustic imaging is a promising biomedical imaging modality that can visualize both structural and functional information of biological tissue. Because of its easiness to be integrated with conventional ultrasound imaging systems, numerous studies have been conducted to develop and apply clinical photoacoustic imaging systems. However, most of the systems were not suitable for general-purpose clinical applications due to one of the following reasons: target specific design, immobility, inaccessible operation sequence, and lack of hand-held operation. This study demonstrates a real-time clinical photoacoustic and ultrasound imaging system, which can overcome the limitations of the previous systems for successful clinical translation.

A Digital-Enhanced Chip-Scale Photoacoustic Sensor System for Blood Core Temperature Monitoring and In Vivo Imaging

Monolithic integration of photoacoustic (PA) sensor with compact size, lightweight, and low power consumption is attractive to be implemented on wearable medical devices for in vivo blood metabolic sensing and imaging. This work presents a miniaturized chip-scale mixed-signal photoacoustic sensor system which can achieve coherent lock-in function to detect weak target PA signals noninvasively at in vivo scenarios of poor signal to noise ratio (SNR) and strong interferences. A low-noise amplifier (LNA), a 3rd order Butterworth low-pass filter (LPF), and a variable-gain amplifier (VGA) chain with 10 MHz cutoff frequency are implemented on-chip to attain a high-quality sensing performance with 50-dB dynamic range. A Gilbert-cell type multiplier is integrated on-chip to fulfill the coherent lock-in process on acquired PA signals in a closed-loop process with an embedded FPGA system. Fabricated in 65-nm CMOS technology, the prototype PA sensor system demonstrated 50 μV sensitivity. The functions of the chip-scale PA sensor system enhanced by coherent lock-in process were validated through the experiments on temperature monitoring and vessel imaging. The PA receiver chip occupies an area of 0.6 mm² and consumes 20 mW at a 1.8-V supply.

Minimally invasive photoacoustic imaging: Current status and future perspectives

Photoacoustic imaging (PAI) is an emerging biomedical imaging modality that is based on optical absorption contrast, capable of revealing distinct spectroscopic signatures of tissue at high spatial resolution and large imaging depths. However, clinical applications of conventional non-invasive PAI systems have been restricted to examinations of tissues at depths less than a few cm due to strong light attenuation. Minimally invasive photoacoustic imaging (miPAI) has greatly extended the landscape of PAI by delivering excitation light within tissue through miniature fibre-optic probes. In the past decade, various miPAI systems have been developed with demonstrated applicability in several clinical fields. In this article, we present an overview of the current status of miPAI and our thoughts on future perspectives.

Review on practical photoacoustic microscopy

Photoacoustic imaging (PAI) has many interesting advantages, such as deep imaging depth, high image resolution, and high contrast to intrinsic and extrinsic chromophores, enabling morphological, functional, and molecular imaging of living subjects. Photoacoustic microscopy (PAM) is one form of the PAI inheriting its characteristics and is useful in both preclinical and clinical research. Over the years, PAM systems have been evolved in several forms and each form has its relative advantages and disadvantages. Thus, to maximize the benefits of PAM for a specific application, it is important to configure the PAM system optimally by targeting a specific application. In this review, we provide practical methods for implementing a PAM system to improve the resolution, signal-to-noise ratio (SNR), and imaging speed. In addition, we review the preclinical and the clinical applications of PAM and discuss the current challenges and the scope for future developments.

Real-time delay-multiply-and-sum beamforming with coherence factor for in vivo clinical photoacoustic imaging of humans

In the clinical photoacoustic (PA) imaging, ultrasound (US) array transducers are typically used to provide B-mode images in real-time. To form a B-mode image, delay-and-sum (DAS) beamforming algorithm is the most commonly used algorithm because of its ease of implementation. However, this algorithm suffers from low image resolution and low contrast drawbacks. To address this issue, delay-multiply-and-sum (DMAS) beamforming algorithm has been developed to provide enhanced image quality with higher contrast, and narrower main lobe compared but has limitations on the imaging speed for clinical applications. In this paper, we present an enhanced real-time DMAS algorithm with modified coherence factor (CF) for clinical PA imaging of humans in vivo. Our algorithm improves the lateral resolution and signal-to-noise ratio (SNR) of original DMAS beamformer by suppressing the background noise and side lobes using the coherence of received signals. We optimized the computations of the proposed DMAS with CF (DMAS-CF) to achieve real-time frame rate imaging on a graphics processing unit (GPU). To evaluate the proposed algorithm, we implemented DAS and DMAS with/without CF on a clinical US/PA imaging system and quantitatively assessed their processing speed and image quality. The processing time to reconstruct one B-mode image using DAS, DAS with CF (DAS-CF), DMAS, and DMAS-CF algorithms was 7.5, 7.6, 11.1, and 11.3 ms, respectively, all achieving the real-time imaging frame rate. In terms of the image quality, the proposed DMAS-CF algorithm improved the lateral resolution and SNR by 55.4% and 93.6 dB, respectively, compared to the DAS algorithm in the phantom imaging experiments. We believe the proposed DMAS-CF algorithm and its real-time implementation contributes significantly to the improvement of imaging quality of clinical US/PA imaging system.

MRI and PET image fusion using the nonparametric density model and the theory of variable-weight

Medical image fusion is important in the field of clinical diagnosis because it can improve the availability of information contained in images. Magnetic Resonance Imaging (MRI) provides excellent anatomical details as well as functional information on regional changes in physiology, hemodynamics, and tissue composition. In contrast, although the spatial resolution of Positron Emission Tomography (PET) provides is lower than that an MRI, PET is capable of depicting the tissue's molecular and pathological activities that are not available from MRI. Fusion of MRI and PET may allow us to combine the advantages of both imaging modalities and achieve more precise localization and characterization of abnormalities. Previous image fusion algorithms, based on the estimation theory, assume that all distortions follow Gaussian distribution and are therefore susceptible to the model mismatch problem. To overcome this mismatch problem, we propose a new image fusion method with multi-resolution and nonparametric density models (MRNDM). The RGB space registered from the source multi-modal medical images is first transformed into a generalized intensity-hue-saturation space (GIHS), and then is decomposed into the low- and high-frequency components using the non-subsampled contourlet transform (NSCT). Two different fusion rules, which are based on the nonparametric density model and the theory of variable-weight, are developed and used to fuse low- and high-frequency coefficients. The fused images are constructed by performing the inverse of the NSCT operation with all composite coefficients. Our experimental results demonstrate that the quality of images fused from PET and MRI brain images using our proposed method MRNDM is higher than that of those fused using six previous fusion methods.

Automated registration of magnetic resonance imaging and optoacoustic tomography data for experimental studies

Multimodal imaging combining optoacoustic tomography (OAT) with magnetic resonance imaging (MRI) enables spatiotemporal resolution complementarity, improves accurate quantification, and thus yields more insights into physiology and pathophysiology. However, only manual landmark based coregistration of OAT-MRI has been used so far. We developed a toolbox (RegOA), which frames an automated registration pipeline to align OAT with high-field MR images based on mutual information. We assessed the performance of the registration method using images acquired on one phantom with fiducial markers and in vivo/ex vivo data of mouse heads/brain. The accuracy and robustness of the registration are improved using a two-step registration method with preprocessing of OAT and MRI data. The major advantages of our approach are minimal user input and quantitative assessment of the registration error. The registration with MR and standard reference atlas enables regional information extraction, facilitating the accurate, objective, and rapid analysis of large groups of rodent OAT and MR images.

Real-Time Photoacoustic Thermometry Combined With Clinical Ultrasound Imaging and High-Intensity Focused Ultrasound

High-intensity focused ultrasound (HIFU) treatment is a promising non-invasive method for killing or destroying the diseased tissues by locally delivering thermal and mechanical energy without damaging surrounding normal tissues. In HIFU, measuring the temperature at the site of delivery is important for improving therapeutic efficacy, controlling safety, and appropriately planning a treatment. Several researchers have proposed photoacoustic thermometry for monitoring HIFU treatment, but they had many limitations, including the inability to image while the HIFU is on, inability to provide two-dimensional monitoring, and the inability to be used clinically. In this paper, we propose a novel integrated real-time photoacoustic thermometry system for HIFU treatment monitoring. The system provides ultrasound B-mode imaging, photoacoustic structural imaging, and photoacoustic thermometry during HIFU treatment in real-time for both in vitro and in vivo environments, without any interference from the strong therapeutic HIFU waves. We have successfully tested the real-time photoacoustic thermometry by investigating the relationship between the photoacoustic amplitude and the measured temperature with in vitro phantoms and in vivo tumor-bearing mice. The results show the feasibility of a real-time photoacoustic thermometry system for safe and effective monitoring of HIFU treatment.

Bi2Se3 nanoplates for contrast-enhanced photoacoustic imaging at 1064 nm

Photoacoustic (PA) imaging is a high-resolution biomedical imaging modality, which can be used to visualize biological tissues located beyond the limited penetration depth of existing optical imaging techniques. An optical wavelength of 1064 nm is of great interest in PA imaging due to low intrinsic absorption at this wavelength. Reduced absorption implies an increased depth of imaging, which enables several new clinical applications such as bladder imaging, gastrointestinal (GI) imaging, and sentinel lymph node (SLN) imaging. In addition, a 1064 nm Nd:YAG laser system enables a high power, cost-effective, and compact laser-based PA imaging system. However, at this wavelength, due to low intrinsic contrast, high absorption exogenous PA contrast agents are necessary for imaging. To this end, we present new Bi2Se3 nanoplates as PA contrast agents at 1064 nm wavelength for PA imaging. We successfully synthesized Bi2Se3 nanoplates and they exhibited relatively strong PA signals at 1064 nm. We confirmed the increased imaging depth of penetration by imaging the Bi2Se3-containing tube located 4.6 cm deep in biological tissues. We present in vivo PA imaging of the bladder, GI tract, and SLN in mice using a Bi2Se3 contrast agent establishing the clinical feasibility of these agents with a clinical photoacoustic/ultrasound imaging system. Our results confirm that Bi2Se3 nanoplates are promising PA contrast agents at 1064 nm that offer a high optical absorbance in the second NIR region providing a high contrast imaging and increased depth of penetration.

Handheld Real-Time LED-Based Photoacoustic and Ultrasound Imaging System for Accurate Visualization of Clinical Metal Needles and Superficial Vasculature to Guide Minimally Invasive Procedures

Ultrasound imaging is widely used to guide minimally invasive procedures, but the visualization of the invasive medical device and the procedure’s target is often challenging. Photoacoustic imaging has shown great promise for guiding minimally invasive procedures, but clinical translation of this technology has often been limited by bulky and expensive excitation sources. In this work, we demonstrate the feasibility of guiding minimally invasive procedures using a dual-mode photoacoustic and ultrasound imaging system with excitation from compact arrays of light-emitting diodes (LEDs) at 850 nm. Three validation experiments were performed. First, clinical metal needles inserted into biological tissue were imaged. Second, the imaging depth of the system was characterized using a blood-vessel-mimicking phantom. Third, the superficial vasculature in human volunteers was imaged. It was found that photoacoustic imaging enabled needle visualization with signal-to-noise ratios that were 1.2 to 2.2 times higher than those obtained with ultrasound imaging, over insertion angles of 26 to 51 degrees. With the blood vessel mimicking phantom, the maximum imaging depth was 38 mm. The superficial vasculature of a human middle finger and a human wrist were clearly visualized in real-time. We conclude that the LED-based system is promising for guiding minimally invasive procedures with peripheral tissue targets.

Clinical photoacoustic imaging platforms

Photoacoustic imaging (PAI) is a new promising medical imaging technology available for diagnosing and assessing various pathologies. PAI complements existing imaging modalities by providing information not currently available for diagnosing, e.g., oxygenation level of the underlying tissue. Currently, researchers are translating PAI from benchside to bedside to make unique clinical advantages of PAI available for patient care. The requirements for a successful clinical PAI system are; deeper imaging depth, wider field of view, and faster scan time than the laboratory-level PAI systems. Currently, many research groups and companies are developing novel technologies for data acquisition/signal processing systems, detector geometry, and an acoustic sensor. In this review, we summarize state-of-the-art clinical PAI systems with three types of the imaging transducers: linear array transducer, curved linear array transducer, and volumetric array transducer. We will also discuss the limitations of the current PAI systems and describe latest techniques being developed to address these for further enhancing the image quality of PAI for successful clinical translation.

Compressed Sensing With a Gaussian Scale Mixture Model for Limited View Photoacoustic Computed Tomography In Vivo

Photoacoustic computed tomography using an ultrasonic array is an attractive noninvasive imaging modality for many biomedical applications. However, the potentially long data acquisition time of array-based photoacoustic computed tomography—usually due to the required time-multiplexing for multiple laser pulses—decreases its applicability for rapid disease diagnoses and the successive monitoring of physiological functions. Compressed sensing is used to improve the imaging speed of photoacoustic computed tomography by decreasing the amount of acquired data; however, the imaging quality can be limited when fewer measurements are used, as traditional compressed sensing considers only the sparsity of the signals in the imaging process. In this work, an advanced compressed sensing reconstruction framework with a Wiener linear estimation-based Gaussian scale mixture model was developed for limited view photoacoustic computed tomography. In this method, the structure dependencies of signals in the wavelet domain were incorporated into the imaging framework through the Gaussian scale mixture model, and an operator based on the Wiener linear estimation was designed to filter the reconstruction artifacts. Phantom and human forearm imaging were performed to verify the developed method. The results demonstrated that compressed sensing with a Wiener linear estimation-based Gaussian scale mixture model more effectively suppressed the reconstruction artifacts of sparse-sampling photoacoustic computed tomography and recovered photoacoustic images with a higher contrast-to-noise ratio and edge resolution than the traditional compressed sensing method. This work may promote the development of low-cost photoacoustic computed tomography techniques with rapid data acquisition and enhance the performance of photoacoustic computed tomography in various biomedical studies.