In vivo virtual intraoperative surgical photoacoustic microscopy

Segmentation and Quantitative Analysis of Photoacoustic Imaging: A Review

Photoacoustic imaging is an emerging biomedical imaging technique that combines optical contrast and ultrasound resolution to create unprecedented light absorption contrast in deep tissue. Thanks to its fusional imaging advantages, photoacoustic imaging can provide multiple structural and functional insights into biological tissues such as blood vasculatures and tumors and monitor the kinetic movements of hemoglobin and lipids. To better visualize and analyze the regions of interest, segmentation and quantitative analyses were used to extract several biological factors, such as the intensity level changes, diameter, and tortuosity of the tissues. Over the past 10 years, classical segmentation methods and advances in deep learning approaches have been utilized in research investigations. In this review, we provide a comprehensive review of segmentation and quantitative methods that have been developed to process photoacoustic imaging in preclinical and clinical experiments. We focus on the parametric reliability of quantitative analysis for semantic and instance-level segmentation. We also introduce the similarities and alternatives of deep learning models in qualitative measurements using classical segmentation methods for photoacoustic imaging.

Spherical-matching hyperbolic-array photoacoustic computed tomography

Linear-array photoacoustic computed tomography (LA-PACT), for its flexibility and simplicity, has great potential in providing anatomical and functional information of tissues. However, the limited coverage view impedes the LA-PACT obtaining high-quality images. In this study, a photoacoustic tomographic system with a hyperbolic-array transducer was developed for stereoscopic PA imaging of carotid artery. The hyperbolic-array PACT increases the receiving sensitivity for PA signal detection due to its transducer's geometric structure matching with the spherical wave. The control phantom experiment shows that the proposed system can expand the angular coverage of ∼1/3 more than that of the LA-PACT system, and the volumetric PA images of rat's carotid artery demonstrates the potential of the system for carotid artery imaging. Furthermore, volumetric imaging of the human forearm verifies that the system has significant capability in human imaging, which indicates that it has bright prospect for assisting diagnosis in the vascular disease.

Comprehensive review of surgical microscopes: technology development and medical applications

SIGNIFICANCE

Surgical microscopes provide adjustable magnification, bright illumination, and clear visualization of the surgical field and have been increasingly used in operating rooms. State-of-the-art surgical microscopes are integrated with various imaging modalities, such as optical coherence tomography (OCT), fluorescence imaging, and augmented reality (AR) for image-guided surgery.

AIM

This comprehensive review is based on the literature of over 500 papers that cover the technology development and applications of surgical microscopy over the past century. The aim of this review is threefold: (i) providing a comprehensive technical overview of surgical microscopes, (ii) providing critical references for microscope selection and system development, and (iii) providing an overview of various medical applications.

APPROACH

More than 500 references were collected and reviewed. A timeline of important milestones during the evolution of surgical microscope is provided in this study. An in-depth technical overview of the optical system, mechanical system, illumination, visualization, and integration with advanced imaging modalities is provided. Various medical applications of surgical microscopes in neurosurgery and spine surgery, ophthalmic surgery, ear-nose-throat (ENT) surgery, endodontics, and plastic and reconstructive surgery are described.

RESULTS

Surgical microscopy has been significantly advanced in the technical aspects of high-end optics, bright and shadow-free illumination, stable and flexible mechanical design, and versatile visualization. New imaging modalities, such as hyperspectral imaging, OCT, fluorescence imaging, photoacoustic microscopy, and laser speckle contrast imaging, are being integrated with surgical microscopes. Advanced visualization and AR are being added to surgical microscopes as new features that are changing clinical practices in the operating room.

CONCLUSIONS

The combination of new imaging technologies and surgical microscopy will enable surgeons to perform challenging procedures and improve surgical outcomes. With advanced visualization and improved ergonomics, the surgical microscope has become a powerful tool in neurosurgery, spinal, ENT, ophthalmic, plastic and reconstructive surgeries.

Photoacoustic viscoelasticity imaging for the detection of acute hepatitis: a feasibility study

Biomechanical assessments are essential for the understanding of physiological states and the characterization of certain tissue pathologies such as liver cirrhosis. In this work, we showed by the photoacoustic viscoelasticity (PAVE) imaging that obvious mechanical change was also observed in the development of the acute hepatitis owing to the hepatocyte enlargement and intracellular fluid increment, indicating that the PAVE technique can be developed as a supplementary method for detecting acute hepatitis in future. The feasibility of the PAVE imaging is validated by a group of agar phantoms. Furthermore, acute hepatitis pathological animal models were established and imaged ex vivo and in situ by the PAVE technique to demonstrate its capability for the mechanical characterization of acute hepatitis, and the imaging results were consistent with pathological results. The feasibility study of detecting acute hepatitis by the PAVE technique proved that this method has potential to be developed as a clinical biomechanical imaging method to supplement current clinical strategy for liver disease detection.

Image Overlay Surgery Based on Augmented Reality: A Systematic Review

Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. AR-based image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient's position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient's body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients' recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach facilitates an easy development of mobile AR applications and thus the implementation of AR-based image overlay surgery in clinical practice.

Clinical Utility of Intraoperative Tympanomastoidectomy Assessment Using a Surgical Microscope Integrated with an Optical Coherence Tomography

Significant technical and optical advances are required for intraoperative optical coherence tomography (OCT) to be utilized during otological surgeries. Integrating OCT with surgical microscopy makes it possible to evaluate soft tissue in real-time and at a high resolution. Herein, we describe an augmented-reality, intraoperative OCT/microscope system with an extended working distance of 280 mm, providing more space for surgical manipulation than conventional techniques. We initially performed ex vivo experiments to evaluate system performance. In addition, we validated the system by performing preliminary clinical assessments of tympanomastoidectomy outcomes in six patients with chronic otitis media. The system evaluated residual inflammation in the region-of-interest of the mastoid bone. Most importantly, the system intraoperatively revealed the connection between the graft and the remnant tympanic membrane. The extended working distance allows otological surgeons to evaluate the status of both the mastoid bone and tympanic membrane during manipulation, affording full intraoperative imaging.

Fast photoacoustic imaging systems using pulsed laser diodes: a review

Photoacoustic imaging (PAI) is a newly emerging imaging modality for preclinical and clinical applications. The conventional PAI systems use Q-switched Nd:YAG/OPO (Optical Parametric Oscillator) nanosecond lasers as excitation sources. Such lasers are expensive, bulky, and imaging speed is limited because of low pulse repetition rate. In recent years, the semiconductor laser technology has advanced to generate high-repetitions rate near-infrared pulsed lasers diodes (PLDs) which are reliable, less-expensive, hand-held, and light-weight, about 200 g. In this article, we review the development and demonstration of PLD based PAI systems for preclinical and clinical applications reported in recent years.

In Vivo Near Infrared Virtual Intraoperative Surgical Photoacoustic Optical Coherence Tomography

Since its first implementation in otolaryngological surgery nearly a century ago, the surgical microscope has improved the accuracy and the safety of microsurgeries. However, the microscope shows only a magnified surface view of the surgical region. To overcome this limitation, either optical coherence tomography (OCT) or photoacoustic microscopy (PAM) has been independently combined with conventional surgical microscope. Herein, we present a near-infrared virtual intraoperative photoacoustic optical coherence tomography (NIR-VISPAOCT) system that combines both PAM and OCT with a conventional surgical microscope. Using optical scattering and absorption, the NIR-VISPAOCT system simultaneously provides surgeons with real-time comprehensive biological information such as tumor margins, tissue structure, and a magnified view of the region of interest. Moreover, by utilizing a miniaturized beam projector, it can back-project 2D cross-sectional PAM and OCT images onto the microscopic view plane. In this way, both microscopic and cross-sectional PAM and OCT images are concurrently displayed on the ocular lens of the microscope. To verify the usability of the NIR-VISPAOCT system, we demonstrate simulated surgeries, including in vivo image-guided melanoma resection surgery and in vivo needle injection of carbon particles into a mouse thigh. The proposed NIR-VISPAOCT system has potential applications in neurosurgery, ophthalmological surgery, and other microsurgeries.

Real-time Near-infrared Virtual Intraoperative Surgical Photoacoustic Microscopy

We developed a near infrared (NIR) virtual intraoperative surgical photoacoustic microscopy (NIR-VISPAM) system that combines a conventional surgical microscope and an NIR light photoacoustic microscopy (PAM) system. NIR-VISPAM can simultaneously visualize PA B-scan images at a maximum display rate of 45 Hz and display enlarged microscopic images on a surgeon's view plane through the ocular lenses of the surgical microscope as augmented reality. The use of the invisible NIR light eliminated the disturbance to the surgeon's vision caused by the visible PAM excitation laser in a previous report. Further, the maximum permissible laser pulse energy at this wavelength is approximately 5 times more than that at the visible spectral range. The use of a needle-type ultrasound transducer without any water bath for acoustic coupling can enhance convenience in an intraoperative environment. We successfully guided needle and injected carbon particles in biological tissues ex vivo and in melanoma-bearing mice in vivo.

A PDMS-Based 2-Axis Waterproof Scanner for Photoacoustic Microscopy

Optical-resolution photoacoustic microscopy (OR-PAM) is an imaging tool to provide in vivo optically sensitive images in biomedical research. To achieve a small size, fast imaging speed, wide scan range, and high signal-to-noise ratios (SNRs) in a water environment, we introduce a polydimethylsiloxane (PDMS)-based 2-axis scanner for a flexible and waterproof structure. The design, theoretical background, fabrication process and performance of the scanner are explained in details. The designed and fabricated scanner has dimensions of 15 × 15 × 15 mm along the X, Y and Z axes, respectively. The characteristics of the scanner are tested under DC and AC conditions. By pairing with electromagnetic forces, the maximum scanning angles in air and water are 18° and 13° along the X and Y axes, respectively. The measured resonance frequencies in air and water are 60 and 45 Hz along the X axis and 45 and 30 Hz along the Y axis, respectively. Finally, OR-PAM with high SNRs is demonstrated using the fabricated scanner, and the PA images of micro-patterned samples and microvasculatures of a mouse ear are successfully obtained with high-resolution and wide-field of view. OR-PAM equipped with the 2-axis PDMS based waterproof scanner has lateral and axial resolutions of 3.6 μm and 26 μm, respectively. This compact OR-PAM system could potentially and widely be used in preclinical and clinical applications.