Image-guidance for surgical procedures

Guidelines for management of actual or suspected inadvertent intra-arterial injection of sclerosants

BACKGROUND

Inadvertent intra-arterial injection of sclerosants is an uncommon adverse event of both ultrasound-guided and direct vision sclerotherapy. This complication can result in significant tissue or limb loss and significant long-term morbidity.

OBJECTIVES

To provide recommendations for diagnosis and immediate management of an unintentional intra-arterial injection of sclerosing agents.

METHODS

An international and multidisciplinary expert panel representing the endorsing societies and relevant specialities reviewed the published biomedical, scientific and legal literature and developed the consensus-based recommendations.

RESULTS

Actual and suspected cases of an intra-arterial sclerosant injection should be immediately transferred to a facility with a vascular/interventional unit. Digital Subtraction Angiography (DSA) is the key investigation to confirm the diagnosis and help select the appropriate intra-arterial therapy for tissue ischaemia. Emergency endovascular intervention will be required to manage the risk of major limb ischaemia. This includes intra-arterial administration of vasodilators to reduce vasospasm, and anticoagulants and thrombolytic agents to mitigate thrombosis. Mechanical thrombectomy, other endovascular interventions and even open surgery may be required. Lumbar sympathetic block may be considered but has a high risk of bleeding. Systemic anti-inflammatory agents, anticoagulants, and platelet inhibitors and modifiers would complement the intra-arterial endovascular procedures. For risk of minor ischaemia, systemic oral anti-inflammatory agents, anticoagulants, vasodilators and antiplatelet treatments are recommended.

CONCLUSION

Inadvertent intra-arterial injection is an adverse event of both ultrasound-guided and direct vision sclerotherapy. Medical practitioners performing sclerotherapy must ensure completion of a course of formal training (specialty or subspecialty training, or equivalent recognition) in the management of venous and lymphatic disorders (phlebology), and be personally proficient in the use of duplex ultrasound in vascular (both arterial and venous) applications, to diagnose and provide image guidance to venous procedure. Expertise in diagnosis and immediate management of an intra-arterial injection is essential for all practitioners performing sclerotherapy.

Ultrasound and Photoacoustic Imaging for the Guidance of Laser Ablation Procedures

The accuracy and efficacy of laser ablation procedures depend on the accurate placement of the laser applicator within the diseased tissue, monitoring the real-time temperature during the ablation procedure, and mapping the extent of the ablated region. Ultrasound (US) imaging has been widely used to guide ablation procedures. While US imaging offers significant advantages for guiding ablation procedures, its limitations include low imaging contrast, angular dependency, and limited ability to monitor the temperature. Photoacoustic (PA) imaging is a relatively new imaging modality that inherits the advantages of US imaging and offers enhanced capabilities for laser-guided ablations, such as accurate, angle-independent tracking of ablation catheters, the potential for quantitative thermometry, and monitoring thermal lesion formation. This work provides an overview of ultrasound-guided procedures and how different US-related artifacts limit their utility, followed by introducing PA as complementary to US as a solution to address the existing limitations and improve ablation outcomes. Furthermore, we highlight the integration of PA-driven features into existing US-guided laser ablation systems, along with their limitations and future outlooks. Integrated US/PA-guided laser ablation procedures can lead to safer and more precise treatment outcomes.

Fluorescence imaging of peripheral nerve function and structure

Peripheral nerve injuries are common and can cause catastrophic consequences. Although peripheral nerves have notable regenerative capacity, full functional recovery is often challenging due to a number of factors, including age, the type of injury, and delayed healing, resulting in chronic disorders that cause lifelong miseries and significant financial burdens. Fluorescence imaging, among the various techniques, may be the key to overcome these restrictions and improve the prognosis because of its feasibility and dynamic real-time imaging. Intraoperative dynamic fluorescence imaging allows the visualization of the morphological structure of the nerve so that surgeons can reduce the incidence of medically induced injury. Axoplasmic transport-based neuroimaging allows the visualization of the internal transport function of the nerve, facilitating early, objective, and accurate assessment of the degree of regenerative repair, allowing early intervention in patients with poor recovery, thereby improving prognosis. This review briefly discusses peripheral nerve fluorescent dyes that have been reported or could potentially be employed, with a focus on their role in visualizing the nerve's function and anatomy.

Unsupervised registration of 3D knee implant components to biplanar X-ray images

BACKGROUND

Registration of three-dimensional (3D) knee implant components to radiographic images provides the 3D position of the implants which aids to analyze the component alignment after total knee arthroplasty.

METHODS

We present an automatic 3D to two-dimensional (2D) registration using biplanar radiographic images based on a hybrid similarity measure integrating region and edge-based information. More precisely, this measure is herein defined as a weighted combination of an edge potential field-based similarity, which represents the relation between the external contours of the component projections and an edge potential field estimated on the two radiographic images, and an object specificity property, which is based on the distinction of the region-label inside and outside of the object.

RESULTS

The accuracy of our 3D/2D registration algorithm was assessed on a sample of 64 components (32 femoral components and 32 tibial components). In our tests, we obtained an average of the root mean square error (RMSE) of 0.18 mm, which is significantly lower than that of both single similarity methods, supporting our hypothesis of better stability and accuracy with the proposed approach.

CONCLUSION

Our method, which provides six accurate registration parameters (three rotations and three translations) without requiring any fiducial markers, makes it possible to perform the important analyses on the rotational alignment of the femoral and tibial components on a large number of cases. In addition, this method can be extended to register other implants or bones.

Niche preclinical and clinical applications of photoacoustic imaging with endogenous contrast

In the past decade, photoacoustic (PA) imaging has attracted a great deal of popularity as an emergent diagnostic technology owing to its successful demonstration in both preclinical and clinical arenas by various academic and industrial research groups. Such steady growth of PA imaging can mainly be attributed to its salient features, including being non-ionizing, cost-effective, easily deployable, and having sufficient axial, lateral, and temporal resolutions for resolving various tissue characteristics and assessing the therapeutic efficacy. In addition, PA imaging can easily be integrated with the ultrasound imaging systems, the combination of which confers the ability to co-register and cross-reference various features in the structural, functional, and molecular imaging regimes. PA imaging relies on either an endogenous source of contrast (e.g., hemoglobin) or those of an exogenous nature such as nano-sized tunable optical absorbers or dyes that may boost imaging contrast beyond that provided by the endogenous sources. In this review, we discuss the applications of PA imaging with endogenous contrast as they pertain to clinically relevant niches, including tissue characterization, cancer diagnostics/therapies (termed as theranostics), cardiovascular applications, and surgical applications. We believe that PA imaging's role as a facile indicator of several disease-relevant states will continue to expand and evolve as it is adopted by an increasing number of research laboratories and clinics worldwide.

Medical instrument detection in ultrasound: a review

Medical instrument detection is essential for computer-assisted interventions, since it facilitates clinicians to find instruments efficiently with a better interpretation, thereby improving clinical outcomes. This article reviews image-based medical instrument detection methods for ultrasound-guided (US-guided) operations. Literature is selected based on an exhaustive search in different sources, including Google Scholar, PubMed, and Scopus. We first discuss the key clinical applications of medical instrument detection in the US, including delivering regional anesthesia, biopsy taking, prostate brachytherapy, and catheterization. Then, we present a comprehensive review of instrument detection methodologies, including non-machine-learning and machine-learning methods. The conventional non-machine-learning methods were extensively studied before the era of machine learning methods. The principal issues and potential research directions for future studies are summarized for the computer-assisted intervention community. In conclusion, although promising results have been obtained by the current (non-) machine learning methods for different clinical applications, thorough clinical validations are still required.

Convolutional-de-convolutional neural networks for recognition of surgical workflow

Computer-assisted surgery (CAS) has occupied an important position in modern surgery, further stimulating the progress of methodology and technology. In recent years, a large number of computer vision-based methods have been widely used in surgical workflow recognition tasks. For training the models, a lot of annotated data are necessary. However, the annotation of surgical data requires expert knowledge and thus becomes difficult and time-consuming. In this paper, we focus on the problem of data deficiency and propose a knowledge transfer learning method based on artificial neural network to compensate a small amount of labeled training data. To solve this problem, we propose an unsupervised method for pre-training a Convolutional-De-Convolutional (CDC) neural network for sequencing surgical workflow frames, which performs neural convolution in space (for semantic abstraction) and neural de-convolution in time (for frame level resolution) simultaneously. Specifically, through neural convolution transfer learning, we only fine-tuned the CDC neural network to classify the surgical phase. We performed some experiments for validating the model, and it showed that the proposed model can effectively extract the surgical feature and determine the surgical phase. The accuracy (Acc), recall, precision (Pres) of our model reached 91.4, 78.9, and 82.5%, respectively.

Calibration Method of Projectional Geometry for X-ray C-arm Fluoroscopy Using Sinogram Data

X-ray imaging represents the most commonly used imaging modality in X-ray-guided vascular intervention procedures. Computed tomography (CT) data ensure that the procedure is performed accurately and safely by providing medical staff with positional information of the body part before starting the procedure. In particular, accurate geometric information of the imaging equipment is essential to accurately calculate the three-dimensional (3D) position of catheters used in delicate operations. However, it is difficult to gather this information before surgery. Therefore, this study proposes a novel calibration method that can be used immediately before a procedure and can guarantee the stability of the procedure. The calibration was performed without additional radiography using sinogram data obtained during the 3D CT imaging process, and both the accuracy and calculation time available in the vascular intervention theater were allowable. The experimental results show that the best angular conditions in terms of calculations and accuracy are between −40 and 40 degrees in angular range and 1.6 degrees in angular interval. Consequently, we achieved a calculation time of 2.92 s and an average accuracy of 0.36 mm, thus meeting our goal of accuracy below 1 mm within a minute of computational time.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Accuracy of Stereovision-Updated Versus Preoperative CT-Based Image Guidance in Multilevel Lumbar Pedicle Screw Placement: A Cadaveric Swine Study

Change in vertebral position between preoperative imaging and the surgical procedure reduces the accuracy of image-guided spinal surgery, requiring repeated imaging and surgical field registration, a process that takes time and exposes patients to additional radiation. We developed a handheld, camera-based, deformable registration system (intraoperative stereovision, iSV) to register the surgical field automatically and compensate for spinal motion during surgery without further radiation exposure.

Methods

We measured motion-induced errors in image-guided lumbar pedicle screw placement in 6 whole-pig cadavers using state-of-the-art commercial spine navigation (StealthStation; Medtronic) and iSV registration that compensates for intraoperative vertebral motion. We induced spinal motion by using preoperative computed tomography (pCT) of the lumbar spine performed in the supine position with accentuated lordosis and performing surgery with the animal in the prone position. StealthStation registration of pCT occurred using metallic fiducial markers implanted in each vertebra, and iSV data were acquired to perform a deformable registration between pCT and the surgical field. Sixty-eight pedicle screws were placed in 6 whole-pig cadavers using iSV and StealthStation registrations in random order of vertebral level, relying only on image guidance without invoking the surgeon's judgment. The position of each pedicle screw was assessed with post-procedure CT and confirmed via anatomical dissection. Registration errors were assessed on the basis of implanted fiducials.

Results

The frequency and severity of pedicle screw perforation were lower for iSV registration compared with StealthStation (97% versus 68% with Grade 0 medial perforation for iSV and StealthStation, respectively). Severe perforation occurred only with StealthStation (18% versus 0% for iSV). The overall time required for iSV registration (computational efficiency) was ∼10 to 15 minutes and was comparable with StealthStation registration (∼10 min). The mean target registration error was smaller for iSV relative to StealthStation (2.81 ± 0.91 versus 8.37 ± 1.76 mm).

Conclusions

Pedicle screw placement was more accurate with iSV registration compared with state-of-the-art commercial navigation based on preoperative CT when alignment of the spine changed during surgery.

Clinical Relevance

The iSV system compensated for intervertebral motion, which obviated the need for repeated vertebral registration while providing efficient, accurate, radiation-free navigation during open spinal surgery.

Hand-Held Stereovision System for Image Updating in Open Spine Surgery

BACKGROUND

Image guidance in open spinal surgery is compromised by changes in spinal alignment between preoperative images and surgical positioning. We evaluated registration of stereo-views of the surgical field to compensate for vertebral alignment changes.

OBJECTIVE

To assess accuracy and efficiency of an optically tracked hand-held stereovision (HHS) system to acquire images of the exposed spine during surgery.

METHODS

Standard midline posterior approach exposed L1 to L6 in 6 cadaver porcine spines. Fiducial markers were placed on each vertebra as "ground truth" locations. Spines were positioned supine with accentuated lordosis, and preoperative computed tomography (pCT) was acquired. Spines were re-positioned in a neutral prone posture, and locations of fiducials were acquired with a tracked stylus. Intraoperative stereovision (iSV) images were acquired and 3-dimensional (3D) surfaces of the exposed spine were reconstructed. HHS accuracy was assessed in terms of distances between reconstructed fiducial marker locations and their tracked counterparts. Level-wise registrations aligned pCT with iSV to account for changes in spine posture. Accuracy of updated computed tomography (uCT) was assessed using fiducial markers and other landmarks.

RESULTS

Acquisition time for each image pair was <1 s. Mean reconstruction time was <1 s for each image pair using batch processing, and mean accuracy was 1.2 ± 0.6 mm across 6 cases. Mean errors of uCT were 3.1 ± 0.7 and 2.0 ± 0.5 mm on the dorsal and ventral sides, respectively.

CONCLUSION

Results suggest that a portable HHS system offers potential to acquire accurate image data from the surgical field to facilitate surgical navigation during open spine surgery.

Intraoperative visualization of nerves using a myelin protein-zero specific fluorescent tracer

BACKGROUND

Surgically induced nerve damage is a common but debilitating side effect in oncological surgery. With the aim to use fluorescence guidance to enable nerve-sparing interventions in future surgery, a fluorescent tracer was developed that specifically targets myelin protein zero (P0).

RESULTS

Truncated homotypic P0 protein-based peptide sequences were C-terminally functionalized with the far-red cyanine dye Cy5. The lead compound Cy5-P0101-125 was selected after initial solubility, (photo)physical and in vitro evaluation (including P0-blocking experiments). Cy5-P0101-125 (KD = 105 ± 17 nM) allowed in vitro and ex vivo P0-related staining. Furthermore, Cy5-P0101-125  enabled in vivo fluorescence imaging of the Sciatic nerve in mice after local intravenous (i.v.) administration and showed compatibility with a clinical fluorescence laparoscope during evaluation in a porcine model undergoing robot-assisted surgery. Biodistribution data revealed that i.v. administered [111In]In-DTPA-P0101-125 does not enter the central nervous system (CNS).

CONCLUSION

P0101-125 has proven to be a potent nerve-specific agent that is able to target P0/myelin under in vitro, ex vivo, and in vivo conditions without posing a threat for CNS-related toxicity.

3D augmentation of the surgical video stream: Toward a modular approach

BACKGROUND AND OBJECTIVE

We present an original approach to the development of augmented reality (AR) real-time solutions for robotic surgery navigation. The surgeon operating the robotic system through a console and a visor experiences reduced awareness of the operatory scene. In order to improve the surgeon's spatial perception during robot-assisted minimally invasive procedures, we provide him/her with a solid automatic software system to position, rotate and scale in real-time the 3D virtual model of a patient's organ aligned over its image captured by the endoscope.

METHODS

We observed that the surgeon may benefit differently from the 3D augmentation during each stage of the surgical procedure; moreover, each stage may present different visual elements that provide specific challenges and opportunities to exploit for organ detection strategies implementation. Hence we integrate different solutions, each dedicated to a specific stage of the surgical procedure, into a single software system.

RESULTS

We present a formal model that generalizes our approach, describing a system composed of integrated solutions for AR in robot-assisted surgery. Following the proposed framework, and application has been developed which is currently used during in vivo surgery, for extensive testing, by the Urology unity of the San Luigi Hospital, in Orbassano (To), Italy.

CONCLUSIONS

The main contribution of this paper is in presenting a modular approach to the tracking problem during in-vivo robotic surgery, whose efficacy from a medical point of view has been assessed in cited works. The segmentation of the whole procedure in a set of stages allows associating the best tracking strategy to each of them, as well as to re-utilize implemented software mechanisms in stages with similar features.

Fluorescence Imaging of Nerves During Surgery

OBJECTIVE

This review details the agents for fluorescence-guided nerve imaging in both preclinical and clinical use to identify factors important in selecting nerve-specific fluorescent agents for surgical procedures.

BACKGROUND

Iatrogenic nerve injury remains a significant cause of morbidity in patients undergoing surgical procedures. Current real-time identification of nerves during surgery involves neurophysiologic nerve stimulation, which has practical limitations. Intraoperative fluorescence-guided imaging provides a complimentary means of differentiating tissue types and pathology. Recent advances in fluorescence-guided nerve imaging have shown promise, but the ideal agent remains elusive.

METHODS

In February 2018, PubMed was searched for articles investigating peripheral nerve fluorescence. Key terms used in this search include: "intraoperative, nerve, fluorescence, peripheral nerve, visualization, near infrared, and myelin." Limits were set to exclude articles exclusively dealing with central nervous system targets or written in languages other than English. References were cross-checked for articles not otherwise identified.

RESULTS

Of the nonspecific agents, tracers that rely on axonal transport showed the greatest tissue specificity; however, neurovascular dyes already enjoy wide clinical use. Fluorophores specific to nerve moieties result in excellent nerve to background ratios. Although noteworthy findings on tissue specificity, toxicity, and route of administration specific to each fluorescent agent were reported, significant data objectively quantifying nerve-specific fluorescence and toxicity are lacking.

CONCLUSIONS

Fluorescence-based nerve enhancement has advanced rapidly over the past 10 years with potential for continued utilization and progression in translational research. An ideal agent would be easily administered perioperatively, would not cross the blood-brain barrier, and would fluoresce in the near-infrared spectrum. Agents administered systemically that target nerve-specific moieties have shown the greatest promise. Based on the heterogeneity of published studies and methods for reporting outcomes, it appears that the development of an optimal nerve imaging agent remains challenging.

Monocular 3D Probe Tracking for Generating Sub-Surface Optical Property Maps From Diffuse Optical Spectroscopic Imaging

OBJECTIVE

Diffuse optical spectroscopic imaging (DOSI) is a promising biophotonic technology for clinical tissue assessment, but is currently hampered by difficult wide area assessment. A co-integrative optical imaging system is proposed for dense sub-surface optical property spatial assessment.

METHODS

The proposed system fuses a co-aligned set of camera frames and diffuse optical spectroscopy measurements to generate spatial sub-surface optical property maps. A 3D rigid body motion estimation model was developed by fitting automatically detected target features to an a priori geometric model using a single overhead camera. Point-wise optical properties were measured across the tissue using frequency domain photon migration DOSI. The 3D probe trajectory and temporal optical property data were fused to generate 2D spatial optical property maps, which were projected onto the tissue image using pre-calibrated camera parameters.

RESULTS

The system demonstrated sub-millimeter positional accuracy (error 0.24 ± 0.35 mm) across different probe speeds (1.0-3.8 cm/s), and displacement accuracy in overhead ([Formula: see text] mm) and tilted (0.51 ± 0.51 mm) camera orientations. Unstructured scans on a tumor inclusion phantom showed strong contrast under different probe paths, and significant ( ) changes in optical properties in an in vivo leg cuff occlusion protocol with spatial anatomy localization.

CONCLUSION

The proposed co-integrative optical imaging system generated dense sub-surface optical property distributions across wide tissue areas with sub-millimeter accuracy at different probe speeds and trajectories, and does not require pre-planned probe route for tissue assessment.

SIGNIFICANCE

This system provides a valuable tool for real-time non-invasive tissue health and cancer screening, and enables longitudinal disease progression assessment through unstructured probe-based optical tissue assessment.

A framework for evaluating correspondence between brain images using anatomical fiducials

Accurate spatial correspondence between template and subject images is a crucial step in neuroimaging studies and clinical applications like stereotactic neurosurgery. In the absence of a robust quantitative approach, we sought to propose and validate a set of point landmarks, anatomical fiducials (AFIDs), that could be quickly, accurately, and reliably placed on magnetic resonance images of the human brain. Using several publicly available brain templates and individual participant datasets, novice users could be trained to place a set of 32 AFIDs with millimetric accuracy. Furthermore, the utility of the AFIDs protocol is demonstrated for evaluating subject-to-template and template-to-template registration. Specifically, we found that commonly used voxel overlap metrics were relatively insensitive to focal misregistrations compared to AFID point-based measures. Our entire protocol and study framework leverages open resources and tools, and has been developed with full transparency in mind so that others may freely use, adopt, and modify. This protocol holds value for a broad number of applications including alignment of brain images and teaching neuroanatomy.

An Automatic Spatial Registration Method for Image-Guided Neurosurgery System

OBJECTIVE

This study aimed to investigate the feasibility of an automatic marker-free patient-to-image spatial registration method based on the 4-points congruent sets (4PCS) and iterative closest point (ICP) algorithm for the image-guided neurosurgery system (IGNS).

METHODS

A portable scanner was used to obtain the point cloud of the patient's entire head. The 4PCS algorithm, which is resilient to noise and outliers, automatically registered the point cloud in the patient space to the surface reconstructed from the patient's preoperative images in the image space without any assumptions about initial alignment. A variant of the ICP algorithm was then used to finish the fine registration. Two phantoms and 3 patients' experiments were performed to demonstrate the effectiveness of the proposed method.

RESULTS

In the phantom experiments, the mean target registration error of 15 targets on the surface of the rigid and the elastic phantoms were 1.02 ± 0.18 mm and 1.27 ± 0.36 mm, respectively. In the clinical experiments, the mean target registration error of 7 targets on the first, second and third patient's head were 1.88 ± 0.19 mm, 1.84 ± 0.19 mm, and 1.89 ± 0.18 mm, respectively, which was sufficient to meet clinical requirements. The registration accuracy and registration time using the proposed method are better than that using the method based on manually coarse registration and automatic fine registration.

CONCLUSIONS

It is feasible to use the automatic spatial registration method based on the 4PCS and ICP algorithm for the IGNS. Moreover, it can replace the spatial registration method based on manually selected anatomical landmarks combined with the automatic fine registration in the currently used IGNS.

Regional-surface-based registration for image-guided neurosurgery: effects of scan modes on registration accuracy

PURPOSE

The conventional surface-based method only registers the facial zone with preoperative point cloud, resulting in low accuracy away from the facial area. Acquiring a point cloud of the entire head for registration can improve registration accuracy in all parts of the head. However, it takes a long time to collect a point cloud of the entire head. It may be more practical to selectively scan part of the head to ensure high registration accuracy in the surgical area of interest. In this study, we investigate the effects of different scan regions on registration errors in different target areas when using a surface-based registration method.

METHODS

We first evaluated the correlation between the laser scan resolution and registration accuracy to determine an appropriate scan resolution. Then, with the appropriate resolution, we explored the effects of scan modes on registration error in computer simulation experiments, phantom experiments and two clinical cases. The scan modes were designed based on different combinations of five zones of the head surface, i.e., the sphenoid-frontal zone, parietal zone, left temporal zone, right temporal zone and occipital zone. In the phantom experiment, a handheld scanner was used to acquire a point cloud of the head. A head model containing several tumors was designed, enabling us to calculate the target registration errors deep in the brain to evaluate the effect of regional-surface-based registration.

RESULT

The optimal scan modes for tumors located in the sphenoid-frontal, parietal and temporal areas are mode 4 (i.e., simultaneously scanning the sphenoid-frontal zone and the temporal zone), mode 4 and mode 6 (i.e., simultaneously scanning the sphenoid-frontal zone, the temporal zone and the parietal zone), respectively. For the tumor located in the occipital area, no modes were able to achieve reliable accuracy.

CONCLUSION

The results show that selecting an appropriate scan resolution and scan mode can achieve reliable accuracy for use in sphenoid-frontal, parietal and temporal area surgeries while effectively reducing the operation time.

An Accurate Recognition of Infrared Retro-Reflective Markers in Surgical Navigation

Marker-based optical tracking systems (OTS) are widely used in clinical image-guided therapy. However, the emergence of ghost markers, which is caused by the mistaken recognition of markers and the incorrect correspondences between marker projections, may lead to tracking failures for these systems. Therefore, this paper proposes a strategy to prevent the emergence of ghost markers by identifying markers based on the features of their projections, finding the correspondences between marker projections based on the geometric information provided by markers, and fast-tracking markers in a 2D image between frames based on the sizes of their projections. Apart from validating its high robustness, the experimental results show that the proposed strategy can accurately recognize markers, correctly identify their correspondences, and meet the requirements of real-time tracking.

An experimental and numerical study on tactile neuroimaging: A novel minimally invasive technique for intraoperative brain imaging

BACKGROUND

The success of tumour neurosurgery is highly dependent on the ability to accurately localize the operative target, which may shift during the operation. Performing intraoperative brain imaging is crucial in minimally invasive neurosurgery to detect the effect of brain shift on the tumour's location, and to maximize the efficiency of tumour resection.

METHOD

The major objective of this research is to introduce tactile neuroimaging as a novel minimally invasive technique for intraoperative brain imaging. To investigate the feasibility of the proposed method, an experimental and numerical study was first performed on silicone phantoms mimicking the brain tissue with a tumour. Then the study was extended to a clinical model with the meningioma tumour.

RESULTS

The stress distribution on the brain surface has high potential to intraoperatively localize the tumour.

CONCLUSION

Results suggest that tactile neuroimaging can be used to provide non-invasive and real-time intraoperative data on a tumour's features.

Planning, guidance, and quality assurance of pelvic screw placement using deformable image registration

Percutaneous pelvic screw placement is challenging due to narrow bone corridors surrounded by vulnerable structures and difficult visual interpretation of complex anatomical shapes in 2D x-ray projection images. To address these challenges, a system for planning, guidance, and quality assurance (QA) is presented, providing functionality analogous to surgical navigation, but based on robust 3D-2D image registration techniques using fluoroscopy images already acquired in routine workflow. Two novel aspects of the system are investigated: automatic planning of pelvic screw trajectories and the ability to account for deformation of surgical devices (K-wire deflection). Atlas-based registration is used to calculate a patient-specific plan of screw trajectories in preoperative CT. 3D-2D registration aligns the patient to CT within the projective geometry of intraoperative fluoroscopy. Deformable known-component registration (dKC-Reg) localizes the surgical device, and the combination of plan and device location is used to provide guidance and QA. A leave-one-out analysis evaluated the accuracy of automatic planning, and a cadaver experiment compared the accuracy of dKC-Reg to rigid approaches (e.g. optical tracking). Surgical plans conformed within the bone cortex by 3-4 mm for the narrowest corridor (superior pubic ramus) and  >5 mm for the widest corridor (tear drop). The dKC-Reg algorithm localized the K-wire tip within 1.1 mm and 1.4° and was consistently more accurate than rigid-body tracking (errors up to 9 mm). The system was shown to automatically compute reliable screw trajectories and accurately localize deformed surgical devices (K-wires). Such capability could improve guidance and QA in orthopaedic surgery, where workflow is impeded by manual planning, conventional tool trackers add complexity and cost, rigid tool assumptions are often inaccurate, and qualitative interpretation of complex anatomy from 2D projections is prone to trial-and-error with extended fluoroscopy time.

In vivo comparison of two navigation systems for abdominal percutaneous needle intervention

PURPOSE

To compare the accuracy of a Kinect-Optical navigation system with an electromagnetic (EM) navigation system for percutaneous liver needle intervention.

MATERIALS AND METHODS

Five beagles with nine artificial tumors were used for validation. The Veran IG4 EM navigation system and a custom-made Kinect-Optical navigation system were used. Needle insertions into each tumor were conducted with these two guidance methods. The target positioning error (TPE) and the time cost of the puncture procedures were evaluated.

RESULTS

A total of 18 needle insertions were performed to evaluate the navigation accuracy of the two guidance approaches. The targeting error was 6.78 ± 3.22 mm and 8.72 ± 3.5 mm for the Kinect-Optical navigation system and the EM navigation system, respectively. There is no statistically significant difference in the TPE between the Kinect-Optical navigation system and the EM navigation system (p = 0.229). The processing time with the Kinect-Optical system (10 min) is similar to that of the Veran IG4 system (12 min).

CONCLUSIONS

The accuracy of the Kinect-Optical navigation system is comparable to that of the EM navigation system.

A Surface-Based Spatial Registration Method Based on Sense Three-Dimensional Scanner

OBJECTIVE

The purpose of this study was to investigate the feasibility of a surface-based registration method based on a low-cost, hand-held Sense three-dimensional (3D) scanner in image-guided neurosurgery system.

METHODS

The scanner was calibrated prior and fixed on a tripod before registration. During registration, a part of the head surface was scanned at first and the spatial position of the adapter was recorded. Then the scanner was taken off from the tripod and the entire head surface was scanned by moving the scanner around the patient's head. All the scan points were aligned to the recorded spatial position to form a unique point cloud of the head by the automatic mosaic function of the scanner. The coordinates of the scan points were transformed from the device space to the adapter space by a calibration matrix, and then to the patient space. A 2-step patient-to-image registration method was then performed to register the patient space to the image space.

RESULTS

The experimental results showed that the mean target registration error of 15 targets on the surface of the phantom was 1.61±0.09 mm. In a clinical experiment, the mean target registration error of 7 targets on the patient's head surface was 2.50±0.31 mm, which was sufficient to meet clinical requirements.

CONCLUSIONS

It is feasible to use the Sense 3D scanner for patient-to-image registration, and the low-cost Sense 3D scanner can take the place of the current used scanner in the image-guided neurosurgery system.

Automated Catheter Navigation With Electromagnetic Image Guidance

This paper describes a novel method of controlling an endoscopic catheter by using an automated catheter tensioning system with the objective of providing clinicians with improved manipulation capabilities within the patient. Catheters are used in many clinical procedures to provide access to the cardiopulmonary system. Control of such catheters is performed manually by the clinicians using a handle, typically actuating a single or opposing set of pull wires. Such catheters are generally actuated in a single plane, requiring the clinician to rotate the catheter handle to navigate the system. The automation system described here allows closed-loop control of a custom bronchial catheter in tandem with an electromagnetic tracking of the catheter tip and image guidance by using a 3D Slicer. An electromechanical drive train applies tension to four pull wires to steer the catheter tip, with the applied force constantly monitored through force sensing load cells. The applied tension is controlled through a PC connected joystick. An electromagnetic sensor embedded in the catheter tip enables constant real-time position tracking, whereas a working channel provides a route for endoscopic instruments. The system is demonstrated and tested in both a breathing lung model and a preclinical animal study. Navigation to predefined targets in the subject's airways by using the joystick while using virtual image guidance and electromagnetic tracking was demonstrated. Average targeting times were 29 and 10 s, respectively, for the breathing lung and live animal studies. This paper presents the first reported remote controlled bronchial working channel catheter utilizing electromagnetic tracking and has many implications for future development in endoscopic and catheter-based procedures.

4D interventional device reconstruction from biplane fluoroscopy

PURPOSE

Biplane angiography systems provide time resolved 2D fluoroscopic images from two different angles, which can be used for the positioning of interventional devices such as guidewires and catheters. The purpose of this work is to provide a novel algorithm framework, which allows the 3D reconstruction of these curvilinear devices from the 2D projection images for each time frame. This would allow creating virtual projection images from arbitrary view angles without changing the position of the gantries, as well as virtual endoscopic 3D renderings.

METHODS

The first frame of each time sequence is registered to and subtracted from the following frame using an elastic grid registration technique. The images are then preprocessed by a noise reduction algorithm using directional adaptive filter kernels and a ridgeness filter that emphasizes curvilinear structures. A threshold based segmentation of the device is then performed, followed by a flux driven topology preserving thinning algorithm to extract the segments of the device centerline. The exact device path is determined using Dijkstra's algorithm to minimize the curvature and distance between adjacent segments as well as the difference to the device path of the previous frame. The 3D device centerline is then reconstructed using epipolar geometry.

RESULTS

The accuracy of the reconstruction was measured in a vascular head phantom as well as in a cadaver head and a canine study. The device reconstructions are compared to rotational 3D acquisitions. In the phantom experiments, an average device tip accuracy of 0.35 ± 0.09 mm, a Hausdorff distance of 0.65 ± 0.32 mm, and a mean device distance of 0.54 ± 0.33 mm were achieved. In the cadaver head and canine experiments, the device tip was reconstructed with an average accuracy of 0.26 ± 0.20 mm, a Hausdorff distance of 0.62 ± 0.08 mm, and a mean device distance of 0.41 ± 0.08 mm. Additionally, retrospective reconstruction results of real patient data are presented.

CONCLUSIONS

The presented algorithm is a novel approach for the time resolved 3D reconstruction of interventional devices from biplane fluoroscopic images, thus allowing the creation of virtual projection images from arbitrary view angles as well as virtual endoscopic 3D renderings. Availability of this technique would enhance the ability to accurately position devices in minimally invasive endovascular procedures.

Accuracy evaluation of a 3D-printed individual template for needle guidance in head and neck brachytherapy

To transfer the preplan for the head and neck brachytherapy to the clinical implantation procedure, a preplan-based 3D-printed individual template for needle insertion guidance had previously been designed and used. The accuracy of needle insertion using this kind template was assessed in vivo In the study, 25 patients with head and neck tumors were implanted with ¹²⁵I radioactive seeds under the guidance of the 3D-printed individual template. Patients were divided into four groups based on the site of needle insertion: the parotid and masseter region group (nine patients); the maxillary and paranasal region group (eight patients); the submandibular and upper neck area group (five patients); and the retromandibular region group (six patients). The distance and angular deviations between the preplanned and placed needles were compared, and the complications and time required for needle insertion were assessed. The mean entrance point distance deviation for all 619 needles was 1.18 ± 0.81 mm, varying from 0.857 ± 0.545 to 1.930 ± 0.843 mm at different sites. The mean angular deviation was 2.08 ± 1.07 degrees, varying from 1.85 ± 0.93 to 2.73 ± 1.18 degrees at different sites. All needles were manually inserted to their preplanned positions in a single attempt, and the mean time to insert one needle was 7.5 s. No anatomical complications related to inaccurately placed implants were observed. Using the 3D-printed individual template for the implantation of ¹²⁵I radioactive seeds in the head and neck region can accurately transfer a CT-based preplan to the brachytherapy needle insertion procedure. Moreover, the addition of individual template guidance can reduce the time required for implantation and minimize the damage to normal tissues.

Virtual implantation of a novel LVAD: toward computer-assisted surgery for heart failure

BACKGROUND

Mechanical and hemodynamic factors are among the determinants of patient-device interaction and early-term and long-term outcomes in left ventricular assist device (LVAD) recipients.

MATERIAL AND METHODS

We are currently developing computer simulation tools aimed at (1) analyze the intrathoracic and intracavitary positioning of LVADs after implantation and establish correlation with clinical outcomes; (2) assist surgeons in the choice of device and of left ventricular coring site for optimized intrathoracic placement and function; and (3) facilitate the planning of less-invasive LVAD implantation. A virtual representation of LVAD (mesh device component) was created through cone-beam computed tomography and semiautomatic segmentation. A modular framework software (CamiTK, Grenoble, France) was used to create a three-dimensional representation of patients' computed tomography (CT) scan and incorporate the mesh device component for virtual implantation.

RESULTS

Device reconstruction was included into a dedicated software with the purposes of virtual implantation, based on the preoperative CT scan of surgical candidates.

CONCLUSIONS

We present herein the first digital reconstruction of the novel HeartMate 3 LVAD. Virtual implantation on the basis of preoperative CT scan is feasible within a user-friendly interactive software. Future studies will be focused on correlation with clinical variables.

Algorithm for simulation of craniotomies assisted by peripheral for 3D virtual navigation

Neurosurgical procedures require high precision and an accurate localization of the structures. For that reason and due to the advances in 3D visualization, the software for planning and training neurosurgeries has become an important tool for neurosurgeons and students, but the manipulation of the 3D structures is not always easy for the staff that usually works with 2D images. This paper describes a system developed in open source software that allows performing a virtual craniotomy (a common procedure in neurosurgery that enables the access to intracranial lesions) in 3D slicer; the system includes a peripheral input in order to permit the manipulation of the 3D structures according to camera movements and to guide the movement of the craniotomy tool.

Recent advances in surgical planning & navigation for tumor biopsy and resection

This paper highlights recent advancements in imaging technologies for surgical planning and navigation in tumor biopsy and resection which need high-precision in detection and characterization of lesion margin in preoperative planning and intraoperative navigation. Multimodality image-guided surgery platforms brought great benefits in surgical planning and operation accuracy via registration of various data sets with information on morphology [X-ray, magnetic resonance (MR), computed tomography (CT)], function connectivity [functional magnetic resonance imaging (fMRI), diffusion tensor imaging (DTI), rest-status fMRI], or molecular activity [positron emission tomography (PET)]. These image-guided platforms provide a correspondence between the pre-operative surgical planning and intra-operative procedure. We envisage that the combination of advanced multimodal imaging, three-dimensional (3D) printing, and cloud computing will play increasingly important roles in planning and navigation of surgery for tumor biopsy and resection in the coming years.

Toward real-time remote processing of laparoscopic video

Laparoscopic surgery is a minimally invasive surgical technique where surgeons insert a small video camera into the patient's body to visualize internal organs and use small tools to perform surgical procedures. However, the benefit of small incisions has a drawback of limited visualization of subsurface tissues, which can lead to navigational challenges in the delivering of therapy. Image-guided surgery uses the images to map subsurface structures and can reduce the limitations of laparoscopic surgery. One particular laparoscopic camera system of interest is the vision system of the daVinci-Si robotic surgical system (Intuitive Surgical, Sunnyvale, California). The video streams generate approximately 360 MB of data per second, demonstrating a trend toward increased data sizes in medicine, primarily due to higher-resolution video cameras and imaging equipment. Processing this data on a bedside PC has become challenging and a high-performance computing (HPC) environment may not always be available at the point of care. To process this data on remote HPC clusters at the typical 30 frames per second (fps) rate, it is required that each 11.9 MB video frame be processed by a server and returned within 1/30th of a second. The ability to acquire, process, and visualize data in real time is essential for the performance of complex tasks as well as minimizing risk to the patient. As a result, utilizing high-speed networks to access computing clusters will lead to real-time medical image processing and improve surgical experiences by providing real-time augmented laparoscopic data. We have performed image processing algorithms on a high-definition head phantom video (1920 × 1080 pixels) and transferred the video using a message passing interface. The total transfer time is around 53 ms or 19 fps. We will optimize and parallelize these algorithms to reduce the total time to 30 ms.

A new markerless patient-to-image registration method using a portable 3D scanner

PURPOSE

Patient-to-image registration is critical to providing surgeons with reliable guidance information in the application of image-guided neurosurgery systems. The conventional point-matching registration method, which is based on skin markers, requires expensive and time-consuming logistic support. Surface-matching registration with facial surface scans is an alternative method, but the registration accuracy is unstable and the error in the more posterior parts of the head is usually large because the scan range is limited. This study proposes a new surface-matching method using a portable 3D scanner to acquire a point cloud of the entire head to perform the patient-to-image registration.

METHODS

A new method for transforming the scan points from the device space into the patient space without calibration and tracking was developed. Five positioning targets were attached on a reference star, and their coordinates in the patient space were measured prior. During registration, the authors moved the scanner around the head to scan its entire surface as well as the positioning targets, and the scanner generated a unique point cloud in the device space. The coordinates of the positioning targets in the device space were automatically detected by the scanner, and a spatial transformation from the device space to the patient space could be calculated by registering them to their coordinates in the patient space that had been measured prior. A three-step registration algorithm was then used to register the patient space to the image space. The authors evaluated their method on a rigid head phantom and an elastic head phantom to verify its practicality and to calculate the target registration error (TRE) in different regions of the head phantoms. The authors also conducted an experiment with a real patient's data to test the feasibility of their method in the clinical environment.

RESULTS

In the phantom experiments, the mean fiducial registration error between the device space and the patient space, the mean surface registration error, and the mean TRE of 15 targets on the surface of each phantom were 0.34 ± 0.01 mm and 0.33 ± 0.02 mm, 1.17 ± 0.02 mm and 1.34 ± 0.10 mm, and 1.06 ± 0.11 mm and 1.48 ± 0.21 mm, respectively. When grouping the targets according to their positions on the head, high accuracy was achieved in all parts of the head, and the TREs were similar across different regions. The authors compared their method with the current surface registration methods that use only a part of the facial surface on the elastic phantom, and the mean TRE of 15 targets was 1.48 ± 0.21 mm and 1.98 ± 0.53 mm, respectively. In a clinical experiment, the mean TRE of seven targets on the patient's head surface was 1.92 ± 0.18 mm, which was sufficient to meet clinical requirements.

CONCLUSIONS

The proposed surface-matching registration method provides sufficient registration accuracy even in the posterior area of the head. The 3D point cloud of the entire head, including the facial surface and the back of the head, can be easily acquired using a portable 3D scanner. The scanner does not need to be calibrated prior or tracked by the optical tracking system during scanning.

Augmented reality in the surgery of cerebral aneurysms: a technical report

BACKGROUND

Augmented reality is the overlay of computer-generated images on real-world structures. It has previously been used for image guidance during surgical procedures, but it has never been used in the surgery of cerebral aneurysms.

OBJECTIVE

To report our experience of cerebral aneurysm surgery aided by augmented reality.

METHODS

Twenty-eight patients with 39 unruptured aneurysms were operated on in a prospective manner with augmented reality. Preoperative 3-dimensional image data sets (angio-magnetic resonance imaging, angio-computed tomography, and 3-dimensional digital subtraction angiography) were used to create virtual segmentations of patients' vessels, aneurysms, aneurysm necks, skulls, and heads. These images were injected intraoperatively into the eyepiece of the operating microscope. An example case of an unruptured posterior communicating artery aneurysm clipping is illustrated in a video.

RESULTS

The described operating procedure allowed continuous monitoring of the accuracy of patient registration with neuronavigation data and assisted in the performance of tailored surgical approaches and optimal clipping with minimized exposition.

CONCLUSION

Augmented reality may add to the performance of a minimally invasive approach, although further studies need to be performed to evaluate whether certain groups of aneurysms are more likely to benefit from it. Further technological development is required to improve its user friendliness.

Training for planning tumour resection: augmented reality and human factors

Planning surgical interventions is a complex task, demanding a high degree of perceptual, cognitive, and sensorimotor skills to reduce intra- and post-operative complications. This process requires spatial reasoning to coordinate between the preoperatively acquired medical images and patient reference frames. In the case of neurosurgical interventions, traditional approaches to planning tend to focus on providing a means for visualizing medical images, but rarely support transformation between different spatial reference frames. Thus, surgeons often rely on their previous experience and intuition as their sole guide is to perform mental transformation. In case of junior residents, this may lead to longer operation times or increased chance of error under additional cognitive demands. In this paper, we introduce a mixed augmented-/virtual-reality system to facilitate training for planning a common neurosurgical procedure, brain tumour resection. The proposed system is designed and evaluated with human factors explicitly in mind, alleviating the difficulty of mental transformation. Our results indicate that, compared to conventional planning environments, the proposed system greatly improves the nonclinicians' performance, independent of the sensorimotor tasks performed ( ). Furthermore, the use of the proposed system by clinicians resulted in a significant reduction in time to perform clinically relevant tasks ( ). These results demonstrate the role of mixed-reality systems in assisting residents to develop necessary spatial reasoning skills needed for planning brain tumour resection, improving patient outcomes.

Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations

BACKGROUND

Augmented reality technology has been used for intraoperative image guidance through the overlay of virtual images, from preoperative imaging studies, onto the real-world surgical field. Although setups based on augmented reality have been used for various neurosurgical pathologies, very few cases have been reported for the surgery of arteriovenous malformations (AVM). We present our experience with AVM surgery using a system designed for image injection of virtual images into the operating microscope's eyepiece, and discuss why augmented reality may be less appealing in this form of surgery.

METHODS

N = 5 patients underwent AVM resection assisted by augmented reality. Virtual three-dimensional models of patients' heads, skulls, AVM nidi, and feeder and drainage vessels were selectively segmented and injected into the microscope's eyepiece for intraoperative image guidance, and their usefulness was assessed in each case.

RESULTS

Although the setup helped in performing tailored craniotomies, in guiding dissection and in localizing drainage veins, it did not provide the surgeon with useful information concerning feeder arteries, due to the complexity of AVM angioarchitecture.

CONCLUSION

The difficulty in intraoperatively conveying useful information on feeder vessels may make augmented reality a less engaging tool in this form of surgery, and might explain its underrepresentation in the literature. Integrating an AVM's hemodynamic characteristics into the augmented rendering could make it more suited to AVM surgery.

Image-guided intraoperative radiation therapy: current developments and future perspectives

Intraoperative electron beam radiation therapy (IOERT) procedures involve the delivery of radiation to a target area during surgery by means of a specific applicator. This treatment is currently planned by means of specific systems that incorporate tools for both surgical simulation and radiation dose distribution estimation. Although the planning step improves treatment quality and facilitates follow-up, the actual position of the patient, the applicator and other tools during the surgical procedure is unknown. Image-guided navigation technologies could be introduced in IOERT treatments, but an innovative solution that overcomes the limitations of these systems in complex surgical scenarios is needed. A recent publication describes a multi-camera optical tracking system integrated in IOERT workflow. This technology has shown appropriate accuracy in phantom experiments, and could also be of interest in other surgical interventions, where the restrictions solved by this system are also present.

Prototype nerve-specific near-infrared fluorophores

Nerve preservation is an important issue during most surgery because accidental transection or injury results in significant morbidity, including numbness, pain, weakness, or paralysis. Currently, nerves are still identified only by gross appearance and anatomical location during surgery, without intraoperative image guidance. Near-infrared (NIR) fluorescent light, in the wavelength range of 650-900 nm, has the potential to provide high-resolution, high-sensitivity, and real-time avoidance of nerve damage, but only if nerve-specific NIR fluorophores can be developed. In this study, we evaluated a series of Oxazine derivatives to highlight various peripheral nerve structures in small and large animals. Among the targeted fluorophores, Oxazine 4 has peak emission near into the NIR, which provided nerve-targeted signal in the brachial plexus and sciatic nerve for up to 12 h after a single intravenous injection. In addition, recurrent laryngeal nerves were successfully identified and highlighted in real time in swine, which could be preserved during the course of thyroid resection. Although optical properties of these agents are not yet optimal, chemical structure analysis provides a basis for improving these prototype nerve-specific NIR fluorophores even further.

Surgical planning of Isshiki type I thyroplasty using an open-source Digital Imaging and Communication in Medicine viewer OsiriX

CONCLUSION

Preoperative surgical planning of Isshiki type I thyroplasty with the Digital Imaging and Communication in Medicine (DICOM) viewer OsiriX can be used for strategic and predictable type I thyroplasty.

OBJECTIVES

The aim of this study was to determine the efficacy of the preoperative planning of type I thyroplasty using the DICOM viewer OsiriX.

METHODS

Five patients with unilateral vocal cord paralysis and severe breathy dysphonia were included in this study. Multidetector computed tomography (CT) DICOM data were obtained and breath holding was performed during image acquisition. Using three-dimensional multiplanar reconstruction, we outlined the optimal location for a window. Type I thyroplasty was performed using Isshiki's original method, and only the placement of the window was decided according to the preoperative simulation point. To verify the advantages of this method, we compared our data with the previous data for total operation time and voice quality at 3 months after the operation without the DICOM viewer planning.

RESULTS

All patients are free from dysphonia and their glottic closures have resolved satisfactorily. Postoperative CT revealed that appropriate implant positioning resulted in successful surgical intervention. The comparison of total operation time showed that with the new method, operation duration was shortened by 12 min.