Computer-assisted 3D ultrasound-guided neurosurgery: technological contributions, including multimodal registration and advanced display, demonstrating future perspectives

Challenges and Opportunities of Intraoperative 3D Ultrasound With Neuronavigation in Relation to Intraoperative MRI

Introduction

Neuronavigation greatly improves the surgeons ability to approach, assess and operate on brain tumors, but tends to lose its accuracy as the surgery progresses and substantial brain shift and deformation occurs. Intraoperative MRI (iMRI) can partially address this problem but is resource intensive and workflow disruptive. Intraoperative ultrasound (iUS) provides real-time information that can be used to update neuronavigation and provide real-time information regarding the resection progress. We describe the intraoperative use of 3D iUS in relation to iMRI, and discuss the challenges and opportunities in its use in neurosurgical practice.

Methods

We performed a retrospective evaluation of patients who underwent image-guided brain tumor resection in which both 3D iUS and iMRI were used. The study was conducted between June 2020 and December 2020 when an extension of a commercially available navigation software was introduced in our practice enabling 3D iUS volumes to be reconstructed from tracked 2D iUS images. For each patient, three or more 3D iUS images were acquired during the procedure, and one iMRI was acquired towards the end. The iUS images included an extradural ultrasound sweep acquired before dural incision (iUS-1), a post-dural opening iUS (iUS-2), and a third iUS acquired immediately before the iMRI acquisition (iUS-3). iUS-1 and preoperative MRI were compared to evaluate the ability of iUS to visualize tumor boundaries and critical anatomic landmarks; iUS-3 and iMRI were compared to evaluate the ability of iUS for predicting residual tumor.

Results

Twenty-three patients were included in this study. Fifteen patients had tumors located in eloquent or near eloquent brain regions, the majority of patients had low grade gliomas (11), gross total resection was achieved in 12 patients, postoperative temporary deficits were observed in five patients. In twenty-two iUS was able to define tumor location, tumor margins, and was able to indicate relevant landmarks for orientation and guidance. In sixteen cases, white matter fiber tracts computed from preoperative dMRI were overlaid on the iUS images. In nineteen patients, the EOR (GTR or STR) was predicted by iUS and confirmed by iMRI. The remaining four patients where iUS was not able to evaluate the presence or absence of residual tumor were recurrent cases with a previous surgical cavity that hindered good contact between the US probe and the brainsurface.

Conclusion

This recent experience at our institution illustrates the practical benefits, challenges, and opportunities of 3D iUS in relation to iMRI.

Twenty Years of Cerebral Ultrasound Perfusion Imaging-Is the Best yet to Come?

Over the past 20 years, ultrasonic cerebral perfusion imaging (UPI) has been introduced and validated applying different data acquisition and processing approaches. Clinical data were collected mainly in acute stroke patients. Some efforts were undertaken in order to compare different technical settings and validate results to gold standard perfusion imaging. This review illustrates the evolution of the method, explicating different technical aspects and milestones achieved over time. Up to date, advancements of ultrasound technology as well as data processing approaches enable semi-quantitative, gold standard proven identification of critically hypo-perfused tissue in acute stroke patients. The rapid distribution of CT perfusion over the past 10 years has limited the clinical need for UPI. However, the unexcelled advantage of mobile application raises reasonable expectations for future applications. Since the identification of intracerebral hematoma and large vessel occlusion can also be revealed by ultrasound exams, UPI is a supplementary multi-modal imaging technique with the potential of pre-hospital application. Some further applications are outlined to highlight the future potential of this underrated bedside method of microcirculatory perfusion assessment.

Position and Orientation Errors in a Neuronavigation Procedure: A Stepwise Protocol Using a Cranial Phantom

OBJECTIVE

Neuronavigation procedures demand high precision and accuracy. Despite this need, there are still few studies analyzing errors in such procedures. The aim of this study was to use a custom-built cranial phantom to measure target position and orientation errors in different phases of a simulated neuronavigation procedure.

METHODS

A cranial phantom with 10 target sites was designed and imaged with computed tomography and magnetic resonance. A segmentation of a cloud of points of the phantom (ground truth) was obtained using an optical tracking system and compared with the images (imaging phase). Targets and trajectories were then planned with neuronavigation software and compared with the ground truth (planning phase). The same plan was used to identify the points in real space after image-to-phantom registration and calculate the final error of the procedure by comparison with the ground truth (registration and execution phase).

RESULTS

The mean errors after the imaging phase were 1.11 ± 0.42 mm and 3.23° ± 1.69° for position and orientation, respectively. After planning the mean errors were 1.10 ± 0.39 mm and 5.55° ± 2.91°. The global errors after the registration and mechanical execution were 3.93 ± 1.70 mm and 3.65° ± 1.29°.

CONCLUSIONS

After a stepwise analysis, registration and mechanical execution were the main contributors to the global position error.

Determination of optimal ultrasound planes for the initialisation of image registration during endoscopic ultrasound-guided procedures

PURPOSE

Navigation of endoscopic ultrasound (EUS)-guided procedures of the upper gastrointestinal (GI) system can be technically challenging due to the small fields-of-view of ultrasound and optical devices, as well as the anatomical variability and limited number of orienting landmarks during navigation. Co-registration of an EUS device and a pre-procedure 3D image can enhance the ability to navigate. However, the fidelity of this contextual information depends on the accuracy of registration. The purpose of this study was to develop and test the feasibility of a simulation-based planning method for pre-selecting patient-specific EUS-visible anatomical landmark locations to maximise the accuracy and robustness of a feature-based multimodality registration method.

METHODS

A registration approach was adopted in which landmarks are registered to anatomical structures segmented from the pre-procedure volume. The predicted target registration errors (TREs) of EUS-CT registration were estimated using simulated visible anatomical landmarks and a Monte Carlo simulation of landmark localisation error. The optimal planes were selected based on the 90th percentile of TREs, which provide a robust and more accurate EUS-CT registration initialisation. The method was evaluated by comparing the accuracy and robustness of registrations initialised using optimised planes versus non-optimised planes using manually segmented CT images and simulated ([Formula: see text]) or retrospective clinical ([Formula: see text]) EUS landmarks.

RESULTS

The results show a lower 90th percentile TRE when registration is initialised using the optimised planes compared with a non-optimised initialisation approach (p value [Formula: see text]).

CONCLUSIONS

The proposed simulation-based method to find optimised EUS planes and landmarks for EUS-guided procedures may have the potential to improve registration accuracy. Further work will investigate applying the technique in a clinical setting.

Simultaneous Multi-Structure Segmentation and 3D Nonrigid Pose Estimation in Image-Guided Robotic Surgery

In image-guided robotic surgery, segmenting the endoscopic video stream into meaningful parts provides important contextual information that surgeons can exploit to enhance their perception of the surgical scene. This information provides surgeons with real-time decision-making guidance before initiating critical tasks such as tissue cutting. Segmenting endoscopic video is a challenging problem due to a variety of complications including significant noise attributed to bleeding and smoke from cutting, poor appearance contrast between different tissue types, occluding surgical tools, and limited visibility of the objects' geometries on the projected camera views. In this paper, we propose a multi-modal approach to segmentation where preoperative 3D computed tomography scans and intraoperative stereo-endoscopic video data are jointly analyzed. The idea is to segment multiple poorly visible structures in the stereo/multichannel endoscopic videos by fusing reliable prior knowledge captured from the preoperative 3D scans. More specifically, we estimate and track the pose of the preoperative models in 3D and consider the models' non-rigid deformations to match with corresponding visual cues in multi-channel endoscopic video and segment the objects of interest. Further, contrary to most augmented reality frameworks in endoscopic surgery that assume known camera parameters, an assumption that is often violated during surgery due to non-optimal camera calibration and changes in camera focus/zoom, our method embeds these parameters into the optimization hence correcting the calibration parameters within the segmentation process. We evaluate our technique on synthetic data, ex vivo lamb kidney datasets, and in vivo clinical partial nephrectomy surgery with results demonstrating high accuracy and robustness.

Intraoperative ultrasound in spinal tumor surgery

PURPOSE

Intraoperative ultrasound (ioUS) has become increasingly widespread in brain tumor surgery but it is not yet a standard procedure in spinal surgery. We analyzed intraoperative ultrasonographic findings of different spinal tumors and their influence on the surgical strategy.

METHODS

We evaluated patients who underwent surgery for spinal tumor (extradural, intradural extramedullary, intradural intramedullary) removal, with ultrasound (US) guidance. Intraoperative standard B-mode images were acquired using a 3-11 MHz linear US probe. Before tumor removal the lesion was identified on the two axes and measured and defined as hyperechoic, isoechoic or hypoechoic. Other characteristics of the lesions were considered: the presence of calcifications, cystic/necrotic areas, diffuse or circumscribed appearance, and the relationships with the surrounding anatomical structures.

RESULTS

In all 34 cases it was possible to visualize the lesion, as well as the surrounding neural structures (like dura mater, dentate ligament, arachnoid membranes) and vascular structures. In 9 out of 34 cases, ioUS showed that the surgical approach was not wide enough: therefore it was necessary to enlarge the bony approach before dural opening. In 8 intramedullary cases, ioUS was used to correctly tailor the myelotomy.

CONCLUSIONS

We present our ioUS series findings along with some pictorial essays of different spinal tumors treated at our institution. IoUS is a valuable tool to detect spinal lesions, evaluate the surgical approach and plan the surgical strategy considering the position and relationships of the lesion with bony, neural and vascular structures.

Three-dimensional endovascular navigation with electromagnetic tracking: ex vivo and in vivo accuracy

PURPOSE

To evaluate the accuracy of a 3-dimensional (3D) navigation system using electromagnetically tracked tools to explore its potential in patients.

METHODS

The 3D navigation accuracy was quantified on a phantom and in a porcine model using the same setup and vascular interventional suite. A box-shaped phantom with 16 markers was scanned in 5 different positions using computed tomography (CT). The 3D navigation system registered each CT volume in the magnetic field. A tracked needle was pointed at the physical markers, and the spatial distances between the tracked needle positions and the markers were calculated. Contrast-enhanced CT images were acquired from 6 swine. The 3D navigation system registered each CT volume in the magnetic field. An electromagnetically tracked guidewire and catheter were visualized in the 3D image and navigated to 4 specified targets. At each target, the spatial distance between the tracked guidewire tip position and the actual position, verified by a CT control, was calculated.

RESULTS

The mean accuracy on the phantom was 1.28±0.53 mm, and 90% of the measured distances were ≤1.90 mm. The mean accuracy in swine was 4.18±1.76 mm, and 90% of the measured distances were ≤5.73 mm.

CONCLUSION

This 3D navigation system demonstrates good ex vivo accuracy and is sufficiently accurate in vivo to explore its potential for improved endovascular navigation.

Endovascular image-guided navigation: validation of two volume-volume registration algorithms

The limited volume covered by intraoperatively acquired CT scans makes the use of navigation systems difficult. Preoperative images cover a larger volume of interest. Hence, reliable registration of high quality preoperative to intraoperative CT will provide the necessary image information required for navigation. This study evaluates two algorithms (Siemens, CAMP) for volume-volume registration for usage during endovascular navigation. Twenty patients treated for abdominal aortic aneurysm were scanned with pre-, intra- and postoperative CT. Six data sets were excluded due to variations in image acquisition parameters and severe artifacts. Fourteen intra- and postoperative datasets were registered ten times with both algorithms, altogether 140 registrations for each program. In all data sets five specified landmarks placed by two radiologists were used to evaluate registration accuracy. The distance between the paired landmarks in the registered intra- and postoperative volumes was measured and the root mean square value calculated. Reference registrations were based on rigid body registration of the five landmarks in the intra- and postoperative volumes. Registration accuracy (mean ± SD) was for Siemens 5.05 ± 4.74 mm, for CAMP 4.02 ± 1.52 mm and for the reference registrations 2.72 ± 1.18 mm. The registration algorithms differed significantly, p < 0.001.

[New developments in computer-assisted surgery (CAS). From intraoperative imaging to ultrasound-based navigation]

Ever faster processor capacity is having an impact on computer-assisted or computer-aided surgery (CAS). The fusion of different imaging modalities enables functional data such as PET-CT, for example, to be available in image-guided surgery. Referencing of image data is the key to precise navigation. Intraoperative data acquisition is a new approach to improving accuracy. Thus, intraoperative CT conducted under navigational support enables automatic referencing of up-to-date image data. Alternatively, intraoperative magnetic resonance imaging or intraoperative sonography can be performed. Ultrasound systems have already been successfully integrated in existing navigational systems to compensate for intraoperative tissue shifting. Ultrasound systems may play a role in the future as a single modality in image-guided surgery in soft tissue of the neck and skull bone.

A surface registration method for quantification of intraoperative brain deformations in image-guided neurosurgery

Intraoperative brain deformations decrease accuracy in image-guided neurosurgery. Approaches to quantify these deformations based on 3-D reconstruction of cortectomy surfaces have been described and have shown promising results regarding the extrapolation to the whole brain volume using additional prior knowledge or sparse volume modalities. Quantification of brain deformations from surface measurement requires the registration of surfaces at different times along the surgical procedure, with different challenges according to the patient and surgical step. In this paper, we propose a new flexible surface registration approach for any textured point cloud computed by stereoscopic or laser range approach. This method includes three terms: the first term is related to image intensities, the second to Euclidean distance, and the third to anatomical landmarks automatically extracted and continuously tracked in the 2-D video flow. Performance evaluation was performed on both phantom and clinical cases. The global method, including textured point cloud reconstruction, had accuracy within 2 mm, which is the usual rigid registration error of neuronavigation systems before deformations. Its main advantage is to consider all the available data, including the microscope video flow with higher temporal resolution than previously published methods.

Role of pre- and intraoperative imaging and neuronavigation in neurosurgery

Advances in neuroimaging acquisition, computing and image processing have enabled neurosurgeons to use radiological imaging to guide both preoperative planning and intraoperative guidance. In preoperative planning, imaging may be used to evaluate surgical risks, choose the best method of intervention and select the safest surgical approach. Neuronavigation may be useful in designing the surgical flap and alerting the surgeon of surrounding anatomy. Finally, intraoperative imaging may be used to define brain shift associated with the resection of intracranial lesions, assist in more complete lesion resection, and monitor for certain intraoperative complications. In the following review, we briefly examine the history of neuroradiology for neurosurgery, neuronavigation and intraoperative imaging and trace their advances to current systems in use. We will also highlight new experimental applications of neuroimaging that are currently being refined.

2D/3D registration of endoscopic ultrasound to CT volume data

This paper describes a computer-aided navigation system using image fusion to support endoscopic interventions such as the accurate collection of biopsy specimens. An endoscope provides the physician with real-time ultrasound (US) and a video image. An image slice that corresponds to the corresponding image from the US scan head is derived from a preoperative computed tomography (CT) or magnetic resonance image volume data set using oblique reformatting and displayed side by side with the US image. The position of the image acquired by the US scan head is determined by a miniaturized electromagnetic tracking system (EMTS) after calibrating the endoscope's scan head. The transformation between the patient coordinate system and the preoperative data set is calculated using a 2D/3D registration. This is achieved by calibrating an intraoperative interventional CT slice with an optical tracking system (OTS) using the same algorithm as for the US calibration. The slice is then used for 2D/3D registration with the coordinate system of the preoperative volume. The fiducial registration error (FRE) for the US calibration was 2.0 mm +/- 0.4 mm; the interventional CT FRE was 0.36 +/- 0.12 mm; and the 2D/3D registration target registration error (TRE) was 1.8 +/- 0.3 mm. The point-to-point registration between the OTS and the EMTS had an FRE of 0.9 +/- 0.4 mm. Finally, we found an overall TRE for the complete system to be 3.9 +/- 0.6 mm.

A comparative analysis of coregistered ultrasound and magnetic resonance imaging in neurosurgery

OBJECTIVE

This work presents qualitative and quantitative side-by-side comparisons of oblique coregistered magnetic resonance imaging (MRI) scans and ultrasound images obtained during 35 neurosurgical procedures.

METHODS

Spatially registered series of ultrasound images were recorded for subsequent off-line evaluation and comparison with corresponding preoperative MRI studies. The degree of misalignment was reduced by reregistering the target volume directly with segmented features.

RESULTS

The initial apparent spatial misalignment of the target volume after craniotomy ranged from 0.11 to 8.73 mm (mean, 4.01 mm). After reregistration, the mutual information in overlapping segmented features was increased, presumably evidence of a better alignment locally. Additionally, the degree of feature congruence, which was assessed quantitatively through a convex hull approximation, demonstrated that the ultrasound volume was consistently smaller than its MRI counterpart.

CONCLUSION

Although intraoperative ultrasound tends to be difficult to interpret by itself, when accurately coregistered with preoperative MRI scans, its potential utility as a navigational guide is enhanced.

Navigation in laparoscopy--prototype research platform for improved image-guided surgery

The manipulation of the surgical field in laparoscopic surgery, through small incisions with rigid instruments, reduces free sight, dexterity, and tactile feedback. To help overcome some of these drawbacks, we present a prototype research and development platform, CustusX, for navigation in minimally invasive therapy. The system can also be used for planning and follow-up studies. With this platform we can import and display a range of medical images, also real-time data such as ultrasound and X-ray, during surgery. Tracked surgical tools, such as pointers, video laparoscopes, graspers, and various probes, allow surgeons to interactively control the display of medical images during the procedure. This paper introduces navigation technologies and methods for laparoscopic therapy, and presents our software and hardware research platform. Furthermore, we illustrate the use of the system with examples from two pilots performed during laparoscopic therapy. We also present new developments that are currently being integrated into the system for future use in the operating room. Our initial results from pilot studies using this technology with preoperative images and guidance in the retroperitoneum during laparoscopy are promising. Finally, we shortly describe an ongoing multicenter study using this surgical navigation system platform.

Analysis of existing methods for 3D modelling of femurs starting from two orthogonal images and development of a script for a commercial software package

BACKGROUND

At present the interest in medical field about the generation of three-dimensional digital models of anatomical structures increases due to the widespread diffusion of CAS--computer assisted surgery--systems. Most of them are based on CT--computer tomography--or MR--magnetic resonance--data volumes but sometimes this information is not available; there are only few X-ray, ultrasound or fluoroscopic images.

METHODS

This paper describes the study and the development of a script for a commercial software package (3ds Max) able to reconfigure the template model of a femur starting from two orthogonal images representing the specific patient's anatomy.

RESULTS

The script was used in several tests as summarized in this paper and the results appear to be interesting and acceptable, even for the medical experts that evaluated them.

CONCLUSIONS

The script developed in this work allows the generation of the 3D model of a femur in a very simple way (the user interface has been developed obeying to the main usability guidelines) and using a widespread commercial package. The quality of the results can be compared to the quality of more expensive and specialized systems.

Side-branched AAA stent graft insertion using navigation technology: a phantom study

OBJECTIVE

To evaluate the feasibility of a side-branched stent graft inserted in an artificial abdominal aortic aneurysm (AAA), using navigation technology, and to compare procedure duration and dose of radiation with control trials.

METHODS

A custom-made stent graft was inserted into an artificial AAA using navigation technology in combination with fluoroscopy. The navigation technology was based on three-dimensional visualization of computed tomography data and electromagnetic tracking of microposition sensors. The stent graft had integrated position sensors in side branch and introducer and was guided into proper position with the aid of three-dimensional images. Control trials were performed with fluoroscopy alone.

RESULTS

It was feasible to insert a side-branched stent graft using three-dimensional navigation technology. The navigation-guided trials had a significantly lower X-ray load (p < 0.001), but showed no difference in the duration of the procedures (p = 0.34) as compared with controls.

CONCLUSIONS

Inserting a side-branched stent graft in an artificial AAA using navigation technology is feasible. Side-branched stent grafts and navigation systems may become useful in the endovascular treatment of complicated aortic aneurysms.