Laparoscopic navigation pointer for three-dimensional image-guided surgery

Database-driven patient-specific registration error compensation method for image-guided laparoscopic surgery

PURPOSE

A surgical navigation system helps surgeons understand anatomical structures in the operative field during surgery. Patient-to-image registration, which aligns coordinate systems between the CT volume and a positional tracker, is vital for accurate surgical navigation. Although a point-based rigid registration method using fiducials on the body surface is often utilized for laparoscopic surgery navigation, precise registration is difficult due to such factors as soft tissue deformation. We propose a method that compensates a transformation matrix computed using fiducials on the body surface based on the analysis of positional information in the database.

METHODS

We built our database by measuring the positional information of the fiducials and the guidance targets in both the CT volume and positional tracker coordinate systems through previous surgeries. We computed two transformation matrices: using only the fiducials and using only the guidance targets in all the data in the database. We calculated the differences between the two transformation matrices in each piece of data. The compensation transformation matrix was computed by averaging these difference matrices. In this step, we selected the data from the database based on the similarity of the fiducials and the configuration of the guidance targets.

RESULTS

We evaluated our proposed method using 20 pieces of data acquired during laparoscopic gastrectomy for gastric cancer. The locations of blood vessels were used as guidance targets for computing target registration error. The mean target registration errors significantly decreased from 33.0 to 17.1 mm before and after the compensation.

CONCLUSION

This paper described a registration error compensation method using a database for image-guided laparoscopic surgery. Since our proposed method reduced registration error without additional intraoperative measurements during surgery, it increases the accuracy of surgical navigation for laparoscopic surgery.

Image-Aligned Dynamic Liver Reconstruction Using Intra-Operative Field of Views for Minimal Invasive Surgery

During hepatic minimal invasive surgery (MIS), 3-D reconstruction of a liver surface by interpreting the geometry of its soft tissues is achieving attractions. One of the major issues to be addressed in MIS is liver deformation. Moreover, it severely inhibits free sight and dexterity of tissue manipulation, which causes its intra-operative morphology and soft tissue motion altered as compared to its pre-operative shape. While many applications focus on 3-D reconstruction of rigid or semi-rigid scenes, the techniques applied in hepatic MIS must be able to cope with a dynamic and deformable environment. We propose an efficient technique for liver surface reconstruction based on the structure from motion to handle liver deformation. The reconstructed liver will assist surgeons to visualize liver surface more efficiently with better depth perception. We use the intra-operative field of views to generate 3-D template mesh from a dense keypoint cloud. We estimate liver deformation by finding best correspondence between 3-D templates and reconstruct a liver image to calculate translation and rotational motions. Our technique then finely tunes deformed surface by adding smoothness using shading cues. Up till now, this technique is not used for solving the human liver deformation problem. Our approach is tested and validated with synthetic as well as real in vivo data, which reveal that the reconstruction accuracy can be enhanced using our approach even in challenging laparoscopic environments.

Virtual and Augmented Reality in Oncologic Liver Surgery

Virtual reality (VR) and augmented reality (AR) in complex surgery are evolving technologies enabling improved preoperative planning and intraoperative navigation. The basis of these technologies is a computer-based generation of a patient-specific 3-dimensional model from Digital Imaging and Communications in Medicine (DICOM) data. This article provides a state-of-the- art overview on the clinical use of this technology with a specific focus on hepatic surgery. Although VR and AR are still in an evolving stage with only some clinical application today, these technologies have the potential to become a key factor in improving preoperative and intraoperative decision making.

Image-Guided Navigation in Lymph Node Biopsy

Background and Objectives:

Image-guided navigation is an effective intra-operative technology in select surgical sub-specialties. Laparoscopic and open lymph node biopsy are frequently undertaken to obtain adequate tissue of difficult lesions. Image-guided navigation may positively augment the precision and success of surgical lymph node biopsies.

Methods:

In this prospective pilot study, pre-operative imaging was uploaded into the navigation platform software, which superimposed the imaging and the subject's real-time anatomy. This required anatomical landmarks on the subject's body to be spatially registered with the platform using an infrared camera. This was then used to guide dissection and biopsy in laparoscopic and subcutaneous biopsies.

Results:

Image-guided lymph node biopsy was undertaken in 15 cases. Successful biopsy locations included: retroperitoneum, porta hepatis, mesentery, iliac region, para-aortic, axilla, and inguinal region. There was an 87% total absolute success rate in biopsies (89% in laparoscopic image-guided navigation [LIGN] and 83% in subcutaneous image-guided navigation [SIGN]). There was a 92% absolute success rate in lesions with fixed locations. There was a 67% absolute success rate in lesions with mobile locations.

Conclusion:

The investigators successfully incorporated image-guidance into surgical biopsy of lymph nodes in a diverse variety of locations. This image-guided technique for surgical biopsy can accurately and safely localize target lesions minimizing unnecessary dissection, conversion to open procedure, and re-operation for further tissue characterization. This technique was useful in the morbidly obese, instances of limited foci of disease, PET-active lesions, identifying areas of highest PET-avidity, and lesions with critical surrounding anatomy.

Laboratory test of Single Landmark registration method for ultrasound-based navigation in laparoscopy using an open-source platform

Purpose

Test the feasibility of the novel Single Landmark image-to-patient registration method for use in the operating room for future clinical trials. The algorithm is implemented in the open-source platform CustusX, a computer-aided intervention research platform dedicated to intraoperative navigation and ultrasound, with an interface for laparoscopic ultrasound probes.

Methods

The Single Landmark method is compared to fiducial landmark on an IOUSFAN (Kyoto Kagaku Co., Ltd., Japan) soft tissue abdominal phantom and T2 magnetic resonance scans of it.

Results

The experiments show that the accuracy of the Single Landmark registration is good close to the registered point, increasing with the distance from this point (12.4 mm error at 60 mm away from the registered point). In this point, the registration accuracy is mainly dominated by the accuracy of the user when clicking on the ultrasound image. In the presented set-up, the time required to perform the Single Landmark registration is 40% less than for the FLRM.

Conclusion

The Single Landmark registration is suitable for being integrated in a laparoscopic workflow. The statistical analysis shows robustness against translational displacements of the patient and improvements in terms of time. The proposed method allows the clinician to accurately register lesions intraoperatively by clicking on these in the ultrasound image provided by the ultrasound transducer. The Single Landmark registration method can be further combined with other more accurate registration approaches improving the registration at relevant points defined by the clinicians.

Three-Dimensional Liver Surgery Simulation: Computer-Assisted Surgical Planning with Three-Dimensional Simulation Software and Three-Dimensional Printing<sup/>

To perform accurate hepatectomy without injury, it is necessary to understand the anatomical relationship among the branches of Glisson's sheath, hepatic veins, and tumor. In Japan, three-dimensional (3D) preoperative simulation for liver surgery is becoming increasingly common, and liver 3D modeling and 3D hepatectomy simulation by 3D analysis software for liver surgery have been covered by universal healthcare insurance since 2012. Herein, we review the history of virtual hepatectomy using computer-assisted surgery (CAS) and our research to date, and we discuss the future prospects of CAS. We have used the SYNAPSE VINCENT medical imaging system (Fujifilm Medical, Tokyo, Japan) for 3D visualization and virtual resection of the liver since 2010. We developed a novel fusion imaging technique combining 3D computed tomography (CT) with magnetic resonance imaging (MRI). The fusion image enables us to easily visualize anatomic relationships among the hepatic arteries, portal veins, bile duct, and tumor in the hepatic hilum. In 2013, we developed an original software, called Liversim, which enables real-time deformation of the liver using physical simulation, and a randomized control trial has recently been conducted to evaluate the use of Liversim and SYNAPSE VINCENT for preoperative simulation and planning. Furthermore, we developed a novel hollow 3D-printed liver model whose surface is covered with frames. This model is useful for safe liver resection, has better visibility, and the production cost is reduced to one-third of a previous model. Preoperative simulation and navigation with CAS in liver resection are expected to help planning and conducting a surgery and surgical education. Thus, a novel CAS system will contribute to not only the performance of reliable hepatectomy but also to surgical education.

Robust augmented reality registration method for localization of solid organs' tumors using CT-derived virtual biomechanical model and fluorescent fiducials

BACKGROUND

Augmented reality (AR) is the fusion of computer-generated and real-time images. AR can be used in surgery as a navigation tool, by creating a patient-specific virtual model through 3D software manipulation of DICOM imaging (e.g., CT scan). The virtual model can be superimposed to real-time images enabling transparency visualization of internal anatomy and accurate localization of tumors. However, the 3D model is rigid and does not take into account inner structures' deformations. We present a concept of automated AR registration, while the organs undergo deformation during surgical manipulation, based on finite element modeling (FEM) coupled with optical imaging of fluorescent surface fiducials.

METHODS

Two 10 × 1 mm wires (pseudo-tumors) and six 10 × 0.9 mm fluorescent fiducials were placed in ex vivo porcine kidneys (n = 10). Biomechanical FEM-based models were generated from CT scan. Kidneys were deformed and the shape changes were identified by tracking the fiducials, using a near-infrared optical system. The changes were registered automatically with the virtual model, which was deformed accordingly. Accuracy of prediction of pseudo-tumors' location was evaluated with a CT scan in the deformed status (ground truth). In vivo: fluorescent fiducials were inserted under ultrasound guidance in the kidney of one pig, followed by a CT scan. The FEM-based virtual model was superimposed on laparoscopic images by automatic registration of the fiducials.

RESULTS

Biomechanical models were successfully generated and accurately superimposed on optical images. The mean measured distance between the estimated tumor by biomechanical propagation and the scanned tumor (ground truth) was 0.84 ± 0.42 mm. All fiducials were successfully placed in in vivo kidney and well visualized in near-infrared mode enabling accurate automatic registration of the virtual model on the laparoscopic images.

CONCLUSIONS

Our preliminary experiments showed the potential of a biomechanical model with fluorescent fiducials to propagate the deformation of solid organs' surface to their inner structures including tumors with good accuracy and automatized robust tracking.

Assessing Technical Skill in Surgery and Endoscopy: A Set of Metrics and an Algorithm (C-PASS) to Assess Skills in Surgical and Endoscopic Procedures

Historically, the performance of surgeons has been assessed subjectively by senior surgical staff in both training and operating environments. In this work, the position and motion of surgical instruments are analyzed through an objective process, denoted C-PASS, to measure surgeon performance of laparoscopic, endoscopic, and image-guided procedures. To develop C-PASS, clinically relevant performance characteristics were identified. Then measurement techniques for parameters that represented each characteristic were derived, and analytic techniques were implemented to transform these parameters into explicit, robust metrics. The metrics comprise the C-PASS performance assessment method, which has been validated over the last 3 years in studies of laparoscopy and endoscopy. These studies show that C-PASS is straightforward, reproducible, and accurate. It is sufficiently powerful to assess the efficiency of these complex processes. It is likely that C-PASS and similar approaches will improve skills acquisition and learning and also enable the objective comparison of systems and techniques.

Stereotactic navigation for TAMIS-TME

Stereotactic navigation allows for real-time, image-guided surgery, thus providing an augmented working environment for the operator. This technique can be applied to complex minimally invasive surgery for fixed anatomic targets. Transanal minimally invasive surgery represents a new approach to rectal cancer surgery that is technically demanding and introduces the potential for procedure-specific morbidity. Feasibility of stereotactic navigation for TAMIS-TME has been demonstrated, and this could theoretically translate into improved resection quality by improving the surgeon's spatial awareness. The future of minimally invasive surgery as it relates to augmented reality and image-guided surgery is discussed.

Clinical application of a surgical navigation system based on virtual laparoscopy in laparoscopic gastrectomy for gastric cancer

PURPOSE

Knowledge of the specific anatomical information of a patient is important when planning and undertaking laparoscopic surgery due to the restricted field of view and lack of tactile feedback compared to open surgery. To assist this type of surgery, we have developed a surgical navigation system that presents the patient's anatomical information synchronized with the laparoscope position. This paper presents the surgical navigation system and its clinical application to laparoscopic gastrectomy for gastric cancer.

METHODS

The proposed surgical navigation system generates virtual laparoscopic views corresponding to the laparoscope position recorded with a three-dimensional (3D) positional tracker. The virtual laparoscopic views are generated from preoperative CT images. A point-based registration aligns coordinate systems between the patient's anatomy and image coordinates. The proposed navigation system is able to display the virtual laparoscopic views using the registration result during surgery.

RESULTS

We performed surgical navigation during laparoscopic gastrectomy in 23 cases. The navigation system was able to present the virtual laparoscopic views in synchronization with the laparoscopic position. The fiducial registration error was calculated in all 23 cases, and the average was 14.0 mm (range 6.1-29.8).

CONCLUSION

The proposed surgical navigation system can provide CT-derived patient anatomy aligned to the laparoscopic view in real time during surgery. This system enables accurate identification of vascular anatomy as a guide to vessel clamping prior to total or partial gastrectomy.

Progressive internal landmark registration for surgical navigation in laparoscopic gastrectomy for gastric cancer

PURPOSE

A surgical navigation system supports the comprehension of anatomical information during surgery. Patient-to-image registration is the alignment process between CT volume and patient coordinate systems. Achieving accurate registration in the surgical navigation of laparoscopic surgery is very challenging due to soft tissue deformation. This paper presents a new patient-to-image registration method based on internal anatomical landmarks for improving registration accuracy in the surgical navigation of laparoscopic gastrectomy for gastric cancer.

METHODS

Our proposed registration method progressively utilizes internal anatomical landmarks. In laparoscopic gastrectomy for gastric cancer, the surgeon cuts the blood vessels around the stomach. The positions of the cut vessels are sequentially used as fiducials for registration during surgery. The proposed method uses a weighted point-based registration method for computing the transformation matrix using the fiducials both on the body surface and on the blood vessels. When a blood vessel is cut during surgery, the proposed progressive registration method measures the cut vessel's position and computes a transformation matrix by adding the cut vessel as a fiducial.

RESULTS

We applied our proposed progressive registration method using the positional information of the blood vessels acquired during laparoscopic gastrectomy in 20 cases. We evaluated it using target registration error in four blood vessels. The average target registration error in the four blood vessels was 12.6 mm and ranged from 2.1 to 32.9 mm.

CONCLUSION

Since the proposed progressive registration can reduce registration error, our proposed method is very useful for the surgical navigation of laparoscopic gastrectomy. Our proposed progressive registration method might increase the accuracy of surgical navigation in laparoscopic gastrectomy.

A Novel Ultrasound-Based Registration for Image-Guided Laparoscopic Liver Ablation

Background Patient-to-image registration is a core process of image-guided surgery (IGS) systems. We present a novel registration approach for application in laparoscopic liver surgery, which reconstructs in real time an intraoperative volume of the underlying intrahepatic vessels through an ultrasound (US) sweep process. Methods An existing IGS system for an open liver procedure was adapted, with suitable instrument tracking for laparoscopic equipment. Registration accuracy was evaluated on a realistic phantom by computing the target registration error (TRE) for 5 intrahepatic tumors. The registration work flow was evaluated by computing the time required for performing the registration. Additionally, a scheme for intraoperative accuracy assessment by visual overlay of the US image with preoperative image data was evaluated. Results The proposed registration method achieved an average TRE of 7.2 mm in the left lobe and 9.7 mm in the right lobe. The average time required for performing the registration was 12 minutes. A positive correlation was found between the intraoperative accuracy assessment and the obtained TREs. Conclusions The registration accuracy of the proposed method is adequate for laparoscopic intrahepatic tumor targeting. The presented approach is feasible and fast and may, therefore, not be disruptive to the current surgical work flow.

Image-guided real-time navigation for transanal total mesorectal excision: a pilot study

BACKGROUND

Frameless stereotaxy for real-time, image-guided surgery has been most utilized for neurological and orthopedic surgery. Recently, our center has reported the application of real-time navigation for transanal total mesorectal excision.

METHODS

During a 5-month period (June 2013-October 2013), three male patients underwent transanal minimally invasive surgery for total mesorectal excision with image-guided real-time navigation during the transanal portion of the operation. This was completed using a frameless stereotactic navigational system as shown in a demonstration video. Male patients with anterior, locally advanced rectal cancer were selected for enrollment into the pilot study.

RESULTS

Three male patients (mean age 69) underwent transanal total mesorectal excision with stereotactic navigation during a 5-month study period. Mean operative time was 402 min, and there were no intra-operative complications recorded. The mean distance from anal verge of the tumor was 6.3 cm (range 4-8 cm). The navigational accuracy was computed to be ±3.69 mm (range ±3.20 to ±4.02 mm). The average navigation setup time was 47 min, not including scan time. The surgical specimens were found to have completely intact mesorectal envelopes (Quirke 3) in all cases. All margins, including radial and distal margins, were negative. Mean postoperative length of stay was 5 days. At a median of 18-month follow-up, there was no evidence of locoregional recurrence or distant metastatic disease.

CONCLUSION

This is the first pilot series to report the use of frameless stereotactic navigation for TAMIS-TME. Stereotactic navigation for transanal total mesorectal excision is shown to be feasible, and may aid in providing colorectal surgeons with the ability to better perform safe, high-quality surgery in select cases.

Navigation surgery using an augmented reality for pancreatectomy

AIM

The aim of this study was to evaluate the utility of navigation surgery using augmented reality technology (AR-based NS) for pancreatectomy.

METHODS

The 3D reconstructed images from CT were created by segmentation. The initial registration was performed by using the optical location sensor. The reconstructed images were superimposed onto the real organs in the monitor display. Of the 19 patients who had undergone hepatobiliary and pancreatic surgery using AR-based NS, the accuracy, visualization ability, and utility of our system were assessed in five cases with pancreatectomy.

RESULTS

The position of each organ in the surface-rendering image corresponded almost to that of the actual organ. Reference to the display image allowed for safe dissection while preserving the adjacent vessels or organs. The locations of the lesions and resection line on the targeted organ were overlaid on the operating field. The initial mean registration error was improved to approximately 5 mm by our refinements. However, several problems such as registration accuracy, portability and cost still remain.

CONCLUSION

AR-based NS contributed to accurate and effective surgical resection in pancreatectomy. The pancreas appears to be a suitable organ for further investigations. This technology is promising to improve surgical quality, training, and education.

Spatial orientation in pathway surgery

BACKGROUND

In the field of minimally invasive surgery, specifically in pathway surgery (i.e. minimal invasive procedures carried out transluminally or through instrument-created pathways), spatial disorientation is a common experience to endoscopists. In this article, two effects that may cause spatial disorientation in pathway surgery, 'control-display compatibility' and 'local disorientation', were studied.

METHOD

A custom-developed simulator Endo-PaC was developed and used for mimicking pathway surgical scenarios. In Study 1, two ways of control-display alignment, normal mapping and mirrored mapping, were tested in combination with two control devices, thumb control and wrist control, in an orienting task using Endo-PaC. In Study 2, a tethered viewpoint was added to the virtual instrument tip. It was hypothesized that the visible tip would provide a cue of orientating direction in the reference frame during the instrument navigation. In both studies, novice participants were involved, and their performance was evaluated with regard to task time, path length travelled by the virtual tip, time and number of warnings, and subjective workload and personal preference.

RESULTS

In Study 1, normal-thumb and normal-wrist mapping yielded significantly lower means than mirrored-thumb and mirrored-wrist control for all investigated objective and subjective performance measurements. Out of 24 participants, 20 participants preferred normal control mapping. In Study 2, participants performed the task in shorter time and with shorter path length when the tip was visible tip on the monitor using a tethered viewpoint, but with a lower number and time of warnings without a visible tip.

CONCLUSION

The results of our studies show that eliminating the visual-display misalignment would greatly improve novice participants' performance, reduce the training time and their cognitive workload. A visible tip on the monitor would provide strong direction cue and shorten the performance time, but might introduce collision errors to novices and therefore requires longer training time.

Clinical application of navigation surgery using augmented reality in the abdominal field

This article presents general principles and recent advancements in the clinical application of augmented reality-based navigation surgery (AR based NS) for abdominal procedures and includes a description of our clinical trial and subsequent outcomes. Moreover, current problems and future aspects are discussed. The development of AR-based NS in the abdomen is delayed compared with another field because of the problem of intraoperative organ deformations or the existence of established modalities. Although there are a few reports on the clinical use of AR-based NS for digestive surgery, sophisticated technologies in urology have often been reported. However, the rapid widespread use of video- or robot assisted surgeries requires this technology. We have worked to develop a system of AR-based NS for hepatobiliary and pancreatic surgery. Then we developed a short rigid scope that enables surgeons to obtain 3D view. We recently focused on pancreatic surgery, because intraoperative organ shifting is minimal. The position of each organ in overlaid image almost corresponded with that of the actual organ with about 5 mm of mean registration errors. Intraoperative information generated from this system provided us with useful navigation. However, AR-based NS has several problems to overcome such as organ deformity, evaluation of utility, portability or cost.

Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance

BACKGROUND

The minimally invasive surgeon cannot use 'sense of touch' to orientate surgical resection, identifying important structures (vessels, tumors, etc.) by manual palpation. Robotic research has provided technology to facilitate laparoscopic surgery; however, robotics has yet to solve the lack of tactile feedback inherent to keyhole surgery. Misinterpretation of the vascular supply and tumor location may increase the risk of intraoperative bleeding and worsen dissection with positive resection margins.

METHODS

Augmented reality (AR) consists of the fusion of synthetic computer-generated images (three-dimensional virtual model) obtained from medical imaging preoperative work-up and real-time patient images with the aim of visualizing unapparent anatomical details.

RESULTS

In this article, we review the most common modalities used to achieve surgical navigation through AR, along with a report of a case of robotic duodenopancreatectomy using AR guidance complemented with the use of fluorescence guidance.

CONCLUSIONS

The presentation of this complex and high-technology case of robotic duodenopancreatectomy, and the overview of current technology that has made it possible to use AR in the operating room, highlights the needs for further evolution and the windows of opportunity to create a new paradigm in surgical practice.

Risk factors and management of conversions to an open approach in laparoscopic liver resection: analysis of 265 consecutive cases

BACKGROUND

As a consequence of continuous technical developments in liver surgery, laparoscopic liver resection (LLR) is increasingly performed worldwide.

METHODS

Between January 2004 and December 2011, 265 LLR were performed in 242 patients for various diseases. The experience of LLR is reported focusing on risk factors of conversion and their management.

RESULTS

The overall conversion rate was 17/265 (6.4%), equally distributed over the period of the study. Statistically significant factors for conversion were found to be LLR of the postero-superior (P-S) segments (SI, SIVa; SVII; SVIII) (12.7% converted versus 2.5% non-converted groups, P = 0.01) and a major compared with a minor hepatectomy (15.2% vs. 4.6%, P = 0.02 respectively). A R0 resection was achieved in 93.2% of cases. According to Dindo's classification, complications were recorded as grade I (n = 20); grade II (6); grade III (11) and grade IV(1) events (total morbidity rate of 14%). Univariate analysis identified a major hepatectomy and resection involving P-S segments as prognostic factors for conversion whereas multivariate analysis identified the latter as an independent risk factor [P = 0.003, odds ratio (OR) = 5.9, 95% confidence interval (CI) = 1.8-18.8].

CONCLUSIONS

LLR can be safely performed with low overall morbidity. According to this experience and irrespective of the learning curve, resections of P-S segments were identified as an independent risk factor for conversion in LLR.

Laparoscopic navigated liver resection: technical aspects and clinical practice in benign liver tumors

Laparoscopic liver resection has been performed mostly in centers with an extended expertise in both hepatobiliary and laparoscopic surgery and only in highly selected patients. In order to overcome the obstacles of this technique through improved intraoperative visualization we developed a laparoscopic navigation system (LapAssistent) to register pre-operatively reconstructed three-dimensional CT or MRI scans within the intra-operative field. After experimental development of the navigation system, we commenced with the clinical use of navigation-assisted laparoscopic liver surgery in January 2010. In this paper we report the technical aspects of the navigation system and the clinical use in one patient with a large benign adenoma. Preoperative planning data were calculated by Fraunhofer MeVis Bremen, Germany. After calibration of the system including camera, laparoscopic instruments, and the intraoperative ultrasound scanner we registered the surface of the liver. Applying the navigated ultrasound the preoperatively planned resection plane was then overlain with the patient's liver. The laparoscopic navigation system could be used under sterile conditions and it was possible to register and visualize the preoperatively planned resection plane. These first results now have to be validated and certified in a larger patient collective. A nationwide prospective multicenter study (ProNavic I) has been conducted and launched.

An on-board surgical tracking and video augmentation system for C-arm image guidance

PURPOSE

Conventional tracker configurations for surgical navigation carry a variety of limitations, including limited geometric accuracy, line-of-sight obstruction, and mismatch of the view angle with the surgeon's-eye view. This paper presents the development and characterization of a novel tracker configuration (referred to as "Tracker-on-C") intended to address such limitations by incorporating the tracker directly on the gantry of a mobile C-arm for fluoroscopy and cone-beam CT (CBCT).

METHODS

A video-based tracker (MicronTracker, Claron Technology Inc., Toronto, ON, Canada) was mounted on the gantry of a prototype mobile isocentric C-arm next to the flat-panel detector. To maintain registration within a dynamically moving reference frame (due to rotation of the C-arm), a reference marker consisting of 6 faces (referred to as a "hex-face marker") was developed to give visibility across the full range of C-arm rotation. Three primary functionalities were investigated: surgical tracking, generation of digitally reconstructed radiographs (DRRs) from the perspective of a tracked tool or the current C-arm angle, and augmentation of the tracker video scene with image, DRR, and planning data. Target registration error (TRE) was measured in comparison with the same tracker implemented in a conventional in-room configuration. Graphics processing unit (GPU)-accelerated DRRs were generated in real time as an assistant to C-arm positioning (i.e., positioning the C-arm such that target anatomy is in the field-of-view (FOV)), radiographic search (i.e., a virtual X-ray projection preview of target anatomy without X-ray exposure), and localization (i.e., visualizing the location of the surgical target or planning data). Video augmentation included superimposing tracker data, the X-ray FOV, DRRs, planning data, preoperative images, and/or intraoperative CBCT onto the video scene. Geometric accuracy was quantitatively evaluated in each case, and qualitative assessment of clinical feasibility was analyzed by an experienced and fellowship-trained orthopedic spine surgeon within a clinically realistic surgical setup of the Tracker-on-C.

RESULTS

The Tracker-on-C configuration demonstrated improved TRE (0.87 ± 0.25) mm in comparison with a conventional in-room tracker setup (1.92 ± 0.71) mm (p < 0.0001) attributed primarily to improved depth resolution of the stereoscopic camera placed closer to the surgical field. The hex-face reference marker maintained registration across the 180° C-arm orbit (TRE = 0.70 ± 0.32 mm). DRRs generated from the perspective of the C-arm X-ray detector demonstrated sub- mm accuracy (0.37 ± 0.20 mm) in correspondence with the real X-ray image. Planning data and DRRs overlaid on the video scene exhibited accuracy of (0.59 ± 0.38) pixels and (0.66 ± 0.36) pixels, respectively. Preclinical assessment suggested potential utility of the Tracker-on-C in a spectrum of interventions, including improved line of sight, an assistant to C-arm positioning, and faster target localization, while reducing X-ray exposure time.

CONCLUSIONS

The proposed tracker configuration demonstrated sub- mm TRE from the dynamic reference frame of a rotational C-arm through the use of the multi-face reference marker. Real-time DRRs and video augmentation from a natural perspective over the operating table assisted C-arm setup, simplified radiographic search and localization, and reduced fluoroscopy time. Incorporation of the proposed tracker configuration with C-arm CBCT guidance has the potential to simplify intraoperative registration, improve geometric accuracy, enhance visualization, and reduce radiation exposure.

Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device

PURPOSE

In endoscopic surgery, limited views and lack of tactile sensation restrict the surgeon's abilities and cause stress to the surgeon. Therefore, an intra-operative navigation system is strongly recommended. We developed an augmented reality (AR) navigation system based on preoperative CT imaging. The purpose of this study is to evaluate the usefulness, feasibility, and accuracy of this system using laparoscopic splenectomy in children.

METHODS

Volume images were reconstructed by three-dimensional (3D) viewer application. We used an optical tracking system for registration between volume image and body surface markers. The AR visualization was superimposed preoperative 3D CT images onto captured laparoscopic live images. This system was applied to six cases of laparoscopic splenectomy in children. To evaluate registration accuracy, distances from the marker position to the volume data were calculated.

RESULTS

The operator recognized the hidden vascular variation of the splenic artery and vein, accessory spleen, and pancreatic tail by overlaying an image onto a laparoscopic live image. The registration accuracy of six cases was 5.30 ± 0.08, 5.71 ± 1.70, 10.1 ± 0.60, 18.8 ± 3.56, 4.06 ± 1.71, and 7.05 ± 4.71.

CONCLUSION

This navigation system provides real-time anatomical information, which cannot be otherwise visualized without navigation. The registration accuracy was acceptable in clinical operation.

Navigated laparoscopic ultrasound in abdominal soft tissue surgery: technological overview and perspectives

PURPOSE

Two-dimensinal laparoscopic ultrasound (LUS) is commonly used for many laparoscopic procedures, but 3D LUS and navigation technology are not conventional tools in the clinic. Navigated LUS can help the user understand and interpret the ultrasound images in relation to the laparoscopic view and preoperative images. When combined with information from MRI or CT, navigated LUS has the potential to provide information about anatomic shifts during the procedure. In this paper, we present an overview of the ongoing technological research and development related to LUS combined with navigation technology, The purpose of this overview is threefold: (1) an introduction for those new to the field of navigated LUS; (2) an overview for those working in the field and; and (3) as a reference for those searching for literature on technological developments related to navigation in ultrasound-guided laparoscopic surgery.

METHODS

Databases were searched to identify relevant publications from the last 10 years.

RESULTS

We were able to identify 18 key papers in the area of navigated LUS for the abdomen, originating from about 10-11 groups. We present the literature overview, including descriptions of our own experience in the field, and a discussion of the important clinical and technological aspects related to navigated LUS.

CONCLUSIONS

LUS integrated with miniaturized tracking technology is likely to play an important role in guiding future laparoscopic surgery.

Augmented reality in laparoscopic surgical oncology

Minimally invasive surgery represents one of the main evolutions of surgical techniques aimed at providing a greater benefit to the patient. However, minimally invasive surgery increases the operative difficulty since the depth perception is usually dramatically reduced, the field of view is limited and the sense of touch is transmitted by an instrument. However, these drawbacks can currently be reduced by computer technology guiding the surgical gesture. Indeed, from a patient's medical image (US, CT or MRI), Augmented Reality (AR) can increase the surgeon's intra-operative vision by providing a virtual transparency of the patient. AR is based on two main processes: the 3D visualization of the anatomical or pathological structures appearing in the medical image, and the registration of this visualization on the real patient. 3D visualization can be performed directly from the medical image without the need for a pre-processing step thanks to volume rendering. But better results are obtained with surface rendering after organ and pathology delineations and 3D modelling. Registration can be performed interactively or automatically. Several interactive systems have been developed and applied to humans, demonstrating the benefit of AR in surgical oncology. It also shows the current limited interactivity due to soft organ movements and interaction between surgeon instruments and organs. If the current automatic AR systems show the feasibility of such system, it is still relying on specific and expensive equipment which is not available in clinical routine. Moreover, they are not robust enough due to the high complexity of developing a real-time registration taking organ deformation and human movement into account. However, the latest results of automatic AR systems are extremely encouraging and show that it will become a standard requirement for future computer-assisted surgical oncology. In this article, we will explain the concept of AR and its principles. Then, we will review the existing interactive and automatic AR systems in digestive surgical oncology, highlighting their benefits and limitations. Finally, we will discuss the future evolutions and the issues that still have to be tackled so that this technology can be seamlessly integrated in the operating room.

Upgrade of an optical navigation system with a permanent electromagnetic position control: a first step towards "navigated control" for liver surgery

INTRODUCTION

The main problems of navigation in liver surgery are organ movement and deformation. With a combination of direct optical and indirect electromagnetic tracking technology, visualisation and positional control of surgical instruments within three-dimensional ultrasound data and registration of organ movements can be realised simultaneously.

METHODS

Surgical instruments for liver resection were localised with an infrared-based navigation system (Polaris). Movements of the organ itself were registered using an electromagnetic navigation system (Aurora). The combination of these two navigation techniques and a new surgical navigation procedure focussed on a circumscribed critical dissection area were applied for the first time in liver resections.

RESULTS

This new technique was effectively implemented. The position of the surgical instrument was localised continuously. Repeated position control with observation of the navigation screen was not necessary. During surgical resection, a sonic warning signal was activated when the surgical instrument entered a "no touch" area--an area of reduced safety margin.

CONCLUSION

Optical tracking of surgical instruments and simultaneous electromagnetic registration of organ position is feasible in liver resection.

Novel Hands-Free Pointer Improves Instruction Efficiency in Laparoscopic Surgery

To improve instruction efficiency during advanced laparoscopic surgery, a hands-free, head-controlled, multimonitor pointer was developed. One instructor guided 20 trainees to locate critical points on a simulated laparoscopic cholecystectomy model. Twenty points, visible to the instructor only, were selected on a photo of a partially dissected gallbladder placed within a laparoscopic trainer box. For each trainee, the points were randomized to 2 groups of 10 points with the instructor providing verbal guidance only or guidance assisted by the head-controlled pointer that appeared on both the instructor's and trainees' monitors. The primary outcome was the time to locate 10 points. Total time was shorter with the pointer than with verbal guidance alone (65 ± 14 vs 119 ± 34 seconds, P < .001). The average of mean individual times to locate each point was shorter with the pointer than without (5.4 ± 0.5 vs 11.9 ± 2.4 seconds, P < .001). The instructor's efficiency improved over time with both verbal guidance ( P = .007) and with the pointer ( P = .001). The benefit of pointer instruction was greater in trainees with laparoscopic experience compared with those without experience ( P = .006). Use of a hands-free pointer improved instruction efficiency in simulated laparoscopy. Experienced surgeons benefited the most.

Spatial Mental Representation: Implications for Navigation System Design

Similarities exist in how people process and represent spatial information and in the factors that contribute to disorientation, whether one is moving through airspace, on the ground, or surgically within the body. As such, design principles for presenting spatial information should bear similarities across these domains but also be somewhat specific to each. In this chapter, we review research in spatial cognition and its application to navigation system design for within-vehicle, aviation, and endoscopic navigation systems. Taken together, the research suggests three general principles for navigation system design consideration. First, multimedia displays should present spatial information visually and action and description information verbally. Second, display organizations should meet users' dynamic navigational goals. Third, navigation systems should be adaptable to users' spatial information preferences. Designers of adaptive navigation display technologies can maximize the effectiveness of those technologies by appealing to the basic spatial cognition processes employed by all users while conforming to user's domain-specific requirements.

Three-dimensional image processing system for the ureter and urethra using endoscopic video

We developed new software that enables three-dimensional (3D) images of the ureter and urethra to be automatically processed from the video image of a conventional endoscope. The entire image of the ureter or urethra is displayed opened in a picture like a 3D map. The locations of lesions and the 3D structure of the prostatic urethra are depicted using this method, which enables quantitative evaluation of lesions and the obstructive potential of the prostatic urethra. Application of the technology in the urinary tract is discussed.

Navigation in laparoscopy--prototype research platform for improved image-guided surgery

The manipulation of the surgical field in laparoscopic surgery, through small incisions with rigid instruments, reduces free sight, dexterity, and tactile feedback. To help overcome some of these drawbacks, we present a prototype research and development platform, CustusX, for navigation in minimally invasive therapy. The system can also be used for planning and follow-up studies. With this platform we can import and display a range of medical images, also real-time data such as ultrasound and X-ray, during surgery. Tracked surgical tools, such as pointers, video laparoscopes, graspers, and various probes, allow surgeons to interactively control the display of medical images during the procedure. This paper introduces navigation technologies and methods for laparoscopic therapy, and presents our software and hardware research platform. Furthermore, we illustrate the use of the system with examples from two pilots performed during laparoscopic therapy. We also present new developments that are currently being integrated into the system for future use in the operating room. Our initial results from pilot studies using this technology with preoperative images and guidance in the retroperitoneum during laparoscopy are promising. Finally, we shortly describe an ongoing multicenter study using this surgical navigation system platform.

Image-guided surgery of liver metastases by three-dimensional ultrasound-based optoelectronic navigation

BACKGROUND

Vessel-oriented surgery and tumour-free resection margins are essential for resection of liver metastases to preserve liver parenchyma and improve oncological outcome. Preoperative three-dimensional models reconstructed from imaging data could facilitate surgical planning with the use of navigation technology.

METHODS

Thirty-three patients with central and/or impalpable liver metastases were scheduled for navigated hepatic resection. Intraoperative three-dimensional ultrasonography and an infrared-based optical tracking system were used for data registration and image-guided surgery. Postoperative three-dimensional data were compared with the preoperative virtual surgical plan to assess the accuracy of navigation, and clinical results were compared with those of a matched control group of 32 patients.

RESULTS

Navigation was successful in 32 of 33 patients. Realization of the preoperative plan and R0 resection was achieved in 30 of these 32 patients. The median discrepancy between the planned and actual vascular dissection level was 6 (range 0-11) mm. There was a reduced rate of R1 resection in the navigated group compared with the control group (two versus four patients), and more parenchyma was preserved.

CONCLUSION

Three-dimensional ultrasound-based optoelectronic navigation technology improves intraoperative orientation and enables parenchyma-preserving surgery with high precision.

Development and validation of a three dimensional ultrasound based navigation system for tumor resection

BACKGROUND

Intraoperative navigation is a rapidly emerging procedure in orthopaedic surgery and neurosurgery. For abdominal tumors (e.g. liver metastasis) and soft tissue tumors there is only limited experience with navigation techniques due to problems of organ shift and tissue deformation. We have developed a navigation system for tumor resection in soft tissue based on 3D ultrasound imaging and optical tracking.

METHODS

Two different modes of navigation were evaluated and compared with conventional surgery in an experimental soft tissue model. Both techniques were based on 3D ultrasound and an optical tracking system for intraoperative real time registration of surgical instruments. These two techniques were used: a) Indirect navigation with ultrasound guided insertion of a tracked hook needle into the tumor; and b) Direct navigation using a 3D image which was obtained with an optically tracked 3D ultrasound probe. It was the aim of both techniques to achieve a circumferential resection margin of 2cm around the tumor.

RESULTS

A total of 23 resections were performed consisting of indirect (n=7) and direct (n=10) navigation and conventional surgery (n=6) as gold standard. For indirect navigation a median deviation from the ideal resection margin (accuracy) of 0.32cm was measured. Direct navigation showed an accuracy of 0.16cm compared to 0.42cm with conventional surgery. Navigated surgery showed for both techniques a significant increase of resection accuracy compared to conventional resection (p<0.05).

CONCLUSION

3D ultrasound based indirect and direct optoelectronic navigation for resection of soft tissue tumors is feasible and may improve intraoperative orientation with increased surgical precision.

Side-branched AAA stent graft insertion using navigation technology: a phantom study

OBJECTIVE

To evaluate the feasibility of a side-branched stent graft inserted in an artificial abdominal aortic aneurysm (AAA), using navigation technology, and to compare procedure duration and dose of radiation with control trials.

METHODS

A custom-made stent graft was inserted into an artificial AAA using navigation technology in combination with fluoroscopy. The navigation technology was based on three-dimensional visualization of computed tomography data and electromagnetic tracking of microposition sensors. The stent graft had integrated position sensors in side branch and introducer and was guided into proper position with the aid of three-dimensional images. Control trials were performed with fluoroscopy alone.

RESULTS

It was feasible to insert a side-branched stent graft using three-dimensional navigation technology. The navigation-guided trials had a significantly lower X-ray load (p < 0.001), but showed no difference in the duration of the procedures (p = 0.34) as compared with controls.

CONCLUSIONS

Inserting a side-branched stent graft in an artificial AAA using navigation technology is feasible. Side-branched stent grafts and navigation systems may become useful in the endovascular treatment of complicated aortic aneurysms.

Using an image-guided navigation system for localization of small pulmonary nodules before thoracoscopic surgery

BACKGROUND

Video-assisted thoracic surgery (VATS) provides a minimally invasive means to resect small pulmonary nodules (SPN). However, thoracoscopy has limits in the detection of small nodules, which are invisible and/or impalpable during surgery. Methods to localize such lesions, including methylene blue injection or the introduction of a hookwire under the guidance of computed tomography (CT), have some limitations. We are developing a new technique using image-guided navigation system for localization of small pulmonary nodules before thoracoscopic surgery.

METHODS

Four pigs underwent spiral-computed tomography (CT) scanning after they were given percutaneously created pulmonary lesions. The CT data were transmitted to a StealthStation navigation system, and with the help of the probe the lesions were located and resected under thoracoscopy.

RESULTS

A total of 20 lesions were created. Nodules were located at an average distance of 15.6 mm from the pleural surface. All the lesions were successfully localized, and biopsy specimens revealed successful resection of target material.

CONCLUSION

This method can provide appropriate guidance to small pulmonary nodules and prove effective in immediately facilitating subsequent thoracoscopic resection.

Retroperitoneal hand-assisted laparoscopic surgery for endoscopic adrenalectomy

Although hand-assisted laparoscopic surgery (HALS) is very common in various laparoscopic procedures, it is rarely used for retroperitoneal endoscopic adrenalectomy because of the small working area. The authors evaluate HALS in endoscopic adrenalectomy with respect to its use as a rescue procedure in complicated cases. In their department, 47 patients underwent endoscopic adrenalectomies between 1998 and 2004. Mainly because of complicated anatomy, three primary aldosteronism cases were converted to retroperitoneal HALS. This involved making an additional 6 cm skin incision, into which the surgeon's left hand was inserted, with the palm used to create a sufficient visual field and working area. The fingers were used for tactile sensation and blunt resection. For these three cases, successful retroperitoneal HALS in endoscopic adrenalectomy resulted in no mortality or morbidity. These findings indicate that this procedure is a feasible technique for complicated benign adrenal tumor cases.