Incorporation of a laser range scanner into image-guided liver surgery: surface acquisition, registration, and tracking

Intraoperative liver deformation and organ motion caused by ventilation, laparotomy, and pneumoperitoneum in a porcine model for image-guided liver surgery

BACKGROUND

Image-guidance promises to make complex situations in liver interventions safer. Clinical success is limited by intraoperative organ motion due to ventilation and surgical manipulation. The aim was to assess influence of different ventilatory and operative states on liver motion in an experimental model.

METHODS

Liver motion due to ventilation (expiration, middle, and full inspiration) and operative state (native, laparotomy, and pneumoperitoneum) was assessed in a live porcine model (n = 10). Computed tomography (CT)-scans were taken for each pig for each possible combination of factors. Liver motion was measured by the vectors between predefined landmarks along the hepatic vein tree between CT scans after image segmentation.

RESULTS

Liver position changed significantly with ventilation. Peripheral regions of the liver showed significantly higher motion (maximal Euclidean motion 17.9 ± 2.7 mm) than central regions (maximal Euclidean motion 12.6 ± 2.1 mm, p < 0.001) across all operative states. The total average motion measured 11.6 ± 0.7 mm (p < 0.001). Between the operative states, the position of the liver changed the most from native state to pneumoperitoneum (14.6 ± 0.9 mm, p < 0.001). From native state to laparotomy comparatively, the displacement averaged 9.8 ± 1.2 mm (p < 0.001). With pneumoperitoneum, the breath-dependent liver motion was significantly reduced when compared to other modalities. Liver motion due to ventilation was 7.7 ± 0.6 mm during pneumoperitoneum, 13.9 ± 1.1 mm with laparotomy, and 13.5 ± 1.4 mm in the native state (p < 0.001 in all cases).

CONCLUSIONS

Ventilation and application of pneumoperitoneum caused significant changes in liver position. Liver motion was reduced but clearly measurable during pneumoperitoneum. Intraoperative guidance/navigation systems should therefore account for ventilation and intraoperative changes of liver position and peripheral deformation.

Stereovision-based integrated system for point cloud reconstruction and simulated brain shift validation

Intraoperative soft tissue deformation, referred to as brain shift, compromises the application of current image-guided surgery navigation systems in neurosurgery. A computational model driven by sparse data has been proposed as a cost-effective method to compensate for cortical surface and volumetric displacements. We present a mock environment developed to acquire stereoimages from a tracked operating microscope and to reconstruct three-dimensional point clouds from these images. A reconstruction error of 1 mm is estimated by using a phantom with a known geometry and independently measured deformation extent. The microscope is tracked via an attached tracking rigid body that facilitates the recording of the position of the microscope via a commercial optical tracking system as it moves during the procedure. Point clouds, reconstructed under different microscope positions, are registered into the same space to compute the feature displacements. Using our mock craniotomy device, realistic cortical deformations are generated. When comparing our tracked microscope stereo-pair measure of mock vessel displacements to that of the measurement determined by the independent optically tracked stylus marking, the displacement error was [Formula: see text] on average. These results demonstrate the practicality of using tracked stereoscopic microscope as an alternative to laser range scanners to collect sufficient intraoperative information for brain shift correction.

Improving Registration Robustness for Image-Guided Liver Surgery in a Novel Human-to-Phantom Data Framework

In open image-guided liver surgery (IGLS), a sparse representation of the intraoperative organ surface can be acquired to drive image-to-physical registration. We hypothesize that uncharacterized error induced by variation in the collection patterns of organ surface data limits the accuracy and robustness of an IGLS registration. Clinical validation of such registration methods is challenged due to the difficulty in obtaining data representative of the true state of organ deformation. We propose a novel human-to-phantom validation framework that transforms surface collection patterns from in vivo IGLS procedures (n = 13) onto a well-characterized hepatic deformation phantom for the purpose of validating surface-driven, volumetric nonrigid registration methods. An important feature of the approach is that it centers on combining workflow-realistic data acquisition and surgical deformations that are appropriate in behavior and magnitude. Using the approach, we investigate volumetric target registration error (TRE) with both current rigid IGLS and our improved nonrigid registration methods. Additionally, we introduce a spatial data resampling approach to mitigate the workflow-sensitive sampling problem. Using our human-to-phantom approach, TRE after routine rigid registration was 10.9 ± 0.6 mm with a signed closest point distance associated with residual surface fit in the range of ±10 mm, highly representative of open liver resections. After applying our novel resampling strategy and improved deformation correction method, TRE was reduced by 51%, i.e., a TRE of 5.3 ± 0.5 mm. This paper reported herein realizes a novel tractable approach for the validation of image-to-physical registration methods and demonstrates promising results for our correction method.

ConoSurf: Open-source 3D scanning system based on a conoscopic holography device for acquiring surgical surfaces

Background

A difficulty in computer‐assisted interventions is acquiring the patient's anatomy intraoperatively. Standard modalities have several limitations: low image quality (ultrasound), radiation exposure (computed tomography) or high costs (magnetic resonance imaging). An alternative approach uses a tracked pointer; however, the pointer causes tissue deformation and requires sterilizing. Recent proposals, utilizing a tracked conoscopic holography device, have shown promising results without the previously mentioned drawbacks.

Methods

We have developed an open‐source software system that enables real‐time surface scanning using a conoscopic holography device and a wide variety of tracking systems, integrated into pre‐existing and well‐supported software solutions.

Results

The mean target registration error of point measurements was 1.46 mm. For a quick guidance scan, surface reconstruction improved the surface registration error compared with point‐set registration.

Conclusions

We have presented a system enabling real‐time surface scanning using a tracked conoscopic holography device. Results show that it can be useful for acquiring the patient's anatomy during surgery.

A novel method for texture-mapping conoscopic surfaces for minimally invasive image-guided kidney surgery

PURPOSE

Organ-level registration is critical to image-guided therapy in soft tissue. This is especially important in organs such as the kidney which can freely move. We have developed a method for registration that combines three-dimensional locations from a holographic conoscope with an endoscopically obtained textured surface. By combining these data sources clear decisions as to the tissue from which the points arise can be made.

METHODS

By localizing the conoscope's laser dot in the endoscopic space, we register the textured surface to the cloud of conoscopic points. This allows the cloud of points to be filtered for only those arising from the kidney surface. Once a valid cloud is obtained we can use standard surface registration techniques to perform the image-space to physical-space registration. Since our methods use two distinct data sources we test for spatial accuracy and characterize temporal effects in phantoms, ex vivo porcine and human kidneys. In addition we use an industrial robot to provide controlled motion and positioning for characterizing temporal effects.

RESULTS

Our initial surface acquisitions are hand-held. This means that we take approximately 55 s to acquire a surface. At that rate we see no temporal effects due to acquisition synchronization or probe speed. Our surface registrations were able to find applied targets with submillimeter target registration errors.

CONCLUSION

The results showed that the textured surfaces could be reconstructed with submillimetric mean registration errors. While this paper focuses on kidney applications, this method could be applied to any anatomical structures where a line of sight can be created via open or minimally invasive surgical techniques.

A clinically applicable laser-based image-guided system for laparoscopic liver procedures

PURPOSE

Laser range scanners (LRS) allow performing a surface scan without physical contact with the organ, yielding higher registration accuracy for image-guided surgery (IGS) systems. However, the use of LRS-based registration in laparoscopic liver surgery is still limited because current solutions are composed of expensive and bulky equipment which can hardly be integrated in a surgical scenario.

METHODS

In this work, we present a novel LRS-based IGS system for laparoscopic liver procedures. A triangulation process is formulated to compute the 3D coordinates of laser points by using the existing IGS system tracking devices. This allows the use of a compact and cost-effective LRS and therefore facilitates the integration into the laparoscopic setup. The 3D laser points are then reconstructed into a surface to register to the preoperative liver model using a multi-level registration process.

RESULTS

Experimental results show that the proposed system provides submillimeter scanning precision and accuracy comparable to those reported in the literature. Further quantitative analysis shows that the proposed system is able to achieve a patient-to-image registration accuracy, described as target registration error, of [Formula: see text].

CONCLUSIONS

We believe that the presented approach will lead to a faster integration of LRS-based registration techniques in the surgical environment. Further studies will focus on optimizing scanning time and on the respiratory motion compensation.

Stereovision to MR image registration for cortical surface displacement mapping to enhance image-guided neurosurgery

PURPOSE

A surface registration method is presented to align intraoperative stereovision (iSV) with preoperative magnetic resonance (pMR) images, which utilizes both geometry and texture information to extract tissue displacements as part of the overall process of compensating for intraoperative brain deformation in order to maintain accurate neuronavigational image guidance during surgery.

METHODS

A sum-of-squared-difference rigid image registration was first executed to detect lateral shift of the cortical surface and was followed by a mutual-information-based block matching method to detect local nonrigid deformation caused by distention or collapse of the cortical surface. Ten (N = 10) surgical cases were evaluated in which an independent point measurement of a dominant cortical surface feature location was recorded with a tracked stylus in each case and compared to its surface-registered counterpart. The full three-dimensional (3D) displacement field was also extracted to drive a biomechanical brain deformation model, the results of which were reconciled with the reconstructed iSV surface as another form of evaluation.

RESULTS

Differences between the tracked stylus coordinates of cortical surface features and their surface-registered locations were 1.94 ± 0.59 mm on average across the ten cases. When the complete displacement map derived from surface registration was utilized, the resulting images generated from mechanical model updates were consistent in terms of both geometry (1-2 mm of model misfit) and texture, and were generated with less than 10 min of computational time. Analysis of the surface-registered 3D displacements indicate that the magnitude of motion ranged from 4.03 to 9.79 mm in the ten patient cases, and the amount of lateral shift was not related statistically to the direction of gravity (p = 0.73 ≫ 0.05) or the craniotomy size (p = 0.48 ≫ 0.05) at the beginning of surgery.

CONCLUSIONS

The iSV-pMR surface registration method utilizes texture and geometry information to extract both global lateral shift and local nonrigid movement of the cortical surface in 3D. The results suggest small differences exist in surface-registered locations when compared to positions measured independently with a coregistered stylus and when the full iSV surface was aligned with model-updated MR. The effectiveness and efficiency of the registration method is also minimally disruptive to surgical workflow.

A new markerless patient-to-image registration method using a portable 3D scanner

PURPOSE

Patient-to-image registration is critical to providing surgeons with reliable guidance information in the application of image-guided neurosurgery systems. The conventional point-matching registration method, which is based on skin markers, requires expensive and time-consuming logistic support. Surface-matching registration with facial surface scans is an alternative method, but the registration accuracy is unstable and the error in the more posterior parts of the head is usually large because the scan range is limited. This study proposes a new surface-matching method using a portable 3D scanner to acquire a point cloud of the entire head to perform the patient-to-image registration.

METHODS

A new method for transforming the scan points from the device space into the patient space without calibration and tracking was developed. Five positioning targets were attached on a reference star, and their coordinates in the patient space were measured prior. During registration, the authors moved the scanner around the head to scan its entire surface as well as the positioning targets, and the scanner generated a unique point cloud in the device space. The coordinates of the positioning targets in the device space were automatically detected by the scanner, and a spatial transformation from the device space to the patient space could be calculated by registering them to their coordinates in the patient space that had been measured prior. A three-step registration algorithm was then used to register the patient space to the image space. The authors evaluated their method on a rigid head phantom and an elastic head phantom to verify its practicality and to calculate the target registration error (TRE) in different regions of the head phantoms. The authors also conducted an experiment with a real patient's data to test the feasibility of their method in the clinical environment.

RESULTS

In the phantom experiments, the mean fiducial registration error between the device space and the patient space, the mean surface registration error, and the mean TRE of 15 targets on the surface of each phantom were 0.34 ± 0.01 mm and 0.33 ± 0.02 mm, 1.17 ± 0.02 mm and 1.34 ± 0.10 mm, and 1.06 ± 0.11 mm and 1.48 ± 0.21 mm, respectively. When grouping the targets according to their positions on the head, high accuracy was achieved in all parts of the head, and the TREs were similar across different regions. The authors compared their method with the current surface registration methods that use only a part of the facial surface on the elastic phantom, and the mean TRE of 15 targets was 1.48 ± 0.21 mm and 1.98 ± 0.53 mm, respectively. In a clinical experiment, the mean TRE of seven targets on the patient's head surface was 1.92 ± 0.18 mm, which was sufficient to meet clinical requirements.

CONCLUSIONS

The proposed surface-matching registration method provides sufficient registration accuracy even in the posterior area of the head. The 3D point cloud of the entire head, including the facial surface and the back of the head, can be easily acquired using a portable 3D scanner. The scanner does not need to be calibrated prior or tracked by the optical tracking system during scanning.

Generalized iterative most likely oriented-point (G-IMLOP) registration

PURPOSE

The need to align multiple representations of anatomy is a problem frequently encountered in clinical applications. A new algorithm for feature-based registration is presented that solves this problem by aligning both position and orientation information of the shapes being registered.

METHODS

The iterative most likely oriented-point (IMLOP) algorithm and its generalization (G-IMLOP) to the anisotropic noise case are described. These algorithms may be understood as probabilistic variants of the popular iterative closest point (ICP) algorithm. A probabilistic model provides the framework, wherein both position information and orientation information are simultaneously optimized. Like ICP, the proposed algorithms iterate between correspondence and registration subphases. Efficient and optimal solutions are presented for implementing each subphase of the proposed methods.

RESULTS

Experiments based on human femur data demonstrate that the IMLOP and G-IMLOP algorithms provide a strong accuracy advantage over ICP, with G-IMLOP providing additional accuracy improvement over IMLOP for registering data characterized by anisotropic noise. Furthermore, the proposed algorithms have increased ability to robustly identify an accurate versus inaccurate registration result.

CONCLUSION

The IMLOP and G-IMLOP algorithms provide a cohesive framework for incorporating orientation data into the registration problem, thereby enabling improvement in accuracy as well as increased confidence in the quality of registration outcomes. For shape data having anisotropic uncertainty in position and/or orientation, the anisotropic noise model of G-IMLOP enables further gains in registration accuracy to be achieved.

Near Real-Time Computer Assisted Surgery for Brain Shift Correction Using Biomechanical Models

Conventional image-guided neurosurgery relies on preoperative images to provide surgical navigational information and visualization. However, these images are no longer accurate once the skull has been opened and brain shift occurs. To account for changes in the shape of the brain caused by mechanical (e.g., gravity-induced deformations) and physiological effects (e.g., hyperosmotic drug-induced shrinking, or edema-induced swelling), updated images of the brain must be provided to the neuronavigation system in a timely manner for practical use in the operating room. In this paper, a novel preoperative and intraoperative computational processing pipeline for near real-time brain shift correction in the operating room was developed to automate and simplify the processing steps. Preoperatively, a computer model of the patient's brain with a subsequent atlas of potential deformations due to surgery is generated from diagnostic image volumes. In the case of interim gross changes between diagnosis, and surgery when reimaging is necessary, our preoperative pipeline can be generated within one day of surgery. Intraoperatively, sparse data measuring the cortical brain surface is collected using an optically tracked portable laser range scanner. These data are then used to guide an inverse modeling framework whereby full volumetric brain deformations are reconstructed from precomputed atlas solutions to rapidly match intraoperative cortical surface shift measurements. Once complete, the volumetric displacement field is used to update, i.e., deform, preoperative brain images to their intraoperative shifted state. In this paper, five surgical cases were analyzed with respect to the computational pipeline and workflow timing. With respect to postcortical surface data acquisition, the approximate execution time was 4.5 min. The total update process which included positioning the scanner, data acquisition, inverse model processing, and image deforming was ~11-13 min. In addition, easily implemented hardware, software, and workflow processes were identified for improved performance in the near future.

Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization

Time-of-flight (ToF) cameras calculate depth maps by reconstructing phase shifts of amplitude-modulated signals. For broad illumination of transparent objects, reflections from multiple scene points can illuminate a given pixel, giving rise to an erroneous depth map. We report here a sparsity-regularized solution that separates K interfering components using multiple modulation frequency measurements. The method maps ToF imaging to the general framework of spectral estimation theory and has applications in improving depth profiles and exploiting multiple scattering.

Evolution of image-guided liver surgery: transition from open to laparoscopic procedures

INTRODUCTION

Indications for liver surgery to treat primary and secondary hepatic malignancies are broadening. Utilizing data from B-mode or 2-D intraoperative ultrasound, it is often challenging to replicate the findings from preoperative CT or MRI scans. Additional data from more recently developed image-guidance technology, which registers preoperative axial imaging to a 3-D real-time model, may be used to improve operative planning, locate difficult to find hepatic tumors, and guide ablations.

METHODS

Laparoscopic liver procedures are often more challenging than their open counterparts. Image-guidance technology can assist in overcoming some of the obstacles to minimally invasive liver procedures by enhancing ultrasound findings and ablation guidance. This manuscript describes a protocol that evaluated an open image-guidance system, and a subsequent protocol that directly compared, for validation, a laparoscopic with an open image-guidance system. Both protocols were limited to ablations within the liver.

DISCUSSION

The laparoscopic image-guidance system successfully creates a 3-D model at both 7 and 14 mm Hg that is similar to the open 3-D model. Ultimately, improving intraoperative image guidance can help expand the ability to perform both laparoscopic and open liver surgeries.

A study on the theoretical and practical accuracy of conoscopic holography-based surface measurements: toward image registration in minimally invasive surgery

BACKGROUND

Registered medical images can assist with surgical navigation and enable image-guided therapy delivery. In soft tissues, surface-based registration is often used and can be facilitated by laser surface scanning. Tracked conoscopic holography (which provides distance measurements) has been recently proposed as a minimally invasive way to obtain surface scans. Moving this technique from concept to clinical use requires a rigorous accuracy evaluation, which is the purpose of our paper.

METHODS

We adapt recent non-homogeneous and anisotropic point-based registration results to provide a theoretical framework for predicting the accuracy of tracked distance measurement systems. Experiments are conducted a complex objects of defined geometry, an anthropomorphic kidney phantom and a human cadaver kidney.

RESULTS

Experiments agree with model predictions, producing point RMS errors consistently < 1 mm, surface-based registration with mean closest point error < 1 mm in the phantom and a RMS target registration error of 0.8 mm in the human cadaver kidney.

CONCLUSIONS

Tracked conoscopic holography is clinically viable; it enables minimally invasive surface scan accuracy comparable to current clinical methods that require open surgery.

Comparison study of intraoperative surface acquisition methods for surgical navigation

Soft-tissue image-guided interventions often require the digitization of organ surfaces for providing correspondence from medical images to the physical patient in the operating room. In this paper, the effect of several inexpensive surface acquisition techniques on target registration error and surface registration error (SRE) for soft tissue is investigated. A systematic approach is provided to compare image-to-physical registrations using three different methods of organ spatial digitization: 1) a tracked laser-range scanner (LRS), 2) a tracked pointer, and 3) a tracked conoscopic holography sensor (called a conoprobe). For each digitization method, surfaces of phantoms and biological tissues were acquired and registered to CT image volume counterparts. A comparison among these alignments demonstrated that registration errors were statistically smaller with the conoprobe than the tracked pointer and LRS (p<0.01). In all acquisitions, the conoprobe outperformed the LRS and tracked pointer: for example, the arithmetic means of the SRE over all data acquisitions with a porcine liver were 1.73 ± 0.77 mm, 3.25 ± 0.78 mm, and 4.44 ± 1.19 mm for the conoprobe, LRS, and tracked pointer, respectively. In a cadaveric kidney specimen, the arithmetic means of the SRE over all trials of the conoprobe and tracked pointer were 1.50 ± 0.50 mm and 3.51 ± 0.82 mm, respectively. Our results suggest that tissue displacements due to contact force and attempts to maintain contact with tissue, compromise registrations that are dependent on data acquired from a tracked surgical instrument and we provide an alternative method (tracked conoscopic holography) of digitizing surfaces for clinical usage. The tracked conoscopic holography device outperforms LRS acquisitions with respect to registration accuracy.

Image-Guided Abdominal Surgery and Therapy Delivery

Image-Guided Surgery has become the standard of care in intracranial neurosurgery providing more exact resections while minimizing damage to healthy tissue. Moving that process to abdominal organs presents additional challenges in the form of image segmentation, image to physical space registration, organ motion and deformation. In this paper, we present methodologies and results for addressing these challenges in two specific organs: the liver and the kidney.

Design and evaluation of an optically-tracked single-CCD laser range scanner

PURPOSE

Acquisition of laser range scans of an organ surface has the potential to efficiently provide measurements of geometric changes to soft tissue during a surgical procedure. A laser range scanner design is reported here which has been developed to drive intraoperative updates to conventional image-guided neurosurgery systems.

METHODS

The scanner is optically-tracked in the operating room with a multiface passive target. The novel design incorporates both the capture of surface geometry (via laser illumination) and color information (via visible light collection) through a single-lens onto the same charge-coupled device (CCD). The accuracy of the geometric data was evaluated by scanning a high-precision phantom and comparing relative distances between landmarks in the scans with the corresponding ground truth (known) distances. The range-of-motion of the scanner with respect to the optical camera was determined by placing the scanner in common operating room configurations while sampling the visibility of the reflective spheres. The tracking accuracy was then analyzed by fixing the scanner and phantom in place, perturbing the optical camera around the scene, and observing variability in scan locations with respect to a tracked pen probe ground truth as the camera tracked the same scene from different positions.

RESULTS

The geometric accuracy test produced a mean error and standard deviation of 0.25 ± 0.40 mm with an RMS error of 0.47 mm. The tracking tests showed that the scanner could be tracked at virtually all desired orientations required in the OR set up, with an overall tracking error and standard deviation of 2.2 ± 1.0 mm with an RMS error of 2.4 mm. There was no discernible difference between any of the three faces on the lasers range scanner (LRS) with regard to tracking accuracy.

CONCLUSIONS

A single-lens laser range scanner design was successfully developed and implemented with sufficient scanning and tracking accuracy for image-guided surgery.

Organ surface deformation measurement and analysis in open hepatic surgery: method and preliminary results from 12 clinical cases

The incidence of soft tissue deformation has been well documented in neurosurgical procedures and is known to compromise the spatial accuracy of image-guided surgery systems.Within the context of image-guided liver surgery (IGLS), no detailed method to study and analyze the observed organ shape change between preoperative imaging and the intra-operative presentation has been developed. Contrary to the studies of deformation in neurosurgical procedures, the majority of deformation in IGLS is imposed prior to resection and due to laparotomy and mobilization. As such, methods of analyzing the organ shape change must be developed to use the intra-operative data (e.g. laser range scan (LRS) surfaces) acquired with the organ in its fully deformed shape. To achieve this end we use a signed closest point distance deformation metric computed after rigid alignment of the intra-operative LRS data with organ surfaces generated from the preoperative tomograms. The rigid alignment between the intra-operative LRS surfaces and pre-operative image data was computed with a feature weighted surface registration algorithm. In order to compare the deformation metrics across patients, an inter-patient non-rigid registration of the pre-operative CT images was performed. Given the inter-patient liver registrations, an analysis was performed to determine the potential similarities in the distribution of measured deformation between patients for which similar procedures had been performed. The results of the deformation measurement and analysis indicates the potential for soft tissue deformation to compromise surgical guidance information and suggests a similarity in imposed deformation among similar procedure types.

Model-updated image-guided liver surgery: preliminary results using surface characterization

The current protocol for image guidance in open abdominal liver tumor removal surgeries involves a rigid registration between the patient's operating room space and the pre-operative diagnostic image-space. Systematic studies have shown that the liver can deform up to 2 cm during surgeries in a non-rigid fashion thereby compromising the accuracy of these surgical navigation systems. Compensating for intra-operative deformations using mathematical models has shown promising results. In this work, we follow up the initial rigid registration with a computational approach that is geared towards minimizing the residual closest point distances between the un-deformed pre-operative surface and the rigidly registered intra-operative surface. We also use a surface Laplacian equation based filter that generates a realistic deformation field. Preliminary validation of the proposed computational framework was performed using phantom experiments and clinical trials. The proposed framework improved the rigid registration errors for the phantom experiments on average by 43%, and 74% using partial and full surface data, respectively. With respect to clinical data, it improved the closest point residual error associated with rigid registration by 54% on average for the clinical cases. These results are highly encouraging and suggest that computational models can be used to increase the accuracy of image-guided open abdominal liver tumor removal surgeries.

Fast-MICP for frameless image-guided surgery

PURPOSE

In image-guided surgery (IGS) systems, image-to-physical registration is critical for reliable anatomical information mapping and spatial guidance. Conventional stereotactic frame-based or fiducial-based approaches provide accurate registration but are not patient-friendly. This study proposes a frameless cranial IGS system that uses computer vision techniques to replace the frame or fiducials with the natural features of the patient.

METHODS

To perform a cranial surgery with the proposed system, the facial surface of the patient is first reconstructed by stereo vision. Accuracy is ensured by capturing parallel-line patterns projected from a calibrated LCD projector. Meanwhile, another facial surface is reconstructed from preoperative computed tomography (CT) images of the patient. The proposed iterative closest point (ICP)-based algorithm [fast marker-added ICP (Fast-MICP)] is then used to register the two facial data sets, which transfers the anatomical information from the CT images to the physical space.

RESULTS

Experimental results reveal that the Fast-MICP algorithm reduces the computational cost of marker-added ICP (J.-D. Lee et al., "A coarse-to-fine surface registration algorithm for frameless brain surgery," in Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society, 2007, pp. 836-839) to 10% and achieves comparable registration accuracy, which is under 3 mm target registration error (TRE). Moreover, two types of optical-based spatial digitizing devices can be integrated for further surgical navigation. Anatomical information or image-guided surgical landmarks can be projected onto the patient to obtain an immersive augmented reality environment.

CONCLUSION

The proposed frameless IGS system with stereo vision obtains TRE of less than 3 mm. The proposed Fast-MICP registration algorithm reduces registration time by 90% without compromising accuracy.

Minimally invasive holographic surface scanning for soft-tissue image registration

Recent advances in registration have extended intrasurgical image guidance from its origins in bone-based procedures to new applications in soft tissues, thus enabling visualization of spatial relationships between surgical instruments and subsurface structures before incisions begin. Preoperative images are generally registered to soft tissues through aligning segmented volumetric image data with an intraoperatively sensed cloud of organ surface points. However, there is currently no viable noncontact minimally invasive scanning technology that can collect these points through a single laparoscopic port, which limits wider adoption of soft-tissue image guidance. In this paper, we describe a system based on conoscopic holography that is capable of minimally invasive surface scanning. We present the results of several validation experiments scanning ex vivo biological and phantom tissues with a system consisting of a tracked, off-the-shelf, relatively inexpensive conoscopic holography unit. These experiments indicate that conoscopic holography is suitable for use with biological tissues, and can provide surface scans of comparable quality to existing clinically used laser range scanning systems that require open surgery. We demonstrate experimentally that conoscopic holography can be used to guide a surgical needle to desired subsurface targets with an average tip error of less than 3 mm.

Vessel-Based Non-Rigid Registration of MR/CT and 3D Ultrasound for Navigation in Liver Surgery

OBJECTIVE

Computer assisted planning of liver surgery based on preoperative computed tomography (CT) or magnetic resonance imaging (MRI) data can be an important aid to operability decisions and visualization of individual patients' 3D anatomy. A navigation system based on intraoperative 3D ultrasound may help the surgeon to precisely localize vessels, vascular territories or tumors. The preoperative planning must be transferred to the intraoperative ultrasound data and thus to the patient on the operating table. Due to deformations of the liver between planning and surgery, a fast non-rigid registration method is needed.

MATERIALS AND METHODS

A feature-based non-rigid registration approach based on the centerlines of the portal veins has been developed. The combination of an iterative closest point (ICP) approach and Multilevel B-Spline transformations offers a fast registration method. The vessels are segmented and their centerlines extracted from preoperative CT/MRI and intraoperative 3D Powerdoppler ultrasound data. Anatomical corresponding points on the centerlines of both modalities are determined in each iteration of the ICP algorithm. The search for corresponding points is restricted to a given search radius and the direction of the vessels is incorporated.

RESULTS

The algorithm has been evaluated on two transcutaneous and one intraoperative clinical ultrasound data set from three different patients. Only a very few vessel segments were not assigned correctly compared to manual assignments. Using non-rigid transformations improved the root mean square target registration error of the vessels by approximately 3-5 mm.

CONCLUSIONS

The proposed registration method is fast enough for clinical application in liver surgery. Initial accuracy results are promising and must be further evaluated, particularly in the operating room.

Role of image-guidance systems during NOTES

Natural orifice translumenal endoscopic surgery (NOTES) is a developing field with the potential to revolutionize our approach to abdominal surgery. Performing operations via a flexible endoscope introduced through a natural orifice presents several challenges to physicians. Orientation and interpretation of the endoscopic video image can be difficult. The surgeon must also learn to operate with the camera and instruments "in line." Advances in technology are currently addressing the challenges of NOTES. Image-guided navigation could potentially provide invaluable assistance during NOTES. Real-time information on spatial positioning and orientation as well as assistance with the identification of anatomy and localization of pathology are some of the possibilities. Image-guided surgery has become commonplace in disciplines such as neurosurgery where the anatomy is relatively rigid. To become widespread in intra-abdominal procedures and NOTES, advances that will allow systems to adapt to moving and deforming anatomy are needed. This article reviews the basics of image-guided surgery, the various image-guided systems, and their potential application to NOTES.

Range data pre-processing for the evaluation of torso shape and symmetry in scoliosis

Pre-processing range scans of the human torso for evaluating shape and symmetry changes in scoliosis are non-trivial. First, stray points from surrounding artefacts are often arbitrarily positioned and not amenable to automatic removal. Second, the asymmetrical alignment of the arms and neck makes cropping them difficult. Third, despite a plethora of methods, removal of holes by surface approximation for this niche application remains a challenge particularly in obscure regions like the sides and armpits. This paper proposes a novel surface approximation method and incorporates it into an integrated procedure for pre-processing range scans of the torso that includes interactive tools for cropping stray points and extremities. The new method, spline-fitted moving least squares (MLS), makes use of the Bezier curve and MLS algorithms. Numeric and clinical tests on scans of 30 volunteers, with and without scoliosis, show that the proposed method outperforms its constituent methods and a commercially available graphics package for this application.

Estimation of operative line of resection using preoperative image and nonrigid registration

Even though accurate diagnosis of organs is done using preoperative images such as CT or MRI, these information are not directly used in the operating room, because organs are nonrigid and their shapes change with time. In this paper, we propose to obtain an intraoperative image of an open organ and fuse the image with a preoperative image. The intraoperative image is obtained from a three-dimensional laser scanner. The registration of preoperative image to the intraoperative image can relate the information from the preoperative image to the open organ in the operating room. We do this by registering preoperative images to intraoperative images. An algorithm based on Robust Point Matching method is developed for this nonrigid image registration problem. We also propose a new metric called Non Overlapping Ratio to determine the registration error. The experiments demonstrate that the proposed method is capable of achieving region of interest estimation within 1.51 mm mean distance error and 0.66% Non Overlapping Ratio.

Image guided medialization laryngoplasty

Techniques that originate in computer graphics and computer vision have found prominent applications in the medical domain. In this paper, we have seamlessly developed techniques from computer graphics and computer vision together with domain knowledge from medicine to develop an image guided surgical system for medialization laryngoplasty. The technical focus of this paper is to register the preoperative radiological data to the intraoperative anatomical structure of the patient. With careful analysis of the real-world surgical environment, we have developed an ICP-based partial shape matching algorithm to register the partially visible anatomical structure to the preoperative CT data. We extracted distinguishable features from the human thyroid cartilage surface and applied image space template matching to find the initial guess for the shape matching. The experimental result shows that our feature-based partial shape matching method has better performance and robustness compared with original ICP-based shape matching method. Although this paper concentrates on the medialization laryngoplasty procedure, its generality makes our methods ideal for future applications in other image guided surgical areas.

Feasibility study for image-guided kidney surgery: assessment of required intraoperative surface for accurate physical to image space registrations

A notable complication of applying current image-guided surgery techniques of soft tissue to kidney resections (nephrectomies) is the limited field of view of the intraoperative kidney surface. This limited view constrains the ability to obtain a sufficiently geometrically descriptive surface for accurate surface-based registrations. The authors examined the effects of the limited view by using two orientations of a kidney phantom to model typical laparoscopic and open partial nephrectomy views. Point-based registrations, using either rigidly attached markers or anatomical landmarks as fiducials, served as initial alignments for surface-based registrations. Laser range scanner (LRS) obtained surfaces were registered to the phantom's image surface using a rigid iterative closest point algorithm. Subsets of each orientation's LRS surface were used in a robustness test to determine which parts of the surface yield the most accurate registrations. Results suggest that obtaining accurate registrations is a function of the percentage of the total surface and of geometric surface properties, such as curvature. Approximately 28% of the total surface is required regardless of the location of that surface subset. However, that percentage decreases when the surface subset contains information from opposite ends of the surface and/or unique anatomical features, such as the renal artery and vein.

Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation

A successful surface-based image-to-physical space registration in image-guided liver surgery (IGLS) is critical to provide reliable guidance information to surgeons and pertinent surface displacement data for use in deformation correction algorithms. The current protocol used to perform the image-to-physical space registration involves an initial pose estimation provided by a point based registration of anatomical landmarks identifiable in both the preoperative tomograms and the intraoperative presentation. The surface based registration is then performed via a traditional iterative closest point (ICP) algorithm between the preoperative liver surface, segmented from the tomographic image set, and an intraoperatively acquired point cloud of the liver surface provided by a laser range scanner. Using this more conventional method, the registration accuracy can be compromised by poor initial pose estimation as well as tissue deformation due to the laparotomy and liver mobilization performed prior to tumor resection. In order to increase the robustness of the current surface-based registration method used in IGLS, we propose the incorporation of salient anatomical features, identifiable in both the preoperative image sets and intraoperative liver surface data, to aid in the initial pose estimation and play a more significant role in the surface-based registration via a novel weighting scheme. Examples of such salient anatomical features are the falciform groove region as well as the inferior ridge of the liver surface. In order to validate the proposed weighted patch registration method, the alignment results provided by the proposed algorithm using both single and multiple patch regions were compared with the traditional ICP method using six clinical datasets. Robustness studies were also performed using both phantom and clinical data to compare the resulting registrations provided by the proposed algorithm and the traditional method under conditions of varying initial pose. The results provided by the robustness trials and clinical registration comparisons suggest that the proposed weighted patch registration algorithm provides a more robust method with which to perform the image-to-physical space registration in IGLS. Furthermore, the implementation of the proposed algorithm during surgical procedures does not impose significant increases in computation or data acquisition times.

Laparoscopic surface scanning and subsurface targeting: Implications for image-guided laparoscopic liver surgery

Segmental liver resection and locoregional ablative therapies are dependent upon accurate tumor localization to ensure safety as well as acceptable oncologic results. Because of the liver's limited external landmarks and complex internal anatomy, such tumor localization poses a technical challenge. Image guided therapies (IGT) address this problem by mapping the real-time, intraoperative position of surgical instruments onto preoperative tomographic imaging through a process called registration. Accuracy is critical to IGT and is a function of: 1) the registration technique, 2) the tissue characteristics, and 3) imaging techniques. The purpose of this study is to validate a novel method of registration using an endoscopic Laser Range Scanner (eLRS) and demonstrate its applicability to laparoscopic liver surgery. Six radiopaque targets were inserted into an ex-vivo bovine liver and a computed tomography (CT) scan was obtained. Using the eLRS, the liver surface was scanned and a surface-based registration was constructed to predict the position of the intraparenchymal targets. The target registration error (TRE) achieved using our surface-based registration was 2.4 +/- 1.0 mm. A comparable TRE using traditional fiducial-based registration was 2.6 +/- 1.7 mm. Compared to traditional fiducial-based registration, laparoscopic surface scanning is able to predict the location of intraparenchymal liver targets with similar accuracy and rate of data acquisition.

Concepts and preliminary data toward the realization of image-guided liver surgery

Image-guided surgery provides navigational assistance to the surgeon by displaying the surgical probe position on a set of preoperative tomograms in real time. In this study, the feasibility of implementing image-guided surgery concepts into liver surgery was examined during eight hepatic resection procedures. Preoperative tomographic image data were acquired and processed. Accompanying intraoperative data on liver shape and position were obtained through optically tracked probes and laser range scanning technology. The preoperative and intraoperative representations of the liver surface were aligned using the iterative closest point surface matching algorithm. Surface registrations resulted in mean residual errors from 2 to 6 mm, with errors of target surface regions being below a stated goal of 1 cm. Issues affecting registration accuracy include liver motion due to respiration, the quality of the intraoperative surface data, and intraoperative organ deformation. Respiratory motion was quantified during the procedures as cyclical, primarily along the cranial-caudal direction. The resulting registrations were more robust and accurate when using laser range scanning to rapidly acquire thousands of points on the liver surface and when capturing unique geometric regions on the liver surface, such as the inferior edge. Finally, finite element models recovered much of the observed intraoperative deformation, further decreasing errors in the registration. Image-guided liver surgery has shown the potential to provide surgeons with important navigation aids that could increase the accuracy of targeting lesions and the number of patients eligible for surgical resection.

Comparison of laser surface scanning and fiducial marker–based registration in frameless stereotaxy

✓The authors compared the accuracy of laser surface scanning patient registration using the commercially available Fazer (Medtronic, Inc.) with the conventional registration procedure based on fiducial markers (FMs) in computer-assisted surgery.

Four anatomical head specimens were prepared with 10 titanium microscrews placed at defined locations and scanned with a 16-slice spiral computed tomography unit. To compare the two registration methods, each method was applied five times for each cadaveric specimen; thus data were obtained from 40 registrations. Five microscrews (selected following a randomization protocol) were used for each FM-based registration; the other five FMs were selected for coordinate measurements by touching with a point measurement stylus. Coordinates of these points were also measured manually on the screen of the navigation computer. Coordinates were measured in the same manner after laser surface registration.

The root mean square error as calculated by the navigation system ranged from 1.3 to 3.2 mm (mean 1.8 mm) with the Fazer and from 0.3 to 1.8 mm (mean 1.0 mm) with FM-based registration. The overall mean deviations (the arithmetic mean of the mean deviations of measurements on the four specimens) were 3.0 mm (standard deviation [SD] range 1.4–2.6 mm) with the Fazer and 1.4 mm (SD range 0.4–0.9 mm) with the FMs. The Fazer registration scans 300 surface points. Statistical tests showed the difference in the accuracy of these methods to be highly significant.

In accordance with the findings of other groups, the authors concluded that the inclusion of a larger number of registration points might improve the accuracy of Fazer registration.

A prototype biosensor-integrated image-guided surgery system

BACKGROUND

In this study we investigated the integration of a Raman spectroscopy-based biosensor with an image-guided surgery system. Such a system would provide a surgeon with both a diagnosis of the tissue being analysed (e.g. cancer) and localization information displayed within an imaging modality of choice. This type of mutual and registered information could lead to faster diagnoses and enable more accurate tissue resections.

METHODS

A test bed consisting of a portable Raman probe attached to a passively articulated mechanical arm was used to scan and classify objects within a phantom skull.

RESULTS

The prototype system was successfully able to track the Raman probe, classify objects within the phantom skull, and display the classifications on medical imaging data within a virtual reality environment.

CONCLUSION

We discuss the implementation of the integrated system, its accuracy and improvements to the system that will enhance its usefulness and further the field of sensor-based computer-assisted surgery.

Image-guidance for surgical procedures

Contemporary imaging modalities can now provide the surgeon with high quality three- and four-dimensional images depicting not only normal anatomy and pathology, but also vascularity and function. A key component of image-guided surgery (IGS) is the ability to register multi-modal pre-operative images to each other and to the patient. The other important component of IGS is the ability to track instruments in real time during the procedure and to display them as part of a realistic model of the operative volume. Stereoscopic, virtual- and augmented-reality techniques have been implemented to enhance the visualization and guidance process. For the most part, IGS relies on the assumption that the pre-operatively acquired images used to guide the surgery accurately represent the morphology of the tissue during the procedure. This assumption may not necessarily be valid, and so intra-operative real-time imaging using interventional MRI, ultrasound, video and electrophysiological recordings are often employed to ameliorate this situation. Although IGS is now in extensive routine clinical use in neurosurgery and is gaining ground in other surgical disciplines, there remain many drawbacks that must be overcome before it can be employed in more general minimally-invasive procedures. This review overviews the roots of IGS in neurosurgery, provides examples of its use outside the brain, discusses the infrastructure required for successful implementation of IGS approaches and outlines the challenges that must be overcome for IGS to advance further.

Application of soft tissue modelling to image-guided surgery

The deformation of soft tissue compromises the accuracy of image-guided surgery based on preoperative images, and restricts its applicability to surgery on or near bony structures. One way to overcome these limitations is to combine biomechanical models with sparse intraoperative data, in order to realistically warp the preoperative image to match the surgical situation. We detail the process of biomechanical modelling in the context of image-guided surgery. We focus in particular on the finite element method, which is shown to be a promising approach, and review the constitutive relationships which have been suggested for representing tissue during surgery. Appropriate intraoperative measurements are required to constrain the deformation, and we discuss the potential of the modalities which have been applied to this task. This technology is on the verge of transition into clinical practice, where it promises to increase the guidance accuracy and facilitate less invasive interventions. We describe here how soft tissue modelling techniques have been applied to image-guided surgery applications.

Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements

Image-guided liver surgery requires the ability to identify and compensate for soft tissue deformation in the organ. The predeformed state is represented as a complete three-dimensional surface of the organ, while the intraoperative data is a range scan point cloud acquired from the exposed liver surface. The first step is to rigidly align the coordinate systems of the intraoperative and preoperative data. Most traditional rigid registration methods minimize an error metric over the entire data set. In this paper, a new deformation-identifying rigid registration (DIRR) is reported that identifies and aligns minimally deformed regions of the data using a modified closest point distance cost function. Once a rigid alignment has been established, deformation is accounted for using a linearly elastic finite element model (FEM) and implemented using an incremental framework to resolve geometric nonlinearities. Boundary conditions for the incremental formulation are generated from intraoperatively acquired range scan surfaces of the exposed liver surface. A series of phantom experiments is presented to assess the fidelity of the DIRR and the combined DIRR/FEM approaches separately. The DIRR approach identified deforming regions in 90% of cases under conditions of realistic surgical exposure. With respect to the DIRR/FEM algorithm, subsurface target errors were correctly located to within 4 mm in phantom experiments.

A method to track cortical surface deformations using a laser range scanner

This paper reports a novel method to track brain shift using a laser-range scanner (LRS) and nonrigid registration techniques. The LRS used in this paper is capable of generating textured point-clouds describing the surface geometry/intensity pattern of the brain as presented during cranial surgery. Using serial LRS acquisitions of the brain's surface and two-dimensional (2-D) nonrigid image registration, we developed a method to track surface motion during neurosurgical procedures. A series of experiments devised to evaluate the performance of the developed shift-tracking protocol are reported. In a controlled, quantitative phantom experiment, the results demonstrate that the surface shift-tracking protocol is capable of resolving shift to an accuracy of approximately 1.6 mm given initial shifts on the order of 15 mm. Furthermore, in a preliminary in vivo case using the tracked LRS and an independent optical measurement system, the automatic protocol was able to reconstruct 50% of the brain shift with an accuracy of 3.7 mm while the manual measurement was able to reconstruct 77% with an accuracy of 2.1 mm. The results suggest that a LRS is an effective tool for tracking brain surface shift during neurosurgery.