Medical navigation system for otologic surgery based on hybrid registration and virtual intraoperative computed tomography

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks

PURPOSE

Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery.

METHODS

CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances.

RESULTS

The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of 0.65 ± 0.60 mm for the incus and 0.48 ± 0.32 mm for the round window. The average Hausdorff distance for these 2 targets was 0.98 ± 0.60 mm and 0.78 ± 0.34 mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18 s.

CONCLUSIONS

A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.

Image-to-Patient Registration in Computer-Assisted Surgery of Head and Neck: State-of-the-Art, Perspectives, and Challenges

Today, image-guided systems play a significant role in improving the outcome of diagnostic and therapeutic interventions. They provide crucial anatomical information during the procedure to decrease the size and the extent of the approach, to reduce intraoperative complications, and to increase accuracy, repeatability, and safety. Image-to-patient registration is the first step in image-guided procedures. It establishes a correspondence between the patient's preoperative imaging and the intraoperative data. When it comes to the head-and-neck region, the presence of many sensitive structures such as the central nervous system or the neurosensory organs requires a millimetric precision. This review allows evaluating the characteristics and the performances of different registration methods in the head-and-neck region used in the operation room from the perspectives of accuracy, invasiveness, and processing times. Our work led to the conclusion that invasive marker-based methods are still considered as the gold standard of image-to-patient registration. The surface-based methods are recommended for faster procedures and applied on the surface tissues especially around the eyes. In the near future, computer vision technology is expected to enhance these systems by reducing human errors and cognitive load in the operating room.

Image-guided surgery and novel intraoperative devices for enhanced visualisation in general and paediatric surgery: a review

Fluorescence guided surgery, augmented reality, and intra-operative imaging devices are rapidly pervading the field of surgical interventions, equipping the surgeon with powerful tools capable of enhancing the surgical visualisation of anatomical normal and pathological structures. There is a wide range of possibilities in the adult population to use these novel technologies and devices in the guidance for surgical procedures and minimally invasive surgeries. Their applications and their use have also been increasingly growing in the field of paediatric surgery, where the detailed visualisation of small anatomical structures could reduce procedure time, minimising surgical complications and ultimately improve the outcome of surgery. This review aims to illustrate the mechanisms underlying these innovations and their main applications in the clinical setting.

Freehand Stereotactic Image-Guidance Tailored to Neurotologic Surgery

Hypothesis: The use of freehand stereotactic image-guidance with a target registration error (TRE) of μ_TRE + 3σ_TRE < 0.5 mm for navigating surgical instruments during neurotologic surgery is safe and useful. Background: Neurotologic microsurgery requires work at the limits of human visual and tactile capabilities. Anatomy localization comes at the expense of invasiveness caused by exposing structures and using them as orientation landmarks. In the absence of more-precise and less-invasive anatomy localization alternatives, surgery poses considerable risks of iatrogenic injury and sub-optimal treatment. There exists an unmet clinical need for an accurate, precise, and minimally-invasive means for anatomy localization and instrument navigation during neurotologic surgery. Freehand stereotactic image-guidance constitutes a solution to this. While the technology is routinely used in medical fields such as neurosurgery and rhinology, to date, it is not used for neurotologic surgery due to insufficient accuracy of clinically available systems. Materials and Methods: A freehand stereotactic image-guidance system tailored to the needs of neurotologic surgery-most importantly sub-half-millimeter accuracy-was developed. Its TRE was assessed preclinically using a task-specific phantom. A pilot clinical trial targeting N = 20 study participants was conducted (ClinicalTrials.gov ID: NCT03852329) to validate the accuracy and usefulness of the developed system. Clinically, objective assessment of the TRE is impossible because establishing a sufficiently accurate ground-truth is impossible. A method was used to validate accuracy and usefulness based on intersubjectivity assessment of surgeon ratings of corresponding image-pairs from the microscope/endoscope and the image-guidance system. Results: During the preclinical accuracy assessment the TRE was measured as 0.120 ± 0.05 mm (max: 0.27 mm, μ_TRE + 3σ_TRE = 0.27 mm, N = 310). Due to the COVID-19 pandemic, the study was terminated early after N = 3 participants. During an endoscopic cholesteatoma removal, a microscopic facial nerve schwannoma removal, and a microscopic revision cochlear implantation, N = 75 accuracy and usefulness ratings were collected from five surgeons each grading 15 image-pairs. On a scale from 1 (worst rating) to 5 (best rating), the median (interquartile range) accuracy and usefulness ratings were assessed as 5 (4-5) and 4 (4-5) respectively. Conclusion: Navigating surgery in the tympanomastoid compartment and potentially in the lateral skull base with sufficiently accurate freehand stereotactic image-guidance (μ_TRE + 3σ_TRE < 0.5 mm) is feasible, safe, and useful. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT03852329.

Automatically Addressing System for Ultrasound-Guided Renal Biopsy Training Based on Augmented Reality

Chronic kidney disease has become one of the diseases with the highest morbidity and mortality in kidney diseases, and there are still some problems in surgery. During the operation, the surgeon can only operate on two-dimensional ultrasound images and cannot determine the spatial position relationship between the lesion and the medical puncture needle in real-time. The average number of punctures per patient will reach 3 to 4, Increasing the incidence of complications after a puncture. This article starts with ultrasound-guided renal biopsy navigation training, optimizes puncture path planning, and puncture training assistance. The augmented reality technology, combined with renal puncture surgery training was studied. This paper develops a prototype ultrasound-guided renal biopsy surgery training system, which improves the accuracy and reliability of the system training. The system is compared with the VR training system. The results show that the augmented reality training platform is more suitable as a surgical training platform. Because it takes a short time and has a good training effect.

Evolution and Stagnation of Image Guidance for Surgery in the Lateral Skull: A Systematic Review 1989-2020

Objective: Despite three decades of pre-clinical and clinical research into image guidance solutions as a more accurate and less invasive alternative for instrument and anatomy localization, translation into routine clinical practice for surgery in the lateral skull has not yet happened. The aim of this review is to identify challenges that need to be solved in order to provide image guidance solutions that are safe and beneficial for use during lateral skull surgery and to synthesize factors that facilitate the development of such solutions. Methods: Literature search was conducted via PubMed using terms relating to image guidance and the lateral skull. Data extraction included the following variables: image guidance error, imaging resolution, image guidance system, tracking technology, registration method, study endpoints, clinical target application, and publication year. A subsequent search of FDA 510(k) database for identified image guidance systems and extraction of the year of approval, intended use, and indications for use was performed. The study objectives and endpoints were subdivided in three time phases and summarized. Furthermore, it was analyzed which factors correlated with the image guidance error. Factor values for which an error ≤0.5 mm (μ_error + 3σ_error) was measured in more than one study were identified and inspected for time trends. Results: A descriptive statistics-based summary of study objectives and findings separated in three time intervals is provided. The literature provides qualitative and quantitative evidence that image guidance systems must provide an accuracy ≤0.5 mm (μ_error + 3σ_error) for their safe and beneficial application during surgery in the lateral skull. Spatial tracking accuracy and precision and medical image resolution both correlate with the image guidance accuracy, and all of them improved over the years. Tracking technology with accuracy ≤0.05 mm, computed tomography imaging with slice thickness ≤0.2 mm, and registration based on bone-anchored titanium fiducials are components that provide a sufficient setting for the development of sufficiently accurate image guidance. Conclusion: Image guidance systems must reliably provide an accuracy ≤0.5 mm (μ_error + 3σ_error) for their safe and beneficial use during surgery in the lateral skull. Advances in tracking and imaging technology contribute to the improvement of accuracy, eventually enabling the development and wide-scale adoption of image guidance solutions that can be used safely and beneficially during lateral skull surgery.

Augmented reality for image guidance in transoral robotic surgery

With the advent of precision surgery, there have been attempts to integrate imaging with robotic systems to guide sound oncologic surgical resections while preserving critical structures. In the confined space of transoral robotic surgery (TORS), this offers great potential given the proximity of structures. In this cadaveric experiment, we describe the use of a 3D virtual model displayed in the surgeon's console with the surgical field in view, to facilitate image-guided surgery at the oropharynx where there is significant soft tissue deformation. We also utilized the 3D model that was registered to the maxillary dentition, allowing for real-time image overlay of the internal carotid artery system. This allowed for real-time visualization of the internal carotid artery system that was qualitatively accurate on cadaveric dissection. Overall, this shows that virtual models and image overlays can be useful in image-guided surgery while approaching different sites in head and neck surgery with TORS.

An Accurate Recognition of Infrared Retro-Reflective Markers in Surgical Navigation

Marker-based optical tracking systems (OTS) are widely used in clinical image-guided therapy. However, the emergence of ghost markers, which is caused by the mistaken recognition of markers and the incorrect correspondences between marker projections, may lead to tracking failures for these systems. Therefore, this paper proposes a strategy to prevent the emergence of ghost markers by identifying markers based on the features of their projections, finding the correspondences between marker projections based on the geometric information provided by markers, and fast-tracking markers in a 2D image between frames based on the sizes of their projections. Apart from validating its high robustness, the experimental results show that the proposed strategy can accurately recognize markers, correctly identify their correspondences, and meet the requirements of real-time tracking.

Novel Thoracoscopic Navigation System With Augmented Real-Time Image Guidance for Chest Wall Tumors

BACKGROUND

We developed a thoracoscopic surgical navigation system with real-time augmented image guidance to assess the potential benefits for minimally invasive resection of chest wall tumors. The accuracy of localization of tumor and resection margin and the effect on task workload and confidence were evaluated in a chest wall tumor phantom.

METHODS

After scanning a realistic tumor phantom by cone-beam computed tomography and registering the data into the system, three-dimensional contoured tumor and resection margin was displayed. Fifteen surgeons were asked to localize the tumor margin and surgical margins with the thoracoscope alone. The same procedure was performed with the surgical navigation system activated, and results were compared between each attempt. A questionnaire and National Aeronautics and Space Administration Task Load Index were completed after.

RESULTS

The surgical navigation system significantly reduced localization error for the medial (p = 0.002) and superior tumor margin (p < 0.001), which was difficult to visualize by thoracoscopy alone. All surgical resection margins were improved circumferentially, including margins that were readily visible by thoracoscopy. National Aeronautics and Space Administration Task Load Index response scores showed a statistically significant reduction in workload in all subscales. There was a more than 50% mean reduction in workload for performance (10.1 vs 4.4, p = 0.001) and frustration (13.0 vs 5.4, p = 0.001).

CONCLUSIONS

This study showed that the thoracoscopic surgical navigation system providing augmented image guidance decreased tumor localization error for regions difficult to visualize thoracoscopically and also reduced surgical margin error circumferentially, regardless of thoracoscopic visibility. This system also reduced workload and increased surgeon's confidence in localizing challenging chest wall tumors.

Application of computer-assisted navigation systems in oral and maxillofacial surgery

The oral and maxillofacial region has a complicated anatomy with critical contiguous organs, including the brain, eyes, vital teeth, and complex networks of nerves and blood vessels. Therefore, advances in basic scientific research within the field of intraoperative oral and maxillofacial surgery have enabled the introduction of the features of these techniques into routine clinical practice to ensure safe and reliable surgery. A navigation system provides a useful guide for safer and more accurate complex in oral and maxillofacial surgery. The effectiveness of a navigation system for oral and maxillofacial surgery has been indicated by clinical applications in maxillofacial trauma surgery including complex midfacial fractures and orbital trauma reconstruction, foreign body removal, complex dentoalveolar surgery, skull base surgery including surgery of the temporomandibular joint (TMJ), and orthognathic surgery. However, some fundamental issues remain involving the mobility of the mandible and difficulty in updating images intraoperatively. This report presents an overview and feasible applications of available navigation systems with a focus on the clinical feasibility of the application of navigation systems in the field of oral and maxillofacial surgery and solutions to current problems.

Navigation accuracy after automatic- and hybrid-surface registration in sinus and skull base surgery

Objective

Computer-aided-surgery in ENT surgery is mainly used for sinus surgery but navigation accuracy still reaches its limits for skull base procedures. Knowledge of navigation accuracy in distinct anatomical regions is therefore mandatory. This study examined whether navigation accuracy can be improved in specific anatomical localizations by using hybrid registration technique.

Study design

Experimental phantom study.

Setting

Operating room.

Subjects and methods

The gold standard of screw registration was compared with automatic LED-mask-registration alone, and in combination with additional surface matching. 3D-printer-based skull models with individual fabricated silicone skin were used for the experiments. Overall navigation accuracy considering 26 target fiducials distributed over each skull was measured as well as the accuracy on selected anatomic localizations.

Results

Overall navigation accuracy was <1.0 mm in all cases, showing the significantly lowest values after screw registration (0.66 ± 0.08 mm), followed by hybrid registration (0.83± 0.08 mm), and sole mask registration (0.92 ± 0.13 mm).On selected anatomic localizations screw registration was significantly superior on the sphenoid sinus and on the internal auditory canal. However, mask registration showed significantly better accuracy results on the midface. Navigation accuracy on skull base localizations could be significantly improved by the combination of mask registration and additional surface matching.

Conclusion

Overall navigation accuracy gives no sufficient information regarding navigation accuracy in a distinct anatomic area. The non-invasive LED-mask-registration proved to be an alternative in clinical routine showing best accuracy results on the midface. For challenging skull base procedures a hybrid registration technique is recommendable which improves navigation accuracy significantly in this operating field. Invasive registration procedures are reserved for selected challenging skull base operations where the required high precision warrants the invasiveness.

Development of a surgical navigation system based on 3D Slicer for intraoperative implant placement surgery

Implant placement has been widely used in various kinds of surgery. However, accurate intraoperative drilling performance is essential to avoid injury to adjacent structures. Although some commercially-available surgical navigation systems have been approved for clinical applications, these systems are expensive and the source code is not available to researchers. 3D Slicer is a free, open source software platform for the research community of computer-aided surgery. In this study, a loadable module based on Slicer has been developed and validated to support surgical navigation. This research module allows reliable calibration of the surgical drill, point-based registration and surface matching registration, so that the position and orientation of the surgical drill can be tracked and displayed on the computer screen in real time, aiming at reducing risks. In accuracy verification experiments, the mean target registration error (TRE) for point-based and surface-based registration were 0.31±0.06mm and 1.01±0.06mm respectively, which should meet clinical requirements. Both phantom and cadaver experiments demonstrated the feasibility of our surgical navigation software module.

Boolean Combinations of Implicit Functions for Model Clipping in Computer-Assisted Surgical Planning

This paper proposes an interactive method of model clipping for computer-assisted surgical planning. The model is separated by a data filter that is defined by the implicit function of the clipping path. Being interactive to surgeons, the clipping path that is composed of the plane widgets can be manually repositioned along the desirable presurgical path, which means that surgeons can produce any accurate shape of the clipped model. The implicit function is acquired through a recursive algorithm based on the Boolean combinations (including Boolean union and Boolean intersection) of a series of plane widgets' implicit functions. The algorithm is evaluated as highly efficient because the best time performance of the algorithm is linear, which applies to most of the cases in the computer-assisted surgical planning. Based on the above stated algorithm, a user-friendly module named SmartModelClip is developed on the basis of Slicer platform and VTK. A number of arbitrary clipping paths have been tested. Experimental results of presurgical planning for three types of Le Fort fractures and for tumor removal demonstrate the high reliability and efficiency of our recursive algorithm and robustness of the module.

New method for identifying abnormal milling states of an otological drill

Surgeons are continuing to strive toward achieving higher quality minimally invasive surgery. With the growth of modern technology, intelligent medical devices are being used to improve the safety of surgery. Milling beyond the bone tissue wall is a common abnormal milling state in ear surgery, as well as entanglement of the drill bit with the cotton swab, which will do harm to the patient's encephalic tissues. Various methods have been investigated by engineers and surgeons in an effort to avoid this type of abnormal milling state during surgery. This paper outlines a new method for identifying these two types of abnormal milling states. Five surgeons were invited to perform experiments on calvarial bones. The average recognition rate for otological drill milling through a bone tissue wall was 93%, with only 2% of normal millings being incorrectly identified as milling faults. The average recognition rate for entanglement of the drill bit with a cotton swab was 92%, with only 2% of normal millings being identified as milling faults. The method presented here can be adapted to the needs of the individual surgeon and reliably identify milling faults.

Accuracy assessment of a marker-free method for registration of CT and stereo images applied in image-guided implantology: a phantom study

To assess the accuracy of a proposed marker-free registration method as opposed to the conventional marker-based method using an image-guided dental system, and investigating the best configurations of anatomical landmarks for various surgical fields in a phantom study, a CT-compatible dental phantom consisting of implanted targets was used. Two marker-free registration methods were evaluated, first using dental anatomical landmarks and second, using a reference marker tool. Six implanted markers, distributed in the inner space of the phantom were used as the targets; the values of target registration error (TRE) for each target were measured and compared with the marker-based method. Then, the effects of different landmark configurations on TRE values, measured using the Parsiss IV Guided Navigation system (Parsiss, Tehran, Iran), were investigated to find the best landmark arrangement for reaching the minimum registration error in each target region. It was proved that marker-free registration can be as precise as the marker-based method. This has a great impact on image-guided implantology systems whereby the drawbacks of fiducial markers for patient and surgeon are removed. It was also shown that smaller values of TRE could be achieved by using appropriate landmark configurations and moving the center of the landmark set closer to the surgery target. Other common factors would not necessarily decrease the TRE value so the conventional rules accepted in the clinical community about the ways to reduce TRE should be adapted to the selected field of dental surgery.

A region-based anatomical landmark configuration for sinus surgery using image guided navigation system: a phantom-study

PURPOSE

To evaluate the current beliefs about the ways to reduce target registration error (TRE) values in image guided Sinus surgery by rearranging the fiducial configuration, and investigating the best configurations for various surgical fields in a phantom study.

METHODS

A new CT-compatible skull phantom consisting of implanted targets was designed to enable direct measurement of TRE in four fields of sinus surgery, Frontal, Ethmoid, Sphenoid and Maxillary. The effects of different landmark configurations on TRE values, measured by the Parsiss-IV navigation system were investigated to find the best landmark arrangement for each region, and compared to the TRE prediction formula to assess the clinically accepted landmark selection approaches based on this formula.

RESULTS

It was shown that smaller values of TRE could be attained by arranging the center of the fiducials to be more focused on the surgery target. The addition of more fiducials and keeping non-linear arrangement of landmark would not necessarily decrease the TRE value.

CONCLUSION

Optimizing the landmark configuration is important for increasing the localization accuracy in image guided sinus surgery. The common beliefs accepted in the clinical community about the ways to reduce the TRE are very general and should be adapted to specific field of image guided surgery.

A new nasopharyngeal dynamic reference frame improves accuracy in navigated skull base targets

OBJECTIVE

We questioned whether the position of the dynamic reference frame (DRF) influences the application accuracy in electromagnetically navigated cranial procedures. A carrier for an electromagnetic DRF was developed, which could be fixed at the posterior edge of the vomer near the center of the head. This nasopharyngeal DRF was compared with a standard DRF fixed to the surface of the forehead.

METHODS

Image coordinates and real-world coordinates were co-registered and the total target error (TTE) was measured in the frontal and the lateral skull base of formalin fixed human head. At each anatomical site, 10 targets served for TTE determinations and 5 different fiducial combinations were used for registration.

RESULTS

With the nasopharyngeal DRF, lower TTE values (2.8 ± 1.4 mm; mean ± SD) were observed when compared with the forehead DRF (3.7 ± 2.8 mm; P = .004). TTEs of both anatomical sites investigated were significantly lower when using the nasopharyngeal DRF (frontal skull base 3.4 vs 2.1 mm, P = .005 and lateral skull base 3.9 vs 3.5 mm, P = .013) than with the standard forehead mounted one.

CONCLUSION

Positioning the DRF in the center of the head significantly improved the application accuracy of targets in the skull base with electromagnetic navigation by 25%.

[Clinical experience with navigation functions for temporal bone surgery : interim result after 40 patients]

PROBLEM

Although robust algorithms for registration and segmentation are available, the majority of surgical approaches to the temporal bone are nowadays made without navigation assistance. Beside instrument navigation (IN), functions such as distance control (DC) and navigated control (NC) can be used. This study analyzes the application of these navigation functionalities in lateral skull base and middle ear surgery.

PATIENTS AND METHODS

A total of 41 patients with an indication of temporal bone approaches were included. The navigation was realized with an optoelectric navigation system with both non-invasive and invasive markers. Parameters such as surgical time, Level of Quality (LoQ) index, and Change of Surgical Strategy (COS) index were evaluated.

RESULTS

In 14.6% of patients, the conventional mode of IN was used. In 70.7% of cases, the function DC was also used. In another 14.6% of cases, the function NC was used to control the speed of the drill. The facial nerve was the dominant segmented risk structure for active navigation. The time for setup was on average 7.78 min. The LoQ index score was on average 66 points. In 17% of the patients, surgeons evaluated the assistance mode as "necessary for the surgery". No technical-related complications were recorded.

CONCLUSION

This study proves the usability of navigation technology for temporal bone surgery in clinical routine. DC and NC are two additional features for higher acceptance of navigation in microscopic surgery.

Compact tracking of surgical instruments through structured markers

Virtual and augmented reality surgery calls for reliable and efficient tracking of the surgical instruments in the virtual or real operating theatre. The most diffused approach uses three or more not aligned markers, attached to each instrument and surveyed by a set of cameras. However, the structure required to carry the markers does modify the instrument's mass distribution and can interfere with surgeon movements. To overcome these problems, we propose here a new methodology, based on structured markers, to compute the six degrees of freedom of a surgical instrument. Two markers are attached on the instrument axis and one of them has a stripe painted over its surface. We also introduce a procedure to compute with high accuracy the markers center on the cameras image, even when partially occluded by the instrument's axis or by other structures. Experimental results demonstrate the reliability and accuracy of the proposed approach. The introduction of structured passive markers can open new possibilities to accurate tracking, combining markers detection with real-time image processing.

Navigation system with real-time finite element analysis for minimally invasive surgery

This paper presents a navigation system for minimally invasive surgery, especially laparoscopic surgery in which operates in abdomen. Conventional navigation systems show virtual images by superimposing models of target tissues on real endoscopic images. Since soft tissues within the abdomen are deformed during the surgery, the navigation system needs to provide surgeons reliable information by deforming the models according to their biomechanical behavior. However, conventional navigation systems don't consider the tissue deformation during the surgery. We have been developing a new real-time FEM-based simulation for deforming a soft tissue model by using neural network[1]. The network is called the neuroFEM. The incorporation of the neuroFEM into the navigation leads to improve the accuracy of the navigation system. In this paper, we propose a new navigation system with a framework of the neuroFEM.

Warning navigation system using real-time safe region monitoring for otologic surgery

PURPOSE

We developed a surgical navigation system that warns the surgeon with auditory and visual feedback to protect the facial nerve with real-time monitoring of the safe region during drilling.

METHODS

Warning navigation modules were developed and integrated into a free open source software platform. To obtain high registration accuracy, we used a high-precision laser-sintered template of the patient's bone surface to register the computed tomography (CT) images. We calculated the closest distance between the drill tip and the surface of the facial nerve during drilling. When the drill tip entered the safe regions, the navigation system provided an auditory and visual signal which differed in each safe region. To evaluate the effectiveness of the system, we performed phantom experiments for maintaining a given safe margin from the facial nerve when drilling bone models, with and without the navigation system. The error of the safe margin was measured on postoperative CT images. In real surgery, we evaluated the feasibility of the system in comparison with conventional facial nerve monitoring.

RESULTS

The navigation accuracy was submillimeter for the target registration error. In the phantom study, the task with navigation ([Formula: see text] mm) was more successful with smaller error, than the task without navigation ([Formula: see text] mm, [Formula: see text]). The clinical feasibility of the system was confirmed in three real surgeries.

CONCLUSIONS

This system could assist surgeons in preserving the facial nerve and potentially contribute to enhanced patient safety in the surgery.

Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device

PURPOSE

In endoscopic surgery, limited views and lack of tactile sensation restrict the surgeon's abilities and cause stress to the surgeon. Therefore, an intra-operative navigation system is strongly recommended. We developed an augmented reality (AR) navigation system based on preoperative CT imaging. The purpose of this study is to evaluate the usefulness, feasibility, and accuracy of this system using laparoscopic splenectomy in children.

METHODS

Volume images were reconstructed by three-dimensional (3D) viewer application. We used an optical tracking system for registration between volume image and body surface markers. The AR visualization was superimposed preoperative 3D CT images onto captured laparoscopic live images. This system was applied to six cases of laparoscopic splenectomy in children. To evaluate registration accuracy, distances from the marker position to the volume data were calculated.

RESULTS

The operator recognized the hidden vascular variation of the splenic artery and vein, accessory spleen, and pancreatic tail by overlaying an image onto a laparoscopic live image. The registration accuracy of six cases was 5.30 ± 0.08, 5.71 ± 1.70, 10.1 ± 0.60, 18.8 ± 3.56, 4.06 ± 1.71, and 7.05 ± 4.71.

CONCLUSION

This navigation system provides real-time anatomical information, which cannot be otherwise visualized without navigation. The registration accuracy was acceptable in clinical operation.

Fast-MICP for frameless image-guided surgery

PURPOSE

In image-guided surgery (IGS) systems, image-to-physical registration is critical for reliable anatomical information mapping and spatial guidance. Conventional stereotactic frame-based or fiducial-based approaches provide accurate registration but are not patient-friendly. This study proposes a frameless cranial IGS system that uses computer vision techniques to replace the frame or fiducials with the natural features of the patient.

METHODS

To perform a cranial surgery with the proposed system, the facial surface of the patient is first reconstructed by stereo vision. Accuracy is ensured by capturing parallel-line patterns projected from a calibrated LCD projector. Meanwhile, another facial surface is reconstructed from preoperative computed tomography (CT) images of the patient. The proposed iterative closest point (ICP)-based algorithm [fast marker-added ICP (Fast-MICP)] is then used to register the two facial data sets, which transfers the anatomical information from the CT images to the physical space.

RESULTS

Experimental results reveal that the Fast-MICP algorithm reduces the computational cost of marker-added ICP (J.-D. Lee et al., "A coarse-to-fine surface registration algorithm for frameless brain surgery," in Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society, 2007, pp. 836-839) to 10% and achieves comparable registration accuracy, which is under 3 mm target registration error (TRE). Moreover, two types of optical-based spatial digitizing devices can be integrated for further surgical navigation. Anatomical information or image-guided surgical landmarks can be projected onto the patient to obtain an immersive augmented reality environment.

CONCLUSION

The proposed frameless IGS system with stereo vision obtains TRE of less than 3 mm. The proposed Fast-MICP registration algorithm reduces registration time by 90% without compromising accuracy.

An effective point-based registration tool for surgical navigation

BACKGROUND

Surgical navigation assists in endoscopic surgeries by enabling surgeons to see concealed lesions and surrounding organs. Successful surgical navigation depends on accurate registration between a medical image and a patient. For accurate point-based registration, it is important to determine the matching order and positions of the markers correctly. It is particularly difficult to determine the order and positions when part of the markers cannot be located on the patient's body or when they cannot be identified in the images.

METHODS

By using the automatic marker-matching option of the proposed tool, an optimum registration result can be obtained even with the partial loss of markers. In addition, this tool provides an intuitive marker selection interface that displays the registration error of each marker pair in different colors.

RESULTS

The efficiency of the described tool in terms of the registration accuracy and time has been confirmed in more than 70 clinical applications. The fiducial registration errors were 1.28 + or - 1.09 mm in ear, nose, and throat surgery and 3.55 + or - 1.30 mm in liver tumor ablation therapy.

CONCLUSIONS

The proposed automatic matching scheme with marker selection interface was particularly effective where the markers were partly lost or incorrectly identified.