High-fidelity haptic and visual rendering for patient-specific simulation of temporal bone surgery

A Narrative Review on 3-Dimensional Visualization Techniques in Neurosurgical Education, Simulation, and Planning

BACKGROUND

High-fidelity visualization of anatomical organs is crucial for neurosurgical education, simulation, and planning. This becomes much more important for minimally invasive neurosurgical procedures. Realistic anatomical visualization can allow resident surgeons to learn visual cues and orient themselves with the complex 3-dimensional (3D) anatomy. Achieving full fidelity in 3D medical visualization is an active area of research; however, the prior reviews focus on the application area and lack the underlying technical principles. Accordingly, the present study attempts to bridge this gap by providing a narrative review of the techniques used for 3D visualization.

METHODS

We conducted a literature review on 3D medical visualization technology from 2018 to 2023 using the PubMed and Google Scholar search engines. The cross-referenced manuscripts were extensively studied to find literature that discusses technology relevant to 3D medical visualization. We also compiled and ran software applications that were accessible to us in order to better understand them.

RESULTS

We present the underlying fundamental technology used in 3D medical visualization in the context of neurosurgical education, simulation, and planning. Further, we discuss and categorize a few important applications based on the 3D visualization techniques they use.

CONCLUSIONS

The visualization of virtual human organs has not yet achieved a level of realism close to reality. This gap is largely due to the interdisciplinary nature of this research, population diversity, and validation complexities. With the advancements in computational resources and automation of 3D visualization pipelines, next-gen applications may offer enhanced medical 3D visualization fidelity.

Further Validity Evidence for Patient-Specific Virtual Reality Temporal Bone Surgical Simulation

OBJECTIVE

Patient-specific virtual reality (VR) simulation of cochlear implant (CI) surgery potentially enables preoperative rehearsal and planning. We aim to gather supporting validity evidence for patient-specific simulation through the analysis of virtual performance and comparison with postoperative imaging.

METHODS

Prospective, multi-institutional study. Pre- and postoperative cone-beam CT scans of CI surgical patients were obtained and processed for patient-specific VR simulation. The virtual performances of five trainees and four attendings were recorded and (1) compared with volumes removed during actual surgery as determined in postoperative imaging, and (2) assessed using the Copenhagen Cochlear Implant Surgery Assessment Tool (CISAT) by two blinded raters. The volumes compared were cortical mastoidectomy, facial recess, and round window (RW) cochleostomy as well as violation of the facial nerve and chorda.

RESULTS

Trainees drilled more volume in the cortical mastoidectomy and facial recess, whereas attendings drilled more volume for the RW cochleostomy and made more violations. Except for the cochleostomy, attendings removed volumes closer to that determined in postoperative imaging. Trainees achieved a higher CISAT performance score compared with attendings (22.0 vs. 18.4 points) most likely due to lack of certain visual cues.

CONCLUSION

We found that there were differences in performance of trainees and attendings in patient-specific VR simulation of CI surgery as assessed by raters and in comparison with actual drilled volumes. The presented approach of volume comparison is novel and might be used for further validation of patient-specific VR simulation before clinical implementation for preoperative rehearsal in temporal bone surgery.

LEVEL OF EVIDENCE

n/a Laryngoscope, 134:1403-1409, 2024.

Fully immersive virtual reality for skull-base surgery: surgical training and beyond

PURPOSE

A virtual reality (VR) system, where surgeons can practice procedures on virtual anatomies, is a scalable and cost-effective alternative to cadaveric training. The fully digitized virtual surgeries can also be used to assess the surgeon's skills using measurements that are otherwise hard to collect in reality. Thus, we present the Fully Immersive Virtual Reality System (FIVRS) for skull-base surgery, which combines surgical simulation software with a high-fidelity hardware setup.

METHODS

FIVRS allows surgeons to follow normal clinical workflows inside the VR environment. FIVRS uses advanced rendering designs and drilling algorithms for realistic bone ablation. A head-mounted display with ergonomics similar to that of surgical microscopes is used to improve immersiveness. Extensive multi-modal data are recorded for post-analysis, including eye gaze, motion, force, and video of the surgery. A user-friendly interface is also designed to ease the learning curve of using FIVRS.

RESULTS

We present results from a user study involving surgeons with various levels of expertise. The preliminary data recorded by FIVRS differentiate between participants with different levels of expertise, promising future research on automatic skill assessment. Furthermore, informal feedback from the study participants about the system's intuitiveness and immersiveness was positive.

CONCLUSION

We present FIVRS, a fully immersive VR system for skull-base surgery. FIVRS features a realistic software simulation coupled with modern hardware for improved realism. The system is completely open source and provides feature-rich data in an industry-standard format.

Virtual Reality Simulation for the Middle Cranial Fossa Approach: A Validation Study

BACKGROUND AND OBJECTIVES

Virtual reality (VR) surgical rehearsal is an educational tool that exists in a safe environment. Validation is necessary to establish the educational value of this platform. The middle cranial fossa (MCF) is ideal for simulation because trainees have limited exposure to this approach and it has considerable complication risk. Our objectives were to assess the face, content, and construct validities of an MCF VR simulation, as well as the change in performance across serial simulations.

METHODS

Using high-resolution volumetric data sets of human cadavers, the authors generated a high-fidelity visual and haptic rendering of the MCF approach using CardinalSim software. Trainees from Neurosurgery and Otolaryngology-Head and Neck Surgery at two Canadian academic centers performed MCF dissections on this VR platform. Randomization was used to assess the effect of enhanced VR interaction. Likert scales were used to assess the face and content validities. Performance metrics and pre- and postsimulation test scores were evaluated. Construct validity was evaluated by examining the effect of the training level on simulation performance.

RESULTS

Twenty trainees were enrolled. Face and content validities were achieved in all domains. Construct validity, however, was not demonstrated. Postsimulation test scores were significantly higher than presimulation test scores ( P < .001 ). Trainees demonstrated statistically significant improvement in the time to complete dissections ( P < .001 ), internal auditory canal skeletonization ( P < .001 ), completeness of the anterior petrosectomy ( P < .001 ), and reduced number of injuries to critical structures ( P = .001 ).

CONCLUSION

This MCF VR simulation created using CardinalSim demonstrated face and content validities. Construct validity was not established because no trainee included in the study had previous MCF approach experience, which further emphasizes the importance of simulation. When used as a formative educational adjunct in both Neurosurgery and Otolaryngology-Head and Neck Surgery, this simulation has the potential to enhance understanding of the complex anatomic relationships of critical neurovascular structures.

Development of a virtual reality-based zygomatic implant surgery training system with global collision detection and optimized finite element method model

BACKGROUND AND OBJECTIVE

Zygomatic implant surgery is challenging due to the complex structure of the zygomatic bone, limited visual range during surgery, and lengthy implant path. Moreover, traditional training methods are costly, and experimental subjects are scarce.

METHODS

To overcome these challenges, we propose a novel training system that integrates visual, haptic, and auditory feedback to create a more immersive surgical experience. The system uses dynamic bounding volume hierarchy (BVH) and Symplectic Euler to detect global collisions between surgical tools and models, while an optimized finite element method (FEM) model simulates soft tissue and detects collisions. Compared to previous works, our system achieves global rigid-body collisions between surgical tools and patient models, while also providing stable and realistic simulation and collisions of soft tissues. This advancement offers a more realistic simulation for zygomatic implant surgery.

RESULTS

We conducted three experiments and evaluations. The first experiment measured the axial force generated during the zygomatic implant simulation process and compared it with actual surgery, demonstrating the realistic force rendering feedback of our system. The second evaluation involved 15 novice surgeons who experienced the system and completed a questionnaire survey focusing on five aspects. The results showed satisfactory evaluations. The third experiment involved six surgeons who underwent in-depth training for two hours daily and were tested on the first, third, and fifth days. We collected data and combined it with the doctors' feedback to prove that our system can improve surgeons' proficiency in zygomatic implant surgery and provide a novel training solution for this procedure.

CONCLUSION

We have innovatively integrated global collision detection and optimized soft tissue simulation into our system. Furthermore, we have conducted experimental validation to demonstrate the effectiveness of this implementation. We present a novel solution for zygomatic implant surgery training.

The user experience design of a novel microscope within SurgiSim, a virtual reality surgical simulator

PURPOSE

Virtual reality (VR) simulation has the potential to advance surgical education, procedural planning, and intraoperative guidance. "SurgiSim" is a VR platform developed for the rehearsal of complex procedures using patient-specific anatomy, high-fidelity stereoscopic graphics, and haptic feedback. SurgiSim is the first VR simulator to include a virtual operating room microscope. We describe the process of designing and refining the VR microscope user experience (UX) and user interaction (UI) to optimize surgical rehearsal and education.

METHODS

Human-centered VR design principles were applied in the design of the SurgiSim microscope to optimize the user's sense of presence. Throughout the UX's development, the team of developers met regularly with surgeons to gather end-user feedback. Supplemental testing was performed on four participants.

RESULTS

Through observation and participant feedback, we made iterative design upgrades to the SurgiSim platform. We identified the following key characteristics of the VR microscope UI: overall appearance, hand controller interface, and microscope movement.

CONCLUSION

Our design process identified challenges arising from the disparity between VR and physical environments that pertain to microscope education and deployment. These roadblocks were addressed using creative solutions. Future studies will investigate the efficacy of VR surgical microscope training on real-world microscope skills as assessed by validated performance metrics.

Virtual reality and haptic interfaces for civilian and military open trauma surgery training: A systematic review

OBJECTIVE

Virtual (VR), augmented (AR), mixed reality (MR) and haptic interfaces make additional avenues available for surgeon assessment, guidance and training. We evaluated applications for open trauma and emergency surgery to address the question: Have new computer-supported interface developments occurred that could improve trauma training for civilian and military surgeons performing open, emergency, non-laparoscopic surgery?

DESIGN

Systematic literature review.

SETTING AND PARTICIPANTS

Faculty, University of Maryland School of Medicine, Baltimore., Maryland; Womack Army Medical Center, Fort Bragg, North Carolina; Temple University, Philadelphia, Pennsylvania; Uniformed Services University of Health Sciences, and Walter Reed National Military Medical Center, Bethesda, Maryland.

METHODS

Structured literature searches identified studies using terms for virtual, augmented, mixed reality and haptics, as well as specific procedures in trauma training courses. Reporting bias was assessed. Study quality was evaluated by the Kirkpatrick's Level of evidence and the Machine Learning to Asses Surgical Expertise (MLASE) score.

RESULTS

Of 422 papers identified, 14 met inclusion criteria, included 282 enrolled subjects, 20% were surgeons, the remainder students, medics and non-surgeon physicians. Study design was poor and sample sizes were low. No data analyses were beyond descriptive and the highest outcome types were procedural success, subjective self-reports, except three studies used validated metrics. Among the 14 studies, Kirkpatrick's level of evidence was level zero in five studies, level 1 in 8 and level 2 in one. Only one study had MLASE Score greater than 9/20. There was a high risk of bias in 6 studies, uncertain bias in 5 studies and low risk of bias in 3 studies.

CONCLUSIONS

There was inadequate evidence that VR,MR,AR or haptic interfaces can facilitate training for open trauma surgery or replace cadavers. Because of limited testing in surgeons, deficient study and technology design, risk of reporting bias, no current well-designed studies of computer-supported technologies have shown benefit for open trauma, emergency surgery nor has their use shown improved patient outcomes. Larger more rigorously designed studies and evaluations by experienced surgeons are required for a greater variety of procedures and skills.

COMPETENCIES

Medical Knowledge, Practice Based Learning and Improvement, Patient Care, Systems-Based Practice.

The effect of virtual reality on temporal bone anatomy evaluation and performance

PURPOSE

There is only limited data on the application of virtual reality (VR) for the evaluation of temporal bone anatomy. The aim of the present study was to compare the VR environment to traditional cross-sectional viewing of computed tomography images in a simulated preoperative planning setting in novice and expert surgeons.

METHODS

A novice (n = 5) and an expert group (n = 5), based on their otosurgery experience, were created. The participants were asked to identify 24 anatomical landmarks, perform 11 distance measurements between surgically relevant anatomical structures and 10 fiducial markers on five cadaver temporal bones in both VR environment and cross-sectional viewings in PACS interface. The data on performance time and user-experience (i.e., subjective validation) were collected.

RESULTS

The novice group made significantly more errors (p < 0.001) and with significantly longer performance time (p = 0.001) in cross-sectional viewing than the expert group. In the VR environment, there was no significant differences (errors and time) between the groups. The performance of novices improved faster in the VR. The novices showed significantly faster task performance (p = 0.003) and a trend towards fewer errors (p = 0.054) in VR compared to cross-sectional viewing. No such difference between the methods were observed in the expert group. The mean overall scores of user-experience were significantly higher for VR than cross-sectional viewing in both groups (p < 0.001).

CONCLUSION

In the VR environment, novices performed the anatomical evaluation of temporal bone faster and with fewer errors than in the traditional cross-sectional viewing, which supports its efficiency for the evaluation of complex anatomy.

Mixed reality in the reconstruction of orbital floor: An experimental and clinical evaluative study

Introduction:

Materials and Methods:

Results:

Discussion:

Augmented Reality for Retrosigmoid Craniotomy Planning

While medical imaging data have traditionally been viewed on two-dimensional (2D) displays, augmented reality (AR) allows physicians to project the medical imaging data on patient's bodies to locate important anatomy. We present a surgical AR application to plan the retrosigmoid craniotomy, a standard approach to access the posterior fossa and the internal auditory canal. As a simple and accurate alternative to surface landmarks and conventional surgical navigation systems, our AR application augments the surgeon's vision to guide the optimal location of cortical bone removal. In this work, two surgeons performed a retrosigmoid approach 14 times on eight cadaver heads. In each case, the surgeon manually aligned a computed tomography (CT)-derived virtual rendering of the sigmoid sinus on the real cadaveric heads using a see-through AR display, allowing the surgeon to plan and perform the craniotomy accordingly. Postprocedure CT scans were acquired to assess the accuracy of the retrosigmoid craniotomies with respect to their intended location relative to the dural sinuses. The two surgeons had a mean margin of davg = 0.6 ± 4.7 mm and davg = 3.7 ± 2.3 mm between the osteotomy border and the dural sinuses over all their cases, respectively, and only positive margins for 12 of the 14 cases. The intended surgical approach to the internal auditory canal was successfully achieved in all cases using the proposed method, and the relatively small and consistent margins suggest that our system has the potential to be a valuable tool to facilitate planning a variety of similar skull-base procedures.

Finite element simulation and integration of CEM43 °C and Arrhenius Models for ultrasonic-assisted skull bone grinding: A thermal dose model

The aim of the study was to develop a novel automated setup for bone grinding to limit the temperature to below 43 °C. The feasibility of using ultrasonic actuation during bone osteotomy was explored with different machining variables, such as rotational speed, feed rate and ultrasonic frequency, in terms of the criterion variable (i.e., temperature). A thermal dose model based on the CEM43 °C and the Arrhenius model was developed for the prediction of tissue damage during bone grinding. CEM43 °C is a normalizing method to convert the time-temperature relationship into an equivalent number of minutes at 43 °C. For every degree rise in temperature above 43 °C, the cell viability significantly increased. The temperature generated during bone grinding was measured with an infrared thermography technique. The increase in temperature above threshold levels of 43 °C and 47 °C may harm the bone tissues and cause thermogenesis and osteonecrosis, respectively. A finite-element simulation was conducted to visualise the spatial and temporal distribution of temperature on the bone surface after bone grinding. Furthermore, simulation results were used to measure the depth of thermogenesis and osteonecrosis at the grinding site. Evaluation of the optimised set of bone grinding process parameters was supported with analysis of variance at the 95% confidence level.

Fully automated preoperative segmentation of temporal bone structures from clinical CT scans

Middle- and inner-ear surgery is a vital treatment option in hearing loss, infections, and tumors of the lateral skull base. Segmentation of otologic structures from computed tomography (CT) has many potential applications for improving surgical planning but can be an arduous and time-consuming task. We propose an end-to-end solution for the automated segmentation of temporal bone CT using convolutional neural networks (CNN). Using 150 manually segmented CT scans, a comparison of 3 CNN models (AH-Net, U-Net, ResNet) was conducted to compare Dice coefficient, Hausdorff distance, and speed of segmentation of the inner ear, ossicles, facial nerve and sigmoid sinus. Using AH-Net, the Dice coefficient was 0.91 for the inner ear; 0.85 for the ossicles; 0.75 for the facial nerve; and 0.86 for the sigmoid sinus. The average Hausdorff distance was 0.25, 0.21, 0.24 and 0.45 mm, respectively. Blinded experts assessed the accuracy of both techniques, and there was no statistical difference between the ratings for the two methods (p = 0.93). Objective and subjective assessment confirm good correlation between automated segmentation of otologic structures and manual segmentation performed by a specialist. This end-to-end automated segmentation pipeline can help to advance the systematic application of augmented reality, simulation, and automation in otologic procedures.

PWD-3DNet: A Deep Learning-Based Fully-Automated Segmentation of Multiple Structures on Temporal Bone CT Scans

The temporal bone is a part of the lateral skull surface that contains organs responsible for hearing and balance. Mastering surgery of the temporal bone is challenging because of this complex and microscopic three-dimensional anatomy. Segmentation of intra-temporal anatomy based on computed tomography (CT) images is necessary for applications such as surgical training and rehearsal, amongst others. However, temporal bone segmentation is challenging due to the similar intensities and complicated anatomical relationships among critical structures, undetectable small structures on standard clinical CT, and the amount of time required for manual segmentation. This paper describes a single multi-class deep learning-based pipeline as the first fully automated algorithm for segmenting multiple temporal bone structures from CT volumes, including the sigmoid sinus, facial nerve, inner ear, malleus, incus, stapes, internal carotid artery and internal auditory canal. The proposed fully convolutional network, PWD-3DNet, is a patch-wise densely connected (PWD) three-dimensional (3D) network. The accuracy and speed of the proposed algorithm was shown to surpass current manual and semi-automated segmentation techniques. The experimental results yielded significantly high Dice similarity scores and low Hausdorff distances for all temporal bone structures with an average of 86% and 0.755 millimeter (mm), respectively. We illustrated that overlapping in the inference sub-volumes improves the segmentation performance. Moreover, we proposed augmentation layers by using samples with various transformations and image artefacts to increase the robustness of PWD-3DNet against image acquisition protocols, such as smoothing caused by soft tissue scanner settings and larger voxel sizes used for radiation reduction. The proposed algorithm was tested on low-resolution CTs acquired by another center with different scanner parameters than the ones used to create the algorithm and shows potential for application beyond the particular training data used in the study.

Virtual reality improves the accuracy of simulated preoperative planning in temporal bones: a feasibility and validation study

PURPOSE

Consumer-grade virtual reality (VR) has recently enabled various medical applications, but more evidence supporting their validity is needed. We investigated the accuracy of simulated surgical planning in a VR environment (VR) with temporal bones and compared it to conventional cross-sectional image viewing in picture archiving and communication system (PACS) interface.

METHODS

Five experienced otologic surgeons measured significant anatomic structures and fiducials on five fresh-frozen cadaveric temporal bones in VR and cross-sectional viewing. Primary image data were acquired by computed tomography. In total, 275 anatomical landmark measurements and 250 measurements of the distance between fiducials were obtained with both methods. Distance measurements between the fiducials were confirmed by physical measurement obtained by Vernier caliper. The experts evaluated the subjective validity of both methods on a 5-point Likert scale qualitative survey.

RESULTS

A strong correlation based on intraclass coefficient was found between the methods on both the anatomical (r > 0.900) and fiducial measurements (r > 0.916). Two-tailed paired t-test and Bland-Altman plots demonstrated high equivalences between the VR and cross-sectional viewing with mean differences of 1.9% (p = 0.396) and 0.472 mm (p = 0.065) for anatomical and fiducial measurements, respectively. Gross measurement errors due to the misidentification of fiducials occurred more frequently in the cross-sectional viewing. The mean face and content validity rating for VR were significantly better compared to cross-sectional viewing (total mean score 4.11 vs 3.39, p < 0.001).

CONCLUSION

Our study supports good accuracy and reliability of VR environment for simulated surgical planning in temporal bones compared to conventional cross-sectional visualization.

Robotics in Pediatric Otolaryngology-Head and Neck Surgery and Advanced Surgical Planning

Robotic surgery has been shown to be feasible and successful in several areas of pediatric head and neck surgery. However, adoption has been limited. Robotic surgery may be better integrated into practice with advanced preoperative surgical planning and the design of new robotic platforms with instrumentation specific for the application. With continued investigations, computer-aided surgical planning techniques including three-dimensional printing, virtual reality, multiobjective cost function for optimization of approach, mirror image overlay, and flexible robotic instruments may demonstrate value and utility over current practice.

Augmented reality in neurosurgical navigation: a survey

BACKGROUND

Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems.

METHODS

In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey.

RESULTS

Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks.

CONCLUSIONS

Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.

Assessment of a virtual reality temporal bone surgical simulator: a national face and content validity study

BACKGROUND

Trainees in Otolaryngology-Head and Neck Surgery must gain proficiency in a variety of challenging temporal bone surgical techniques. Traditional teaching has relied on the use of cadavers; however, this method is resource-intensive and does not allow for repeated practice. Virtual reality surgical training is a growing field that is increasingly being adopted in Otolaryngology. CardinalSim is a virtual reality temporal bone surgical simulator that offers a high-quality, inexpensive adjunct to traditional teaching methods. The objective of this study was to establish the face and content validity of CardinalSim through a national study.

METHODS

Otolaryngologists and resident trainees from across Canada were recruited to evaluate CardinalSim. Ethics approval and informed consent was obtained. A face and content validity questionnaire with questions categorized into 13 domains was distributed to participants following simulator use. Descriptive statistics were used to describe questionnaire results, and either Chi-square or Fishers exact tests were used to compare responses between junior residents, senior residents, and practising surgeons.

RESULTS

Sixty-two participants from thirteen different Otolaryngology-Head and Neck Surgery programs were included in the study (32 practicing surgeons; 30 resident trainees). Face validity was achieved for 5 out of 7 domains, while content validity was achieved for 5 out of 6 domains. Significant differences between groups (p-value of < 0.05) were found for one face validity domain (realistic ergonomics, p = 0.002) and two content validity domains (teaching drilling technique, p = 0.011 and overall teaching utility, p = 0.006). The assessment scores, global rating scores, and overall attitudes towards CardinalSim, were universally positive. Open-ended questions identified limitations of the simulator.

CONCLUSION

CardinalSim met acceptable criteria for face and content validity. This temporal bone virtual reality surgical simulation platform may enhance surgical training and be suitable for patient-specific surgical rehearsal for practicing Otolaryngologists.

Mastoid notch as a landmark for localization of the transverse-sigmoid sinus junction

Background

The top of the mastoid notch (TMN) is close to the transverse-sigmoid sinus junction. The spatial position relationship between the TMN and the key points (the anterosuperior and inferomedial points of the transverse-sigmoid sinus junction, ASTS and IMTS) can be used as a novel method to precisely locate the sinus junction during lateral skull base craniotomy.

Methods

Forty-three dried adult skull samples (21 from males and 22 from females) were included in the study. A rectangular coordinate system on the lateral surface of the skull was defined to assist the analysis. According to sex and skull side, the data were divided into 4 groups: male&left, male&right, female&left and female&right. The distances from the ASTS and IMTS to the TMN were evaluated on the X-axis and Y-axis, symbolized as ASTS&TMN_x, ASTS&TMN_y, IMTS&TMN_x and IMTS&TMN_y.

Results

Among the four groups, there was no significant difference in ASTS&TMN_x (p = 0.05) and ASTS&TMN_y (p = 0.3059), but there were significant differences in IMTS&TMN_x (p < 0.001) and IMTS&TMN_y (p = 0.01), and multiple comparisons indicated that there were significant differences between male&left and female&left both in IMTS&TMN_x (p = 0.0006) and in IMTS&TMN_y (p = 0.0081). In general, the ASTS was located 1.92 mm anterior to the TMN on the X-axis and 27.01 mm superior to the TMN on the Y-axis. For the male skulls, the IMTS was located 3.60 mm posterior to the TMN on the X-axis and 14.40 mm superior to the TMN on the Y-axis; for the female skulls, the IMTS was located 7.84 mm posterior to the TMN on the X-axis and 19.70 mm superior to the TMN on the Y-axis.

Conclusions

The TMN is a useful landmark for accurately locating the ASTS and IMTS.

Haptic Rendering of Diverse Tool-Tissue Contact Constraints During Dental Implantation Procedures

Motor skill learning of dental implantation surgery is difficult for novices because it involves fine manipulation of different dental tools to fulfill a strictly pre-defined procedure. Haptics-enabled virtual reality training systems provide a promising tool for surgical skill learning. In this paper, we introduce a haptic rendering algorithm for simulating diverse tool-tissue contact constraints during dental implantation. Motion forms of an implant tool can be summarized as the high degree of freedom (H-DoF) motion and the low degree of freedom (L-DoF) motion. During the H-DoF state, the tool can move freely on bone surface and in free space with 6 DoF. While during the L-DoF state, the motion degrees are restrained due to the constraints imposed by the implant bed. We propose a state switching framework to simplify the simulation workload by rendering the H-DoF motion state and the L-DoF motion state separately, and seamless switch between the two states by defining an implant criteria as the switching judgment. We also propose the virtual constraint method to render the L-DoF motion, which are different from ordinary drilling procedures as the tools should obey different axial constraint forms including sliding, drilling, screwing and perforating. The virtual constraint method shows efficiency and accuracy in adapting to different kinds of constraint forms, and consists of three core steps, including defining the movement axis, projecting the configuration difference, and deriving the movement control ratio. The H-DoF motion on bone surface and in free space is simulated through the previously proposed virtual coupling method. Experimental results illustrated that the proposed method could simulate the 16 different phases of the complete implant procedures of the Straumann® Bone Level(BL) Implants Φ4.8–L12 mm. According to the output force curve, different contact constraints could be rendered with steady and continuous output force during the operation procedures.

Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype

Mixed reality (MR), a technology which supplements the real world with virtual objects, is increasingly becoming available as a teaching tool in medical education. The Microsoft HoloLens device allows operators to experience MR using a head-mounted device without interfering with their physical reality, stimulating a realistic learning experience using virtual objects. This project aimed to develop a MR anatomy teaching application with HoloLens for exploring the anatomy of the temporal bone. The educational application was developed from a multidisciplinary collaboration between undergraduate and postgraduate students across several academic disciplines with Medtronic, a medical technology company. 3D anatomical models were built using ZBrush and Blender, while the HoloLens1 application was developed using Windows 10, Visual Studio 2017, Unity and Mixed Reality Toolkit (MRTK). Modules developed within the application included a basic HoloLens tutorial, a virtual temporal bone with surgical anatomy landmarks and free drilling of the temporal bone. The basic tutorial allows the operator to adapt to the MR environment prior to exploring the anatomical landmarks of the 3D temporal bone. The free drilling of the temporal bone using vertex displacement and texture stretching replicates a real-time bone drilling experience and allows the operator to explore the anatomical relationship between different otological structures.

The future of otology

BACKGROUND

The field of otology is increasingly at the forefront of innovation in science and medicine. The inner ear, one of the most challenging systems to study, has been rendered much more open to inquiry by recent developments in research methodology. Promising advances of potential clinical impact have occurred in recent years in biological fields such as auditory genetics, ototoxic chemoprevention and organ of Corti regeneration. The interface of the ear with digital technology to remediate hearing loss, or as a consumer device within an intelligent ecosystem of connected devices, is receiving enormous creative energy. Automation and artificial intelligence can enhance otological medical and surgical practice. Otology is poised to enter a new renaissance period, in which many previously untreatable ear diseases will yield to newly introduced therapies.

OBJECTIVE

This paper speculates on the direction otology will take in the coming decades.

CONCLUSION

Making predictions about the future of otology is a risky endeavour. If the predictions are found wanting, it will likely be because of unforeseen revolutionary methods.

Real-Time Needle Force Modeling for VR-Based Renal Biopsy Training with Respiratory Motion Using Direct Clinical Data

Realistic tool-tissue interactive modeling has been recognized as an essential requirement in the training of virtual surgery. A virtual basic surgical training framework integrated with real-time force rendering has been recognized as one of the most immersive implementations in medical education. Yet, compared to the original intraoperative data, there has always been an argument that these data are represented by lower fidelity in virtual surgical training. In this paper, a dynamic biomechanics experimental framework is designed to achieve a highly immersive haptic sensation during the biopsy therapy with human respiratory motion; it is the first time to introduce the idea of periodic extension idea into the dynamic percutaneous force modeling. Clinical evaluation is conducted and performed in the Yunnan First People's Hospital, which not only demonstrated a higher fitting degree (AVG: 99.36%) with the intraoperation data than previous algorithms (AVG: 87.83%, 72.07%, and 66.70%) but also shows a universal fitting range with multilayer tissue. 27 urologists comprising 18 novices and 9 professors were invited to the VR-based training evaluation based on the proposed haptic rendering solution. Subjective and objective results demonstrated higher performance than the existing benchmark training simulator. Combining these in a systematic approach, tuned with specific fidelity requirements, haptically enabled medical simulation systems would be able to provide a more immersive and effective training environment.

A review of simulation applications in temporal bone surgery

Background

Temporal bone surgery is a technically challenging and high-risk procedure in an anatomically complex area. Safe temporal bone surgery emphasizes a consummate anatomic understanding and technique development that requires the guidance of an experienced otologic surgeon and years of practice. Temporal bone simulation can augment otologic surgical training and enable rehearsal of surgical procedures.

Objectives

The purpose of this article is to provide an updated review of temporal bone simulation platforms and their uses.

Data Sources

PubMed literature search. Search terms included temporal bone, temporal bone simulation, virtual reality (VR), and presurgical planning and rehearsal.

Discussion

Various simulation platforms such as cadaveric bone, three-dimensional (3D) printed models, and VR simulation have been used for temporal bone surgery training. However, each simulation method has its drawbacks. There is a need to improve upon current simulation platforms to enhance surgical training and skills assessment, as well as a need to explore other clinically significant applications of simulation, such as preoperative planning and rehearsal, in otologic surgery.

Conclusions

There is no replacement for actual surgical experience, but high-fidelity temporal bone models such as those produced with 3D printing and computer simulation have emerged as promising tools in otolaryngologic surgery. Improvements in the fidelity of both 3D printed and VR simulators as well as integration of a standardized assessment format would allow for an expansion in the use of these simulation platforms in training and assessment.

Level of Evidence

5.

Validation of a rhinologic virtual surgical simulator for performing a Draf 3 endoscopic frontal sinusotomy

BACKGROUND

We recently introduced a patient-specific rhinologic virtual surgical environment (VSE) that has shown potential for surgical rehearsal of various skull base lesions. Our aim in this study was to validate the usefulness of the rhinology VSE in performing the Draf 3 procedure.

METHODS

An outside-in Draf 3 procedure was performed on 4 cadaver heads. Computed tomography (CT) scans were obtained before and after cadaver dissection (CD). Pre-dissection CT scans were used to construct a cadaver-specific VSE. A virtual Draf 3 dissection (VD) was performed using the same technique. Validation was conducted by comparing the final common frontal outflow tract. A subjective comparison of the post-dissection endoscopic findings (CD vs VD) and an objective measurement using the post-dissection CT scan for the CD and the reconstructed CT scan obtained from the data after the VD was performed.

RESULTS

Subjective overall resemblance of the 2 dissections (CD vs VD) assessed by the 4-point Likert scale (0-3) was 2.5 (median interquartile range [IQR], 0.25) for the 4 cadavers. The median difference for the anteroposterior dimension of the frontal neo-ostium (CD vs VD) assessed in the midsagittal view was 0.11 mm, whereas the median difference for the lateral dimension assessed in the coronal view was 2.71 mm. Thus, no statistical difference was observed.

CONCLUSION

VD showed nearly matching results with the actual cadaver dissection. With further validation, our rhinologic VSE may be used for presurgical planning and rehearsal before the actual Draf 3 procedure is performed in the operating room.

Morphological analysis of sigmoid sinus anatomy: clinical applications to neurotological surgery

OBJECTIVES

The primary objective of this study was to use high-resolution micro-CT images to create accurate three-dimensional (3D) models of several intratemporal structures, and to compare several surgically important dimensions within the temporal bone. The secondary objective was to create a statistical shape model (SSM) of a dominant and non-dominant sigmoid sinus (SS) to provide a template for automated segmentation algorithms.

METHODS

A free image processing software, 3D Slicer, was utilized to create three-dimensional reconstructions of the SS, jugular bulb (JB), facial nerve (FN), and external auditory canal (EAC) from micro-CT scans. The models were used to compare several clinically important dimensions between the dominant and non-dominant SS. Anatomic variability of the SS was also analyzed using SSMs generated using the Statismo software framework.

RESULTS

Three-dimensional models from 38 temporal bones were generated and analyzed. Right dominance was observed in 74% of the paired SSs. All distances were significantly shorter on the dominant side (p < 0.05), including: EAC - SS (dominant: 13.7 ± 3.4 mm; non-dominant: 15.3 ± 2.7 mm), FN - SS (dominant: 7.2 ± 1.8 mm; non-dominant: 8.1 ± 2.3 mm), 2nd genu FN - superior tip of JB (dominant: 8.7 ± 2.2 mm; non-dominant: 11.2 ± 2.6 mm), horizontal distance between the superior tip of JB - descending FN (dominant: 9.5 ± 2.3 mm; non-dominant: 13.2 ± 3.5 mm), and horizontal distance between the FN at the stylomastoid foramen - JB (dominant: 5.4 ± 2.2 mm; non-dominant: 7.7 ± 2.1). Analysis of the SSMs indicated that SS morphology is most variable at its junction with the transverse sinus, and least variable at the JB.

CONCLUSIONS

This is the first known study to investigate the anatomical variation and relationships of the SS using high resolution scans, 3D  models and statistical shape analysis. This analysis seeks to guide neurotological surgical approaches and provide a template for automated segmentation and surgical simulation.

A review of virtual cutting methods and technology in deformable objects

BACKGROUND

Virtual cutting of deformable objects has been a research topic for more than a decade and has been used in many areas, especially in surgery simulation.

METHODS

We refer to the relevant literature and briefly describe the related research. The virtual cutting method is introduced, and we discuss the benefits and limitations of these methods and explore possible research directions.

RESULTS

Virtual cutting is a category of object deformation. It needs to represent the deformation of models in real time as accurately, robustly and efficiently as possible. To accurately represent models, the method must be able to: (1) model objects with different material properties; (2) handle collision detection and collision response; and (3) update the geometry and topology of the deformable model that is caused by cutting.

CONCLUSION

Virtual cutting is widely used in surgery simulation, and research of the cutting method is important to the development of surgery simulation.

Atlas-Based Segmentation of Temporal Bone Anatomy

PURPOSE

To develop a time-efficient automated segmentation approach that could identify critical structures in the temporal bone for visual enhancement and use in surgical simulation software.

METHODS

An atlas-based segmentation approach was developed to segment the cochlea, ossicles, semicircular canals (SCCs), and facial nerve in normal temporal bone CT images. This approach was tested in images of 26 cadaver bones (13 left, 13 right). The results of the automated segmentation were compared to manual segmentation visually and using DICE metric, average Hausdorff distance, and volume similarity.

RESULTS

The DICE metrics were greater than 0.8 for the cochlea, malleus, incus, and the SCCs combined. It was slightly lower for the facial nerve. The average Hausdorff distance was less than one voxel for all structures, and the volume similarity was 0.86 or greater for all structures except the stapes.

CONCLUSIONS

The atlas-based approach with rigid body registration of the otic capsule was successful in segmenting critical structures of temporal bone anatomy for use in surgical simulation software.

Virtual Reality and Augmented Reality in Plastic Surgery: A Review

Recently, virtual reality (VR) and augmented reality (AR) have received increasing attention, with the development of VR/AR devices such as head-mounted displays, haptic devices, and AR glasses. Medicine is considered to be one of the most effective applications of VR/AR. In this article, we describe a systematic literature review conducted to investigate the state-of-the-art VR/AR technology relevant to plastic surgery. The 35 studies that were ultimately selected were categorized into 3 representative topics: VR/AR-based preoperative planning, navigation, and training. In addition, future trends of VR/AR technology associated with plastic surgery and related fields are discussed.

Anatomy-Specific Virtual Reality Simulation in Temporal Bone Dissection: Perceived Utility and Impact on Surgeon Confidence

Objective

To evaluate the effect of anatomy-specific virtual reality (VR) surgical rehearsal on surgeon confidence and temporal bone dissection performance.

Study Design

Prospective pre- and poststudy of a novel virtual surgical rehearsal platform.

Setting

Academic otolaryngology–head and neck surgery residency training programs.

Subjects and Methods

Sixteen otolaryngology–head and neck surgery residents from 2 North American training institutions were recruited. Surveys were administered to assess subjects' baseline confidence in performing 12 subtasks of cortical mastoidectomy with facial recess. A cadaver temporal bone was randomly assigned to each subject. Cadaver specimens were scanned with a clinical computed tomography protocol, allowing the creation of anatomy-specific models for use in a VR surgical rehearsal platform. Subjects then rehearsed a virtual mastoidectomy on data sets derived from their specimens. Surgical confidence surveys were administered again. Subjects then dissected assigned cadaver specimens, which were blindly graded with a modified Welling scale. A final survey assessed the perceived utility of rehearsal on dissection performance.

Results

Of 16 subjects, 14 (87.5%) reported a significant increase in overall confidence after conducting an anatomy-specific VR rehearsal. A significant correlation existed between perceived utility of rehearsal and confidence improvement. The effect of rehearsal on confidence was dependent on trainee experience and the inherent difficulty of the surgical subtask. Postrehearsal confidence correlated strongly with graded dissection performance. Subjects rated anatomy-specific rehearsal as having a moderate to high contribution to their dissection performance.

Conclusion

Anatomy-specific virtual rehearsal improves surgeon confidence in performing mastoid dissection, dependent on surgeon experience and task difficulty. The subjective confidence gained through rehearsal correlates positively with subsequent objective dissection performance.

Expert subjective comparison of haptic models for bone-drill interaction

PURPOSE

A haptic algorithm to simulate the interaction between a surgical drill and bone using a constraint-based algorithm has been previously demonstrated. However, there has been no blinded study to determine whether this algorithm is preferred by professionals who commonly use this type of system METHODS: Fourteen otologic surgeons were presented with a spring-damper model and a constraint-based model of drill-bone interaction rendered on a low-cost haptic device with only linear feedback. The participants were blinded as to what algorithm they were using. They then answered survey questions about their opinions of the models.

RESULTS

The surgeons overwhelmingly preferred the constraint-based model. They generally preferred the constraint-based model in the individual questions as well.

CONCLUSIONS

Follow-up work can be done to fine-tune the parameters in the model, but this study shows that a sophisticated algorithm can make a significant difference even on a low-fidelity haptic device.