Surgery guided by mixed reality: presentation of a proof of concept

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

The application of mixed reality technique in oromaxillo-facial reconstruction with the perforator flap for malignant tumor patients

Objectives

The integration of quantitative imaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) with mixed reality (MR) technology holds promise for enhancing the diagnosis, prognosis, and treatment monitoring of cancer. This study compares the characteristics and effects of MR and color Doppler ultrasound (CDU) in the localization of perforator blood vessels in the lower extremities.

Methods

Two techniques were used to locate the perforator vessels in 40 cases of maxillofacial defect repair using perforator flaps from the lower extremities. The number of perforator vessels located in the flap area and the actual number of perforator vessels explored during the surgery were recorded. The recognition rate was calculated and the operation time and blood loss were recorded for each case.

Results

The recognition rates of MR technology and CDU in perforating vessels of the lower limbs were 93.9% and 97.2%, respectively (p > 0.05). The operation time was 52-74 minutes, 65-88 minutes (p > 0.05). The average bleeding volumes were 24 and 56 ml (p < 0.05), respectively. All perforator flaps were alive. One flap had a crisis and recovered after emergency exploratory treatment. Thirty donor sites of the lower extremities were directly sutured, and wounds were closed by abdominal skin grafting in 10 cases.

Conclusion

MR technology for successfully identifying perforator vessels can shorten the operation time, reduce the amount of bleeding in the donor site, and reduce trauma to the donor site.

Artificial intelligence in shoulder and elbow surgery: overview of current and future applications

Artificial intelligence (AI) is amongst the most rapidly growing technologies in orthopedic surgery. With the exponential growth in healthcare data, computing power, and complex predictive algorithms, this technology is poised to aid providers in data processing and clinical decision support throughout the continuum of orthopedic care. Understanding the utility and limitations of this technology is vital to practicing orthopedic surgeons, as these applications will become more common place in everyday practice. AI has already demonstrated its utility in shoulder and elbow surgery for imaging-based diagnosis, predictive modeling of clinical outcomes, implant identification, and automated image segmentation. The future integration of AI and robotic surgery represents the largest potential application of AI in shoulder and elbow surgery with the potential for significant clinical and financial impact. This editorial's purpose is to summarize common AI terms, provide a framework to understand and interpret AI model results, and discuss current applications and future directions within shoulder and elbow surgery.

Three-dimensional planning, navigation, patient-specific instrumentation and mixed reality in shoulder arthroplasty: a digital orthopedic renaissance

Accurate component placement in shoulder arthroplasty is crucial for avoiding complications, achieving superior biomechanical performance and optimizing functional outcomes. Shoulder and elbow surgeons have explored various methods to improve surgical understanding and precise execution including preoperative planning with 3D computed tomography (CT), patient-specific instrumentation (PSI), intraoperative navigation, and mixed reality (MR). 3D preoperative planning facilitated by CT scans and advanced software, enhances surgical precision, influences decision-making for implant types and approaches, reduces errors in guide pin placement, and contributes to cost-effectiveness. Navigation demonstrates benefits in reducing malpositioning, optimizing baseplate stability, improving humeral cut, and potentially conserving bone stock, although challenges such as varied operating times and costs warrant further investigation. The personalized patient care and enhanced operational efficiency associated with PSI are not only attractive for achieving desired component positions but also hold promise for improved outcomes in complex cases involving glenoid bone loss. Augmented reality (AR) and virtual reality (VR) technologies play a pivotal role in reshaping shoulder arthroplasty. They offer benefits in preoperative planning, intraoperative guidance, and interactive surgery. Studies demonstrate their effectiveness in AR-guided guidewire placement, providing real-time surgical advice during reverse total shoulder arthroplasty (RTSA). Additionally, these technologies show promise in orthopedic training, delivering superior realism and accelerating learning compared to conventional methods.

Validation of mixed-reality surgical navigation for glenoid axis pin placement in shoulder arthroplasty using a cadaveric model

BACKGROUND

Mixed reality may offer an alternative for computer-assisted navigation in shoulder arthroplasty. The purpose of this study was to determine the accuracy and precision of mixed-reality guidance for the placement of the glenoid axis pin in cadaver specimens. This step is essential for accurate glenoid placement in total shoulder arthroplasty.

METHODS

Fourteen cadaveric shoulders underwent simulated shoulder replacement surgery by 7 experienced shoulder surgeons. The surgeons exposed the cadavers through a deltopectoral approach and then used mixed-reality surgical navigation to insert a guide pin in a preplanned position and trajectory in the glenoid. The mixed-reality system used the Microsoft Hololens 2 headset, navigation software, dedicated instruments with fiducial marker cubes, and a securing pin. Computed tomography scans obtained before and after the procedure were used to plan the surgeries and determine the difference between the planned and executed values for the entry point, version, and inclination. One specimen had to be discarded from the analysis because the guide pin was removed accidentally before obtaining the postprocedure computed tomography scan.

RESULTS

Regarding the navigated entry point on the glenoid, the mean difference between planned and executed values was 1.7 ± 0.8 mm; this difference was 1.2 ± 0.6 mm in the superior-inferior direction and 0.9 ± 0.8 mm in the anterior-posterior direction. The maximum deviation from the entry point for all 13 specimens analyzed was 3.1 mm. Regarding version, the mean difference between planned and executed version values was 1.6° ± 1.2°, with a maximum deviation in version for all 13 specimens of 4.1°. Regarding inclination, the mean angular difference was 1.7° ± 1.5°, with a maximum deviation in inclination of 5°.

CONCLUSIONS

The mixed-reality navigation system used in this study allowed surgeons to insert the glenoid guide pin on average within 2 mm from the planned entry point and within 2° of version and inclination. The navigated values did not exceed 3 mm or 5°, respectively, for any of the specimens analyzed. This approach may help surgeons more accurately place the definitive glenoid component.

The Value of Computed Tomography-Based Planning in Shoulder Arthroplasty Compared to Intra-/Interobserver Reliability of X-ray Planning

Background: Reversed total shoulder arthroplasty (RTSA) is an established surgery for many pathologies of the shoulder and the demand continues to rise with an aging population. Preoperative planning is mandatory to support the surgeon's understanding of the patient's individual anatomy and, therefore, is crucial for the patient's outcome. Methods: In this observational study, we identified 30 patients who underwent RTSA with two- and three-dimensional preoperative planning. Each patient underwent new two-dimensional planning from a medical student and an orthopedic resident as well as through a mid-volume and high-volume shoulder surgeon, which was repeated after a minimum of 4 weeks. The intra- and interobserver reliability was then analyzed and compared to the 3D planning and the implanted prosthesis. The evaluated parameters were the size of the pegged glenoid baseplate, glenosphere, and humeral short stem. Results: The inter-rater reliability showed higher deviations in all four raters compared to the 3D planning of the base plate, glenosphere, and shaft. The intra-rater reliability showed a better correlation in more experienced raters, especially in the planning of the shaft. Conclusions: Our study shows that 3D planning is more accurate than traditional planning on plain X-rays, despite experienced shoulder surgeons showing better results in 2D planning than inexperienced ones.

Comparing Different Registration and Visualization Methods for Navigated Common Femoral Arterial Access-A Phantom Model Study Using Mixed Reality

Mixed reality (MxR) enables the projection of virtual three-dimensional objects into the user's field of view via a head-mounted display (HMD). This phantom model study investigated three different workflows for navigated common femoral arterial (CFA) access and compared it to a conventional sonography-guided technique as a control. A total of 160 punctures were performed by 10 operators (5 experts and 5 non-experts). A successful CFA puncture was defined as puncture at the mid-level of the femoral head with the needle tip at the central lumen line in a 0° coronary insertion angle and a 45° sagittal insertion angle. Positional errors were quantified using cone-beam computed tomography following each attempt. Mixed effect modeling revealed that the distance from the needle entry site to the mid-level of the femoral head is significantly shorter for navigated techniques than for the control group. This highlights that three-dimensional visualization could increase the safety of CFA access. However, the navigated workflows are infrastructurally complex with limited usability and are associated with relevant cost. While navigated techniques appear as a potentially beneficial adjunct for safe CFA access, future developments should aim to reduce workflow complexity, avoid optical tracking systems, and offer more pragmatic methods of registration and instrument tracking.

The application of extended reality technology-assisted intraoperative navigation in orthopedic surgery

Extended reality (XR) technology refers to any situation where real-world objects are enhanced with computer technology, including virtual reality, augmented reality, and mixed reality. Augmented reality and mixed reality technologies have been widely applied in orthopedic clinical practice, including in teaching, preoperative planning, intraoperative navigation, and surgical outcome evaluation. The primary goal of this narrative review is to summarize the effectiveness and superiority of XR-technology-assisted intraoperative navigation in the fields of trauma, joint, spine, and bone tumor surgery, as well as to discuss the current shortcomings in intraoperative navigation applications. We reviewed titles of more than 200 studies obtained from PubMed with the following search terms: extended reality, mixed reality, augmented reality, virtual reality, intraoperative navigation, and orthopedic surgery; of those 200 studies, 69 related papers were selected for abstract review. Finally, the full text of 55 studies was analyzed and reviewed. They were classified into four groups-trauma, joint, spine, and bone tumor surgery-according to their content. Most of studies that we reviewed showed that XR-technology-assisted intraoperative navigation can effectively improve the accuracy of implant placement, such as that of screws and prostheses, reduce postoperative complications caused by inaccurate implantation, facilitate the achievement of tumor-free surgical margins, shorten the surgical duration, reduce radiation exposure for patients and surgeons, minimize further damage caused by the need for visual exposure during surgery, and provide richer and more efficient intraoperative communication, thereby facilitating academic exchange, medical assistance, and the implementation of remote healthcare.

Machine Learning in Spine Surgery: A Narrative Review

Artificial intelligence and machine learning (ML) can offer revolutionary advances in their application to the field of spine surgery. Within the past 5 years, novel applications of ML have assisted in surgical decision-making, intraoperative imaging and navigation, and optimization of clinical outcomes. ML has the capacity to address many different clinical needs and improve diagnostic and surgical techniques. This review will discuss current applications of ML in the context of spine surgery by breaking down its implementation preoperatively, intraoperatively, and postoperatively. Ethical considerations to ML and challenges in ML implementation must be addressed to maximally benefit patients, spine surgeons, and the healthcare system. Areas for future research in augmented reality and mixed reality, along with limitations in generalizability and bias, will also be highlighted.

Glenoid component placement in reverse shoulder arthroplasty assisted with augmented reality through a head-mounted display leads to low deviation between planned and postoperative parameters

BACKGROUND

Navigated augmented reality (AR) through a head-mounted display (HMD) may lead to accurate glenoid component placement in reverse shoulder arthroplasty (RSA). The purpose of this study is to evaluate the deviation between planned, intra- and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement assisted by a navigated AR through HMD during RSA.

METHODS

Both shoulders of 6 fresh frozen human cadavers, free from fractures or other bony pathologies, were used. Preoperative computed tomography (CT) scans were used for the 3-dimensional (3D) planning. The glenoid component placement was assisted using a navigated AR system through an HMD in all specimens. Intraoperative inclination, retroversion, depth, and rotation were measured by the system. A postoperative CT scan was performed. The pre- and postoperative 3D CT scan reconstructions were superimposed to calculate the deviation between planned and postoperative inclination, retroversion, entry point, depth, and rotation of the glenoid component placement. Additionally, a comparison between intra- and postoperative values was calculated. Outliers were defined as >10° inclination, >10° retroversion, >3 mm entry point.

RESULTS

The registration algorithm of the scapulae prior to the procedure was correctly completed for all cases. The deviations between planned and postoperative values were 1.0° ± 0.7° for inclination, 1.8° ± 1.3° for retroversion, 1.1 ± 0.4 mm for entry point, 0.7 ± 0.6 mm for depth, and 1.7° ± 1.6° for rotation. The deviation between intra- and postoperative values were 0.9° ± 0.8° for inclination, 1.2° ± 1.1° for retroversion, 0.6 ± 0.5 mm for depth, and 0.3° ± 0.2° for rotation. There were no outliers between planned and postoperative parameters.

CONCLUSION

In this study, the use of a navigated AR system through an HMD for RSA led to low deviation between planned and postoperative values and between intra- and postoperative parameters.

Applications of Extended Reality in Orthopaedic Surgery

➤ Extended reality is a term that encompasses different modalities, including virtual reality, augmented reality, and mixed reality.➤ Although fully immersive virtual reality has benefits for developing procedural memory and technical skills, augmented and mixed reality are more appropriate modalities for preoperative planning and intraoperative utilization.➤ Current investigations on the role of extended reality in preoperative planning and intraoperative utilization are still in the early stages, but preliminarily show that extended reality technologies can help surgeons to be more accurate and efficient.

Mixed reality in primary retroperitoneal tumour surgery: Evaluation of preoperative and intraoperative application value

OBJECTIVE

To evaluate the feasibility and application value of mixed reality technology (MR) in Primary retroperitoneal tumour (PRT) surgery.

METHODS

From 276 patients who underwent PRT resection at the First Affiliated Hospital of Xi'an Jiaotong University, we screened 46 patients who underwent MR-assisted retroperitoneal tumour resection and 46 patients who underwent tumour resection without MR assistance. The intraoperative and postoperative recovery of the patients in both groups were compared, and the reliability and validity of the application of MR were further examined using the Likert scale.

RESULTS

There was a significant difference in the mean intraoperative bleeding volume between the two groups, but it was reduced in the MR group. The results of the Likert scale showed higher scores in the MR group than non-MR group.

CONCLUSIONS

MR can be used to assist PRT resection and has great potential to improve the rate of complete retroperitoneal tumour resection.

Technological innovations in shoulder replacement: current concepts and the future of robotics in total shoulder arthroplasty

BACKGROUND

Total shoulder arthroplasty (TSA) has been rapidly evolving over the last several decades, with innovative technological strategies being investigated and developed in order to achieve optimal component precision and joint alignment and stability, preserve implant longevity, and improve patient outcomes. Future advancements such as robotic-assisted surgeries, augmented reality, artificial intelligence, patient-specific instrumentation (PSI) and other peri- and preoperative planning tools will continue to revolutionize TSA. Robotic-assisted arthroplasty is a novel and increasingly popular alternative to the conventional arthroplasty procedure in the hip and knee but has not yet been investigated in the shoulder. Therefore, the purpose of this study was to conduct a narrative review of the literature on the evolution and projected trends of technological advances and robotic assistance in total shoulder arthroplasty.

METHODS

A narrative synthesis method was employed for this review, rather than a meta-analysis or systematic review of the literature. This decision was based on 2 primary factors: (1) the lack of eligible, peer-reviewed studies with high-quality level of evidence available for review on robotic-assisted shoulder arthroplasty, and (2) a narrative review allows for a broader scope of content analysis, including a comprehensive review of all technological advances-including robotics-within the field of TSA. A general literature search was performed using PubMed, Embase, and Cochrane Library databases. These databases were queried by 2 independent reviewers from database inception through November 11, 2022, for all articles investigating the role of robotics and technology assistance in total shoulder arthroplasty. Inclusion criteria included studies describing "shoulder arthroplasty" and "robotics."

RESULTS

After exclusion criteria were applied, 4 studies on robotic-assisted TSA were described in the review. Given the novelty of this technology and limited data on robotics in TSA, these studies consisted of a literature review, nonvalidated experimental biomechanical studies in sawbones models, and preclinical proof-of-concept cadaveric studies using prototype robotic technology primarily in conjunction with PSI. The remaining studies described the technological advancements in TSA, including PSI, computer-assisted navigation, artificial intelligence, machine learning, and virtual, augmented, and mixed reality. Although not yet commercially available, robotic-assisted TSA confers the theoretical advantages of precise humeral head cuts for restoration of proximal humerus anatomy, more accurate glenoid preparation, and improved soft-tissue assessment in limited early studies.

CONCLUSION

The evidence for the use of robotics in total hip arthroplasty and total knee arthroplasty demonstrates improved component accuracy, more precise radiographic measurements, and improved early/mid-term patient-reported and functional outcomes. Although no such data currently exist for shoulder arthroplasty given that the technology has not yet been commercialized, the lessons learned from robotic hip and knee surgery in conjunction with its rapid adoption suggests robotic-assisted TSA is on the horizon of innovation. By achieving a better understanding of the past, present, and future innovations in TSA through this narrative review, orthopedic surgeons can be better prepared for future applications.

Mixed Reality Improves 3D Visualization and Spatial Awareness of Bone Tumors for Surgical Planning in Orthopaedic Oncology: A Proof of Concept Study

Introduction

In orthopedic oncology, computer navigation and 3D-printed guides facilitate precise osteotomies only after surgical exposure. Before surgeries start, it is challenging to mentally process and superimpose the virtual medical images onto patients' anatomy for preoperative surgical planning. Mixed Reality (MR) is an immersive technology merging real and virtual worlds, and users can interact with digital objects in real time. Through Head-Mounted Displays, surgeons directly visualize holographic models that overlaid on tumor patients. The technology may facilitate surgical planning before skin incisions.

Methods

Nine bone tumor patients were included (July 2021 - Dec 2022). There were six primary bone sarcomas, two benign bone tumors, and one revision pelvic prosthesis. MR applications were created using patients' preoperative medical images. The surgeon examined each patient clinically using the conventional method of viewing 2D images and MR via HMD, Hololens 2. A Likert-Scale (LS) questionnaire and The National Aeronautics and Space Administration-Task Load Index (NASA-TLX) score were used to evaluate and compare the effectiveness of surgical planning and the surgeon's clinical cognitive workload for the two methods.

Results

The qualitative survey of the LS questionnaire suggested that the MR group had superior spatial awareness of tumors and was considered more effective as a preoperative planning tool than the conventional group. For NASA-TLX scores, the overall cognitive workload was lower in MR 3D hologram group than in the 2D Group for preoperative clinical assessment. When using MR technology with HMDs, the surgeon reported no discomfort.

Conclusion

MR technology may improve 3D visualization and spatial awareness of bone tumors in patients' anatomies and may facilitate surgical planning before skin incisions in orthopedic oncology surgery. With less cognitive load and better ergonomics, surgeons can focus on patients and surgical tasks with MR technology. Further studies must investigate whether MR technology improves clinical outcomes.

Accuracy of Femoral Tunnel Localization With Mixed Reality Technology-Assisted Single-Bundle ACL Reconstruction

Background

It is clinically challenging to accurately drill femoral and tibial tunnels to reconstruct the anterior cruciate ligament (ACL). Mixed reality (MR) technology, a further development of virtual reality technology, presents virtual scene information in real time and establishes an interactive feedback information loop among the real world, the virtual world, and the user.

Purpose/Hypothesis

The purpose of this study was to investigate the structural and early clinical outcomes of ACL reconstruction assisted by MR technology. It was hypothesized that MR technology would improve the accuracy of tunnel localization.

Study Design

Cohort study; Level of evidence, 3.

Methods

Included were 44 patients at a single institution who underwent arthroscopic single-bundle ACL reconstruction between June 2020 and March 2022. Reconstruction with the aid of MR technology was performed in 21 patients (MR group), and conventional arthroscopic reconstruction was performed in 23 patients. Postoperatively, the parameters related to the bone tunnel positioning were compared by computed tomography imaging with 3-dimensional (3D) reconstruction, and 12-month postoperative clinical outcomes were assessed with the Lysholm and International Knee Documentation Committee scores.

Results

There was no statistically significant difference in projection angles in the coronal, axial, or sagittal plane between the preoperative virtually created tunnel guide pin and the actual tunnel (P > .05 for all). In the MR group, the center of the femoral tunnel exit was closer to the apex of the lateral femoral condyle along the proximal-distal axis (14.07 ± 4.12 vs 17.49 ± 6.24 mm for the conventional group; P < .05) and the graft bending angle was lower (117.71° ± 8.08° vs 127.81° ± 11.91° for the conventional group; P < .05). The scatterplot of the femoral tunnel location distribution showed that the entrance and exit points in the MR group were more concentrated and closer to the ideal location of the preoperative design than in the conventional group. Patients in both groups had significant preoperative-to-postoperative improvement based on outcome scores (P < .001 for all), with no significant difference between groups.

Conclusion

ACL reconstruction with the aid of MR technology allowed for more accurate positioning and orientation of the femoral tunnel during surgery when compared with conventional reconstruction.

Emerging Technologies in Shoulder Arthroplasty: Navigation, Mixed Reality, and Preoperative Planning

Shoulder arthroplasty is a rapidly improving and utilized management for end-stage arthritis that is associated with improved functional outcomes, pain relief, and long-term implant survival. Accurate placement of the glenoid and humeral components is critical for improved outcomes. Traditionally, preoperative planning was limited to radiographs and 2-dimensional computed tomography (CT); however, 3-dimensional CT is becoming more commonly utilized and necessary to understand complex glenoid and humeral deformities. To further increase accurate component placement, intraoperative assistive devices-patient-specific instrumentation, navigation, and mixed reality-minimize malpositioning, increase surgeon accuracy, and maximize fixation. These intraoperative technologies likely represent the future of shoulder arthroplasty.

Next step trauma and orthopaedic surgery: integration of augmented reality for reduction and nail implantation of tibial fractures

INTRODUCTION

There is a tremendous scope of hardware and software development going on in augmented reality (AR), also in trauma and orthopaedic surgery. However, there are only a few systems available for intra-operative 3D imaging and guidance, most of them rely on peri- and intra-operative X-ray imaging. Especially in complex situations such as pelvic surgery or multifragmentary multilevel fractures, intra-operative 3D imaging and implant tracking systems have proven to be of great advantage for the outcome of the surgery and can help reduce X-ray exposure, at least for the surgical team (Ochs et al. in Injury 41:1297 1305, 2010). Yet, the current systems do not provide the ability to have a dynamic live view from the perspective of the surgeon. Our study describes a prototype AR-based system for live tracking which does not rely on X-rays.

MATERIALS AND METHODS

A protype live-view intra-operative guidance system using an AR head-mounted device (HMD) was developed and tested on the implantation of a medullary nail in a tibia fracture model. Software algorithms that allow live view and tracking of the implant, fracture fragments and soft tissue without the intra-operative use of X-rays were derived.

RESULTS

The implantation of a medullar tibia nail is possible while only relying on AR-guidance and live view without the intra-operative use of X-rays.

CONCLUSIONS

The current paper describes a feasibility study with a prototype of an intra-operative dynamic live tracking and imaging system that does not require intra-operative use of X-rays and dynamically adjust to the perspective of the surgeons due to an AR HMD. To our knowledge, the current literature does not describe any similar systems. This could be the next step in surgical imaging and education and a promising way to improve patient care.

Magnetic resonance imaging based neurosurgical planning on hololens 2: A feasibility study in a paediatric hospital

Objective

The goal of this work is to show how to implement a mixed reality application (app) for neurosurgery planning based on neuroimaging data, highlighting the strengths and weaknesses of its design.

Methods

Our workflow explains how to handle neuroimaging data, including how to load morphological, functional and diffusion tensor imaging data into a mixed reality environment, thus creating a first guide of this kind. Brain magnetic resonance imaging data from a paediatric patient were acquired using a 3 T Siemens Magnetom Skyra scanner. Initially, this raw data underwent specific software pre-processing and were subsequently transformed to ensure seamless integration with the mixed reality app. After that, we created three-dimensional models of brain structures and the mixed reality environment using Unity™ engine together with Microsoft® HoloLens 2™ device. To get an evaluation of the app we submitted a questionnaire to four neurosurgeons. To collect data concerning the performance of a user session we used Unity Performance Profiler.

Results

The use of the interactive features, such as rotating, scaling and moving models and browsing through menus, provided by the app had high scores in the questionnaire, and their use can still be improved as suggested by the performance data collected. The questionnaire's average scores were high, so the overall experiences of using our mixed reality app were positive.

Conclusion

We have successfully created a valuable and easy-to-use neuroimaging data mixed reality app, laying the foundation for more future clinical uses, as more models and data derived from various biomedical images can be imported.

Augmented reality through head-mounted display for navigation of baseplate component placement in reverse total shoulder arthroplasty: a cadaveric study

BACKGROUND

To achieve an optimal clinical outcome in reverse total shoulder arthroplasty (RSA), accurate placement of the components is essential. The recently introduced navigation technology of augmented reality (AR) through head-mounted displays (HMD) offers a promising new approach to visualize the anatomy and navigate component positioning in various orthopedic surgeries. We hypothesized that AR through HMD is feasible, reliable, and accurate for guidewire placement in RSA baseplate positioning.

METHODS

Twelve human cadaver shoulders were scanned with computed tomography (CT) and RSA baseplate positioning was 3-D planned using dedicated software. The shoulders were prepared through a deltopectoral approach and an augmented reality hologram was superimposed using the HMD Microsoft HoloLense. The central guidewire was then navigated through the HMD to achieve the planned entry point and trajectory. Postoperatively, the shoulders were CT-scanned a second time and the deviation from the planning was calculated.

RESULTS

The mean deviation of the entry point was 3.5 mm  ± 1.7 mm (95% CI 2.4 mm; 4.6 mm). The mean deviation of the planned trajectory was 3.8°  ± 1.7° (95% CI 2.6°; 4.9°).

CONCLUSION

Augmented reality seems feasible and reliable for baseplate guidewire positioning in reverse total shoulder arthroplasty. The achieved values were accurate.

Application of Augmented Reality to Maxillary Resections: A Three-Dimensional Approach to Maxillofacial Oncologic Surgery

In the relevant global context, although virtual reality, augmented reality, and mixed reality have been emerging methodologies for several years, only now have technological and scientific advances made them suitable for revolutionizing clinical care and medical settings through the provision of advanced features and improved healthcare services. Over the past fifteen years, tools and applications using augmented reality (AR) have been designed and tested in the context of various surgical and medical disciplines, including maxillofacial surgery. The purpose of this paper is to show how a marker-less AR guidance system using the Microsoft^® HoloLens 2 can be applied in mandible and maxillary demolition surgery to guide maxillary osteotomies. We describe three mandibular and maxillary oncologic resections performed during 2021 using AR support. In these three patients, we applied a marker-less tracking method based on recognition of the patient's facial profile. The surgeon, using HoloLens 2 smart glasses, could see the virtual surgical planning superimposed on the patient's anatomy. We showed that performing osteotomies under AR guidance is feasible and viable, as demonstrated by comparison with osteotomies performed using CAD-CAM cutting guides. This technology has advantages and disadvantages. However, further research is needed to improve the stability and robustness of the marker-less tracking method applied to patient face recognition.

The use of mixed reality in dentistry

Augmented reality (AR) was first described in the literature in the 1990s. It has been shown as a futuristic concept in television, film and media, and now in the twenty-first century has become a reality. AR is defined as an interactive experience of a real-world environment where the object that resides in the real world is enhanced by computer-generated perceptual information.Microsoft HoloLens is a mixed reality device which has the capability to provide a real-time, three-dimensional platform using multiple sensors and holographic processing to display information and even simulate a virtual world. With rapidly evolving technology and virtual learning on the increase, the HoloLens technology can be used as a vital tool for dental education and surgical planning. However, within dentistry at present, there is limited research regarding its benefits and potential.The authors would like to demonstrate the use of HoloLens in three common oral surgery procedures and how it can be used to distinguish anatomy and benefit surgical planning, aid in patient communication and play a role in dental education.

Case Report: Application of Mixed Reality Combined With A Surgical Template for Precise Periapical Surgery

Objective

The etiology of apical diseases is diverse, and most are due to incomplete root canal therapy. The common clinical manifestations include gingival abscess, fistula and bone destruction. The currently existing limitation of procedures is that surgeons cannot visually evaluate the surgical areas. We sought to combine mixed reality (MR) technology with a 3-dimensional (3D) printed surgical template to achieve visualization in apical surgery. Notably, no reports have described this application.

Methods

We created visual 3D (V3D) files and transferred them into the HoloLens system. We explained the surgical therapy plan to the patient using a mixed reality head-mounted display (MR-HMD). Then, the 3D information was preliminarily matched with the operative area, and the optimal surgical approach was determined by combining this information with 3D surgical guide plate technology.

Results

We successfully developed a suitable surgical workflow and confirmed the optimal surgical approach from the buccal side. We completely exposed the apical lesion and removed the inflammatory granulation tissue.

Conclusion

We are the first group to use the MR technique in apical surgery. We integrated the MR technique with a 3D surgical template to successfully accomplish the surgery. Desirable outcomes using minimally invasive therapy could be achieved with the MR technique.

Innovations in Shoulder Arthroplasty

Innovations currently available with anatomic total shoulder arthroplasty include shorter stem designs and augmented/inset/inlay glenoid components. Regarding reverse shoulder arthroplasty (RSA), metal augmentation, including custom augments, on both the glenoid and humeral side have expanded indications in cases of bone loss. In the setting of revision arthroplasty, humeral options include convertible stems and newer tools to improve humeral implant removal. New strategies for treatment and surgical techniques have been developed for recalcitrant shoulder instability, acromial fractures, and infections after RSA. Finally, computer planning, navigation, PSI, and augmented reality are imaging options now available that have redefined preoperative planning and indications as well intraoperative component placement. This review covers many of the innovations in the realm of shoulder arthroplasty.

Review and Future/Potential Application of Mixed Reality Technology in Orthopaedic Oncology

In orthopaedic oncology, surgical planning and intraoperative execution errors may result in positive tumor resection margins that increase the risk of local recurrence and adversely affect patients’ survival. Computer navigation and 3D-printed resection guides have been reported to address surgical inaccuracy by replicating the surgical plans in complex cases. However, limitations include surgeons’ attention shift from the operative field to view the navigation monitor and expensive navigation facilities in computer navigation surgery. Practical concerns are lacking real-time visual feedback of preoperative images and the lead-time in manufacturing 3D-printed objects. Mixed Reality (MR) is a technology of merging real and virtual worlds to produce new environments with enhanced visualizations, where physical and digital objects coexist and allow users to interact with both in real-time. The unique MR features of enhanced medical images visualization and interaction with holograms allow surgeons real-time and on-demand medical information and remote assistance in their immediate working environment. Early application of MR technology has been reported in surgical procedures. Its role is unclear in orthopaedic oncology. This review aims to provide orthopaedic tumor surgeons with up-to-date knowledge of the emerging MR technology. The paper presents its essential features and clinical workflow, reviews the current literature and potential clinical applications, and discusses the limitations and future development in orthopaedic oncology. The emerging MR technology adds a new dimension to digital assistive tools with a more accessible and less costly alternative in orthopaedic oncology. The MR head-mounted display and hand-free control may achieve clinical point-of-care inside or outside the operating room and improve service efficiency and patient safety. However, lacking an accurate hologram-to-patient matching, an MR platform dedicated to orthopaedic oncology, and clinical results may hinder its wide adoption. Industry-academic partnerships are essential to advance the technology with its clinical role determined through future clinical studies.

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/t4hl_Anh_kM

Glenoid Component Placement Assisted by Augmented Reality Through a Head-Mounted Display During Reverse Shoulder Arthroplasty

Component positioning is a key factor for avoiding complications and improving functional outcomes in reverse shoulder arthroplasty. Preoperative planning can improve component positioning. However, translating the preoperative plan into the surgical procedure can be challenging. This is particularly the case for the glenoid component positioning in severe deformity or limited visualization of the scapula. Different computational-assisted techniques have been developed to aid implementation of the preoperative plan into the surgical procedure. Navigated augmented reality (AR) refers to the real world augmented with virtual real-time information about the position and orientation of instruments and components. This information can be presented through a head-mounted display (HMD), which enables the user to visualize the virtual information directly overlaid onto the real world. Navigated AR systems through HMD have been validated for shoulder arthroplasty using phantoms and cadavers. This article details a step-by-step guide use of a navigated AR system through HMD, in the placement of the glenoid bony-augmented component.

Mixed Reality Technology in Total Knee Arthroplasty: An Updated Review With a Preliminary Case Report

Background

Augmented reality and mixed reality have been used to help surgeons perform complex surgeries. With the development of technology, mixed reality (MR) technology has been used to improve the success rate of complex hip arthroplasty due to its unique advantages. At present, there are few reports on the application of MR technology in total knee arthroplasty. We presented a case of total knee arthroplasty with the help of mixed reality technology.

Case Presentation

We presented a case of a 71-year-old woman who was diagnosed with bilateral knee osteoarthritis with varus deformity, especially on the right side. After admission, the right total knee arthroplasty was performed with the assistance of MR technology. Before the operation, the three-dimensional virtual model of the knee joint of the patient was reconstructed for condition analysis, operation plan formulation, and operation simulation. During the operation, the three-dimensional virtual images of the femur and tibia coincided with the real body of the patient, showing the osteotomy plane designed before the operation, which can accurately guide the completion of osteotomy and prosthesis implantation.

Conclusions

As far as we know, this is the first report on total knee arthroplasty under the guidance of mixed reality technology.

Three dimensional patient-specific guides for guide pin positioning in reverse shoulder arthroplasty: An experimental study on different glenoid types

INTRODUCTION

Incorrect positioning is one of the main factors for glenoid component loosening in reverse shoulder arthroplasty and component placement can be challenging. This study aimed to assess whether Patient-Specific Instrumentation (PSI) provides better guide pin positioning accuracy and is superior to standard guided and freehand instrumentation methods in cases of glenoid bone deformity.

MATERIALS AND METHODS

Based on the Walch classification, five different scapula types were acquired by computed tomography (CT). For each type, two different surgeons placed a guide pin into the scapula using three different methods: freehand method, conventional non-patient-specific guide, and PSI guide. Each method was repeated five times by both surgeons. In these experiments, a total of 150 samples of scapula models were used (5 × 2 × 3 × 5 = 150). Post-operative CT scans of the samples with the guide pin were digitally assessed and the accuracy of the pin placement was determined by comparison to the preoperative planning on a three-dimensional (3D) model.

RESULTS

The PSI method showed accuracies to the preoperative plan of 2.68 (SD 2.10) degrees for version angle (p < .05), 2.59 (SD 2.68) degrees for inclination angle (p < .05), and 1.55 (SD 1.26) mm for entry point offset (p < .05). The mean and standard deviation errors compared to planned values of version angle, inclination angle, and entry point offset were statistically significant for the PSI method for the type C defected glenoid and non-arthritic glenoid.

CONCLUSION

Using the PSI guide created by an image processing software tool for guide pin positioning showed advantages in glenoid component positioning over other methods, for defected and intact glenoid types, but correlation with clinical outcomes should be examined.

The Evolution of Reverse Total Shoulder Arthroplasty-From the First Steps to Novel Implant Designs and Surgical Techniques

PURPOSE OF REVIEW

The purpose of this review is to summarize recent literature regarding the latest design modifications and biomechanical evolutions of reverse total shoulder arthroplasty and their impact on postoperative outcomes.

RECENT FINDINGS

Over the past decade, worldwide implantation rates of reverse total shoulder arthroplasty have drastically increased for various shoulder pathologies. While Paul Grammont's design principles first published in 1985 for reverse total shoulder arthroplasty remained unchanged, several adjustments were made to address postoperative clinical and biomechanical challenges such as implant glenoid loosening, scapular notching, or limited range of motion in order to maximize functional outcomes and increase the longevity of reverse total shoulder arthroplasty. However, the adequate and stable fixation of prosthetic components can be challenging, especially in massive osteoarthritis with concomitant bone loss. To overcome such issues, surgical navigation and patient-specific instruments may be a viable tool to improve accurate prosthetic component positioning. Nevertheless, larger clinical series on the accuracy and possible complications of this novel technique are still missing.

Thermal Ablation of Liver Tumors Guided by Augmented Reality: An Initial Clinical Experience

Background: Over the last two decades, augmented reality (AR) has been used as a visualization tool in many medical fields in order to increase precision, limit the radiation dose, and decrease the variability among operators. Here, we report the first in vivo study of a novel AR system for the guidance of percutaneous interventional oncology procedures. Methods: Eight patients with 15 liver tumors (0.7−3.0 cm, mean 1.56 + 0.55) underwent percutaneous thermal ablations using AR guidance (i.e., the Endosight system). Prior to the intervention, the patients were evaluated with US and CT. The targeted nodules were segmented and three-dimensionally (3D) reconstructed from CT images, and the probe trajectory to the target was defined. The procedures were guided solely by AR, with the position of the probe tip was subsequently confirmed by conventional imaging. The primary endpoints were the targeting accuracy, the system setup time, and targeting time (i.e., from the target visualization to the correct needle insertion). The technical success was also evaluated and validated by co-registration software. Upon completion, the operators were assessed for cybersickness or other symptoms related to the use of AR. Results: Rapid system setup and procedural targeting times were noted (mean 14.3 min; 12.0−17.2 min; 4.3 min, 3.2−5.7 min, mean, respectively). The high targeting accuracy (3.4 mm; 2.6−4.2 mm, mean) was accompanied by technical success in all 15 lesions (i.e., the complete ablation of the tumor and 13/15 lesions with a >90% 5-mm periablational margin). No intra/periprocedural complications or operator cybersickness were observed. Conclusions: AR guidance is highly accurate, and allows for the confident performance of percutaneous thermal ablations.

Looking through the Lens: The Reality of Telesurgical Support with Interactive Technology Using Microsoft's HoloLens 2

Reality technologies in the orthopaedic arena have been increasing in use over the last decade, including virtual reality (VR), augmented reality (AR), and mixed reality (MR). MR is one of the most recent innovations and perhaps the most promising for improving the overall surgical experience. The purpose of this case report was to demonstrate a complex total knee arthroplasty case where unplanned remote assistance was used for telesurgical support using the HoloLens 2.

Mixed-Reality-Assisted Puncture of the Common Femoral Artery in a Phantom Model

Percutaneous femoral arterial access is daily practice in a variety of medical specialties and enables physicians worldwide to perform endovascular interventions. The reported incidence of percutaneous femoral arterial access complications is 3–18% and often results from suboptimal puncture location due to insufficient visualization of the target vessel. The purpose of this proof-of-concept study was to evaluate the feasibility and the positional error of a mixed-reality (MR)-assisted puncture of the common femoral artery in a phantom model using a commercially available navigation system. In total, 15 MR-assisted punctures were performed. Cone-beam computed tomography angiography (CTA) was used following each puncture to allow quantification of positional error of needle placements in the axial and sagittal planes. Technical success was achieved in 14/15 cases (93.3%) with a median axial positional error of 1.0 mm (IQR 1.3) and a median sagittal positional error of 1.1 mm (IQR 1.6). The median duration of the registration process and needle insertion was 2 min (IQR 1.0). MR-assisted puncture of the common femoral artery is feasible with acceptable positional errors in a phantom model. Future studies should aim to measure and reduce the positional error resulting from MR registration.

Accuracy of Mixed Reality Combined With Surgical Navigation Assisted Oral and Maxillofacial Tumor Resection

OBJECTIVE

To evaluate the feasibility and accuracy of mixed reality combined with surgical navigation in oral and maxillofacial tumor surgery.

METHODS

Retrospective analysis of data of seven patients with oral and maxillofacial tumors who underwent surgery between January 2019 and January 2021 using a combination of mixed reality and surgical navigation. Virtual surgical planning and navigation plan were based on preoperative CT datasets. Through IGT-Link port, mixed reality workstation was synchronized with surgical navigation, and surgical planning data were transferred to the mixed reality workstation. Osteotomy lines were marked with the aid of both surgical navigation and mixed reality images visualized through HoloLens. Frozen section examination was used to ensure negative surgical margins. Postoperative CT datasets were obtained 1 week after the surgery, and chromatographic analysis of virtual osteotomies and actual osteotomies was carried out. Patients received standard oncological postoperative follow-up.

RESULTS

Of the seven patients, four had maxillary tumors and three had mandibular tumors. There were total of 13 osteotomy planes. Mean deviation between the planned osteotomy plane and the actual osteotomy plane was 1.68 ± 0.92 mm; the maximum deviation was 3.46 mm. Chromatographic analysis showed error of ≤3 mm for 80.16% of the points. Mean deviations of maxillary and mandibular osteotomy lines were approximate (1.60 ± 0.93 mm vs. 1.86 ± 0.93 mm). While five patients had benign tumors, two had malignant tumors. Mean deviations of osteotomy lines was comparable between patients with benign and malignant tumors (1.48 ± 0.74 mm vs. 2.18 ± 0.77 mm). Intraoperative frozen pathology confirmed negative resection margins in all cases. No tumor recurrence or complications occurred during mean follow-up of 15.7 months (range, 6-26 months).

CONCLUSION

The combination of mixed reality technology and surgical navigation appears to be feasible, safe, and effective for tumor resection in the oral and maxillofacial region.

Artificial Intelligence in Adult Spinal Deformity

Artificial Intelligence is gaining traction in medicine for its ease of use and advancements in technology. This study evaluates the current literature on the use of artificial intelligence in adult spinal deformity.

Virtual and augmented reality for surgical training and simulation in knee arthroplasty

BACKGROUND

Immersive virtual reality (IVR), augmented reality and mixed reality form a spectrum of extended reality technology integration that has gained popularity in orthopaedics recently. This review article examines the role of extended reality technologies in knee arthroplasty.

METHODS

Existing literature on the applications of extended reality technologies in preoperative planning and intraoperative navigation were reviewed. A sample workflow of a novel IVR simulator for improving surgical training was also provided to demonstrate its utility in educating trainees on knee arthroplasty techniques.

RESULTS

Extended reality technologies enable the surgeon to visualise patient-specific anatomy in real-time, enhancing preoperative planning and providing intraoperative guidance. IVR technology has the potential to revolutionise modern surgical training and optimise surgical performance in a cost-efficient manner, with current evidence demonstrating favourable immediate skill acquisition and transfer.

CONCLUSIONS

Extended reality technologies have a myriad of potential applications in orthopaedic surgery. Further research is needed to evaluate the cost-effectiveness of its incorporation into training programmes.

Mixed reality for visualization of orthopedic surgical anatomy

In the modern era, preoperative planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing the safety and accuracy of surgical interventions. In the field of orthopedic surgery, the increase in safety and accuracy improves treatment quality and orthopedic patient outcomes. Artificial reality technologies, which include virtual reality (VR), augmented reality (AR), and mixed reality (MR), use digital images obtained from computed tomography or magnetic resonance imaging. VR replaces the user’s physical environment with one that is computer generated. AR and MR have been defined as technologies that permit the fusing of the physical with the virtual environment, enabling the user to interact with both physical and virtual objects. MR has been defined as a technology that, in contrast to AR, enables users to visualize the depth and perspective of the virtual models. We aimed to shed light on the role that MR can play in the visualization of orthopedic surgical anatomy. The literature suggests that MR could be a valuable tool in orthopedic surgeon’s hands for visualization of the anatomy. However, we remark that confusion exists in the literature concerning the characteristics of MR. Thus, a more clear description of MR is needed in orthopedic research, so that the potential of this technology can be more deeply understood.

Characterization of Structural Bone Properties through Portable Single-Sided NMR Devices: State of the Art and Future Perspectives

Nuclear Magnetic Resonance (NMR) is a well-suited methodology to study bone composition and structural properties. This is because the NMR parameters, such as the T2 relaxation time, are sensitive to the chemical and physical environment of the 1H nuclei. Although magnetic resonance imaging (MRI) allows bone structure assessment in vivo, its cost limits the suitability of conventional MRI for routine bone screening. With difficulty accessing clinically suitable exams, the diagnosis of bone diseases, such as osteoporosis, and the associated fracture risk estimation is based on the assessment of bone mineral density (BMD), obtained by the dual-energy X-ray absorptiometry (DXA). However, integrating the information about the structure of the bone with the bone mineral density has been shown to improve fracture risk estimation related to osteoporosis. Portable NMR, based on low-field single-sided NMR devices, is a promising and appealing approach to assess NMR properties of biological tissues with the aim of medical applications. Since these scanners detect the signal from a sensitive volume external to the magnet, they can be used to perform NMR measurement without the need to fit a sample inside a bore of a magnet, allowing, in principle, in vivo application. Techniques based on NMR single-sided devices have the potential to provide a high impact on the clinical routine because of low purchasing and running costs and low maintenance of such scanners. In this review, the development of new methodologies to investigate structural properties of trabecular bone exploiting single-sided NMR devices is reviewed, and current limitations and future perspectives are discussed.

Mixed Reality Assisted Percutaneous Scaphoid Fixation: A Proposed New Surgical Technique

The success of percutaneous fixation of non or minimally displaced scaphoid waist fractures is reliant on optimal placement of the screw. This can be challenging for surgeons to achieve, potentially involving a large volume of intraoperative imaging, and surgical time. Mixed reality (MR) is a new technology that allows the projection of holographic imagery within the surgeon's field of vision intraoperatively. This imagery can include surgical planning data and 3D reconstructions of a patient's anatomy that can be used in order to aid the surgeon in achieving accuracy. We describe a technique for how this novel technology might be used in the future to fix scaphoid fractures-MR assisted percutaneous scaphoid fixation. This is done using cadaveric modeling. MR assisted percutaneous scaphoid fixation may have the potential to aid surgeons in achieving an optimal guidewire placement with the ability to reduce surgical time and radiation exposure.

An update on reverse total shoulder arthroplasty: current indications, new designs, same old problems

Reverse total shoulder arthroplasty (RTSA) was originally developed because of unsatisfactory results with anatomic shoulder arthroplasty options for the majority of degenerative shoulder conditions and fractures.

After initial concerns about RTSA longevity, indications were extended to primary osteoarthritis with glenoid deficiency, massive cuff tears in younger patients, fracture, tumour and failed anatomic total shoulder replacement.

Traditional RTSA by Grammont has undergone a number of iterations such as glenoid lateralization, reduced neck-shaft angle, modular, stemless components and onlay systems.

The incidence of complications such as dislocation, notching and acromial fractures has also evolved.

Computer navigation, 3D planning and patient-specific implantation have been in use for several years and mixed-reality guided implantation is currently being trialled.

Controversies in RTSA include lateralization, stemless humeral components, subscapularis repair and treatment of acromial fractures.

Cite this article: EFORT Open Rev 2021;6:189-201. DOI: 10.1302/2058-5241.6.200085

Augmented reality for base plate component placement in reverse total shoulder arthroplasty: a feasibility study

BACKGROUND

Accurate glenoid positioning in reverse total shoulder arthroplasty (RSA) is important to achieve satisfying functional outcome and prosthesis longevity. Optimal component placement can be challenging, especially in severe glenoid deformities. The use of patient-specific instruments (PSI) and 3D computer-assisted optical tracking navigation (NAV) are already established methods to improve surgical precision. Augmented reality technology (AR) promises similar results at low cost and ease of use. With AR, the planned component placement can be superimposed to the surgical situs and shown directly in the operating field using a head mounted display. We introduce a new navigation technique using AR via head mounted display for surgical navigation in this feasibility study, aiming to improve and enhance the surgical planning.

METHODS

3D surface models of ten human scapulae were printed from computed tomography (CT) data of cadaver scapulae. Guidewire positioning of the central back of the glenoid baseplate was planned with a dedicated computer software. A hologram of the planned guidewire with dynamic navigation was then projected onto the 3D-created models of the cadaver shoulders. The registration of the plan to the anatomy was realized by digitizing the glenoid surface and the base of the coracoid with optical tracking using a fiducial marker. After navigated placement of the central guidewires, another CT imaging was recorded, and the 3D model was superimposed with the preoperative planning to analyze the deviation from the planned and executed central guides trajectory and entry point.

RESULTS

The mean deviation of the ten placed guidewires from the planned trajectory was 2.7° ± 1.3° (95% CI 1.9°; 3.6°). The mean deviation to the planned entry point of the ten placed guidewires measured 2.3 mm ± 1.1 mm (95% CI 1.5 mm; 3.1 mm).

CONCLUSION

AR may be a promising new technology for highly precise surgical execution of 3D preoperative planning in RSA.

[Application of mixed reality technique for the surgery of oral and maxillofacial tumors]

OBJECTIVE

To explore the application of mixed reality technique for the surgery of oral and maxillofacial tumors.

METHODS

In this study, patients with a diagnosis of an oral and maxillofacial tumor who were referred to Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology from December 2018 to January 2020 were selected. The preoperative contrast-enhanced computed tomography data of the patients were imported into StarAtlas Holographic Medical Imaging System (Visual 3D Corp., Beijing, China). Three-dimensional (3D) model of tumor and key structures, such as skeleton and vessels were reconstructed to three-dimensionally present the spatial relationship between them, followed with the key structures delineation and preoperative virtual surgical planning. By using mixed reality technique, the real-time 3D model was displayed stereotactically in the surgical site. While keeping sterile during operation, the surgeon could use simple gestures to adjust the 3D model, and observed the location, range, and size of tumor and the key structures adjacent to the tumor. Mixed reality technique was used to assist the operation: 3D model registration was performed for guidance before tumor excision; intraoperative real-time verification was performed during tumor exposure and after excision of the tumor. The Likert scale was used to evaluate the application of mixed reality technique after the operation.

RESULTS

Eight patients underwent mixed reality assisted tumor resection, and all of them successfully completed the operation. The average time of the 3D model registration was 12.0 minutes. In all the cases, the surgeon could intuitively and three-dimensionally observe the 3D model of the tumor and the surrounding anatomical structures, and could adjust the model during the operation. The results of the Likert scale showed that mixed reality technique got high scores in terms of perceptual accuracy, helping to locate the anatomical parts, the role of model guidance during surgery, and the potential for improving surgical safety (4.22, 4.19, 4.16, and 4.28 points respectively). Eight patients healed well without perioperative complications.

CONCLUSION

By providing real-time stereotactic visualization of anatomy of surgical site and guiding the operation process through 3D model, mixed reality technique could improve the accuracy and safety of the excision of oral and maxillofacial tumors.

[Application of mixed reality technique for the surgery of oral and maxillofacial tumors]

OBJECTIVE

To explore the application of mixed reality technique for the surgery of oral and maxillofacial tumors.

METHODS

In this study, patients with a diagnosis of an oral and maxillofacial tumor who were referred to Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology from December 2018 to January 2020 were selected. The preoperative contrast-enhanced computed tomography data of the patients were imported into StarAtlas Holographic Medical Imaging System (Visual 3D Corp., Beijing, China). Three-dimensional (3D) model of tumor and key structures, such as skeleton and vessels were reconstructed to three-dimensionally present the spatial relationship between them, followed with the key structures delineation and preoperative virtual surgical planning. By using mixed reality technique, the real-time 3D model was displayed stereotactically in the surgical site. While keeping sterile during operation, the surgeon could use simple gestures to adjust the 3D model, and observed the location, range, and size of tumor and the key structures adjacent to the tumor. Mixed reality technique was used to assist the operation: 3D model registration was performed for guidance before tumor excision; intraoperative real-time verification was performed during tumor exposure and after excision of the tumor. The Likert scale was used to evaluate the application of mixed reality technique after the operation.

RESULTS

Eight patients underwent mixed reality assisted tumor resection, and all of them successfully completed the operation. The average time of the 3D model registration was 12.0 minutes. In all the cases, the surgeon could intuitively and three-dimensionally observe the 3D model of the tumor and the surrounding anatomical structures, and could adjust the model during the operation. The results of the Likert scale showed that mixed reality technique got high scores in terms of perceptual accuracy, helping to locate the anatomical parts, the role of model guidance during surgery, and the potential for improving surgical safety (4.22, 4.19, 4.16, and 4.28 points respectively). Eight patients healed well without perioperative complications.

CONCLUSION

By providing real-time stereotactic visualization of anatomy of surgical site and guiding the operation process through 3D model, mixed reality technique could improve the accuracy and safety of the excision of oral and maxillofacial tumors.

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergence-accommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error.

[Application of mixed reality in oromaxillofacial head and neck oncology surgery: a preliminary study]

Mixed reality (MR), characterized by the ability to integrate digital data into human real feeling, is a new technique in medical imaging and surgical navigation. MR has tremendous value in surgery, but its application in oromaxillofacial head and neck oncology surgery is not yet reported. This paper reports the application of MR in oromaxillofacial head and neck oncology surgery. The merits, demerits, and present research situations and prospects of MR are further discussed.

The Role of Machine Learning in Spine Surgery: The Future Is Now

The recent influx of machine learning centered investigations in the spine surgery literature has led to increased enthusiasm as to the prospect of using artificial intelligence to create clinical decision support tools, optimize postoperative outcomes, and improve technologies used in the operating room. However, the methodology underlying machine learning in spine research is often overlooked as the subject matter is quite novel and may be foreign to practicing spine surgeons. Improper application of machine learning is a significant bioethics challenge, given the potential consequences of over- or underestimating the results of such studies for clinical decision-making processes. Proper peer review of these publications requires a baseline familiarity of the language associated with machine learning, and how it differs from classical statistical analyses. This narrative review first introduces the overall field of machine learning and its role in artificial intelligence, and defines basic terminology. In addition, common modalities for applying machine learning, including classification and regression decision trees, support vector machines, and artificial neural networks are examined in the context of examples gathered from the spine literature. Lastly, the ethical challenges associated with adapting machine learning for research related to patient care, as well as future perspectives on the potential use of machine learning in spine surgery, are discussed specifically.