An integrated augmented reality surgical navigation platform using multi-modality imaging for guidance

Progress in Computer-Assisted Navigation for Total Knee Arthroplasty in Treating Knee Osteoarthritis with Extra-Articular Deformity

Total knee arthroplasty (TKA) is a well-established treatment for end-stage knee osteoarthritis. However, in patients with concomitant extra-articular deformities, conventional TKA techniques may lead to unsatisfactory outcomes and higher complication rates. This review summarizes the application of navigated TKA for treating knee osteoarthritis with extra-articular deformities. The principles and potential benefits of computer navigation systems, including improved component alignment, soft tissue balancing, and restoration of mechanical axis, are discussed. Research studies demonstrate that navigated TKA can effectively correct deformities, relieve pain, and improve postoperative joint function and quality of life compared with conventional methods. The advantages of navigated TKA in terms of surgical precision, lower complication rates, and superior functional recovery are highlighted. Despite challenges like the learning curve and costs, navigated TKA is an increasingly indispensable tool for achieving satisfactory outcomes in TKA for knee osteoarthritis patients with extra-articular deformities.

Application of image recognition-based tracker-less augmented reality navigation system in a series of sawbone trials

BACKGROUND

This study introduced an Augmented Reality (AR) navigation system to address limitations in conventional high tibial osteotomy (HTO). The objective was to enhance precision and efficiency in HTO procedures, overcoming challenges such as inconsistent postoperative alignment and potential neurovascular damage.

METHODS

The AR-MR (Mixed Reality) navigation system, comprising HoloLens, Unity Engine, and Vuforia software, was employed for pre-clinical trials using tibial sawbone models. CT images generated 3D anatomical models, projected via HoloLens, allowing surgeons to interact through intuitive hand gestures. The critical procedure of target tracking, essential for aligning virtual and real objects, was facilitated by Vuforia's feature detection algorithm.

RESULTS

In trials, the AR-MR system demonstrated significant reductions in both preoperative planning and intraoperative times compared to conventional navigation and metal 3D-printed surgical guides. The AR system, while exhibiting lower accuracy, exhibited efficiency, making it a promising option for HTO procedures. The preoperative planning time for the AR system was notably shorter (4 min) compared to conventional navigation (30.5 min) and metal guides (75.5 min). Intraoperative time for AR lasted 8.5 min, considerably faster than that of conventional navigation (31.5 min) and metal guides (10.5 min).

CONCLUSIONS

The AR navigation system presents a transformative approach to HTO, offering a trade-off between accuracy and efficiency. Ongoing improvements, such as the incorporation of two-stage registration and pointing devices, could further enhance precision. While the system may be less accurate, its efficiency renders it a potential breakthrough in orthopedic surgery, particularly for reducing unnecessary harm and streamlining surgical procedures.

Does Freehand, Patient-specific Instrumentation or Surgical Navigation Perform Better for Allograft Reconstruction After Tumor Resection? A Preclinical Synthetic Bone Study

BACKGROUND

Joint-sparing resection of periarticular bone tumors can be challenging because of complex geometry. Successful reconstruction of periarticular bone defects after tumor resection is often performed with structural allografts to allow for joint preservation. However, achieving a size-matched allograft to fill the defect can be challenging because allograft sizes vary, they do not always match a patient's anatomy, and cutting the allograft to perfectly fit the defect is demanding.

QUESTIONS/PURPOSES

(1) Is there a difference in mental workload among the freehand, patient-specific instrumentation, and surgical navigation approaches? (2) Is there a difference in conformance (quantitative measure of deviation from the ideal bone graft), elapsed time during reconstruction, and qualitative assessment of goodness-of-fit of the allograft reconstruction among the approaches?

METHODS

Seven surgeons used three modalities in the same order (freehand, patient-specific instrumentation, and surgical navigation) to fashion synthetic bone to reconstruct a standardized bone defect. National Aeronautics and Space Administration (NASA) mental task load index questionnaires and procedure time were captured. Cone-beam CT images of the shaped allografts were used to measure conformance (quantitative measure of deviation from the ideal bone graft) to a computer-generated ideal bone graft model. Six additional (senior) surgeons blinded to modality scored the quality of fit of the allografts into the standardized tumor defect using a 10-point Likert scale. We measured conformance using the root-mean-square metric in mm and used ANOVA for multipaired comparisons (p < 0.05 was significant).

RESULTS

There was no difference in mental NASA total task load scores among the freehand, patient-specific instrumentation, and surgical navigation techniques. We found no difference in conformance root-mean-square values (mean ± SD) between surgical navigation (2 ± 0 mm; mean values have been rounded to whole numbers) and patient-specific instrumentation (2 ± 1 mm), but both showed a small improvement compared with the freehand approach (3 ± 1 mm). For freehand versus surgical navigation, the mean difference was 1 mm (95% confidence interval [CI] 0.5 to 1.1; p = 0.01). For freehand versus patient-specific instrumentation, the mean difference was 1 mm (95% CI -0.1 to 0.9; p = 0.02). For patient-specific instrumentation versus surgical navigation, the mean difference was 0 mm (95% CI -0.5 to 0.2; p = 0.82). In evaluating the goodness of fit of the shaped grafts, we found no clinically important difference between surgical navigation (median [IQR] 7 [6 to 8]) and patient-specific instrumentation (median 6 [5 to 7.8]), although both techniques had higher scores than the freehand technique did (median 3 [2 to 4]). For freehand versus surgical navigation, the difference of medians was 4 (p < 0.001). For freehand versus patient-specific instrumentation, the difference of medians was 3 (p < 0.001). For patient-specific instrumentation versus surgical navigation, the difference of medians was 1 (p = 0.03). The mean ± procedural times for freehand was 16 ± 10 minutes, patient-specific instrumentation was 14 ± 9 minutes, and surgical navigation techniques was 24 ± 8 minutes. We found no differences in procedures times across three shaping modalities (freehand versus patient-specific instrumentation: mean difference 2 minutes [95% CI 0 to 7]; p = 0.92; freehand versus surgical navigation: mean difference 8 minutes [95% CI 0 to 20]; p = 0.23; patient-specific instrumentation versus surgical navigation: mean difference 10 minutes [95% CI 1 to 19]; p = 0.12).

CONCLUSION

Based on surgical simulation to reconstruct a standardized periarticular bone defect after tumor resection, we found a possible small advantage to surgical navigation over patient-specific instrumentation based on qualitative fit, but both techniques provided slightly better conformance of the shaped graft for fit into the standardized post-tumor resection bone defect than the freehand technique did. To determine whether these differences are clinically meaningful requires further study. The surgical navigation system presented here is a product of laboratory research development, and although not ready to be widely deployed for clinical practice, it is currently being used in a research operating room setting for patient care. This new technology is associated with a learning curve, capital costs, and potential risk. The reported preliminary results are based on a preclinical synthetic bone tumor study, which is not as realistic as actual surgical scenarios.

CLINICAL RELEVANCE

Surgical navigation systems are an emerging technology in orthopaedic and reconstruction surgery, and understanding their capabilities and limitations is paramount for clinical practice. Given our preliminary findings in a small cohort study with one scenario of standardized synthetic periarticular bone tumor defects, future investigations should include different surgical scenarios using allograft and cadaveric specimens in a more realistic surgical setting.

3D bioprinting: a review and potential applications for Mohs micrographic surgery

Mohs Micrographic Surgery (MMS) is effective for treating common cutaneous malignancies, but complex repairs may often present challenges for reconstruction. This paper explores the potential of three-dimensional (3D) bioprinting in MMS, offering superior outcomes compared to traditional methods. 3D printing technologies show promise in advancing skin regeneration and refining surgical techniques in dermatologic surgery. A PubMed search was conducted using the following keywords: "Three-dimensional bioprinting" OR "3-D printing" AND "Mohs" OR "Mohs surgery" OR "Surgery." Peer-reviewed English articles discussing medical applications of 3D bioprinting were included, while non-peer-reviewed and non-English articles were excluded. Patients using 3D MMS models had lower anxiety scores (3.00 to 1.7, p < 0.0001) and higher knowledge assessment scores (5.59 or 93.25% correct responses), indicating better understanding of their procedure. Surgical residents using 3D models demonstrated improved proficiency in flap reconstructions (p = 0.002) and knowledge assessment (p = 0.001). Additionally, 3D printing offers personalized patient care through tailored surgical guides and anatomical models, reducing intraoperative time while enhancing surgical. Concurrently, efforts in tissue engineering and regenerative medicine are being explored as potential alternatives to address organ donor shortages, eliminating autografting needs. However, challenges like limited training and technological constraints persist. Integrating optical coherence tomography with 3D bioprinting may expedite grafting, but challenges remain in pre-printing grafts for complex cases. Regulatory and ethical considerations are paramount for patient safety, and further research is needed to understand long-term effects and cost-effectiveness. While promising, significant advancements are necessary for full utilization in MMS.

Exploring the potential role for extended reality in Mohs micrographic surgery

Mohs micrographic surgery (MMS) is a cornerstone of dermatological practice. Virtual reality (VR) and augmented reality (AR) technology, initially used for entertainment, have entered healthcare, offering real-time data overlaying a surgeon's view. This paper explores potential applications of VR and AR in MMS, emphasizing their advantages and limitations. We aim to identify research gaps to facilitate innovation in dermatological surgery. We conducted a PubMed search using the following: "augmented reality" OR "virtual reality" AND "Mohs" or "augmented reality" OR "virtual reality" AND "surgery." Inclusion criteria were peer-reviewed articles in English discussing these technologies in medical settings. We excluded non-peer-reviewed sources, non-English articles, and those not addressing these technologies in a medical context. VR alleviates patient anxiety and enhances patient satisfaction while serving as an educational tool. It also aids physicians by providing realistic surgical simulations. On the other hand, AR assists in real-time lesion analysis, optimizing incision planning, and refining margin control during surgery. Both of these technologies offer remote guidance for trainee residents, enabling real-time learning and oversight and facilitating synchronous teleconsultations. These technologies may transform dermatologic surgery, making it more accessible and efficient. However, further research is needed to validate their effectiveness, address potential challenges, and optimize seamless integration. All in all, AR and VR enhance real-world environments with digital data, offering real-time surgical guidance and medical insights. By exploring the potential integration of these technologies in MMS, our study identifies avenues for further research to thoroughly understand the role of these technologies to redefine dermatologic surgery, elevating precision, surgical outcomes, and patient experiences.

Research on augmented reality navigation of in vitro fenestration of stent-graft based on deep learning and virtual-real registration

OBJECTIVES

In vitro fenestration of stent-graft (IVFS) demands high-precision navigation methods to achieve optimal surgical outcomes. This study aims to propose an augmented reality (AR) navigation method for IVFS, which can provide in situ overlay display to locate fenestration positions.

METHODS

We propose an AR navigation method to assist doctors in performing IVFS. A deep learning-based aorta segmentation algorithm is used to achieve automatic and rapid aorta segmentation. The Vuforia-based virtual-real registration and marker recognition algorithm are integrated to ensure accurate in situ AR image.

RESULTS

The proposed method can provide three-dimensional in situ AR image, and the fiducial registration error after virtual-real registration is 2.070 mm. The aorta segmentation experiment obtains dice similarity coefficient of 91.12% and Hausdorff distance of 2.59, better than conventional algorithms before improvement.

CONCLUSIONS

The proposed method can intuitively and accurately locate fenestration positions, and therefore can assist doctors in performing IVFS.

Registration Sanity Check for AR-guided Surgical Interventions: Experience From Head and Face Surgery

Achieving and maintaining proper image registration accuracy is an open challenge of image-guided surgery. This work explores and assesses the efficacy of a registration sanity check method for augmented reality-guided navigation (AR-RSC), based on the visual inspection of virtual 3D models of landmarks. We analyze the AR-RSC sensitivity and specificity by recruiting 36 subjects to assess the registration accuracy of a set of 114 AR images generated from camera images acquired during an AR-guided orthognathic intervention. Translational or rotational errors of known magnitude up to ±1.5 mm/±15.5°, were artificially added to the image set in order to simulate different registration errors. This study analyses the performance of AR-RSC when varying (1) the virtual models selected for misalignment evaluation (e. g., the model of brackets, incisor teeth, and gingival margins in our experiment), (2) the type (translation/rotation) of registration error, and (3) the level of user experience in using AR technologies. Results show that: 1) the sensitivity and specificity of the AR-RSC depends on the virtual models (globally, a median true positive rate of up to 79.2% was reached with brackets, and a median true negative rate of up to 64.3% with incisor teeth), 2) there are error components that are more difficult to identify visually, 3) the level of user experience does not affect the method. In conclusion, the proposed AR-RSC, tested also in the operating room, could represent an efficient method to monitor and optimize the registration accuracy during the intervention, but special attention should be paid to the selection of the AR data chosen for the visual inspection of the registration accuracy.

Public health implications of computer-aided diagnosis and treatment technologies in breast cancer care

Breast cancer remains a significant public health issue, being a leading cause of cancer-related mortality among women globally. Timely diagnosis and efficient treatment are crucial for enhancing patient outcomes, reducing healthcare burdens and advancing community health. This systematic review, following the PRISMA guidelines, aims to comprehensively synthesize the recent advancements in computer-aided diagnosis and treatment for breast cancer. The study covers the latest developments in image analysis and processing, machine learning and deep learning algorithms, multimodal fusion techniques and radiation therapy planning and simulation. The results of the review suggest that machine learning, augmented and virtual reality and data mining are the three major research hotspots in breast cancer management. Moreover, this paper discusses the challenges and opportunities for future research in this field. The conclusion highlights the importance of computer-aided techniques in the management of breast cancer and summarizes the key findings of the review.

Exposure to Extended Reality and Artificial Intelligence-Based Manifestations: A Primer on the Future of Hip and Knee Arthroplasty

BACKGROUND

Software-infused services, from robot-assisted and wearable technologies to artificial intelligence (AI)-laden analytics, continue to augment clinical orthopaedics - namely hip and knee arthroplasty. Extended reality (XR) tools, which encompass augmented reality, virtual reality, and mixed reality technology, represent a new frontier for expanding surgical horizons to maximize technical education, expertise, and execution. The purpose of this review is to critically detail and evaluate the recent developments surrounding XR in the field of hip and knee arthroplasty and to address potential future applications as they relate to AI.

METHODS

In this narrative review surrounding XR, we discuss (1) definitions, (2) techniques, (3) studies, (4) current applications, and (5) future directions. We highlight XR subsets (augmented reality, virtual reality, and mixed reality) as they relate to AI in the increasingly digitized ecosystem within hip and knee arthroplasty.

RESULTS

A narrative review of the XR orthopaedic ecosystem with respect to XR developments is summarized with specific emphasis on hip and knee arthroplasty. The XR as a tool for education, preoperative planning, and surgical execution is discussed with future applications dependent upon AI to potentially obviate the need for robotic assistance and preoperative advanced imaging without sacrificing accuracy.

CONCLUSION

In a field where exposure is critical to clinical success, XR represents a novel stand-alone software-infused service that optimizes technical education, execution, and expertise but necessitates integration with AI and previously validated software solutions to offer opportunities that improve surgical precision with or without the use of robotics and computed tomography-based imaging.

Using augmented reality to guide bone conduction device implantation

Exact placement of bone conduction implants requires avoidance of critical structures. Existing guidance technologies for intraoperative placement have lacked widespread adoption given accessibility challenges and significant cognitive loading. The purpose of this study is to examine the application of augmented reality (AR) guided surgery on accuracy, duration, and ease on bone conduction implantation. Five surgeons surgically implanted two different types of conduction implants on cadaveric specimens with and without AR projection. Pre- and postoperative computer tomography scans were superimposed to calculate centre-to-centre distances and angular accuracies. Wilcoxon signed-rank testing was used to compare centre-to-centre (C-C) and angular accuracies between the control and experimental arms. Additionally, projection accuracy was derived from the distance between the bony fiducials and the projected fiducials using image guidance coordinates. Both operative time (4.3 ± 1.2 min. vs. 6.6 ± 3.5 min., p = 0.030) and centre-to-centre distances surgery (1.9 ± 1.6 mm vs. 9.0 ± 5.3 mm, p < 0.001) were significantly less in augmented reality guided surgery. The difference in angular accuracy, however, was not significantly different. The overall average distance between the bony fiducial markings and the AR projected fiducials was 1.7 ± 0.6 mm. With direct intraoperative reference, AR-guided surgery enhances bone conduction implant placement while reduces operative time when compared to conventional surgical planning.

Skull-Base Surgery—A Narrative Review on Current Approaches and Future Developments in Surgical Navigation

Surgical navigation technology combines patient imaging studies with intraoperative real-time data to improve surgical precision and patient outcomes. The navigation workflow can also include preoperative planning, which can reliably simulate the intended resection and reconstruction. The advantage of this approach in skull-base surgery is that it guides access into a complex three-dimensional area and orients tumors intraoperatively with regard to critical structures, such as the orbit, carotid artery and brain. This enhances a surgeon’s capabilities to preserve normal anatomy while resecting tumors with adequate margins. The aim of this narrative review is to outline the state of the art and the future directions of surgical navigation in the skull base, focusing on the advantages and pitfalls of this technique. We will also present our group experience in this field, within the frame of the current research trends.

Visualization, registration and tracking techniques for augmented reality guided surgery: a review

Augmented reality (AR) surgical navigation has developed rapidly in recent years. This paper reviews and analyzes the visualization, registration, and tracking techniques used in AR surgical navigation systems, as well as the application of these AR systems in different surgical fields. The types of AR visualization are divided into two categories ofin situvisualization and nonin situvisualization. The rendering contents of AR visualization are various. The registration methods include manual registration, point-based registration, surface registration, marker-based registration, and calibration-based registration. The tracking methods consist of self-localization, tracking with integrated cameras, external tracking, and hybrid tracking. Moreover, we describe the applications of AR in surgical fields. However, most AR applications were evaluated through model experiments and animal experiments, and there are relatively few clinical experiments, indicating that the current AR navigation methods are still in the early stage of development. Finally, we summarize the contributions and challenges of AR in the surgical fields, as well as the future development trend. Despite the fact that AR-guided surgery has not yet reached clinical maturity, we believe that if the current development trend continues, it will soon reveal its clinical utility.

Treatment of Skull Base Diseases: A Multidisciplinary Challenge

The skull base has always been regarded as a frontier by surgeons and radiation oncologists since it represents the interface between the intracranial and the extracranial compartment and hosts several critical anatomical structures with an extremely complex and close relationship [...].

Stereotactic co-axial projection imaging for augmented reality neuronavigation: a proof-of-concept study

BACKGROUND

Lack of intuitiveness and poor hand-eye coordination present a major technical challenge in neurosurgical navigation.

METHODS

We developed an integrated dexterous stereotactic co-axial projection imaging (sCPI) system featuring orthotopic image projection for augmented reality (AR) neurosurgical navigation. The performance characteristics of the sCPI system, including projection resolution and navigation accuracy, were quantitatively verified. The resolution of the sCPI was tested with a USAF1951 resolution test chart. The stereotactic navigation accuracy of the sCPI was measured using a calibration panel with a 7×7 circle array pattern. In benchtop validation, the navigation accuracy of the sCPI and the BrainLab Kick Navigation Station was compared using a skull phantom with 8 intracranial targets. Finally, we demonstrated the potential clinical application of sCPI through a clinical trial.

RESULTS

The resolution test showed that the resolution of the sCPI was 1.3 mm. In a stereotactic navigation accuracy test, the maximum and minimum error of the sCPI was 2.9 and 0.3 mm, and the mean error was 1.5 mm. The stereotactic navigation accuracy test also showed that the navigation error of the sCPI would increase with the pitch and yaw angle, but there was no obvious difference in navigation errors caused by different yaw directions, which meant that the navigation error is unbiased across all directions. The benchtop validation showed that the average navigation errors for the sCPI system and the Kick Navigation Station were 1.4±0.8 and 1.8±0.7 mm, the medians were 1.3 and 1.9 mm, and the average preparation times were 3 min 24 sec and 6 min 8 sec, respectively. The clinical feasibility of sCPI-assisted neurosurgical navigation was demonstrated in a clinical study. In comparison with the BrainLab device, the sCPI system required less time for preoperative preparation and enhanced the clinician experience in intraoperative visualization and navigation.

CONCLUSIONS

The sCPI technique can be potentially used in many surgical applications for intuitive visualization of medical information and intraoperative guidance of surgical trajectories.

Development of a cadaveric head and neck cancer model and three-dimensional analysis of margins in surgical navigation-aided ablations

INTRODUCTION

The adequacy of the surgical resection is the main controllable variable that is in the hands of the surgical team. There exists an unmet need to increase the rate of negative margins, particularly in cancers invading the craniofacial area. The study aimed 1) at developing a gross tumor model to be utilized for research, educational, and training purposes and 2) establishing the 3-dimensional relationship between the outer surface of the surgical specimen and tumor surface and test the effect of guiding ablations on cadavers with surgical navigation (SN).

MATERIAL AND METHODS

Seven cadaver heads were employed to create 24 craniofacial tumor models. Simulation of tumor resections was performed by 8 surgeons. Fourteen and 10 resections were performed with and without SN-guidance, respectively. Gross specimens underwent computed tomography and 3-dimensional analysis through dedicated software. Task load was assessed through a validated questionnaire. Tumor model reliability was studied based on visual analogue scale rate by surgeons and radiologists.

RESULTS

SN reduced the rate of margin involvement, particularly by decreasing the percentage of the gross specimen outer surface involvement in areas uncovered by normal bony structures. The workload of SN-aided ablations was found to be medium-to-somewhat-high. Tumor model reliability was deemed satisfactory except for the extension to bony structures.

CONCLUSIONS

A gross tumor model for head and neck cancers involving the craniofacial area was developed and resulted satisfactorily reliable from both a surgical and radiologic standpoint. SN reduced the rate of margin involvement, particularly by improving delineation of bone-uncovered areas.

Augmented Reality and Intraoperative Navigation in Sinonasal Malignancies: A Preclinical Study

Objective

To report the first use of a novel projected augmented reality (AR) system in open sinonasal tumor resections in preclinical models and to compare the AR approach with an advanced intraoperative navigation (IN) system.

Methods

Four tumor models were created. Five head and neck surgeons participated in the study performing virtual osteotomies. Unguided, AR, IN, and AR + IN simulations were performed. Statistical comparisons between approaches were obtained. Intratumoral cut rate was the main outcome. The groups were also compared in terms of percentage of intratumoral, close, adequate, and excessive distances from the tumor. Information on a wearable gaze tracker headset and NASA Task Load Index questionnaire results were analyzed as well.

Results

A total of 335 cuts were simulated. Intratumoral cuts were observed in 20.7%, 9.4%, 1.2,% and 0% of the unguided, AR, IN, and AR + IN simulations, respectively (p < 0.0001). The AR was superior than the unguided approach in univariate and multivariate models. The percentage of time looking at the screen during the procedures was 55.5% for the unguided approaches and 0%, 78.5%, and 61.8% in AR, IN, and AR + IN, respectively (p < 0.001). The combined approach significantly reduced the screen time compared with the IN procedure alone.

Conclusion

We reported the use of a novel AR system for oncological resections in open sinonasal approaches, with improved margin delineation compared with unguided techniques. AR improved the gaze-toggling drawback of IN. Further refinements of the AR system are needed before translating our experience to clinical practice.