3D Visualization and Augmented Reality for Orthopedics

Facial augmented reality based on hierarchical optimization of similarity aspect graph

BACKGROUND

The existing face matching method requires a point cloud to be drawn on the real face for registration, which results in low registration accuracy due to the irregular deformation of the patient's skin that makes the point cloud have many outlier points.

METHODS

This work proposes a non-contact pose estimation method based on similarity aspect graph hierarchical optimization. The proposed method constructs a distance-weighted and triangular-constrained similarity measure to describe the similarity between views by automatically identifying the 2D and 3D feature points of the face. A mutual similarity clustering method is proposed to construct a hierarchical aspect graph with 3D pose as nodes. A Monte Carlo tree search strategy is used to search the hierarchical aspect graph for determining the optimal pose of the facial 3D model, so as to realize the accurate registration of the facial 3D model and the real face.

RESULTS

The proposed method was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with four advanced pose calibration methods. The proposed method obtained average fusion errors of 1.13 ± 0.20 mm and 0.92 ± 0.08 mm in head phantom and volunteer experiments, respectively, which exhibits the best fusion performance among all comparison methods.

CONCLUSIONS

Our experiments proved the effectiveness of the proposed pose estimation method in facial augmented reality.

Single-center experience with Knee+™ augmented reality navigation system in primary total knee arthroplasty

BACKGROUND

Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years, as they enhance precision compared to conventional hardware. The expansion of computer assistance is evolving with the employment of augmented reality. Yet, the accuracy of augmented reality navigation systems has not been determined.

AIM

To examine the accuracy of component alignment and restoration of the affected limb's mechanical axis in primary total knee arthroplasty (TKA), utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon.

METHODS

From May 2021 to December 2021, 30 patients, 25 women and five men, underwent a primary unilateral TKA. Revision cases were excluded. A preoperative radiographic procedure was performed to evaluate the limb's axial alignment. All patients were operated on by the same team, without a tourniquet, utilizing three distinct prostheses with the assistance of the Knee+™ augmented reality navigation system in every operation. Postoperatively, the same radiographic exam protocol was executed to evaluate the implants' position, orientation and coronal plane alignment. We recorded measurements in 3 stages regarding femoral varus and flexion, tibial varus and posterior slope. Firstly, the expected values from the Augmented Reality system were documented. Then we calculated the same values after each cut and finally, the same measurements were recorded radiologically after the operations. Concerning statistical analysis, Lin's concordance correlation coefficient was estimated, while Wilcoxon Signed Rank Test was performed when needed.

RESULTS

A statistically significant difference was observed regarding mean expected values and radiographic measurements for femoral flexion measurements only (Z score = 2.67, P value = 0.01). Nonetheless, this difference was statistically significantly lower than 1 degree (Z score = -4.21, P value < 0.01). In terms of discrepancies in the calculations of expected values and controlled measurements, a statistically significant difference between tibial varus values was detected (Z score = -2.33, P value = 0.02), which was also statistically significantly lower than 1 degree (Z score = -4.99, P value < 0.01).

CONCLUSION

The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized. Augmented reality navigation systems can bolster orthopaedic surgeons' accuracy in achieving precise axial alignment. However, further research is required to further evaluate their efficacy and potential.

Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery

Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon's perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present a marker-less approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a median of 100% successful registrations, a median target registration error of 2.7 mm, a median screw trajectory error of 1.6°and a median screw entry point error of 2.3 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a median target registration error of 1.0 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.

Virtual and Augmented Reality in Spine Surgery: An Era of Immersive Healthcare

In the dynamic realm of spinal surgery, the integration of virtual reality (VR) and augmented reality (AR) technologies is heralding a transformative era. These cutting-edge tools are not only reshaping the training landscape for surgical trainees, offering immersive and interactive experiences but are also enhancing the surgical precision of seasoned professionals in the operating room. While the potential of VR and AR is vast, their adoption is tempered by significant costs and challenges in seamless integration. As the spinal surgery community looks ahead, it becomes imperative to emphasize the validation, reliability, and thorough cost-benefit analysis of these technologies. This article delves into the current applications, benefits, challenges, and future trajectories of VR and AR in spinal surgery, underscoring their pivotal role in the evolution of immersive healthcare.

Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery

Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.

User's image perception improved strategy and application of augmented reality systems in smart medical care: A review

BACKGROUND

Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures.

METHODS

This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field.

RESULTS

A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment.

CONCLUSION

This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.

From Diagnosis to Therapy: The Role of Virtual and Augmented Reality in Orthopaedic Trauma Surgery

By reducing procedure-related problems, advancements in computer-assisted surgery (CAS) and surgical training aim to boost operative precision and enhance patient safety. Orthopaedic training and practice have started to change as a result of the incorporation of reality technologies like virtual reality (VR), augmented reality (AR), and mixed reality (MR) into CAS. Today's trainees can engage in realistic and highly involved operational simulations without supervision. With the coronavirus disease 2019 (COVID-19) pandemic, there is a greater need for breakthrough technology adoption. VR is an interactive technology that enables personalised care and could support successful patient-centered rehabilitation. It is a valid and trustworthy evaluation method for determining joint range of motion, function, and balance in physical rehabilitation. It may make it possible to customise care, encourage patients, boost compliance, and track their advancement. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. As little patient observation is needed, this may lessen the workload clinicians must bear. The ability to use it for home-based therapy is often available commercially as well.

The objectives of this review are to evaluate the technology available, comprehend the available evidence regarding the benefit, and take into account implementation problems in clinical practice. The use of this technology, its practical and moral ramifications, and how it will affect orthopaedic doctors and their patients are also covered. This review offers a current and thorough analysis of the reality technologies and their uses in orthopaedic surgery.

Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions

PURPOSE OF REVIEW

Imaging technologies (X-ray, CT, MRI, and ultrasound) have revolutionized orthopedic surgery, allowing for the more efficient diagnosis, monitoring, and treatment of musculoskeletal aliments. The current review investigates recent literature surrounding the impact of augmented reality (AR) imaging technologies on orthopedic surgery. In particular, it investigates the impact that AR technologies may have on provider cognitive burden, operative times, occupational radiation exposure, and surgical precision and outcomes.

RECENT FINDINGS

Many AR technologies have been shown to lower provider cognitive burden and reduce operative time and radiation exposure while improving surgical precision in pre-clinical cadaveric and sawbones models. So far, only a few platforms focusing on pedicle screw placement have been approved by the FDA. These technologies have been implemented clinically with mixed results when compared to traditional free-hand approaches. It remains to be seen if current AR technologies can deliver upon their multitude of promises, and the ability to do so seems contingent upon continued technological progress. Additionally, the impact of these platforms will likely be highly conditional on clinical indication and provider type. It remains unclear if AR will be broadly accepted and utilized or if it will be reserved for niche indications where it adds significant value. One thing is clear, orthopedics' high utilization of pre- and intra-operative imaging, combined with the relative ease of tracking rigid structures like bone as compared to soft tissues, has made it the clear beachhead market for AR technologies in medicine.

Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review

BACKGROUND & AIM

Utilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice.

METHODS & RESULTS

PRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits.

CONCLUSION

Whilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons' specific needs should increase uptake.

Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy

PURPOSE

During the minimally invasive knee surgery, surgeons insert surgical instruments and arthroscopy through small incisions, and implement treatment assisted by 2D arthroscopic images. However, this 2D arthroscopic navigation faces several problems. Firstly, the guidance information is displayed on a screen away from the surgical area, which makes hand/eye coordination difficult. Secondly, the small incision limits the surgeons to view the internal knee structures only from an arthroscopic camera. In addition, arthroscopic images commonly appear obscure visions.

METHODS

To solve these problems, we proposed a novel in-situ augmented reality navigation system with the enhanced arthroscopic information. Firstly, intraoperative anatomical locations were obtained by using arthroscopic images and arthroscopy calibration. Secondly, tissue properties-based model deformation method was proposed to update the 3D preoperative knee model with anatomical location information. Then, the updated model was further rendered with glasses-free real 3D display for achieving the global in-situ augmented reality view. In addition, virtual arthroscopic images were generated from the updated preoperative model to provide the anatomical information of the operation area.

RESULTS

Experimental results demonstrated that virtual arthroscopic images could reflect the correct structure information with a mean error of 0.32 mm. Compared with 2D arthroscopic navigation, the proposed augmented reality navigation reduced the targeting errors by 2.10 mm and 2.70 mm for the experiments of knee phantom and in-vitro swine knee, respectively.

CONCLUSION

Our navigation method is helpful for minimally invasive knee surgery since it can provide the global in-situ information and detail anatomical information.

Evaluation of 3-Dimensional Magnetic Resonance Imaging (3D MRI) in Diagnosing Anterior Talofibular Ligament Injury

BACKGROUND It is challenging to entirely show the anterior talofibular ligament (ATFL) and accurately diagnose ATFL injury with traditional 2-dimensional (2D) magnetic resonance imaging (MRI). With the introduction of 3.0T MRI, a 3-dimensional (3D) MRI sequence can achieve images with high spatial resolution. This study aimed to evaluate the accuracy of 3D MRI and compare it with 2D MRI in diagnosing ATFL injury. MATERIAL AND METHODS This was a prospective study in which 45 patients with clinically suspected ATFL injury underwent 2D MRI, 3D MRI, and 3D model reconstruction followed by arthroscopic surgery between February 2018 and April 2019. Two radiologists who had over 11 and 13 years of musculoskeletal experience assessed the injury of ATFL in consensus without any clinical clues. Arthroscopic surgery results were the standard reference of MRI accuracy. RESULTS The 3D MRI results of ATFL injury showed the sensitivity of diagnosis of complete tears of 83% and specificity of 82%. The partial tears diagnosis sensitivity was 78%, and specificity was 100%. The sensitivity of diagnosis of sprains was 100%, and the specificity was 97%. The 3D MRI accuracy of diagnosis was 98% for no injury, 98% for sprain, 91% for partial tear, and 82% for complete tear. The difference in the diagnosis of sprain and partial tears by 3D MRI and 2D MRI was statistically significant (P<0.05). A 3D reconstruction model was successfully created for all patients. CONCLUSIONS 3D MRI may be a reliable and accurate method to detect ATFL injury. The 3D reconstruction model using 3D MRI sequences has excellent prospects in application.

Augmented reality in neurosurgical navigation: a survey

BACKGROUND

Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems.

METHODS

In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey.

RESULTS

Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks.

CONCLUSIONS

Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.