1
|
Shao L, Fu T, Lin Y, Xiao D, Ai D, Zhang T, Fan J, Song H, Yang J. Facial augmented reality based on hierarchical optimization of similarity aspect graph. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 248:108108. [PMID: 38461712 DOI: 10.1016/j.cmpb.2024.108108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Revised: 02/05/2024] [Accepted: 02/29/2024] [Indexed: 03/12/2024]
Abstract
BACKGROUND The existing face matching method requires a point cloud to be drawn on the real face for registration, which results in low registration accuracy due to the irregular deformation of the patient's skin that makes the point cloud have many outlier points. METHODS This work proposes a non-contact pose estimation method based on similarity aspect graph hierarchical optimization. The proposed method constructs a distance-weighted and triangular-constrained similarity measure to describe the similarity between views by automatically identifying the 2D and 3D feature points of the face. A mutual similarity clustering method is proposed to construct a hierarchical aspect graph with 3D pose as nodes. A Monte Carlo tree search strategy is used to search the hierarchical aspect graph for determining the optimal pose of the facial 3D model, so as to realize the accurate registration of the facial 3D model and the real face. RESULTS The proposed method was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with four advanced pose calibration methods. The proposed method obtained average fusion errors of 1.13 ± 0.20 mm and 0.92 ± 0.08 mm in head phantom and volunteer experiments, respectively, which exhibits the best fusion performance among all comparison methods. CONCLUSIONS Our experiments proved the effectiveness of the proposed pose estimation method in facial augmented reality.
Collapse
Affiliation(s)
- Long Shao
- School of Computer Science & Technology, Beijing Institute of Technology, Beijing 100081, China
| | - Tianyu Fu
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China.
| | - Yucong Lin
- School of Medical Technology, Beijing Institute of Technology, Beijing 100081, China
| | - Deqiang Xiao
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Danni Ai
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| | - Tao Zhang
- Department of Stomatology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100730, China
| | - Jingfan Fan
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China.
| | - Hong Song
- School of Computer Science & Technology, Beijing Institute of Technology, Beijing 100081, China.
| | - Jian Yang
- Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
| |
Collapse
|
2
|
Sakellariou E, Alevrogiannis P, Alevrogianni F, Galanis A, Vavourakis M, Karampinas P, Gavriil P, Vlamis J, Alevrogiannis S. Single-center experience with Knee+™ augmented reality navigation system in primary total knee arthroplasty. World J Orthop 2024; 15:247-256. [PMID: 38596188 PMCID: PMC10999969 DOI: 10.5312/wjo.v15.i3.247] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/29/2023] [Revised: 01/15/2024] [Accepted: 03/04/2024] [Indexed: 03/15/2024] Open
Abstract
BACKGROUND Computer-assisted systems obtained an increased interest in orthopaedic surgery over the last years, as they enhance precision compared to conventional hardware. The expansion of computer assistance is evolving with the employment of augmented reality. Yet, the accuracy of augmented reality navigation systems has not been determined. AIM To examine the accuracy of component alignment and restoration of the affected limb's mechanical axis in primary total knee arthroplasty (TKA), utilizing an augmented reality navigation system and to assess whether such systems are conspicuously fruitful for an accomplished knee surgeon. METHODS From May 2021 to December 2021, 30 patients, 25 women and five men, underwent a primary unilateral TKA. Revision cases were excluded. A preoperative radiographic procedure was performed to evaluate the limb's axial alignment. All patients were operated on by the same team, without a tourniquet, utilizing three distinct prostheses with the assistance of the Knee+™ augmented reality navigation system in every operation. Postoperatively, the same radiographic exam protocol was executed to evaluate the implants' position, orientation and coronal plane alignment. We recorded measurements in 3 stages regarding femoral varus and flexion, tibial varus and posterior slope. Firstly, the expected values from the Augmented Reality system were documented. Then we calculated the same values after each cut and finally, the same measurements were recorded radiologically after the operations. Concerning statistical analysis, Lin's concordance correlation coefficient was estimated, while Wilcoxon Signed Rank Test was performed when needed. RESULTS A statistically significant difference was observed regarding mean expected values and radiographic measurements for femoral flexion measurements only (Z score = 2.67, P value = 0.01). Nonetheless, this difference was statistically significantly lower than 1 degree (Z score = -4.21, P value < 0.01). In terms of discrepancies in the calculations of expected values and controlled measurements, a statistically significant difference between tibial varus values was detected (Z score = -2.33, P value = 0.02), which was also statistically significantly lower than 1 degree (Z score = -4.99, P value < 0.01). CONCLUSION The results indicate satisfactory postoperative coronal alignment without outliers across all three different implants utilized. Augmented reality navigation systems can bolster orthopaedic surgeons' accuracy in achieving precise axial alignment. However, further research is required to further evaluate their efficacy and potential.
Collapse
Affiliation(s)
- Evangelos Sakellariou
- The 3rd Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, KAT General Hospital, Athens 14561, Greece
| | - Panagiotis Alevrogiannis
- The 1st Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, Attikon General Hospital, Athens 14561, Greece
| | - Fani Alevrogianni
- Department of Anesthesiology, KAT General Hospital, Athens 14561, Greece
| | - Athanasios Galanis
- The 3rd Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, KAT General Hospital, Athens 14561, Greece
| | - Michail Vavourakis
- The 3rd Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, KAT General Hospital, Athens 14561, Greece
| | - Panagiotis Karampinas
- The 3rd Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, KAT General Hospital, Athens 14561, Greece
| | - Panagiotis Gavriil
- The 1st Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, Attikon General Hospital, Athens 14561, Greece
| | - John Vlamis
- The 3rd Department of Orthopaedic Surgery, National & Kapodistrian University of Athens, KAT General Hospital, Athens 14561, Greece
| | - Stavros Alevrogiannis
- Department of Robotic Hip & Knee Orthopaedic Surgery, Metropolitan General Hospital, Athens 14561, Greece
| |
Collapse
|
3
|
Liebmann F, von Atzigen M, Stütz D, Wolf J, Zingg L, Suter D, Cavalcanti NA, Leoty L, Esfandiari H, Snedeker JG, Oswald MR, Pollefeys M, Farshad M, Fürnstahl P. Automatic registration with continuous pose updates for marker-less surgical navigation in spine surgery. Med Image Anal 2024; 91:103027. [PMID: 37992494 DOI: 10.1016/j.media.2023.103027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 10/29/2023] [Accepted: 11/09/2023] [Indexed: 11/24/2023]
Abstract
Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon's perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present a marker-less approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a median of 100% successful registrations, a median target registration error of 2.7 mm, a median screw trajectory error of 1.6°and a median screw entry point error of 2.3 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a median target registration error of 1.0 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.
Collapse
Affiliation(s)
- Florentin Liebmann
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland; Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland.
| | - Marco von Atzigen
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland; Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland
| | - Dominik Stütz
- Computer Vision and Geometry Group, ETH Zurich, Zurich, Switzerland
| | - Julian Wolf
- Product Development Group, ETH Zurich, Zurich, Switzerland
| | - Lukas Zingg
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Daniel Suter
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Nicola A Cavalcanti
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland; Department of Orthopedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Laura Leoty
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Hooman Esfandiari
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Jess G Snedeker
- Laboratory for Orthopaedic Biomechanics, ETH Zurich, Zurich, Switzerland
| | - Martin R Oswald
- Computer Vision and Geometry Group, ETH Zurich, Zurich, Switzerland; Computer Vision Lab, University of Amsterdam, Amsterdam, Netherlands
| | - Marc Pollefeys
- Computer Vision and Geometry Group, ETH Zurich, Zurich, Switzerland; Microsoft Mixed Reality and AI Zurich Lab, Zurich, Switzerland
| | - Mazda Farshad
- Department of Orthopedics, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| | - Philipp Fürnstahl
- Research in Orthopedic Computer Science, Balgrist University Hospital, University of Zurich, Zurich, Switzerland
| |
Collapse
|
4
|
Hasan S, Miller A, Higginbotham D, Saleh ES, McCarty S. Virtual and Augmented Reality in Spine Surgery: An Era of Immersive Healthcare. Cureus 2023; 15:e43964. [PMID: 37746489 PMCID: PMC10516442 DOI: 10.7759/cureus.43964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/23/2023] [Indexed: 09/26/2023] Open
Abstract
In the dynamic realm of spinal surgery, the integration of virtual reality (VR) and augmented reality (AR) technologies is heralding a transformative era. These cutting-edge tools are not only reshaping the training landscape for surgical trainees, offering immersive and interactive experiences but are also enhancing the surgical precision of seasoned professionals in the operating room. While the potential of VR and AR is vast, their adoption is tempered by significant costs and challenges in seamless integration. As the spinal surgery community looks ahead, it becomes imperative to emphasize the validation, reliability, and thorough cost-benefit analysis of these technologies. This article delves into the current applications, benefits, challenges, and future trajectories of VR and AR in spinal surgery, underscoring their pivotal role in the evolution of immersive healthcare.
Collapse
Affiliation(s)
- Sazid Hasan
- Department of Orthopedic Surgery, Oakland University William Beaumont School of Medicine, Rochester, USA
| | - Alex Miller
- Department of Orthopedic Surgery, Beaumont Hospital, Royal Oak, USA
| | - Devan Higginbotham
- Department of Orthopedic Surgery, Wayne State University Detroit Medical Center, Detroit, USA
| | - Ehab S Saleh
- Department of Orthopedic Surgery, Oakland University William Beaumont School of Medicine, Rochester, USA
| | - Scott McCarty
- Department of Orthopedic Surgery, Wayne State University Detroit Medical Center, Detroit, USA
| |
Collapse
|
5
|
León-Muñoz VJ, Moya-Angeler J, López-López M, Lisón-Almagro AJ, Martínez-Martínez F, Santonja-Medina F. Integration of Square Fiducial Markers in Patient-Specific Instrumentation and Their Applicability in Knee Surgery. J Pers Med 2023; 13:jpm13050727. [PMID: 37240897 DOI: 10.3390/jpm13050727] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 04/23/2023] [Accepted: 04/23/2023] [Indexed: 05/28/2023] Open
Abstract
Computer technologies play a crucial role in orthopaedic surgery and are essential in personalising different treatments. Recent advances allow the usage of augmented reality (AR) for many orthopaedic procedures, which include different types of knee surgery. AR assigns the interaction between virtual environments and the physical world, allowing both to intermingle (AR superimposes information on real objects in real-time) through an optical device and allows personalising different processes for each patient. This article aims to describe the integration of fiducial markers in planning knee surgeries and to perform a narrative description of the latest publications on AR applications in knee surgery. Augmented reality-assisted knee surgery is an emerging set of techniques that can increase accuracy, efficiency, and safety and decrease the radiation exposure (in some surgical procedures, such as osteotomies) of other conventional methods. Initial clinical experience with AR projection based on ArUco-type artificial marker sensors has shown promising results and received positive operator feedback. Once initial clinical safety and efficacy have been demonstrated, the continued experience should be studied to validate this technology and generate further innovation in this rapidly evolving field.
Collapse
Affiliation(s)
- Vicente J León-Muñoz
- Department of Orthopaedic Surgery and Traumatology, Hospital General Universitario Reina Sofía, 30003 Murcia, Spain
- Instituto de Cirugía Avanzada de la Rodilla (ICAR), 30005 Murcia, Spain
| | - Joaquín Moya-Angeler
- Department of Orthopaedic Surgery and Traumatology, Hospital General Universitario Reina Sofía, 30003 Murcia, Spain
- Instituto de Cirugía Avanzada de la Rodilla (ICAR), 30005 Murcia, Spain
| | - Mirian López-López
- Subdirección General de Tecnologías de la Información, Servicio Murciano de Salud, 30100 Murcia, Spain
| | - Alonso J Lisón-Almagro
- Department of Orthopaedic Surgery and Traumatology, Hospital General Universitario Reina Sofía, 30003 Murcia, Spain
| | - Francisco Martínez-Martínez
- Department of Orthopaedic Surgery and Traumatology, Hospital Clínico Universitario Virgen de la Arrixaca, 30120 Murcia, Spain
| | - Fernando Santonja-Medina
- Department of Orthopaedic Surgery and Traumatology, Hospital Clínico Universitario Virgen de la Arrixaca, 30120 Murcia, Spain
- Department of Surgery, Pediatrics and Obstetrics & Gynecology, Faculty of Medicine, University of Murcia, 30120 Murcia, Spain
| |
Collapse
|
6
|
Jiang J, Zhang J, Sun J, Wu D, Xu S. User's image perception improved strategy and application of augmented reality systems in smart medical care: A review. Int J Med Robot 2023; 19:e2497. [PMID: 36629798 DOI: 10.1002/rcs.2497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2022] [Revised: 12/26/2022] [Accepted: 01/06/2023] [Indexed: 01/12/2023]
Abstract
BACKGROUND Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures. METHODS This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field. RESULTS A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment. CONCLUSION This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system.
Collapse
Affiliation(s)
- Jingang Jiang
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin, Heilongjiang, China.,Robotics & Its Engineering Research Center, Harbin University of Science and Technology, Harbin, Heilongjiang, China
| | - Jiawei Zhang
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin, Heilongjiang, China
| | - Jianpeng Sun
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin, Heilongjiang, China
| | - Dianhao Wu
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin, Heilongjiang, China
| | - Shuainan Xu
- Key Laboratory of Advanced Manufacturing and Intelligent Technology, Ministry of Education, Harbin University of Science and Technology, Harbin, Heilongjiang, China
| |
Collapse
|
7
|
Augmented reality (AR) and fracture mapping model on middle-aged femoral neck fracture: A proof-of-concept towards interactive visualization. MEDICINE IN NOVEL TECHNOLOGY AND DEVICES 2022. [DOI: 10.1016/j.medntd.2022.100190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
|
8
|
Gupta A, Ambade R. From Diagnosis to Therapy: The Role of Virtual and Augmented Reality in Orthopaedic Trauma Surgery. Cureus 2022; 14:e29099. [PMID: 36249662 PMCID: PMC9557249 DOI: 10.7759/cureus.29099] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2022] [Accepted: 09/12/2022] [Indexed: 11/28/2022] Open
Abstract
By reducing procedure-related problems, advancements in computer-assisted surgery (CAS) and surgical training aim to boost operative precision and enhance patient safety. Orthopaedic training and practice have started to change as a result of the incorporation of reality technologies like virtual reality (VR), augmented reality (AR), and mixed reality (MR) into CAS. Today's trainees can engage in realistic and highly involved operational simulations without supervision. With the coronavirus disease 2019 (COVID-19) pandemic, there is a greater need for breakthrough technology adoption. VR is an interactive technology that enables personalised care and could support successful patient-centered rehabilitation. It is a valid and trustworthy evaluation method for determining joint range of motion, function, and balance in physical rehabilitation. It may make it possible to customise care, encourage patients, boost compliance, and track their advancement. AR supplementation in orthopaedic surgery has shown promising results in pre-clinical settings, with improvements in surgical accuracy and reproducibility, decreased operating times, and less radiation exposure. As little patient observation is needed, this may lessen the workload clinicians must bear. The ability to use it for home-based therapy is often available commercially as well. The objectives of this review are to evaluate the technology available, comprehend the available evidence regarding the benefit, and take into account implementation problems in clinical practice. The use of this technology, its practical and moral ramifications, and how it will affect orthopaedic doctors and their patients are also covered. This review offers a current and thorough analysis of the reality technologies and their uses in orthopaedic surgery.
Collapse
|
9
|
Augmented Reality (AR) in Orthopedics: Current Applications and Future Directions. Curr Rev Musculoskelet Med 2021; 14:397-405. [PMID: 34751894 DOI: 10.1007/s12178-021-09728-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/27/2021] [Indexed: 01/05/2023]
Abstract
PURPOSE OF REVIEW Imaging technologies (X-ray, CT, MRI, and ultrasound) have revolutionized orthopedic surgery, allowing for the more efficient diagnosis, monitoring, and treatment of musculoskeletal aliments. The current review investigates recent literature surrounding the impact of augmented reality (AR) imaging technologies on orthopedic surgery. In particular, it investigates the impact that AR technologies may have on provider cognitive burden, operative times, occupational radiation exposure, and surgical precision and outcomes. RECENT FINDINGS Many AR technologies have been shown to lower provider cognitive burden and reduce operative time and radiation exposure while improving surgical precision in pre-clinical cadaveric and sawbones models. So far, only a few platforms focusing on pedicle screw placement have been approved by the FDA. These technologies have been implemented clinically with mixed results when compared to traditional free-hand approaches. It remains to be seen if current AR technologies can deliver upon their multitude of promises, and the ability to do so seems contingent upon continued technological progress. Additionally, the impact of these platforms will likely be highly conditional on clinical indication and provider type. It remains unclear if AR will be broadly accepted and utilized or if it will be reserved for niche indications where it adds significant value. One thing is clear, orthopedics' high utilization of pre- and intra-operative imaging, combined with the relative ease of tracking rigid structures like bone as compared to soft tissues, has made it the clear beachhead market for AR technologies in medicine.
Collapse
|
10
|
Ettinger M. [Digital technologies]. DER ORTHOPADE 2021; 50:785-788. [PMID: 34505909 DOI: 10.1007/s00132-021-04156-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 08/12/2021] [Indexed: 10/20/2022]
Affiliation(s)
- Max Ettinger
- MHH Klinik für Orthopädie, Annastift, Anna-von-Borries-Str. 1-7, 30625, Hannover, Deutschland.
| |
Collapse
|
11
|
Ha J, Parekh P, Gamble D, Masters J, Jun P, Hester T, Daniels T, Halai M. Opportunities and challenges of using augmented reality and heads-up display in orthopaedic surgery: A narrative review. J Clin Orthop Trauma 2021; 18:209-215. [PMID: 34026489 PMCID: PMC8131920 DOI: 10.1016/j.jcot.2021.04.031] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 03/28/2021] [Accepted: 04/29/2021] [Indexed: 12/16/2022] Open
Abstract
BACKGROUND & AIM Utilization of augmented reality (AR) and heads-up displays (HUD) to aid orthopaedic surgery has the potential to benefit surgeons and patients alike through improved accuracy, safety, and educational benefits. With the COVID-19 pandemic, the opportunity for adoption of novel technology is more relevant. The aims are to assess the technology available, to understand the current evidence regarding the benefit and to consider challenges to implementation in clinical practice. METHODS & RESULTS PRISMA guidelines were used to filter the literature. Of 1004 articles returned the following exclusion criteria were applied: 1) reviews/commentaries 2) unrelated to orthopaedic surgery 3) use of other AR wearables beyond visual aids leaving 42 papers for review.This review illustrates benefits including enhanced accuracy and reduced time of surgery, reduced radiation exposure and educational benefits. CONCLUSION Whilst there are obstacles to overcome, there are already reports of technology being used. As with all novel technologies, a greater understanding of the learning curve is crucial, in addition to shielding our patients from this learning curve. Improvements in usability and implementing surgeons' specific needs should increase uptake.
Collapse
Affiliation(s)
- Joon Ha
- Queen Elizabeth Hospital, London, UK,Corresponding author.
| | | | | | - James Masters
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), UK
| | - Peter Jun
- University of Alberta, Edmonton, Canada
| | | | | | - Mansur Halai
- St Michael's Hospital, University of Toronto, Canada
| |
Collapse
|
12
|
Chen F, Cui X, Han B, Liu J, Zhang X, Liao H. Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 201:105952. [PMID: 33561710 DOI: 10.1016/j.cmpb.2021.105952] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Accepted: 01/21/2021] [Indexed: 06/12/2023]
Abstract
PURPOSE During the minimally invasive knee surgery, surgeons insert surgical instruments and arthroscopy through small incisions, and implement treatment assisted by 2D arthroscopic images. However, this 2D arthroscopic navigation faces several problems. Firstly, the guidance information is displayed on a screen away from the surgical area, which makes hand/eye coordination difficult. Secondly, the small incision limits the surgeons to view the internal knee structures only from an arthroscopic camera. In addition, arthroscopic images commonly appear obscure visions. METHODS To solve these problems, we proposed a novel in-situ augmented reality navigation system with the enhanced arthroscopic information. Firstly, intraoperative anatomical locations were obtained by using arthroscopic images and arthroscopy calibration. Secondly, tissue properties-based model deformation method was proposed to update the 3D preoperative knee model with anatomical location information. Then, the updated model was further rendered with glasses-free real 3D display for achieving the global in-situ augmented reality view. In addition, virtual arthroscopic images were generated from the updated preoperative model to provide the anatomical information of the operation area. RESULTS Experimental results demonstrated that virtual arthroscopic images could reflect the correct structure information with a mean error of 0.32 mm. Compared with 2D arthroscopic navigation, the proposed augmented reality navigation reduced the targeting errors by 2.10 mm and 2.70 mm for the experiments of knee phantom and in-vitro swine knee, respectively. CONCLUSION Our navigation method is helpful for minimally invasive knee surgery since it can provide the global in-situ information and detail anatomical information.
Collapse
Affiliation(s)
- Fang Chen
- Department of Computer Science and Engineering, Nanjing University of Aeronautics and Astronautics, MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, Nanjing, China.
| | - Xiwen Cui
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Boxuan Han
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Jia Liu
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Xinran Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Hongen Liao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China.
| |
Collapse
|
13
|
Xu Y, He L, Han Y, Duan D, Ouyang L. Evaluation of 3-Dimensional Magnetic Resonance Imaging (3D MRI) in Diagnosing Anterior Talofibular Ligament Injury. Med Sci Monit 2021; 27:e927920. [PMID: 33453097 PMCID: PMC7816539 DOI: 10.12659/msm.927920] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022] Open
Abstract
BACKGROUND It is challenging to entirely show the anterior talofibular ligament (ATFL) and accurately diagnose ATFL injury with traditional 2-dimensional (2D) magnetic resonance imaging (MRI). With the introduction of 3.0T MRI, a 3-dimensional (3D) MRI sequence can achieve images with high spatial resolution. This study aimed to evaluate the accuracy of 3D MRI and compare it with 2D MRI in diagnosing ATFL injury. MATERIAL AND METHODS This was a prospective study in which 45 patients with clinically suspected ATFL injury underwent 2D MRI, 3D MRI, and 3D model reconstruction followed by arthroscopic surgery between February 2018 and April 2019. Two radiologists who had over 11 and 13 years of musculoskeletal experience assessed the injury of ATFL in consensus without any clinical clues. Arthroscopic surgery results were the standard reference of MRI accuracy. RESULTS The 3D MRI results of ATFL injury showed the sensitivity of diagnosis of complete tears of 83% and specificity of 82%. The partial tears diagnosis sensitivity was 78%, and specificity was 100%. The sensitivity of diagnosis of sprains was 100%, and the specificity was 97%. The 3D MRI accuracy of diagnosis was 98% for no injury, 98% for sprain, 91% for partial tear, and 82% for complete tear. The difference in the diagnosis of sprain and partial tears by 3D MRI and 2D MRI was statistically significant (P<0.05). A 3D reconstruction model was successfully created for all patients. CONCLUSIONS 3D MRI may be a reliable and accurate method to detect ATFL injury. The 3D reconstruction model using 3D MRI sequences has excellent prospects in application.
Collapse
Affiliation(s)
- Yan Xu
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China (mainland)
| | - Lei He
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China (mainland)
| | - Yu Han
- Department of Orthopedic Surgery, Shanghai Key Laboratory of Orthopedic Implants, Shanghai Ninth People's Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, China (mainland)
| | - Deyu Duan
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China (mainland)
| | - Liu Ouyang
- Department of Orthopedics, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, Hubei, China (mainland)
| |
Collapse
|
14
|
Liu T, Tai Y, Zhao C, Wei L, Zhang J, Pan J, Shi J. Augmented reality in neurosurgical navigation: a survey. Int J Med Robot 2020; 16:e2160. [PMID: 32890440 DOI: 10.1002/rcs.2160] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 08/19/2020] [Accepted: 08/29/2020] [Indexed: 11/12/2022]
Abstract
BACKGROUND Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems. METHODS In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey. RESULTS Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks. CONCLUSIONS Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Tao Liu
- Yunnan Key Lab of Opto-electronic Information Technology, Yunnan Normal University, Kunming, China
| | - Yonghang Tai
- Yunnan Key Lab of Opto-electronic Information Technology, Yunnan Normal University, Kunming, China
| | - Chengming Zhao
- Yunnan Key Lab of Opto-electronic Information Technology, Yunnan Normal University, Kunming, China
| | - Lei Wei
- Institute for Intelligent Systems Research and Innovation, Deakin University, Geelong, VIC, Australia
| | - Jun Zhang
- Yunnan Key Lab of Opto-electronic Information Technology, Yunnan Normal University, Kunming, China
| | - Junjun Pan
- State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China
| | - Junsheng Shi
- Yunnan Key Lab of Opto-electronic Information Technology, Yunnan Normal University, Kunming, China
| |
Collapse
|
15
|
Zhang M, Wang L. Re: Francesco Porpiglia, Enrico Checcucci, Daniele Amparore, et al. Three-dimensional Augmented Reality Robot-assisted Partial Nephrectomy in Case of Complex Tumours (PADUA ≥ 10): A New Intraoperative Tool Overcoming the Ultrasound Guidance. Eur Urol. In press. https://doi.org/10.1016/j.eururo.2019.11.024. Eur Urol 2020; 77:e161-e162. [PMID: 32303382 DOI: 10.1016/j.eururo.2020.03.037] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2020] [Accepted: 03/23/2020] [Indexed: 11/24/2022]
Affiliation(s)
- Mengda Zhang
- Department of Urology, The Third Xiangya Hospital, Central South University, Changsha, Hunan, P.R. China; Department of Urology, Xiangya Hospital, Central South University, Changsha, Hunan, P.R. China
| | - Long Wang
- Department of Urology, The Third Xiangya Hospital, Central South University, Changsha, Hunan, P.R. China; Department of Urology, Xiangya Hospital, Central South University, Changsha, Hunan, P.R. China.
| |
Collapse
|
16
|
Verhey JT, Haglin JM, Verhey EM, Hartigan DE. Virtual, augmented, and mixed reality applications in orthopedic surgery. Int J Med Robot 2020; 16:e2067. [DOI: 10.1002/rcs.2067] [Citation(s) in RCA: 56] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Revised: 12/11/2019] [Accepted: 12/11/2019] [Indexed: 12/18/2022]
Affiliation(s)
| | | | | | - David E. Hartigan
- Mayo Clinic Alix School of Medicine Scottsdale Arizona
- Department of Orthopedic SurgeryMayo Clinic Phoenix Arizona
| |
Collapse
|
17
|
Ferrari V, Carbone M, Condino S, Cutolo F. Are augmented reality headsets in surgery a dead end? Expert Rev Med Devices 2019; 16:999-1001. [PMID: 31725347 DOI: 10.1080/17434440.2019.1693891] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Affiliation(s)
- Vincenzo Ferrari
- Dipartimento di Ingegneria dell'Informazione, Università di Pisa, Pisa, Italy
| | - Marina Carbone
- Dipartimento di Ingegneria dell'Informazione, Università di Pisa, Pisa, Italy
| | - Sara Condino
- Dipartimento di Ingegneria dell'Informazione, Università di Pisa, Pisa, Italy
| | - Fabrizio Cutolo
- Dipartimento di Ingegneria dell'Informazione, Università di Pisa, Pisa, Italy
| |
Collapse
|