Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference

Is Overlain Display a Right Choice for AR Navigation? A Qualitative Study of Head-Mounted Augmented Reality Surgical Navigation on Accuracy for Large-Scale Clinical Deployment

BACKGROUND

During the course of the past two decades, head-mounted augmented reality surgical navigation (HMARSN) systems have been increasingly employed in a variety of surgical specialties as a result of both advancements in augmented reality-related technologies and surgeons' desires to overcome some drawbacks inherent to conventional surgical navigation systems. In the present time, most experimental HMARSN systems adopt overlain display (OD) that overlay virtual models and planned routes of surgical tools on corresponding physical tissues, organs, lesions, and so forth, in a surgical field so as to provide surgeons with an intuitive and direct view to gain better hand-eye coordination as well as avoid attention shift and loss of sight (LOS), among other benefits during procedures. Yet, its system accuracy, which is the most crucial performance indicator of any surgical navigation system, is difficult to ascertain because it is highly subjective and user-dependent. Therefore, the aim of this study was to review presently available experimental OD HMARSN systems qualitatively, explore how their system accuracy is affected by overlain display, and find out if such systems are suited to large-scale clinical deployment.

METHOD

We searched PubMed and ScienceDirect with the following terms: head mounted augmented reality surgical navigation, and 445 records were returned in total. After screening and eligibility assessment, 60 papers were finally analyzed. Specifically, we focused on how their accuracies were defined and measured, as well as whether such accuracies are stable in clinical practice and competitive with corresponding commercially available systems.

RESULTS AND CONCLUSIONS

The primary findings are that the system accuracy of OD HMARSN systems is seriously affected by a transformation between the spaces of the user's eyes and the surgical field, because measurement of the transformation is heavily individualized and user-dependent. Additionally, the transformation itself is potentially subject to changes during surgical procedures, and hence unstable. Therefore, OD HMARSN systems are not suitable for large-scale clinical deployment.

RibMR - A Mixed Reality Visualization System for Rib Fracture Localization in Surgical Stabilization of Rib Fractures: Phantom, Preclinical, and Clinical Studies

In surgical stabilization of rib fractures (SSRF), the current standard relies on preoperative CT imaging and often incorporates ultrasound (US) imaging. As an alternative, mixed reality (MR) technology holds promise for improving rib fracture localization. This study presents an MR-based visualization system designed for SSRF in a clinical setting. We developed RibMR - a visualization system using an MR head-mounted display that projects a patient-specific 3D hologram onto the patient. RibMR enables the localization of rib fractures in relation to the patient's anatomy. We conducted phantom study using a human mannequin, a preclinical study with two healthy patients, and clinical study with two patients to evaluate RibMR and compared it to US practice. RibMR localized rib fractures with an average accuracy of 0.38 ± 0.21 cm in phantom, 3.75 ± 2.45 cm in preclinical, and 1.47 ± 1.33 cm in clinical studies. RibMR took an average time (minutes) of 4.42 ± 0.98 for the phantom, 8.03 ± 3.67 for the preclinical, and 8.76 ± 0.65 for the clinical studies. Compared to US, RibMR located more fractures, including fractures occluded by other structures, with higher accuracy, faster speed, and improved localization rate. All participating surgeons provided positive feedback regarding accuracy, visualization quality, and usability. RibMR enabled accurate and time-efficient localization of rib fractures and showed better performance compared to US. RibMR is a promising alternative to US for localizing rib fractures in SSRF.

Digital technologies to unlock safe and sustainable opportunities for medical device and healthcare sectors with a focus on the combined use of digital twin and extended reality applications: A review

Medical devices have increased in complexity where there is a pressing need to consider design thinking and specialist training for manufacturers, healthcare and sterilization providers, and regulators. Appropriately addressing this consideration will positively inform end-to-end supply chain and logistics, production, processing, sterilization, safety, regulation, education, sustainability and circularity. There are significant opportunities to innovate and to develop appropriate digital tools to help unlock efficiencies in these important areas. This constitutes the first paper to create an awareness of and to define different digital technologies for informing and enabling medical device production from a holistic end-to-end life cycle perspective. It describes the added-value of using digital innovations to meet emerging opportunities for many disposable and reusable medical devices. It addresses the value of accessing and using integrated multi-actor HUBs that combine academia, industry, healthcare, regulators and society to help meet these opportunities. Such as cost-effective access to specialist pilot facilities and expertise that converges digital innovation, material science, biocompatibility, sterility assurance, business model and sustainability. It highlights the marked gap in academic R&D activities (PRISMA review of best publications conducted between January 2010 and January 2024) and the actual list of U.S. FDA's approved and marketed artificial intelligence/machine learning (AI/ML), and augmented reality/virtual reality (AR/VR) enabled-medical devices for different healthcare applications. Bespoke examples of benefits underlying future use of digital tools includes potential implementation of machine learning for supporting and enabling parametric release of sterilized products through efficient monitoring of critical process data (complying with ISO 11135:2014) that would benefit stakeholders. This paper also focuses on the transformative potential of combining digital twin with extended reality innovations to inform efficiencies in medical device design thinking, supply chain and training to inform patient safety, circularity and sustainability.

Acetabular Revision with McMinn Cup: Development and Application of a Patient-Specific Targeting Device

BACKGROUND

Surgeries of severe periacetabular bone defects (Paprosky ≥ 2B) are a major challenge in current practice. Although solutions are available for this serious clinical problem, they all have their disadvantages as well as their advantages. An alternative method of reconstructing such extensive defects was the use of a cup with a stem to solve these revision situations. As the instrumentation offered is typically designed for scenarios where a significant bone defect is not present, our unique technique has been developed for implantation in cases where reference points are missing. Our hypothesis was that a targeting device designed based on the CT scan of a patient's pelvis could facilitate the safe insertion of the guiding wire.

METHODS

Briefly, our surgical solution consists of a two-step operation. If periacetabular bone loss was found to be more significant during revision surgery, all implants were removed, and two titanium marker screws in the anterior iliac crest were percutaneously inserted. Next, by applying the metal artifact removal (MAR) algorithm, a CT scan of the pelvis was performed. Based on that, the dimensions and positioning of the cup to be inserted were determined, and a patient-specific 3D printed targeting device made of biocompatible material was created to safely insert the guidewire, which is essential to the implantation process.

RESULTS

In this study, medical, engineering, and technical tasks related to the design, the surgical technique, and experiences from 17 surgical cases between February 2018 and July 2021 are reported. There were no surgical complications in any cases. The implant had to be removed due to septic reasons (independently from the technique) in a single case, consistent with the septic statistics for this type of surgery. There was not any perforation of the linea terminalis of the pelvis due to the guiding method. The wound healing of patients was uneventful, and the implant was fixed securely. Following rehabilitation, the joints were able to bear weight again. After one to four years of follow-up, the patient satisfaction level was high, and the gait function of the patients improved a lot in all cases.

CONCLUSIONS

Our results show that CT-based virtual surgical planning and, based on it, the use of a patient-specific 3D printed aiming device is a reliable method for major hip surgeries with significant bone loss. This technique has also made it possible to perform these operations with minimal X-ray exposure.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Systematic review of techniques used to validate the registration of augmented-reality images using a head-mounted device to navigate surgery

Augmented-reality (AR) head-mounted devices (HMD) allow the wearer to have digital images superposed on to their field of vision. They are being used to superpose annotations on to the surgical field akin to a navigation system. This review examines published validation studies on HMD-AR systems, their reported protocols, and outcomes. The aim was to establish commonalities and an acceptable registration outcome. Multiple databases were systematically searched for relevant articles between January 2015 and January 2021. Studies that examined the registration of AR content using a HMD to guide surgery were eligible for inclusion. The country of origin, year of publication, medical specialty, HMD device, software, and method of registration, were recorded. A meta-analysis of the mean registration error was conducted. A total of 4784 papers were identified, of which 23 met the inclusion criteria. They included studies using HoloLens (Microsoft) (n = 22) and nVisor ST60 (NVIS Inc) (n = 1). Sixty-six per cent of studies were in hard tissue specialties. Eleven studies reported registration errors using pattern markers (mean (SD) 2.6 (1.8) mm), and four reported registration errors using surface markers (mean (SD) 3.8 (3.7) mm). Three studies reported registration errors using manual alignment (mean (SD) 2.2 (1.3) mm). The majority of studies in this review used in-house software with a variety of registration methods and reported errors. The mean registration error calculated in this study can be considered as a minimum acceptable standard. It should be taken into consideration when procedural applications are selected.

UltrARsound: in situ visualization of live ultrasound images using HoloLens 2

Purpose

Augmented Reality (AR) has the potential to simplify ultrasound (US) examinations which usually require a skilled and experienced sonographer to mentally align narrow 2D cross-sectional US images in the 3D anatomy of the patient. This work describes and evaluates a novel approach to track retroreflective spheres attached to the US probe using an inside-out technique with the AR glasses HoloLens 2. Finally, live US images are displayed in situ on the imaged anatomy.

Methods

The Unity application UltrARsound performs spatial tracking of the US probe and attached retroreflective markers using the depth camera integrated into the AR glasses—thus eliminating the need for an external tracking system. Additionally, a Kalman filter is implemented to improve the noisy measurements of the camera. US images are streamed wirelessly via the PLUS toolkit to HoloLens 2. The technical evaluation comprises static and dynamic tracking accuracy, frequency and latency of displayed images.

Results

Tracking is performed with a median accuracy of 1.98 mm/1.81\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document}∘ for the static setting when using the Kalman filter. In a dynamic scenario, the median error was 2.81 mm/1.70\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$^\circ $$\end{document}∘. The tracking frequency is currently limited to 20 Hz. 83% of the displayed US images had a latency lower than 16 ms.

Conclusions

In this work, we showed that spatial tracking of retroreflective spheres with the depth camera of HoloLens 2 is feasible, achieving a promising accuracy for in situ visualization of live US images. For tracking, no additional hardware nor modifications to HoloLens 2 are required making it a cheap and easy-to-use approach. Moreover, a minimal latency of displayed images enables a real-time perception for the sonographer.

HoloLens 1 vs. HoloLens 2: Improvements in the New Model for Orthopedic Oncological Interventions

This work analyzed the use of Microsoft HoloLens 2 in orthopedic oncological surgeries and compares it to its predecessor (Microsoft HoloLens 1). Specifically, we developed two equivalent applications, one for each device, and evaluated the augmented reality (AR) projection accuracy in an experimental scenario using phantoms based on two patients. We achieved automatic registration between virtual and real worlds using patient-specific surgical guides on each phantom. They contained a small adaptor for a 3D-printed AR marker, the characteristic patterns of which were easily recognized using both Microsoft HoloLens devices. The newest model improved the AR projection accuracy by almost 25%, and both of them yielded an RMSE below 3 mm. After ascertaining the enhancement of the second model in this aspect, we went a step further with Microsoft HoloLens 2 and tested it during the surgical intervention of one of the patients. During this experience, we collected the surgeons’ feedback in terms of comfortability, usability, and ergonomics. Our goal was to estimate whether the improved technical features of the newest model facilitate its implementation in actual surgical scenarios. All of the results point to Microsoft HoloLens 2 being better in all the aspects affecting surgical interventions and support its use in future experiences.

Augmented Reality to Assist Skin Paddle Harvesting in Osteomyocutaneous Fibular Flap Reconstructive Surgery: A Pilot Evaluation on a 3D-Printed Leg Phantom

Background

Augmented Reality (AR) represents an evolution of navigation-assisted surgery, providing surgeons with a virtual aid contextually merged with the real surgical field. We recently reported a case series of AR-assisted fibular flap harvesting for mandibular reconstruction. However, the registration accuracy between the real and the virtual content needs to be systematically evaluated before widely promoting this tool in clinical practice. In this paper, after description of the AR based protocol implemented for both tablet and HoloLens 2 smart glasses, we evaluated in a first test session the achievable registration accuracy with the two display solutions, and in a second test session the success rate in executing the AR-guided skin paddle incision task on a 3D printed leg phantom.

Methods

From a real computed tomography dataset, 3D virtual models of a human leg, including fibula, arteries and skin with planned paddle profile for harvesting, were obtained. All virtual models were imported into Unity software to develop a marker-less AR application suitable to be used both via tablet and via HoloLens 2 headset. The registration accuracy for both solutions was verified on a 3D printed leg phantom obtained from the virtual models, by repeatedly applying the tracking function and computing pose deviations between the AR-projected virtual skin paddle profile and the real one transferred to the phantom via a CAD/CAM cutting guide. The success rate in completing the AR-guided task of skin paddle harvesting was evaluated using CAD/CAM templates positioned on the phantom model surface.

Results

On average, the marker-less AR protocol showed comparable registration errors (ranging within 1-5 mm) for tablet-based and HoloLens-based solution. Registration accuracy seems to be quite sensitive to ambient light conditions. We found a good success rate in completing the AR-guided task within an error margin of 4 mm (97% and 100% for tablet and HoloLens, respectively). All subjects reported greater usability and ergonomics for HoloLens 2 solution.

Conclusions

Results revealed that the proposed marker-less AR based protocol may guarantee a registration error within 1-5 mm for assisting skin paddle harvesting in the clinical setting. Optimal lightening conditions and further improvement of marker-less tracking technologies have the potential to increase the efficiency and precision of this AR-assisted reconstructive surgery.

Surgical Navigation, Augmented Reality, and 3D Printing for Hard Palate Adenoid Cystic Carcinoma En-Bloc Resection: Case Report and Literature Review

Adenoid Cystic Carcinoma is a rare and aggressive tumor representing less than 1% of head and neck cancers. This malignancy often arises from the minor salivary glands, being the palate its most common location. Surgical en-bloc resection with clear margins is the primary treatment. However, this location presents a limited line of sight and a high risk of injuries, making the surgical procedure challenging. In this context, technologies such as intraoperative navigation can become an effective tool, reducing morbidity and improving the safety and accuracy of the procedure. Although their use is extended in fields such as neurosurgery, their application in maxillofacial surgery has not been widely evidenced. One reason is the need to rigidly fixate a navigation reference to the patient, which often entails an invasive setup. In this work, we studied three alternative and less invasive setups using optical tracking, 3D printing and augmented reality. We evaluated their precision in a patient-specific phantom, obtaining errors below 1 mm. The optimum setup was finally applied in a clinical case, where the navigation software was used to guide the tumor resection. Points were collected along the surgical margins after resection and compared with the real ones identified in the postoperative CT. Distances of less than 2 mm were obtained in 90% of the samples. Moreover, the navigation provided confidence to the surgeons, who could then undertake a less invasive and more conservative approach. The postoperative CT scans showed adequate resection margins and confirmed that the patient is free of disease after two years of follow-up.

Augmented Reality as a Tool to Guide PSI Placement in Pelvic Tumor Resections

Patient-specific instruments (PSIs) have become a valuable tool for osteotomy guidance in complex surgical scenarios such as pelvic tumor resection. They provide similar accuracy to surgical navigation systems but are generally more convenient and faster. However, their correct placement can become challenging in some anatomical regions, and it cannot be verified objectively during the intervention. Incorrect installations can result in high deviations from the planned osteotomy, increasing the risk of positive resection margins. In this work, we propose to use augmented reality (AR) to guide and verify PSIs placement. We designed an experiment to assess the accuracy provided by the system using a smartphone and the HoloLens 2 and compared the results with the conventional freehand method. The results showed significant differences, where AR guidance prevented high osteotomy deviations, reducing maximal deviation of 54.03 mm for freehand placements to less than 5 mm with AR guidance. The experiment was performed in two versions of a plastic three-dimensional (3D) printed phantom, one including a silicone layer to simulate tissue, providing more realism. We also studied how differences in shape and location of PSIs affect their accuracy, concluding that those with smaller sizes and a homogeneous target surface are more prone to errors. Our study presents promising results that prove AR's potential to overcome the present limitations of PSIs conveniently and effectively.

Complex Bone Tumors of the Trunk-The Role of 3D Printing and Navigation in Tumor Orthopedics: A Case Series and Review of the Literature

The combination of 3D printing and navigation promises improvements in surgical procedures and outcomes for complex bone tumor resection of the trunk, but its features have rarely been described in the literature. Five patients with trunk tumors were surgically treated in our institution using a combination of 3D printing and navigation. The main process includes segmentation, virtual modeling and build preparation, as well as quality assessment. Tumor resection was performed with navigated instruments. Preoperative planning supported clear margin multiplanar resections with intraoperatively adaptable real-time visualization of navigated instruments. The follow-up ranged from 2–15 months with a good functional result. The present results and the review of the current literature reflect the trend and the diverse applications of 3D printing in the medical field. 3D printing at hospital sites is often not standardized, but regulatory aspects may serve as disincentives. However, 3D printing has an increasing impact on precision medicine, and we are convinced that our process represents a valuable contribution in the context of patient-centered individual care.

Mixed reality and three dimensional printed models for resection of maxillary tumor: a case report

In the field of oral and maxillofacial surgery, many institutions have recently begun using three-dimensional printers to create three-dimensional models and mixed reality in a variety of diseases. Here, we report the actual situation model which we made using three-dimensional printer from virtual operation data and the resection that was performed while grasping a maxillary benign tumor and neighboring three-dimensional structure by designing an application for Microsoft® HoloLens, and using Mixed Reality surgery support during the procedure.

Point-of-care manufacturing: a single university hospital's initial experience

Background

The integration of 3D printing technology in hospitals is evolving toward production models such as point-of-care manufacturing. This study aims to present the results of the integration of 3D printing technology in a manufacturing university hospital.

Methods

Observational, descriptive, retrospective, and monocentric study of 907 instances of 3D printing from November 2015 to March 2020. Variables such as product type, utility, time, or manufacturing materials were analyzed.

Results

Orthopedic Surgery and Traumatology, Oral and Maxillofacial Surgery, and Gynecology and Obstetrics are the medical specialties that have manufactured the largest number of processes. Working and printing time, as well as the amount of printing material, is different for different types of products and input data. The most common printing material was polylactic acid, although biocompatible resin was introduced to produce surgical guides. In addition, the hospital has worked on the co-design of custom-made implants with manufacturing companies and has also participated in tissue bio-printing projects.

Conclusions

The integration of 3D printing in a university hospital allows identifying the conceptual evolution to “point-of-care manufacturing.”

Conceptual evolution of 3D printing in orthopedic surgery and traumatology: from "do it yourself" to "point of care manufacturing"

BACKGROUND

3D printing technology in hospitals facilitates production models such as point-of-care manufacturing. Orthopedic Surgery and Traumatology is the specialty that can most benefit from the advantages of these tools. The purpose of this study is to present the results of the integration of 3D printing technology in a Department of Orthopedic Surgery and Traumatology and to identify the productive model of the point-of-care manufacturing as a paradigm of personalized medicine.

METHODS

Observational, descriptive, retrospective and monocentric study of a total of 623 additive manufacturing processes carried out in a Department of Orthopedic Surgery and Traumatology from November 2015 to March 2020. Variables such as product type, utility, time or materials for manufacture were analyzed.

RESULTS

The areas of expertise that have performed more processes are Traumatology, Reconstructive and Orthopedic Oncology. Pre-operative planning is their primary use. Working and 3D printing hours, as well as the amount of 3D printing material used, vary according to the type of product or material delivered to perform the process. The most commonly used 3D printing material for manufacturing is polylactic acid, although biocompatible resin has been used to produce surgical guides. In addition, the hospital has worked on the co-design of customized implants with manufacturing companies.

CONCLUSIONS

The integration of 3D printing in a Department of Orthopedic Surgery and Traumatology allows identifying the conceptual evolution from "Do-It-Yourself" to "POC manufacturing".

Patient-specific desktop 3D-printed guides for pelvic tumour resection surgery: a precision study on cadavers

PURPOSE

3D-printed patient-specific instruments have become a useful tool to improve accuracy in pelvic tumour resections. However, their correct placement can be challenging in some regions due to the morphology of the bone, so it is essential to be aware of the possible placement errors in each region. In this study, we characterize these errors in common pelvic osteotomies.

METHODS

We conducted an experiment with 9 cadaveric specimens, for which we acquired a pre-operative computed tomography scan. Small PSIs were designed for each case following a realistic surgical approach for four regions of the pelvis: iliac crest (C), supra-acetabular (S), ischial (I), and pubic (P). Final surgical placement was based on a post-operative scan. The resulting positions were compared with pre-operative planning, obtaining translations, rotations, and maximum osteotomy deviations in a local reference frame defined based on the bone's morphology.

RESULTS

Mean translations and rotations in the direction of the osteotomy plane were as follows: C = 5.3 mm, 6.7°; S = 1.8 mm, 5.1°; I = 1.5 mm, 3.4°; P = 1.8 mm, 3.5°. Mean translations in the remaining axes were below 2 mm. Maximum osteotomy deviations (75% of cases) were below 11.8 mm in C (7.8 mm for half-length), 7.8 mm in S (5.5 mm for half-length), 5.5 mm in I, and 3.7 mm in P.

CONCLUSION

We have characterized placement errors for small PSIs in four regions of the pelvis. Our results show high errors in C and S PSIs in the direction of the resection plane's normal, and thus large osteotomy deviations. Deviations in short osteotomies in S, I and P and placement errors in the remaining directions were low. The PSIs used in this study are biocompatible and can be produced with a desktop 3D printer, thus minimizing manufacturing cost.

Combining Augmented Reality and 3D Printing to Improve Surgical Workflows in Orthopedic Oncology: Smartphone Application and Clinical Evaluation

During the last decade, orthopedic oncology has experienced the benefits of computerized medical imaging to reduce human dependency, improving accuracy and clinical outcomes. However, traditional surgical navigation systems do not always adapt properly to this kind of interventions. Augmented reality (AR) and three-dimensional (3D) printing are technologies lately introduced in the surgical environment with promising results. Here we present an innovative solution combining 3D printing and AR in orthopedic oncological surgery. A new surgical workflow is proposed, including 3D printed models and a novel AR-based smartphone application (app). This app can display the patient’s anatomy and the tumor’s location. A 3D-printed reference marker, designed to fit in a unique position of the affected bone tissue, enables automatic registration. The system has been evaluated in terms of visualization accuracy and usability during the whole surgical workflow. Experiments on six realistic phantoms provided a visualization error below 3 mm. The AR system was tested in two clinical cases during surgical planning, patient communication, and surgical intervention. These results and the positive feedback obtained from surgeons and patients suggest that the combination of AR and 3D printing can improve efficacy, accuracy, and patients’ experience.

Mixed reality as a novel tool for diagnostic and surgical navigation in orthopaedics

Purpose

This study presents a novel surgical navigation tool developed in mixed reality environment for orthopaedic surgery. Joint and skeletal deformities affect all age groups and greatly reduce the range of motion of the joints. These deformities are notoriously difficult to diagnose and to correct through surgery.

Method

We have developed a surgical tool which integrates surgical instrument tracking and augmented reality through a head mounted display. This allows the surgeon to visualise bones with the illusion of possessing “X-ray” vision. The studies presented below aim to assess the accuracy of the surgical navigation tool in tracking a location at the tip of the surgical instrument in holographic space.

Results

Results show that the average accuracy provided by the navigation tool is around 8 mm, and qualitative assessment by the orthopaedic surgeons provided positive feedback in terms of the capabilities for diagnostic use.

Conclusions

More improvements are necessary for the navigation tool to be accurate enough for surgical applications, however, this new tool has the potential to improve diagnostic accuracy and allow for safer and more precise surgeries, as well as provide for better learning conditions for orthopaedic surgeons in training.

Development and Pre-Clinical Analysis of Spatiotemporal-Aware Augmented Reality in Orthopedic Interventions

Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. As a consequence, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. The main contribution of this paper is to reveal how exemplary workflows are redefined by taking full advantage of head-mounted displays when entirely co-registered with the imaging system at all times. The awareness of the system from the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. Our system achieved an error of 4.76 ± 2.91mm for placing K-wire in a fracture management procedure, and yielded errors of 1.57 ± 1.16° and 1.46 ± 1.00° in the abduction and anteversion angles, respectively, for total hip arthroplasty (THA). We compared the results with the outcomes from baseline standard operative and non-immersive AR procedures, which had yielded errors of [4.61mm, 4.76°, 4.77°] and [5.13mm, 1.78°, 1.43°], respectively, for wire placement, and abduction and anteversion during THA. We hope that our holistic approach towards improving the interface of surgery not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications and provide novel approaches of documenting procedures for training purposes.

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergence-accommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error.

Role of the orthopaedic surgeon in 3D printing: current applications and legal issues for a personalized medicine

3D printing (I3D) is an additive manufacturing technology with a growing interest in medicine and especially in the specialty of orthopaedic surgery and traumatology. There are numerous applications that add value to the personalised treatment of patients: advanced preoperative planning, surgeries with specific tools for each patient, customised orthotic treatments, personalised implants or prostheses and innovative development in the field of bone and cartilage tissue engineering. This paper provides an update on the role that the orthopaedic surgeon and traumatologist plays as a user and prescriber of this technology and a review of the stages required for the correct integration of I3D into the hospital care flow, from the necessary resources to the current legal recommendations.

The influence of different patient positions on the preoperative 3D planning for surgical resection of soft tissue sarcoma in the lower limb-a cadaver pilot study

INTRODUCTION

Complete surgical resection remains the mainstay of the treatment of soft tissue sarcomas. Intraoperative positioning of the patient is dictated by tumor location, whereas preoperative imaging is always performed in the supine position. The effect of changing the patient position on the exact location of the tumor with regard to neurovascular structures and bone is unknown.

MATERIAL AND METHODS

Two fresh frozen cadavers (pelvis and legs) were thawed and warmed. Three standardized tumor models were implanted in the thigh and calf. MR/CT images of the cadavers were obtained sequentially in four different patient positions. The minimal distance of each "tumor" to neurovascular structures was measured on axial MR images and the 3D shift of the center of the tumor to the bone was measured after segmentation of the CT images.

RESULTS

A significant difference of the minimal distance of the "tumor" to the femoral artery (P = 0.019/0.023) and a significantly greater number of deviations of more than 5mm/10 mm in the thigh between the supine position and the other positions compared to two supine positions (p = 0.027/0.028) were seen. The center of the "tumor" compared to the bone shifted significantly in the thigh (P < 0.001/0.002) but not the lower leg.

CONCLUSION

Obtaining images in the same patient position as the planned tumor resection may become particularly relevant if computer assisted surgery, which is based on preoperative imaging, is introduced into soft tissue sarcoma surgery as the patient position significantly influences the spatial position of the tumor.

Value of the surgeon's sightline on hologram registration and targeting in mixed reality

Purpose

Mixed reality (MR) is being evaluated as a visual tool for surgical navigation. Current literature presents unclear results on intraoperative accuracy using the Microsoft HoloLens 1^®. This study aims to assess the impact of the surgeon’s sightline in an inside-out marker-based MR navigation system for open surgery.

Methods

Surgeons at Akershus University Hospital tested this system. A custom-made phantom was used, containing 18 wire target crosses within its inner walls. A CT scan was obtained in order to segment all wire targets into a single 3D-model (hologram). An in-house software application (CTrue), developed for the Microsoft HoloLens 1, uploaded 3D-models and automatically registered the 3D-model with the phantom. Based on the surgeon’s sightline while registering and targeting (free sightline /F/or a strictly perpendicular sightline /P/), 4 scenarios were developed (FF-PF-FP-PP). Target error distance (TED) was obtained in three different working axes-(XYZ).

Results

Six surgeons (5 males, age 29–62) were enrolled. A total of 864 measurements were collected in 4 scenarios, twice. Scenario PP showed the smallest TED in XYZ-axes mean = 2.98 mm ± SD 1.33; 2.28 mm ± SD 1.45; 2.78 mm ± SD 1.91, respectively. Scenario FF showed the largest TED in XYZ-axes with mean = 10.03 mm ± SD 3.19; 6.36 mm ± SD 3.36; 16.11 mm ± SD 8.91, respectively. Multiple comparison tests, grouped in scenarios and axes, showed that the majority of scenario comparisons had significantly different TED values (p < 0.05). Y-axis always presented the smallest TED regardless of scenario tested.

Conclusion

A strictly perpendicular working sightline in relation to the 3D-model achieves the best accuracy results. Shortcomings in this technology, as an intraoperative visual cue, can be overcome by sightline correction. Incidentally, this is the preferred working angle for open surgery.

Extended Reality in Patient Care and Pharmacy Practice: A Viewpoint

The evolution of technology has given practitioners and educators more tools to better treat, manage, and educate both patients and future pharmacists. The objective of this viewpoint publication is to describe the current use of extended reality (XR) in pharmacy and propose ways in which pharmacy practice and education may benefit from incorporation of this technology. While these tools have been used for decades by many other professions, pharmacy is starting to adopt XR in professional and educational practice. XR (virtual reality, mixed reality, and augmented reality) is being used in various aspects of pharmacy care and education, such as pain management, diabetes self-care, cross-checking of prescriptions, treatments for addiction, and (in limited ways) patient and pharmacy education. There is great potential for further integration of XR into pharmacy practice and pharmacy education to ultimately improve patient care and education as well as pharmacy education.

Extended Reality in Patient Care and Pharmacy Practice: A Viewpoint

The evolution of technology has given practitioners and educators more tools to better treat, manage, and educate both patients and future pharmacists. The objective of this viewpoint publication is to describe the current use of extended reality (XR) in pharmacy and propose ways in which pharmacy practice and education may benefit from incorporation of this technology. While these tools have been used for decades by many other professions, pharmacy is starting to adopt XR in professional and educational practice. XR (virtual reality, mixed reality, and augmented reality) is being used in various aspects of pharmacy care and education, such as pain management, diabetes self-care, cross-checking of prescriptions, treatments for addiction, and (in limited ways) patient and pharmacy education. There is great potential for further integration of XR into pharmacy practice and pharmacy education to ultimately improve patient care and education as well as pharmacy education.