Towards a First-Person Perspective Mixed Reality Guidance System for Needle Interventions

Effects of Augmented Reality-Based Remote Mentoring on Task Performance and Communication: A Simulation Study in the Context of Emergency Medical Services

Background: Augmented reality head-worn displays (HWDs) may enable efficient remote support in the prehospital environment due to their hand-free operability, their "see-what-I-see" features, and their ability to superimpose digital content over the environment. Methods: In this simulation-based randomized controlled study, a remote mentor used either a phone or HWD to instruct 23 physicians on how to insert a Multi-Lumen Access Catheter into a mannequin. In the phone condition, information could be exchanged only verbally. In the HWD condition, the mentor could additionally see the participant's first-person view and show reference images. We hypothesized that participants who received instructions via the HWD would achieve better procedural performance (lower task completion times, fewer errors advancing the catheter) and exhibit different communication patterns than participants who received instructions via phone. Results: The HWD did not significantly reduce task completion times or errors during catheter advancement. However, by analyzing the frequency of communication events with a Poisson regression, we could demonstrate that with the HWD, the mentor had to request situation reports less often (p < 0.001) but provided more instructions (p = 0.004) and more feedback (p = 0.008). As a possible consequence, participants in the HWD condition rated their workload as lower than participants who used a phone to communicate (p = 0.45). Conclusion: The study demonstrates that HWD-based telemedicine systems can be rated positively by physicians, can benefit communication, and can provide more opportunities for the detection of clinical errors.

Comparing Different Registration and Visualization Methods for Navigated Common Femoral Arterial Access-A Phantom Model Study Using Mixed Reality

Mixed reality (MxR) enables the projection of virtual three-dimensional objects into the user's field of view via a head-mounted display (HMD). This phantom model study investigated three different workflows for navigated common femoral arterial (CFA) access and compared it to a conventional sonography-guided technique as a control. A total of 160 punctures were performed by 10 operators (5 experts and 5 non-experts). A successful CFA puncture was defined as puncture at the mid-level of the femoral head with the needle tip at the central lumen line in a 0° coronary insertion angle and a 45° sagittal insertion angle. Positional errors were quantified using cone-beam computed tomography following each attempt. Mixed effect modeling revealed that the distance from the needle entry site to the mid-level of the femoral head is significantly shorter for navigated techniques than for the control group. This highlights that three-dimensional visualization could increase the safety of CFA access. However, the navigated workflows are infrastructurally complex with limited usability and are associated with relevant cost. While navigated techniques appear as a potentially beneficial adjunct for safe CFA access, future developments should aim to reduce workflow complexity, avoid optical tracking systems, and offer more pragmatic methods of registration and instrument tracking.

Simulation of Image-Guided Microwave Ablation Therapy Using a Digital Twin Computational Model

Emerging computational tools such as healthcare digital twin modeling are enabling the creation of patient-specific surgical planning, including microwave ablation to treat primary and secondary liver cancers. Healthcare digital twins (DTs) are anatomically one-to-one biophysical models constructed from structural, functional, and biomarker-based imaging data to simulate patient-specific therapies and guide clinical decision-making. In microwave ablation (MWA), tissue-specific factors including tissue perfusion, hepatic steatosis, and fibrosis affect therapeutic extent, but current thermal dosing guidelines do not account for these parameters. This study establishes an MR imaging framework to construct three-dimensional biophysical digital twins to predict ablation delivery in livers with 5 levels of fat content in the presence of a tumor. Four microwave antenna placement strategies were considered, and simulated microwave ablations were then performed using 915 MHz and 2450 MHz antennae in Tumor Naïve DTs (control), and Tumor Informed DTs at five grades of steatosis. Across the range of fatty liver steatosis grades, fat content was found to significantly increase ablation volumes by approximately 29-l42% in the Tumor Naïve and 55-60% in the Tumor Informed DTs in 915 MHz and 2450 MHz antenna simulations. The presence of tumor did not significantly affect ablation volumes within the same steatosis grade in 915 MHz simulations, but did significantly increase ablation volumes within mild-, moderate-, and high-fat steatosis grades in 2450 MHz simulations. An analysis of signed distance to agreement for placement strategies suggests that accounting for patient-specific tumor tissue properties significantly impacts ablation forecasting for the preoperative evaluation of ablation zone coverage.

Mixed reality for the assessment of aortoiliac anatomy in patients with abdominal aortic aneurysm prior to open and endovascular repair: Feasibility and interobserver agreement

OBJECTIVES

The objective is to evaluate the feasibility and interobserver agreement of a Mixed Reality Viewer (MRV) in the assessment of aortoiliac vascular anatomy of abdominal aortic aneurysm (AAA) patients.

METHODS

Fifty preoperative computed tomography angiographies (CTAs) of AAA patients were included. CTAs were assessed in a mixed reality (MR) environment with respect to aortoiliac anatomy according to a standardized protocol by two experienced observers (Mixed Reality Viewer, MRV, Brainlab AG, Germany). Additionally, all CTAs were independently assessed applying the same protocol by the same observers using a conventional DICOM viewer on a two-dimensional screen with multi-planar reconstructions (Conventional viewer, CV, GE Centricity PACS RA1000 Workstation, GE, United States). The protocol included four sets of items: calcification, dilatation, patency, and tortuosity as well as the number of lumbar and renal arteries. Interobserver agreement (IA, Cohen's Kappa, κ) was calculated for every item set.

RESULTS

All CTAs could successfully be displayed in the MRV (100%). The MRV demonstrated equal or better IA in the assessment of anterior and posterior calcification (κMRV: 0.68 and 0.61, κCV: 0.33 and 0.45, respectively) as well as tortuosity (κMRV: 0.60, κCV: 0.48) and dilatation (κMRV: 0.68, κCV: 0.67). The CV demonstrated better IA in the assessment of patency (κMRV: 0.74, κCV: 0.93). The CV also identified significantly more lumbar arteries (CV: 379, MRV: 239, p < 0.01).

CONCLUSIONS

The MRV is a feasible imaging viewing technology in clinical routine. Future efforts should aim at improving hologram quality and enabling accurate registration of the hologram with the physical patient.

Augmenting Image-Guided Procedures through In Situ Visualization of 3D Ultrasound via a Head-Mounted Display

Medical ultrasound (US) is a commonly used modality for image-guided procedures. Recent research systems providing an in situ visualization of 2D US images via an augmented reality (AR) head-mounted display (HMD) were shown to be advantageous over conventional imaging through reduced task completion times and improved accuracy. In this work, we continue in the direction of recent developments by describing the first AR HMD application visualizing real-time volumetric (3D) US in situ for guiding vascular punctures. We evaluated the application on a technical level as well as in a mixed-methods user study with a qualitative prestudy and a quantitative main study, simulating a vascular puncture. Participants completed the puncture task significantly faster when using 3D US AR mode compared to 2D US AR, with a decrease of 28.4% in time. However, no significant differences were observed regarding the success rate of vascular puncture (2D US AR-50% vs. 3D US AR-72%). On the technical side, the system offers a low latency of 49.90 ± 12.92 ms and a satisfactory frame rate of 60 Hz. Our work shows the feasibility of a system that visualizes real-time 3D US data via an AR HMD, and our experiments show, furthermore, that this may offer additional benefits in US-guided tasks (i.e., reduced task completion time) over 2D US images viewed in AR by offering a vividly volumetric visualization.

Mixed Reality in der Gefäßchirurgie – ein Scoping Review

Hintergrund „Mixed Reality“ (MR) erlaubt die Projektion von virtuellen Objekten in das Sichtfeld des Anwenders durch ein Head-mounted Display (HMD). Im gefäßchirurgischen Behandlungsspektrum könnten MR-Anwendungen in Zukunft einen Nutzen darstellen. Im folgenden Scoping Review soll eine Orientierung über die aktuelle Anwendung der genannten Technologien im Bereich der Gefäßchirurgie gegeben und Forschungsziele für die Zukunft definiert werden. Material und Methoden Es erfolgte eine systematische Literaturrecherche in PubMed (MEDLINE) mit den Suchbegriffen „aorta“, „intervention“, „endovsacular intervention“, „vascular surgery“, „aneurysm“, „endovascular“, „vascular access“ jeweils in Kombination mit „mixed reality“ oder „augmented reality“. Die Suche erfolgte nach PRISMA-Leitlinie (Preferred Reporting Items for Systematic reviews and Meta-Analyses) für Scoping Reviews. Ergebnisse Aus 547 Literaturstellen konnten 8 relevante Studien identifiziert werden. Die Suchergebnisse konnten in 2 Anwendungskategorien eingeteilt werden: (1) MR mit dem Ziel des Informationsmanagements und zur Verbesserung der periprozeduralen Ergonomie gefäßchirurgischer Eingriffe (n = 3) sowie (2) MR mit dem Ziel der intraoperativen Navigation bei gefäßchirurgischen Eingriffen (n = 5). Die Registrierung des physischen Patienten mit dem virtuellen Objekt und das Tracking von Instrumenten in der MR-Umgebung zur intraoperativen Navigation ist dabei im Fokus des wissenschaftlichen Interesses und konnte technisch erfolgreich am Phantom- und Tiermodell gezeigt werden. Die bisher vorgestellten Methoden sind jedoch mit hohem infrastrukturellem Aufwand und relevanten Limitationen verbunden. Schlussfolgerung Der Einsatz von MR im Bereich der Gefäßchirurgie ist grundsätzlich vielversprechend. Für die Zukunft sollten alternative, pragmatische Registrierungsmethoden mit entsprechender Quantifizierung des Positionierungsfehlers angestrebt werden. Die entwickelten Soft- und Hardwarelösungen sollten auf das Anforderungsprofil der Gefäßchirurgie angepasst werden. Das elektromagnetische Instrumenten-Tracking erscheint als sinnvolle, komplementäre Technologie zur Umsetzung der MR-assistierten Navigation.

Mixed-Reality-Assisted Puncture of the Common Femoral Artery in a Phantom Model

Percutaneous femoral arterial access is daily practice in a variety of medical specialties and enables physicians worldwide to perform endovascular interventions. The reported incidence of percutaneous femoral arterial access complications is 3–18% and often results from suboptimal puncture location due to insufficient visualization of the target vessel. The purpose of this proof-of-concept study was to evaluate the feasibility and the positional error of a mixed-reality (MR)-assisted puncture of the common femoral artery in a phantom model using a commercially available navigation system. In total, 15 MR-assisted punctures were performed. Cone-beam computed tomography angiography (CTA) was used following each puncture to allow quantification of positional error of needle placements in the axial and sagittal planes. Technical success was achieved in 14/15 cases (93.3%) with a median axial positional error of 1.0 mm (IQR 1.3) and a median sagittal positional error of 1.1 mm (IQR 1.6). The median duration of the registration process and needle insertion was 2 min (IQR 1.0). MR-assisted puncture of the common femoral artery is feasible with acceptable positional errors in a phantom model. Future studies should aim to measure and reduce the positional error resulting from MR registration.