Virtual and augmented reality: potential applications in radiology

Smartphone Augmented Reality Outperforms Conventional CT Guidance for Composite Ablation Margins in Phantom Models

PURPOSE

To develop and evaluate a smartphone augmented reality (AR) system for a large 50-mm liver tumor ablation with treatment planning for composite overlapping ablation zones.

MATERIALS AND METHODS

A smartphone AR application was developed to display tumor, probe, projected probe paths, ablated zones, and real-time percentage of the ablated target tumor volume. Fiducial markers were attached to phantoms and an ablation probe hub for tracking. The system was evaluated with tissue-mimicking thermochromic phantoms and gel phantoms. Four interventional radiologists performed 2 trials each of 3 probe insertions per trial using AR guidance versus computed tomography (CT) guidance approaches in 2 gel phantoms. Insertion points and optimal probe paths were predetermined. On Gel Phantom 2, serial ablated zones were saved and continuously displayed after each probe placement/adjustment, enabling feedback and iterative planning. The percentages of tumor ablated for AR guidance versus CT guidance, and with versus without display of recorded ablated zones, were compared among interventional radiologists with pairwise t-tests.

RESULTS

The means of percentages of tumor ablated for CT freehand and AR guidance were 36% ± 7 and 47% ± 4 (P = .004), respectively. The mean composite percentages of tumor ablated for AR guidance were 43% ± 1 (without) and 50% ± 2 (with display of ablation zone) (P = .033). There was no strong correlation between AR-guided percentage of ablation and years of experience (r < 0.5), whereas there was a strong correlation between CT-guided percentage of ablation and years of experience (r > 0.9).

CONCLUSIONS

A smartphone AR guidance system for dynamic iterative large liver tumor ablation was accurate, performed better than conventional CT guidance, especially for less experienced interventional radiologists, and enhanced more standardized performance across experience levels for ablation of a 50-mm tumor.

Night and Day, Why Radiologists Need Play

What is play and why does it matter for radiologists? Play can take many forms in the workplace, including organic, managed, task-related, diversionary, and resistive forms of play, and play may also take the form of authentic self-expression and creation. In this review article, we will discuss the benefits of play including improved problem solving, gaining perspective, and stress reduction, and also provide low-tech and high-tech examples of beneficial play for the radiology team in work and personal contexts.

Radiation exposure in augmented fluoroscopic bronchoscopy procedures: a comprehensive analysis for patients and physicians

Cancer is a major health challenge and causes millions of deaths worldwide each year, and the incidence of lung cancer has increased. Augmented fluoroscopic bronchoscopy (AFB) procedures, which combine bronchoscopy and fluoroscopy, are crucial for diagnosing and treating lung cancer. However, fluoroscopy exposes patients and physicians to radiation, and therefore, the procedure requires careful monitoring. The National Council on Radiation Protection and Measurement and the International Commission on Radiological Protection have emphasised the importance of monitoring patient doses and ensuring occupational radiation safety. The present study evaluated radiation doses during AFB procedures, focusing on patient skin doses, the effective dose, and the personal dose equivalent to the eye lens for physicians. Skin doses were measured using thermoluminescent dosimeters. Peak skin doses were observed on the sides of the patients' arms, particularly on the side closest to the x-ray tube. Differences in the procedures and experience of physicians between the two hospitals involved in this study were investigated. AFB procedures were conducted more efficiently at Hospital A than at Hospital B, resulting in lower effective doses. Cone-beam computed tomography (CT) contributes significantly to patient effective doses because it has higher radiographic parameters. Despite their higher radiographic parameters, AFB procedures resulted in smaller skin doses than did image-guided interventional and CT fluoroscopy procedures. The effective doses differed between the two hospitals of this study due to workflow differences, with cone-beam CT playing a dominant role. No significant differences in left and right eyeHp(3) values were observed between the hospitals. For both hospitals, theHp(3) values were below the recommended limits, indicating that radiation monitoring may not be required for AFB procedures. This study provides insights into radiation exposure during AFB procedures, concerning radiation dosimetry, and safety for patients and physicians.

The Multidomain Metaverse Cancer Care Digital Platform: Development and Usability Study

BACKGROUND

As cancer treatment methods have diversified and the importance of self-management, which lowers the dependence rate on direct hospital visits, has increased, effective cancer care education and management for health professionals and patients have become necessary. The metaverse is in the spotlight as a means of digital health that allows users to engage in cancer care education and management beyond physical constraints. However, it is difficult to find a multipurpose medical metaverse that can not only be used in the field but also complements current cancer care.

OBJECTIVE

This study aimed to develop an integrated metaverse cancer care platform, Dr. Meta, and examine its usability.

METHODS

We conducted a multicenter, cross-sectional survey between November and December 2021. A descriptive analysis was performed to examine users' experiences with Dr. Meta. In addition, a supplementary open-ended question was used to ask users for their suggestions and improvements regarding the platform.

RESULTS

Responses from 70 Korean participants (male: n=19, 27% and female: n=51, 73%) were analyzed. More than half (n=37, 54%) of the participants were satisfied with Dr. Meta; they responded that it was an interesting and immersive platform (n=50, 72%). Less than half perceived no discomfort when using Dr. Meta (n=34, 49%) and no difficulty in wearing and operating the device (n=30, 43%). Furthermore, more than half (n=50, 72%) of the participants reported that Dr. Meta would help provide non-face-to-face and noncontact services. More than half also wanted to continue using this platform in the future (n=41, 59%) and recommended it to others (n=42, 60%).

CONCLUSIONS

We developed a multidomain metaverse cancer care platform that can support both health professionals and patients in non-face-to-face cancer care. The platform was uniquely disseminated and implemented in multiple regional hospitals and showed the potential to perform successful cancer care.

Study Smarter: Applying the Science of Learning to Radiology

Lifelong learning is critical to a successful career in radiology, but many learners use inefficient or ineffective studying strategies developed from personal experience. Decades of cognitive psychology research have identified several techniques that consistently improve knowledge consolidation and retrieval. To improve their knowledge and ultimately patient care, radiologists should strive to understand and adopt these learning techniques. The first part of this article reviews several evidence-based learning principles, including active retrieval and the testing effect, spaced repetition, interleaving, deliberate practice, and growth mind-set. The second part provides practical suggestions on how to incorporate these principles into radiology learning, both during training and beyond.

Bone biopsies guided by augmented reality: a pilot study

PURPOSE

To test the technical feasibility of an augmented reality (AR) navigation system to guide bone biopsies.

METHODS

We enrolled patients subjected to percutaneous computed tomography (CT)-guided bone biopsy using a novel AR navigation system. Data from prospectively enrolled patients (AR group) were compared with data obtained retrospectively from previous standard CT-guided bone biopsies (control group). We evaluated the following: procedure duration, number of CT passes, patient's radiation dose (dose-length product), complications, and specimen adequacy. Technical success was defined as the ability to complete the procedure as planned, reaching the target center. Technical efficacy was assessed evaluating specimen adequacy.

RESULTS

Eight patients (4 males) aged 58 ± 24 years (mean ± standard deviation) were enrolled in the AR group and compared with 8 controls (4 males) aged 60 ± 15 years. No complications were observed. Procedure duration, number of CT passes, and radiation dose were 22 ± 5 min, 4 (median) [4, 6 interquartile range] and 1,034 ± 672 mGy*cm for the AR group and 23 ± 5 min, 9 [7.75, 11.25], and 1,954 ± 993 mGy*cm for controls, respectively. No significant differences were observed for procedure duration (p = 0.878). Conversely, number of CT passes and radiation doses were significantly lower for the AR group (p < 0.001 and p = 0.021, respectively). Technical success and technical efficacy were 100% for both groups.

CONCLUSIONS

This AR navigation system is safe, feasible, and effective; it can decrease radiation exposure and number of CT passes during bone biopsies without increasing duration time.

RELEVANCE STATEMENT

This augmented reality (AR) navigation system is a safe and feasible guidance for bone biopsies; it may ensure a decrease in the number of CT passes and patient's radiation dose.

KEY POINTS

• This AR navigation system is a safe guidance for bone biopsies. • It ensures decrease of number of CT passes and patient's radiation exposure. • Procedure duration was similar to that of standard CT-guided biopsy. • Technical success was 100% as in all patients the target was reached. • Technical efficacy was 100% as the specimen was adequate in all patients.

Extended Reality in Diagnostic Imaging-A Literature Review

The utilization of extended reality (ER) has been increasingly explored in the medical field over the past ten years. A comprehensive analysis of scientific publications was conducted to assess the applications of ER in the field of diagnostic imaging, including ultrasound, interventional radiology, and computed tomography. The study also evaluated the use of ER in patient positioning and medical education. Additionally, we explored the potential of ER as a replacement for anesthesia and sedation during examinations. The use of ER technologies in medical education has received increased attention in recent years. This technology allows for a more interactive and engaging educational experience, particularly in anatomy and patient positioning, although the question may be asked: is the technology and maintenance cost worth the investment? The results of the analyzed studies suggest that implementing augmented reality in clinical practice is a positive phenomenon that expands the diagnostic capabilities of imaging studies, education, and positioning. The results suggest that ER has significant potential to improve diagnostic imaging procedures' accuracy and efficiency and enhance the patient experience through increased visualization and understanding of medical conditions. Despite these promising advancements, further research is needed to fully realize the potential of ER in the medical field and to address the challenges and limitations associated with its integration into clinical practice.

A comparison of perceived image quality between computer display monitors and augmented reality smart glasses

INTRODUCTION

Augmented-reality (AR) smart glasses provide an alternative to standard computer display monitors (CDM). AR smart glasses may provide an opportunity to improve visualisation during fluoroscopy and interventional radiology (IR) procedures when there can be difficulty in viewing intra-procedural images on a CDM. The aim of this study was to evaluate radiographer perception of image quality (IQ) when comparing CDM and AR smart glasses.

METHODS

38 radiographers attending an international congress evaluated ten fluoroscopic-guided surgery and IR images on both a CDM (1920 × 1200 pixels) and a set of Epson Moverio BT-40 AR smart glasses (1920 × 1080 pixels). Participants provided oral responses to pre-defined IQ questions generated by study researchers. Summative IQ scores for each participant/image were compared between CDM and AR smart glasses.

RESULTS

Of the 38 participants, the mean age was 39 ± 1 years. 23 (60.5%) participants required corrective glasses. In terms of generalisability, participants were from 12 different countries, the majority (n = 9, 23.7%) from the United Kingdom. For eight out of ten images, the AR smart glasses demonstrated a statistically significant increase in perceived IQ (median [IQR] 2.0 [-1.0 to 7.0] points) when compared to the CDM.

CONCLUSION

AR smart glasses appear to show improvements in perceived IQ when compared to a CDM. AR smart glasses could provide an option for improving the experiences of radiographers involved in image-guided procedures and should be subject to further clinical evaluations.

IMPLICATIONS FOR PRACTICE

Opportunities exist to improve perceived IQ for radiographers when reviewing fluoroscopy and IR images. AR smart glasses should be further evaluated as a potential opportunity to improve practice when visual attention is split between positioning equipment and image review.

Extended reality veterinary medicine case studies for diagnostic veterinary imaging instruction: Assessing student perceptions and examination performance

Educational technologies in veterinary medicine aim to train veterinarians faster and improve clinical outcomes. COVID-19 pandemic, shifted face-to-face teaching to online, thus, the need to provide effective education remotely was exacerbated. Among recent technology advances for veterinary medical education, extended reality (XR) is a promising teaching tool. This study aimed to develop a case resolution approach for radiographic anatomy studies using XR technology and assess students' achievement of differential diagnostic skills. Learning objectives based on Bloom's taxonomy keywords were used to develop four clinical cases (3 dogs/1 cat) of spinal injuries utilizing CT scans and XR models and presented to 22 third-year veterinary medicine students. Quantitative assessment (ASMT) of 7 questions probing 'memorization', 'understanding and application', 'analysis' and 'evaluation' was given before and after contact with XR technology as well as qualitative feedback via a survey. Mean ASMT scores increased during case resolution (pre 51.6% (±37%)/post 60.1% (± 34%); p < 0.01), but without significant difference between cases (Kruskal-Wallis H = 2.18, NS). Learning objectives were examined for six questions (Q1-Q6) across cases (C1-4): Memorization improved sequentially (Q1, 2 8/8), while Understanding and Application (Q3,4) showed the greatest improvement (26.7%-76.9%). Evaluation and Analysis (Q5,6) was somewhat mixed, improving (5/8), no change (3/8) and declining (1/8).Positive student perceptions suggest that case studies' online delivery was well received stimulating learning in diagnostic imaging and anatomy while developing visual-spatial skills that aid understanding cross-sectional images. Therefore, XR technology could be a useful approach to complement radiological instruction in veterinary medicine.

Advances and Innovations in Ablative Head and Neck Oncologic Surgery Using Mixed Reality Technologies in Personalized Medicine

The benefit of computer-assisted planning in head and neck ablative and reconstructive surgery has been extensively documented over the last decade. This approach has been proven to offer a more secure surgical procedure. In the treatment of cancer of the head and neck, computer-assisted surgery can be used to visualize and estimate the location and extent of the tumor mass. Nowadays, some software tools even allow the visualization of the structures of interest in a mixed reality environment. However, the precise integration of mixed reality systems into a daily clinical routine is still a challenge. To date, this technology is not yet fully integrated into clinical settings such as the tumor board, surgical planning for head and neck tumors, or medical and surgical education. As a consequence, the handling of these systems is still of an experimental nature, and decision-making based on the presented data is not yet widely used. The aim of this paper is to present a novel, user-friendly 3D planning and mixed reality software and its potential application for ablative and reconstructive head and neck surgery.

"Snap on" or Not? A Validation on the Measurement Tool in a Virtual Reality Application

This multi-rater comparison study aims to validate the measurement tool with a "snap" feature option (SNAP ON vs. SNAP OFF), in a virtual reality (VR) application, ImmersiveView v. 2.1, against a conventional software Mimics Innovation Suite v.22 (MIS). It is hypothesized that these measurement tools are equivalent between SNAP ON, and SNAP OFF, and when compared to MIS, in terms of basic linear and angular measurements. Six (6) raters conducted a set of 40 linear and 15 angular measurements using CT scan data of three objects (L-block, hand model, and dry skull) with fiducial markers. Inter-rater repeatability and intra-rater reproducibility were assessed via inter-class coefficient (ICC). Equivalency between each pair of modules (SNAP ON, SNAP OFF, and MIS) was analyzed via Bland-Altman plots and two one-sided t-tests (TOST) procedure. The ICC for intra-rater repeatability yielded 0.999 to 1.000, and inter-rater reproducibility yielded 0.998 to 1.000, which suggests high degree of intra- and inter-rater reliability. The Bland-Altman plots demonstrated that measurements acquired from SNAP ON, SNAP OFF, and MIS were equivalent. The TOST procedure yielded that the measurements through all three modules are equivalent within ± 0.2 mm interval for distance, and ± 0.3° interval for angular measurements. The measurement tool with the "snap" feature in a newly developed VR application (ImmersiveView v.2.1) has been validated through a multi-rater comparison study. In terms of linear and angular measurements, this VR application, whether the "snap" feature was on or off, was equivalent to each other and to the control software (MIS) under the condition of this study. A strong reliability, both intra-rater repeatability and inter-rater reproducibility, has been found.

Extended reality visualization of medical museum specimens: Online presentation of conjoined twins curated by Dr. Jacob Henle between 1844-1852

Background:

The purpose of this study is to characterize a full-term conjoined twins’ cadaver curated by Dr. Jacob Henle sometime between 1844 and 1852 and demonstrate digital distribution of an old and rare medical museum specimen using an extended reality (XR) model workflow.

Methods:

The cadaver (Preparation 296) is in the Department of Anatomy and Cell Biology at the University of Heidelberg. An XR display workflow comprises image capture, segmentation, and visualization using CT/MR scans derived from the cadaver. Online radiology presentation to medical students focuses on diagnostic characteristics of anatomical systems depicted with XR models.

Results:

Developmental defects in Preparation 296 include duplicated supradiaphragmatic structures and abnormal osteological features. Subdiaphragmatically, the gut is continuous on the right, but terminates at the distal esophagus on the left. One large liver occupies the abdomen with one spleen located on the left side. Observations suggest duplication of the primitive streak and separate notochords rostrally. Duplication occurs near the yolk sac and involves midgut formation while secondary midline fusion of the upper extremities and ribs likely results from the proximity of the embryos during development. Medical students access the model with device agnostic software during the curricular topic “Human Body Plan” that includes embryology concepts covering mechanisms of twinning.

Conclusions:

The workflow enables ease-of-access XR visualizations of an old and rare museum specimen. This study also demonstrates digital distribution and utilization of XR models applicable to embryology education.

Development and Implementation of an Integrated Imaging Informatics Track for Radiology Residents: Our 3-Year Experience

RATIONALE AND OBJECTIVES

Imaging Informatics is an emerging and fast-evolving field that encompasses the management of information during all steps of the imaging value chain. With many information technology tools being essential to the radiologists' day-to-day work, there is an increasing need for qualified professionals with clinical background, technology expertise, and leadership skills. To answer this, we describe our experience in the development and implementation of an Integrated Imaging Informatics Track (I3T) for radiology residents at our institution.

MATERIALS AND METHODS

The I3T was created by a resident-driven initiative funded by an intradepartmental resident grant. Its curriculum is delivered through a combination of monthly small group discussions, operational meetings, recommended readings, lectures, and early exposure to the National Imaging Informatics Course. The track is steered and managed by the I3T Committee, including trainees and faculty advisors. Up to two first-year residents are selected annually based on their curriculum vitae and an interest application. Successful completion of the program requires submission of a capstone project and at least one academic deliverable (national meeting presentation, poster, exhibit, manuscript and/or grant).

RESULTS

In our three-year experience, the seven I3T radiology residents have reported a total of 58 scholarly activities related to Imaging Informatics. I3T residents have assumed leadership roles within our organization and nationally. All residents have successfully carried out their clinical responsibilities.

CONCLUSION

We have developed and implemented an I3T for radiology residents at our institution. These residents have been successful in their clinical, scholarship and leadership pursuits.

Thermal Ablation of Liver Tumors Guided by Augmented Reality: An Initial Clinical Experience

Background: Over the last two decades, augmented reality (AR) has been used as a visualization tool in many medical fields in order to increase precision, limit the radiation dose, and decrease the variability among operators. Here, we report the first in vivo study of a novel AR system for the guidance of percutaneous interventional oncology procedures. Methods: Eight patients with 15 liver tumors (0.7−3.0 cm, mean 1.56 + 0.55) underwent percutaneous thermal ablations using AR guidance (i.e., the Endosight system). Prior to the intervention, the patients were evaluated with US and CT. The targeted nodules were segmented and three-dimensionally (3D) reconstructed from CT images, and the probe trajectory to the target was defined. The procedures were guided solely by AR, with the position of the probe tip was subsequently confirmed by conventional imaging. The primary endpoints were the targeting accuracy, the system setup time, and targeting time (i.e., from the target visualization to the correct needle insertion). The technical success was also evaluated and validated by co-registration software. Upon completion, the operators were assessed for cybersickness or other symptoms related to the use of AR. Results: Rapid system setup and procedural targeting times were noted (mean 14.3 min; 12.0−17.2 min; 4.3 min, 3.2−5.7 min, mean, respectively). The high targeting accuracy (3.4 mm; 2.6−4.2 mm, mean) was accompanied by technical success in all 15 lesions (i.e., the complete ablation of the tumor and 13/15 lesions with a >90% 5-mm periablational margin). No intra/periprocedural complications or operator cybersickness were observed. Conclusions: AR guidance is highly accurate, and allows for the confident performance of percutaneous thermal ablations.

New Approach to Accelerated Image Annotation by Leveraging Virtual Reality and Cloud Computing

Three-dimensional imaging is at the core of medical imaging and is becoming a standard in biological research. As a result, there is an increasing need to visualize, analyze and interact with data in a natural three-dimensional context. By combining stereoscopy and motion tracking, commercial virtual reality (VR) headsets provide a solution to this critical visualization challenge by allowing users to view volumetric image stacks in a highly intuitive fashion. While optimizing the visualization and interaction process in VR remains an active topic, one of the most pressing issue is how to utilize VR for annotation and analysis of data. Annotating data is often a required step for training machine learning algorithms. For example, enhancing the ability to annotate complex three-dimensional data in biological research as newly acquired data may come in limited quantities. Similarly, medical data annotation is often time-consuming and requires expert knowledge to identify structures of interest correctly. Moreover, simultaneous data analysis and visualization in VR is computationally demanding. Here, we introduce a new procedure to visualize, interact, annotate and analyze data by combining VR with cloud computing. VR is leveraged to provide natural interactions with volumetric representations of experimental imaging data. In parallel, cloud computing performs costly computations to accelerate the data annotation with minimal input required from the user. We demonstrate multiple proof-of-concept applications of our approach on volumetric fluorescent microscopy images of mouse neurons and tumor or organ annotations in medical images.

Clinical acceptance of advanced visualization methods: a comparison study of 3D-print, virtual reality glasses, and 3D-display

BACKGROUND

To compare different methods of three-dimensional representations, namely 3D-Print, Virtual Reality (VR)-Glasses and 3D-Display regarding the understanding of the pathology, accuracy of details, quality of the anatomical representation and technical operability and assessment of possible change in treatment in different disciplines and levels of professional experience.

METHODS

Interviews were conducted with twenty physicians from the disciplines of cardiology, oral and maxillofacial surgery, orthopedic surgery, and radiology between 2018 and 2020 at the University Hospital of Basel. They were all presented with three different three-dimensional clinical cases derived from CT data from their area of expertise, one case for each method. During this, the physicians were asked for their feedback written down on a pencil and paper questionnaire.

RESULTS

Concerning the understanding of the pathology and quality of the anatomical representation, VR-Glasses were rated best in three out of four disciplines and two out of three levels of professional experience. Regarding the accuracy of details, 3D-Display was rated best in three out of four disciplines and all levels of professional experience. As to operability, 3D-Display was consistently rated best in all levels of professional experience and all disciplines. Possible change in treatment was reported using 3D-Print in 33%, VR-Glasses in 44%, and 3D-Display in 33% of participants. Physicians with a professional experience of more than ten years reported no change in treatment using any method.

CONCLUSIONS

3D-Print, VR-Glasses, and 3D-Displays are very well accepted, and a relevant percentage of participants with less than ten years of professional work experience could imagine a possible change in treatment using any of these three-dimensional methods. Our findings challenge scientists, technicians, and physicians to further develop these methods to improve the three-dimensional understanding of pathologies and to add value to the education of young and inexperienced physicians.

Emergence of teleradiology, PACS, and other radiology IT solutions in Acta Radiologica

For this historical review, we searched a database containing all the articles published in Acta Radiologica during its 100-year history to find those on the use of information technology (IT) in radiology. After reading the full texts, we selected the presented articles according to major radiology IT domains such as teleradiology, picture archiving and communication systems, image processing, image analysis, and computer-aided diagnostics in order to describe the development as it appeared in the journal. Publications generally follow IT megatrends, but because the contents of Acta Radiologica are mainly clinically oriented, some technology achievements appear later than they do in journals discussing mainly imaging informatics topics.

3D Modeling and Advanced Visualization of the Pediatric Brain, Neck, and Spine

The ready availability of advanced visualization tools on picture archiving and communication systems workstations or even standard laptops through server-based or cloud-based solutions has enabled greater adoption of these techniques. We describe how radiologists can tailor imaging techniques for optimal 3D reconstructions provide a brief overview of the standard and newer "on-screen" techniques. We describe the process of creating 3D printed models for surgical simulation and education, with examples from the authors' institution and the existing literature. Finally, the review highlights current uses and potential future use cases for virtual reality and augmented reality applications in a pediatric neuroimaging setting.

Feasibility of the Application of Holographic Augmented Reality in Endovascular Surgery Using Microsoft HoloLens Head-Mounted Display

OBJECTIVES

Advances in virtual, augmented (AR) and mixed reality have led to the development of wearable technologies including head mounted displays (HMD). The aim of this study was to investigate the feasibility to use HMD during endovascular surgery.

METHODS

We propose an adaptation of AR-HMD using Microsoft HoloLens. Software was developed to enable visualization of the vascular system during endovascular procedures. A video was performed to present an overview of the device and show its use in real conditions.

RESULTS

The device allowed a successful visualization of perioperative angiography during peripheral angioplasty, carotid angioplasty and aortic aneurysm endovascular repair. The device was operated on voice command, preserving the environment sterility.

CONCLUSION

This video illustrated the feasibility of the application of holographic AR during endovascular intervention and brings perspectives to use artificial-intelligence derived tools for image-guided surgery.

Simulation-Based Estimation of the Number of Cameras Required for 3D Reconstruction in a Narrow-Baseline Multi-Camera Setup

Graphical visualization systems are a common clinical tool for displaying digital images and three-dimensional volumetric data. These systems provide a broad spectrum of information to support physicians in their clinical routine. For example, the field of radiology enjoys unrestricted options for interaction with the data, since information is pre-recorded and available entirely in digital form. However, some fields, such as microsurgery, do not benefit from this yet. Microscopes, endoscopes, and laparoscopes show the surgical site as it is. To allow free data manipulation and information fusion, 3D digitization of surgical sites is required. We aimed to find the number of cameras needed to add this functionality to surgical microscopes. For this, we performed in silico simulations of the 3D reconstruction of representative models of microsurgical sites with different numbers of cameras in narrow-baseline setups. Our results show that eight independent camera views are preferable, while at least four are necessary for a digital surgical site. In most cases, eight cameras allow the reconstruction of over 99% of the visible part. With four cameras, still over 95% can be achieved. This answers one of the key questions for the development of a prototype microscope. In future, such a system can provide functionality which is unattainable today.

Comparison of Smartphone Augmented Reality, Smartglasses Augmented Reality, and 3D CBCT-guided Fluoroscopy Navigation for Percutaneous Needle Insertion: A Phantom Study

PURPOSE

To compare needle placement performance using an augmented reality (AR) navigation platform implemented on smartphone or smartglasses devices to that of CBCT-guided fluoroscopy in a phantom.

MATERIALS AND METHODS

An AR application was developed to display a planned percutaneous needle trajectory on the smartphone (iPhone7) and smartglasses (HoloLens1) devices in real time. Two AR-guided needle placement systems and CBCT-guided fluoroscopy with navigation software (XperGuide, Philips) were compared using an anthropomorphic phantom (CIRS, Norfolk, VA). Six interventional radiologists each performed 18 independent needle placements using smartphone (n = 6), smartglasses (n = 6), and XperGuide (n = 6) guidance. Placement error was defined as the distance from the needle tip to the target center. Placement time was recorded. For XperGuide, dose-area product (DAP, mGy*cm²) and fluoroscopy time (sec) were recorded. Statistical comparisons were made using a two-way repeated measures ANOVA.

RESULTS

The placement error using the smartphone, smartglasses, or XperGuide was similar (3.98 ± 1.68 mm, 5.18 ± 3.84 mm, 4.13 ± 2.38 mm, respectively, p = 0.11). Compared to CBCT-guided fluoroscopy, the smartphone and smartglasses reduced placement time by 38% (p = 0.02) and 55% (p = 0.001), respectively. The DAP for insertion using XperGuide was 3086 ± 2920 mGy*cm², and no intra-procedural radiation was required for augmented reality.

CONCLUSIONS

Smartphone- and smartglasses-based augmented reality reduced needle placement time and radiation exposure while maintaining placement accuracy compared to a clinically validated needle navigation platform.

Registration Techniques for Clinical Applications of Three-Dimensional Augmented Reality Devices

Many clinical procedures would benefit from direct and intuitive real-time visualization of anatomy, surgical plans, or other information crucial to the procedure. Three-dimensional augmented reality (3D-AR) is an emerging technology that has the potential to assist physicians with spatial reasoning during clinical interventions. The most intriguing applications of 3D-AR involve visualizations of anatomy or surgical plans that appear directly on the patient. However, commercially available 3D-AR devices have spatial localization errors that are too large for many clinical procedures. For this reason, a variety of approaches for improving 3D-AR registration accuracy have been explored. The focus of this review is on the methods, accuracy, and clinical applications of registering 3D-AR devices with the clinical environment. The works cited represent a variety of approaches for registering holograms to patients, including manual registration, computer vision-based registration, and registrations that incorporate external tracking systems. Evaluations of user accuracy when performing clinically relevant tasks suggest that accuracies of approximately 2 mm are feasible. 3D-AR device limitations due to the vergence-accommodation conflict or other factors attributable to the headset hardware add on the order of 1.5 mm of error compared to conventional guidance. Continued improvements to 3D-AR hardware will decrease these sources of error.