See It With Your Own Eyes: Markerless Mobile Augmented Reality for Radiation Awareness in the Hybrid Room

Real-time mixed reality display of dual particle radiation detector data

Radiation source localization and characterization are challenging tasks that currently require complex analyses for interpretation. Mixed reality (MR) technologies are at the verge of wide scale adoption and can assist in the visualization of complex data. Herein, we demonstrate real-time visualization of gamma ray and neutron radiation detector data in MR using the Microsoft HoloLens 2 smart glasses, significantly reducing user interpretation burden. Radiation imaging systems typically use double-scatter events of gamma rays or fast neutrons to reconstruct the incidence directional information, thus enabling source localization. The calculated images and estimated 'hot spots' are then often displayed in 2D angular space projections on screens. By combining a state-of-the-art dual particle imaging system with HoloLens 2, we propose to display the data directly to the user via the head-mounted MR smart glasses, presenting the directional information as an overlay to the user's 3D visual experience. We describe an open source implementation using efficient data transfer, image calculation, and 3D engine. We thereby demonstrate for the first time a real-time user experience to display fast neutron or gamma ray images from various radioactive sources set around the detector. We also introduce an alternative source search mode for situations of low event rates using a neural network and simulation based training data to provide a fast estimation of the source's angular direction. Using MR for radiation detection provides a more intuitive perception of radioactivity and can be applied in routine radiation monitoring, education & training, emergency scenarios, or inspections.

Immersive Radiation Experience for Interventional Radiology with Virtual Reality Radiation Dose Visualization Using Fast Monte Carlo Dose Estimation

For interventional radiology (IR), understanding the precise dose distribution is crucial to reduce the risks of radiation dermatitis to patients and staff. Visualization of dose distribution is expected to support radiation safety efforts immensely. This report presents techniques for perceiving the dose distribution using virtual reality (VR) technology and for estimating the air dose distribution accurately using Monte Carlo simulation for VR dose visualization. We adopted an earlier reported Monte-Carlo-based estimation system for IR and simulated the dose in a geometrical area resembling an IR room with fluoroscopic conditions. Users of our VR system experienced a simulated air dose distribution in the IR room while the irradiation angle, irradiation timing, and lead shielding were controlled. The estimated air dose was evaluated through comparison with measurements taken using a radiophotoluminescence glass dosimeter. Our dose estimation results were consistent with dosimeter readings, showing a 13.5% average mutual difference. The estimated air dose was visualized in VR: users could view a virtual IR room and walk around in it. Using our VR system, users experienced dose distribution changes dynamically with C-arm rotation. Qualitative tests were conducted to evaluate the workload and usability of our VR system. The perceived overall workload score (18.00) was lower than the scores reported in the literature for medical tasks (50.60) and computer activities (54.00). This VR visualization is expected to open new horizons for understanding dose distributions intuitively, thereby aiding the avoidance of radiation injury.

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

Usability Measures in Mobile-Based Augmented Reality Learning Applications: A Systematic Review

The implementation of usability in mobile augmented reality (MAR) learning applications has been utilized in a myriad of standards, methodologies, and techniques. The usage and combination of techniques within research approaches are important in determining the quality of usability data collection. The purpose of this study is to identify, study, and analyze existing usability metrics, methods, techniques, and areas in MAR learning. This study adapts systematic literature review techniques by utilizing research questions and Boolean search strings to identify prospective studies from six established databases that are related to the research context area. Seventy-two articles, consisting of 45 journals, 25 conference proceedings, and two book chapters, were selected through a systematic process. All articles underwent a rigorous selection protocol to ensure content quality according to formulated research questions. Post-synthesis and analysis, the output of this article discusses significant factors in usability-based MAR learning applications. This paper presents five identified gaps in the domain of study, modes of contributions, issues within usability metrics, technique approaches, and hybrid technique combinations. This paper concludes five recommendations based on identified gaps concealing potential of usability-based MAR learning research domains, varieties of unexplored research types, validation of emerging usability metrics, potential of performance metrics, and untapped correlational areas to be discovered.

Extended Reality in Medical Practice

PURPOSE OF REVIEW

Advances in display technology and computing have led to new devices capable of overlaying digital information onto the physical world or incorporating aspects of the physical world into virtual scenes. These combinations of digital and physical environments are referred to as extended realities. Extended reality (XR) devices offer many advantages for medical applications including realistic 3D visualization and touch-free interfaces that can be used in sterile environments. This review introduces extended reality and describes how it can be applied to medical practice.

RECENT FINDINGS

The 3D displays of extended reality devices are valuable in situations where spatial information such as patient anatomy and medical instrument position is important. Applications that take advantage of these 3D capabilities include teaching and pre-operative planning. The utility of extended reality during interventional procedures has been demonstrated with through 3D visualizations of patient anatomy, scar visualization, and real-time catheter tracking with touch-free software control. Extended reality devices have been applied to education, pre-procedural planning, and cardiac interventions. These devices excel in settings where traditional devices are difficult to use, such as in the cardiac catheterization lab. New applications of extended reality in cardiology will continue to emerge as the technology improves.

Machine and deep learning for workflow recognition during surgery

Recent years have seen tremendous progress in artificial intelligence (AI), such as with the automatic and real-time recognition of objects and activities in videos in the field of computer vision. Due to its increasing digitalization, the operating room (OR) promises to directly benefit from this progress in the form of new assistance tools that can enhance the abilities and performance of surgical teams. Key for such tools is the recognition of the surgical workflow, because efficient assistance by an AI system requires this system to be aware of the surgical context, namely of all activities taking place inside the operating room. We present here how several recent techniques relying on machine and deep learning can be used to analyze the activities taking place during surgery, using videos captured from either endoscopic or ceiling-mounted cameras. We also present two potential clinical applications that we are developing at the University of Strasbourg with our clinical partners.

Evaluation of occupational exposure to static magnetic field in MRI sites based on body pose estimation and SMF analytical computation

This paper tackles the problem of estimating exposure to static magnetic field (SMF) in magnetic resonance imaging (MRI) sites using a non-invasive approach. The proposed approach relies on a vision-based system to detect people's body parts and on a mathematical model to compute SMF exposure. A multi-view camera system was used to capture the MRI room, and a vision-based system was applied to detect body parts. The detected localization was then fed into a mathematical model to compute SMF exposure. In this study, we focused on exposure at the neck due to two main reasons. First, according to regulations, the limit of exposure at head and trunk for MR workers is higher than that for the general public. Second, it was easier to attach a dosimeter at the neck to perform measurements, which allowed a quantitative evaluation of our approach. This approach was applied to two scenarios simulating the daily movements of medical workers for which dosimeter measurements were also recorded. The results indicated that the proposed approach predicted occupational SMF exposure with reasonable accuracy compared with the dosimeter measurements. The proposed approach is a simple safe working procedure to estimate the exposure of MR workers at different parts of the body without wearing any marker detection. It can be applied to reduce occupational SMF exposure, without changes in workers' performances. For that reason, our non-invasive proposed method can be used as a simple safety tool to estimate occupational SMF exposure in MR sites. Bioelectromagnetics. 39:503-515, 2018.© 2018 Wiley Periodicals, Inc.