HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training

Applying augmented reality in teaching of surgical residents-telementoring, a "stress-free" way to surgical autonomy?

BACKGROUND

Achieving surgical autonomy can be considered the ultimate goal of surgical training. Innovative head-mounted augmented reality (AR) devices enable visualization of the operating field and teaching from remote. Therefore, utilization of AR glasses may be a novel approach to achieve autonomy. The aim of this pilot study is to analyze the feasibility of AR application in surgical training and to assess its impact on intraoperative stress.

METHODS

A head-mounted RealWear Navigator® 500 glasses and the TeamViewer software were used. Initial "dry lab" testing of AR glasses was performed in combination with the Symbionix LAP Mentor™. Subsequently, residents performed various stage-adapted surgical procedures semi-autonomously (SA) (on-demand consultation of senior surgeon, who is in theatre but not scrubbed) versus permanent remote supervision (senior surgeon not present) via augmented reality (AR) glasses, worn by the resident in theatre. Stress was measured by intraoperative heart rate (Polar® pulse belt) and State-Trait Anxiety Inventory (STAI) questionnaire.

RESULTS

After "dry lab" testing, N = 5 senior residents performed equally N = 25 procedures SA and with AR glasses. For both, open and laparoscopic procedure AR remote assistance showed satisfactory applicability. Utilization of AR significantly reduced intraoperative peak pulse rate from 131 to 119 bpm (p = 0.004), as compared with the semi-autonomous group. Likewise, subjectively perceived stress according to STAI was significantly lower in the AR group (p = 0.011).

CONCLUSION

AR can be applied in surgical training and may help to reduce stress in theatre. In the future, AR has a huge potential to become a stepping stone to surgical autonomy.

The application of augmented reality for improving clinical skills: a scoping review

Augmented reality technology had developed rapidly in recent years and had been applied in many fields, including medical education. Augmented reality had potential to improve students' knowledge and skills in medical education. This scoping review primarily aims to further elaborate the current studies on the implementation of augmented reality in advancing clinical skills. This study was conducted by utilizing electronic databases such as PubMed, Embase, and Web of Science in June 2022 for articles focusing on the use of augmented reality for improving clinical skills. The Rayyan website was used to screen the articles that met the inclusion criteria, which was the application of augmented reality as a learning method in medical education. Total of 37 articles met the inclusion criteria. These publications suggested that using augmented reality could improve clinical skills. The most researched topics explored were laparoscopic surgery skills and ophthalmology were the most studied topic. The research methods applied in the articles fall into two main categories: randomized control trial (RCT) (29.3%) and non-RCT (70.3%). Augmented reality has the potential to be integrated in medical education, particularly to boost clinical studies. Due to limited databases, however, any further studies on the implementation of augmented reality as a method to enhance skills in medical education need to be conducted.

Keep it simple: a crosshair on the screen during proctored laparoscopic surgeries

INTRODUCTION

Inside the operating room, experts use verbal instructions to guide surgical novices through laparoscopic procedures. In this study, we evaluated the use of a crosshair attached to the video monitor, as a hands-free pointing tool to simplify instructions during operation.

METHODS

Ten surgical novices performed two elective laparoscopic cholecystectomies within a week of each other, randomized such that one was performed with and the other without using the crosshair. Directly after operation, questionnaires were completed by the novices and the consultant surgeons. Measures including the comprehensibility of instructions, subjective feeling of safety during preparation, time delays due to different instruction options, and disruptive influence while instructors used the crosshair. Differences in operative performance were evaluated based on the global operative assessment of laparoscopic skills (GOALS) scores.

RESULTS

When the crosshair was used, surgical novices had a better understanding of which anatomical structure should be shown (p = 0.028). Operating time (p = 0.222) and feeling of confidence during preparation did not differ with versus without crosshair use (p = 0.081). All participants stated that the crosshair did not negatively affect the field of vision. In terms of the median GOALS score, the operative performance was improved when the crosshair was used compared with verbal instructions only (median 15, IQR (11; 21) vs. median 12, IQR (5; 19), p < 0.001).

CONCLUSION

The crosshair is a simple, inexpensive, and widely available method to improve communication between instructors and novices in everyday training.

The Role of Augmented Reality in Surgical Training: A Systematic Review

This review aims to provide an update on the role of augmented reality (AR) in surgical training and investigate whether the use of AR improves performance measures compared to traditional approaches in surgical trainees.

PUBMED, EMBASE, Google Scholar, Cochrane Library, British Library and Science Direct were searched following PRIMSA guidelines. All English language original studies pertaining to AR in surgical training were eligible for inclusion. Qualitative analysis was performed and results were categorised according to simulator models, subsequently being evaluated using Messick's framework for validity and McGaghie's translational outcomes for simulation-based learning.

Of the 1132 results retrieved, 45 were included in the study. 29 platforms were identified, with the highest 'level of effectiveness' recorded as 3. In terms of validity parameters, 10 AR models received a strong 'content validity' score of 2.15 models had a 'response processes' score ≥ 1. 'Internal structure' and 'consequences' were largely not discussed. 'Relations to other variables' was the best assessed criterion, with 9 platforms achieving a high score of 2. Overall, the Microsoft HoloLens received the highest level of recommendation for both validity and level of effectiveness.

Augmented reality in surgical education is feasible and effective as an adjunct to traditional training. The Microsoft HoloLens has shown the most promising results across all parameters and produced improved performance measures in surgical trainees. In terms of the other simulator models, further research is required with stronger study designs, in order to validate the use of AR in surgical training.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

Augmented Reality in Surgery: A Scoping Review

Augmented reality (AR) is an innovative system that enhances the real world by superimposing virtual objects on reality. The aim of this study was to analyze the application of AR in medicine and which of its technical solutions are the most used. We carried out a scoping review of the articles published between 2019 and February 2022. The initial search yielded a total of 2649 articles. After applying filters, removing duplicates and screening, we included 34 articles in our analysis. The analysis of the articles highlighted that AR has been traditionally and mainly used in orthopedics in addition to maxillofacial surgery and oncology. Regarding the display application in AR, the Microsoft HoloLens Optical Viewer is the most used method. Moreover, for the tracking and registration phases, the marker-based method with a rigid registration remains the most used system. Overall, the results of this study suggested that AR is an innovative technology with numerous advantages, finding applications in several new surgery domains. Considering the available data, it is not possible to clearly identify all the fields of application and the best technologies regarding AR.

Examining the interaction of different factors on pointing precision when using handheld laser pointers

OBJECTIVE

Laser pointers are common teaching tools used during lessons. The pointing precision may influence the teaching effectiveness. In this study, we examined the effect of four external factors, namely aiming distance, target size, light condition and colour of the laser beam on the pointing precision.

RESULTS

Thirty participants (15 males and 15 females; age = 23.2 ± 4.3) were asked to aim at the target black circles with different sizes (diameters = 4 mm, 8 mm, 12 mm and 16 mm) from five various distances (2 m, 4 m, 6 m, 8 m and 10 m) at two brightness conditions (i.e., bright and dark) using two different coloured laser pointers (red and green). Three aiming parameters, namely number of hits, duration per hit and pointing precision were measured. Results showed that the aiming parameters were the highest with the aiming distance of 2 m and the use of green laser pointer towards larger target sizes regardless of the environmental brightness. Among all factors, aiming distance was the most important external factor that could influence pointing precision.