Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room

OBJECTIVE

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance.

BACKGROUND

Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons.

METHODS

A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context.

RESULTS

From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making.

CONCLUSIONS

The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Eye-tracking in surgery: a systematic review

BACKGROUND

Surgery is constantly evolving with the assistance of rapidly developing novel technology. Eye-tracking devices provide opportunities to monitor the acquisition of surgical skills, gain insight into performance, and enhance surgical practice. The aim of this review was to consolidate the available evidence for the use of eye-tracking in the surgical disciplines.

METHODS

A systematic literature review was conducted in accordance with PRISMA guidelines. A search of OVID Medline, EMBASE, Cochrane library, Scopus, and Science Direct was conducted January 2000 until December 2022. Studies involving eye-tracking in surgical training, assessment and technical innovation were included in the review. Non-surgical procedures, animal studies, and studies not involving surgical participants were excluded from the review.

RESULTS

The search returned a total of 12 054 articles, 80 of which were included in the final analysis and review. Seventeen studies involved eye-tracking in surgical training, 48 surgical assessment, and 20 were focussing on technical aspects of this technology. Twenty-six different eye-tracking devices were used in the included studies. Metrics such as the number of fixations, duration of fixations, dwell time, and cognitive workload were able to differentiate between novice and expert performance. Eight studies demonstrated the effectiveness of gaze-training for improving surgical skill.

CONCLUSION

The current literature shows a broad range of utility for a variety of eye-tracking devices in surgery. There remains a lack of standardization for metric parameters and gaze analysis techniques. Further research is required to validate its use to establish reliability and create uniform practices.

Telestration with augmented reality improves surgical performance through gaze guidance

BACKGROUND

In minimally invasive surgery (MIS), trainees need to learn how to interpret the operative field displayed on the laparoscopic screen. Experts currently guide trainees mainly verbally during laparoscopic procedures. A newly developed telestration system with augmented reality (iSurgeon) allows the instructor to display hand gestures in real-time on the laparoscopic screen in augmented reality to provide visual expert guidance (telestration). This study analysed the effect of telestration guided instructions on gaze behaviour during MIS training.

METHODS

In a randomized-controlled crossover study, 40 MIS naive medical students performed 8 laparoscopic tasks with telestration or with verbal instructions only. Pupil Core eye-tracking glasses were used to capture the instructor's and trainees' gazes. Gaze behaviour measures for tasks 1-7 were gaze latency, gaze convergence and collaborative gaze convergence. Performance measures included the number of errors in tasks 1-7 and trainee's ratings in structured and standardized performance scores in task 8 (ex vivo porcine laparoscopic cholecystectomy).

RESULTS

There was a significant improvement 1-7 on gaze latency [F(1,39) = 762.5, p < 0.01, ηp2 = 0.95], gaze convergence [F(1,39) = 482.8, p < 0.01, ηp2 = 0.93] and collaborative gaze convergence [F(1,39) = 408.4, p < 0.01, ηp2 = 0.91] upon instruction with iSurgeon. The number of errors was significantly lower in tasks 1-7 (0.18 ± 0.56 vs. 1.94 ± 1.80, p < 0.01) and the score ratings for laparoscopic cholecystectomy were significantly higher with telestration (global OSATS: 29 ± 2.5 vs. 25 ± 5.5, p < 0.01; task-specific OSATS: 60 ± 3 vs. 50 ± 6, p < 0.01).

CONCLUSIONS

Telestration with augmented reality successfully improved surgical performance. The trainee's gaze behaviour was improved by reducing the time from instruction to fixation on targets and leading to a higher convergence of the instructor's and the trainee's gazes. Also, the convergence of trainee's gaze and target areas increased with telestration. This confirms augmented reality-based telestration works by means of gaze guidance in MIS and could be used to improve training outcomes.

Scene-dependent, feedforward eye gaze metrics can differentiate technical skill levels of trainees in laparoscopic surgery

INTRODUCTION

In laparoscopic surgery, looking in the target areas is an indicator of proficiency. However, gaze behaviors revealing feedforward control (i.e., looking ahead) and their importance have been under-investigated in surgery. This study aims to establish the sensitivity and relative importance of different scene-dependent gaze and motion metrics for estimating trainee proficiency levels in surgical skills.

METHODS

Medical students performed the Fundamentals of Laparoscopic Surgery peg transfer task while recording their gaze on the monitor and tool activities inside the trainer box. Using computer vision and fixation algorithms, five scene-dependent gaze metrics and one tool speed metric were computed for 499 practice trials. Cluster analysis on the six metrics was used to group the trials into different clusters/proficiency levels, and ANOVAs were conducted to test differences between proficiency levels. A Random Forest model was trained to study metric importance at predicting proficiency levels.

RESULTS

Three clusters were identified, corresponding to three proficiency levels. The correspondence between the clusters and proficiency levels was confirmed by differences between completion times (F2,488 = 38.94, p < .001). Further, ANOVAs revealed significant differences between the three levels for all six metrics. The Random Forest model predicted proficiency level with 99% out-of-bag accuracy and revealed that scene-dependent gaze metrics reflecting feedforward behaviors were more important for prediction than the ones reflecting feedback behaviors.

CONCLUSION

Scene-dependent gaze metrics revealed skill levels of trainees more precisely than between experts and novices as suggested in the literature. Further, feedforward gaze metrics appeared to be more important than feedback ones at predicting proficiency.

Assessment of Core Surgical Skills Using a Mixed Reality Headset - The MoTOR Study

INTRODUCTION

Surgical skill assessment utilises direct observation and feedback by an expert which is potentially subjective, therefore obtaining objective data for hand and eye tracking is essential. Our aim was to evaluate a wearable mixed reality (MR) headset in these domains.

METHODS

Participants with differing levels of surgical expertise [novice (N), intermediate (I) & expert (E)] performed 4 simulated surgical tasks; 2 general dexterity (tasks 1&2) and 2 surgical skills (tasks 3&4) wearing the MR headset capturing their hand and eye movements (median & range). Metrics included hand path length and the speed of each index or thumb tip. Gaze data were also captured. Participant demographics, prior expertise and current experience were captured with an electronic survey. Data were analysed with a Shapiro-Wilk test or ANOVA as appropriate. A p-value of < 0.05 was significant.

RESULTS

Thirty-six participants were analysed (N = 18, I = 8, E = 8). Tasks 1&2 revealed 2 speed outcomes (left index and left-hand speed) which were significant. For tasks 3&4, various outcomes were significant: path length for left hand (N:45 cm vs. I:31 cm vs. E:27 cm, p = 0.03) and right hand (N:48 cm vs. I:29 cm vs. E:28 cm, p = 0.01) and total time (N:456s vs. I:292 vs. E: 245, p = 0.0002). With left-hand-tying, average path length (N:61 cm vs. I:39 vs. E:36, p = 0.04), average speed (N:11 cm/s vs. I:23 vs. E:24, p = 0.03), and total time (N:156s vs. I:43 vs. E:37, p = 0.003) were significant. The gaze-tracking was not statistically significant.

CONCLUSION

The MR headset can be utilised as a valid tool for surgical performance assessment. Outcomes including path length and speed can be valuable metrics captured by the MR Headset during the task completion for detecting surgical proficiency.

Visual gaze patterns in trainee endoscopists - a novel assessment tool

BACKGROUND

Colonoscopy proficiency is significantly influenced by skills achieved during training. Although assessment scores exist, they do not evaluate the impact of visual search strategies and their use is time and labour intensive. Eye-tracking has shown significant differences in visual gaze patterns (VGPs) between expert endoscopists with varying polyp detection rates, so may provide a means of automated assessment and guidance for trainees. This study aimed to assess the feasibility of eye-tracking as a novel assessment method for trainee endoscopists.

METHODS

Eye-tracking glasses were used to record 26 colonoscopies from 12 endoscopy trainees who were assessed with directly observed procedural scores (DOPS), devised by the Joint Advisory Group (JAG) on GI endoscopy, and a visual analogue score of overall competence. A 'total weighted procedure score' (TWPS) was calculated from 1 to 20. Primary outcomes of fixation duration (FixD) and fixation frequency (FixF) were analysed according to areas of interest (AOIs) with the bowel surface and lumen represented by three concentric rings. Correlation was assessed using Pearson's coefficient. Significance was set at p<.050.

RESULTS

Trainees displayed a significant positive correlation between TWPS and FixD (R = 0.943, p<.0001) and FixF (R = 0.936, p<.0001) in the anatomical bowel mucosa peripheries. Conversely, they had significant negative correlations between TWPS and the anatomical bowel lumen (FixD: R= -0.546, p=.004; FixF: R= -0.568, p=.002).

CONCLUSIONS

Higher objective performance scores were associated with VGPs focussing on bowel mucosa. This is consistent with prior analysis showing peripheral VGPs correspond with higher polyp detection rates. Analysis of VGPs, therefore, has potential for training and assessment in colonoscopy.

The Measurement of Cognitive Workload in Surgery Using Pupil Metrics: A Systematic Review and Narrative Analysis

INTRODUCTION

Increased cognitive workload (CWL) is a well-established entity that can impair surgical performance and increase the likelihood of surgical error. The use of pupil and gaze tracking data is increasingly being used to measure CWL objectively in surgery. The aim of this review is to summarize and synthesize the existing evidence that surrounds this.

METHODS

A systematic review was undertaken in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A search of OVID MEDLINE, IEEE Xplore, Web of Science, Google Scholar, APA PsychINFO, and EMBASE was conducted for articles published in English between 1990 and January 2021. In total, 6791 articles were screened and 32 full-text articles were selected based on the inclusion criteria. A narrative analysis was undertaken in view of the heterogeneity of studies.

RESULTS

Seventy-eight percent of selected studies were deemed high quality. The most frequent surgical environment and task studied was surgical simulation (75%) and performance of laparoscopic skills (56%) respectively. The results demonstrated that the current literature can be broadly categorized into pupil, blink, and gaze metrics used in the assessment of CWL. These can be further categorized according to their use in the context of CWL: (1) direct measurement of CWL (n = 16), (2) determination of expertise level (n = 14), and (3) predictors of performance (n = 2).

CONCLUSIONS

Eye-tracking data provide a wealth of information; however, there is marked study heterogeneity. Pupil diameter and gaze entropy demonstrate promise in CWL assessment. Future work will entail the use of artificial intelligence in the form of deep learning and the use of a multisensor platform to accurately measure CWL.

Eye Tracking Use in Surgical Research: A Systematic Review

INTRODUCTION

Eye tracking (ET) is a popular tool to study what factors affect the visual behaviour of surgical team members. To our knowledge, there have been no reviews to date that evaluate the broad use of ET in surgical research. This review aims to identify and assess the quality of this evidence, to synthesize how ET can be used to inform surgical practice, and to provide recommendations to improve future ET surgical studies.

METHODS

In line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic literature review was conducted. An electronic search was performed in MEDLINE, Cochrane Central, Embase, and Web of Science databases up to September 2020. Included studies used ET to measure the visual behaviour of members of the surgical team during surgery or surgical tasks. The included studies were assessed by two independent reviewers.

RESULTS

A total of 7614 studies were identified, and 111 were included for data extraction. Eleven applications were identified; the four most common were skill assessment (41%), visual attention assessment (22%), workload measurement (17%), and skills training (10%). A summary was provided of the various ways ET could be used to inform surgical practice, and three areas were identified for the improvement of future ET studies in surgery.

CONCLUSIONS

This review provided a comprehensive summary of the various applications of ET in surgery and how ET could be used to inform surgical practice, including how to use ET to improve surgical education. The information provided in this review can also aid in the design and conduct of future ET surgical studies.

Laparoscopic surgical skill evaluation with motion capture and eyeglass gaze cameras: A pilot study

INTRODUCTION

An eyeglass gaze camera and a skeletal coordinate camera without sensors attached to the operator's body were used to monitor gaze and movement during a simulated surgical procedure. These new devices have the potential to change skill assessment for laparoscopic surgery. The suitability of these devices for skill assessment was investigated.

MATERIAL AND METHODS

Six medical students, six intermediate surgeons, and four experts performed suturing tasks in a dry box. The tip positions of the instruments were identified from video recordings. Performance was evaluated based on instrument movement, gaze, and skeletal coordination.

RESULTS

Task performance time and skeletal coordinates were not significantly different among skill levels. The total movement distance of the right instrument was significantly different depending on the skill level. The SD of the gaze coordinates was significantly different depending on skill level and was less for experts. The expert's gaze stayed in a small area with little blurring.

CONCLUSIONS

The SD of gaze point coordinates correlates with laparoscopic surgical skill level. These devices may facilitate objective intraoperative skill evaluation in future studies.

Correlating Personal Resourcefulness and Psychomotor Skills: An Analysis of Stress, Visual Attention and Technical Metrics

Modern surgical education is focused on making use of the available technologies in order to train and assess surgical skill acquisition. Innovative technologies for the automatic, objective assessment of nontechnical skills are currently under research. The main aim of this study is to determine whether personal resourcefulness can be assessed by monitoring parameters that are related to stress and visual attention and whether there is a relation between these and psychomotor skills in surgical education. For this purpose, we implemented an application in order to monitor the electrocardiogram (ECG), galvanic skin response (GSR), gaze and performance of surgeons-in-training while performing a laparoscopic box-trainer task so as to obtain technical and personal resourcefulness' metrics. Eight surgeons (6 nonexperts and 2 experts) completed the experiment. A total of 22 metrics were calculated (7 technical and 15 related to personal resourcefulness) per subject. The average values of these metrics in the presence of stressors were compared with those in their absence and depending on the participants' expertise. The results show that both the mean normalized GSR signal and average surgical instrument's acceleration change significantly when stressors are present. Additionally, the GSR and acceleration were found to be correlated, which indicates that there is a relation between psychomotor skills and personal resourcefulness.

Eye-Tracking Indicators of Workload in Surgery: A Systematic Review

BackgroundEye tracking is a powerful tool for unobtrusive and real time assessment of workload in clinical settings. Before the complex eye tracking derived surrogates can be proactively utilized to improve surgical safety, the indications, validity and reliability requires careful evaluation.MethodsWe conducted a systematic review of literature from 2010 to 2020 according to PRISMA guidelines. A search on PubMed, Cochrane, Scopus, Web of science, PsycInfo and Google scholar databases was conducted on July 2020. The following search query was used" ("eye tracking" OR "gaze tracking") AND (surgery OR surgical OR operative OR intraoperative) AND (workload OR stress)". Short papers, no peer reviewed or papers in which eye-tracking methodology was not used to investigate workload or stress factors in surgery, were omitted.ResultsA total of 17 (N = 17) studies were identified eligible to this review. Most of the studies (n = 15) measured workload in simulated setting. Task difficulty and expertise were the most studied factors. Studies consistently showed surgeon's eye movements such as pupil responses, gaze patterns, blinks were associated with the level of perceived workload. However, differences between measurements in operational room and simulated environments have been found.ConclusionPupil responses, blink rate and gaze indices are valid indicators of workload. However, the effect of distractions and non-technical factors on workload is underrepresented aspect in the literature even though recognized as underlying factors in successful surgery.

A cost-effective IoT learning environment for the training and assessment of surgical technical skills with visual learning analytics

BACKGROUND

Surgeons need to train and certify their technical skills. This is usually done with the intervention of experts who monitor and assess trainees. Nevertheless, this is a time-consuming task that is subject to variations among evaluators. In recent decades, subjectivity has been significantly reduced through 1) the introduction of standard curricula, such as the Fundamentals of Laparoscopic Surgery (FLS) program, which measures students' performance in specific exercises, and 2) rubrics, which are widely accepted in the literature and serve to provide feedback about the overall technical skills of the trainees. Although these two elements reduce subjectivity, they do not, however, eliminate the figure of the expert evaluator, and so the process remains time consuming.

OBJECTIVES

The objective of this work is to automate those parts of the work of the expert evaluator that the technology can measure objectively, using sensors to collect evidence, and visualizations to provide feedback. We designed and developed 1) a cost-effective IoT (Internet of Things) learning environment for the training and assessment of surgical technical skills and 2) visualizations supported by the literature on visual learning analytics (VLA) to provide feedback about the exercises (in real time) and overall performance (at the end of the training) of the trainee.

METHODS

A hybrid approach was followed based on previous research for the design of the sensor based IoT learning environment. Previous studies were used as the basis for getting best practices on the tracking of surgical instruments and on the detection of the force applied to the tissue, with a focus on reducing the costs of data collection. The monitoring of the specific exercises required the design of sensors and collection mechanisms from scratch as there is little existing research on this subject. Moreover, it was necessary to design the overall architecture to collect, process, synchronize and communicate the data coming from the different sensors to provide high-level information relevant to the end user. The information to be presented was already validated by the literature and the focus was on how to visualize this information and the optimal time for its presentation to end users. The visualizations were validated with 18 VLA experts assessing the technical aspects of the visualizations and 4 medical experts assessing their functional aspects.

RESULTS

This IoT learning environment amplifies the evaluation mechanisms already validated by the literature, allowing automatic data collection. First, it uses IoT sensors to automatically correct two of the exercises defined in the FLS (peg transfer and precision cutting), providing real-time visualizations. Second it monitors the movement of the surgical instruments and the force applied to the tissues during the exercise, computing 6 of the high-level indicators used by expert evaluators in their rubrics (efficiency, economy of movement, hand tremor, depth perception, bimanual dexterity, and respect for tissue), providing feedback about the technical skills of the trainee using a radar chart with these six indicators at the end of the training (summative visualizations).

CONCLUSIONS

The proposed IoT learning environment is a promising and cost-effective alternative to help in the training and assessment of surgical technical skills. The system shows the trainees' progress and presents new indicators about the correctness of each specific exercise through real-time visualizations, as well as their general technical skills through summative visualizations, aligned with the 6 more frequent indicators in standardized scales. Early results suggest that although both types of visualizations are useful, it is necessary to reduce the cognitive load of the graphs presented in real time during training. Nevertheless, an additional evaluation is needed to confirm these results.

Years of experience is more effective in defining experts in the gaze analysis of laparoscopic suturing task than task duration

In this study, the relationship between gaze patterns, task duration, and years of experience, which are commonly used to define and evaluate expert surgeons in laparoscopic surgery, was identified. Ten surgeons with 1-28 years of experience and six inexperienced students were included. Subjects used forceps to repeat the task of suturing a suture pad. Each subject wore an eye-marking recorder, and gaze points were recorded and analyzed. No significant relationship between task duration and gaze pattern was observed. However, there was a significant relationship between a surgeon's years of experience and the percentage of time spent gazing at the forceps. Subjects with more years of experience operated without looking at the forceps and fixed their gaze on the operational target. Therefore, when analyzing laparoscopic gazing patterns, it may be more appropriate to define an "expert" based on the years of experience rather than task duration.

The Application of a System of Eye Tracking in Laparoscopic Surgery: A New Didactic Tool to Visual Instructions

Introduction: Laparoscopic surgery is an increasingly used technique, but it requires a high degree of learning, and communication between the operating room crew is considerably difficult. The use of eye tracking has been proposed as a didactic and evaluation tool in several settings, including in laparoscopy in simulators.

Objectives: This study aimed to evaluate the usefulness of the use of eye tracking systems (Tobii glasses 2) in laparoscopic surgery as a didactic and assessment tool to improve communication in the operating room and improve patients' security.

Methodology: An anonymous survey was sent to the students and medical teachers of a faculty of medicine and practicing doctors and residents. The message contained an explanation about the use of the Tobii glasses, a link to watch the video showing its use in a laparoscopic surgery, and the survey to complete after watching the video.

Results: The survey was answered by 113 participants (51.3% medical students, 27.4% medical teachers, 18.6% practicing doctors, and 2.7% medicine residents). Eighty-three percent agreed with the usefulness of the “Tobii glasses” in the operating room for improving communication between the main surgeon and the assistant, for learning complex surgery techniques, for obtaining didactic videos, and for indicating anatomical structures. The item scored worst was the price of the glasses.

Conclusions: It is possible to record and project expert gaze patterns in the operating room in real time using the Tobii glasses. This device allows improving communication among the surgical crew and the learning of residents and also improving the security of surgical patients.

A novel gaze-controlled flexible robotized endoscope; preliminary trial and report

Background

Interventional endoluminal therapy is rapidly advancing as a minimally invasive surgical technique. The expanding remit of endoscopic therapy necessitates precision control. Eye tracking is an emerging technology which allows intuitive control of devices. This was a feasibility study to establish if a novel eye gaze-controlled endoscopic system could be used to intuitively control an endoscope.

Methods

An eye gaze-control system consisting of eye tracking glasses, specialist cameras and a joystick was used to control a robotically driven endoscope allowing steering, advancement, withdrawal and retroflexion. Eight experienced and eight non-endoscopists used both the eye gaze system and a conventional endoscope to identify ten targets in two simulated environments: a sphere and an upper gastrointestinal (UGI) model.

Completion of tasks was timed. Subjective feedback was collected from each participant on task load (NASA Task Load Index) and acceptance of technology (Van der Laan scale).

Results

When using gaze-control endoscopy, non-endoscopists were significantly quicker when using gaze-control rather than conventional endoscopy (sphere task 3:54 ± 1:17 vs. 9:05 ± 5:40 min, p = 0.012, and UGI model task 1:59 ± 0:24 vs 3:45 ± 0:53 min, p < .001).

Non-endoscopists reported significantly higher NASA-TLX workload total scores using conventional endoscopy versus gaze-control (80.6 ± 11.3 vs 22.5 ± 13.8, p < .001). Endoscopists reported significantly higher total NASA-TLX workload scores using gaze control versus conventional endoscopy (54.2 ± 16 vs 26.9 ± 15.3, p = 0.012). All subjects reported that the gaze-control had positive ‘usefulness’ and ‘satisfaction’ score of 0.56 ± 0.83 and 1.43 ± 0.51 respectively.

Conclusions

The novel eye gaze-control system was significantly quicker to use and subjectively lower in workload when used by non-endoscopists. Further work is needed to see if this would translate into a shallower learning curve to proficiency versus conventional endoscopy. The eye gaze-control system appears feasible as an intuitive endoscope control system. Hybrid gaze and hand control may prove a beneficial technology to evolving endoscopic platforms.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-021-08556-1.

Using Eye Tracking for Measuring Cognitive Workload During Clinical Simulations: Literature Review and Synthesis

High-fidelity clinical simulations can be used by clinicians to acquire technical (physical ability and knowledge) and non-technical (cognitive and social processes) skills. Excessive cognitive workload contributes to medical errors because of the impact on both technical and non-technical skills. Many studies measure cognitive workload with psychometric instruments that limit the assessment of cognitive workload to a single time period and may involve response bias. Using eye tracking to measure task-evoked pupillary responses allows the measurement of changes in pupil diameter related to the cognitive workload associated with a specific activity. Incorporating eye tracking with high-fidelity clinical simulations provides a reliable and continuous assessment of cognitive workload. The purpose of this literature review is to summarize the use of eye-tracking technology to measure cognitive workload of healthcare providers to generate evidence-based guidelines for measuring cognitive workload during high-fidelity clinical simulations. What this manuscript adds to the body of literature is a summary of best practices related to the different methods of measuring cognitive workload, benefits and limitations of using eye tracking, and high-fidelity clinical simulation design considerations for successful integration of eye tracking.

Eye Tracking Supported Human Factors Testing Improving Patient Training

The handling of left ventricular assist devices (LVADs) can be challenging for patients and requires appropriate training. The devices' usability impacts patients' safety and quality of life. In this study, an eye tracking supported human factors testing was performed to reveal problems during use and test the trainings' effectiveness. In total 32 HeartWare HVAD patients (including 6 pre-VAD patients) and 3 technical experts as control group performed a battery change (BC) and a controller change (CC) as an everyday and emergency scenario on a training device. By tracking the patients' gaze point, task duration and pump-off time were evaluated. Patients with LVAD support ≥1 year showed significantly shorter BC task duration than patients with LVAD support <1 year (p = 0.008). In contrast their CC task duration (p = 0.002) and pump-off times (median = 12.35 s) were higher than for LVAD support patients <1 year (median = 5.3 s) with p = 0.001. The shorter BC task duration for patients with LVAD support ≥1 year indicate that with time patients establish routines and gain confidence using their device. The opposite effect was found for CC task duration and pump-off times. This implies the need for intermittent re-training of less frequent tasks to increase patients' safety.

Computer Vision in the Operating Room: Opportunities and Caveats

Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Contemporary training methods in regional anaesthesia: fundamentals and innovations

Over the past two decades, regional anaesthesia and medical education as a whole have undergone a renaissance. Significant changes in our teaching methods and clinical practice have been influenced by improvements in our theoretical understanding as well as by technological innovations. More recently, there has been a focus on using foundational education principles to teach regional anaesthesia, and the evidence on how to best teach and assess trainees is growing. This narrative review will discuss fundamentals and innovations in regional anaesthesia training. We present the fundamentals in regional anaesthesia training, specifically the current state of simulation-based education, deliberate practice and curriculum design based on competency-based progression. Moving into the future, we present the latest innovations in web-based learning, emerging technologies for teaching and assessment and new developments in alternate reality learning systems.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Pathologist Experience and Concordance in the Diagnosis of Dysplasia in Long-standing Inflammatory Bowel Disease

Surveillance colonoscopies focused to detect dysplasia are recommended to prevent colorectal cancer in patients with long-standing colonic inflammatory bowel disease (IBD). To date, histologic diagnosis and gradation of IBD-related dysplasia has been challenged by a high variability among pathologists. We aimed to analyze the observer characteristics that are correlated with concordance deviations in this diagnosis. Eight pathologists evaluated a set of 125 endoscopic biopsy samples with a representative distribution of nondysplastic and dysplastic lesions from long-standing IBD patients. Two rounds of diagnosis were carried out during a period of 18 months. The κ test was applied to analyze concordance. Pathologists were grouped on the basis of their experience. A subanalysis was performed by eliminating the highly prevalent nondysplastic samples, as well as an analysis after observers' grouping. Overall interobserver agreement was good (κ=0.73), with an even higher pairwise value (κ=0.86) as well as the intraobserver agreement values (best κ=0.85). After eliminating the highly prevalent nondysplastic samples, the interobserver agreement was still moderate to good (best overall κ=0.50; best paired κ=0.72). Notable differences were seen between the pathologists with a high-volume and low-volume practice (best overall κ=0.61 and 0.41, respectively). The agreement in the diagnosis of dysplasia in IBD endoscopic biopsies may have been undervalued over time. This is the first study evaluating pathologists' diagnostic robustness in this field. The results suggest that examining a large volume of samples is the key factor to increase the consistency in the diagnosis and gradation of IBD-related dysplasia.

Performance Optimization Strategies for Complex Endourologic Procedures

OBJECTIVE

To identify and address factors that may impact a surgeon's performance during endourologic procedures.

METHODS

A literature review was performed for articles focusing on surgical ergonomics, education, sports and performance psychology.

RESULTS

As urologists and trainees have become more comfortable approaching complex pathology endoscopically, there remains an opportunity to refine surgeon-related factors and optimize extrinsic factors to maximize efficiency and provide patients with the highest quality outcomes and safety.

CONCLUSION

Medical centers and training programs should strive to include formal lessons on stress-coping mechanisms, communication, and dedicated ergonomic training, as these all play a role in physician well-being and may lead to improved clinical outcomes.