Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations

Feasibility of Wearable Sensors to Assess Cognitive Load During Clinical Performance: Lessons Learned and Blueprint for Success

INTRODUCTION

Cognitive load (CogL) is increasingly recognized as an important resource underlying operative performance. Current innovations in surgery aim to develop objective performance metrics via physiological monitoring from wearable digital sensors. Surgeons have access to consumer technology that could measure CogL but need guidance regarding device selection and implementation. To realize the benefits of surgical performance improvement these methods must be feasible, incorporating human factors usability and design principles. This paper aims to evaluate the feasibility of using wearable sensors to assess CogL, identify the benefits and challenges of implementing devices, and develop guidance for surgeons planning to implement wearable devices in their research or practice.

METHODS

We examined the feasibility of wearable sensors from a series of empirical studies that measured aspects of clinical performance relating to CogL. Across four studies, 84 participants and five sensors were involved in the following clinical settings: (i) real intraoperative surgery; (ii) simulated laparoscopic surgery; and (iii) medical team performance outside the hospital.

RESULTS

Wearable devices worn on the wrist and chest were found to be comfortable. After a learning curve, electrodermal activity data were easily and reliably collected. Devices using photoplethysmography to determine heart rate variability were significantly limited by movement artifact. There was variable success with electroencephalography devices regarding connectivity, comfort, and usability.

CONCLUSIONS

It is feasible to use wearable sensors across various clinical settings, including surgery. There are some limitations, and their implementation is context and device dependent. To scale sensor use in clinical research, surgeons must embrace human factors principles to optimize wearability, usability, reliability, and data security.

Public's Perception and Knowledge of Using Robotics in General Surgery in Eastern Region, Saudi Arabia

BACKGROUND

Robotics in general surgery is a field that involves the use of robotic systems to assist surgeons in performing various types of surgical procedures.

OBJECTIVE

The objective of this study was to evaluate the perception and knowledge of robotic surgery among the Eastern Region's population.

METHODS

This cross-sectional study used an electronic questionnaire that was developed using Google Docs. It included males and females aged above 18 years who lived in the Eastern Province. Participants who were below 18 or above 65 years of age, or non-Saudi, or people who lived in other than the Eastern Region of Saudi Arabia were excluded from the study.

RESULTS

A total of 500 responses were received via the Google Form, and 81 subjects of them were excluded from the study. Approximately half of the participants were aware of the existence of general robotic surgery, while the other half had no prior knowledge about it. When assessing the participants' understanding of how robotic surgery works, a significant proportion provided incorrect responses. In terms of the advantages of general robotic surgery, the most commonly recognized benefit was that it makes the doctor's life easier, followed by more accurate surgical results. However, the participants' understanding of the disadvantages of robotic surgery was not as accurate. A substantial portion of participants were unsure about the disadvantages.

CONCLUSION

The general public of the Eastern Region in Saudi Arabia showed a derated level of knowledge about the use of robotics in general surgery. Furthermore, a major portion of people were unaware of the availability of robotic surgery in Saudi Arabia. Educational programs are warranted to facilitate the implantation of robotic surgery in Saudi Arabia.

Deep Learning Analysis of Surgical Video Recordings to Assess Nontechnical Skills

Importance

Assessing nontechnical skills in operating rooms (ORs) is crucial for enhancing surgical performance and patient safety. However, automated and real-time evaluation of these skills remains challenging.

Objective

To explore the feasibility of using motion features extracted from surgical video recordings to automatically assess nontechnical skills during cardiac surgical procedures.

Design, Setting, and Participants

This cross-sectional study used video recordings of cardiac surgical procedures at a tertiary academic US hospital collected from January 2021 through May 2022. The OpenPose library was used to analyze videos to extract body pose estimations of team members and compute various team motion features. The Non-Technical Skills for Surgeons (NOTSS) assessment tool was employed for rating the OR team's nontechnical skills by 3 expert raters.

Main Outcomes and Measures

NOTSS overall score, with motion features extracted from surgical videos as measures.

Results

A total of 30 complete cardiac surgery procedures were included: 26 (86.6%) were on-pump coronary artery bypass graft procedures and 4 (13.4%) were aortic valve replacement or repair procedures. All patients were male, and the mean (SD) age was 72 (6.3) years. All surgical teams were composed of 4 key roles (attending surgeon, attending anesthesiologist, primary perfusionist, and scrub nurse) with additional supporting roles. NOTSS scores correlated significantly with trajectory (r = 0.51, P = .005), acceleration (r = 0.48, P = .008), and entropy (r = -0.52, P = .004) of team displacement. Multiple linear regression, adjusted for patient factors, showed average team trajectory (adjusted R2 = 0.335; coefficient, 10.51 [95% CI, 8.81-12.21]; P = .004) and team displacement entropy (adjusted R2 = 0.304; coefficient, -12.64 [95% CI, -20.54 to -4.74]; P = .003) were associated with NOTSS scores.

Conclusions and Relevance

This study suggests a significant link between OR team movements and nontechnical skills ratings by NOTSS during cardiac surgical procedures, suggesting automated surgical video analysis could enhance nontechnical skills assessment. Further investigation across different hospitals and specialties is necessary to validate these findings.

Standardization and Implementation of a Surgical Coaching Model for Pediatric Urology

To bridge gaps in proficiency and encourage life-long learning following training, coaching models have been utilized in multiple surgical fields; however, not within pediatric urology. In this review of our methodology, we describe the development of a coaching model at a single institution. In our initial experience, the perceived most beneficial aspect of the program was the goal setting process with logistics around debriefs being the most challenging. With our proposed coaching study, we aim to develop a model based upon prior coaching frameworks,1,2 that is feasible and universally adaptable to allow for further advancement of surgical coaching, particularly within the field of pediatric urology.

Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review

When technical requirements are high, and patient outcomes are critical, opportunities for monitoring and improving surgical skills via objective motion analysis feedback may be particularly beneficial. This narrative review synthesises work on technical and non-technical surgical skills, collaborative task performance, and pose estimation to illustrate new opportunities to advance cardiothoracic surgical performance with innovations from computer vision and artificial intelligence. These technological innovations are critically evaluated in terms of the benefits they could offer the cardiothoracic surgical community, and any barriers to the uptake of the technology are elaborated upon. Like some other specialities, cardiothoracic surgery has relatively few opportunities to benefit from tools with data capture technology embedded within them (as is possible with robotic-assisted laparoscopic surgery, for example). In such cases, pose estimation techniques that allow for movement tracking across a conventional operating field without using specialist equipment or markers offer considerable potential. With video data from either simulated or real surgical procedures, these tools can (1) provide insight into the development of expertise and surgical performance over a surgeon's career, (2) provide feedback to trainee surgeons regarding areas for improvement, (3) provide the opportunity to investigate what aspects of skill may be linked to patient outcomes which can (4) inform the aspects of surgical skill which should be focused on within training or mentoring programmes. Classifier or assessment algorithms that use artificial intelligence to 'learn' what expertise is from expert surgical evaluators could further assist educators in determining if trainees meet competency thresholds. With collaborative efforts between surgical teams, medical institutions, computer scientists and researchers to ensure this technology is developed with usability and ethics in mind, the developed feedback tools could improve cardiothoracic surgical practice in a data-driven way.

Collaborative Team Training in Virtual Reality is Superior to Individual Learning For Performing Complex Open Surgery: A Randomized Controlled Trial

OBJECTIVE

To assess whether multiplayer immersive Virtual Reality (iVR) training was superior to single-player training for the acquisition of both technical and nontechnical skills in learning complex surgery.

BACKGROUND

Superior teamwork in the operating room (OR) is associated with improved technical performance and clinical outcomes. iVR can successfully train OR staff individually; however, iVR team training has yet to be investigated.

METHODS

Forty participants were randomized to individual or team iVR training. Individually trained participants practiced alongside virtual avatar counterparts, whereas teams trained live in pairs. Both groups underwent 5 iVR training sessions over 6 weeks. Subsequently, they completed a real-life assessment in which they performed anterior approach total hip arthroplasty surgery on a high-fidelity model with real equipment in a simulated OR. Teams performed together, and individually trained participants were randomly paired up. Videos were marked by 2 blinded assessors recording the 'Non-Operative Technical Skills for Surgeons, Oxford NOn-TECHnical Skills II and Scrub Practitioners' List of Intraoperative Non-Technical Skills' scores. Secondary outcomes were procedure duration and the number of technical errors.

RESULTS

Teams outperformed individually trained participants for nontechnical skills in the real-world assessment (Non-Operative Technical Skills for Surgeons: 13.1±1.5 vs 10.6±1.6, P = 0.002, Non-TECHnical Skills II score: 51.7 ± 5.5 vs 42.3 ± 5.6, P = 0.001 and Scrub Practitioners' List of Intraoperative Non-Technical Skills: 10 ± 1.2 vs 7.9 ± 1.6, P = 0.004). They completed the assessment 33% faster (28.2 minutes ± 5.5 vs 41.8 ± 8.9, P < 0.001), and made fewer than half the number of technical errors (10.4 ± 6.1 vs 22.6 ± 5.4, P < 0.001).

CONCLUSIONS

Multiplayer training leads to faster surgery with fewer technical errors and the development of superior nontechnical skills.

A tool to assess nontechnical skills of perfusionists in the cardiac operating room

OBJECTIVES

This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability.

METHODS

An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists' Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed "good performance," and 2 videos showed "poor performance." Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported.

RESULTS

The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on "poor performance" and "good performance" videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level.

CONCLUSIONS

The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.

The implications of fragmented practice in hepatopancreatic surgery

BACKGROUND

Familiarity with the surgical work environment has been demonstrated to improve outcomes. We sought to investigate the impact of the rate of fragmented practice on textbook outcomes, a validated composite outcome representing an "optimal" postoperative course.

METHODS

Patients who underwent a hepatic or pancreatic surgical procedure between 2013 and 2017 were identified from the Medicare Standard Analytic Files. The rate of fragmented practice was defined as the surgeon's volume over the study period relative to the number of facilities practiced at. The association between the rate of fragmented practice and textbook outcomes was assessed using multivariable logistic regression.

RESULTS

A total of 37,599 patients were included (pancreatic: n = 23,701, 63.0%; hepatic: n = 13,898, 37.0%). After controlling for relevant characteristics, patients who underwent surgery by surgeons in higher rate of fragmented practice categories had lower odds of achieving a textbook outcome (reference: low rate of fragmented practice; intermediate rate of fragmented practice: odds ratio = 0.88 [95% confidence interval 0.84-0.93]; high rate of fragmented practice: odds ratio = 0.58 [95% confidence interval 0.54-0.61]) (both P < .001). Of note, the adverse effect of a high rate of fragmented practice on the achievement of textbook outcomes remained substantial, regardless of the county-level social vulnerability index [high rate of fragmented practice; low social vulnerability index: odds ratio = 0.58 (95% confidence interval 0.52-0.66); intermediate social vulnerability index: odds ratio = 0.56 (95% confidence interval 0.52-0.61); high social vulnerability index: odds ratio = 0.60 (95% confidence interval 0.54-0.68)] (all P < .001). Patients in intermediate and high social vulnerability index counties had 19% and 37% greater odds of undergoing surgery by a high rate of fragmented practice surgeon (reference: low social vulnerability index; intermediate social vulnerability index: odds ratio = 1.19 [95% confidence interval 1.12-1.26]; high social vulnerability index: odds ratio = 1.37 [95% confidence interval 1.28-1.46]).

CONCLUSION

Owing to the impact of the rate of fragmented practice on postoperative outcomes, decreasing fragmentation of care may be an important target for quality initiatives and a means to alleviate social disparities in surgical care.

Senior surgeons as role models in the operating theatre: a thematic analysis through the lens of aristotelian ethics

BACKGROUND

Surgeons are commonly evaluated by surgical skills and outcomes rather than their character traits. We sought to examine role model behaviours of senior surgeons through the lens of Aristotelian (virtue) ethics.

METHODS

Semi-structured focus group interviews were undertaken of anaesthetic trainees at a large university hospital NHS Foundation Trust and transcripts were subjected to thematic analysis to yield themes and subthemes. Participation of the trainees was entirely voluntary and focus groups were conducted using Zoom™.

RESULTS

The overarching themes identified were 'Teamwork makes the dream work', 'Captain of the ship' and 'Strong foundations'.

CONCLUSION

We hope to take lessons learnt in conjunction with our previous work to help refocus surgical training towards a process of character reformation, rather than simply imparting technical skills to trainees.

Intraoperative Assessment of Non-Technical Skills for Surgeons (NOTSS) and Qualitative Description of their Effects on Intraoperative Performance

OBJECTIVE

The aims of this study were (1) to evaluate the feasibility of using the Non-Technical Skills for Surgeons (NOTSS) rating tool in assessing surgeons' non-technical skills behaviors in live operations, and (2) to describe the effect of NOTSS on intraoperative performance.

SETTING DESIGN

This study was conducted in an academic hospital in North America. Two observers independently conducted direct non-participant observations using the NOTSS rating tool to assess non-technical skills, and to document examples of effective or ineffective non-technical skills behaviors. Observers took field notes to document non-technical skill gaps that were not captured by the NOTSS rating tool, and situations or scenarios that presented challenges for accurate assessment. Interclass correlation estimates and 95% confidence intervals were calculated to assess the validity of the NOTSS rating tool. Deductive thematic qualitative data analysis was used for field notes and NOTSS behavior descriptions.

PARTICIPANTS

Participants were general surgeons performing either minimally invasive (robotic assisted or laparoscopic surgery), or open procedures.

RESULTS

We observed 18 surgeries, involving 6 surgeons, 11 residents and one fellow resulting in 37 hours of direct observations. The mean NOTSS score was 3.8 (SD 0.41) for situation awareness, 3.75 (SD 0.47) for decision-making, 3.71 (SD 0.39) for communication and teamwork, and 3.76 (SD 0.38) for leadership. The inter-rater reliability ranged between 0.65 and 0.80 for each NOTSS categories. The observers documented examples of effective non-technical skills behaviors and examples of behaviors that need improvement. Furthermore, we described the effect of each observed behavior on intraoperative performance. One challenge to NOTSS use in live surgery was that observers had to infer situation awareness, decision-making, and coping with pressure as these were not easily observed without attending surgeons articulating their underlying thought process.

CONCLUSION

The use of the NOTSS tool in live surgery is a valid and practical tool to document observed behaviors and their effect on intraoperative performance in order to provide constructive feedback to surgeons. One notable limitation is that without specific articulation by the surgeon of their underlying thought process the observer must infer specific elements. By documenting specific real-world events with high inter-rater reliability and adequate surgeon score variation the process can be used to provide useful feedback for improvement.

Assessing Team Situational Awareness in the Operating Room via Computer Vision

Situational awareness (SA) at both individual and team levels, plays a critical role in the operating room (OR). During the pre-incision time-out, the entire OR team comes together to deploy the surgical safety checklist (SSC). Worldwide, the implementation of the SSC has been shown to reduce intraoperative complications and mortality among surgical patients. In this study, we investigated the feasibility of applying computer vision analysis on surgical videos to extract team motion metrics that could differentiate teams with good SA from those with poor SA during the pre-incision time-out. We used a validated observation-based tool to assess SA, and a computer vision software to measure body position and motion patterns in the OR. Our findings showed that it is feasible to extract surgical team motion metrics captured via off-the-shelf OR cameras. Entropy as a measure of the level of team organization was able to distinguish surgical teams with good and poor SA. These findings corroborate existing studies showing that computer vision-based motion metrics have the potential to integrate traditional observation-based performance assessments in the OR.

Non-technical Skills for Medical Students: Validating the Tools of the Trade

The Medical Students’ Non-Technical Skills (Medi-StuNTS) is a behavioural marker scheme (BMS) designed to assess non-technical skills (NTS) in medical students in emergency simulations. This study aimed to assess the evidence for validity and usability of Medi-StuNTS by naive, near-peer educators. Nine doctors assessed four students in simulations of common medical emergencies. The scores were used to assess inter-rater reliability, inter-class correlation, and observability. Students and assessors completed questionnaires that assessed the tool’s usability and consequence.

Inter-rater agreement across all skill elements was “high” with rWG scores >0.8. An inter-class correlation was “good” with ICC3K kappa scores of 0.86 and 0.89 overall, when measured per simulation and per skills element respectively. Overall skill observability was high (>80%) except for coping with stress. Assessors found the tool “difficult to use” but “useful for feeding back in a constructive way”. Students appreciated the comprehensiveness of the feedback as well as knowing what to expect during debriefs.

This study has shown that the Medi-StuNTS BMS has good usability and evidence of validity in naive assessors and near-peer educators. It shows the particularly good internal structure and overall beneficial consequences. Further study will be necessary to understand how best to deploy it in formative and summative contexts.

Integrative Review of Non-Technical Skills Frameworks to Apply for Air Medical Transfer of Pregnant Women

OBJECTIVE

To identify the most suitable non-technical skills framework to adapt and apply to the air medical transfer of pregnant women.

DATA SOURCES

Embase, PsycINFO, PubMed, MEDLINE, Web of Science, CINAHL, Science Direct, and Google Scholar.

STUDY SELECTION

We retrieved potentially relevant articles using a predefined combination of keywords extended with truncation and Boolean operators. Database and manual reference searches yielded 569 peer-reviewed articles. We included articles if they presented empirical data and described non-technical or cognitive competency skills frameworks for health care professionals. We discussed any ambiguities regarding inclusion, and they were resolved by consensus. We retained 71 full-text articles for final review.

DATA EXTRACTION

We coded extracted data under four criteria: non-technical skill categories, context of use, psychometric properties, and rating system. We generated descriptive summary tables of the characteristics of existing non-technical skills frameworks based on publication year, method of development, clinical setting, clinical specialty, routine/crisis-based performance, and team/individual performance.

DATA SYNTHESIS

We identified 42 non-technical skills frameworks from a variety of health care settings. We critically examined context of use and how use in various clinical settings may align with air transfers of pregnant women. Our findings illustrate the importance of team-based and routine performance rather than crisis-focused skills. Maintaining situational awareness throughout all stages of the transfer and communicating effectively with team members, the pregnant woman, and her partner are skills that are particularly important to ensure good outcomes.

CONCLUSION

We selected the Global Assessment of Obstetric Team Performance as the most suitable non-technical skills framework to adapt to the clinical setting of air medical transfer of pregnant women. We considered the clinical specialty, specific non-technical skills required in the setting, the framework's properties, and the requirement to focus on routine team performance.

The role of teamwork and non-technical skills for improving emergency surgical outcomes: an international perspective

The assurance of patient safety in emergency general surgery remains challenging due to the patients’ high-risk underlying conditions and the wide variability in emergency surgical care provided around the globe. The authors of this article convened as an expert panel on patient safety in surgery at the 8^th International Conference of the World Society of Emergency Surgery (WSES) in Edinburgh, Scotland, on September 7–10, 2021. This review article represents the proceedings from the expert panel discussions at the WSES congress and was designed to provide an international perspective on optimizing teamwork and non-technical skills in emergency general surgery.

Measuring and Improving Emotional Intelligence in Surgery: A Systematic Review

OBJECTIVE

Evaluate how emotional intelligence (EI) has been measured among surgeons and to investigate interventions implemented for improving EI.

SUMMARY BACKGROUND

EI has relevant applications in surgery given its alignment with nontechnical skills. In recent years, EI has been measured in a surgical context to evaluate its relationship with measures such as surgeon burnout and the surgeon-patient relationship.

METHODS

A systematic review was conducted by searching MEDLINE, EMBASE, CINAHL, and PSYCINFO databases using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. MeSH terms and keywords included "emotional intelligence," "surgery," and "surgeon." Eligible studies included an EI assessment of surgeons, surgical residents, and/or medical students within a surgical context.

RESULTS

The initial search yielded 4627 articles. After duplicate removal, 4435 articles were screened by title and abstract and 49 articles proceeded to a full-text read. Three additional articles were found via hand search. A total of 37 articles were included. Studies varied in surgical specialties, settings, and outcome measurements. Most occurred in general surgery, residency programs, and utilized self-report surveys to estimate EI. Notably, EI improved in all studies utilizing an intervention.

CONCLUSIONS

The literature entailing the intersection between EI and surgery is diverse but still limited. Generally, EI has been demonstrated to be beneficial in terms of overall well-being and job satisfaction while also protecting against burnout. EI skills may provide a promising modifiable target to achieve desirable outcomes for both the surgeon and the patient. Future studies may emphasize the relevance of EI in the context of surgical teamwork.

Team emergency assessment measure (TEAM) of non-technical skills: The Brazilian Portuguese version of the TEAM tool

OBJECTIVES

The aim of this study was to conduct the translation and cross-cultural adaptation of the original Team Emergency Assessment Measure (TEAM) tool into the Brazilian Portuguese language and investigate the internal consistency, inter-rater reliability, and concurrent validity of this new version (bp-TEAM).

METHODS

Independent medical translators performed forward and backward translations of the TEAM tool between English and Portuguese, creating the bp-TEAM. The authors selected 23 videos from final-year medical students during in-situ emergency simulations. Three independent raters assessed all the videos using the bp-TEAM and provided a score for each of the 12 items of the tool. The authors assessed the internal consistency and the inter-rater reliability of the tool.

RESULTS

Raters assessed all 23 videos. Internal consistency was assessed among the 11 items of the bp-TEAM from one rater, yielding a Cronbach's alpha of 0.89. inter-item correlation analysis yielded a mean correlation coefficient rho of 0.46. Inter-rater reliability analysis among the three raters yielded an intraclass correlation coefficient of 0.86 (95% CI 0.83‒0.89), p < 0.001.

CONCLUSION

The Brazilian Portuguese version of the TEAM tool presented acceptable psychometric properties, similar to the original English version.

The impact of team familiarity on intra and postoperative cardiac surgical outcomes

BACKGROUND

Familiarity among cardiac surgery team members may be an important contributor to better outcomes and thus serve as a target for enhancing outcomes.

METHODS

Adult cardiac surgical procedures (n = 4,445) involving intraoperative providers were evaluated at a tertiary hospital between 2016 and 2020. Team familiarity (mean of prior cardiac surgeries performed by participating surgeon/nonsurgeon pairs within 2 years before the operation) were regressed on cardiopulmonary bypass duration (primary-an intraoperative measure of care efficiency) and postoperative complication outcomes (major morbidity, mortality), adjusting for provider experience, surgeon 2-year case volume before the surgery, case start time, weekday, and perioperative risk factors. The relationship between team familiarity and outcomes was assessed across predicted risk strata.

RESULTS

Median (interquartile range) cardiopulmonary bypass duration was 132 (91-192) minutes, and 698 (15.7%) patients developed major postoperative morbidity. The relationship between team familiarity and cardiopulmonary bypass duration significantly differed across predicted risk strata (P = .0001). High (relative to low) team familiarity was associated with reduced cardiopulmonary bypass duration for medium-risk (-24 minutes) and high-risk (-27 minutes) patients. Increasing team familiarity was not significantly associated with the odds of major morbidity and mortality.

CONCLUSION

Team familiarity, which was predictive of improved intraoperative efficiency without compromising major postoperative outcomes, may serve as a novel quality improvement target in the setting of cardiac surgery.

Surgeons Are Not Pilots: Is the Aviation Safety Paradigm Relevant to Modern Surgical Practice?

Error in surgery is common, although not always consequential. Surgical outcomes are often compared to safety data from commercial aviation. This industry's performance is frequently referenced as an example of high-reliability that should be reproduced in clinical practice. Consequently, the aviation-surgery analogy forms the conceptual framework for much patient safety research, advocating for the translation of aviation safety tools to the healthcare setting. Nevertheless, overuse or incorrect application of this paradigm can be misleading and may result in ineffective quality improvement interventions. This article discusses the validity and relevance of the aviation-surgery comparison, providing the necessary context to improve its application at the bedside. It addresses technical and human factors training, as well as more novel performance domains such as professional culture and optimization of operators' condition. These are used to determine whether the aviation-surgery analogy is a valuable source of cross-professional learning or simply another safety cliché.

Surgical Simulation in East, Central, and Southern Africa: A Multinational Survey

BACKGROUND

High-income countries have increased the use of simulation-based training and assessment for surgical education. Learners in low- and middle-income countries may have different educational needs and levels of autonomy but they and their patients could equally benefit from the procedural training simulation provides. We sought to characterize the current state of surgical skills simulation in East, Central, and Southern Africa and determine residents' perception and future interest in such activities.

METHODS

A survey was created via collaboration and revision between trainees and educators with experiences spanning high-income countries and low- and middle-income countries. The survey was administered on paper to 76 trainees (PGY2-3) who were completing the College of Surgeons of East, Central, and Southern Africa (COSECSA) Membership of the College of Surgeons examination in Kampala, Uganda in December 2019. Data from paper responses were summarized using descriptive statistics and frequencies.

RESULTS

We received responses from 43 trainees (57%) from 11 countries in sub-Saharan Africa who participated in the examination. Fifty-eight percent of respondents reported having dedicated space for surgical skills simulation training, and most (91%) had participated in some form of simulation activity at some point in their training. However, just 16% used simulation as a regular part of training. The majority of trainees (90%) felt that surgical skills learned in simulation were transferrable to the operating room and agreed it should be a required part of training. Seventy-one percent of trainees felt that simulation could objectively measure technical skills, and 73% percent of respondents agreed that simulation should be integrated into formal assessment. However, residents split on whether proficiency in simulation should be achieved prior to operative experience (54%) and if nontechnical skills could be measured (51%). The most common cited barriers to the integration of surgical simulation into residents' education were lack of suitable tools and models (85%), funding (73%), and maintenance of facilities (49%).

CONCLUSIONS

Residents from East, Central, and Southern Africa strongly agree that simulation is a valuable educational tool and ought to be required during their surgical residency. Barriers to achieving this goal include availability of affordable tools, adequate funding and confidence in the value of the educational experience. Trainees affirm further efforts are necessary to make simulation more widely available in these contexts.

Teaching and Evaluating Nontechnical Skills for General Surgery

Surgical training programs have long used quantitative measures of knowledge, as well as subjective evaluation of technical skills, to define the competence of trainees. However, a growing body of literature has shown the importance of nontechnical surgical skills as vital components of quality surgical care. Institutions must train nontechnical surgical skills, including leadership, communication, teamwork, situational awareness, and decision making, and incorporate these attributes into their evaluative processes to maximally enhance surgical performance at every career stage.

Surgical Sabermetrics: Applying Athletics Data Science to Enhance Operative Performance

Mini-abstract: Surgical sabermetrics is advanced analytics of digitally recorded surgical training and operative procedures to enhance insight, support professional development, and optimize clinical and safety outcomes. This perspectives article illustrates how surgery can leverage data science approaches in athletics and industry to transform individual and team performance in the operating room.

Gathering Validity Evidence to Adapt the Non-technical Skills for Surgeons (NOTSS) Assessment Tool to the United States Context

BACKGROUND

Nontechnical skills are of increasing focus for safe and effective performance in the operating room. Assessment tools have been developed in Europe, Africa, and Asia but not adapted to the unique aspects of surgical delivery in the United States. Our objective was to use the Non-Technical Skills for Surgeons (NOTSS) assessment tool as a basis to establish consensus on essential nontechnical skills for surgical trainees and practicing surgeons in the U.S surgical context.

STUDY DESIGN

A mixed-methods research design was used in the form of a modified Delphi process to build consensus on essential NOTSS. A panel of surgical experts from hospitals across the U.S used this iterative process in 4 rounds to generate, rate, and classify behaviors. The primary outcome was consensus on behaviors as being essential for surgeons to achieve the best patient outcomes in the operating room, with a median rating of ≥6 on a 7-point scale for inclusion.

RESULTS

A total of 10 surgical experts participated. One hundred and thirty eight behaviors were generated in Round 1, and reduced to 100 behaviors in Rounds 2 and 3 based on application of inclusion criteria. The final skill list consisted of behaviors in Situation Awareness (n = 26), Decision Making (n = 18), Teamwork (n = 25), and Leadership (n = 31). No additional NOTSS categories or elements emerged from the analysis. In Round 4, all 100 behaviors were successfully grouped into 12 nontechnical skills elements. Labels and definitions were reworded to reflect the U.S. context, and an appropriate assessment scale was selected.

CONCLUSIONS

A panel of surgical experts from across the U.S. reached consensus on the essential NOTSS to achieve the best patient outcomes in the operating room. These behaviors form an empirical basis for the first context-specific nontechnical skills assessment and training tool for practicing surgeons in the U.S.

Superiority of living animal models in microsurgical training: beyond technical expertise

Background

Many studies are investigating the role of living and nonliving models to train microsurgeons. There is controversy around which modalities account for the best microsurgical training. In this study, we aim to provide a systematic literature review of the practical modalities in microsurgery training and compare the living and nonliving models, emphasizing the superiority of the former. We introduce the concept of non-technical skill acquisition in microsurgical training with the use of living laboratory animals in the context of a novel proposed curriculum.

Methods

A literature search was conducted on PubMed/Medline and Scopus within the past 11 years based on a combination of the following keywords: “microsurgery,” “training,” “skills,” and “models.” The online screening process was performed by two independent reviewers with the Covidence tool. A total of 101 papers was identified as relevant to our study. The protocol was reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.

Results

Living models offer the chance to develop both technical and non-technical competencies (i.e., leadership, situation awareness, decision-making, communication, and teamwork). Prior experience with ex vivo tissues helps residents consolidate basic skills prior to performing more advanced techniques in the living tissues. Trainees reported a higher satisfaction rate with the living models.

Conclusions

The combination of living and nonliving training microsurgical models leads to superior results; however, the gold standard remains the living model. The validity of the hypothesis that living models enhance non-technical skills remains to be confirmed.

Level of evidence: Not ratable.

Novel Assessments of Technical and Nontechnical Cardiac Surgery Quality: Protocol for a Mixed Methods Study

BACKGROUND

Of the 150,000 patients annually undergoing coronary artery bypass grafting, 35% develop complications that increase mortality 5 fold and expenditure by 50%. Differences in patient risk and operative approach explain only 2% of hospital variations in some complications. The intraoperative phase remains understudied as a source of variation, despite its complexity and amenability to improvement.

OBJECTIVE

The objectives of this study are to (1) investigate the relationship between peer assessments of intraoperative technical skills and nontechnical practices with risk-adjusted complication rates and (2) evaluate the feasibility of using computer-based metrics to automate the assessment of important intraoperative technical skills and nontechnical practices.

METHODS

This multicenter study will use video recording, established peer assessment tools, electronic health record data, registry data, and a high-dimensional computer vision approach to (1) investigate the relationship between peer assessments of surgeon technical skills and variability in risk-adjusted patient adverse events; (2) investigate the relationship between peer assessments of intraoperative team-based nontechnical practices and variability in risk-adjusted patient adverse events; and (3) use quantitative and qualitative methods to explore the feasibility of using objective, data-driven, computer-based assessments to automate the measurement of important intraoperative determinants of risk-adjusted patient adverse events.

RESULTS

The project has been funded by the National Heart, Lung and Blood Institute in 2019 (R01HL146619). Preliminary Institutional Review Board review has been completed at the University of Michigan by the Institutional Review Boards of the University of Michigan Medical School.

CONCLUSIONS

We anticipate that this project will substantially increase our ability to assess determinants of variation in complication rates by specifically studying a surgeon's technical skills and operating room team member nontechnical practices. These findings may provide effective targets for future trials or quality improvement initiatives to enhance the quality and safety of cardiac surgical patient care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/22536.

Teamwork and Surgical Team-Based Training

Effective teamwork, both in and out of the operating room, is an essential component of safe and efficient surgical performance. There are multiple available assessment tools for evaluating teamwork and important contributors to teamwork such as safety culture and nontechnical skills. Multiple types of interventions exist to improve and train providers on teamwork, and many have been demonstrated to improve not only teamwork but also patient outcomes. Teamwork strategies can be adapted to different contexts, based on provider needs and resources.

Continuing surgical education of non-technical skills

Background

The non-technical skills for surgeons (NOTSS) system was developed as a tool to assess surgical skills for patient safety during surgery. This study aimed to develop a NOTSS-based training system for surgical trainees to acquire non-technical skills using a chest surgery scenario in a wet lab.

Materials and methods

Trainees were categorized into three subgroups according to the years of experience as follows: Level A: 6 years or more; Level B: 3–5 years; and Level C: 1–2 years. Three stages of surgical procedure were designed: 1. chest wall resection and right upper lobe lobectomy, 2. right middle lobe sleeve lobectomy, and 3. right lower lobe lobectomy. One instructor was assigned to each operation table, who evaluated each participant's NOTSS scores consisting of 16 elements.

Results

When comparing average NOTSS score of all the three procedures, significant differences were observed between Level A, B, and C trainees. As an example of varying elements by procedure, Level A trainees demonstrated differences in Situation Awareness, and a significant difference was observed in Level C trainees regarding the elements of Decision Making. On the contrary, no significant difference was observed among Level B trainees. In the comparison between first-time and experienced participants, a significant improvement was observed in some elements in Level B and C trainees.

Conclusion

This study highlights the usefulness and feasibility of the NOTSS scoring system for surgeons with different experiences and the effectiveness of providing feedback to trainees during intraoperative handoffs in a wet lab.

•

Team training was effective for trainees of different levels.

•

Surgeons need continuing education of a non-technical skills.

Non-technical skills in paramedicine: A scoping review

This scoping review aims to identify which non-technical skills have been empirically identified in the literature, to create the first list of empirically identified non-technical skills for paramedics and allied health personnel. A five-stage scoping literature was undertaken in March 2020. The search retrieved a total of 4756 citations. A total of 93 studies met the inclusion criteria and were analyzed for data charting. A total of 26 non-technical skills were identified in the literature. The top five non-technical skills included decision-making (33%, n = 31), communication (24%, n = 23), empathy (17%, n = 16), leadership (12%, n = 12), and ethics (10%, n = 10). Furthermore, only five studies investigated the assessment or measurement of non-technical skills. This scoping review identified 26 non-technical skills that had been investigated in the paramedic literature to create the first list of empirically based desirable non-technical skills for a paramedic. Subsequently, research can then begin to focus on identifying the link that these have to paramedic practice and patient safety.

Non-technical skills and otolaryngology: systematic review

OBJECTIVES

This study aimed to assess the published literature on non-technical skills in otolaryngology surgery and examine the applicability of any research to others' practice, and to explore how the published literature can identify areas for further development and guide future research.

METHODS

A systematic review was conducted using the following key words: 'otolaryngology', 'otorhinolaryngology', 'ENT', 'ENT surgery', 'ear, nose and throat surgery', 'head and neck surgery', 'thyroid surgery', 'parathyroid surgery', 'otology', 'rhinology', 'laryngology' 'skull base surgery', 'airway surgery', 'non-technical skills', 'non technical skills for surgeons', 'NOTSS', 'behavioural markers' and 'behavioural assessment tool'.

RESULTS

Three publications were included in the review - 1 randomised, controlled trial and 2 cohort studies - involving 78 participants. All were simulation-based studies involving training otolaryngology surgeons.

CONCLUSION

Little research has been undertaken on non-technical skills in otolaryngology. Training surgeons' non-technical skill levels are similar across every tested aspect. The research already performed can guide further studies, particularly amongst non-training otolaryngology surgeons and in both emergency and elective non-simulated environments.

Advancing spinal fellowship training: an international multi-centre educational perspective

PURPOSE

The purpose of this article is to review the importance of contemporary spine surgery fellowships and educational strategies to assist with fellowship design and delivery.

METHODS

Spine surgery fellowship includes trainees from orthopaedic and neurosurgical backgrounds and is increasingly indicated for individuals wishing to pursue spine surgery as a career, recognizing how spinal surgery evolved significantly in scope and complexity. We combine expert opinion with a review of the literature and international experience to expound spine fellowship training.

RESULTS

Contemporary learning techniques include boot camps at the start of fellowship which may reinforce previous clinical learning and help prepare fellows for their new clinical roles. There is good evidence that surgical specialty training boot camps improve clinical skills, knowledge and trainee confidence prior to embarking upon new clinical roles with increasing levels of responsibility. Furthermore, as simulation techniques and technologies take on an increasing role in medical and surgical training, we found evidence that trainees' operative skills and knowledge can improve with simulated operations, even if just carried out briefly. Finally, we found evidence to suggest a role for establishing competence-based objectives for training in specific operative and technical procedures. Competence-based objectives are helpful for trainees and trainers to highlight gaps in a trainee's skill set that may then be addressed during training.

CONCLUSIONS

Spinal fellowships may benefit from certain contemporary strategies that assist design and delivery of training in a safe environment. Interpersonal factors that promote healthy teamwork may contribute to an environment conducive to learning. These slides can be retrieved under Electronic Supplementary Material.

Non-technical skills: a review of training and evaluation in urology

PURPOSE

With non-technical skills (NTS) deficits being recognised as a major cause for error in surgery, there is an increasing interest in their training and evaluation. A growing number of training courses are emerging and some NTS curricula have also been created. Many different training methods are described in the literature but there is still uncertainty with regards to their optimum combination within a curriculum.

METHODS

A literature review of the electronic database Medline was performed. All articles published before December 2018 were screened by abstract and included if deemed relevant by the author. The included articles' reference lists were also screened for further relevant studies.

RESULTS

Simulation training is accepted as the most effective way to train NTS. Within simulation training, it is shown that the 'igloo' full immersion/distributed simulation environment is appropriate for teaching NTS in urological scenarios where a designated operating room or space is not available. The use of multiple settings, for example wards and clinics as well as the operating room, is advantageous, as is training in an interprofessional team. Classroom teaching also plays a role in NTS training as an adjunct to simulation, with evidence that it improves some parameters of NTS. All levels, including qualified surgeons, benefit from NTS training; however, adaptation to both trainee level and specialty is important. Although less time consuming, training juniors and seniors together mainly benefits juniors, and training NTS at the same time as technical skills detracts from the quality of teaching. Debriefing is an important part of training and should be well structured; there are many debriefing models in existence, allowing for choice of method based on examiner preference and participant demographic. Furthermore, examiners should be well briefed in their task and trained in NTS assessment.

CONCLUSION

To move forward, studies should combine tried and tested learning techniques into a curriculum covering all training levels, which should then be validated and followed up long term to ensure a positive impact on patient safety.

Improving Safety through Non-Technical Skills in Chemical Plants: The Validity of a Questionnaire for the Self-Assessment of Workers

This research is aimed at developing a questionnaire for the self-assessment of non-technical skills (NTS) leading to safety in the chemical sector and at analysing the properties of its scales in terms of construct validity. The research involved 269 Italian employees from three chemical plants of an international company, who occupied low⁻medium levels in the organizational hierarchy. Results showed a good level of validity and reliability of the instrument and suggested that communication, situational awareness, decision-making, and fatigue/stress management are the four most important NTS for safety in the chemical sector.

Improving the working environment for the delivery of safe surgical care in the UK: a qualitative cross-sectional analysis

OBJECTIVES

The aim of this study was to identify current problems and potential solutions to improve the working environment for the delivery of safe surgical care in the UK.

DESIGN

Prospective, questionnaire-based cross-sectional study.

SETTING/PARTICIPANTS

Following validation, an electronic questionnaire was distributed to postgraduate local education and training board distribution lists, the Royal College of Surgeons of Edinburgh (RCSEd) mailing lists and trainee organisations. This consisted of a single open-ended question inviting five open-ended responses. Throughout the 13-week study period, the survey was also published on a number of social media platforms.

RESULTS

A total of 505 responders completed the survey, of which 35% were consultants, 30% foundation doctors, 17% specialty trainees, 11% specialty doctors, 5% core trainees and <1% surgical nurse practitioners. A total of 2238 free-text answers detailed specific actions to improve the working environment. These responses were individually coded and then grouped into nine categories (staff resources, non-staff resources, support, working conditions, communication and team work, systems improvement, patient centred, training and education, and miscellaneous).

CONCLUSIONS

The results of this study have identified a number of key areas that, if addressed, may improve the environment for the delivery of safer surgical care. Common themes that emerged across all grades included: increased front-line staff; a return to a 'firm' structure to improve team continuity; greater senior support; and improved hospital facilities to help staff rest and recuperate. While unlimited funding remains unrealistic, many of the suggestions could be implemented in a cost-neutral fashion and include insightful ideas for remodelling or restructuring the workforce to improve the efficiency of the surgical team. The findings of this study formed the basis of a set of recommendations published by the RCSEd as a discussion paper.