Relationship between intraoperative non-technical performance and technical events in bariatric surgery

Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions

OBJECTIVE

Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI).

METHODS

A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases.

RESULTS

The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case.

CONCLUSION

An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.

How experienced robotic nurses adapt to the Hugo™ RAS system

No studies have reported on the impact at team level of the Medtronic Hugo™ RAS system. We described the work patterns and learning curves of an experienced robotic nurse team adapting to the new robotic system. We prospectively recorded the robotic nurse team's preoperative, perioperative, and postoperative tasks on the first 30 robotic procedures performed. The data were descriptively analyzed, and Gantt Charts were created for a timeline overview of the work patterns. We compared the operative times between the Medtronic Hugo™ RAS and the Davinci® system. The preoperative phase seemed to improve with a median time of 94 min (IQR 81-107). After 20 surgeries, the work pattern became more consistent where the scrub and circulating nurses worked simultaneously. There was no noticeable improvement for the perioperative and postoperative phases with a stable median time of 170 min (IQR 135-189) and 26 min (IQR 22-31). We found that the work pattern seemed to stabilize after 20 surgeries but with a continued decrease in preoperative time without a learning curve plateau. The robotic nurse team suffered from few breaks and long working hours because only a few nurses at our facility were trained in the Hugo™ system.

Operating room organization and surgical performance: a systematic review

BACKGROUND

Organizational factors may influence surgical outcomes, regardless of extensively studied factors such as patient preoperative risk and surgical complexity. This study was designed to explore how operating room organization determines surgical performance and to identify gaps in the literature that necessitate further investigation.

METHODS

We conducted a systematic review according to PRISMA guidelines to identify original studies in Pubmed and Scopus from January 1, 2000 to December 31, 2019. Studies evaluating the association between five determinants (team composition, stability, teamwork, work scheduling, disturbing elements) and three outcomes (operative time, patient safety, costs) were included. Methodology was assessed based on criteria such as multicentric investigation, accurate population description, and study design.

RESULTS

Out of 2625 studies, 76 met inclusion criteria. Of these, 34 (44.7%) investigated surgical team composition, 15 (19.7%) team stability, 11 (14.5%) teamwork, 9 (11.8%) scheduling, and 7 (9.2%) examined the occurrence of disturbing elements in the operating room. The participation of surgical residents appeared to impact patient outcomes. Employing specialized and stable teams in dedicated operating rooms showed improvements in outcomes. Optimization of teamwork reduced operative time, while poor teamwork increased morbidity and costs. Disturbances and communication failures in the operating room negatively affected operative time and surgical safety.

CONCLUSION

While limited, existing scientific evidence suggests that operating room staffing and environment significantly influences patient outcomes. Prioritizing further research on these organizational drivers is key to enhancing surgical performance.

Nontechnical Skills for Intraoperative Team Members

Nontechnical skills, defined as the set of cognitive and social skills used by individuals and teams to reduce error and improve performance in complex systems, have become increasingly recognized as a key contributor to patient safety. Efforts to characterize, quantify, and teach nontechnical skills in the context of perioperative care continue to evolve. This review article summarizes the essential behaviors for safety, described in taxonomies for nontechnical skills assessments developed for intraoperative clinical team members (eg, surgeons, anesthesiologists, scrub practitioners, perfusionists). Furthermore, the authors describe emerging methods to advance understanding of the impact of nontechnical skills on perioperative outcomes.

Which, how, and what? Using digital tools to train surgical skills; a systematic review and meta-analysis

Background

Digital tools like digital box trainers and VR seem promising in delivering safe and tailored practice opportunities outside of the surgical clinic, yet understanding their efficacy and limitations is essential. This study investigated Which digital tools are available to train surgical skills, How these tools are used, How effective they are, and What skills they are intended to teach.

Methods

Medline, Embase, and Cochrane libraries were systematically reviewed for randomized trials, evaluating digital skill-training tools based on objective outcomes (skills scores and completion time) in surgical residents. Digital tools effectiveness were compared against controls, wet/dry lab training, and other digital tools. Tool and training factors subgroups were analysed, and studies were assessed on their primary outcomes: technical and/or non-technical.

Results

The 33 included studies involved 927 residents and six digital tools; digital box trainers, (immersive) virtual reality (VR) trainers, robot surgery trainers, coaching and feedback, and serious games. Digital tools outperformed controls in skill scores (SMD 1.66 [1.06, 2.25], P < 0.00001, I2 = 83 %) and completion time (SMD -1.05 [-1.72, -0.38], P = 0.0001, I2 = 71 %). There were no significant differences between digital tools and lab training, between tools, or in other subgroups. Only two studies focussed on non-technical skills.

Conclusion

While the efficacy of digital tools in enhancing technical surgical skills is evident - especially for VR-trainers -, there is a lack of evidence regarding non-technical skills, and need to improve methodological robustness of research on new (digital) tools before they are implemented in curricula.

Key message

This study provides critical insight into the increasing presence of digital tools in surgical training, demonstrating their usefulness while identifying current challenges, especially regarding methodological robustness and inattention to non-technical skills.

Collaborative Team Training in Virtual Reality is Superior to Individual Learning For Performing Complex Open Surgery: A Randomized Controlled Trial

OBJECTIVE

To assess whether multiplayer immersive Virtual Reality (iVR) training was superior to single-player training for the acquisition of both technical and nontechnical skills in learning complex surgery.

BACKGROUND

Superior teamwork in the operating room (OR) is associated with improved technical performance and clinical outcomes. iVR can successfully train OR staff individually; however, iVR team training has yet to be investigated.

METHODS

Forty participants were randomized to individual or team iVR training. Individually trained participants practiced alongside virtual avatar counterparts, whereas teams trained live in pairs. Both groups underwent 5 iVR training sessions over 6 weeks. Subsequently, they completed a real-life assessment in which they performed anterior approach total hip arthroplasty surgery on a high-fidelity model with real equipment in a simulated OR. Teams performed together, and individually trained participants were randomly paired up. Videos were marked by 2 blinded assessors recording the 'Non-Operative Technical Skills for Surgeons, Oxford NOn-TECHnical Skills II and Scrub Practitioners' List of Intraoperative Non-Technical Skills' scores. Secondary outcomes were procedure duration and the number of technical errors.

RESULTS

Teams outperformed individually trained participants for nontechnical skills in the real-world assessment (Non-Operative Technical Skills for Surgeons: 13.1±1.5 vs 10.6±1.6, P = 0.002, Non-TECHnical Skills II score: 51.7 ± 5.5 vs 42.3 ± 5.6, P = 0.001 and Scrub Practitioners' List of Intraoperative Non-Technical Skills: 10 ± 1.2 vs 7.9 ± 1.6, P = 0.004). They completed the assessment 33% faster (28.2 minutes ± 5.5 vs 41.8 ± 8.9, P < 0.001), and made fewer than half the number of technical errors (10.4 ± 6.1 vs 22.6 ± 5.4, P < 0.001).

CONCLUSIONS

Multiplayer training leads to faster surgery with fewer technical errors and the development of superior nontechnical skills.

Kinematic data profile and clinical outcomes in robotic inguinal hernia repairs: a pilot study

BACKGROUND

Surgical training requires clinical knowledge and technical skills to operate safely and optimize clinical outcomes. Technical skills are hard to measure. The Intuitive Data Recorder (IDR), (Sunnyvale, CA) allows for the measurement of technical skills using objective performance indicators (OPIs) from kinematic event data. Our goal was to determine whether OPIs improve with surgeon experience and whether they are correlated with clinical outcomes for robotic inguinal hernia repair (RIHR).

METHODS

The IDR was used to record RIHRs from six surgeons. Data were obtained from 98 inguinal hernia repairs from February 2022 to February 2023. Patients were called on postoperative days 5-10 and asked to take the Carolina Comfort Scale (CCS) survey to evaluate acute clinical outcomes. A Pearson test was run to determine correlations between OPIs from the IDR with a surgeon's yearly RIHR experience and with CCS scores. Linear regression was then run for correlated OPIs.

RESULTS

Multiple OPIs were correlated with surgeon experience. Specifically, for the task of peritoneal flap exploration, we found that 23 OPIs were significantly correlated with surgeons' 1-year RIHR case number. Total angular motion distance of the left arm instrument had a correlation of - 0.238 (95% CI - 0.417, - 0.042) for RIHR yearly case number. Total angular motion distance of right arm instrument was also negatively correlated with RIHR in 1 year with a correlation of - 0.242 (95% CI - 0.420, - 0.046). For clinical outcomes, wrist articulation of the surgeon's console positively correlated with acute sensation scores from the CCS with a correlation of 0.453 (95% CI 0.013, 0.746).

CONCLUSIONS

This study defines multiple OPIs that correlate with surgeon experience and with outcomes. Using this knowledge, surgical simulation platforms can be designed to teach patterns to surgical trainees that are associated with increased surgical experience and with improved postoperative outcomes.

Exploring the competencies of operating room nurses in mobile surgical teams based on the Onion Model: a qualitative study

BACKGROUND

With the frequent occurrence of public health emergencies, conflicts and natural disasters around the world, mobile surgical teams are becoming more crucial. The competency of the operating room (OR) nurse has a substantial impact on the effectiveness and quality of the surgical team's treatment, still there is limited knowledge about OR nurse competencies in mobile surgical teams. This study aimed to explore the competencies of OR nurses in mobile surgical teams based on the Onion Model.

METHODS

We conducted a qualitative descriptive study of participants from 10 mobile surgical teams in 2022. Twenty-one surgical team members were interviewed, including 15 OR nurses, four surgeons, and two anesthesiologists. Data were collected through semi-structured interviews. The data were analyzed using Mayring's content analysis.

RESULTS

Twenty-eight competencies were found in the data analysis, which were grouped into four major domains using the Onion Model. From the outer layer to the inner layer were knowledge and skills, professional abilities, professional quality, and personal traits. The qualitative data revealed several novel competencies, including triage knowledge, self and mutual medical aid, outdoor survival skills, and sense of discipline.

CONCLUSIONS

The application of the Onion Model promotes the understanding of competency and strengthens the theoretical foundations of this study. New competencies can enrich the content of the competencies of OR nurses. The results of this study can be used for clinical recruitment, evaluation and training of OR nurses in mobile surgical teams. This study encourages further research to develop competency assessment tools and training programs for OR nurses.

"Stand-up straight!": human pose estimation to evaluate postural skills during orthopedic surgery simulations

PURPOSE

Surgery simulators can be used to learn technical and non-technical skills and, to analyse posture. Ergonomic skill can be automatically detected with a Human Pose Estimation algorithm to help improve the surgeon's work quality. The objective of this study was to analyse the postural behaviour of surgeons and identify expertise-dependent movements. Our hypothesis was that hesitation and the occurrence of surgical instruments interfering with movement (defined as interfering movements) decrease with expertise.

MATERIAL AND METHODS

Sixty surgeons with three expertise levels (novice, intermediate, and expert) were recruited. During a training session using an arthroscopic simulator, each participant's movements were video-recorded with an RGB camera. A modified OpenPose algorithm was used to detect the surgeon's joints. The detection frequency of each joint in a specific area was visualized with a heatmap-like approach and used to calculate a mobility score.

RESULTS

This analysis allowed quantifying surgical movements. Overall, the mean mobility score was 0.823, 0.816, and 0.820 for novice, intermediate and expert surgeons, respectively. The mobility score alone was not enough to identify postural behaviour differences. A visual analysis of each participants' movements highlighted expertise-dependent interfering movements.

CONCLUSION

Video-recording and analysis of surgeon's movements are a non-invasive approach to obtain quantitative and qualitative ergonomic information in order to provide feedback during training. Our findings suggest that the interfering movements do not decrease with expertise but differ in function of the surgeon's level.

Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study

BACKGROUND

The hybrid room (HR) is a complex, high-risk environment, requiring teams (surgeons, anesthesiologists, nurses, technologists) to master various skills, including the 'As Low As Reasonably Achievable' principle of radiation safety. This prospective single center cohort reports the first use of the Operating Room Black Box (ORBB) in a HR. This medical data recording system captures procedural and audio-visual data to facilitate structured team performance analysis.

METHODS

Patients planned for endovascular repair of an infrarenal abdominal aortic aneurysm (EVAR) or treatment of symptomatic iliac-femoral-popliteal atherosclerotic disease (Peripheral Vascular Interventions or PVI) were included. Validated measures and established assessment tools were used to assess (non-)technical skills, radiation safety performance and environmental distractions.

RESULTS

Six EVAR and sixteen PVI procedures were captured. Technical performance for one EVAR was rated 19/35 on the procedure-specific scale, below the 'acceptable' score of 21. Technical skills were rated above acceptable in all PVI procedures. Shared decision making and leadership were rated highly in 12/22 cases, whereas surgical communication and nurses' task management were rated low in 14/22 cases. Team members rarely stepped back from the C-arm during digital subtraction angiography. Radiation safety behavior was scored below 'acceptable' in 14/22 cases. A median (interquartile range) number of 12 (6-23) auditory distractions was observed per procedure.

CONCLUSIONS

The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. Strengths and weaknesses were identified in team members' (non-)technical and radiation safety practices. This technology has the potential to improve vascular surgical practice, though human input remains crucial. (NCT04854278).

Creating a Practical Transformational Change Management Model for Novel Artificial Intelligence-Enabled Technology Implementation in the Operating Room

Objective

To identify change management (CM) strategies for implementing novel artificial intelligence and similar novel technologies in operating rooms and create a new CM model for future trials and applications inspired by the abovementioned strategies and established models.

Methods

Key phases of technology implementation were defined, and strategies for transformational CM were created and applied in a recent CM experience at our institution between October 15, 2020 and October 15, 2021. We appraised existing CM models and propose the newly created model.

Results

The key phases of the technology implementation were as follows: (1) team assembly; (2) committee approvals; (3) CM; and (4) system installation and go-live. Key strategies were (1) assemble team with necessary expertise; (2) anticipate potential institutional cultural and regulatory hurdles; (3) add agility to project planning and execution; (4) accommodate institutional culture and regulations; (5) early clinical partner buy-in and stakeholder engagement; and (6) consistent communication, all of which contributed to the new CM model creation.

Conclusion

Key CM strategies and a new CM model addressing the unique needs and characteristics of operating room novel technology implementation were identified and created. The new model may be customized and tested for individual institution and project’s needs and characteristics.

Intraoperative Assessment of Non-Technical Skills for Surgeons (NOTSS) and Qualitative Description of their Effects on Intraoperative Performance

OBJECTIVE

The aims of this study were (1) to evaluate the feasibility of using the Non-Technical Skills for Surgeons (NOTSS) rating tool in assessing surgeons' non-technical skills behaviors in live operations, and (2) to describe the effect of NOTSS on intraoperative performance.

SETTING DESIGN

This study was conducted in an academic hospital in North America. Two observers independently conducted direct non-participant observations using the NOTSS rating tool to assess non-technical skills, and to document examples of effective or ineffective non-technical skills behaviors. Observers took field notes to document non-technical skill gaps that were not captured by the NOTSS rating tool, and situations or scenarios that presented challenges for accurate assessment. Interclass correlation estimates and 95% confidence intervals were calculated to assess the validity of the NOTSS rating tool. Deductive thematic qualitative data analysis was used for field notes and NOTSS behavior descriptions.

PARTICIPANTS

Participants were general surgeons performing either minimally invasive (robotic assisted or laparoscopic surgery), or open procedures.

RESULTS

We observed 18 surgeries, involving 6 surgeons, 11 residents and one fellow resulting in 37 hours of direct observations. The mean NOTSS score was 3.8 (SD 0.41) for situation awareness, 3.75 (SD 0.47) for decision-making, 3.71 (SD 0.39) for communication and teamwork, and 3.76 (SD 0.38) for leadership. The inter-rater reliability ranged between 0.65 and 0.80 for each NOTSS categories. The observers documented examples of effective non-technical skills behaviors and examples of behaviors that need improvement. Furthermore, we described the effect of each observed behavior on intraoperative performance. One challenge to NOTSS use in live surgery was that observers had to infer situation awareness, decision-making, and coping with pressure as these were not easily observed without attending surgeons articulating their underlying thought process.

CONCLUSION

The use of the NOTSS tool in live surgery is a valid and practical tool to document observed behaviors and their effect on intraoperative performance in order to provide constructive feedback to surgeons. One notable limitation is that without specific articulation by the surgeon of their underlying thought process the observer must infer specific elements. By documenting specific real-world events with high inter-rater reliability and adequate surgeon score variation the process can be used to provide useful feedback for improvement.

Analyzing and Discussing Human Factors Affecting Surgical Patient Safety Using Innovative Technology: Creating a Safer Operating Culture

INTRODUCTION

Surgical errors often occur because of human factor-related issues. A medical data recorder (MDR) may be used to analyze human factors in the operating room. The aims of this study were to assess intraoperative safety threats and resilience support events by using an MDR and to identify frequently discussed safety and quality improvement issues during structured postoperative multidisciplinary debriefings using the MDR outcome report.

METHODS

In a cross-sectional study, 35 standard laparoscopic procedures were performed and recorded using the MDR. Outcome data were analyzed using the automated Systems Engineering Initiative for Patient Safety model. The video-assisted MDR outcome report reflects on safety threat and resilience support events (categories: person, tasks, tools and technology, psychical and external environment, and organization). Surgeries were debriefed by the entire team using this report. Qualitative data analysis was used to evaluate the debriefings.

RESULTS

A mean (SD) of 52.5 (15.0) relevant events were identified per surgery. Both resilience support and safety threat events were most often related to the interaction between persons (272 of 360 versus 279 of 400). During the debriefings, communication failures (also category person) were the main topic of discussion.

CONCLUSIONS

Patient safety threats identified by the MDR and discussed by the operating room team were most frequently related to communication, teamwork, and situational awareness. To create an even safer operating culture, educational and quality improvement initiatives should aim at training the entire operating team, as it contributes to a shared mental model of relevant safety issues.

Perceptions of safety culture and recording in the operating room: understanding barriers to video data capture

OBJECTIVE

Recording in the operating room is an important tool to help surgical teams improve their performance. This is becoming more feasible using the Operating Room Black Box, a comprehensive data capture platform. Operating room (OR) staff, however, may voice reasonable concerns as recording initiatives are implemented. The objective of this study was to assess pre-implementation attitudes of OR staff toward operative recording and explore the relationship of these attitudes to the themes of (1) safety culture, (2) impostor syndrome, and (3) privacy concerns.

METHODS

This cross-sectional survey study measured staff members' beliefs and opinions of operative recording and used three previously validated tools (safety attitudes questionnaire, clance impostor phenomenon scale, and dispositional privacy concern) to assess personal and professional factors. Concepts were correlated using Pearson's correlation coefficient.

RESULTS

Forty-three staff members participated in this study, with a response rate of 45% (n = 43/96, 20/22 nurses, 9/11 gynecologists, 14/63 anesthesiologists). Opinions of operative data capture were generally positive (5-point Likert scale, mean = 3.81, SD = 0.91). Nurses tended to have more favorable opinions of the OR Black Box as compared to gynecologists and anesthesiologists, though this did not reach statistical significance (4.15 vs. 3.67 vs 3.43, p = 0.06). Impostor syndrome characteristics correlated with concerns about litigation related to recording (r =  - 0.32, p = 0.04).

CONCLUSION

There are personal and professional attributes of the OR team that impact perceptions of the OR Black Box and implications around privacy and litigation. Addressing these concerns may facilitate successful implementation of the OR Black Box and improve team communication and patient safety in the OR.

Identification of Main Influencers of Surgical Efficiency and Variability Using Task-Level Objective Metrics: A Five-Year Robotic Sleeve Gastrectomy Case Series

Objective

Surgical efficiency and variability are critical contributors to optimal outcomes, patient experience, care team experience, and total cost to treat per disease episode. Opportunities remain to develop scalable, objective methods to quantify surgical behaviors that maximize efficiency and reduce variability. Such objective measures can then be used to provide surgeons with timely and user-specific feedbacks to monitor performances and facilitate training and learning. In this study, we used objective task-level analysis to identify dominant contributors toward surgical efficiency and variability across the procedural steps of robotic-assisted sleeve gastrectomy (RSG) over a five-year period for a single surgeon. These results enable actionable insights that can both complement those from population level analyses and be tailored to an individual surgeon's practice and experience.

Methods

Intraoperative video recordings of 77 RSG procedures performed by a single surgeon from 2015 to 2019 were reviewed and segmented into surgical tasks. Surgeon-initiated events when controlling the robotic-assisted surgical system were used to compute objective metrics. A series of multi-staged regression analysis were used to determine: if any specific tasks or patient body mass index (BMI) statistically impacted procedure duration; which objective metrics impacted critical task efficiency; and which task(s) statistically contributed to procedure variability.

Results

Stomach dissection was found to be the most significant contributor to procedure duration (β = 0.344, p< 0.001; R = 0.81, p< 0.001) followed by surgical inactivity and stomach stapling. Patient BMI was not found to be statistically significantly correlated with procedure duration (R = −0.01, p = 0.90). Energy activation rate, a robotic system event-based metric, was identified as a dominant feature in predicting stomach dissection duration and differentiating earlier and later case groups. Reduction of procedure variability was observed between earlier (2015-2016) and later (2017-2019) groups (IQR = 14.20 min vs. 6.79 min). Stomach dissection was found to contribute most to procedure variability (β = 0.74, p < 0.001).

Conclusions

A surgical task-based objective analysis was used to identify major contributors to surgical efficiency and variability. We believe this data-driven method will enable clinical teams to quantify surgeon-specific performance and identify actionable opportunities focused on the dominant surgical tasks impacting overall procedure efficiency and consistency.

Executive summary of the artificial intelligence in surgery series

As opportunities for artificial intelligence to augment surgical care expand, the accompanying surge in published literature has generated both substantial enthusiasm and grave concern regarding the safety and efficacy of artificial intelligence in surgery. For surgeons and surgical data scientists, it is increasingly important to understand the state-of-the-art, recognize knowledge and technology gaps, and critically evaluate the deluge of literature accordingly. This article summarizes the experiences and perspectives of a global, multi-disciplinary group of experts who have faced development and implementation challenges, overcome them, and produced incipient evidence thereof. Collectively, evidence suggests that artificial intelligence has the potential to augment surgeons via decision-support, technical skill assessment, and the semi-autonomous performance of tasks ranging from resource allocation to patching foregut defects. Most applications remain in preclinical phases. As technologies and their implementations improve and positive evidence accumulates, surgeons will face professional imperatives to lead the safe, effective clinical implementation of artificial intelligence in surgery. Substantial challenges remain; recent progress in using artificial intelligence to achieve performance advantages in surgery suggests that remaining challenges can and will be overcome.

Effects of shared mental models in teams performing video-assisted thoracoscopic surgery lobectomy

BACKGROUND

Important non-technical skills enable operating teams to establish shared mental models (SMMs). The importance of SMMs in regards to surgical performance and peri-operative outcomes remains to be investigated. The aim of this study was to explore whether shared mental models (SMMs) of team resources and the current situation, respectively, were predictive of technical skills, duration of surgery, and amount of intra-operative bleeding in video-assisted thoracoscopic surgery (VATS).

METHODS

A prospective multi-center observational study was conducted at four tertiary academic hospitals during VATS lobectomy procedures. Data included pre-operative and post-operative questionnaires answered by each of the six team members to measure the SMMs; thoracoscopic video recordings assessed using the previously validated VATS lobectomy Assessment Tool (VATSAT); surgery-related time stamps; and amount (volume) of intra-operative bleeding. Linear regression analyses were conducted to adjust for confounders.

RESULTS

Fifty-eight lobectomy procedures were included. Median (interquartile range) VATSAT score was 33.3 (scale 8-40) duration of surgery 101 min (88-123), and amount of intra-operative bleeding 100 ml (20-150). The mean (± SD) of teams' SMMs of the current situation was 20 (± 5). They were not predictive of the surgeons' technical skills, but every one point increase in SMM score significantly predicted a 1 min 52 s decrease in duration of surgery and an 11% decrease in amount of bleeding. The SMMs of team resources were not predictive of any outcomes.

CONCLUSION

VATS teams' superior SMMs of the current situation related to significantly shorter duration of surgery and decreased intra-operative bleeding, indicating an effect on team performance and patient care.

TRIAL REGISTRATION

NCT02999113 at http://www.clinicaltrials.gov .

Step, Error, and Event Frameworks in Endovascular Aortic Repair

OBJECTIVE

Competency-based surgical education requires detailed and actionable feedback to ensure adequate and efficient skill development. Comprehensive operative capture systems such as the Operating Room Black Box (ORBB; Surgical Safety Technologies, Inc), which continuously records and synchronizes multiple sources of intraoperative data, have recently been integrated into hybrid rooms to provide targeted feedback to endovascular teams. The objective of this study is to develop step, error, and event frameworks to evaluate technical performance in elective endovascular aortic repair (EVAR) comprehensively captured by the ORBB (Surgical Safety Technologies, Inc; Toronto, Canada).

METHODS

This study is based upon a modified Delphi consensus process to create evaluation frameworks for steps, errors, and events in EVAR. International experts from Vascular Surgery and Interventional Radiology were identified, based on their records of publications and invited presentations, or serving on relevant journal editorial boards. In an initial open-ended survey round, experts were asked to volunteer a comprehensive list of steps, errors, and events for a standard EVAR of an infrarenal aorto-iliac aneurysm (AAA). In subsequent survey rounds, the identified items were presented to the expert panel to rate on a 5-point Likert scale. Delphi survey rounds were repeated until the process reached consensus with a predefined agreement threshold (Cronbach α>0.7). The final frameworks were constructed with items achieving an agreement (responses of 4 or 5) from greater than 70% of experts.

RESULTS

Of 98 invited proceduralists, 38 formed the expert consensus panel (39%), consisting of 29 vascular surgeons and 9 interventional radiologists, with 34% from North America and 66% from Europe. Consensus criteria were met following the third round of the Delphi consensus process (Cronbach α=0.82-0.93). There were 15, 32, and 25 items in the error, step, and event frameworks, respectively (within-item agreement=74%-100%).

CONCLUSION

A detailed evaluation tool for the procedural steps, errors, and events in infrarenal EVAR was developed. This tool will be validated on recorded procedures in future work: It may focus skill development on common errors and hazardous steps. This tool might be used to provide high-quality feedback on technical performance of trainees and experienced surgeons alike, thus promoting surgical mastery.

RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

Background

Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams’ NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS.

Methodology

Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool—RAS-NOTECHS—was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations).

Results

Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement.

Conclusions

RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-021-08474-2.

Nontechnical Skills (NTS) in the Undergraduate Surgical and Anesthesiology Curricula: Are We Adequately Preparing Medical Students?

OBJECTIVES

Nontechnical skills (NTS) encompass interpersonal, cognitive, and personal resource skills that can mitigate surgical errors and improve patient outcomes. However, inconsistencies in medical student awareness around NTS suggest limited exposure to these skills. This study aimed to determine the prevalence and content of NTS in medical school surgery and anesthesiology education.

DESIGN AND SETTING

Learning objectives from clerkship core surgery and anesthesiology rotations were collected from Canadian anglophone medical schools. Two raters independently classified each objective under one of the Non-Technical Skills for Surgeons (NOTSS) or Anaesthetists' Non-Technical Skills (ANTS) "Categories" and "Elements" of NTS, or as a non-NTS objective. Rater disagreements were resolved by group consensus. Group discussion was also held to identify examples of objectives that could help develop future curricula. Descriptive statistics were used to determine the number of NTS objectives from each school and within each NOTSS and ANTS Categories and Elements.

RESULTS

Learning objectives were obtained from 12 out of 14 Canadian medical schools. A total of 2116 surgery objectives and 571 anesthesiology objectives were reviewed. Of these, 16 (0.76%) and 26 (4.55%) were identified as NTS objectives in surgery and anesthesiology, respectively. Of the NOTSS and ANTS Categories, "Situation Awareness" and "Decision Making" were represented by only one objective each in both specialties. Approximately half of the NOTSS and ANTS Elements were not represented by a single objective. Group discussion yielded examples of NTS objectives that were excellent, could use improvement, or were too vague to be classified as NTS.

CONCLUSIONS

A paucity of objectives in the clerkship perioperative curricula involve NTS. These findings suggest that NTS are unlikely being adequately introduced as critical skillsets of surgeons and anesthesiologists in undergraduate perioperative education. Future curriculum development should involve greater medical student exposure to NTS as key components of their surgery and anesthesiology education.

Computer Vision in the Operating Room: Opportunities and Caveats

Effectiveness of computer vision techniques has been demonstrated through a number of applications, both within and outside healthcare. The operating room environment specifically is a setting with rich data sources compatible with computational approaches and high potential for direct patient benefit. The aim of this review is to summarize major topics in computer vision for surgical domains. The major capabilities of computer vision are described as an aid to surgical teams to improve performance and contribute to enhanced patient safety. Literature was identified through leading experts in the fields of surgery, computational analysis and modeling in medicine, and computer vision in healthcare. The literature supports the application of computer vision principles to surgery. Potential applications within surgery include operating room vigilance, endoscopic vigilance, and individual and team-wide behavioral analysis. To advance the field, we recommend collecting and publishing carefully annotated datasets. Doing so will enable the surgery community to collectively define well-specified common objectives for automated systems, spur academic research, mobilize industry, and provide benchmarks with which we can track progress. Leveraging computer vision approaches through interdisciplinary collaboration and advanced approaches to data acquisition, modeling, interpretation, and integration promises a powerful impact on patient safety, public health, and financial costs.

Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty

INTRODUCTION

Immersive Virtual Reality (iVR) is a novel technology which can enhance surgical training in a virtual environment without supervision. However, it is untested for the training to select, assemble and deliver instrumentation in orthopaedic surgery-typically performed by scrub nurses. This study investigates the impact of an iVR curriculum on this facet of the technically demanding revision total knee arthroplasty.

MATERIALS AND METHODS

Ten scrub nurses completed training in four iVR sessions over a 4-week period. Initially, nurses completed a baseline real-world assessment, performing their role with real equipment in a simulated operation assessment. Each subsequent iVR session involved a guided mode, where the software taught participants the procedural choreography and assembly of instrumentation in a simulated operating room. In the latter three sessions, nurses also undertook an assessment in iVR. Outcome measures were related to procedural sequence, duration of surgery and efficiency of movement. Transfer of skills from iVR to the real world was assessed in a post-training simulated operation assessment. A pre- and post-training questionnaire assessed the participants knowledge, confidence and anxiety.

RESULTS

Operative time reduced by an average of 47% across the 3 unguided sessions (mean 55.5 ± 17.6 min to 29.3 ± 12.1 min, p > 0.001). Assistive prompts reduced by 75% (34.1 ± 16.8 to 8.6 ± 8.8, p < 0.001), dominant hand motion by 28% (881.3 ± 178.5 m to 643.3 ± 119.8 m, p < 0.001) and head motion by 36% (459.9 ± 99.7 m to 292.6 ± 85.3 m, p < 0.001). Real-world skill improved from 11% prior to iVR training to 84% correct post-training. Participants reported increased confidence and reduced anxiety in scrubbing for rTKA procedures (p < 0.001).

CONCLUSIONS

For scrub nurses, unfamiliarity with complex surgical procedures or equipment is common. Immersive VR training improved their understanding, technical skills and efficiency. These iVR-learnt skills transferred into the real world.

How to Bring Surgery to the Next Level: Interpretable Skills Assessment in Robotic-Assisted Surgery

Introduction

A surgeon's technical skills are an important factor in delivering optimal patient care. Most existing methods to estimate technical skills remain subjective and resource intensive. Robotic-assisted surgery (RAS) provides a unique opportunity to develop objective metrics using key elements of intraoperative surgeon behavior which can be captured unobtrusively, such as instrument positions and button presses. Recent studies have shown that objective metrics based on these data (referred to as objective performance indicators [OPIs]) correlate to select clinical outcomes during robotic-assisted radical prostatectomy. However, the current OPIs remain difficult to interpret directly and, therefore, to use within structured feedback to improve surgical efficiencies.

Methods

We analyzed kinematic and event data from da Vinci surgical systems (Intuitive Surgical, Inc., Sunnyvale, CA, USA) to calculate values that can summarize the use of robotic instruments, referred to as OPIs. These indicators were mapped to broader technical skill categories of established training protocols. A data-driven approach was then applied to further sub-select OPIs that distinguish skill for each technical skill category within each training task. This subset of OPIs was used to build a set of logistic regression classifiers that predict the probability of expertise in that skill to identify targeted improvement and practice. The final, proposed feedback using OPIs was based on the coefficients of the logistic regression model to highlight specific actions that can be taken to improve.

Results

We determine that for the majority of skills, only a small subset of OPIs (2-10) are required to achieve the highest model accuracies (80-95%) for estimating technical skills within clinical-like tasks on a porcine model. The majority of the skill models have similar accuracy as models predicting overall expertise for a task (80-98%). Skill models can divide a prediction into interpretable categories for simpler, targeted feedback.

Conclusion

We define and validate a methodology to create interpretable metrics for key technical skills during clinical-like tasks when performing RAS. Using this framework for evaluating technical skills, we believe that surgical trainees can better understand both what can be improved and how to improve.

Artificial intelligence in cardiothoracic surgery

The tremendous and rapid technological advances that humans have achieved in the last decade have definitely impacted how surgical tasks are performed in the operating room (OR). As a high-tech work environment, the contemporary OR has incorporated novel computational systems into the clinical workflow, aiming to optimize processes and support the surgical team. Artificial intelligence (AI) is increasingly important for surgical decision making to help address diverse sources of information, such as patient risk factors, anatomy, disease natural history, patient values and cost, and assist surgeons and patients to make better predictions regarding the consequences of surgical decisions. In this review, we discuss the current initiatives that are using AI in cardiothoracic surgery and surgical care in general. We also address the future of AI and how high-tech ORs will leverage human-machine teaming to optimize performance and enhance patient safety.

Non-technical skills and device-related interruptions in minimally invasive surgery

BACKGROUND

Device-related interruptions in the operating room (OR) may create stress among health care providers and delays. Although non-technical skills (NTS) of the OR teams, such as situational awareness and communication, are expected to influence device-related interruptions, empirical data on this relationship are limited.

METHODS

We performed a prospective cohort study of 144 consecutive elective laparoscopic operations during 13 months. A data capture system called the OR Black Box® was used to characterize device-related interruptions, NTS, and distractions. Device-related interruptions were classified according to a priori established categories. Positive and negative NTS instances were identified according to validated measurement tools specific for nurses and surgeons. We assessed the relationship between NTS and device-related interruptions after adjusting for potential confounders.

RESULTS

A total of 86 device-related interruptions occurred in 48 of 144 operations (33%). They were most frequently classified as device failure (54%) followed by improper assembly (19%) and disconnection (14%). Medians of 1 [interquartile range (IQR) 0-3] and 1 (IQR 0-2) negative NTS instance per operation were demonstrated by nurses and surgeons, respectively. Medians of 28 (IQR 15-38) and 40 (IQR 28-118) positive NTS instances per operation were demonstrated by nurses and surgeons. In a multivariable analysis, a higher frequency of negative NTS instances demonstrated by nurses was associated with device-related interruptions after risk adjustment (Odds Ratio 1.33, p = 0.02).

CONCLUSIONS

In elective laparoscopic operations, an increased likelihood of device-related interruptions in the OR was associated with more frequent negative NTS demonstrations by nursing teams.

Do 30-Day Reoperation Rates Adequately Measure Quality in Orthopedic Surgery?

BACKGROUND

Unplanned reoperation rates represent an important metric in monitoring quality in orthopedic surgery. Previous studies have focused on 30-day reoperation rates, not accounting for complications that may arise beyond this time. This study aimed to understand the frequency, timing, and procedure type of orthopedic reoperations, as well as the complications leading up to these reoperations over a 1-year period.

METHODS

A single-center, retrospective cohort study reviewed all orthopedic surgeries performed within a three-year period and subsequently identified reoperations within a year following the initial case. Exclusion criteria for reoperations included those that were planned, involved a different body part, or had a different laterality from the first operation. The cases were analyzed by procedure type, timing of reoperation, and causes of reoperation.

RESULTS

Of the 10,449 orthopedic surgeries performed between 2012 and 2015, 947 (9.1%) were unplanned reoperations within 1 year. Most (775; 81.8%) unplanned reoperations occurred after 30 days. Infections/wound complications (58.2%) were the most common reason for unplanned reoperations at 1 month from the initial operation, and mechanical complications (49.5%) predominated at the 6-months-to-1-year time frame.

CONCLUSION

This study demonstrated that the current paradigm of focusing on reoperations occurring within 30 days of the initial operation captures only a fraction of unplanned reoperations. Stratification of this metric by time and precipitating complication type provides additional information that quality improvement programs may target. A 1-year unplanned reoperation rate could be used as a broad indicator of surgical quality across institutions.

An Interdisciplinary Review of Surgical Data Recording Technology Features and Legal Considerations

Surgical data recording technology has great promise to generate patient safety and quality data that can be utilized to potentially reduce medical errors. Variations of these systems aim to improve surgical technique, develop better training simulation, and promote adverse event investigation similar to the aims of black box technology utilized in other industries. However, many unknowns remain for surgical data recording utilization in operating rooms and clinical settings in the United States. This includes the need to appropriately design systems so they collect meaningful and useful data that can be discussed by surgical team members in an open and safe environment to optimize clinical care processes. In order to better understand the clinical and regulatory environment for surgical data recording systems, we conducted an interdisciplinary review to identify key technology approaches, and assess legal and regulatory implications associated with this potentially disruptive technology. We found technology ranging from audio and visual data, to systems utilizing mobile applications, and kinematic data capture. The data collected present legal questions over ownership of information and privacy, along with regulatory issues at the federal and state levels. The benefits of these data should be balanced with the need to develop appropriate policies and regulations that protect the interests of both clinicians and patients in order to encourage further innovation and better realize the potential of surgical data recording technology to improve clinical decision making and patient safety outcomes.

Surgical Fires and Burns: A 5-Year Analysis of Medico-legal Cases

Surgical fires and unintended intraoperative burns cause serious patient harm, yet surveillance data are lacking in Canada. Medico-legal data provide unique descriptions of these events which can inform burn prevention strategies. We extracted 5 years of data on closed (2012-2016) medico-legal cases involving surgical fires and burns from the database of our organization which, in 2016, provided medico-legal support to >93,000 Canadian physicians. We performed a retrospective descriptive analysis of contributing factors using an in-house coding system and case reviews. We identified 53 eligible burn cases: 26 from thermal sources (49.1%), 16 from fires (30.2%), 5 from chemical sources (9.4%), and 6 from undetermined sources (11.3%). Common burn sources were electrosurgical equipment, lasers, lighting, and improper temperatures (causing thermal burns), cautery or lasers combined with supplemental oxygen and/or a flammable fuel source (causing fire), and improperly applied solutions including antiseptics (causing chemical burns). Nontechnical factors also contributed to patient outcomes, such as nonadherence to protocols (15 cases, 28.3%), failures in surgical team communication (3 cases, 5.7%), and lost situational awareness leading to delays in recognizing and treating burns (7 cases, 13.2%). This retrospective study highlights a need for improved surgical safety interventions to address surgical fires and burns. These interventions could include: effectively implemented surgical safety protocols, surgical team communication strategies, and raising awareness about preventing, diagnosing, and managing surgical burns.

Evidence that surgical performance predicts clinical outcomes

PURPOSE

Assessment of surgeon performance in the operating room has been identified as a direct method of measuring surgical quality. Studies published in urology and other surgical disciplines have investigated this link directly by measuring surgeon and team performance using methodology supported by validity evidence. This article highlights the key findings of these studies and associated underlying concepts.

METHODS

​Seminal literature from urology and related areas of research was used to inform this review of the performance-outcome relationship in surgery. Current efforts to further our understanding of this concept are discussed, including relevant quality improvement and educational interventions that utilize this relationship.

RESULTS

Evidence from multiple surgical specialties and procedures has established the association between surgeon skill and clinically significant patient outcomes. Novel methods of measuring performance utilize surgeon kinematics and artificial intelligence techniques to more reliably and objectively quantify surgical performance.

CONCLUSIONS

Future directions include the use of this data to create interventions for quality improvement, as well as innovate the credentialing and recertification process for practicing surgeons.