Reliability of the assessment of non-technical skills by using video-recorded trauma resuscitations

Identifying and addressing mentorship gaps in European trauma and emergency surgical training. Results from the Young European Society of Trauma and Emergency Surgery (yESTES) mentorship survey

PURPOSE

European training pathways for surgeons dedicated to treating severely injured and critically ill surgical patients lack a standardized approach and are significantly influenced by diverse organizational and cultural backgrounds. This variation extends into the realm of mentorship, a vital component for the holistic development of surgeons beyond mere technical proficiency. Currently, a comprehensive understanding of the mentorship landscape within the European trauma care (visceral or skeletal) and emergency general surgery (EGS) communities is lacking. This study aims to identify within the current mentorship environment prevalent practices, discern existing gaps, and propose structured interventions to enhance mentorship quality and accessibility led by the European Society for Trauma and Emergency Surgery (ESTES).

METHODS

Utilizing a structured survey conceived and promoted by the Young section of the European Society of Trauma and Emergency Surgery (yESTES), we collected and analyzed responses from 123 ESTES members (both surgeons in practice and in training) across 20 European countries. The survey focused on mentorship experiences, challenges faced by early-career and female surgeons, the integration of non-technical skills (NTS) in mentorship, and the perceived role of surgical societies in facilitating mentorship.

RESULTS

Findings highlighted a substantial mentorship experience gap, with 74% of respondents engaging in mostly informal mentorship, predominantly centered on surgical training. Notably, mentorship among early-career surgeons and trainees was less reported, uncovering a significant early-career gap. Female surgeons, representing a minority within respondents, reported a disproportionately poorer access to mentorship. Moreover, while respondents recognized the importance of NTS, these were inadequately addressed in current mentorship practices. The current mentorship input of surgical societies, like ESTES, is viewed as insufficient, with a call for structured programs and initiatives such as traveling fellowships and remote mentoring.

CONCLUSIONS

Our survey underscores critical gaps in the current mentorship landscape for trauma and EGS in Europe, particularly for early-career and female surgeons. A clear need exists for more formalized, inclusive mentorship programs that adequately cover both technical and non-technical skills. ESTES could play a pivotal role in addressing these gaps through structured interventions, fostering a more supportive, inclusive, and well-rounded surgical community.

Nontechnical Skills for Intraoperative Team Members

Nontechnical skills, defined as the set of cognitive and social skills used by individuals and teams to reduce error and improve performance in complex systems, have become increasingly recognized as a key contributor to patient safety. Efforts to characterize, quantify, and teach nontechnical skills in the context of perioperative care continue to evolve. This review article summarizes the essential behaviors for safety, described in taxonomies for nontechnical skills assessments developed for intraoperative clinical team members (eg, surgeons, anesthesiologists, scrub practitioners, perfusionists). Furthermore, the authors describe emerging methods to advance understanding of the impact of nontechnical skills on perioperative outcomes.

Video-based Coaching: Current Status and Role in Surgical Practice, Part II- Practical Implementation and Risk Mitigation from the Society for Surgery of the Alimentary Tract, Health Care Quality and Outcomes Committee

INTRODUCTION

Video-based surgical coaching is gaining traction within the surgical community. It has an increasing adoption rate and growing recognition of its utility, especially an advanced continuous professional growth tool, for continued educational purposes. This method offers instructional flexibility in real-time remote settings and asynchronous feedback scenarios. In our first paper, we delineated fundamental principles for video-based coaching, emphasizing the customization of feedback to suit individual surgeon's needs.

METHOD

In this second part of the series, we review into practical applications of video-based coaching, focusing on quality improvements in a team-based setting, such as the trauma bay. Additionally, we address the potential risks associated with surgical video recording, storage, and distribution, particularly regarding medicolegal aspects. We propose a comprehensive framework to facilitate the implementation of video coaching within individual healthcare institutions.

RESULTS

Our paper examines the legal and ethical framework and explores the potential benefits and challenges, offering insights into the real-world implications of this educational approach.

CONCLUSION

This paper contributes to the discourse on integrating video-based coaching into continuous professional development. It aims to facilitate informed decision-making in healthcare institutions, considering the adoption of this innovative educational quality tool.

Evaluation of video review tools for assessing non-technical skills in emergency department resuscitation teams: a systematic review

BACKGROUND AND IMPORTANCE

Use of video review in medicine is established in contexts such as surgery. Although not widely used in the emergency department (ED), some centres use it to evaluate non-technical skills (NTS) to support teaching and quality improvement.

OBJECTIVE

There is no consensus on assessment of NTS using video review in the ED and the purpose of this review was to identify tools used in this context.

DESIGN, SETTING AND PARTICIPANTS

Studies were identified using Embase, Medline, CINAHL and Google Scholar. Inclusion criterion for the review was NTS of resuscitation teams working within the ED were assessed using video review. A systematic search method was used, and results were synthesised after search criteria was checked by two independent reviewers. Authors settled on the same 9 studies eligible for inclusion.

OUTCOME MEASURES AND ANALYSIS

Reliability and validity of tools identified for use in this context. Due to the heterogeneity of studies, no meta-analysis occurred.

MAIN RESULTS

There are 9 studies included in the review. The review was registered with PROSPERO (Ref No: CRD42022306129). Four unique tools were identified - 6 studies used T-NOTECHS, 1 used TTCA-24, 1 used CALM and 1 used the Communication tool. T-NOTECHS is validated in the literature for use in this context.

CONCLUSION

T-NOTECHS is the tool of choice for assessing ED teams in this context.

Training Anesthesiology Residents to Care for the Traumatically Injured in the United States

Training and education for trauma anesthesiology have been predicated on 2 primary pathways: learning through peripheral "complex, massive transfusion cases"-an assumption that is flawed due to the unique demands, skills, and knowledge of trauma anesthesiology-or learning through experiential education, which is also incomplete due to its unpredictable and variable exposure. Residents may receive training from senior physicians who may not maintain a trauma-focused continuing medical education. Further compounding the issue is the lack of fellowship-trained clinicians and standardized curricula. The American Board of Anesthesiology (ABA) provides a section for trauma education in its Initial Certification in Anesthesiology Content Outline. However, many trauma-related topics also fall under other subspecialties, and the outline excludes "nontechnical" skills. This article focuses on the training of anesthesiology residents and proposes a tier-based approach to teaching the ABA outline by including lectures, simulation, problem-based learning discussions, and case-based discussions that are proctored in conducive environments by knowledgeable facilitators.

Variation of in-hospital trauma team staffing: new resuscitation, new team

Background

Non-technical errors, such as insufficient communication or leadership, are a major cause of medical failures during trauma resuscitation. Research on staffing variation among trauma teams on teamwork is still in their infancy. In this study, the extent of variation in trauma team staffing was assessed. Our hypothesis was that there would be a high variation in trauma team staffing.

Methods

Trauma team composition of consecutive resuscitations of injured patients were evaluated using videos. All trauma team members that where part of a trauma team during a trauma resuscitation were identified and classified during a one-week period. Other outcomes were number of unique team members, number of new team members following the previous resuscitation and new team members following the previous resuscitation in the same shift (Day, Evening, Night).

Results

All thirty-two analyzed resuscitations had a unique trauma team composition and 101 unique members were involved. A mean of 5.71 (SD 2.57) new members in teams of consecutive trauma resuscitations was found, which was two-third of the trauma team. Mean team members present during trauma resuscitation was 8.38 (SD 1.43). Most variation in staffing was among nurses (32 unique members), radiology technicians (22 unique members) and anesthetists (19 unique members). The least variation was among trauma surgeons (3 unique members) and ER physicians (3 unique members).

Conclusion

We found an extremely high variation in trauma team staffing during thirty-two consecutive resuscitations at our level one trauma center which is incorporated in an academic teaching hospital. Further research is required to explore and prevent potential negative effects of staffing variation in trauma teams on teamwork, processes and patient related outcomes.

Evaluation of validity and reliability of video analysis and live observations to assess trauma team performance

INTRODUCTION

A trauma resuscitation is dynamic and complex process in which failures could lead to serious adverse events. In several trauma centers, evaluation of trauma resuscitation is part of a hospital's quality assessment program. While video analysis is commonly used, some hospitals use live observations, mainly due to ethical and medicolegal concerns. The aim of this study was to compare the validity and reliability of video analysis and live observations to evaluate trauma resuscitations.

METHODS

In this prospective observational study, validity was assessed by comparing the observed adherence to 28 advanced trauma life support (ATLS) guideline related tasks by video analysis to life observations. Interobserver reliability was assessed by calculating the intra class coefficient of observed ATLS related tasks by live observations and video analysis.

RESULTS

Eleven simulated and thirteen real-life resuscitations were assessed. Overall, the percentage of observed ATLS related tasks performed during simulated resuscitations was 10.4% (P < 0.001) higher when the same resuscitations were analysed using video compared to live observations. During real-life resuscitations, 8.7% (p < 0.001) more ATLS related tasks were observed using video review compared to live observations. In absolute terms, a mean of 2.9 (during simulated resuscitations) respectively 2.5 (during actual resuscitations) ATLS-related tasks per resuscitation were not identified using live observers, that were observed through video analysis. The interobserver variability for observed ATLS related tasks was significantly higher using video analysis compared to live observations for both simulated (video analysis: ICC 0.97; 95% CI 0.97-0.98 vs. live observation: ICC 0.69; 95% CI 0.57-0.78) and real-life witnessed resuscitations (video analyse 0.99; 95% CI 0.99-1.00 vs live observers 0.86; 95% CI 0.83-0.89).

CONCLUSION

Video analysis of trauma resuscitations may be more valid and reliable compared to evaluation by live observers. These outcomes may guide the debate to justify video review instead of live observations.

Closed loop communication in the trauma bay: identifying opportunities for team performance improvement through a video review analysis

Objectives

Communication among trauma team members in the trauma bay is vulnerable to errors, which may impact patient outcomes. We used the previously validated trauma-non-technical skills (T-NOTECHS) tool to identify communication gaps during patient management in the trauma bay and to inform development strategies to improve team performance.

Methods

Two reviewers independently assessed non-technical skills of team members through video footage at Sunnybrook Health Sciences Centre. Team performance was measured using T-NOTECHS across five domains using a five-point Likert scale (lower score indicating worse performance): (1) leadership; (2) cooperation and resource management; (3) communication and interaction; (4) assessment and decision making; (5) situation awareness/coping with stress. Secondary outcomes assessed the number of callouts, closed loop communications and parallel conversations.

Results

The study included 55 trauma activations. Injury severity score (ISS) was used as a measure of trauma severity. A case with an ISS score ≥ 16 was considered severe. ISS was ≥ 16 in 37% of cases. Communication and interaction scored significantly lower compared to all other domains (p < 0.0001). There were significantly more callouts and completed closed loop communications in more severe cases compared to less severe cases (p = 0.017 for both). Incomplete closed loop communications and parallel conversations were identified, irrespective of case severity.

Conclusion

A lower communication score was identified using T-NOTECHS, attributed to incomplete closed loop communications and parallel conversations. Through video review of trauma team activations, opportunities for improvement in communication can be identified by the T-NOTECHS tool, as well as specifically identifying callouts and closed loop communication. This process may be useful for trauma programs as part of a quality improvement program on communication skills and team performance.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43678-022-00295-z.