Actual vs perceived performance debriefing in surgery: practice far from perfect

Artificial Intelligence-Based Video Feedback to Improve Novice Performance on Robotic Suturing Skills: A Pilot Study

Introduction: Automated skills assessment can provide surgical trainees with objective, personalized feedback during training. Here, we measure the efficacy of artificial intelligence (AI)-based feedback on a robotic suturing task. Materials and Methods: Forty-two participants with no robotic surgical experience were randomized to a control or feedback group and video-recorded while completing two rounds (R1 and R2) of suturing tasks on a da Vinci surgical robot. Participants were assessed on needle handling and needle driving, and feedback was provided via a visual interface after R1. For feedback group, participants were informed of their AI-based skill assessment and presented with specific video clips from R1. For control group, participants were presented with randomly selected video clips from R1 as a placebo. Participants from each group were further labeled as underperformers or innate-performers based on a median split of their technical skill scores from R1. Results: Demographic features were similar between the control (n = 20) and feedback group (n = 22) (p > 0.05). Observing the improvement from R1 to R2, the feedback group had a significantly larger improvement in needle handling score (0.30 vs -0.02, p = 0.018) when compared with the control group, although the improvement of needle driving score was not significant when compared with the control group (0.17 vs -0.40, p = 0.074). All innate-performers exhibited similar improvements across rounds, regardless of feedback (p > 0.05). In contrast, underperformers in the feedback group improved more than the control group in needle handling (p = 0.02). Conclusion: AI-based feedback facilitates surgical trainees' acquisition of robotic technical skills, especially underperformers. Future research will extend AI-based feedback to additional suturing skills, surgical tasks, and experience groups.

Intraoperative feedback: A survey of surgical trainees' perspective

INTRODUCTION

Intraoperative feedback can be associated with improved surgical performance. Quality feedback can reduce the time required by trainees to achieve proficiency in psychomotor skills. Operative training time has become increasingly limited, and it has become imperative to use surgical training time effectively.

AIM

In this survey, we assessed trainees' perspectives of intraoperative feedback. We included several aspects of feedback including its occurrence, quality, and potential barriers.

METHODS

All surgical trainees in a single centre were invited to complete an electronic questionnaire. Participants were anonymised. We summarised data using descriptive statistics.

RESULTS

Most trainees (85%) reported they had the opportunity to share their training goals with trainers. Just under three-quarters of trainees felt they always or sometimes got timely feedback. Only half of the trainees were signposted to feedback and 23% felt feedback was not part of their department's culture. Half of the trainees did not always feel comfortable asking for feedback from their trainers stating their reasons as fear of criticism, lack of time and competing clinical commitments.

CONCLUSION

There is no denying the importance of feedback on operative performance, however, this survey shows that many of the pillars of quality feedback are poorly adhered to.

The use of an educational time-out in thyroid and parathyroid surgery to move the needle in periprocedural education

BACKGROUND

As surgical training shifts toward a competency-based paradigm, deliberate practice for procedures must be a point of focus. The purpose of this study was to assess the impact of an educational time-out intervention on educational experience and operative performance in endocrine surgery.

METHODS

For 12 months, third-year general surgery residents used the educational time-out to establish an operative step of focus for thyroidectomy and parathyroidectomy procedures. Data were collected using the System for Improving and Measuring Procedural Learning application and post-rotation surveys. The Zwisch scale was used to classify supervision, with meaningful autonomy defined as passive help or supervision only.

RESULTS

Eight residents and 3 attending surgeons performed the educational time-out for a total of 211 operations (93% completion rate). At the end of each rotation, there was improvement in the frequency of goal setting. There was strong agreement (90%) that the intervention strengthened the educational experience. For most cases (52%), the residents were rated at active help. Residents performed a median of 3/6 thyroidectomy steps at meaningful autonomy and a median of 2/5 parathyroidectomy steps at meaningful autonomy. Review of the qualitative data revealed that optimal feedback was provided in 46% of cases.

CONCLUSION

The educational time-out strengthened educational experiences. Stepwise procedural data revealed the varying levels of supervision that exist within an operation. Broader implementation of this intervention could facilitate competency-based procedural education.

The Effect of Anonymity on Quality of Resident Feedback

OBJECTIVE

Quality feedback is critical to facilitate better performance and quicker learning. However, faculty may be hesitant to provide quality constructive feedback, especially in written form, due to fear of retaliation. We evaluated the impact of faculty anonymity on the quality of faculty-to-resident feedback.

DESIGN

A retrospective review was undertaken of faculty evaluation of resident performance from 2017 to 2018, when evaluations were identifiable, compared to 2018-2019, when evaluations were anonymous. Evaluations included 27 individual items with Likert type scoring and 2 open-ended questions. Open-ended responses and overall performance were de-identified and scored by 2 reviewers independently using the task, performance gap, action scoring model. Comparisons between groups were performed with the Wilcoxon-Mann-Whitney test.

SETTING

Tertiary Care Institution, University of Michigan, Ann Arbor, MI PARTICIPANTS: 415 resident performance evaluations were available for analysis with 251 in the identifiable group and 164 in the anonymous group.

RESULTS

The average composite score for the identifiable group was 105.2 and 103.4 in the anonymous group (p = 0.22). The effect size of the impact on composite score was small (Cohen's d 0.084, 95% CI -0.11-0.28). There was excellent inter-rater reliability. There were no differences between feedback groups for any of the 3 components of task, performance gap, and action model. While average scores for all 3 components were low, action scores were lowest.

CONCLUSIONS

Anonymity did not significantly impact faculty evaluations of resident performance. The quality of open-ended feedback on written evaluations was generally poor, especially in identifying actions for continued performance improvement. Additional mechanisms to improve feedback quality should be sought.

Growth, Engagement, and Belonging in the Clinical Learning Environment: the Role of Psychological Safety and the Work Ahead

Psychological safety is the perception that an environment is safe for interpersonal risk taking, exposing vulnerability, and contributing perspectives without fear of being shamed, blamed, or ignored. The presence of psychological safety has been associated with improved team learning and innovation, leader inclusivity, and team members' sense of belonging. In medical education, psychological safety has additional benefits: it allows learners to be present in the moment and to focus on the tasks at hand, and reduces trainee focus on image. Several key features of psychologically safe environments have already been described, including the presence of high-quality relationships, the absence of social positioning, a learner-driven and flexible learning agenda, the lack of formal assessment, and time for debriefing. However, many of the structures and cultural traditions in medical education are in clear opposition to these features. This paper describes the current barriers to psychological safety in medical education, and sets out an agenda for change. In accordance with benefits seen in other sectors, we anticipate that an emphasis on relationships and psychological safety will support the learning, inclusion, and success of medical trainees.

A Retrospective Review of Serious Surgical Incidents in 5 Large UK Teaching Hospitals: A System-Based Approach

OBJECTIVES

Surgical incidents are the most common serious patient safety incidents worldwide. We conducted a review of serious surgical incidents recorded in 5 large teaching hospitals located in one London NHS trust to identify possible contributing factors and propose recommendations for safer healthcare systems.

METHODS

We searched the Datix system for all serious surgical incidents that occurred in any operating room, excluding critical care departments, and were recorded between October 2014 and December 2016. We used the London Protocol system analysis framework, which involved a 2-stage approach. A brief description of each incident was produced, and an expert panel analyzed these incidents to identify the most likely contributing factors and what changes should be recommended.

RESULTS

One thousand fifty-one surgical incidents were recorded, 14 of which were categorized as "serious" with contributing factors relating to task, equipment and resources, teamwork, work environmental, and organizational and management. Operating room protocols were found to be unavailable, outdated, or not followed correctly in 8 incidents studied. The World Health Organization surgical safety checklist was not adhered to in 8 incidents, with the surgical and anesthetic team not informed about faulty equipment or product shortages before surgery. The lack of effective communication within multidisciplinary teams and inadequate medical staffing levels were perceived to have contributed.

CONCLUSIONS

Multiple factors contributed to the occurrence of serious surgical incidents, many of which related to human failures and faulty equipment. The use of faulty equipment needs to be recognized as a major risk within departments and promptly addressed.

Immediate faculty feedback using debriefing timing data and conversational diagrams

BACKGROUND

Debriefing is an essential skill for simulation educators and feedback for debriefers is recognised as important in progression to mastery. Existing assessment tools, such as the Debriefing Assessment for Simulation in Healthcare (DASH), may assist in rating performance but their utility is limited by subjectivity and complexity. Use of quantitative data measurements for feedback has been shown to improve performance of clinicians but has not been studied as a focus for debriefer feedback.

METHODS

A multi-centre sample of interdisciplinary debriefings was observed. Total debriefing time, length of individual contributions and demographics were recorded. DASH scores from simulation participants, debriefers and supervising faculty were collected after each event. Conversational diagrams were drawn in real-time by supervising faculty using an approach described by Dieckmann. For each debriefing, the data points listed above were compiled on a single page and then used as a focus for feedback to the debriefer.

RESULTS

Twelve debriefings were included (µ = 6.5 simulation participants per event). Debriefers receiving feedback from supervising faculty were physicians or nurses with a range of experience (n = 7). In 9/12 cases the ratio of debriefer to simulation participant contribution length was ≧ 1:1. The diagrams for these debriefings typically resembled a fan-shape. Debriefings (n = 3) with a ratio < 1:1 received higher DASH ratings compared with the ≧ 1:1 group (p = 0.038). These debriefings generated star-shaped diagrams. Debriefer self-rated DASH scores (µ = 5.08/7.0) were lower than simulation participant scores (µ = 6.50/7.0). The differences reached statistical significance for all 6 DASH elements. Debriefers evaluated the 'usefulness' of feedback and rated it 'highly' (µ= 4.6/5).

CONCLUSION

Basic quantitative data measures collected during debriefings may represent a useful focus for immediate debriefer feedback in a healthcare simulation setting.

Exploring the nature and focus of feedback when using video playback in gynecology laparoscopy training

BACKGROUND

Feedback about intraoperative performance remains a cornerstone of surgical training. Video playback offers one potential method for more effective feedback to surgical residents. More research is needed to better understand this method. This study explores the nature of instructional interactions and feedback in the operating room (OR) and when using video playback during post-operative review in obstetrics and gynecology (OBGYN) training.

METHOD

This case study occurred between September 2016 and February 2017. Three OBGYN residents and five OBGYN supervising surgeons were involved in six laparoscopic cases. Intraoperative and video playback dialogues were recorded and analysed, the former deductively using codes identified from published literature, and the latter both deductively, using the same codes, and inductively, with codes that emerged from the data during analysis.

RESULTS

1090 intraoperative interactions were identified within 376 minutes of dialogue. Most interactions were didactic, instructing the resident how to use an instrument to perform a task. Deductive analysis of postoperative video playback review identified 146 interactions within 155 minutes. While the most common interaction type remained didactic, a teaching component was included more often. It became apparent that deductive analysis using the intraoperative codes did not adequately capture the nature and focus of feedback during video playback. Hermeneutic phenomenological analysis identified more dialogic video playback sessions with more resident-initiated questions and reflection.

CONCLUSIONS

This study demonstrates that the nature of feedback during video playback is fundamentally different from that in the OR, offering a greater potential for collaborative and improved learning.

Intraoperative Teaching and Evaluation in General Surgery

The operating room continues to be the location where surgical residents develop both technical and nontechnical skills, ultimately culminating with them being capable of safe and independent practice. The process of intraoperative instruction is, by necessity, moving from an apprentice-based model where skills are acquired somewhat randomly through repeated exposure and evaluation is done in a global gestalt fashion. Modern surgical education demands that intraoperative instruction be intentional and that evaluation provides formative and summative feedback. This chapter describes some best practice approaches to intraoperative teaching and evaluation.

Learning from safety incidents in high-reliability organizations: a systematic review of learning tools that could be adapted and used in healthcare

OBJECTIVE

A high-reliability organization (HRO) is an organization that has sustained almost error-free performance, despite operating in hazardous conditions where the consequences of errors could be catastrophic. A number of tools and initiatives have been used within HROs to learn from safety incidents, some of which have the potential to be adapted and used in healthcare. We conducted a systematic review to identify any learning tools deemed to be effective that could be adapted and used by multidisciplinary teams in healthcare following a patient safety incident.

METHODS

This review followed the Preferred Reporting Items for Systematic Reviews and MetaAnalyses for Protocols reporting guidelines and was registered with the PROSPERO (CRD42017071528). A search of databases was carried out in January 2021, from the date of their commencement. We conducted a search on electronic databases such as Web of Science, Science Direct, MEDLINE in Process Jan 1950-present, EMBASE Jan 1974-present, CINAHL 1982-present, PsycINFO 1967-present, Scopus and Google Scholar. We also searched the grey literature including reports from government agencies, relevant doctoral dissertations and conference proceedings. A customized data extraction form was used to capture pertinent information from included studies and Critical Appraisal Skills Programme tool to appraise on their quality.

RESULTS

A total of 5921 articles were identified, with 964 duplicate articles removed and 4932 excluded at the title (4055), abstract (510) and full-text (367) stages. Twenty-five articles were included in the review. Learning tools identified included debriefing, simulation, crew resource management and reporting systems to disseminate safety messages. Debriefing involved deconstructing incidents using reflective questions, whilst simulation training involved asking staff to relive the event again by performing the task(s) in a role-play scenario. Crew resource management is a set of training procedures that focus on communication, leadership and decision-making. Sophisticated incident-reporting systems provide valuable information on hazards and were widely recommended as a way of disseminating key safety messages following safety incidents. These learning tools were found to have a positive impact on learning if conducted soon after the incident with efficient facilitation.

CONCLUSION

Healthcare organizations should find ways to adapt to the learning tools or initiatives used in HROs following safety incidents. It is challenging to recommend any specific one as all learning tools have shown considerable promise. However, the way these tools or initiatives are implemented is critical, and so further work is needed to explore how to successfully embed them into healthcare organizations so that everyone at every level of the organization embraces them.

Education Time Out and Debrief: Structured Implementation of Perioperative Resident Technical Education Discussion

BACKGROUND

We designed a model for pre- and postoperative discussions between faculty and trainees to maximize educational yield of cases and accelerate residents' technical development. We sought to study its effect on surgical education via participant perceptions and longitudinal validated performance evaluations.

STUDY DESIGN

Our model included preoperative collaborative technical goal-setting, specific to the resident, or "Time Out," and immediate postoperative granular feedback guided by validated evaluation tools, or "Debrief." We encouraged routine use for two 3-month rotations. We administered surveys with Likert scale and open-ended questions before and after implementation to assess adoption and perceptions. Likert scale survey data were analyzed using Mann-Whitney U tests; reported time durations were analyzed using t-tests. At 2 time points per rotation, designated faculty evaluated participating residents using the Objective Structured Assessment of Technical Skills (OSATS) for open/endoscopic cases or Global Evaluative Assessment of Robotic Skills (GEARS). OSATS and GEARS data were analyzed using paired t-tests.

RESULTS

Before our intervention, we noted significant differences between attending and resident physicians' perceptions of the frequency, importance, and challenges of perioperative educational discussions. After our intervention, these disparities resolved. In addition, participants reported significantly improved satisfaction with pre- and postoperative educational discussions (p = 0.01). Use of the model did not require increased time per participants' report. Paired GEARS/OSATS were completed for 9 trainees during the intervention, with faculty ratings revealing significant improvement in resident technical skills (p = 0.03).

CONCLUSIONS

Our structured model for perioperative educational discussions, consisting of the preoperative "Education Time Out" and postoperative "Education Debrief," significantly improved faculty and resident satisfaction and was associated with measurable improvements in resident technical skills without requiring significantly more time.

General Surgery Residents' Perception of Feedback: We Can Do Better

BACKGROUND

Feedback (FB) regarding perioperative care is essential in general surgery residents' (GSRs) training. We hypothesized that FB would be distributed unevenly across preoperative (PrO), intraoperative (IO), and postoperative (PO) continuum of the perioperative period. We aimed to compare results between university- and community-hospital settings planning to institute structured, formalized FB in a large health care system operating multiple surgery residency programs in departments that are linked strategically.

METHODS

Quantitative, cross-sectional, Likert scale anonymous surveys were distributed to all GSRs (categorical and preliminary; university: community 1:2). Twenty-five questions considered frequency and perceived quality of FB in PrO, IO, and PO settings. Data were tabulated using REDCap and analyzed in Microsoft Excel using the Mann-Whitney U test, with α = 0.05. Comparisons were made between university- and community-hospital settings, between junior (Post-Graduate Year (PGY) 1-3) and senior (PGY 4-5) GSRs, and by gender.

RESULTS

Among 115 GSRs surveyed, 83 (72%) responded. Whereas 93% reported receiving some FB within the past year, 46% reported receiving FB ≤ 20% of the time. A majority (58%) found FB to be helpful ≥ 80% of the time. Among GSRs, 77%, 24%, and 64% reported receiving PrO, IO, or PO FB ≤ 20% of the time, respectively, but 52% also believed that FB was lacking in all 3 areas. Most GSRs wanted designated time for PrO planning FB (82%) and PO FB (87%), respectively. Thirty-six percent of GSRs reported that senior/chief (i.e., PGY-4/PGY-5 GSRs) took them through cases ≥40% of the time; notably,78% reported that FB from senior/chief GSRs was equally or more valuable than FB from attending surgeons. A majority (78%) reported that attending surgeons stated explicitly when they were providing FB only ≤20% of the time. GSRs at the community hospital campuses reported receiving a higher likelihood of "any" FB, IO FB, and PO FB (p < 0.05). Most GSRs surveyed preferred a structured format and designated times for debriefing and evaluation of performance. Subanalyses of gender and GSR level of training showed no differences.

CONCLUSIONS

FB during GSR training varies across the perioperative continuum of care. Community programs seem to do better than University Programs. More work need to be done to elucidate why differences exist between the frequency of FB at University and Community programs. Further, data show particularly low FB outside of the operating room. Ideally, according to respondents, FB would be provided in a structured format and at designated times for debriefing and evaluation of performance, which poses a challenge considering the temporal dynamism of general surgery services.

Leadership-Specific Feedback Practices in Surgical Residency: A Qualitative Study

OBJECTIVE

The importance of feedback is well recognized in surgical training. Although there is increased focus on leadership as an essential competency in surgical training, it is unclear whether surgical residents receive effective feedback on leadership performance. We performed an exploratory qualitative study with surgical residents to understand current leadership-specific feedback practices in one surgical training program.

DESIGN

We conducted semistructured interviews with surgical residents. Using line-by-line coding in an iterative process, we focused on feedback on leadership performance to capture both semantic and conceptual data.

SETTING

The general surgery residency program at the University of Michigan, a tertiary care, academic institution.

PARTICIPANTS

Residents were purposively selected to include key informants and comprise a balanced sample with respect to postgraduate year, gender, and race.

RESULTS

Four major themes were identified during the thematic analysis: (1) the importance of feedback for leadership development in residency; (2) inadequacy of current feedback mechanisms; (3) barriers to giving and receiving leadership-specific feedback; and (4) resident-driven recommendations for better leadership feedback.

CONCLUSIONS

Many surgical residents do not receive effective leadership feedback, although they express strong desire for formal evaluation of leadership skills. Establishing avenues for feedback on leadership performance will help bridge this gap. Additionally, training to give and receive leadership-specific feedback may improve the quality and incorporation of delivered feedback for developing surgeon-leaders.

Enhancing Formative Feedback in Orthopaedic Training: Development and Implementation of a Competency-Based Assessment Framework

OBJECTIVE

The purpose of this study was to develop, implement, and evaluate the effectiveness of an assessment framework aimed at improving formative feedback practices in a Canadian orthopaedic postgraduate training program.

METHODS

Tool development began in 2014 and took place in 4 phases, each building upon the previous and informing the next. The reliability, validity, and educational impact of the tools were assessed on an ongoing basis, and changes were made accordingly.

RESULTS

One hundred eighty-two tools were completed and analyzed during the study period. Quantitative results suggested moderate to excellent agreement between raters (intraclass correlation coefficient = 0.54-0.93), and an ability of the tools to discriminate between learners at different stages of training (p's < 0.05). Qualitative data suggested that the tools improved both the quality and quantity of formative feedback given by assessors and had begun to foster a culture change around assessment in the program.

CONCLUSIONS

The tool development, implementation, and evaluation processes detailed in this article can serve as a model for other training programs to consider as they move towards adopting competency-based approaches and refining current assessment practices.

The Resident Report Card: A Tool for Operative Feedback and Evaluation of Technical Skills

BACKGROUND

Competency-based medical education surgical curriculums will require frequent, recorded trainee performance evaluations. It is our hypothesis that written feedback after each operation can be used to chart surgical progress, can identify underperforming trainees, and will prove beneficial for resident learning.

METHODS

The resident report card (RRC) is an online, easy-to-use evaluation tool designed to facilitate the creation and distribution of resident technical assessments. RRC data were collected from urologic trainees and analyzed using ANOVA and post hoc testing to confirm our hypothesis. A standardized survey was sent to residents, gauging their views on the RRC.

RESULTS

Over a 5-y period, 958 RRCs with the resident listed as the primary operator were collected across 29 different procedures. Resident cohort and individual performance scores stratified by postgraduate year (PGY) were shown to significantly improve when all procedures (cohort, 6.5 ± 1.9 [PGY-1] to 9.1 ± 1.0 [PGY-5]; individual [resident M], 8.8 ± 1.8 [PGY-3] to 9.4 ± 0.7 [PGY-5], P < 0.01) and specific procedures (laparoscopic donor nephrectomy: cohort, 7.3 ± 1.3 [PGY-3] to 8.9 ± 1.0 [PGY-5]; individual [resident I], 7.2 ± 1.3 [PGY-3] to 9.5 ± 0.6 [PGY-5], P < 0.01) were analyzed. Individual residents were able to be compared to their own peer group and to the average scores across all evaluated residents. Surveyed residents were overwhelmingly positive about the RRC.

CONCLUSIONS

The RRC adds further evidence to the fact that standardized, formative, and timely assessment can capture trainee performance over time and against comparator cohorts in an acceptable format to residents and academic training programs.

A study to explore specific stressors and coping strategies in primary dental care practice

Background and aims It is widely acknowledged that dentists experience occupational stress. This qualitative study aimed to explore previously identified specific stressors in more detail in order to inform the development of a future stress management programme.Method Two focus groups of dentists (N: 7 &6) were conducted to explore, in more detail, nine specific stressors and concepts; being out of one's comfort zone, zoning out from the patient, celebrating the positive aspects of work, thinking aloud, the effect of hurting patients, the impact of perfectionism, responsibility for patient's self-care, the emotional impact of difficult situations as a foundation dentist. Participants were also asked for their views on the structure and contents of the proposed stress management package. Verbatim transcripts were subjected to thematic analysis.Results and discussion Dentists described the impact of these stressors and their current coping methods; thematic analysis revealed nine themes which covered the above concepts and a further overall theme of need for control. The findings are elaborated in connection to their relevant stress, coping and emotion psychological theory. Their implications for personal well-being and clinical outcomes are discussed.Conclusion Dentists' stressful and coping experiences are complex and it is essential that any stress management programme reflects this and that the skills are easily accessible and sustainable within the context of a busy dental practice.

Quality of Operative Performance Feedback Given to Thoracic Surgery Residents Using an App-Based System

OBJECTIVE

Residents frequently report inadequate feedback both in quantity and quality. The study evaluates the quality of faculty feedback about operative performance given using an app-based system.

DESIGN

Residents requested operative performance evaluation from faculty on a real-time basis using the "Zwisch Me!!" mobile application which allows faculty to provide brief written feedback. Qualitative analysis of feedback was performed using grounded theory.

SETTING

The 7 academic medical centers with thoracic surgery training programs.

PARTICIPANTS

Volunteer thoracic surgery residents in both integrated and traditional training pathways and their affiliated cardiothoracic faculty.

RESULTS

Residents (n = 33) at 7 institutions submitted a total of 596 evaluations to faculty (n = 48). Faculty acknowledged the evaluation request in 476 cases (80%) and in 350 cases (74%) provided written feedback. Initial open coding generated 12 categories of feedback type. We identified 3 overarching themes. The first theme was the tone of the feedback. Encouraging elements were identified in 162 comments (46%) and corrective elements in 230 (65%). The second theme was the topic of the feedback. Surgical technique was the most common category at 148 comments (42.2%) followed by preparation for case (n = 69, 19.7%). The final theme was the specificity of the feedback. Just over half of comments (n = 190, 54.3%) contained specific feedback, which could be applied to future cases. However, 51 comments (14.6%) contained no useful information for the learners.

CONCLUSIONS

An app-based system resulted in thoracic surgery residents receiving identifiable feedback in a high proportion of cases. In over half of comments the feedback was specific enough to allow improvement. Feedback was better quality when addressing error prevention and surgical technique but was less useful when addressing communication, flow of the case, and assisting. Faculty development around feedback should focus on making feedback specific and actionable, avoiding case descriptions, or simple platitudes.

SimZones: An Organizational Innovation for Simulation Programs and Centers

The complexity and volume of simulation-based learning programs have increased dramatically over the last decade, presenting several major challenges for those who lead and manage simulation programs and centers. The authors present five major issues affecting the organization of simulation programs: (1) supporting both single- and double-loop learning experiences; (2) managing the training of simulation teaching faculty; (3) optimizing the participant mix, including individuals, professional groups, teams, and other role-players, to ensure learning; (4) balancing in situ, node-based, and center-based simulation delivery; and (5) organizing simulation research and measuring value. They then introduce the SimZones innovation, a system of organization for simulation-based learning, and explain how it can alleviate the problems associated with these five issues.Simulations are divided into four zones (Zones 0-3). Zone 0 simulations include autofeedback exercises typically practiced by solitary learners, often using virtual simulation technology. Zone 1 simulations include hands-on instruction of foundational clinical skills. Zone 2 simulations include acute situational instruction, such as clinical mock codes. Zone 3 simulations involve authentic, native teams of participants and facilitate team and system development.The authors also discuss the translation of debriefing methods from Zone 3 simulations to real patient care settings (Zone 4), and they illustrate how the SimZones approach can enable the development of longitudinal learning systems in both teaching and nonteaching hospitals. The SimZones approach was initially developed in the context of the Boston Children's Hospital Simulator Program, which the authors use to illustrate this innovation in action.

Perioperative feedback in surgical training: A systematic review

BACKGROUND

Changes in surgical training have raised concerns about residents' operative exposure and preparedness for independent practice. One way of addressing this concern is by optimizing teaching and feedback in the operating room (OR). The objective of this study was to perform a systematic review on perioperative teaching and feedback.

METHODS

A systematic literature search identified articles from 1994 to 2014 that addressed teaching, feedback, guidance, or debriefing in the perioperative period. Data was extracted according to ENTREQ guidelines, and a qualitative analysis was performed.

RESULTS

Thematic analysis of the 26 included studies identified four major topics. Observation of teaching behaviors in the OR described current teaching practices. Identification of effective teaching strategies analyzed teaching behaviors, differentiating positive and negative teaching strategies. Perceptions of teaching behaviors described resident and attending satisfaction with teaching in the OR. Finally models for delivering structured feedback cited examples of feedback strategies and measured their effectiveness.

CONCLUSIONS

This study provides an overview of perioperative teaching and feedback for surgical trainees and identifies a need for improved quality and quantity of structured feedback.

Quality of interdisciplinary postsimulation debriefing: 360° evaluation

Introduction

Debriefing is widely perceived to be the most important component of simulation-based training. This study aimed to explore the value of 360° evaluation of debriefing by examining expert debriefing evaluators, debriefers and learners’ perceptions of the quality of interdisciplinary debriefings.

Method

This was a cross-sectional observational study. 41 teams, consisting of 278 learners, underwent simulation-based team training. Immediately following the postsimulation debriefing session, debriefers and learners rated the quality of debriefing using the validated Objective Structured Assessment of Debriefing (OSAD) framework. All debriefing sessions were video-recorded and subsequently rated by evaluators trained to proficiency in assessing debriefing quality.

Results

Expert debriefing evaluators and debriefers’ perceptions of debriefing quality differed significantly; debriefers perceived the quality of debriefing they provided more favourably than expert debriefing evaluators (40.98% of OSAD ratings provided by debriefers were ≥+1 point greater than expert debriefing evaluators’ ratings). Further, learner perceptions of the quality of debriefing differed from both expert evaluators and debriefers’ perceptions: weak agreement between learner and expert evaluators’ perceptions was found on 2 of 8 OSAD elements (learner engagement and reflection); similarly weak agreement between learner and debriefer perceptions was found on just 1 OSAD element (application).

Conclusions

Debriefers and learners’ perceptions of debriefing quality differ significantly. Both groups tend to perceive the quality of debriefing far more favourably than external evaluators. An overconfident debriefer may fail to identify elements of debriefing that require improvement. Feedback provided by learners to debriefers may be of limited value in facilitating improvements. We recommend periodic external evaluation of debriefing quality.

Resident education in orthopaedic trauma

As residency training programmes around the globe move towards competency-based medical education (CBME), there is a need to review current teaching and assessment practices as they relate to education in orthopaedic trauma. Assessment is the cornerstone of CBME, as it not only helps to determine when a trainee is fit to practice independently, but it also provides feedback on performance and guides the development of competence. Although a standardised core knowledge base for trauma care has been developed by the leading national accreditation bodies and international agencies that teach and perform research in orthopaedic trauma, educators have not yet established optimal methods for assessing trainees’ performance in managing orthopaedic trauma patients.

This review describes the existing knowledge from the literature on assessment in orthopaedic trauma and highlights initiatives that have recently been undertaken towards CBME in the United Kingdom, Canada and the United States.

In order to support a CBME approach, programmes need to improve the frequency and quality of assessments and improve on current formative and summative feedback techniques in order to enhance resident education in orthopaedic trauma.

Cite this article: Bone Joint J 2016;98-B:1320–5.

Comprehensive feedback on trainee surgeons' non-technical skills

OBJECTIVE

This study aimed to explore the content of conversations, feedback style, and perceived usefulness of feedback to trainee surgeons when conversations were stimulated by a tool for assessing surgeons' non-technical skills.

METHODS

Trainee surgeons and their supervisors used the Non-Technical Skills for Surgeons in Denmark tool to stimulate feedback conversations. Audio recordings of post-operation feedback conversations were collected. Trainees and supervisors provided questionnaire responses on the usefulness and comprehensiveness of the feedback. The feedback conversations were qualitatively analyzed for content and feedback style. Usefulness was investigated using a scale from 1 to 5 and written comments were qualitatively analyzed.

RESULTS

Six trainees and six supervisors participated in eight feedback conversations. Eighty questionnaires (response rate 83 percent) were collected from 13 trainees and 12 supervisors. Conversations lasted median eight (2-15) minutes. Supervisors used the elements and categories in the tool to structure the content of the conversations. Supervisors tended to talk about the trainees' actions and their own frames rather than attempting to understand the trainees' perceptions. Supervisors and trainees welcomed the feedback opportunity and agreed that the conversations were useful and comprehensive.

CONCLUSIONS

The content of the feedback conversations reflected the contents of the tool and the feedback was considered useful and comprehensive. However, supervisors talked primarily about their own frames, so in order for the feedback to reach its full potential, supervisors may benefit from training techniques to stimulate a deeper reflection among trainees.