What factors influence attending surgeon decisions about resident autonomy in the operating room?

Behind the pattern: General surgery residsent autonomy in robotic surgery

Objective

Robotic surgery is increasingly utilized and common in general surgery training programs. This study sought to better understand the factors that influence resident operative autonomy in robotic surgery. We hypothesized that resident seniority, surgeon work experience, surgeon robotic-assisted surgery (RAS) case volume, and procedure type influence general surgery residents' opportunities for autonomy in RAS as measured by percentage of resident individual console time (ICT).

Methods

General surgery resident ICT data for robotic cholecystectomy (RC), inguinal hernia (RIH), and ventral hernia (RVH) operations performed on the dual-console Da Vinci surgical robotic system between July 2019 and June 2021 were extracted. Cases with postgraduate year (PGY) 2-5 residents participating as a console surgeon were included. A sequential explanatory mixed-methods approach was undertaken to explore the ICT results and we conducted secondary qualitative interviews with surgeons. Descriptive statistics and thematic analysis were applied.

Results

Resident ICT data from 420 robotic cases (IH 200, RC 121, and VH 99) performed by 20 junior residents (PGY2-3), 18 senior residents (PGY4-5), and 9 attending surgeons were extracted. The average ICT per case was 26.8 % for junior residents and 42.4 % for senior residents. Compared to early-career surgeons, surgeons with over 10 years' work experience gave less ICT to junior (18.2 % vs. 32.0 %) and senior residents (33.9 % vs. 56.6 %) respectively. Surgeons' RAS case volume had no correlation with resident ICT (r = 0.003, p = 0.0003). On average, residents had the most ICT in RC (45.8 %), followed by RIH (36.7 %) and RVH (28.6 %). Interviews with surgeons revealed two potential reasons for these resident ICT patterns: 1) Surgeon assessment of resident training year/experience influenced decisions to grant ICT; 2) Surgeons' perceived operative time pressure inversely affected resident ICT.

Conclusions

This study suggests resident ICT/autonomy in RC, RIH, and RVH are influenced by resident seniority level, surgeon work experience, and procedure type, but not related to surgeon RAS case volume. Design and implementation of an effective robotic training program must consider the external pressures at conflict with increased resident operative autonomy and seek to mitigate them.

Factors necessary for entrustment decision-making in surgical operating rooms: A modified Delphi study

Objective

Resident autonomy in an operation theatre has been directly linked with patient safety in healthcare. The objective of this study was to identify the factors necessary for making resident-entrustment decisions from the perspective of a supervisor/consultant viewpoint. The second objective was to develop a checklist for assessing resident readiness for independent work.

Materials and methods

This study employed a mixed-method Delphi approach. In the first stage, a comprehensive literature review and a qualitative exploratory study produced a list of factors related to residents. The second phase involved content validation by a panel of experts, followed by a two-round Delphi study with 20 expert panelists.

Results

A total of 49 resident-related factors for entrustment were identified, which were reduced to 46 after content validation. During Delphi Round I, 17 factors were fully accepted, 7 were rejected and 22 items were partially accepted. Out of the 39 items sent to Delphi Round II, 23 items were accepted and 16 were rejected. A final 23-item checklist was formed based on the following factors; 6 Cognitive (knowledgeable, risk manager, safe doctor, general manager, field of interest, communicator), 5 Psychomotor (past performer, competent, ability to pick critical findings, ability to act situationally, decision maker) and 12 affective (responsible, leader, honest, empathetic, ethical, receptive, humble, emotionally intelligent, motivated, accountable, team player, disciplined) factors.

Conclusion

The study resulted in the formation of a checklist based on the factors necessary for entrustment decision-making in surgical operating rooms. Some of the novel contextual factors were 'general manager', 'field of interest', 'ability to pick critical findings', 'accountable', 'risk manager', and 'past performer'. This framework offers a guideline for supervisors and residents to evaluate progress throughout the residency program. The developed tool demonstrates good content validity and is suitable for entrustment assessment following construct validation.

Why Not This Case? Differences Between Resident and Attending Operative Cases at Teaching Hospitals

INTRODUCTION

Previous studies have focused on outcomes pertaining to resident operative autonomy, but there has been little academic work examining the types of patients and cases where autonomy is afforded. We sought to describe the differences between surgical patient populations in teaching cases where residents are and are not afforded autonomy.

METHODS

We examined all general and vascular operations at Veterans Affairs teaching hospitals from 2004 to 2019 using Veterans Affairs Surgical Quality Improvement Program. Level of resident supervision is prospectively recorded by the operating room nurse at the time of surgery: attending primary (AP): the attending performs the case with or without a resident; attending resident (AR): the resident performs the case with the attending scrubbed; resident primary (RP): resident operating with supervising attending not scrubbed. Resident (R) cases refer to AR + RP. Patient demographics, comorbidities, level of supervision, and top cases within each group were evaluated.

RESULTS

A total of 618,578 cases were analyzed; 154,217 (24.9%) were AP, 425,933 (68.9%) AR, and 38,428 (6.2%) RP. Using work relative value unit as a surrogate for complexity, RP was the least complex compared to AP and AR (10.4/14.4/14.8, P < 0.001). RP also had a lower proportion of American Society of Anesthesiologists 3 and 4 + 5 patients (P < 0.001), were younger (P < 0.001), and generally had lower comorbidities. The most common RP cases made up a higher proportion of all RP cases than they did for AP/AR and demonstrated several core competencies (hernia, cholecystectomy, appendectomy, amputation). R cases, however, were generally sicker than AP cases.

CONCLUSIONS

In the small proportion of cases where residents were afforded autonomy, we found they were more focused on the core general surgery cases on lower risk patients. This selection bias likely demonstrates appropriate attending judgment in affording autonomy. However, this cohort consisted of many "sicker" patients and those factors alone should not disqualify resident involvement.

Understanding and addressing varying perceptions of autonomy

Recently, there has been a major shift in the concept of resident autonomy in the operating room. As a result, surgical residents' independence has decreased during their training years. This change has been secondary to multiple factors, including fragmented attending resident interactions, hospital demands for productivity, operating room efficiency, and the public's perception of resident participation in surgery. Multiple gender, personality, and racial biases have also influenced the autonomy of surgical residents. In this paper, we have analyzed the impact of all these factors on the current state of resident autonomy after reviewing relevant literature. We have proposed a strategy to increase resident autonomy via increased resident and faculty interactions, case planning, and encouraged recruitment of diverse vascular surgery trainees and faculty.

Evaluation Tools for Assessing Autonomy of Surgical Residents in the Operating Room and Factors Influencing Access to Autonomy: A Systematic Literature Review

BACKGROUND

Surgical residents in France lack a clear pedagogical framework for achieving autonomy in the operating room. The progressive acquisition of surgical autonomy is a determining factor in the confidence of operators for their future independent practice. Currently, there is no autonomy scale commonly used in Europe. The objective of this study is to identify existing tools for quantifying the autonomy of residents and the factors that influence it.

MATERIALS AND METHODS

We conducted a qualitative systematic review following the recommendations of the Systematic Review Without Meta-Analysis (SWiM) guidelines. Publications were extracted from the MEDLINE (PubMed), EMBASE, and PSYCINFO databases. All publications without date restrictions up to July 2022 were identified.

RESULTS

Among the 231 identified publications, 21 met the inclusion criteria. Seventeen publications used a graded autonomy assessment tool by the student and/or the teacher, while 4 used evaluations by an observing third party. We found 8 different autonomy scales, with the Zwisch Scale representing 57.1% of the cases. Factors influencing autonomy were diverse, including the work context, experience, and gender of the resident and their teacher.

DISCUSSION

We found heterogeneity in the tools used to "measure" the autonomy of a resident in the operating room. The SIMPL tool or the Zwisch Scale appear to be the most frequently used tools. The relationship between autonomy, performance, confidence, and knowledge may require multidimensional tools that encompass various areas of competence, but this could make their daily application more challenging. The factors influencing autonomy are numerous; and understanding them would improve teaching in the operating room. There is a significant lack of data on surgical autonomy in France, as well as a lack of evaluation in the field of gynecology-obstetrics worldwide.

Perceptions and Guiding Strategies to Regulate Entrusted Autonomy of Residents in the Operating Room: A Systematic Literature Review

OBJECTIVE

To provide a systematic literature review of intraoperative entrusted autonomy for surgical residents. Specifically, perceptions from residents and supervising surgeons, supervising behavior and influencing factors on intraoperative teaching and learning are analyzed.

BACKGROUND

Increasing demands on surgical training and the need for effective development of technical skills, amplify the importance of making the most of intraoperative teaching and learning opportunities in the operating room. It is critical for residents to gain the greatest benefit from every surgical case and to achieve operative competence.

METHODS

A systematic literature search identified 921 articles from 2000 to 2022 that addressed surgical education/training, intraoperative supervision/teaching, autonomy and entrustment. 40 studies with heterogeneous designs and methodologies were included.

RESULTS

Four themes were established in the analysis: patient safety, learner, learning environment and supervising surgeon. The patient is identified as the primary responsibility during intraoperative teaching and learning. Supervisors continuously guard patient safety as well as the resident's learning process. Ideal intraoperative learning occurs when the resident has optimal entrusted autonomy during the procedure matching with the current surgical skills level. A safe learning environment with dedicated time for learning are prerequisites for both supervising surgeons and residents. Supervising surgeons' own preferences and confidence levels also play an important role.

CONCLUSIONS

This systematic literature review identifies patient safety as the overriding principle for supervising surgeons when regulating residents' entrusted autonomy. When the supervisor's responsibility toward the patient has been met, there is room for intraoperative teaching and learning. In this process the learner, the learning environment and the supervising surgeon's own preferences all intertwine, creating a triangular responsibility. This review outlines the challenge of establishing an equilibrium in this triangle and the broad arsenal of strategies supervising surgeons use to keep it in balance.

Autonomy Maps: Building a Shared Mental Model for Progressive Resident Operative Autonomy on the Road to Entrustable Professional Activities (EPAs)

Our residents expressed dissatisfaction with operative autonomy and faculty feedback regarding technical skills. They reported variability among faculty regarding allowed operative autonomy. Our goals were to establish a shared mental model among residents and faculty regarding intraoperative performance expectations. We asked faculty to assign a level of expected autonomy (Zwisch scale) for various steps of common procedures according to the resident post-graduate year. Through an iterative process, the maps were standardized across service lines. The resulting "Autonomy Maps" were distributed to the faculty and residents. We held educational sessions and set expectations for use. Selected benchmarks were incorporated into resident end-of-rotation assessment forms. Initial operative case mapping identified variability in faculty expectations for a given post-graduate year and procedure. Residents reported improved satisfaction with understanding expectations regarding operative performance. Establishing autonomy benchmarks facilitated more specific feedback regarding residents' technical skills. Faculty expectations for resident operative autonomy are variable. Autonomy Maps provide structure for a shared mental model between faculty and residents for progressive operative autonomy and serve as a framework for expectations that improve resident satisfaction. Case-specific technical benchmarks are useful tools for assessing residents' technical milestones.

Empowering the Resident Learner

Surgical resident autonomy is an essential element of the transition from supervised training to independent practice. However, in an age of duty hour restrictions, legislative constraints, increased litigation, and heightened societal expectations, training an autonomous resident proves increasingly difficult. To tackle these barriers to training successful surgeons, the American College of Surgeons Committee on Resident Education initiated the Resident Mentored Autonomy Project. As a subdivision of this project, the Empowered Learner research team here presents a framework for training the self-directed and empowered surgical resident learner. There are many strategies by which surgical faculty, program directors and chairs, and residents themselves may engage to improve resident operative autonomy.

'It depends': The complexity of allowing residents to fail from the perspective of clinical supervisors

PURPOSE

Clinical supervisors acknowledge that they sometimes allow trainees to fail for educational purposes. What remains unknown is how supervisors decide whether to allow failure in a specific instance. Given the high stakes nature of these decisions, such knowledge is necessary to inform conversations about this educationally powerful and clinically delicate phenomenon.

MATERIALS AND METHODS

19 supervisors participated in semi-structured interviews to explore how they view their decision to allow failure in clinical training. Following constructivist grounded theory methodology, the iteratively collected data and analysis were informed by theoretical sampling.

RESULTS

Recalling instances when they considered allowing residents to fail for educational purposes, supervisors characterized these as intuitive, in-the-moment decisions. In their post hoc reflections, they could articulate four factors that they believed influenced these decisions: patient, supervisor, trainee, and environmental factors. While patient factors were reported as primary, the factors appear to interact in dynamic and nonlinear ways, such that supervisory decisions about allowing failure may not be predictable from one situation to the next.

CONCLUSIONS

Clinical supervisors make many decisions in the moment, and allowing resident failure appears to be one of them. Upon reflection, supervisors understand their decisions to be shaped by recurring factors in the clinical training environment. The complex interplay among these factors renders predicting such decisions difficult, if not impossible. However, having a language for these dynamic factors can support clinical educators to have meaningful discussions about this high-stakes educational strategy.

The long game: Evolution of clinical decision making throughout residency and fellowship

BACKGROUND

The purpose of this study was to explore the trajectory of autonomy in clinical decision making.

METHODS

We conducted a qualitative secondary analysis of interviews with 45 residents and fellows from the General Surgery and Obstetrics & Gynecology departments across all clinical postgraduate years (PGY) using convenience sampling. Each interview was recorded, transcribed and iteratively analyzed using a framework method.

RESULTS

A total of 16 junior residents, 22 senior residents and 7 fellows participated in 12 original interviews. Early in training residents take their abstract ideas about disease processes and make them concrete in their applications to patient care. A transitional stage follows in which residents apply concepts to concrete patient care. Chief residents re-abstract their concrete technical and clinical knowledge to prepare for future surgical practice.

CONCLUSIONS

Understanding where each learner is on this pathway will assist development of curriculum that fosters resident readiness for practice at each PGY level.

Transition from trainee to educator in the operating room: A needs assessment and framework to support junior faculty

BACKGROUND

Transitioning from trainee to attending surgeon requires learners to become educators. The purpose of this study is to evaluate educational strategies utilized by surgeons, define gaps in preparation for operative teaching, and identify opportunities to support this transition.

METHODS

A web-based, Association of Surgical Education approved survey was distributed to attending surgeons.

RESULTS

There were 153 respondents. Narrating actions was the most frequently reported educational model, utilized by 74% of junior faculty [JF] (0-5yrs) and 63% of senior faculty [SF] (>6yrs). Other models used included educational time-outs (29% JF, 27% SF), BID teaching model (36% JF, 51% SF), and Zwisch model (13% JF, 25% SF). Compared with 91% JF, 65% SF reported struggling with instruction (p < 0.001). Five themes emerged as presenting difficulty during the resident to attending transition: lack of relationships, ongoing learning, systems-based, cognitive load, impression management.

CONCLUSIONS

Our results represent a needs assessment in the transition from learner to educator in the OR.

Correlation of Supervised Independence and Performance with Procedure Difficulty amongst Surgical Residents Stratified by Post Graduate Year

PURPOSE

This study investigates the role of procedure difficulty on attending ratings of supervised levels of independence and procedural performance amongst general surgery residents, while accounting for case complexity.

METHODS

Attending ratings for residents were obtained from System for Improving and Measuring Procedural Learning (SIMPL) database. Current procedural terminology (CPT) codes were used to match procedures to a corresponding work relative value unit (wRVU) as a surrogate for procedure difficulty. Three categories of wRVU (<13.07, 13.07-22, >22) were identified using recursive partitioning. Procedures were also divided into 'Core' or 'Advanced' as defined by the American Board of Surgery Surgical Council on Resident Education (SCORE). Temporal advancement in resident skill was accounted for through academic quarterly analysis. A generalized estimating equations (GEE) approach was used to form separate multivariable logistic regression models for meaningful autonomy (MA) and satisfactory performance (SP) adjusted for potential clustering by program, subject, and rater. Models were further adjusted for core/advanced procedures, attending rated complexity, and academic quarter.

RESULTS

A total of 33,281 ratings were analyzed. Overall, 51.6% were rated as MA and 44.4% as SP. For core procedures, surgical residents rated as MA (53.5%) and SP (45.7%), which was twice as high as those for advance procedures (MA-29.2%, SP-29.0%). MA and SP both decreased with increasing wRVU (Figure 2 &3). Using a wRVU<13.07 as a reference, the adjusted odds ratios of MA and SP were significantly lower with increasing procedure difficulty, 0.44 for wRVU 13.07-22.0 and 0.24 for wRVU >22.00 (Table 3). Post graduate year (PGY) 5 residents in the final quarter of training obtain MA in 95.5% and SP 92.9% for core procedures with wRVU <13.07 (Table 4).

CONCLUSION

Increasing procedural difficulty is independently associated with decreases in meaningful autonomy and satisfactory performance. As residents approach graduation the level of meaningful autonomy and satisfactory performance both reach high levels for common core procedures but decrease as procedural difficulty increases.

Factors Influencing the Entrustment of Resident Operative Autonomy: Comparing Perceptions of General Surgery Residents and Attending Surgeons

BACKGROUND

General surgery residents may be underprepared for practice, due in part to declining operative autonomy during training. The factors that influence entrustment of autonomy in the operating room are unclear.

OBJECTIVE

To identify and compare the factors that residents and faculty consider influential in entrustment of operative autonomy.

METHODS

An anonymous survey of 29-item Likert-type scale (1-7, 1 = strongly disagree, 7 = strongly agree), 9 multiple-choice, and 4 open-ended questions was sent to 70 faculty and 45 residents in a large ACGME-approved general surgery residency program comprised of university, county, and VA hospitals in 2018.

RESULTS

Sixty (86%) faculty and 38 (84%) residents responded. Faculty were more likely to identify resident-specific factors such as better resident reputation and higher skill level as important in fostering entrustment. Residents were more likely to identify environmental factors such as a focus on efficiency and a litigious malpractice environment as impeding entrustment. Both groups agreed that work hour restrictions do not decrease autonomy and entrustment does not increase risk to patients. More residents considered low faculty confidence level as a barrier to operative autonomy, while more faculty considered lower resident clinical skill as a barrier. Improvement in resident preparation for cases was cited as an important intervention that could enhance entrustment.

CONCLUSIONS

Differences in perspectives exist between general surgery residents and faculty regarding entrustment of autonomy. Residents cite environmental and attending-related factors, while faculty cite resident-specific factors as most influential. Residents and faculty both agree that entrustment is integral to surgical training.

Operative Confidence Among U.S General Surgery Residents

BACKGROUND

General surgery residents (GSRs) must develop operative autonomy skills to practice independently after graduation. We aim to investigate perceived confidence and operative autonomy of GSR physicians in order to identify and address influential factors.

METHODS

A 28-question anonymous online survey was distributed to 23 United States general surgery residency programs. Multivariable logistic regression was used for calculating the adjusted odds ratio (aOR) for binary outcomes. Significance was defined as P-values ≤ .05 or 95% confidence intervals (CIs) >1 or <1.

RESULTS

There were 120/558 (21.5%) GSR respondents. General surgery residents with >200 overall operative case volume reported significantly higher confidence with minor cases (P = .05) and major cases (P = .02). General surgery residents that performed both minor and major surgeries reported higher confidence with minor cases at 85.7% compared to GSRs that performed mostly minor surgeries (64.7%) and mostly major surgeries (62.5%). General surgery residents who performed >50 minor surgeries during their PGY 1 and 2 were less confident with major cases than GSRs who performed <50 minor surgeries (aOR: 19.98, 95% CI: 1.26, 318). General surgery residents from community teaching hospitals reported higher confidence with major and minor cases than GSRs from university teaching hospitals and combined programs.

CONCLUSION

Increased case volume, predominant case type, early surgical experience during PGY 1 and 2 years, and training at community teaching hospitals were identified as the most important factors that positively influence perception of operative confidence and autonomy among GSRs. These may have important implications in the development of future surgeons.

General Surgery Resident Autonomy: Truth and Myth

Within general surgery education circles, the state of autonomy for residents in surgery training programs has been of growing concern. Although there is no direct evidence showing less autonomy in modern surgical training, multiple surrogates have been cited as reasons for concern. Many reasons have been given for lost autonomy including the 80-hour work week, financial constraints, concerns over quality of patient care, patient expectations, new and innovative technologies, legal limitations, and public opinion. This article discusses the current state of general surgery resident autonomy, why autonomy is important, barriers to autonomy, and ways to support autonomy.

Residents' Perceptions of Faculty Behaviors and Resident Burnout: a Cross-Sectional Survey Study Across a Large Health Care Organization

BACKGROUND

Data suggests the learning environment factors influence resident well-being. The authors conducted an assessment of how residents' perceptions of faculty-resident relationships, faculty professional behaviors, and afforded autonomy related to resident burnout.

METHODS

All residents at one organization were surveyed in 2019 using two items from the Maslach Burnout Inventory and the faculty relationship subscale of the Johns Hopkins Learning Environment Scale (JHLES, range 6 to 30). Residents were also asked about faculty professional behaviors (range 0 to 30), and satisfaction with autonomy across various clinical settings.

RESULTS

A total of 762/1146 (66.5%) residents responded to the survey. After adjusting for age, gender, postgraduate year, and specialty, lower (less favorable) JHLES-faculty relationship subscale score (parameter estimate, - 3.08, 95% CI - 3.75, - 2.41, p < 0.0001), fewer observed faculty professional behaviors (parameter estimate, - 3.34, 95% CI - 4.02, - 2.67, p < 0.0001), and lower odds of satisfaction with autonomy in the intensive care settings (OR 0.46, 95% CI 0.30, 0.70, p = 0.001), but not other care settings, were reported by residents with burnout in comparison to those without. Similar relationships were observed when emotional exhaustion and depersonalization were analyzed separately as continuous variables.

CONCLUSION

In this cohort, resident perceptions of faculty relationships, faculty professional behaviors, and satisfaction with autonomy in the intensive care unit were associated with resident burnout. Additional longitudinal studies are needed to elucidate the direction of these relationships and determine if faculty development can reduce resident burnout.

Difference in Resident Versus Attending Perspective of Competency and Autonomy During Arthroscopic Rotator Cuff Repairs

A noted deficiency in orthopaedic resident education is a lack of intraoperative autonomy; however, no studies exist evaluating this issue. The purpose of this study was to determine whether there is a difference between resident and attending perception of resident competency and autonomy during arthroscopic rotator cuff repairs and whether increased perceived competency leads to more autonomy.

Medicine and surgery residents' perspectives on the impact of COVID-19 on graduate medical education

The COVID-19 crisis has had an unprecedented impact on resident education and well-being: social distancing guidelines have limited patient volumes and forced virtual learning, while personal protective equipment (PPE) shortages, school/daycare closures, and visa restrictions have served as additional stressors. Our study aimed to analyze the effects of COVID-19 crisis-related stressors on residents' professional and personal lives. In April 2020, we administered a survey to residents at a large academic hospital system in order to assess the impact of the pandemic on residency training after >6 weeks of a modified schedule. The primary outcome was to determine which factors or resident characteristics were related to stress during the pandemic. Our secondary goals were to examine which resident characteristics were related to survey responses. Data were analyzed with regression analyses. Ninety-six of 205 residents completed the survey (47% response rate). For our primary outcome, anxiety about PPE (P < 0.001), female gender (P = 0.03), and the interaction between female gender and anxiety about PPE (P = 0.04) were significantly related to increased stress during the COVID-19 pandemic. Secondary analyses suggested that medicine residents were more comfortable than surgical residents using telemedicine (P > 0.001). Additionally, compared to juniors, seniors believed that the pandemic was more disruptive, modified schedules were effective, and virtual meetings were less effective while virtual lectures were more effective (all P ≤ 0.05) Furthermore, the pandemic experience has allowed seniors in particular to feel more confident to lead in future health crises (P ≤ 0.05). Medicine and surgery residency programs should be cognizant of and closely monitor the effects of COVID-19 crisis-related factors on residents' stress and anxiety levels. Transparent communication, telemedicine, online lectures/meetings, procedure simulations, advocacy groups, and wellness resources may help to mitigate some of the challenges posed by the pandemic.

Use of a Secure Web-Based Data Management Platform to Track Resident Operative Performance and Program Educational Quality Over Time

OBJECTIVE

In surgery residency programs, Accreditation Council for Graduate Medical Education mandated performance assessment can include assessment in the operating room to demonstrate that necessary quality and autonomy goals are achieved by the conclusion of training. For the past 3 years, our institution has used The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) instrument to assess and track operative skills. Evaluation is accomplished in near real-time using a secure web-based platform for data management and analytics (Firefly). Simultaneous to access of the platform's case logging function, the O-SCORE instrument is delivered to faculty members for rapid completion, facilitating quality, and timeliness of feedback. We sought to demonstrate the platform's utility in detecting operative performance changes over time in response to focused educational interventions based on stored case log and O-SCORE data.

DESIGN

Stored resident performance assessments for the most frequently performed laparoscopic procedures (cholecystectomy, appendectomy, inguinal hernia repair, ventral hernia repair) were examined for 3 successive academic years (2016-2019). During this time, 4 of 36 residents had received program-assigned supplemental simulation training to improve laparoscopic skills. O-SCORE data for these residents were extracted from peer data, which were used for comparisons. Assigned training consisted of a range of videoscopic and virtual reality skills drills with performance objectives. O-SCORE responses were converted to integers and autonomy scores for items pertaining to technical skill were compared before and after educational interventions (Student's t-tests). These scores were also compared to aggregate scores in the nonintervention group. Bayesian-modeled learning curves were used to characterize patterns of improvement over time.

SETTING

University of Massachusetts Medical School-Baystate Surgery Residency and Baystate Medical Center PARTICIPANTS: General surgery residents (n = 36) RESULTS: During the period of review, 3325 resident cases were identified meeting the case type criteria. As expected, overall autonomy increased with the number of cases performed. The 4 residents who had been assigned supplemental training (6-18 months) had preintervention score averages that were lower than that of the nonintervention group (2.25 ± 0.43 vs 3.57 ± 1.02; p < 0.0001). During the respective intervention periods, all 4 residents improved autonomy scores (increase to 3.40 ± 0.61; p < 0.0001). Similar improvements were observed for tissue handling, instrument handling, bimanual dexterity, visuospatial skill, and operative efficiency component skills. Postintervention scores were not significantly different compared to scores for the non-intervention group. Bayesian-modeled learning curves showed a similar pattern of postintervention performance improvement.

CONCLUSIONS

The data management platform proved to be an effective tool to track responses to supplemental training that was deemed necessary to close defined skills gaps in laparoscopic surgery. This could be seen both in individual and in aggregated data. We were gratified that at the conclusion of the supplemental training, O-SCORE results for the intervention group were not different than those seen in the non-intervention group.

Repeat Attending Exposure Influences Operative Autonomy in Endocrine Surgical Procedures

PURPOSE

There is concern that graduating surgery residents are not prepared for independent practice. This study aimed to identify predictors of performance, autonomy, and readiness for independence ratings of trainees by attendings for thyroidectomy and parathyroidectomy with respect to repeated resident-attending exposure. We hypothesized that increased exposure with a particular attending increases resident autonomy.

METHODS

All residents and faculty at a single institution performing parathyroidectomy or thyroidectomy were invited to complete an operative performance evaluation at case competition using the Zwisch scale to measure performance and autonomy for individual operative steps. In addition, each survey evaluated the trainee's readiness for practice in a straightforward procedure as a binary variable. Categorical variables were evaluated via Chi-squared or Fisher's exact tests and ordinal variables were evaluated with Wilcoxon or Kruskal-Wallis tests. Multivariable analysis was conducted with random effects logistic regression, and learning curves were generated for each procedure.

RESULTS

Operative performance evaluations were obtained from 36 individual learners and 6 faculty members, with a total of 145 evaluations for parathyroidectomy and 116 for thyroidectomy. On bivariate analysis, readiness for practice ratings was significantly associated with increasing chronologic procedure number, but not resident gender or case difficulty. The multivariable model demonstrated that increasing chronologic procedure number, while a significant predictor without accounting for exposure, did not remain a significant predictor of practice-readiness for parathyroidectomy when accounting for resident-attending exposure. Bivariate analysis comparing resident and attending ratings showed no difference between the 2, but there were significant differences in autonomy and performance scores for both groups of raters. Trainees rated by attendings as independence ready completed a median of 7 parathyroidectomies and 5 thyroidectomies. Descriptive learning curves generated serve as a model of the multistate nature that residents undergo when moving from novice to proficiency.

CONCLUSIONS

Not surprisingly, the more operations residents perform with a single attending, the higher their ratings for performance and autonomy from that individual, with increased exposure allowing improved performance with less attending autonomy. By contrast, our data also show that repeated exposure between resident and attending may confound the use of procedural numbers alone when predicting resident ability in the operating room.

Transferencia de autonomía operatoria en apendicectomía abierta y laparoscópica: Un estudio de métodos mixtos desde la perspectiva de residentes y supervisores

Introducción. Existe información limitada sobre el desarrollo apropiado de curvas quirúrgicas de aprendizaje, con altos niveles de autonomía, en residentes de cirugía general en Colombia. El objetivo de este estudio fue caracterizar los niveles de autonomía, para la realización de apendicectomía laparoscópica o abierta en un programa de especialización, desde la perspectiva de médicos residentes y supervisores.  Métodos. Estudio de métodos mixtos que se realizó en dos fases. La primera fase incluyó la recolección prospectiva de la información de cada procedimiento (apendicectomía abierta o laparoscópica), realizado entre agosto de 2015 y diciembre de 2018, en la que participaron 29 médicos residentes. Cada residente evaluó su función (cirujano, ayudante), el nivel de supervisión y el nivel de autonomía intraoperatoria mediante la Escala de Zwisch (EZ). En la segunda fase (cualitativa), se realizaron entrevistas a un total de 15 cirujanos generales que supervisaron la práctica de los residentes con preguntas que buscaban explicar los hallazgos cuantitativos. Resultados. Se analizaron 1732 intervenciones: 629 (36 %) se realizaron por vía abierta y 1103 (63 %) por vía laparoscópica. El 81,4 % (n=1411) de los procedimientos fueron realizados en hospitales privados. La percepción global de autonomía reportada por los residentes de acuerdo con la Escala de Zwisch tuvo nivel A 28,9 % (n=500), nivel B 18,1 % (n=313), nivel C 30,4 % (n=526) y nivel D 22,7 % (n=393). El 35,2 % (n=388) de apendicectomías laparoscópicas y el 17,8% (n=112) por vía abierta fueron realizadas con un nivel A, mientras el 19,5 % (n=215) de apendicectomías laparoscópicas y el 28,2 % (n=178) por vía abierta fueron realizadas con un nivel D. La explicación dada de los hallazgos cuantitativos fue la frecuencia de apendicectomías abiertas en hospitales públicos, aspectos relacionados con la transferencia de autonomía hacia el residente y el aumento progresivo en el nivel de autonomía avanzada entre 2015-2018. Discusión. Se encontró un mayor nivel de autonomía en la realización de apendicectomía por vía abierta comparada con la vía laparoscópica, y los niveles de autonomía fueron mayores en hospitales públicos. La explicación a estos hallazgos estuvo relacionada con el contexto clínico y profesional de los residentes.

Operative Self-Confidence, Hesitation, and Ability Assessment of Surgical Trainees in Rural Kenya

BACKGROUND

The education of surgical trainees is ideally designed to produce surgeons with both confidence and competence. This involves the development of autonomy in the operating room. Factors associated with autonomy and entrustment have been studied in high-resource settings. In a resource-limited context, where autonomy is solely at the discretion of faculty, and there are fewer external constraints to restrict it, we hypothesized that assessment of a trainee's performance would be dependent upon reported confidence levels of both faculty and trainees in those trainees' abilities.

MATERIALS AND METHODS

At a teaching hospital in rural Kenya, operative experience surveys were administered to eleven general surgery trainees (PGY1-5) and six faculty paired dyads immediately following operative procedures in May 2016 to elicit self-reported assessments of confidence, hesitation, and ability as measured by the Zwisch Scale. We examined factors related to learning and used dyadic structural equation models to understand factors related to the assessment of ability.

RESULTS

There were 107 paired surveys among 136 trainees and 130 faculty evaluations. Faculty scrubbed into 76 (72%) cases. In comparison to trainees, faculty were more likely to give a higher average score for confidence (4.08 versus 3.90; P value: 0.005), a lower score for hesitation (2.67 versus 2.84; P value: 0.001), and a lower score for the ability to perform the operation independently (2.73 versus 3.02; P value: 0.01). Faculty and trainee perceptions of hesitation influenced their ability scores. Trainee hesitation (OR 12.1; 1.2-127.6, P = 0.04) predicted whether trainees reported experiencing learning.

CONCLUSIONS

Between trainees and faculty at a teaching program in rural Kenya, assessment scores of confidence, hesitation, and ability differ in value but remain fairly correlated. Hesitation is predictive of ability assessment, as well as self-reported learning opportunities. Focus upon identifying when trainees hesitate to proceed with a case may yield important educational opportunities.

Expectations versus reality: trainee participation on the robotic console in academic surgery

INTRODUCTION

Trainees underestimate the amount of operative autonomy they receive, whereas faculty overestimate; this has not been studied in robotics. We aimed to assess the perceptions and expectations of our general surgery trainees and faculty on robotic console participation in academic surgery.

METHODS

A survey was administered to general surgery robotic faculty and trainees eligible to sit at the console. Participants estimated the average percentage of trainee console participation time (CPT) per case for robotic cholecystectomies (CCY) and inguinal hernia repairs (IHR) from January to June 2019. Trainees were additionally asked what CPT they expected according to their training level (novice or senior). Expected CPTs were compared to actual CPTs extracted from robotic console logs during the same time frame.

RESULTS

Survey response rate was 80% for faculty (4 of 5) and 65% for trainees (15 of 23). Novices expected a higher CPT than they perceived in CCY (42.8% ± 14.8% vs 19.0% ± 17.2%, p = 0.03) and IHR (36.1% ± 17.6% vs. 10.7% ± 13.7%, p = 0.01), but in actuality, they did more CPT than perceived (by 34.9% in CCY, p < 0.01; 14% in IHR, p = 0.10). Senior trainees accurately perceived their CPT in IHR, but expected a higher CPT by 15.9% (p = 0.04). In CCY, seniors perceived a 23.8% higher CPT than in reality (p = 0.04). Faculty generally overperceived trainee CPT by 12.8-16.3% (p > 0.05). Compared to faculty, novices perceived lower CPTs in both CCY by 29.9% (p = 0.16) and IHR by 26.8% (p = 0.07), but seniors tended to agree with the faculty-perceived CPTs (p > 0.05).

CONCLUSION

Our robotic trainees expect to do more on the console than they perceive. Faculty think they allow their trainees more participation than in reality. Compared to faculty perception, novice trainees perceive a much lower level of trainee participation than senior trainees do. Expectation setting and standardizing learning curves are important for robotic surgery training.

Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate?

PURPOSE

Medical educators have developed no standard way to assess the operative performance of surgical residents. Most residency programs use end-of-rotation (EOR) evaluations for this purpose. Recently, some programs have implemented workplace-based "microassessment" tools that faculty use to immediately rate observed operative performance. The authors sought to determine (1) the degree to which EOR evaluations correspond to workplace-based microassessments and (2) which factors most influence EOR evaluations and directly observed workplace-based performance ratings and how the influence of those factors differs for each assessment method.

METHOD

In 2017, the authors retrospectively analyzed EOR evaluations and immediate postoperative assessment ratings of surgical trainees from a university-based training program from the 2015-2016 academic year. A Bayesian multivariate mixed model was constructed to predict operative performance ratings for each type of assessment.

RESULTS

Ratings of operative performance from EOR evaluations vs workplace-based microassessment ratings had a Pearson correlation of 0.55. Postgraduate year (PGY) of training was the most important predictor of operative performance ratings on EOR evaluations: Model estimates ranged from 0.62 to 1.75 and increased with PGY. For workplace-based assessment, operative autonomy rating was the most important predictor of operative performance (coefficient = 0.74).

CONCLUSIONS

EOR evaluations are perhaps most useful in assessing the ability of a resident to become a surgeon compared with other trainees in the same PGY of training. Workplace-based microassessments may be better for assessing a trainee's ability to perform specific procedures autonomously, thus perhaps providing more insight into a trainee's true readiness for operative independence.

Tracking operative autonomy and performance in otolaryngology training using smartphone technology: A single institution pilot study

BACKGROUND

In the era of duty hour restrictions, otolaryngology residents may not gain the operative experience necessary to function autonomously by the end of training. This study quantifies residents' autonomy during key indicator cases, defined by the Accreditation Council for Graduate Medical Education.

STUDY DESIGN

Prospective cohort study.

METHODS

Faculty and residents at a large academic institution were surveyed on the surgical autonomy trainees should achieve for otolaryngology key indicator surgeries at each training level. Residents and faculty used the mobile application "System for Improving and Measuring Procedural Learning" (SIMPL) between December 2017 and July 2018 to log trainees' operative autonomy during cases on a validated four-level Zwisch scale, from "show and tell" to "supervision only."

RESULTS

The study included 40 participants (23 residents and 17 attendings). The survey response rate was 83%. In surveys, residents overestimated the autonomy PGY5 residents should achieve for parotidectomy, rhinoplasty, thyroid/parathyroidectomy, and airway procedures compared with faculty (P < .05). Using SIMPL, 833 evaluations were logged of which 253 were paired evaluations for key indicator cases. Comparing survey predictions with actual cases logged in SIMPL, residents and faculty overestimated the autonomy achieved by senior trainees performing mastoidectomy (PGY5, P < .05) and ethmoidectomy (PGY4/5, P < .05); both felt that senior residents should operate with between "passive help" and "supervision only" whereas residents actually had "passive help." Residents overestimated their autonomy during rhinoplasty (PGY5, P = .017) and parotidectomy (PGY5, P = .007) while attendings accurately expected chief residents to have "passive help."

CONCLUSIONS

Resident surgical autonomy varies across otolaryngology procedures. Multicenter studies are needed to elucidate national trends.

LEVEL OF EVIDENCE

2.

Can a Structured, Video-Based Cadaver Curriculum Demonstrating Proficiency Enhance Resident Operative Autonomy?

INTRODUCTION

Autonomy is of foremost concern in the current era of surgical residency, and it is especially important to trainees when considering their surgical education. Factors impacting trainee independence include the restriction of clinical work hours and the development of advanced minimally invasive techniques such as robotics, which requires separate technical education outside of conventional surgical education. Moreover, when residents are left to learn fundamental exposures via their clinical experience alone, they run the risk of not being exposed to some fundamental skills based on case volume and type. The Department of Surgery at Emory University developed a cadaver-based simulation curriculum to standardize exposure to fundamental operative skills and enhance proficiency outside the operating room, with the larger aim of improving resident autonomy.

METHODS

Residents were assigned to small groups led by a chief resident with an even distribution of postgraduate year (PGY) levels. Each group participated in core surgical exposures and fundamental maneuvers on a cadaver over a 6-hour session. Residents were tested on skills according to their PGY level about 1 month after the course. Testing included recitation of the skill in an oral boards format, highlighting major steps, followed by performance of the skill. All steps were video-recorded with no resident identifiers. These were reviewed by 2 independent, blinded faculty examiners who assigned proficiency grades to each resident video.

RESULTS

Three hundred and thirty-three individual procedure evaluations were done over the 5-year period. Senior residents (PGY3-5) had 86% pass rate while junior residents (PGY1-2) had 70% pass rate. Overall, 21% of residents failed to achieve competence in their assigned skills. Junior residents were less likely to achieve competence compared to senior residents. Faculty graders had improved congruence in grading as the course progressed through the 5 years. The most recent 2 years had >80% congruence in faculty grading compared to less than 50% congruence in the first 2 years. 81% of attendings agreed this course positively influenced the granting of autonomy in the operating room.

CONCLUSIONS

A cadaveric skills course focused on fundamental maneuvers with objective confirmation of achieving competency is a viable adjunct to clinical operative experience. Video-recorded evaluation, of these fundamental skills improved both resident and attending confidence in trainee operative skill.

The Role of Manual Dexterity and Cognitive Functioning in Enhancing Resident Operative Autonomy

INTRODUCTION

Autonomy, both operative and nonoperative, is one of the most critical aspects of successful surgical training. Both surgeon and resident share the responsibility of achieving this goal. We hypothesize that operative autonomy is distinct and depends, for the most part, on the resident's manual dexterity, knowledge of, and preparation for the procedure.

METHODS

Over a period of 4 academic years, between July 2014 and June 2018, a total of 958 Global Rating Scale of Operative Performance evaluations were completed by 32 general and subspecialty faculty surgeons for 35 residents. Elective procedures were evaluated, including 165 (17.2%) by postgraduate year (PGY)1 residents, 253 (26.4%) by PGY2, 199 (20.8%) by PGY3, 147 (15.3%) by PGY4, and 194 (20.3%) by PGY5. The procedures evaluated were: 261 (27.2%) hernia repairs; 178 (18.6%) cholecystectomies; 102 (10.6%) colorectal and anal procedures; 73 (7.6%) vascular procedures; 56 (5.8%) thyroid and parathyroidectomies; 39 (4.1%) foregut (esophagus and stomach) procedures; 38 (4%) skin, soft tissue, and breast; 92 (10%) hepatopancreatic; 20 (2.1%) pediatric procedures; and 99 (10.3%) other procedures including amputations, cardiothoracic, and solid organs procedures. Each resident was scored from 1 to 5 (1 lowest, 5 highest) in each of the following categories of Global Rating Scale of Operative Performance: respect for tissue (RT), time and motion (T&M), instrument handling (IH), knowledge of the instrument (KI), flow of operation (FO) and resident's preparation for the procedure (RP). Resident operative autonomy (ROA) was assessed using the Zwisch scale, a 4-point scale describing faculty supervision behaviors associated with different degrees of resident autonomy (1: Show and Tell, 2: Active Help, 3: Passive Help, and 4: Supervision Only).

RESULTS

Correlation and ordinal regression analyses were conducted to examine the relationship between ROA and manual dexterity (RT, T&M, IH, and FO), and cognitive functioning (knowledge of instruments and resident preparation). Results indicated a positive correlation between ROA and RT (r = 0.528, p < 0.001), T&M (r = 0.630, p < 0.001), IH (r = 0.597, p < 0.001), KI (r = 0.490, p < 0.001), FO (r = 0.637, p < 0.001), and RP (r = 0.525, p < 0.001). Additionally, there was a weak inverse correlation between ROA and the number of years the surgeon had been in practice (r = -0.127, p = 0.001). The significant predictors of resident autonomy found by the ordinal logistic regression include time and motion (p < 0.001), flow of operation (p < 0.001), and resident's preparation for the procedure (p < 0.001).

CONCLUSIONS

Resident operative autonomy is a product of shared responsibility between the faculty and resident. However, residents' inherent and/or acquired skills and preparation for the operative procedures play a critical role. Residents should be advised to use available resources such as simulation to augment their skills preoperatively and to enhance their autonomy in the operating room.

Improving the Intraoperative Educational Experience: Understanding the Role of Confidence in the Resident-Attending Relationship

OBJECTIVE

With recent changes to graduate medical education, the balance between resident autonomy and need for supervision impacts the educational and training experience of residents. The objective of this study was to understand the relationship between the confidence of attendings and residents and their different perspectives of perceived educational experience and autonomy in the operating room (OR). We hypothesized that the attending's confidence in the resident would be an important factor in improving the educational experience and resident's autonomy in the OR.

DESIGN

Self-reported confidence-rating and operative experience surveys were administered to teams of post-graduate year (PGY 1) through PGY 5 surgical residents and attendings in two temporal sets (Early: Sept-Dec 2015, n = 20; Late: Jan-Apr 2016, n = 22). A second "end-of-year" survey was distributed to residents (n = 9, 37.5% response) and attendings (n = 10, 35% response) asking questions regarding their educational experience and operative experience during the past year.

SETTING

Large rural teaching hospital.

PARTICIPANTS

Nineteen general surgery residents (PGY 1 - 5) and 14 general surgery attendings.

RESULTS

Resident perception of confidence differs from junior to senior residents, and that there was discordance between resident's confidence and skill as perceived by attendings, particularly in senior residents. Results also showed that attending's confidence in residents was positively correlated with attending's perceived educational experience in the OR. Residents and attendings both indicated attending's confidence in residents as an important factor in increasing resident autonomy in the OR, thus the attending's confidence in residents could have a positive impact on resident autonomy and educational experience in the OR.

CONCLUSIONS

We have demonstrated a relationship between self-confidence for residents and improved confidence from attendings in residents' capabilities. Based on these findings, we would propose identifying methods to expand resident's awareness of surgical situations and develop attending's confidence in residents.

Assessments of Otolaryngology Resident Operative Experiences Using Mobile Technology: A Pilot Study

Objectives

Surgical education has shifted from the Halstedian model of “see one, do one, teach one” to a competency-based model of training. Otolaryngology residency programs can benefit from a fast and simple system to assess residents’ surgical skills. In this quality initiative, we hypothesize that a novel smartphone application called System for Improving and Measuring Procedural Learning (SIMPL) could be applied in an otolaryngology residency to facilitate the assessment of resident operative experiences.

Methods

The Plan Do Study Act method of quality improvement was used. After researching tools of surgical assessment and trialing SIMPL in a resident-attending pair, we piloted SIMPL across an otolaryngology residency program. Faculty and residents were trained to use SIMPL to rate resident operative performance and autonomy with a previously validated Zwisch Scale.

Results

Residents (n = 23) and faculty (n = 17) were trained to use SIMPL using a standardized curriculum. A total of 833 assessments were completed from December 1, 2017, to June 30, 2018. Attendings completed a median 20 assessments, and residents completed a median 14 self-assessments. All evaluations were resident initiated, and attendings had a 78% median response rate. Evaluations took residents a median 22 seconds to complete; 126 unique procedures were logged, representing all 14 key indicator cases for otolaryngology.

Discussion

This is the first residency-wide application of a mobile platform to track the operative experiences of otolaryngology residents.

Implications for Practice

We adapted and implemented a novel assessment tool in a large otolaryngology program. Future multicenter studies will benchmark resident operative experiences nationwide.

Identifying and Addressing High Priority Issues in General Surgery Training and Education

BACKGROUND

Complex problems are often easier to address when multiple entities collaborate. The Procedural Learning and Safety Collaborative (PLSC) was established to address complex problems in general surgery residency training by connectively engaging multiple residency programs in addressing progressive research questions.

STUDY DESIGN

Recently, PLSC members held a national symposium which included leadership from several leading surgical societies to come to a consensus on what are the most critical issues in general surgery education.

RESULTS

This paper describes the process used and the end result of this process. This paper describes the process used and the end result of this process.

Progressive Autonomy in the Era of Increased Supervision: AOA Critical Issues

The observation of decreased resident autonomy, ultimately influencing the readiness of a new graduate to practice, has been supported with a number of recent surveys. This perceived lack of autonomy is felt to be due, in part, to many reasons, including duty-hour regulations, increased supervision requirements, patient safety measures, concern for complication rates, and other performance measures. Pressure on faculty members to have increased clinical productivity may not allow for more resident autonomy.Increased clinical exposure to improve resident independence may come from several suggested areas. First, restructuring the residency program to allow for more clinical time may be one way to improve education. Second, increased use of surgical simulation will allow for more experience to develop technical skills within a controlled environment. Surgical simulators can be used to acquire new skills and also as a means of assessing competence. Third, competency-based education (CBE) has been offered as a way to improve resident education. At its core, CBE offers criterion-based assessments for residents and faculty that allow for more frequent feedback.