Opportunities to Increase Workplace Feedback: A Multi-institutional Examination

OBJECTIVE

While feedback is an essential component of resident education, there are few large-scale studies examining when and under what conditions formative feedback is provided. Workplace-based assessment systems offer an opportunity to identify factors influencing when faculty provides feedback to trainees. Influential factors affecting feedback may provide targets for increasing and improving feedback in resident education.

DESIGN

Data on whether dictated feedback was provided were obtained from the Society for Improving Medical Professional Learning (SIMPL) mobile application. We used generalized linear mixed effects models to identify the degree to which faculty members, procedures, surgical case characteristics, and trainee performance were associated with whether narrative feedback was provided using SIMPL.

SETTING

This study was conducted using data from members of the SIMPL collaborative.

PARTICIPANTS

67,434 evaluations from 70 general surgery programs were included from 2015 to 2021. Of these, 25,355 evaluations included dictated feedback.

RESULTS

Approximately 61% of the variation in whether dictated feedback was provided was attributable to the individual faculty member. Compared to residents who achieved autonomy ratings of "Active Help," residents who achieved ratings of "Supervision Only" (odds ratio (OR) = 0.80, 95% confidence interval (CI) = 0.72, 0.88) had a lower likelihood of receiving dictated feedback. Residents who achieved ratings of "Intermediate" (OR = 0.81, CI = 0.74, 0.89), "Practice-Ready" (OR = 0.50, CI = 0.45, 0.57), or "Exceptional (OR = 0.64, CI = 0.54, 0.76) showed a lower likelihood of receiving dictated feedback compared to those rated as "Inexperienced." Cases rated as "High" in terms of complexity were associated with an increased likelihood of having dictation (OR = 1.35, CI = 1.26, 1.44).

CONCLUSIONS

The largest contributing factor for whether dictated feedback is included in a SIMPL evaluation are factors specific to the attending surgeon. Resident performance, resident autonomy, and case complexity had only modest associations with feedback decisions. Efforts to improve the amount of formative feedback for trainees should be directed towards reducing the variation in which attending surgeons elect to provide feedback.

Understanding General Surgery Applicant Expectations and Perceptions in the Virtual Interview Process

OBJECTIVE

The COVID-19 pandemic has played a lasting role on residency recruitment through the virtual interview process. The objective of this study was to 1) examine general surgery applicants' priorities and perceptions following pre-interview virtual open houses and 2) to assess applicant expectations and efficacy of the virtual interview day process.

DESIGN/SETTING/PARTICIPANTS

This study utilized two voluntary and anonymous cross-sectional surveys administered via email to evaluate the virtual interview process of a general surgery residency program. The first was administered to registrants following completion of three open houses of various topics. The second was administered following each interview day. The post-open house survey had 78 respondents, two excluded for no open house attendance. The post-interview survey was completed by 44 applicants (62.9% response rate).

RESULTS

Majority of respondents reported that attending virtual open houses made them want to apply to (90.9%) and improved their perception of the program (94.7%). Applicants who felt a sense of obligation to attend open houses (68.4%) were significantly more likely to feel that they contributed to the stress and time commitment of applications (81.8% vs 18.2%, p=0.028). Interview expectations were identified in recurrent themes: 1. Clear organization with breaks, 2. Interactive resident sessions, 3. Meetings with program leadership, 4. Additional information unavailable on other resources. The pre-interview social and interview day improved 90.2% of the applicants' perceptions of the program. The interview significantly improved applicants' ability to assess nearly all aspects of the program, notably resident camaraderie and culture (30.8% vs 97.4%, p=0.01) and strengths and weaknesses (30.8% vs 92.3%, p=0.04).

CONCLUSIONS

While virtual open houses can improve applicants' perceptions and desire to apply to a program, the associated stress and obligation should be considered. Virtual interviews should provide information unavailable using other resources and provide avenues for conveying the resident culture and camaraderie.

Comparison of preoperative, intraoperative, and follow-up functional luminal imaging probe measurements in patients undergoing myotomy for achalasia

BACKGROUND AND AIMS

The functional luminal imaging probe (FLIP) is a novel catheter-based device that measures esophagogastric junction (EGJ) distensibility index (DI) in real time. Previous studies have demonstrated DI to be a predictor of post-treatment clinical outcomes in patients with achalasia. We sought to evaluate EGJ DI in patients with achalasia before, during, and after peroral endoscopic myotomy (POEM) and laparoscopic Heller myotomy (LHM) and to assess the correlation of DI with postoperative outcomes.

METHODS

DI (defined as the minimum cross-sectional area at the EGJ divided by distensive pressure) was measured at 4 time points in patients undergoing surgical myotomy for achalasia: (1) during outpatient preoperative endoscopy (preoperative DI), (2) at the start of each operation after the induction of anesthesia (induction DI), (3) at the conclusion of each operation (postmyotomy DI), and (4) at routine follow-up endoscopy 12 months postoperatively (follow-up DI). Routine Eckardt symptom score, endoscopy, timed barium esophagram, and pH study were obtained 12 months postoperatively.

RESULTS

Forty-six patients (35 POEM, 11 LHM) underwent FLIP measurements at all 4 time points. Preoperative and induction mean DI were similar for both groups (POEM, 1 vs .9 mm²/mm Hg; LHM, 1.7 vs 1.5 mm²/mm Hg). POEM resulted in a significant increase in DI (induction .9 vs postmyotomy 7 mm²/mm Hg, P < .001). There was a subsequent decrease in DI in the follow-up period (postmyotomy 7 vs follow-up 4.8 mm²/mm Hg, P < .01), but DI at follow-up was still significantly improved from preoperative values (P < .001). For LHM patients, DI also increased as a result of surgery (induction 1.5 vs postmyotomy 5.9 mm²/mm Hg, P < .001); however, the increase was smaller than in POEM patients (DI increase 4.4 vs 6.2 mm²/mm Hg, P < .05). After LHM, DI also decreased in the follow-up period, but this change was not statistically significant (5.9 vs 4.4 mm²/mm Hg, P = .29). LHM patients with erosive esophagitis on follow-up endoscopy had a significantly higher postmyotomy DI compared with those without esophagitis (9.3 vs 4.8 mm²/mm Hg, P < .05).

CONCLUSIONS

EGJ DI improved dramatically as a result of both POEM and LHM, with POEM resulting in a larger increase. Mean DI decreased at intermediate follow-up but remained well above previously established thresholds for symptom recurrence. DI at the conclusion of LHM was predictive of erosive esophagitis in the postoperative period, which supports the potential use of FLIP for calibration of partial fundoplication construction during LHM.

Addressing the gap in laparoscopic common bile duct exploration training for rural surgeons: imparting procedural ability is not enough

BACKGROUND

Laparoscopic common bile duct exploration (LCBDE) is an underutilized therapy for choledocholithiasis. The driving factors of this practice gap are poorly defined. We sought to evaluate the attitudes and practice patterns of surgeons who underwent training courses using an LCBDE simulator.

METHODS

Surgeons completed a half-day simulator-based LCBDE curriculum at national courses, including the American College of Surgeons Advanced Skills Training for Rural Surgeons and the Society of American Gastrointestinal and Endoscopic Surgeons annual meeting. Attitudes were assessed with Likert surveys immediately before and after curriculum completion. Follow-up surveys were distributed electronically.

RESULTS

159 surgeons completed training during six courses. Surgeon attitudes regarding the overall superiority of LCBDE vs. ERCP shifted towards favoring LCBDE after course participation (4.0 vs 3.3; Likert scale 1-5, p < 0.001). 44% of surgeons completed follow-up surveys at a mean of 3 years post-course. Surgeons remained confident in their ability to perform LCBDE, with only 14% rating their skill as a significant barrier to practice, as compared with 43% prior to course participation (p < 0.01). However, only 28% of surgeons saw an increase in LCBDE volume. Deficiencies in operating room (OR) staff knowledge and instrument availability were the most significant barriers to post-course practice implementation and were inversely correlated with LCBDE case volume (ρ = - 0.44 and - 0.47, both p < 0.01). Surgeons for whom OR staff knowledge of LCBDE was not a significant barrier performed nearly 4 times more LCBDE than those who rated staff knowledge as a moderate, strong, or complete barrier.

CONCLUSIONS

Surgeons trained at an LCBDE course retained long-term confidence in their procedural ability. Practice implementation was hindered by deficiencies in OR staff knowledge and instrument availability. Surgeons with knowledgeable operating room staff performed significantly more LCBDEs than those with less capable assistance. These barriers should be addressed in future curricula to improve procedural adoption.

Entrustable Professional Activity-Based Summative Performance Assessment in the Surgery Clerkship

OBJECTIVE

The objectives of this study were to 1) assess the performance Entrustable Professional Activities (EPAs) when integrated into the summative assessment of third-year medical students on the surgery clerkship and 2) to compare EPAs to traditional clinical performance assessment tools.

DESIGN

EPA assessments were collected prospectively from a minimum of 4 evaluators at the completion of each surgical clerkship rotation from November 2019 to June 2019. Overall EPA-based clinical performance scores were calculated as the sum of the mean EPA score from each evaluator. A rating of overall clinical performance called the clinical performance appraisal (CPA) was also collected. EPA ratings were compared to the CPA score, National Board of Medical Examiners exam score, objective structured clinical exam scores, and final clerkship grade.

SETTING

Northwestern Memorial Hospital, a tertiary care teaching institution in Chicago, IL.

RESULTS

Overall, 446 evaluations (111 students) were included in the analysis. The aggregate EPA scores ranged from 11.6-24.0 (mean 19.9 ± 2.0), and the CPA scores ranged from 4.4-9.0 (mean 7.6 ± 0.7). The variance among learners in EPA scores was significantly higher than CPA scores (p < 0.001). The aggregate EPA scores correlated well with CPA scores (Spearman's rho 0.803) but had lesser, positive correlations with the objective structured clinical exam (rho 0.153) and National Board of Medical Examiners (rho 0.265) scores. When all EPA scores were included in ordinal logistic regression, only EPA 6, oral presentation of patients, was independently associated with students' final grades (OR: 10.05, 95%CI 1.41-71.80; p = 0.02).

CONCLUSION

Integration of EPAs for use in clinical performance assessment of medical students is feasible within a surgery clerkship. Compared to a global clinical performance assessment, EPA-based assessment provided better discrimination of clinical performance among learners. Use of EPAs may better identify advanced learners and those that need additional time.

Improving trauma tertiary survey performance and missed injury identification using an education-based quality improvement initiative

BACKGROUND

Performance of a trauma tertiary survey (TTS) reduces rates of missed injuries, but performance has been inconsistent at trauma centers. The objectives of this study were to assess whether quality improvement (QI) efforts would increase the frequency of TTS documentation and determine if TTS documentation would increase identification of traumatic injuries. Our hypothesis was that QI efforts would improve documentation of the TTS.

METHODS

Before-and-after analysis of QI interventions at a level 1 trauma center was performed. The interventions included an electronic template for TTS documentation, customized educational sessions, and emphasis from trauma leadership on TTS performance. The primary outcome was documentation of the TTS. Detection of additional injuries based on tertiary evaluation was a secondary outcome. Associations between outcomes and categorical patient and encounter characteristics were assessed using χ2 tests.

RESULTS

Overall, 592 trauma encounters were reviewed (296 preimplementation and 296 postimplementation). Trauma tertiary survey documentation was significantly higher after implementation of the interventions (30.1% preimplementation vs. 85.1% postimplementation, p < 0.001). Preimplementation documentation of the TTS was less likely earlier in the academic year (14.3% first academic quarter vs. 46.5% last academic quarter, p < 0.001), but this temporal pattern was no longer evident postimplementation (88.5% first academic quarter vs. 77.9% last academic quarter, p = 0.126). Patients were more likely to have a missed traumatic injury diagnosed on TTS postimplementation (1.7% in preimplementation vs. 5.7% postimplementation, p = 0.009).

CONCLUSION

Documentation of the TTS and missed injury detection rates were significantly increased following implementation of a bundle of QI interventions. The association between time of year and documentation of the TTS was also attenuated, likely through reduction of the resident learning curve. Targeted efforts to improve TTS performance may improve outcomes for trauma patients at teaching hospitals.

LEVEL OF EVIDENCE

Care management, Level IV.

Outcomes of 100 Patients More Than 4 Years After POEM for Achalasia

OBJECTIVE

We aim to describe the long-term follow-up data from our institution's POEM experience.

SUMMARY BACKGROUND DATA

Per-oral endoscopic myotomy (POEM) is a well-established endoscopic therapy for achalasia with excellent short-term efficacy, but long-term outcomes data are limited.

METHODS

Patients older than 4 years removed from POEM for treatment of achalasia were studied. Clinical success was defined as an Eckardt Symptom (ES) score ≤3 and freedom from reintervention for achalasia. Patients underwent esophagogastroduodenoscopy (EGD), high-resolution manometry, impedance planimetry, and timed barium esophagram (TBE) preoperatively and at least 4 years postoperatively. Objective gastroesophageal reflux disease (GERD) was defined LA Grade B or worse esophagitis on EGD.

RESULTS

One hundred and nineteen consecutive patients were included. Five patients died or had catastrophic events unrelated to achalasia or POEM. One hundred of the remaining patients (88%, 100/114) had long-term data available. Clinical follow-up for all patients was greater than 4 years postoperatively and the mean was 55 months. Mean current ES was significantly improved from preop (n = 100, 1 ± 1 vs 7 ± 2, P < 0.001). Overall clinical success was 88% and 92%. Five patients had a current ES >3 and 4 patients required procedural reintervention on the lower esophageal sphincter. Reinterventions were successful in 75% of patients (3/4), with current ES ≤3. The rate of objective GERD was 33% (15/45). Esophageal physiology was improved with a decrease in median integrated relaxation pressure (11 ± 4 vs 33 ± 15 mm Hg, P < 0.001), a decrease in median TBE column height (3 ± 3 vs 13 ± 8 cm, P < 0.001), and an increase in median distensibility index (5.1 ± 2 vs 1.1 ± 1 mm2/mm Hg, P < 0.001).

CONCLUSIONS

POEM provides durable symptom relief and improvement in physiologic esophagogastric junction relaxation parameters over 4.5 years postoperatively. Reinterventions are rare and effective.

Intraoperative use of FLIP is associated with clinical success following POEM for achalasia

BACKGROUND

Esophagogastric junction distensibility index (DI), measured using the functional luminal imaging probe (FLIP), correlates with symptomatic outcomes after interventions for achalasia. The objective of this study was to determine if the intraoperative measurement of DI using FLIP was associated with improved clinical outcomes following per-oral endoscopic myotomy (POEM) for achalasia when compared with procedures in which FLIP was not utilized.

METHODS

Patients undergoing POEM from 2012 to 2017 at a single institution by a single surgeon were studied. Use of FLIP during this time period was based on catheter and technician availability, resulting in two patient cohorts. In patients in whom FLIP was used, operative video recordings were reviewed to determine when DI measurements led to the performance of additional myotomy. Postoperative Eckardt symptom scores (ES) at 12 months and postoperative physiologic studies were compared between patients with and without intraoperative FLIP. Associations were assessed using Mann-Whitney U and Chi-square tests.

RESULTS

143 patients were included in the analysis (61 with intraoperative FLIP and 82 without FLIP). Video recordings were available for 85% of the FLIP cohort. Review of these operative recordings revealed that 65% of patients who underwent FLIP had additional myotomy performed following the initial postmyotomy FLIP measurement. At 12 months after POEM, the FLIP cohort had significantly more clinical successes (defined as ES ≤ 3) than patients in whom FLIP was not used (93% vs. 81%, p < 0.05).

CONCLUSIONS

Use of intraoperative FLIP during POEM resulted in the surgeon performing additional myotomy in over half of cases and was associated with improved clinical outcomes. This study demonstrates the potential for a FLIP-tailored myotomy to improve outcomes in patients undergoing surgical myotomy for achalasia.

Intraoperative FLIP distensibility during POEM varies according to achalasia subtype

BACKGROUND

The functional luminal imaging probe (FLIP) can be used to measure the esophagogastric junction distensibility index (DI) during myotomy for achalasia and increased DI has been shown to predict superior clinical outcomes. The objective of this study was to determine if the intraoperative DI and the changes produced by per oral endoscopic myotomy (POEM) differed between achalasia subtypes.

METHODS

FLIP measurements were performed during POEM for achalasia at a single institution. DI (defined as the minimum cross-sectional area (CSA) at the EGJ divided by distensive pressure) was measured at three time points: after induction of anesthesia, after submucosal tunneling, and after myotomy. Measurements were reported at the 40 mL fill volume for the 8 cm FLIP (EF-325) and at the 60 mL fill volume for the 16 cm FLIP (EF-322). Measurements were compared using chi-square and Kruskal-Wallis tests.

RESULTS

142 patients had intraoperative FLIP performed during POEM for achalasia between 2012 and 2019 (30 type I, 68 type II, 27 type III, and 17 variant). Patients with type I achalasia had a significantly higher induction DI (median 1.7 mm²/mmHg) than type II (0.8 mm²/mmHg), type III (0.9 mm²/mmHg), and variants (1.1 mm²/mmHg; p < 0.001). These differences persisted after submucosal tunneling and final DI after myotomy was also significantly higher in type I patients (median 8.0 mm²/mmHg) compared to type II (5.8 mm²/mmHg), type III (3.9 mm²/mmHg), and variants (5.4 mm²/mmHg; p < 0.001). Achalasia subtypes were found to have similar CSA at all time points, whereas pressure differed with type I having the lowest pressure and type III the highest.

CONCLUSION

The DI at each operative step during POEM was found to differ significantly between achalasia subtypes. These differences in DI were due to pressure, as CSA was similar between subtypes. Achalasia subtype should be accounted for when using FLIP as an intraoperative calibration tool and in future studies examining the relationship between DI and clinical outcomes.

POEM for Esophageal Achalasia

Over the past decade, per-oral endoscopic myotomy has been shown to be a durable minimally invasive approach to the treatment of esophageal achalasia. Patients with suspected achalasia should undergo upper endoscopy, timed barium esophagram, and high-resolution manometry to confirm the diagnosis. The procedure includes several key steps including mucosotomy, submucosal tunneling, selective myotomy of the circular muscle layer, and mucosotomy closure. Specialized endoscopic dissection tools are used during the procedure to access the submucosal space. Common procedural challenges include minor bleeding and capnoperitoneum.

A comprehensive national survey on thoughts of leaving residency, alternative career paths, and reasons for staying in general surgery training

BACKGROUND

General surgery residencies continue to experience high levels of attrition.

METHODS

Survey of general surgery residents administered with the 2018 American Board of Surgery In-Training Examination. Outcomes were consideration of leaving residency, potential alternative career paths, and reasons for staying in residency.

RESULTS

Among 7,409 residents, 930 (12.6%) reported considering leaving residency over the last year. Residents were more likely to consider other general surgery programs (46.2%) if PGY 2/3 (OR: 1.93, 95%CI 1.34-2.77) or reporting frequent duty hour violations (OR: 1.58, 95%CI 1.12-2.24). Consideration of other specialties (47.0%) was more likely if dissatisfied with being a surgeon (OR 2.86, 95%CI 1.92-4.26). Residents were more likely to consider leaving medicine (49.7%) if female (OR: 1.54, 95%CI 1.16-2.06) or dissatisfied with a surgical career (OR: 2.81, 95%CI 1.85-4.27). Common reasons for remaining in residency included a sense of too much invested to leave (65.3%) and career satisfaction (55.5%).

CONCLUSION

Profiles of trainees considering leaving residency exist based on factors associated with alternative careers. This may be a target for future interventions to reduce attrition.

Obtaining Meaningful Assessment in Thoracic Surgery Education

Training in thoracic surgery has evolved immensely over the past decade due to the advent of integrated programs, technological innovations, and regulations on resident duty hours, decreasing the time trainees have to learn. These changes have made assessment of thoracic surgical trainees even more important. Shifts in medical education have increasingly emphasized competency, which has led to novel competency-based assessment tools for clinical and operative assessment. These novel tools take advantage of simulation and modern technology to provide more frequent and comprehensive assessment of the surgical trainee to ensure competence.