Evaluating Surgical Coaching: A Mixed Methods Approach Reveals More Than Surveys Alone

OBJECTIVE

Traditionally, surgical educators have relied upon participant survey data for the evaluation of educational interventions. However, the ability of such subjective data to completely evaluate an intervention is limited. Our objective was to compare resident and attending surgeons' self-assessments of coaching sessions from surveys with independent observations from analysis of intraoperative and postoperative coaching transcripts.

DESIGN

Senior residents were video-recorded operating. Each was then coached by the operative attending in a 1:1 video review session. Teaching points made in the operating room (OR) and in post-OR coaching sessions were coded by independent observers using dialogue analysis then compared using t-tests. Participants were surveyed regarding the degree of teaching dedicated to specific topics and perceived changes in teaching level, resident comfort, educational assessments, and feedback provision between the OR and the post-OR coaching sessions.

SETTING

A single, large, urban, tertiary-care academic institution.

PARTICIPANTS

Ten PGY4 to 5 general surgery residents and 10 attending surgeons.

RESULTS

Although the reported experiences of teaching and coaching sessions by residents and faculty were similar (Pearson correlation coefficient = 0.88), these differed significantly from independent observations. Observers found that residents initiated a greater proportion of teaching points and had more educational needs assessments during coaching, compared to the OR. However, neither residents nor attendings reported a change between the 2 environments with regard to needs assessments nor comfort with asking questions or making suggestions. The only metric on which residents, attendings, and observers agreed was the provision of feedback.

CONCLUSIONS

Participants' perspectives, although considered highly reliable by traditional metrics, rarely aligned with analysis of the associated transcripts from independent observers. Independent observation showed a distinct benefit of coaching in terms of frequency and type of learning points. These findings highlight the importance of seeking different perspectives, data sources, and methodologies when evaluating clinical education interventions. Surgical education can benefit from increased use of dialogue analyses performed by independent observers, which may represent a viewpoint distinct from that obtained by survey methodology.

Factors Associated With Surgery Clerkship Performance and Subsequent USMLE Step Scores

PURPOSE

We conducted an in-depth empirical investigation to achieve a better understanding of the surgery clerkship from multiple perspectives, including the influence of clerkship sequence on performance, the relationship between self-logged work hours and performance, as well as the association between surgery clerkship performance with subsequent USMLE Step exams' scores.

METHOD

The study cohort consisted of medical students graduating between 2015 and 2018 (n = 687). The primary measures of interest were clerkship sequence (internal medicine clerkship before or after surgery clerkship), self-logged work hours during surgery clerkship, surgery NBME subject exam score, surgery clerkship overall grade, and Step 1, Step 2 CK, and Step 3 exam scores. We reported the descriptive statistics and conducted correlation analysis, stepwise linear regression analysis, and variable selection analysis of logistic regression to answer the research questions.

RESULTS

Students who completed internal medicine clerkship prior to surgery clerkship had better performance on surgery subject exam. The subject exam score explained an additional 28% of the variance of the Step 2 CK score, and the clerkship overall score accounted for an additional 24% of the variance after the MCAT scores and undergraduate GPA were controlled.

CONCLUSION

Our finding suggests that the clerkship sequence does matter when it comes to performance on the surgery NBME subject exam. Performance on the surgery subject exam is predictive of subsequent performance on future USMLE Step exams.

The Economics of Academic Advancement Within Surgery

BACKGROUND

The success of an academic surgeon's career is often viewed as directly related to academic appointment; therefore, the sequence of promotion is a demanding, rigorous process. This paper seeks to define the financial implication of academic advancement across different surgical subspecialties.

STUDY DESIGN

Data was collected from the Association of American Medical College's 2015 report of average annual salaries. Assumptions included 30 years of practice, 5 years as assistant professor, and 10 years as associate professor before advancement. The base formula used was: (average annual salary) × (years of practice [30 years - fellowship/research years]) + ($50,000 × years of fellowship/research) = total adjusted lifetime salary income.

RESULTS

There was a significant increase in lifetime salary income with advancement from assistant to associate professor in all subspecialties when compared to an increase from associate to full professor. The greatest increase in income from assistant to associate professor was seen in transplant and cardiothoracic surgery (35% and 27%, respectively). Trauma surgery and surgical oncology had the smallest increases of 8% and 9%, respectively. With advancement to full professor, the increase in lifetime salary income was significantly less across all subspecialties, ranging from 1% in plastic surgery to 8% in pediatric surgery.

CONCLUSION

When analyzing the economics of career advancement in academic surgery, there is a substantial financial benefit in lifetime income to becoming an associate professor in all fields; whereas, advancement to full professor is associated with a drastically reduced economic benefit.

Does a new surgical residency program increase operating room times?

BACKGROUND

Our country faces a shortage of surgeons; hence, we may anticipate the development of new surgery residencies. Therefore, the question of the effect of a new program on operating room times (ORT) is important. Our primary aim was to compare ORT of 3 common procedures done by attendings alone vs ORT of cases with residents.

METHODS

We queried records of 1458 patients from the JFK Medical Center database for laparoscopic cholecystectomy, open inguinal hernia repair, and laparoscopic appendectomy from July 2010 to July 2012. We divided the sample into 2 groups: "attending alone" (2010-2011) and "with residents" (2011-2012). The ORT was calculated by "Cut time" and "Close time," as recorded in the OR. ORT for both groups was calculated using the unpaired t test.

RESULTS

Of the total number of patients, 778 underwent laparoscopic cholecystectomy, 407 underwent open inguinal hernia repair, and 273 underwent laparoscopic appendectomy; of these, 620, 315, and 211 procedures, respectively, were done by the attending alone and 158, 92, and 62, respectively, were done with residents. Differences in ORT for the 3 types of surgery were statistically significant (p < 0.001). There was no statistical significance when comparing the first half with the second half of the academic year for residents' ORT.

CONCLUSIONS

Resident involvement increases ORT. Cost analysis considering OR time and anesthesia time vs federal funding for Graduate Medical Education is complicated. The benefit of new programs in diminishing the shortage of surgeons cannot be underestimated.