National Profile of the ACGME Milestones 1.0 and 2.0 within General Surgery: A Seven-Year National Study from 2014 to 2021

PURPOSE

The Accreditation Council for Graduate Medical Education (ACGME) introduced General Surgery Milestones 1.0 in 2014 and Milestones 2.0 in 2020 as steps toward competency-based training. Analysis will inform residency programs on curriculum development, assessment, feedback, and faculty development. This study describes the distributions and trends for Milestones 1.0 and 2.0 ratings and proportion of residents not achieving the level 4.0 graduation target.

METHODS

A deidentified dataset of milestone ratings for all ACGME-accredited General Surgery residency programs in the United States was used. Medians and interquartile ranges (IQR) were reported for milestone ratings at each PGY level. Percentages of PGY-5s receiving final year ratings of less than 4.0 were calculated. Wilcoxon rank sum tests were used to compare 1.0 and 2.0 median ratings. Kruskal-Wallis and Bonferroni post-hoc tests were used to compare median ratings across time periods and PGY levels. Chi-squared tests were used to compare the proportion of level 4.0 nonachievement under both systems.

RESULTS

Milestones 1.0 data consisted of 13,866 residents and Milestones 2.0 data consisted of 7,633 residents. For 1.0 and 2.0, all competency domain median ratings were higher for subsequent years of training. Milestones 2.0 had significantly higher median ratings at all PGY levels for all competency domains except Medical Knowledge. Percentages of PGY-5 residents not achieving the graduation target in Milestones 1.0 ranged from 27% to 42% and in 2.0 from 5% to 13%. For Milestones 1.0, all subcompetencies showed an increased number of residents achieving the graduation target from 2014 to 2019.

CONCLUSIONS

This study of General Surgery Milestones 1.0 and 2.0 data uncovered significant increases in average ratings and significantly fewer residents not achieving the graduation target under the 2.0 system. We hypothesize that these findings may be related more to rating bias given the change in rating scales, rather than a true increase in resident ability.

Hiatal hernia reporting: time to remove subjectivity?

INTRODUCTION

The size of a hiatal hernia (HH) is a key determinant of the approach for surgical repair. However, endoscopists will often utilize subjective terms, such as "small," "medium," and "large," without any standardized objective correlations. The aim of this study was to identify HHs described using objective axial length measurements versus subjective size allocations and compare them to their corresponding manometry and barium swallow studies.

METHODS AND PROCEDURES

Retrospective chart reviews were conducted on 93 patients diagnosed endoscopically with HHs between 2017 and 2021 at Newton-Wellesley Hospital. Information was collected regarding their HH subjective size assessment, axial length measurement (cm), manometry results, and barium swallow readings. Linear regression models were used to analyze the correlation between the objective endoscopic axial length measurements and manometry measurements. Ordered logistic regression models were used to correlate the ordinal endoscopic and barium swallow subjective size allocations with the continuous axial length measurements and manometry measurements.

RESULTS

Of the 93 endoscopy reports, 42 included a subjective size estimate, 38 had axial length measurement, and 12 gave both. Of the 34 barium swallow reads, only one gave an objective HH size measurement. Axial length measurements were significantly correlated with the manometry measurements (R2 = 0.0957, p = 0.049). The endoscopic subjective size estimates were also closely related to the manometry measurements (R2 = 0.0543, p = 0.0164). Conversely, the subjective size estimates from barium swallow reads were not significantly correlated with the endoscopic axial length measurements (R2 = 0.0143, p = 0.366), endoscopic subjective size estimates (R2 = 0.0481, p = 0.0986), or the manometry measurements (R2 = 0.0418, p = 0.0738). Mesh placement was significantly correlated to pre-operative endoscopic axial length measurement (p = 0.0001), endoscopic subjective size estimate (p = 0.0301), and barium swallow read (p = 0.0211). However, mesh placement was not significantly correlated with pre-operative manometry measurements (0.2227).

CONCLUSIONS

Endoscopic subjective size allocations and objective axial length measurements are associated with pre-operative objective measurements and intra-operative decisions, suggesting both can be used to guide clinical decision making. However, including axial length measurements in endoscopy reports can improve outcomes reporting.

Characterizing the Role of Surgical Vice Chair of Education: Stakeholder Needs and Current Practices

INTRODUCTION

The role of a Department of Surgery (DoS) Vice Chair of Education (VCE) is a relatively new position intended to address the many changes and initiatives in medical education. The primary responsibilities and functions of a surgical VCE are not well described in the literature. This study examines the perceptions of stakeholders and current VCEs to develop a newly established position of VCE within an academic DoS.

METHODS

We conducted semistructured interviews of 13 faculties in our DoS. Each of these faculties hold the role of fellowship or residency program director. We also interviewed 13 VCEs from other departments within the same hospital system. Transcripts of these interviews were analyzed using reflexive thematic analysis, comparing the data gathered from the two populations.

RESULTS

There was a consensus of positivity about the new VCE position by the stakeholders with some residual uncertainty about how it would affect their work. Four themes resulted from analysis of stakeholder interviews including advocacy for education, balance of clinical, research, and educational duties, departmental integration, and faculty development as educators. Three themes were generated from analysis of interviews of VCEs including leverage of experience, framework establishment, and leadership in education.

CONCLUSIONS

Although exact VCE roles and responsibilities will be department dependent, this study may help other institutions craft their VCE positions and create a framework for understanding important stakeholder views.

Trend of minimally invasive and open surgery experience of general surgery residents: Accreditation Council for Graduate Medical Education general surgery case log in Thailand

Purpose

Minimally invasive surgery (MIS) offers patients several benefits, such as smaller incisions, and fast recovery times. General surgery residents should be trained in both open and MIS. We aimed to examine the trends of minimally invasive and open procedures performed by general surgery residents in Thailand.

Methods

A retrospective review of the Royal College of Surgeons of Thailand and Accreditation Council for Graduate Medical Education general surgery case logs from 2007 to 2018 was performed for common open and laparoscopic general surgery operations. The data were grouped by three time periods, which were 2007-2010, 2011-2014, and 2015-2018, and analyzed to explore changes in the operative trends.

Results

For Thai residents, the mean number of laparoscopic operations per person per year increased from 5.97 to 9.36 (56.78% increase) and open increased from 20.02 to 27.16 (35.67% increase). There was a significant increase in the average number of minimally invasive procedures performed among cholecystectomy (5.83, 6.57, 8.10; p < 0.001) and inguinal hernia repair (0.33, 0.35, 0.66; p < 0.001). Compared to general surgery residents in the United States, Thai residents had more experience with open appendectomy, but significantly less experience with all other operations/procedures.

Conclusion

The number of open and minimally invasive procedures performed or assisted by Thai general surgery residents has slowly increased, but generally lags behind residents in the United States. The Thai education program must be updated to improve residents' technical skills in open and laparoscopic surgery to remain competitive with their global partners.

Inpatient Satisfaction With Surgical Resident Care After Elective General and Oncologic Surgery

OBJECTIVE

To investigate inpatient satisfaction with surgical resident care.

BACKGROUND

Surgical trainees are often the primary providers of care to surgical inpatients, yet patient satisfaction with surgical resident care is not well characterized or routinely assessed.

METHODS

English-speaking, general surgery inpatients recovering from elective gastrointestinal and oncologic surgery were invited to complete a survey addressing their satisfaction with surgical resident care. Patients positively identified photos of surgical senior residents and interns before completing a modified version of the Consumer Assessment of Healthcare Providers and Systems Surgical Care Survey (S-CAHPS). Adapted S-CAHPS items were scored using the "top-box" method.

RESULTS

Ninety percent of recruited patients agreed to participate (324/359, mean age=62.2, 50.3% male). Patients were able to correctly identify their seniors and interns 85% and 83% of the time, respectively ( P =0.14). On a 10-point scale, seniors had a mean rating of 9.23±1.27 and interns had a mean rating of 9.01±1.49 ( P =0.14). Ninety-nine percent of patients agreed it was important to help in the education of future surgeons.

CONCLUSIONS

Surgical inpatients were able to recognize their resident physicians with high frequency and rated resident care highly overall, suggesting that they may serve as a willing source of feedback regarding residents' development of core competencies such as interpersonal skills, communication, professionalism, and patient care. Future work should investigate how to best incorporate patient evaluation of surgical resident care routinely into trainee assessment to support resident development.

The Big Sibling program: Impact of a medical student-resident mentorship program during the surgery clerkship

BACKGROUND

We created a Big Sibling mentorship program for medical students and studied the program effects.

METHODS

Between July 2019 to December 2020, students completing their surgery clerkship were paired with a Big Sibling surgical research resident. Participation in and perceptions of the program were assessed by survey.

RESULTS

81 medical students and 25 residents participated with a 79% and 95% survey response rate, respectively. The most valuable topics discussed included ward skills, personal development and career advising. Students who interacted >2 times with their Big Sibling were more likely to perceive the operating room as a positive learning environment, view attendings as role models, and receive mentoring and feedback from residents and attendings (p = 0.03, 0.02, 0.01 respectively). 78% of residents thought the program was a positive experience and no residents found it burdensome.

CONCLUSION

The Big Siblings program enhances the surgery clerkship learning environment. Students who engaged with their Big Sibling had a more positive view of the clerkship and the mentorship provided by residents and attendings.

Effective Senior Surgical Residents as Defined by Their Peers: A Qualitative Content Analysis of Nontechnical Skills Development

OBJECTIVE

This study aims to define an effective senior resident and understand the process of leadership and nontechnical skill development in the transition from junior to senior surgery resident.

SUMMARY BACKGROUND

General surgery residents are responsible for patient care, technically demanding operations, and diverse care team management. However, leadership skill development for the transition from junior to senior resident roles is often overlooked.

METHODS

We conducted 15 semi-structured focus groups with surgery residents from an urban, academic institution. Focus group transcripts were inductively coded. Using content analysis and constant comparative methodology, primary codes were refined into categories and organized into higher-level themes.

RESULTS

Thirty-three general surgery residents completed fifteen focus groups. Six themes were identified. Three themes describe the process of becoming an effective senior resident: how to define a senior resident's scope of practice, the transition process, and the importance of personal investment. Three themes were identified regarding effective seniors: ideal traits, teachable skills, and the team and patient impact.

CONCLUSIONS

Surgery residents define an effective senior resident as the team member with the highest level of experience who manages the big picture of patient care. The transition is improved by personal engagement and acknowledgement of the transition. Ideal traits of effective seniors, including emotional intelligence and inherent personality traits, allow a resident to more naturally assume this role; however, teachable skills, such as communication, expectation setting and competence, can be taught to improve one's effectiveness. The actions of a senior resident impact the team and patient care, underscoring the importance of understanding this role.

Multidisciplinary perceptions on robotic surgical training: the robot is a stimulus for surgical education change

BACKGROUND

It is unclear how to best establish successful robotic training programs or if subspecialty robotic program principles can be adapted for general surgery practice. The objective of this study is to understand the perspectives of high-volume robotic surgical educators on best practices in robotic surgery training and to provide recommendations transferable across surgical disciplines.

METHODS

This multi-institutional qualitative analysis involved semi-structured interviews with high-volume robotic educators from academic general surgery (AGS), community general surgery (CGS), urology (URO), and gynecology (GYN). Purposeful sampling and snowballing ensured high-volume status and geographically balanced representation across four strata. Interviews were transcribed, deidentified, and independently, inductively coded. A codebook was developed and refined using constant comparative method until interrater reliability kappa reached 0.95. A qualitative thematic, framework analysis was completed.

RESULTS

Thirty-four interviews were completed: AGS (n = 9), CGS (n = 8), URO (n = 9), and GYN (n = 8) resulting in 40 codes and four themes. Theme 1: intangibles of culture, resident engagement, and faculty and administrative buy-in are as important as tangibles of robot and simulator access, online modules, and case volumes. Theme 2: robotic OR integration stresses the trainee-autonomy versus patient-safety balance. Theme 3: trainees acquire robotic skills along individual learning curves; benchmark assessments track progress. Theme 4: AGS can learn from URO and GYN through multidisciplinary collaboration but must balance pre-existing training program use with context-specific curricular needs.

CONCLUSIONS

Robotic surgical experts emphasize the importance of universal training paradigms, such as a strong educational culture that balances autonomy and patient safety, collaboration between disciplines, and routine assessments for continuous growth. Often, introduction and acceptance of the robot serves as a stimulus to discuss broader surgical education change.

Interprofessional Trauma Team Training: Leveraging Each Specialties' Expertise to Teach Procedural-Based Skills

OBJECTIVE

The goal of this study was to utilize interprofessional trauma team training to teach procedural-based skills, teamwork, and assess the impact on the procedural comfort and interprofessional collaboration.

DESIGN

Interdisciplinary skills sessions were created to focus on chest tube placement and advanced ultrasound techniques. Chest tube sessions were taught by senior general surgery (GS) residents and faculty. Ultrasound sessions were taught by emergency medicine (EM) fellows and faculty. Mock trauma simulations for EM and GS residents and EM nurses, were developed to also focus on improving interprofessional trauma-bay collaboration. Sessions were held throughout the year for 2 consecutive academic years. After completing skills sessions and trauma scenarios, participants were surveyed on skill comfort, session utility, and willingness to collaborate with the other specialty. Likert scale responses were analyzed by specialty cohort and in aggregate. Free-text feedback responses were analyzed for common themes.

SETTING

Large, tertiary, urban academic medical center PARTICIPANTS: Forty seven EM residents and 32 GS residents completed instructional chest tube and ultrasound simulations, respectively. Twenty two EM residents, 24 GS residents, and 29 EM nurses participated in interprofessional trauma simulations.

RESULTS

For chest tube placement: 71% of EM residents reported feeling uncomfortable with the procedure prior to the session, with 100% reporting improved confidence afterwards. Seventy percent stated the model was realistic. One hundred percent thought it improved their procedural skills. All participants thought it was worthwhile, should be offered again in future years, and planned to incorporate what they learned in their future practice. For the ultrasound sessions: 61% of GS residents felt uncomfortable with the Focused Assessment with Sonography in Trauma prior to the simulation. Ninety four percent reported the improved skill and confidence, and felt the model was realistic. All participants felt sessions were worthwhile, should be offered again, and planned to incorporate what they learned in their future practice. For trauma simulations: 97% of participants felt scenarios were realistic and clinically relevant and planned to incorporate lessons learned in their future clinical practice. All participants thought participation was worthwhile. Ninety seven percent thought it improved their confidence with trauma clinical management and 56% reported it improved their skills. Many participants reported they appreciated learning from the other specialty's perspective, with greater than 95% of all participants reporting improved comfort and willingness to collaborate across disciplines when caring for future trauma patients. All participants requested the simulation sessions continue in future academic years.

CONCLUSION

Interprofessional trauma simulation sessions can harness the unique skill sets of different disciplines to teach procedural-based skills and improve interprofessional collaboration within the trauma bay.

Impact of Rater Training on Residents Technical Skill Assessments: A Randomized Trial

OBJECTIVE

The ACS/APDS Resident Skills Curriculum's Objective Structured Assessment of Technical Skills (OSATS) consists of task-specific checklists and a global rating scale (GRS) completed by raters. Prior work demonstrated a need for rater training. This study evaluates the impact of a rater-training curriculum on scoring discrimination, consistency, and validity for handsewn bowel anastomosis (HBA) and vascular anastomosis (VA).

DESIGN/ METHODS

A rater training video model was developed, which included a GRS orientation and anchoring performances representing the range of potential scores. Faculty raters were randomized to rater training or no rater training and were asked to score videos of resident HBA/VA. Consensus scores were assigned to each video using a modified Delphi process (Gold Score). Trained and untrained scores were analyzed for discrimination and score spread and compared to the Gold Score for relative agreement.

RESULTS

Eight general and eight vascular surgery faculty were randomized to score 24 HBA/VA videos. Rater training increased rater discrimination and decreased rating scale shrinkage for both VA (mean trained score: 2.83, variance 1.88; mean untrained score: 3.1, variance 1.14, p = 0.007) and HBA (mean trained score: 3.52, variance 1.44; mean untrained score: 3.42, variance 0.96, p = 0.033). On validity analyses, a comparison between each rater group vs Gold Score revealed a moderate training impact for VA, trained κ=0.65 vs untrained κ=0.57 and no impact for HBA, R1 κ = 0.71 vs R2 κ = 0.73.

CONCLUSION

A rater-training curriculum improved raters' ability to differentiate performance levels and use a wider range of the scoring scale. However, despite rater training, there was persistent disagreement between faculty GRS scores with no groups reaching the agreement threshold for formative assessment. If technical skill exams are incorporated into high stakes assessments, consensus ratings via a standard setting process are likely a more valid option than individual faculty ratings.

Gaps in Practice Management Skills After Training: A Qualitative Needs Assessment of Early Career Surgeons

PURPOSE

Shifts in American healthcare delivery mechanisms pose significant hurdles to new physicians. Surgeons are particularly susceptible to these changes, but surgical residency educational efforts primarily focus on technical and clinical training to the exclusion of business and management practices. This study conducted a needs assessment of perceived gaps in practice management skills among early career surgeons to guide future training curricula.

METHODS

This study was an exploratory qualitative study following the Consolidated Criteria for Reporting Qualitative Research. Purposive sampling was used to identify early career (<5 years following fellowship completion) surgeons across the United States. A semi-structured interview guide was created from interviews with surgical administrators and physician administrative curricula. Transcripts were de-identified and analyzed using a constructivist grounded theory approach.

RESULTS

Ten surgeons from 6 specialties and 6 institutions were interviewed along with 3 surgeon administrators. Three major domains of need were identified: (1) fundamentals of procedural coding, clinical billing, & compliance, (2) finding/building a practice, and (3) navigating organizational challenges. First, surgeons thought trainees would benefit from a better understanding of reimbursement schema and the basics of health policy. They also thought that more exposure to malpractice litigation, especially for handling case review or expert witness requests, would be helpful for discerning how to handle such issues early in their career. In addition, early career surgeons expressed a desire to have dedicated mentorship time, a primer on evaluating job offers with simulated contract negotiation, and guidance regarding administrative roles. Finally, surgeons requested training in change management techniques, care pathway construction, and the basics of staffing decisions.

CONCLUSIONS

There are significant practice management gaps in surgical training which may be amenable to targeted educational efforts during a residency or fellowship program. Future research will test the generalizability of these findings as well as build curricula that adequately meet these needs.

Simulation versus reality: what can interprofessional simulation teach us about team dynamics in the trauma bay?

Purpose

Surgical consultation and the joint management of trauma patients is a common scenario in the emergency department. The goal of this study was to utilize interprofessional trauma team training to understand the role of simulation and its impact on the overall culture of trauma-related care.

Methods

Interdisciplinary trauma simulation scenarios were completed by 12 groups of emergency medicine residents, general surgery residents, and emergency medicine nurses across two academic years. Following each simulation, a debriefing session was held to reflect on the scenario, focusing on team interactions. Debriefing sessions were audio-recorded, transcribed, deidentified, and independently, inductively coded by two members of the research team. Using the constant comparative method, a codebook was developed and refined until interrater reliability was confirmed with a kappa of > 0.9. Codes were organized into higher level themes.

Results

There were 72 participants, including 23 general surgery residents, 19 emergency medicine residents, and 30 emergency medicine nurses. 214 primary codes were collapsed into 29 coding categories, with 6 emerging themes. Pre-trauma bay impact describes how interactions prior to the trauma scenario can impact how team members communicate, trust one another, and ultimately care for the patient. Role and team identity explores the importance of one knowing their individual role in the trauma bay and how it impacts overall team identity. Resource allocation describes the balance of having appropriate resources to efficiently care for patients while not negatively impacting crowd control or role identity. Impact of the simulation experience highlights the impact of the lower stakes simulation scenario on learning and reflection as well as concerns with simulation fidelity. Trauma leader traits and actions outlines inherent traits and learned actions of trauma leaders that impact how the trauma scenario unfolds. Interprofessional team performance describes the overall performance of the trauma team, including but not limited to the type of communication used, teamwork behaviors, and transition of care of the patient.

Conclusions

Interdisciplinary trauma simulations and structured debriefing sessions provide insights into team dynamics and interprofessional relationships. Simulations and debriefing sessions can promote understanding, respect, and familiarity of team members' roles; recognition of key characteristics of high functioning leaders and teams; and discovery of conflict mitigating strategies for future interdisciplinary team improvement. Simulation sessions allow implementation of quality improvement measures and communication and leadership strategy practice in a safe, collaborative learning environment. The lessons learned from these sessions can encourage participants to reexamine how they interact and function as a team within the real-life trauma bay.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training

BACKGROUND

Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum.

METHODS

Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA.

RESULTS

Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion.

CONCLUSION

A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Virtual Simulation of Intra-operative Decision-Making for Open Abdominal Aortic Aneurysm Repair: A Mixed Methods Analysis

OBJECTIVE

To create and pilot test a novel open abdominal aortic aneurysm (AAA) repair virtual simulation focused on intraoperative decision-making. To identify if the simulation replicated real-time intra-operative decision-making and discover how learners' respond to this type of simulation.

DESIGN

An explanatory sequential mixed methods study. We developed a step-by-step outline of major intra-operative decision points within a standard open AAA repair. Perioperative and intraoperative decision-making trees were developed and coded into an online virtual simulation. The simulation was piloted. Quantitative data was collected from the simulation platform. We then performed a qualitative thematic analysis on feedback from interviewed participants.

SETTING

Four academic general and vascular surgical training programs across the US.

PARTICIPANTS

Seventeen vascular and general surgery trainees and 6 vascular surgery faculty.

RESULTS

Participants spent on average 27 minutes (range: 8-45 minutes) interacting with the interface. 93% of participants reported feeling they were making real intraoperative decisions. 85% said it added to their knowledge base. 96% requested additional simulations. 22 interviews were completed: 241 primary codes were collapsed into 21 parent codes, and 6 emerging themes identified. Themes included the benefit of how (1) "Virtual Learning Could Standardize the Training Experience"; how (2) "Dealing with the Unexpected" as a trainee is an important part of surgical education growth, and that this (3) "Choose Your Own Adventure" virtual format simulates this intraoperative growth experience. Participants requested a (4) "Looping Feature Feedback Diagram" for future simulation iterations and highlighted that (5) "Fancier is Not Necessarily More Educational." Finally, many trainees wondered about (6) "The Attending Impact" from the simulation: if faculty would notice a difference between trainees who did vs did not utilize the simulation for case preparation.

CONCLUSIONS

Operative simulation training should focus on both technical skills and intra-operative decision-making, particularly "dealing with the unexpected." The learners' responses indicate that a low-fidelity, scalable, virtual platform can effectively deliver knowledge and allow for intra-operative decision-making practice in a remote learning environment.

A multisite study of medical student perspectives on the core surgical clerkship

BACKGROUND

The surgical clerkship is the primary surgical learning experience for medical students. This study aims to understand student perspectives on the surgery clerkship both before and after the core surgical rotation.

METHODS

Medical students at 4 academic hospitals completed pre and postclerkship surveys that included open-ended questions regarding (1) student learning goals and concerns and (2) how surgical clerkship learning could be enhanced. Thematic analysis was performed, and interrater reliability was calculated.

RESULTS

Ninety-one percent of students completed both a pre and postclerkship survey (n =162 of 179), generating 320 preclerkship and 270 postclerkship responses. Mean kappa coefficients were 0.83 and 0.82 for pre and postclerkship primary themes, respectively. Thematic analysis identified 5 broad themes: (1) core learning expectations, (2) understanding surgical careers, culture, and work, (3) inhabiting the role of a surgeon, (4) inclusion in the surgical team, and (5) the unique role of the medical student on clinical clerkships. Based on these themes, we propose a learner-centered model of a successful surgical clerkship that satisfies discrete student learning and goals and career objectives while ameliorating the challenges of high-stakes clinical surgical environments such as the operating room.

CONCLUSION

Understanding student perspectives on the surgery clerkship, including preclerkship motivations and concerns and postclerkship reflections on surgical learning, revealed potential targets of intervention to improve the surgery clerkship. Future investigation may elucidate whether the proposed model of the elements of a successful surgery clerkship learning facilitates improvement of the surgical learning environment and enhanced surgical learning.

Implementation of competence by design in Canadian neurosurgery residency programs

OBJECTIVE

The Royal College of Physicians and Surgeons of Canada (RCPSC) recently redesigned the Canadian neurosurgery residency training curriculum by implementing a competency-based model of training known as Competence by Design (CBD) centered around the assessment of Entrustable Professional Activities (EPAs). This sequential explanatory mixed-methods study evaluated potential benefits and pitfalls of CBD in Canadian neurosurgery residency education.

METHODS

Two four-month interval surveys were distributed to all Canadian neurosurgery residents participating in CBD. The surveys assessed important educational components: CBD knowledge of key stakeholders, potential system barriers, and educational/psychological impacts on residents. Paired t-tests were done to assess changes over time. Based on longitudinal survey responses, semi-structured interviews were conducted to investigate in-depth residents' experience with CBD in neurosurgery. The qualitative analysis followed an explanatory approach, and a thematic analysis was performed.

RESULTS

Surveys had 82% average response rate (n = 25). Over time, most residents self-reported that they retrospectively understood concepts around CBD intentions (p = 0.02). Perceived benefits included faculty evaluations with more feedback that was clearer and more objective (53% and 51%). Pitfalls included the amount of time needed to navigate through EPAs (90%) and residents forgetting to initiate EPA forms (71%). There was no significant change over time. During interviews, five key themes were found. Potential solutions identified by residents to enhance their experience included learning analytics data availability, mobile app refinement, and dedicated time to integrate EPAs in the workflow.

CONCLUSION

This study was the first to assess resident-perceived benefits and pitfalls of the neurosurgery CBD training program in an educational framework context. In general, residents believed that theoretical principles behind CBD were valuable, but that technological ability and having enough time to request EPA assessments were significant barriers to success. Long-term studies are required to determine the definitive outcomes of CBD on residents' performance and ultimately, on patient care.

Association of Simulation Participation With Diagnostic Reasoning Scores in Preclinical Students

PURPOSE

The aim of the study was to evaluate for an association between the number of voluntary mannequin simulation sessions completed during the school year with scores on a year-end diagnostic reasoning assessment among second-year medical students.

METHOD

This is retrospective analysis of participation in 0 to 8 extracurricular mannequin simulation sessions on diagnostic reasoning assessed among 129 second-year medical students in an end-of-year evaluation. For the final skills assessment, 2 physicians measured students' ability to reason through a standardized case encounter using the Diagnostic Justification (DXJ) instrument (4 categories each scored 0-3 by raters reviewing students' postencounter written summaries). Rater scores were averaged for a total DXJ score (0-12). To provide additional baseline comparison, zero participation students were divided into 2 groups based on intent to participate: those who signed up for extracurricular sessions but never attended versus those who never expressed interest. Scores across the attendance groups were compared with an analysis of variance and trend analysis.

RESULTS

The class DXJ mean equaled 7.56, with a standard deviation of 2.78 and range of 0 to 12. Post hoc analysis after a significant analysis of variance (F = 4.91, df = 8, 128, P < 0.001) showed those participating in 1 or more extracurricular sessions had significantly higher DXJ scores than those not participating. Students doing 7 extracurricular sessions had significantly higher DXJ scores than those doing 0 and 2 (P < 0.05). Zero attendance groups were not different. A significant linear trend (R = 0.48, F = 38.0, df = 1, 127, P < 0.001) was found with 9 groups. A significant quadratic effect, like a dose-response pattern, was found (F = 18.1, df = 2, 125, P < 0.001) in an analysis including both zero attendance groups, a low (1-4 extracurricular sessions) group and a high (5-8) group.

CONCLUSIONS

Higher year-end diagnostic reasoning scores were associated with increased voluntary participation in extracurricular mannequin-based simulation exercises in an approximate dose-response pattern.

Finding my people: Effects of student identity and vulnerability to Stereotype Threat on sense of belonging in surgery

BACKGROUND

Diversity in surgery lags behind the medical student population. We documented first-year medical students' vulnerability to stereotype threat (VST) and its impact on a sense of belonging in surgery.

METHODS

All first-year medical students at a single academic institution were surveyed. Demographics, VST, anticipated clerkship experience, and sense of belonging were assessed.

RESULTS

44% of students were vulnerable to ST in upcoming clerkships, with the majority worried about surgical clerkships. More student from races/ethnicities underrepresented in medicine (URM; 74%) and sexual minorities (62%) were vulnerable than white (30%) and heterosexual (38%) students respectively (p = 0.001 and p = 0.017). Knowing a surgeon with a shared identity would enhance belonging for most students (84%). VST was higher for those who do not anticipate working with (p < 0.001) or do not know a surgeon (p = 0.0001) who shares their identity.

CONCLUSION

VST significantly influences a student's sense of belonging in surgery. More research is needed to promote inclusivity in surgery.

Challenges of virtual interviewing for surgical fellowships: a qualitative analysis of applicant experiences

BACKGROUND

The COVID-19 pandemic forced surgical fellowship programs to transition from in-person to remote applicant interviews; the virtual interviewing format presented new and unique challenges. We sought to understand applicants' perceived challenges to virtual interviewing for a surgical fellowship program.

METHOD

A grounded theory-based qualitative study was performed utilizing semi-structured interviews with fellowship applicants from the 2020 fellowship match. All Fellowship Council-registered applicants were eligible. We purposefully sampled participants to balance across gender, specialty-choice, and academic versus community-program affiliation. Interviews were inductively analyzed by two researchers for prominent themes.

RESULTS

Fifteen interviews were conducted. Participants were 60% male (n = 9), with 33% (n = 5) from non-academic institutions. They applied for the following fellowships: Advanced Gastrointestinal/Minimal Invasive (55%), Bariatric (30%), Hepatopancreatobiliary (10%) and Surgical Oncology (5%). Four main themes emerged to describe virtual interview process challenges: (1) perceived data deficiency, (2) superficial personal connections, (3) magnification of non-professionalism, and (4) logistical frustrations. Applicants recommend program directors provide more information about the fellowship prior to interview day and offer informal independent interactions with current and previous fellows.

CONCLUSIONS

According to fellowship applicants, virtual interviews resulted in a lack of information for rank-list decision making ultimately requiring them to rely on other information avenues to base their decisions. These applicants have offered advice to fellowship program directors and future applicants to better optimize this process.

Which Patient Do I Attend to First? Night-float Simulation to Assess Surgical Intern's Clinical Prioritization Skills

OBJECTIVE

Night-float rotations require general surgery interns to prioritize multiple competing patients' needs efficiently and accurately. Research is lacking on whether these skills can be taught and to what degree the prioritizations taught match overall attending surgeon expectations.

DESIGN

A night-float situation was developed to simulate the experience of surgical interns responding to multiple patients' needs. Participants were instructed to rank order 10 patient paper-case scenarios with a variety of clinical urgencies. After completing their first ranking, the interns participated in a faculty-facilitated peer discussion (intervention) and were then instructed to re-rank their priorities. Their performance was compared pre- and post-intervention, and to the ranking of 16 surgery faculty attendings.

SETTING

Massachusetts General Hospital, Department of Surgery, Boston, MA.

PARTICIPANTS

Post-graduate year (PGY) 1 surgical residents.

RESULTS

Two classes of general surgery interns (n = 25) completed the prioritization training simulation in the middle of their internship year, one class in 2018 and one in 2020. Agreement between interns regarding patient prioritization ranking increased after the facilitated peer discussion (pre-intervention mean standard deviation = 1.8 versus 0.9 post-intervention; p = 0.03). In the post-intervention mean rank, four cases moved by one position (p < 0.05). The facilitated discussion resulted in increased absolute agreement between individual interns and attendings' ranks (mean agreement 38 ± 17% pre-intervention Vesus mean 50 ± 20% post-intervention, p = 0.02). The scenarios with highest agreement between interns and attendings concerned patients with the most urgent conditions. Scenarios with the lowest agreement were those ranked with medium-to-low urgency.

CONCLUSION

A faculty-led facilitated discussion appears to increase clinical prioritization consistency among surgical interns and better align their prioritizations with expectations of local attending surgeons.

The Resident-Run Minor Surgery Clinic: A Four-Year Analysis of Patient Outcomes, Satisfaction, and Resident Education

OBJECTIVE

A resident-run minor surgery clinic was developed to increase resident procedural autonomy. We evaluated whether 1) there was a significant difference between complications and patient satisfaction when procedures were independently performed by surgical residents vs. a surgical attending and 2) if participation was associated with an increase in resident procedural confidence.

DESIGN

Third year general surgery residents participated in a weekly procedure clinic from 2014-2018. Post-procedure complications and patient satisfaction were compared between patients operated on by residents vs. the staff surgeon. Residents were surveyed regarding their confidence in independently performing a variety of clinic-based patient care tasks.

SETTING

Massachusetts General Hospital General in Boston, MA; an academic tertiary care general surgery residency program.

PARTICIPANTS

Post-graduate year three general surgery residents that ran the clinic as part of a general surgery rotation.

RESULTS

1230 patients underwent 1592 procedures (612 in resident clinic, 980 in attending clinic). There was no significant difference in the 30-day complication rate between patients operated on by the resident vs. attending (2.5% vs. 1.9%, p = 0.49). 459 patient satisfaction surveys were administered with a 79.1% response rate. There was no significant difference in the overall quality of care rating between residents and the attending surgeon (87.5% top-box rating vs. 93.1%, p = 0.15). Twenty-one residents completed both a pre- and post-rotation survey (77.8% response rate). The proportion of residents indicating that they could independently perform a variety of patient care tasks significantly increased across the rotation (all p < 0.05).

CONCLUSION

Mid-level general surgery residents can independently perform office-based procedures without detriment to safety or patient satisfaction. The resident-run procedure clinic serves as an environment for residents to grow in confidence in both technical and non-technical skills. Given the high rate at which patients provide resident feedback, future work may investigate how to best incorporate patient derived evaluations into resident assessment.

Virtual student-transplant patient interactions empower patients and enhance student transplantation knowledge

INTRODUCTION

There is variation in exposure to transplantation in undergraduate medical education. We created a program pairing medical students with transplant patients for semi-structured, virtual encounters and studied the impact on both students and patients using qualitative content analysis.

METHODS

Fifty medical students were paired with transplant recipients and donors for non-medical virtual encounters. Separate focus groups were conducted, deidentified, and analyzed using a constant comparative method.

RESULTS

Three themes related to the student experience emerged: transplant-related relationships, a deeper understanding of the patient's journey to transplant, and alterations of their personal view of organ donation and transplantation. Three themes emerged from the patient's experiences: the benefits of conversations, the patient as a teacher, and spreading the message of organ donation and transplantation.

CONCLUSIONS

This novel program demonstrates that virtual student-patient interactions are a useful approach to engage patients and a unique way to teach medical students about transplantation and donation.

Adaptation to Open-Book Online Examination During the COVID-19 Pandemic

INTRODUCTION

COVID-19 altered medical education systems worldwide as many medical schools quickly changed to online assessment systems. However, the feasibility of online assessment and how it compares to traditional examinations is unclear.

METHODS

We compared 4^th year medical students' online surgery clerkship assessment scores to the traditional written examinations. The percent of correct scores using online open-book examination was compared to the results of the traditional closed-book examination in the previous three rotations. Additional correlation between grade point average(GPA) and examination performance were reviewed.

RESULTS

Compared with the traditional groups, medical students who took the online, open-book examination had a significantly higher mean score in both MCQ(85.21 vs. 77.36, 72.43, 83.00, p<0.001) and essay examinations (187.36 vs. 158.77, 152.17, 152.29, p<0.001), but a significantly lower mean score in short answer examination (60.09 vs. 66.79, 67.73, 64.82, p<0.001). The online open-book examination group had a significantly lower correlation between the essay score and their GPA than the previous traditional groups (z=2.81 p=0.005, z=2.23 p=0.026, z=2.19 p=0.029).

CONCLUSION

Although an online, open-book examination was feasible during the COVID-19 pandemic, this study indicates that mean scores are significantly different which has important implications regarding grading and standard setting. More research is required to assess other effects of this new assessment on long-term knowledge retention and application.

A Needs Assessment of Video-based Education Resources Among General Surgery Residents

BACKGROUND

Video-based education (VBE) is an effective tool for knowledge and skill acquisition for medical students, but its utility is less clear for resident physicians. We sought to determine how to incorporate VBE into a general surgery resident operative curriculum.

METHODS

We conducted a single-institution, survey-based needs assessment of general surgery residents to determine desired content and format of an operative VBE module.

RESULTS

The response rate was 84% (53/63), with 66% senior (postgraduate year ≥3) resident respondents. VBE was the most commonly cited resource that residents used to prepare for an operation (93%) compared with surgical textbooks (89%) and text-based website content (57%). Junior residents were more likely to utilize text-based website content than senior residents (P < 0.01). The three most important operative video components were accuracy, length, and cost. Senior residents significantly preferred videos that were peer-reviewed (P < 0.05) and featured attending surgeons whom they knew (P = 0.03). A majority of residents (59%) believed 5-10 min is the ideal length of an operative video. Across all postgraduate year levels, residents indicated that detailed instruction of each operative step was the most important content of a VBE module. Senior residents believed that the overall indications and details of each step of the operation were the most important contents of VBE for a junior resident.

CONCLUSIONS

At this institution, general surgery residents preferentially use VBE resources for operative preparation. A centralized, standardized operative resource would likely improve resident studying efficiency, but would require personalized learning options to work for both junior and senior surgery residents.

Implementation of the ACS/ APDS Resident Skills Curriculum reveals a need for rater training: An analysis using generalizability theory

BACKGROUND

The American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) Resident Skills Curriculum includes validated task-specific checklists and global rating scales (GRS) for Objective Structured Assessment of Technical Skills (OSATS). However, it does not include instructions on use of these assessment tools. Since consistency of ratings is a key feature of assessment, we explored rater reliability for two skills.

METHODS

Surgical faculty assessed hand-sewn bowel and vascular anastomoses in real-time using the OSATS GRS. OSATS were video-taped and independently evaluated by a research resident and surgical attending. Rating consistency was estimated using intraclass correlation coefficients (ICC) and generalizability analysis.

RESULTS

Three-rater ICC coefficients across 24 videos ranged from 0.12 to 0.75. Generalizability reliability coefficients ranged from 0.55 to 0.8. Percent variance attributable to raters ranged from 2.7% to 32.1%. Pairwise agreement showed considerable inconsistency for both tasks.

CONCLUSIONS

Variability of ratings for these two skills indicate the need for rater training to increase scoring agreement and decrease rater variability for technical skill assessments.

The Surgical Knowledge "Growth Curve": Predicting ABSITE Scores and Identifying "At-Risk" Residents

OBJECTIVE

Resident performance on the American Board of Surgery In-Training Examination (ABSITE) is used for evaluation of surgical knowledge and guides resident selection for institutional remediation programs. Remediation thresholds have historically been based on ABSITE percentile scores; however, this does not account for predictors that can impact a resident's exam performance. We sought to identify predictors of yearly ABSITE performance to help identify residents "at-risk" for performing below their expected growth trajectory.

DESIGN

The knowledge of the residents, as measured by standardized ABSITE scores, was modeled as a function of the corresponding postgraduate year via a linear mixed effects regression model. Additional model covariates included written USMLE-1-3 examination scores, gender, number of practice questions completed, and percentage correct of practice questions. For each resident, the predicted ABSITE standard score along with a 95% bootstrap prediction interval was obtained. Both resident-specific and population-level predictions for ABSITE standard scores were also estimated.

SETTING

The study was conducted at a single, large academic medical center (Massachusetts General Hospital, Boston, MA).

PARTICIPANTS

Six years of general surgery resident score reports at a single institution between 2014 and 2019 were deidentified and analyzed.

RESULTS

A total of 376 score reports from 130 residents were analyzed. Covariates that had a significant effect on the model included USMLE-1 score (PGY1: p = 0.013; PGY2: p = 0.007; PGY3: p = 0.011), USMLE-2 score (PGY1: p < 0.001; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p < 0.001; PGY5: p = 0.032), male gender (PGY1: p = 0.003; PGY2: p < 0.001; PGY3: p < 0.001; PGY4: p = 0.008), and number of practice questions completed (p=0.003). Five residents were identified as having "fallen off" their predicted knowledge curve, including a single resident on 2 occasions. Population prediction curves were obtained at 7 different covariate percentile levels (5%, 10%, 25%, 50%, 75%, 90%, and 95%) that could be used to plot predicted resident knowledge progress.

CONCLUSION

Performance on USMLE-1 and -2 examinations, male gender, and number of practice questions completed were positive predictors of ABSITE performance. Creating residency-wide knowledge growth curves as well as individualized predictive ABSITE performance models allows for more efficient identification of residents potentially at risk for poor ABSITE performance and structured monitoring of surgical knowledge progression.

Medical Students and Patients Benefit from Virtual Non-Medical Interactions Due to COVID-19

BACKGROUND

Patient care restrictions created by the COVID-19 pandemic constrained medical students' ability to interact directly with patients. Additionally, organ transplant recipients faced increasing isolation due to the rise of telemedicine, the importance of social distancing and their immunosuppressed state. We created a pilot program to pair students with transplant patients for structured, virtual encounters and studied its impact on medical students and patients.

METHODS

In May 2020, medical students conducted virtual visits with patients via telephone or video conferencing. Patients and students were surveyed regarding their experiences and independent focus groups were conducted. The survey responses and focus group discussions were deidentified, transcribed, and analyzed for themes.

RESULTS

Ten participating students were in their first, second, or final year of medical school. The 14 patients were liver or kidney transplant recipients or kidney donors. All interactions lasted longer than 30 minutes, with 56% greater than 1 hour. Three themes emerged related to the student experience: improvement of their clinical communication skills, development of knowledge and attitudes related to organ transplantation and donation, and independent management of a patient encounter. Three themes related to the patient experience: appreciation of the opportunity to share their personal patient experience to help educate future physicians, a cathartic and personally illuminating experience and an opportunity to share the message of donation.

CONCLUSIONS

This pilot program provided a novel opportunity for virtual student-patient interactions that was feasible, well-received, and mutually beneficial. The use of virtual non-medical patient experiences allowed for experiential learning during which students learned about both clinical medicine and enhanced their communication skills directly from patients. Additionally, patients were able to engage with medical students in a new way, as teachers of clinical interactions, and reported a high level of satisfaction in addition to deriving personal benefit.

Concordance of Resident and Patient Perceptions of Culturally Dexterous Patient Care Skills

PURPOSE

Disparities in surgical care persist. To mitigate these disparities, we are implementing and testing the Provider Awareness and Cultural dexterity Toolkit for Surgeons (PACTS), a curriculum to improve surgical residents' cultural dexterity during clinical encounters. We analyzed baseline data to look for concordance between residents' self-perceived cultural dexterity skills and patients' perceptions of their skills. We hypothesized that residents would rate their skills in cultural dexterity higher than patients would perceive those skills.

METHODS

Prior to the implementation of the curriculum, surgical residents at 5 academic medical centers completed a self-assessment of their skills in culturally dexterous patient care using a modified version of the Cross-Cultural Care Survey. Randomly selected surgical inpatients at these centers completed a similar survey about the quality of culturally dexterous care provided by a surgery resident on their service. Likert scale responses for both assessments were classified as high (agree/strongly agree) or low (neutral/disagree/strongly disagree) competency. Resident and patient ratings of cultural dexterity were compared. Assessments were considered dexterous if 75% of responses were in the high category. Univariate and multivariate analysis was conducted using STATA 16.

RESULTS

A total of 179 residents from 5 surgical residency programs completed self-assessments prior to receiving the PACTS curriculum, including 88 (49.2%) women and 97 (54.2%) junior residents (PGY 1-2s), of whom 54.7% were White, 19% were Asian, and 8.9% were Black/African American. A total of 494 patients with an average age of 55.1 years were surveyed, of whom 238 (48.2%) were female and 320 (64.8%) were White. Fifty percent of residents viewed themselves as culturally dexterous, while 57% of patients reported receiving culturally dexterous care; this difference was not statistically significant (p = 0.09). Residents who perceived themselves to be culturally dexterous were more likely to self-identify as non-White as compared to White (p < 0.05). On multivariate analysis, White patients were more likely to report highly dexterous care, whereas Black patients were more likely to report poorly dexterous care (p < 0.05).

CONCLUSIONS

At baseline, half of patients reported receiving culturally dexterous care from surgical residents at 5 academic medical centers in the United States. This was consistent with residents' self-assessment of their cultural dexterity skills. White patients were more likely to report receiving culturally dexterous care as compared to non-White patients. Non-White residents were more likely to feel confident in their cultural dexterity skills. A novel curriculum has been designed to improve these interactions between patients and surgical residents.

A qualitative study of the perceived value of participation in a new Department of Surgery Research Residents as teachers program

OBJECTIVE

This study aims to understand the perspectives of surgical residents who completed a Research Residents as Teachers Program (RRATP).

METHODS

Our RRATP included a 6 h workshop followed by formal teaching opportunities across one academic year. Resident teachers participated in semi-structured interviews, which were inductively analyzed for prominent themes.

RESULTS

Eight surgical research residents completed the RRATP workshop and taught 330 h (median = 26 h, range: 8-105). Interview participation rate was 100%; kappa was 0.81. Residents reported four themes: 1) increased knowledge of teaching principles with subsequent teaching changes, specific factors that contributed to their development as a teacher, numerous personal benefits to participation, and broad positive consequences for the surgical department including improved culture and patient care.

CONCLUSION

A RRATP can generate a significant number of formal teaching hours by surgical research residents, who perceive a high value of formal education training to themselves and their surgical residency program.

Identification of Specific Educational Targets to Improve the Student Surgical Clerkship Experience

BACKGROUND

This study describes the relationship between medical student perception of surgery, frequency of positive surgery clerkship activities, and overall surgical clerkship experience.

METHODS

Medical students at four academic hospitals completed pre- and post-clerkship surveys assessing 1) surgery clerkship activities/experiences and 2) perceptions of surgery during the 2017-2018 academic year.

RESULTS

Ninety-one percent of students completed both a pre- and post-clerkship survey (n = 162 of 179). Student perception of surgery significantly improved across the clerkship overall (P < 0.0001) and for 7 of 21 specific items. Eighty-six percent of students agreed that the clerkship was a meaningful experience. Sixty-six percent agreed that the operating room was a positive learning environment. Multivariable logistic regression identified one-on-one mentoring from a resident (OR [95% CI] = 2.12 [1.11-4.04], P = 0.02) and establishing a meaningful relationship with a surgical patient (OR = 2.21 [1.12-4.37], P = 0.02) as activities predictive of student agreement that the surgical clerkship was meaningful. Making an incision (OR = 2.92 [1.54-5.56], P = 0.001) and assisting in dissection (OR = 1.67 [1.03-2.69], P = 0.035) were predictive of student agreement that the operating room was a positive learning environment. Positive student perception of surgery before the clerkship was associated with increased frequency of positive clerkship activities including operative involvement (r = 0.26, P = 0.001) and relationships with surgical attendings (r = 0.20, P = 0.01), residents (r = 0.41, P < 0.0001), and patients (r = 0.24, P = 0.003).

CONCLUSIONS

Interventions to improve surgery clerkship quality should target enhancing student relationships with residents and surgical patients as well as providing opportunity for student operative involvement beyond just suturing. In addition, fostering positive perceptions of surgery in the preclinical period may increase meaningfulness and experience with the later surgery clerkship.

Surgeons With Five or More Actual Cricothyrotomies Perform Significantly Better on a Virtual Reality Simulator

BACKGROUND

Discriminating performance of learners with varying experience is essential to developing and validating a surgical simulator. For rare and emergent procedures such as cricothyrotomy (CCT), the criteria to establish such groups are unclear. This study is to investigate the impact of surgeons' actual CCT experience on their virtual reality simulator performance and to determine the minimum number of actual CCTs that significantly discriminates simulator scores. Our hypothesis is that surgeons who performed more actual CCT cases would perform better on a virtual reality CCT simulator.

METHODS

47 clinicians were recruited to participate in this study at the 2018 annual conference of the Society of American Gastrointestinal and Endoscopic Surgeons. We established groups based on three different experience thresholds, that is, the minimal number of CCT cases performed (1, 5, and 10), and compared simulator performance between these groups.

RESULTS

Participants who had performed more clinical cases manifested higher mean scores in completing CCT simulation tasks, and those reporting at least 5 actual CCTs had significantly higher (P = 0.014) simulator scores than those who had performed fewer cases. Another interesting finding was that classifying participants based on experience level, that is, attendings, fellows, and residents, did not yield statistically significant differences in skills related to CCT.

CONCLUSIONS

The simulator was sensitive to prior experience at a threshold of 5 actual CCTs performed.

Enhancing the Formal Preclinical Curriculum to Improve Medical Student Perception of Surgery

BACKGROUND

This study aims to determine the effect of formal, preclinical curricular interventions on medical students' perceptions of surgeons, surgical learning objectives, and concerns regarding the surgical clerkship.

METHODS

Thirty-eight medical students underwent a newly required, formal introduction to surgery during the preclinical curriculum. Two months later, these students were given surveys regarding their perception of surgery before and after a bootcamp-style transitions to the wards workshop that immediately preceded their core clinical rotations. Student responses were compared to historical peers.

RESULTS

Thirty-seven students participated in the study (97.4%). Relative to historical peers, students demonstrated improved overall perception of surgery (71.2 vs 66.6, p = 0.046). A smaller proportion of students indicated that they were worried about evaluation (18.9% in 2018 vs 55.3% in 2017, p = 0.001) and interactions with surgical educators (18.9% vs 50%, p = 0.005). Students' overall perception of surgery significantly improved after participation in the transition to the wards workshop (71.2 to 77.8, p ≤ 0.0001), as did student agreement with 9 of 21 specific items. Improvement in surgical perception across the bootcamp-style workshop was similar to that of a prior workshop (8.6 in 2018 vs 6.4 in 2017, p = 0.21).

CONCLUSIONS

A preclinical introduction to surgery can have a positive impact on medical student perception of surgery prior to entry to the wards and may mitigate student fears regarding their surgical rotation.

Prevention and Management of Operating Room Fire: An Interprofessional Operating Room Team Simulation Case

INTRODUCTION

Operating room (OR) fire can be a devastating and costly event to patients and health care providers. Prevention and effective management of such fires may present difficulties even for experienced OR staff.

METHODS

This simulation involved a 52-year-old man presenting for excisional biopsy of a cervical lymph node to be performed under sedation. Participants were expected to identify and manage both contained and uncontained fires resulting from ignition by electrosurgical cautery. We conducted weekly multidisciplinary simulations in the mock OR at Massachusetts General Hospital. Participants included surgery and anesthesiology residents, certified registered nurse anesthetists, registered nurses, and surgical technicians. Participants were unaware of the scenario content. Each 90-minute session was divided into three parts: an orientation (10 minutes), the case with rapid cycle debriefing (65 minutes), and a final debriefing with course evaluations (15 minutes). Equipment consisted of a simulation OR with general surgery supplies, general anesthesia equipment, a high-fidelity Laerdal SimMan 3G simulator, a code cart, a defibrillator, dry ice for smoke effects, and a projector with a fire image.

RESULTS

From April to June 2015, 86 participants completed this simulation. Participants reported that the simulation scenario was realistic (80%), was relevant to their clinical practice (93%), changed their practice (82%), and promoted teamwork (80%).

DISCUSSION

Prevention and management of OR fire require collaboration and prompt coordination between anesthesiologists, surgeons, and nurses. This simulation case scenario was implemented to train multidisciplinary learners in the identification and crisis management of such an event.

"Yes, I'm the Doctor": One Department's Approach to Assessing and Addressing Gender-Based Discrimination in the Modern Medical Training Era

While gender-based bias and discrimination (GBD) is known to exist in medical training, there is limited guidance for training programs on how to understand and combat this issue locally. The Massachusetts General Hospital Department of Surgery established the Gender Equity Task Force (GETF) to address GBD in the local training environment. In 2017, members of the GETF surveyed residents in surgery, anesthesia, and internal medicine at 2 academic hospitals to better understand perceived sources, frequency, forms, and effects of GBD. Overall, 371 residents completed the survey (60% response rate, 197 women). Women trainees were more likely to endorse personal experience of GBD and sexual harassment than men (P < .0001), with no effect of specialty on rates of GBD or sexual harassment. Patients and nursing staff were the most frequently identified groups as sources of GBD. While an overwhelming majority of both men (86%) and women (96%) respondents either experienced or observed GBD in the training environment, less than 5% of respondents formally reported such experiences, most frequently citing a belief that nothing would happen. Survey results served as the basis for a variety of interventions addressing nursing staff and patients as sources of GBD, low confidence in formal reporting mechanisms, and the pervasiveness of GBD, including sexual harassment, across specialties. These results reproduce other studies' findings that GBD and sexual harassment disproportionately affect women trainees while demonstrating how individual training programs can incorporate local GBD data when planning interventions to address GBD.

This is Not Brain Surgery: Increasing Neurosurgical Knowledge and Retention in Medical Students Through Usage of a Video-Based Education Curriculum

INTRODUCTION

The fundamental elements of neurosurgical care are essential for all medical students regardless of their ultimate clinical specialty. However, the high-pressured neurosurgical environment leaves limited time for teaching medical students. One potential solution is the use of video-based education modules, but the effectiveness and usage of these modules in surgery is unclear.

METHODS

A 2-wk asynchronous, VBE (video based education) curriculum of four key neurosurgical topics as outlined in the Congress of Neurological Surgeons' curriculum for medical students (Intracranial Hemorrhage, Neuro-Imaging, Hydrocephalus, and Glasgow Coma Scale) was created and implemented for surgery clerkship students (n = 65). On day 1 of the study, each student was randomly assigned to 1 of 2 pairs, given a pretest for knowledge and self-efficacy on all 4 topics, and provided with the link for 2 of the 4 VBE modules. On day 14, a post-test for knowledge and self-efficacy on all four topics was completed. Usage analytics were employed to track views of assigned content.

RESULTS

Students who watched the modules (n = 53) increased their knowledge (+11.0%, P = .001) and self-efficacy (+1.37, P = .001) from pre- to post. Students who did not watch the modules (n = 12) showed no change in knowledge (58.8% vs 58.3%, P = NS), but a significant increase in their self-efficacy ratings (+1.42, P = .009). Learning analytics revealed that vast majority of learners (81%) engaged with the curriculum and watched their assigned videos once or on multiple occasions. 19% of learners did not engage with the curriculum, citing their heavy workload as a primary reason.

CONCLUSION

This study shows that a focused, asynchronous, VBE curriculum in neurosurgery has a significantly positive effect on knowledge and self-efficacy scores amongst medical students. Future studies will investigate how to improve learner compliance and better understand the gap in knowledge improvement vs self-efficacy.

Feasibility and Perceived Usefulness of Using Head-Mounted Cameras for Resident Video Portfolios

BACKGROUND

There is limited guidance on how to longitudinally administer simulation materials or to incorporate video recordings into assessment portfolios of simulated surgical skills.

MATERIALS AND METHODS

We launched a longitudinal weekly simulation curriculum for PGY1-PGY3 surgical residents based on the ACS/APDS Curriculum. Residents underwent monthly objective structured assessment of technical skills (OSATS) while wearing head-mounted cameras. Videos of OSATS performance accrued into individual online video portfolios. Residents were surveyed about their attitudes toward video recording.

RESULTS

Twenty-seven general surgical residents participated, completing 161 OSATS encompassing 11 distinct skills and generating 258 videos of simulated skills performance. The overall survey response rate was 88%. Residents viewed the curriculum favorably overall, and 36.4% of residents accessed their videos. Of those who did not watch their videos, 78.6% cited not having enough time, whereas 28.6% did not think the videos would be useful. Over 95% of surveyed residents expressed interest in having a video library of attending-performed procedures, 59.1% were interested in having their own operations recorded, and 45.5% were interested in video-based coaching.

CONCLUSIONS

Residents viewed longitudinal administration of the ACS/APDS Curriculum positively. Although video recording in simulation is feasible, resident interest may be higher for intraoperative recordings than for simulated skills.

Medical students' perceptions and motivations prior to their surgery clerkship

BACKGROUND

This study aims to determine the effect of a pre-clerkship workshop on medical students' perceptions of surgery and surgeons and to describe their concerns and learning goals.

METHODS

Thirty-nine medical students completed surveys before and after a workshop preceding their surgery clerkship. Quantitative data and free responses that were inductively coded were used to assess effectiveness.

RESULTS

Perceptions from 38 students (response rate = 97.4%) significantly improved for 11 of 21 items. At pre-workshop, the most frequently cited learning goals were improving technical skills (58%), surgical knowledge (53%), and understanding surgical culture and work (53%). Students' top concerns were meeting clerkship demands (68%) and being evaluated (55%). After the workshop, student learning objectives and concerns remained largely unchanged.

CONCLUSIONS

A pre-clerkship workshop improved student perceptions of surgery and surgeons. Understanding students' intrinsic motivations may facilitate future clerkship curriculum improvement via better alignment of educator and student goals and objectives.

Surgical procedural map scoring for decision-making in laparoscopic cholecystectomy

INTRODUCTION

The objective of this study was to determine whether decision-based procedural mapping demonstrates differences in attendings versus residents.

METHODS

Attendings and residents were interviewed about operative decision-making in laparoscopic cholecystectomy (LC) using a cognitive task analysis framework. Interviews were converted into procedural maps. Operative steps, patient factors, and surgeon factors noted by attendings and residents were compared. Two scoring methods were used to compare map structures of attendings versus residents.

RESULTS

Six attendings and six residents were interviewed. There were no significant differences in the number of patient or surgeon factors identified. Attendings had significantly more operative steps (29.67 ± 1.9 vs. 23.3 ± 1.9, p = 0.04) and crosslinks (3.2 ± 0.5 vs. 1 ± 0.4, p = 0.005) in their maps and a higher total score (90.2 ± 8.4 vs. 63.2 ± 3.8, p = 0.015) than residents.

CONCLUSION

LC procedural map scoring for attendings and residents demonstrated significant differences in structural complexity and may provide a useful framework for assessing decision making.

A Pilot Study of Inpatient Satisfaction Rating of Surgical Resident Care

OBJECTIVE

To describe inpatient satisfaction with surgical resident care given the increasing importance of patient satisfaction as a quality metric.

DESIGN

Surgical inpatients were invited to complete a survey that addressed their satisfaction with and attitudes regarding surgical resident care. The survey was based on the Consumer Assessment of Healthcare Providers and Systems Surgical Care Survey (S-CAHPS). Patients were required to positively identify photos of resident physicians prior to providing ratings. Adapted S-CAHPS items were scored using the "top-box" method.

SETTING

Massachusetts General Hospital, a tertiary academic center.

PARTICIPANTS

General surgery inpatients recovering from elective, major abdominal surgery were recruited on postoperative days 2 to 4.

RESULTS

Ninety-one percent of approached patients participated (102/112, mean age = 62.9, 51.6% male). Patients positively identified both seniors and interns 88% of the time. Thirteen seniors and 19 interns were rated, with 1 to 14 evaluations per trainee. Overall quality of care ratings for seniors and interns were 9.35 and 9.09, respectively (0-10 scale, 10 = "best possible care"). Sixty-three percent of senior resident evaluations and 60% of intern evaluations received a score of 10. The proportion of residents receiving top-box scores ranged from 59.5% to 97.7% depending on the item. Forty percent of senior resident and 38% of intern evaluations received top-box scores for all 8 items. Over 96% of patients reported strong or moderate agreement with the statements "I feel it is important to help in the education of future surgeons."

CONCLUSIONS

Surgical inpatients willingly completed ratings about their surgery residents, typically can recognize their resident physicians, and rate quality of care highly. Despite many high ratings, there is room for improvement in some S-CAHPS domains. These results indicate patients are a valuable source of feedback regarding a resident's progress in several core competencies such as interpersonal skills, communication, professionalism, and patient care.

High-Fidelity Emergency Department Thoracotomy Simulator With Beating-Heart Technology and OSATS Tool Improves Trainee Confidence and Distinguishes Level of Skill

OBJECTIVE

Resuscitative Thoracotomy or Emergency Department Thoracotomy (EDT) is a time-sensitive and potentially life-saving procedure. Yet, trainee experience with this procedure is often limited in both clinical and simulation settings. We sought to develop a high-fidelity EDT simulation module and assessment tool to facilitate trainee education.

DESIGN

Using the Kern model for curricular development, a group of expert trauma surgeons identified EDT as a high-stakes, low-frequency procedure. Task analysis identified 5 key steps of EDT: (1) opening chest/rib spreader utilization; (2) pericardiotomy/cardiac repair; (3) open cardiac massage; (4) clamping aorta; and (5) control of pulmonary hilum. A high-fidelity simulator with beating-heart technology was built. The previously validated Objective Structured Assessment of Technical Skills (OSATS) was adapted to create the "EDT-OSATS" which assessed performance along several domains: (1) Surgical technique (key steps); (2) general skills; and (3) global rating. A pilot test was performed to compare board-certified trauma surgeons (i.e., Experts) with categorical general surgery interns (i.e., Novices). Each subject received preparatory materials, completed a presimulation quiz, performed a videotaped procedure on the EDT simulator, and completed a postmodule survey. Two independent raters scored performances using the EDT-OSATS. Groups were compared in descriptive and unadjusted analyses. We hypothesized that our EDT simulation module would distinguish between expert vs novice performance and improve trainee confidence.

SETTING

Simulation laboratory at Massachusetts General Hospital in Boston, MA.

PARTICIPANTS

Trauma surgeons (Experts, n = 6) and categorical general surgery interns (Novices, n = 8).

RESULTS

Experts scored significantly higher than Novices on nearly all components of the EDT-OSATS, including: (1) surgical technique: pericardiotomy (4.2 vs 3.4, p = 0.040), cardiac massage (3.6 vs 2.4, p = 0.028), clamping aorta (4.1 vs 3.3, p = 0.035), control of pulmonary hilum (4.8 vs 3.4, p < 0.001); (2) general skills: time/motion (4.1 vs 2.9, p = 0.011), knowledge and handling of instruments (4.3 vs 3.1, p = 0.004), and (3) global rating (3.9 vs 2.9, p = 0.026). There was no statistical difference between groups on opening chest/rib spreader utilization (3.8 vs 3.3, p = 0.352) or procedure time (204sec vs 227sec, p = 0.401), though Experts scored numerically higher than Novices on every measure. Novices reported significantly increased confidence after the simulation (3.1 vs 1.4, p = 0.001). Ninety-three percent (13/14) of participants found the simulator realistic.

CONCLUSIONS

Our novel high-fidelity beating-heart EDT simulator is realistic and improves trainee confidence in this low-frequency, high-stakes emergency procedure. The EDT-OSATS tool differentiates between performances of experienced surgeons vs novice trainees on the beating-heart simulator. This training module and accompanying assessment instrument hold promise as a learning tool for clinicians who may perform emergency department thoracotomy.

New Challenges for a Core Procedure: Development of a Faculty Workshop for Skills Maintenance for Abdominal Hysterectomy

OBJECTIVE

To describe the development of a low-cost educational module for OB/GYN faculty skills maintenance for total abdominal hysterectomy (TAH), a low frequency core procedure in obstetrics and gynecology.

DESIGN

After review of existing educational tools and utilization of a modified Delphi method to establish consensus regarding key procedural components for skills maintenance, a 2-hour workshop was developed to review knowledge and participate in a simulation focused on the critical steps in performing TAH. An expert in TAH delivered a lecture highlighting important surgical considerations. Participants then rotated through simulation stations for critical steps in TAH: dissecting the bladder, identifying the ureter, and closing the cuff. Knowledge gains were assessed with a written pre- and posttest. Consecutive focus groups were conducted with participants on effectiveness of the workshop, and suggestions for improvement. Ideas identified in the first focus group were incorporated into the second workshop.

SETTING

Massachusetts General Hospital, an academic tertiary care facility with a single Obstetrics and Gynecology faculty group, located in Boston, Massachusetts.

PARTICIPANTS

Eligible participants were recruited via email from full time specialists in General Obstetrics and Gynecology at Massachusetts General Hospital. Of the 25 eligible gynecology faculty subjects, 22 participated (88%).

RESULTS

On pre or post-test comparison, 70% of participants scored higher on the posttest, demonstrating an increase in knowledge of critical TAH surgical steps. Focus group analyses identified the need for increased review and training demonstrations of TAH, and recommended continued offering of the workshop.

CONCLUSIONS

Based on focus group responses and pre or posttest comparisons, the workshop was deemed feasible and enhanced short-term learning. Future directions include utilizing more challenging anatomic models and simulation scenarios and optimizing integration of expert demonstration and individualized coaching, as well as identifying regionally tailored surgical workshop programming.

Identifying and Managing Intraoperative Arrhythmia: A Multidisciplinary Operating Room Team Simulation Case

Introduction

Resuscitation of a critically ill patient is challenging for both novice learners and experienced health care providers. During a critical event, not only is it important to identify the correct underlying diagnosis, it is equally crucial that the appropriate Advance Cardiac Life Support algorithm, medications, and defibrillator modality are implemented. This scenario features a 56-year-old female who presents for excisional biopsy of an inguinal lymph node to evaluate lymphadenopathy concerning for lymphoma. Intraoperatively, she goes into cardiopulmonary arrest. Participants must identify and manage three different scenarios: (1) ventricular fibrillation, (2) unstable ventricular tachycardia, and (3) bradycardia, including the use of the defibrillator.

Method

Weekly simulation sessions were conducted in the in situ simulation operating room at Massachusetts General Hospital. Surgical residents, anesthesiology residents, nurses, and surgical technicians participated in a multidisciplinary operating room team. Each approximately 60-minute session included an orientation, the case, and the debriefing. Equipment included a simulation operating room with general surgery supplies, general anesthesia equipment, a high-fidelity SimMan patient simulator, a code cart, and a defibrillator.

Results

Ninety-one multidisciplinary participants completed this scenario from September to December 2015. Participants reported that the scenario was applicable to their clinical practice (96%), promoted teamwork skills (88%), and encouraged interprofessional learning (94%).

Discussion

Intraoperative cardiac arrest is a devastating event that can result in poor patient outcomes if the care team is not thoroughly prepared for crisis management. This simulation case scenario was implemented to train multidisciplinary learners in the identification and management of such an event.