Administrative considerations when implementing ACS/APDS Skills Curriculum

Template for a program tailored ACS/APDS phase 1 curriculum: From needs assessment to implementation

BACKGROUND

The Phase 1 ACS/APDS skills curriculum standardizes intern training. Despite this, institutional implementation varies and is nationally low. We aimed to use Kern's six-steps to tailor this to our program, providing a framework to improve implementation.

METHODS

Problem identification and general needs assessment were performed. Targeted needs assessment (TNA) of incoming interns ('interns'), current residents, and attendings determined perceived importance of skills and intern's previous experience and confidence. Educational strategies were developed. Learner knowledge was assessed before and after modules, deficiencies identified enabled employment of active learning strategies. Modular and curricular evaluations were completed.

RESULTS

TNA determined all interns had been taught knot tying and suturing, and were most confident with suturing, knot tying, and urethral catheterization. Educational strategies included simulation and lectures. Evaluations demonstrated improvement in test scores (pre-v post-) and skills confidence on curricula completion.

CONCLUSION

Our framework utilizes institutional resources and expertise while focusing on determining existing knowledge, skill, and technical deficiencies of learners. This approach demonstrated improvement in knowledge and confidence, and could improve implementation rates of the Phase 1 curriculum.

Institution-specific utilization of the American College of Surgeons/Association of Program Directors operative skills curriculum: From needs assessment to implementation

BACKGROUND

The American College of Surgeons/Association of Program Directors in Surgery operative skills curriculum standardizes training. However, simulation resources are variable with curriculum implementation institution dependent. Our aim was to use Kern's six steps of curricular development to demonstrate how to tailor the American College of Surgeons/Association of Program Directors in Surgery Phase 2 curriculum to program specific needs.

METHODS

Problem identification and general needs assessment was performed. Targeted needs assessment of general surgery residents and attendings was conducted to determine perceived importance of operative skills and residents' confidence with these skills and attendings perceptions of deficiencies in technical skills using the Objective Structured Assessment of Technical Skills criteria. Educational strategies were developed dependent on program resources. The program was piloted between 2018 to 2019 and implemented in the 2019 to 2020 academic year. Assessment of resident technical skills and resident or faculty teaching skills was performed for each session. Resident confidence with procedures was assessed using the Zwisch scale before and after modules. Curricular evaluations were completed by residents after each module.

RESULTS

The previous curriculum did not comprehensively cover Phase 2 modules and was not tailored to the needs of residents. Targeted needs assessment revealed differences in prioritization of learning for techniques by seniority (most important operation for faculty: laparoscopic cholecystectomy, postgraduate year 4 and 5: laparoscopic partial colectomy, interns: open inguinal/femoral hernia repair). Faculty identified technical skills on which to focus (ie, interns' knowledge of a specific procedure, postgraduate year 4 and 5 flow of the operation and forward planning). Educational strategies employed included wet and dry lab simulations and online materials. Residents reported increased procedural confidence after curriculum completion.

CONCLUSION

This comprehensive implementation of the American College of Surgeons/Association of Program Directors in Surgery Phase 2 skills curriculum effectively used resources and expertise of an institution and focused on the knowledge and technical deficiencies of the target learners. Improvement in learner confidence was demonstrated by this approach.

Feasibility and Perceived Usefulness of Using Head-Mounted Cameras for Resident Video Portfolios

BACKGROUND

There is limited guidance on how to longitudinally administer simulation materials or to incorporate video recordings into assessment portfolios of simulated surgical skills.

MATERIALS AND METHODS

We launched a longitudinal weekly simulation curriculum for PGY1-PGY3 surgical residents based on the ACS/APDS Curriculum. Residents underwent monthly objective structured assessment of technical skills (OSATS) while wearing head-mounted cameras. Videos of OSATS performance accrued into individual online video portfolios. Residents were surveyed about their attitudes toward video recording.

RESULTS

Twenty-seven general surgical residents participated, completing 161 OSATS encompassing 11 distinct skills and generating 258 videos of simulated skills performance. The overall survey response rate was 88%. Residents viewed the curriculum favorably overall, and 36.4% of residents accessed their videos. Of those who did not watch their videos, 78.6% cited not having enough time, whereas 28.6% did not think the videos would be useful. Over 95% of surveyed residents expressed interest in having a video library of attending-performed procedures, 59.1% were interested in having their own operations recorded, and 45.5% were interested in video-based coaching.

CONCLUSIONS

Residents viewed longitudinal administration of the ACS/APDS Curriculum positively. Although video recording in simulation is feasible, resident interest may be higher for intraoperative recordings than for simulated skills.

"I got it on Ebay!": cost-effective approach to surgical skills laboratories

BACKGROUND

Surgical education is witnessing a surge in the use of simulation. However, implementation of simulation is often cost-prohibitive. Online shopping offers a low budget alternative. The aim of this study was to implement cost-effective skills laboratories and analyze online versus manufacturers' prices to evaluate for savings.

MATERIALS AND METHODS

Four skills laboratories were designed for the surgery clerkship from July 2014 to June 2015. Skills laboratories were implemented using hand-built simulation and instruments purchased online. Trademarked simulation was priced online and instruments priced from a manufacturer. Costs were compiled, and a descriptive cost analysis of online and manufacturers' prices was performed. Learners rated their level of satisfaction for all educational activities, and levels of satisfaction were compared.

RESULTS

A total of 119 third-year medical students participated. Supply lists and costs were compiled for each laboratory. A descriptive cost analysis of online and manufacturers' prices showed online prices were substantially lower than manufacturers, with a per laboratory savings of: $1779.26 (suturing), $1752.52 (chest tube), $2448.52 (anastomosis), and $1891.64 (laparoscopic), resulting in a year 1 savings of $47,285. Mean student satisfaction scores for the skills laboratories were 4.32, with statistical significance compared to live lectures at 2.96 (P < 0.05) and small group activities at 3.67 (P < 0.05).

CONCLUSIONS

A cost-effective approach for implementation of skills laboratories showed substantial savings. By using hand-built simulation boxes and online resources to purchase surgical equipment, surgical educators overcome financial obstacles limiting the use of simulation and provide learning opportunities that medical students perceive as beneficial.

Learning curve patterns generated by a training method for laparoscopic small bowel anastomosis

Background

The identification of developmental curve patterns generated by a simulation-based educational method and the variables that can accelerate the learning process will result in cost-effective training. This study describes the learning curves of a simulation-based instructional design (ID) that uses ex vivo animal models to teach laparoscopic latero-lateral small bowel anastomosis.

Methods

Twenty general surgery residents were evaluated on their performance of laparoscopic latero-lateral jejuno-jejunal anastomoses (JJA) and gastro-jejunal anastomoses (GJA), using swine small bowel and stomach on an endotrainer. The ID included the following steps: (1) provision of references and videos demonstrating the surgical technique, (2) creation of an engaging context for learning, (3) critical review of the literature and video on the procedures, (4) demonstration of the critical steps, (5) hands-on practice, (6) in-action instructor’s feedback, (7) quality assessment, (8) debriefing at the end of the session, and (9) deliberate and repetitive practice. Time was recorded from the beginning to the completion of the procedure, along with the presence or absence of anastomotic leaks.

Results

The participants needed to perform 23.8 ± 6.96 GJA (12–35) and 24.2 ± 6.96 JJA (9–43) to attain proficiency. The starting point of the learning curve was higher for the GJA than for the JJA, although the slope and plateau were parallel. Further, four types of learning curves were identified: (1) exponential, (2) rapid, (3) slow, and (4) no tendency. The type of pattern could be predicted after procedure number 8.

Conclusions

These findings may help to identify the learning curve of a trainee early in the developmental process, estimate the number of sessions required to reach a performance goal, determine a trainee’s readiness to practice the procedure on patients, and identify the subjects who lack the innate technical abilities. It may help motivated individuals to become reflective and self-regulated learners. Moreover, the standardization of the ID may help to measure the effectiveness of learning strategies and make comparisons with other educational strategies.

Unanticipated teaching demands rise with simulation training: strategies for managing faculty workload

INTRODUCTION

Using simulation to teach and assess learners represents a powerful approach to training, but one that comes with hidden costs in terms of faculty time, even if programs adopt existing curricula. Some simulators are built to be used independently by learners, but much of the surgical simulation curricula developed for cognitive and psychomotor tasks requires active faculty involvement and low learner-to-faculty teaching ratios to ensure sufficient practice with feedback. The authors hypothesize that the added teaching demands related to simulation have resulted in a significant financial burden to surgery training programs. To date, the effect of simulation-based training on faculty workload has not been estimated objectively and reported in the literature.

METHODS

To test their hypothesis, the authors analyzed data from 2 sources: (1) changes over time (2006-2014) in formal teaching hours and estimated faculty costs at the University of Minnesota, General Surgery Department and (2) a 2014 online survey of general surgery program directors on their use of simulation for teaching and assessment and their perceptions of workload effects.

RESULTS

At the University of Minnesota, the total number of hours spent by department faculty in resident and student simulation events increased from 81 in annual year 2006 to 365 in annual year 2013. Estimated full-time equivalent faculty costs rose by 350% during the same period. Program directors (n = 48) of Association of Program Directors in Surgery reported either a slight (60%) or a significant (33%) increase in faculty workload with the advent of simulation, and moderate difficulty in finding enough instructors to meet this increase. Calling upon leadership for support, using diverse instructor types, and relying on "the dedicated few" represent the most common strategies.

CONCLUSION

To avoid faculty burnout and successfully sustain faculty investment in simulation-based training over time, programs need to be creative in building, sustaining, and managing the instructor workforce.

In-training assessments used in the United States surgical residency programmes

INTRODUCTION

Quality and safety concerns have always been the impetus for evaluating surgical competence. This paper provides a focused overview of key historical events that lead to the development and implementation of surgical training standards and competency assessments in the United States.

METHODS

Focused review of surgical literature.

RESULTS

The following events were found to correlate with the development and implementation of training standards and competency assessments: (i) The Flexner Report issued in 1910; (ii) The American Medical Association's 1928 endorsement of the 'Essentials of Approved Residencies and Fellowships'; and (iii) The formation of several major surgical organizations - American College of Surgeons (1913), American Board of Surgery (1937), Residency Review Committee for Surgery (1950) and Association of Program Directors in Surgery (1966).

DISCUSSION

The process by which competence is assessed in the US surgical training programmes is multifactorial and heavily linked to the structure and function of several national organizations in surgery and medicine.

Evaluation of a novel laparoscopic simulation laboratory curriculum

BACKGROUND

Simulation training may be an invaluable tool in training and assessing surgeons. Questions remain regarding its true impact on acquisition of laparoscopic skills and their transferability to the operating room.

METHODS

Twenty general surgery residents completed 5 exercises: bean drop/rope drill, foam cutting/endoloop, checkerboard, endostitch, and suturing/intracorporeal knot. After baseline scoring, residents had 3 weeks to practice before re-testing. Statistical analysis of objective and subjective scores included the signed rank test or paired t-test, the Kruskal-Walis test, the McNemar test, and the Global Operative Assessment of Laparoscopic Skills (GOALS). P < .05 was considered significant.

RESULTS

Five postgraduate year (PGY) 4 and 15 PGY 1-3 residents completed the curriculum. When compared with baseline, overall scores improved (P < .05) in all categories except for the bean drop/rope drill, which improved on objectively measured tasks only (dropped beads and beads threaded). The foam cutting/endoloop task was mastered by a majority of trainees at baseline leaving no room for improvement.

CONCLUSION

The results of this study reinforce that simulation leads to improvement in laparoscopic skills and that our curriculum is a valid educational tool. Further studies are needed to validate whether this results in improved skills in the operating room.

Development and implementation of an interactive, objective, and simulation-based curriculum for general surgery residents

OBJECTIVE

A Steering Committee of residents and faculty initiated a systematic approach to curriculum development, focusing on competency-based education and emphasizing both resident and faculty engagement in the didactic process.

SETTING

Integrated General Surgery Residency Program at the University of Connecticut School of Medicine, Farmington, Connecticut.

PARTICIPANTS

Postgraduate year (PGY) 1 through 5 general surgery categorical and preliminary residents.

METHODS

A Core Curriculum consisting of 45-minute blocks and 2.5 hours of resident time per week was developed by a steering committee composed of faculty and residents. Each block is assigned a faculty and resident moderator, and has defined competency and knowledge-based objectives. An anonymous online evaluation tool collected residents' perceptions of value and satisfaction with the curriculum utilizing 15 5-point Likert items focusing on conferences, objectives, preparation, and quality of presentations, and materials. Measures were taken at the close of the previous academic year (baseline) and at 6 months and 1 year after implementation. The analysis focused on the percent responding in the 2 highest Likert categories (good/excellent, almost always/always, agree/strongly agree). The resulting dichotomous outcomes were compared with time point using χ(2)-tests of proportion; Kruskal-Wallis statistic was also used to compare the full distribution of responses. All analyses were done using SPSS v. 14 with α = 0.05.

RESULTS

One hundred two surveys were completed on-line (42 at baseline, 38 at 6 months, and 22 at 1 year). All 15 items showed increases from baseline to 1-year follow-up; 9 of the 15 were statistically significant with conferences and presentation quality and interaction showing the greatest improvement.

CONCLUSIONS

Resident satisfaction with the core curriculum, and their self-reported clinical and academic abilities showed improvement after a systematic collaborative faculty-resident approach to curriculum development and implementation.

What is the cost associated with the implementation and maintenance of an ACS/APDS-based surgical skills curriculum?

PURPOSE

The purpose of this study was to evaluate the cost associated with the American College of Surgery (ACS)/Association of Program Directors in Surgery (APDS)-based surgical skills curriculum (SSC) within a general surgery residency program.

METHODS

The Penn Surgical Simulation Center (PSSC) of the University of Pennsylvania was established by the Department of Surgery during the 2006-2007 academic year and became a Level-I ACS Accredited Education Institute in 2008. Each academic year, 38 junior residents are assigned to a 4-week dedicated simulation rotation based on the ACS/APDS-based SSC. In conjunction with voluntary participation by faculty, a salaried educational fellow is responsible for maintaining the schedule and administering the surgical skills training modules. The costs associated with the ACS/APDS-based SSC were divided in initial implementation capital expenses and annual operational maintenance expenses.

RESULTS

The overall capital expenditures associated with the implementation of the curriculum were $4.204 million. These costs included the purchase of low and high-fidelity simulation equipment and initial construction costs to renovate a previous operating room (OR) and recovery suite into the Penn Medicine Clinical Simulation Center (PMCSC) which has housed the PSSC since 2008. The annual operational expenses are $476,000 and include the salary for the educational fellow, disposables, and other supplies, and the PMCSC average student fees. The annual cost per resident for the 4-week dedicated simulation rotation is $12,516. This figure does not include the average cost for teaching efforts including the simulation teaching per participating faculty member which is $30,000 in Relative Teaching Value Units per year.

CONCLUSIONS

The expenditures associated with the implementation and maintenance of the ACS/APDS-based surgical skills curriculum in a surgical residency program are significant. This center's experience might be useful to programs deciding on more cost-effective means of implementing the ACS/APDS-SSC into their training.