Implementation and evaluation of a comprehensive proficiency-based curriculum in an advanced, minimally invasive procedure: a multi-institutional Canadian experience

Transforming the "SEAD": Evaluation of a Virtual Surgical Exploration and Discovery Program and its Effects on Career Decision-Making

OBJECTIVE

The abrupt cessation of in-person education due to the COVID-19 pandemic has made it difficult for preclerkship students to explore a career in surgery. To supplement the lack of exposure, the Surgical Exploration and Discovery (SEAD) program was transitioned to an entirely virtual format. This study aims to describe the virtual SEAD program and evaluate its effectiveness as a career decision-making (CDM) intervention.

DESIGN

The week-long program was delivered on Microsoft Teams, featured 11 surgical specialties, and comprised four activities: live demonstrations, virtual operating room observerships, career talks, and technical skills workshops. The program was evaluated using the four levels of the Kirkpatrick model: (1) reactions, (2) knowledge, (3) CDM behaviors - assessed using the Career Decision-making Difficulties Questionnaire (CDDQ) - and (4) results. The latter was indirectly assessed using CDDQ scores from an in-person SEAD program, where lower CDDQ scores indicate less difficulty with CDM.

SETTING

Faculty of Medicine at the University of Ottawa in Ontario, Canada.

PARTICIPANTS

Forty pre-clerkship students (27 first and 13 second year students) at the University of Ottawa RESULTS: Level 1: 97.5% of participants rated the program as good or very good. Live demonstration and technical skills workshops were the highest rated activities. Level 2: participants' scores on knowledge-based questions about a surgical career significantly increased following the program (pre: 9/25 vs post: 15/25, p = 0.008). Level 3: overall mean CDDQ scores (±SD) decreased difficulties with significantly following the program (pre: 45.6 ± 10.5 vs post: 38.8 ± 10.9, p < 0.001), which indicates decreased CDM difficulties. Level 4: Except for one sub-category, the difference in mean CDDQ scores between the virtual and in-person programs were not significantly different.

CONCLUSION

The program received the positive reactions and significantly increased participants' knowledge. The change in CDDQ scores following the virtual program suggests it may reduce career decision-making difficulties in the short-term. In-person surgical exposure remains important; however, a hybrid model may be valuable in resource limited settings. WC: 300.

International Implementation of a PROficiency based StePwise Endovascular Curricular Training (PROSPECT) in Daily Practice

OBJECTIVE

A randomised controlled trial (RCT) showed superior real life endovascular performance of trainees after PROficiency based StePwise Endovascular Curricular Training (PROSPECT) compared with traditional training. This study aimed to determine whether PROSPECT can be implemented routinely in surgical training (registry) and to identify the associated effects, facilitators, and barriers to implementation.

METHODS

This was a multicentre prospective registry to evaluate real life PROSPECT implementation effects. Cognitive ability was tested with a Multiple Choice Question test (MCQ) (max. score 20 points), while technical skills were assessed with a Global Rating Scale (GRS) (max. score 55 points), Examiner's Checklist (max, score 85 points), and simulator metrics. A cross sectional anonymous survey for both trainees and faculty evaluated the programme's implementation.

RESULTS

The PROSPECT registry was implemented in five centres in four countries over a period of three years. Only 17 of 48 starting trainees completed PROSPECT, resulting in a dropout rate of 65%. Participants who completed the programme showed significant improvement in the GRS (median 26 vs. 44, p = .018) and Examiner's Checklist (median 53.5 vs. 80, p = .028) after the programme. The survey was completed by 13/17 faculty members (76%) and 38/57 trainees who participated either in the initial PROSPECT RCT or the registry (67%). PROSPECT was perceived to provide more endovascular training opportunities, but non-availability of logistical/technical support, lack of training time within the working schedule, and little trainee motivation hindered broad implementation.

CONCLUSION

PROSPECT has a significant effect on performance, and trainees and faculty agree that PROSPECT is a valuable addition to surgical training. However, external and internal factors pose significant barriers to integration of this simulation based programme into daily practice leading to a high dropout rate. To integrate PROSPECT into contemporary surgical training, it should be a prerequisite to treating real patients and protected training time should be provided, combined with accessible support.

Development of an International Standardized Curriculum for Laparoscopic Sleeve Gastrectomy Teaching Utilizing Modified Delphi Methodology

BACKGROUND

The performance of laparoscopic sleeve gastrectomy has increased markedly to become the single-most performed bariatric surgical procedure globally. To date, a means of standardized trainee teaching has not been developed. The aim of this study was to design a laparoscopic curriculum for trainees of bariatric surgery utilizing modified Delphi consensus methodology.

METHODS

A panel of surgeons was assembled to devise an academic framework of technical, non-technical and cognitive skills utilized in the performance of laparoscopic sleeve gastrectomy. The panel invited 18 bariatric surgeons experienced in laparoscopic gastrectomy from 11 countries to rate the items for inclusion in the curriculum to a predefined level of agreement.

RESULTS

A consensus of experts was achieved for 24 of the 30 proposed elements for inclusion within the first round of the curriculum Delphi panel. All components pertaining to anatomical knowledge, peri-operative considerations and non-technical items were accepted. A second round further examined six statements, of which three were accepted. Agreement of the panel was reached for 27 of the cognitive, technical and non-technical components after two rounds. Three statements found no consensus.

CONCLUSIONS

Utilizing modified Delphi methodology, a curriculum outlining the most important components of teaching the procedure of laparoscopic sleeve gastrectomy, has been determined by a consensus of international experts in bariatric surgery. The curriculum is suggested as a standard in proficiency-based training of this procedure. It forms a generic template which facilitates individual jurisdictions to perform content validation, adapting the curriculum to local requirements in teaching the next generation of bariatric surgeons.

An Evaluation of the Surgical Foundations Curriculum: A National Study

BACKGROUND

Canadian Surgical Foundations (SF) residency programs transitioned to competency-based medical education in 2018. It is unknown how well the SF curriculum prepares surgery residents to recognize and manage common perioperative patient presentations. We sought to evaluate the national SF curriculum using the Kirkpatrick model of curriculum evaluation.

METHODS

We administered online surveys to 300 first-year English-speaking surgery residents across Canada to assess self-reported confidence in recognizing and managing 7 common perioperative patient presentations at 3 timepoints: pre-SF (July 2019), mid-SF (December 2019), and post-SF (May 2020). We conducted multistation simulation-based objective structured clinical examinations for surgery residents at our institution pre-SF (August 2019) and mid-SF (December 2019), and collected workplace-based assessment (WBA) data, including entrustment scores and narrative feedback, for 6 preselected entrustable professional activities (EPAs) (July 2019 to May 2020).

RESULTS

Fifty-five residents (18%) completed pre-SF, 31 (10%) completed mid-SF, and 52 (17%) completed post-SF surveys. Residents' confidence in recognizing 6 out of 7 patient presentations was high pre-SF and did not improve significantly during the SF curriculum except for recognizing poor glycemic control (p < 0.01). Residents' confidence in managing 7 out of 7 patient presentations improved significantly (p < 0.05). Objective structured clinical examinations performance did not change significantly between pre-SF and mid-SF (4 [3.5-4.5] vs 4 [3-4]; p = 0.28). Analysis of WBA data showed that residents received high entrustment scores from the start of the SF curriculum. Entrustment scores improved significantly during the SF curriculum for 2 out of 6 EPAs. Only 56% of WBA assessments had narrative feedback, 16% of which had somewhat constructive feedback.

CONCLUSION

Participation in the SF curriculum was associated with improved confidence of surgery residents in managing common perioperative patient presentations, and greater level of entrustment for some EPAs. Consideration should be given to further faculty development to increase the quantity and quality of narrative feedback in the SF curriculum.

Leadership and surgical training part 1: preparing to lead the way?

Every day surgeons lead teams on the wards, in clinics and operating theatres, but most trainees and some surgeons do not consider themselves as leaders. Leadership skills are increasingly important for surgeons, who need knowledge of organizational structure and policy, management strategy and team dynamics to deliver and improve health care in resource-constrained environments. The Royal Australasian College of Surgeons recognizes leadership as one of 10 core surgical competencies but leadership curricula within surgical training programmes are not well defined. There is limited opportunity for formal leadership training and development prior to becoming a consultant.

Identifying technical skills and clinical procedures in surgery for a simulation-based curriculum: a national general needs assessment

BACKGROUND

The efficacy of simulation-based training in surgical education is well known. However, the development of training programs should start with problem identification and a general needs assessment to ensure that the content is aligned with current surgical trainee needs. The objective of the present study is to identify the technical skills and clinical procedures that should be included in a simulation-based curriculum in general surgery.

METHODS

A national, three-round Delphi process was conducted to achieve consensus on which technical skills and clinical procedures should be included in a simulation-based curriculum in general surgery. In total, 87 key opinion leaders were identified and invited to the study.

RESULTS

Round 1 of the Delphi process had a response rate of 64% (56/87) and a total of 245 suggestions. Based on these suggestions, a consolidated list of 51 technical skills or clinical procedures was made. The response rate in Delphi round 2 was 62% (54/87) resulting in a pre-prioritized order of procedures for round 3. The response rate in Delphi round 3 was 65% (35/54). The final list included 13 technical skills and clinical procedures. Training was predominantly requested within general open surgical skills, laparoscopic skills, and endoscopic skills, and a few specific procedures such as appendectomy and cholecystectomy were included in the final prioritized list.

CONCLUSION

Based on the Delphi process 13 technical skills and clinical procedures were included in the final prioritized list, which can serve as a point of departure when developing simulation-based training in surgery.

Development and assessment of a simulation-based curriculum in pediatric surgical education: Conventional wisdom and lessons learned from the national training program in France

Traditional training in pediatric surgery has focused on knowledge acquisition and technical skills. Although recognized as important, the skills relating to communication, collaboration, leadership, management, health advocacy and professionalism have received less formal attention. Recognition of the importance of these softer skills in combination with the greater use of simulation in education prompted us to develop a national simulation-based curriculum in pediatric surgery. This paper describes the development, implementation and three-year follow-up of the mandatory, simulation-based curriculum for pediatric surgery residents in France. We review the available literature on the topic and present our results.