The usefulness of work-based assessments in higher surgical training: A systematic review

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

OBJECTIVES

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency.

DESIGN

A participatory action research model with a mixed methods approach.

SETTING

This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department.

PARTICIPANTS

Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention.

RESULTS

The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4.

CONCLUSION

Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

There is a need for a paradigm shift in laparoscopic surgical training: results of a nationwide survey among teaching hospitals in Switzerland

BACKGROUND

Surgical training curricula have changed little over the past decades. Current advances in surgical techniques, especially in minimally invasive surgery, as well as the rapidly changing socioeconomic environment pose a major challenge for the training of young surgeons. The aim of this survey was to provide a representative overview of the surgical training landscape in Switzerland focusing on laparoscopic surgical training: How do department chairs of teaching hospitals deal with the above challenges, and what should a future training curriculum look like?

METHODS

This is a prospective, questionnaire-based, cross-sectional study among the heads of departments of all certified surgical teaching hospitals in Switzerland.

RESULTS

The overall response rate was 56% (48/86) and 86% (19/22) for tertiary centers. Two-thirds of the centers (32) organize themselves in training networks. Laparoscopic training courses are offered in 25 (52%) hospitals, mainly in tertiary centers. Self-training opportunities exist in 40 (83%) hospitals. In addition to commercial (27) and self-built (7) box trainers, high-fidelity trainers are available in 16 (33%) hospitals. A mandatory training curriculum exists in 7 (15%) facilities, and a training assessment is performed in 15 (31%) institutions. Thirty-two (65%) heads of departments indicated that residents have sufficient practical exposure in the operating room, but the ability to work independently with obtaining the specialist title is seen critically (71%). They state that the surgical catalog does not adequately reflect the manual skills of the resident (64%). The desire is for training to be restructured from a numbers-based to a performance-based curriculum (53%) and for tools to assess residents' manual skills (56%) to be introduced.

CONCLUSIONS

Department chairs stated that the existing curriculum in Switzerland does not meet the requirements of a modern training curriculum. This study highlights the need to create an improved, competency-based curriculum that ensures the training of a new generation of surgeons, taking into account the growing evidence of the effectiveness of state-of-the-art training modalities such as simulation or proficiency-based training.

Specialist training: workplace-based assessments impact on teaching, learning and feedback to support competency-based postgraduate programs

BACKGROUND

Workplace-based assessments (WBAs) are part of a competency-based curriculum where training progression is dependent on the achievement of defined competencies in a real-world clinical environment. There is a significant literature gap on the impact of WBAs implemented in resource constrained countries and their contextual challenges. This study aimed to examine the use, impact, and educational context of WBAs in South African medical specialist training programs drawing on perspectives from both trainees and trainers to identify educational challenges and propose effective solutions.

METHODS

A mixed methods national electronic survey was conducted with specialist medical trainees and supervising trainers from all eight specialist training institutions in South Africa involving 16 specialities. The survey responses were voluntary and anonymous. The survey was closed after seven months when data saturation was achieved. Descriptive statistical analysis was performed using SPSS Version 27 (SPSS Inc, 2012, Chicago, IL) for the quantitative analysis. The thematic coding framework for the qualitative analysis was facilitated by NVivo Version 12 software.

RESULTS

There were 108 ethnically diverse supervising trainers and 248 specialist trainees' survey respondents. Across the 16 medical specialities, 45% of the respondents were using WBAs. Despite contextual resource and staff challenges, this study found that WBAs had a positive impact to Kirkpatrick level 2 in providing actionable feedback to improve competency. WBA users had a significantly higher rating for trainee supervision (p < 0.01), general quality of feedback on trainee competence (< 0.01) and the specialist training program (p = 0.03) compared to WBA non-users. They also had a higher rating for the assessment of the trainee as a professional (p < 0.01); scholar (p < 0.01); communicator (p < 0.01); collaborator (p = 0.001) and leader/manager (p < 0.001) based on the AfriMEDS competency framework. Racism, sexism and favouritism were challenges that negatively affected the training programs.

CONCLUSION

Overall, this study reports that the use of WBAs had a substantially favourable impact on teaching, learning, feedback and supports a competency-based approach to specialist training programs. Addressing the contextual concerns that negatively impact training; training the trainees and trainers about their relationship, roles and responsibilities; and focusing on a trainee-centred, inclusive and empowering teaching approach will help further enhance its effectiveness.

Evaluating the perceptions of workplace-based assessments in surgical training: a systematic review

INTRODUCTION

Workplace-based assessments (WBAs) are intended to maximise learning opportunities in surgical training. There is speculation as to whether mandatory assessments in this form contribute to a tick-box culture. The objective of this review was to investigate surgical trainees' attitudes towards WBAs.

METHODS

This systematic review of qualitative studies was performed in accordance with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) statement. The literature was searched on the Medline®, Embase™, PubMed and Web of Science™ databases on 22 March 2022.

RESULTS

Sixteen studies were included in the review, mostly carried out on users of the Intercollegiate Surgical Curriculum Programme portfolio in the UK. Trainees felt that WBAs were educationally useful, providing opportunity for feedback, but this was overshadowed by a pressure to reach a set annual quota for WBAs and achieve high scores. Other themes included inaccurate recording of WBAs, the role of WBAs as formative or summative assessments, engagement and accessibility of trainers, and lack of time to complete WBAs.

CONCLUSIONS

Negative perceptions about WBAs were widespread among surgical trainees despite a recognition of their capacity to facilitate learning. This review supports the recent removal of the annual quota for WBAs in UK surgical training programmes.

Entrustable professional activities in postgraduate general surgery training: a scoping review protocol

INTRODUCTION

The competency-based medical education (CBME) movement continues to gain momentum in postgraduate physician training, resulting in increasing interest among surgical training programmes on how to implement it effectively. Entrustable professional activities (EPAs) were introduced to connect competencies (characteristics/abilities of learners) and the professional activities to be entrusted to them on qualification/graduation. Although reviews related to the field of general surgery have been published on specific aspects of CBME, for example, workplace-based assessment, there is a paucity of published guidance available for surgeon-educators to plan and implement CBME using an EPA framework. This scoping review aims to provide an overview of the current use of EPAs in general surgery.

METHODS AND ANALYSIS

This scoping review will be based on the original five-stage approach described by Arksey and O'Malley with the addition of protocol-specific items from the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols extension for scoping reviews. The primary research question focuses on describing how EPAs have been used in general surgery and where potential gaps remain that warrant further study. The review will include peer-reviewed journal publications and an extensive review of grey literature sources. The following databases will be searched for published literature from 1 January 2005 to 31 December 2022: PubMed; CINAHL, Africa-Wide Information, PsycInfo and ERIC (via EBSCOhost); Scopus; and SciELO (via Web of Science). Studies that describe the use of EPAs for curriculum design, teaching and/or assessment of competence in postgraduate general surgery training will be eligible for inclusion.

ETHICS AND DISSEMINATION

The institutional ethics board of the University of Cape Town has granted a waiver of formal approval requirement. The dissemination strategy includes publication of results in peer-reviewed journals, presentation at international conferences and presentation to relevant stakeholders as deemed appropriate.

Systematic review and meta-analysis comparing perioperative outcomes of emergency appendectomy performed by trainee vs trained surgeon

BACKGROUND

Appendectomy is a benchmark operation for trainee progression, but this should be weighed against patient safety and perioperative outcomes.

METHODS

Systematic literature review and meta-analysis comparing outcomes of appendectomy performed by trainees versus trained surgeons.

RESULTS

Of 2086 articles screened, 29 studies reporting on 135,358 participants were analyzed. There was no difference in mortality (Odds ratio [OR] 1.08, P = 0.830), overall complications (OR 0.93, P = 0.51), or major complications (OR 0.56, P = 0.16). There was no difference in conversion from laparoscopic to open surgery (OR 0.81, P = 0.12) and in intraoperative blood loss (Mean Difference [MD] 5.58 mL, P = 0.25). Trainees had longer operating time (MD 7.61 min, P < 0.0001). Appendectomy by trainees resulted in shorter duration of hospital stay (MD 0.16 days, P = 0.005) and decreased reoperation rate (OR 0.78, P = 0.05).

CONCLUSIONS

Appendectomy performed by trainees does not compromise patient safety. Due to statistical heterogeneity, further randomized controlled trials, with standardized reported outcomes, are required.

Implementing competency-based medical education in post-graduate ophthalmology training: Understanding key concepts and methodologies and overcoming challenges

The National Medical Council (NMC) has mandated that all post-graduate training shall be competency-based, following the principles of Competency. Based Medical Education (CBME). This aims to serve the dual purpose of standardizing ophthalmology post-graduate training and to overcome the shortcomings of the traditional curriculum by employing innovative and interactive teaching-learning and assessment techniques to produce competent ophthalmologists well equipped with clinical, surgical, and professional skills relevant to current times. CBME marks a paradigm shift from traditional curriculum methodologies and involves the introduction of several new concepts. This article aims to highlight the key principles of CBME and the various teaching-learning and assessment technique methodologies which can be employed for post-graduate training. It also highlights the various challenges that are likely to be faced in its implementation and measures to overcome them.