Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

OBJECTIVES

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency.

DESIGN

A participatory action research model with a mixed methods approach.

SETTING

This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department.

PARTICIPANTS

Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention.

RESULTS

The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4.

CONCLUSION

Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Specialist training: workplace-based assessments impact on teaching, learning and feedback to support competency-based postgraduate programs

BACKGROUND

Workplace-based assessments (WBAs) are part of a competency-based curriculum where training progression is dependent on the achievement of defined competencies in a real-world clinical environment. There is a significant literature gap on the impact of WBAs implemented in resource constrained countries and their contextual challenges. This study aimed to examine the use, impact, and educational context of WBAs in South African medical specialist training programs drawing on perspectives from both trainees and trainers to identify educational challenges and propose effective solutions.

METHODS

A mixed methods national electronic survey was conducted with specialist medical trainees and supervising trainers from all eight specialist training institutions in South Africa involving 16 specialities. The survey responses were voluntary and anonymous. The survey was closed after seven months when data saturation was achieved. Descriptive statistical analysis was performed using SPSS Version 27 (SPSS Inc, 2012, Chicago, IL) for the quantitative analysis. The thematic coding framework for the qualitative analysis was facilitated by NVivo Version 12 software.

RESULTS

There were 108 ethnically diverse supervising trainers and 248 specialist trainees' survey respondents. Across the 16 medical specialities, 45% of the respondents were using WBAs. Despite contextual resource and staff challenges, this study found that WBAs had a positive impact to Kirkpatrick level 2 in providing actionable feedback to improve competency. WBA users had a significantly higher rating for trainee supervision (p < 0.01), general quality of feedback on trainee competence (< 0.01) and the specialist training program (p = 0.03) compared to WBA non-users. They also had a higher rating for the assessment of the trainee as a professional (p < 0.01); scholar (p < 0.01); communicator (p < 0.01); collaborator (p = 0.001) and leader/manager (p < 0.001) based on the AfriMEDS competency framework. Racism, sexism and favouritism were challenges that negatively affected the training programs.

CONCLUSION

Overall, this study reports that the use of WBAs had a substantially favourable impact on teaching, learning, feedback and supports a competency-based approach to specialist training programs. Addressing the contextual concerns that negatively impact training; training the trainees and trainers about their relationship, roles and responsibilities; and focusing on a trainee-centred, inclusive and empowering teaching approach will help further enhance its effectiveness.

The Use, Effectiveness, and Impact of Workplace-Based Assessments on Teaching, Supervision and Feedback Across Surgical Specialties

OBJECTIVE

To investigate the use and effectiveness of Workplace-based assessments (WBAs) and their impact on training, feedback, and perioperative teaching in surgical training programs.

DESIGN

A mixed methods cross-sectional, national electronic survey was conducted with surgical trainees and consultant trainers.

SETTINGS

The trainees and supervising faculty were from all 8 major surgical training universities across 11 surgical disciplines in South Africa.

PARTICIPANTS

A total of 108 surgical trainees and 41 supervising consultant trainers from 11 surgical disciplines across 8 surgical training universities responded to the survey.

RESULTS

The most significant educational gap identified by both the surgical trainees and trainers across all surgical disciplines was inadequate perioperative feedback. A third of the respondents were currently using workplace-based assessments. The WBA users (both trainees and trainers) had a higher rating for the general quality of surgical feedback than WBA nonusers (p = 0.02). WBA users also had a higher rating for the general quality of feedback given to trainees on their skills and competence (p = 0.04) and a higher rating for trainee supervision (p = 0.01) and the specialist training program overall (p = 0.01). The WBA users also had a higher rating for the assessment of competencies such as the trainee as an effective communicator (p < 0.01) and collaborator (p = 0.04).

CONCLUSION

This study found that the use of WBAs enhances the quality and effectiveness of feedback in surgical training programs. We also found that the use of WBAs enhance perioperative teaching and learning and improves the assessment of relational competencies. This was also associated with high ratings for the quality of trainee supervision. Faculty and trainee development, strengthening the trainee-trainer relationship, and integrating iterative stakeholder feedback could help realize the full potential of WBAs to augment surgical training across disciplines.

South African Eye Study (SAES): ethnic differences in central corneal thickness and intraocular pressure

PurposeGlaucoma is the leading cause of irreversible blindness worldwide. South Africa has a diverse population but there is a lack of published ethnic specific normative data. The purpose of the study is to determine the distribution of intraocular pressure (IOP) and central corneal thickness (CCT) values in a multi-ethnic South African population and to determine additional systemic and ocular factors that influence IOP and CCT.Patients and methodsThis cross-sectional study included a total of 402 participants with 706 eyes aged 18-94 years. Participants underwent a standardized interviewer-administered questionnaire for risk factor assessment followed by a full ophthalmic examination. The averages of six IOP readings were measured with an Icare PRO tonometer and CCT was measured with a Pentacam.ResultsThe mean CCT readings in the African, Mixed ethnicity, and Caucasian participants were 514.77±31.86, 531.77±35.17, and 549.97±30.51 μm (P<0.001). The mean IOP in the African, Mixed ethnicity, and Caucasian participants were 15.51±2.49, 15.09±2.12, and 15.13±2.53 mm Hg (P=0.07). Africans had significantly higher IOP than Mixed ethnicity (P=0.034) and Caucasians (P=0.011). Hypertensives had a higher IOP (P=0.03). Age and pseudophakia were associated with a lower IOP (P<0.001) and higher CCT (P<0.001). There was a strongly positive correlation between CCT and IOP (β=0.021; P<0.001).ConclusionsIn the South African Eye Study (SAES), Africans had the thinnest corneas and highest IOP followed by Mixed ethnicity and Caucasians. Including systemic and ocular factors that influence IOP specific to each population and ethnic group, will lead to a more accurate clinical risk stratification in glaucoma management.