Trust, power and learning in workplace-based assessment: The trainee perspective

Development of a Nuclear Safety and Security Integration Assessment Tool for Research Reactors and Associated Facilities

ABSTRACT

Nuclear safety and security are essential elements of radiation protection. Integration of nuclear safety and security provides a means to identify conflict and synergy points. Research has not been performed to enable integrated practices at the facility level. A tool was developed through research to help staff and regulators assess the level of integration practiced within a research reactor. This tool aims to improve the identification of synergistic and conflict points. Eight criteria of nuclear safety and security integration were used to create the integration assessment tool: access control, transportation, emergency response, proper disposal of materials, testing and maintenance, defense in depth, training and education, and culture. The tool's final score can range from 0.0375 to 1, with a score of 1 indicating complete integration. The tool was used by research reactor staff to assess practiced integrative techniques. The testing and maintenance criterion scored the highest level of integration (0.84). Training and education and culture scored the lowest levels of integration (0.50). The areas with the highest scores identified points of actively practiced integration. In contrast, those areas with lower scores indicated a lack of integrative practices. The total integration score was 0.69. This tool determined that the facility practiced an adequate level of integration. By analyzing integration levels with this tool, a measurable standard of integrative practices can be employed to achieve improved radiation protection.

There is no "too small" for frequent workplace-based assessment: Differences between large and small residency programs in anesthesia when using a mobile application to assess EPAs

Background

A competency-based education approach calls for frequent workplace-based assessments (WBA) of Entrustable Professional Activities (EPAs). While mobile applications increase the efficiency, it is not known how many assessments are required for reliable ratings and whether the concept can be implemented in all sizes of residency programs.

Methods

Over 5 months, a mobile app was used to assess 10 different EPAs in daily clinical routine in Swiss anesthesia departments. The data from large residency programs was compared to those from smaller ones. We applied generalizability theory and decision studies to estimate the minimum number of assessments needed for reliable assessments.

Results

From 28 residency programs, we included 3936 assessments by 306 supervisors for 295 residents. The median number of assessments per trainee was 8, with a median of 4 different EPAs assessed by 3 different supervisors. We found no statistically significant differences between large and small programs in the number of assessments per trainee, per supervisor, per EPA, the agreement between supervisors and trainees, and the number of feedback processes stimulated. The average "level of supervision" (LoS, scale from 1 to 5) recorded in larger programs was 3.2 (SD 0.5) compared to 2.7 (SD 0.4) (p<0.05). To achieve a g-coefficient >0.7, at least a random set of 3 different EPAs needed to be assessed, with each EPA rated at least 4 times by 4 different supervisors, resulting in a total of 12 assessments.

Conclusion

Frequent WBAs of EPAs were feasible in large and small residency programs. We found no significant differences in the number of assessments performed. The minimum number of assessments required for a g-coefficient >0.7 was attainable in large and small residency programs.

Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback

The entrustment framework redirects assessment from considering only trainees' competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors' themes predominantly reflected skills related to patient presentations, while trainees' themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees' documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees' assimilation into their roles and formation of trusting relationships.

Gaining Respect and Mitigating Risk: A Qualitative Exploration of How New Attendings Navigate Interpersonal Relationships with Recent Resident Peers

Background

Physicians moving through training experience changes in personal and professional relationships, which can increase stress, uncertainty, and burnout. Social connection can be an important resource but can introduce complexity and conflict. This study aimed to explore how early-career attendings navigate and manage changing organizational and friendship roles with recent resident peers (near-peers) through this critical transition.

Methods

We conducted a reflexive thematic analysis of interviews with early-career attendings working with near-peers from the same institution where they trained. Twenty three of 27 (85%) eligible attendings from two United States health systems participated in semi-structured interviews between April and June 2023.

Results

Familiarity from working at the same institution where new attendings completed training made it more difficult to command authority. Early-career attendings at times struggled with insecurities about their ability to fulfill their new role and challenges from others in recognizing their new attending identity. These tensions could heighten emotions in the clinical setting and spill over into relationships with residents outside the workplace, impacting social lives and well-being. Early-career attendings engaged in strategies to manage the social realm of their transition with near-peers, including prioritizing their organizational role in the clinical setting and mitigating risks to their professional reputation by creating stronger boundaries between their personal and professional lives.

Conclusions

This study provides new insight into how attendings navigate changing personal and professional relationships with recent resident peers and offers strategies on how to manage the social realm of this liminal transition.

Assessment-Seeking Strategies:

Phenomenon: Competency-based medical education (CBME) relies on workplace-based assessment (WBA) to generate formative feedback (assessment for learning-AfL) and make inferences about competence (assessment of learning-AoL). When approaches to CBME rely on residents to initiate WBA, learners experience tension between seeking WBA for learning and for establishing competence. How learners resolve this tension may lead to unintended consequences for both AfL and AoL. We sought to explore the factors that impact both decisions to seek and not to seek WBA and use the findings to build a model of assessment-seeking strategy used by residents. In building this model we consider how the link between WBA and promotion or progression within a program impacts an individual's assessment-seeking strategy. Approach: We conducted 20 semi-structured interviews with internal medicine residents at Queen's University about the factors that influence their decision to seek or avoid WBA. Using grounded theory methodology, we applied a constant comparative analysis to collect data iteratively and identify themes. A conceptual model was developed to describe the interaction of factors impacting the decision to seek and initiate WBA. Findings: Participants identified two main motivations when deciding to seek assessments: the need to fulfill program requirements and the desire to receive feedback for learning. Analysis suggested that these motivations are often at odds with each other. Participants also described several moderating factors that impact the decision to initiate assessments, irrespective of the primary underlying motivation. These included resident performance, assessor factors, training program expectations, and clinical context. A conceptual framework was developed to describe the factors that lead to strategic assessment-seeking behaviors. Insights: Faced with the dual purpose of WBA in CBME, resident behavior in initiating assessment is guided by specific assessment-seeking strategies. Strategies reflect individual underlying motivations, influenced by four moderating factors. These findings have broad implications for programmatic assessment in a CBME context including validity considerations for assessment data used in summative decision-making including readiness for unsupervised practice.

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education

OBJECTIVES

The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency.

DESIGN

A participatory action research model with a mixed methods approach.

SETTING

This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department.

PARTICIPANTS

Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention.

RESULTS

The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4.

CONCLUSION

Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

The Next Era of Assessment: Building a Trustworthy Assessment System

Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.

Specialist training: workplace-based assessments impact on teaching, learning and feedback to support competency-based postgraduate programs

BACKGROUND

Workplace-based assessments (WBAs) are part of a competency-based curriculum where training progression is dependent on the achievement of defined competencies in a real-world clinical environment. There is a significant literature gap on the impact of WBAs implemented in resource constrained countries and their contextual challenges. This study aimed to examine the use, impact, and educational context of WBAs in South African medical specialist training programs drawing on perspectives from both trainees and trainers to identify educational challenges and propose effective solutions.

METHODS

A mixed methods national electronic survey was conducted with specialist medical trainees and supervising trainers from all eight specialist training institutions in South Africa involving 16 specialities. The survey responses were voluntary and anonymous. The survey was closed after seven months when data saturation was achieved. Descriptive statistical analysis was performed using SPSS Version 27 (SPSS Inc, 2012, Chicago, IL) for the quantitative analysis. The thematic coding framework for the qualitative analysis was facilitated by NVivo Version 12 software.

RESULTS

There were 108 ethnically diverse supervising trainers and 248 specialist trainees' survey respondents. Across the 16 medical specialities, 45% of the respondents were using WBAs. Despite contextual resource and staff challenges, this study found that WBAs had a positive impact to Kirkpatrick level 2 in providing actionable feedback to improve competency. WBA users had a significantly higher rating for trainee supervision (p < 0.01), general quality of feedback on trainee competence (< 0.01) and the specialist training program (p = 0.03) compared to WBA non-users. They also had a higher rating for the assessment of the trainee as a professional (p < 0.01); scholar (p < 0.01); communicator (p < 0.01); collaborator (p = 0.001) and leader/manager (p < 0.001) based on the AfriMEDS competency framework. Racism, sexism and favouritism were challenges that negatively affected the training programs.

CONCLUSION

Overall, this study reports that the use of WBAs had a substantially favourable impact on teaching, learning, feedback and supports a competency-based approach to specialist training programs. Addressing the contextual concerns that negatively impact training; training the trainees and trainers about their relationship, roles and responsibilities; and focusing on a trainee-centred, inclusive and empowering teaching approach will help further enhance its effectiveness.

"We just don't have the resources": Supervisor perspectives on introducing workplace-based assessments into medical specialist training in South Africa

BACKGROUND

South Africa (SA) is on the brink of implementing workplace-based assessments (WBA) in all medical specialist training programmes in the country. Despite the fact that competency-based medical education (CBME) has been in place for about two decades, WBA offers new and interesting challenges. The literature indicates that WBA has resource, regulatory, educational and social complexities. Implementing WBA would therefore require a careful approach to this complex challenge. To date, insufficient exploration of WBA practices, experiences, perceptions, and aspirations in healthcare have been undertaken in South Africa or Africa. The aim of this study was to identify factors that could impact WBA implementation from the perspectives of medical specialist educators. The outcomes being reported are themes derived from reported potential barriers and enablers to WBA implementation in the SA context.

METHODS

This paper reports on the qualitative data generated from a mixed methods study that employed a parallel convergent design, utilising a self-administered online questionnaire to collect data from participants. Data was analysed thematically and inductively.

RESULTS

The themes that emerged were: Structural readiness for WBA; staff capacity to implement WBA; quality assurance; and the social dynamics of WBA.

CONCLUSIONS

Participants demonstrated impressive levels of insight into their respective working environments, producing an extensive list of barriers and enablers. Despite significant structural and social barriers, this cohort perceives the impending implementation of WBA to be a positive development in registrar training in South Africa. We make recommendations for future research, and to the medical specialist educational leaders in SA.

Meaning making about performance: A comparison of two specialty feedback cultures

INTRODUCTION

Specialty trainees often struggle to understand how well they are performing, and feedback is commonly seen as a solution to this problem. However, medical education tends to approach feedback as acontextual rather than located in a specialty-specific cultural world. This study therefore compares how specialty trainees in surgery and intensive care medicine (ICM) make meaning about the quality of their performance and the role of feedback conversations in this process.

METHODS

We conducted a qualitative interview study in the constructivist grounded theory tradition. We interviewed 17 trainees from across Australia in 2020, eight from ICM and nine from surgery, and iterated between data collection and analytic discussions. We employed open, focused, axial and theoretical coding.

FINDINGS

There were significant divergences between specialties. Surgical trainees had more opportunity to work directly with supervisors, and there was a strong link between patient outcome and quality of care, with a focus on performance information about operative skills. ICM was a highly uncertain practice environment, where patient outcome could not be relied upon as a source of performance information; valued performance information was diffuse and included tacit emotional support. These different 'specialty feedback cultures' strongly influenced how trainees orchestrated opportunities for feedback, made meaning of their performance in their day-to-day patient care tasks and 'patched together' experiences and inputs into an evolving sense of overall progress.

DISCUSSION

We identified two types of meaning-making about performance: first, trainees' understanding of an immediate performance in a patient-care task and, second, a 'patched together' sense of overall progress from incomplete performance information. This study suggests approaches to feedback should attend to both, but also take account of the cultural worlds of specialty practice, with their attendant complexities. In particular, feedback conversations could better acknowledge the variable quality of performance information and specialty specific levels of uncertainty.

Having a Bad Day Is Not an Option: Learner Perspectives on Learner Handover

PURPOSE

Learner handover is the sharing of learner-related information between supervisors involved in their education. The practice allows learners to build upon previous assessments and can support the growth-oriented focus of competency-based medical education. However, learner handover also carries the risk of biasing future assessments and breaching learner confidentiality. Little is known about learner handover's educational impact, and what is known is largely informed by faculty and institutional perspectives. The purpose of this study was to explore learner handover from the learner perspective.

METHOD

Constructivist grounded theory was used to explore learners' perspectives and beliefs around learner handover. Twenty-nine semistructured interviews were completed with medical students and residents from the University of Ottawa and University of California, San Francisco. Interviews took place between April and December 2020. Using the constant comparative approach, themes were identified through an iterative process.

RESULTS

Learners were generally unaware of specific learner handover practices, although most recognized circumstances where both formal and informal handovers may occur. Learners appreciated the potential for learner handover to tailor education, guide entrustment and supervision decisions, and support patient safety, but worried about its potential to bias future assessments and breach confidentiality. Furthermore, learners were concerned that information-sharing may be more akin to gossip rather than focused on their educational needs and feared unfair scrutiny and irreversible long-term career consequences from one shared mediocre performance. Altogether, these concerns fueled an overwhelming pressure to perform.

CONCLUSIONS

While learners recognized the rationale for learner handover, they feared the possible inadvertent short- and long-term impact on their training and future careers. Designing policies that support transparency and build awareness around learner handover may mitigate unintended consequences that can threaten learning and the learner-supervisor relationship, ensuring learner handover benefits the learner as intended.

Evolution of the feedback conversation in anaesthesia education: a narrative review

Over the past century, education has been a core component for improving patient safety. The initial focus was developing a curriculum and an assessment process. In recent decades, the value of work-based learning has come to the fore. Learning from work, or experiential learning, requires reflection, which is critically dependent on external feedback. Conceptions of feedback have moved from a transactional information transfer from the supervisor to the trainee to a learner-centred and collaborative process occurring in a complex socio-cultural environment. In this narrative review we describe the evolution of the feedback conversation, provide a model synthesising the core concepts of feedback, and offer some guidance for the development of effective feedback in anaesthesia education.

The Use, Effectiveness, and Impact of Workplace-Based Assessments on Teaching, Supervision and Feedback Across Surgical Specialties

OBJECTIVE

To investigate the use and effectiveness of Workplace-based assessments (WBAs) and their impact on training, feedback, and perioperative teaching in surgical training programs.

DESIGN

A mixed methods cross-sectional, national electronic survey was conducted with surgical trainees and consultant trainers.

SETTINGS

The trainees and supervising faculty were from all 8 major surgical training universities across 11 surgical disciplines in South Africa.

PARTICIPANTS

A total of 108 surgical trainees and 41 supervising consultant trainers from 11 surgical disciplines across 8 surgical training universities responded to the survey.

RESULTS

The most significant educational gap identified by both the surgical trainees and trainers across all surgical disciplines was inadequate perioperative feedback. A third of the respondents were currently using workplace-based assessments. The WBA users (both trainees and trainers) had a higher rating for the general quality of surgical feedback than WBA nonusers (p = 0.02). WBA users also had a higher rating for the general quality of feedback given to trainees on their skills and competence (p = 0.04) and a higher rating for trainee supervision (p = 0.01) and the specialist training program overall (p = 0.01). The WBA users also had a higher rating for the assessment of competencies such as the trainee as an effective communicator (p < 0.01) and collaborator (p = 0.04).

CONCLUSION

This study found that the use of WBAs enhances the quality and effectiveness of feedback in surgical training programs. We also found that the use of WBAs enhance perioperative teaching and learning and improves the assessment of relational competencies. This was also associated with high ratings for the quality of trainee supervision. Faculty and trainee development, strengthening the trainee-trainer relationship, and integrating iterative stakeholder feedback could help realize the full potential of WBAs to augment surgical training across disciplines.

It Takes a Village: Optimal Graduate Medical Education Requires a Deliberately Developmental Organization

Coaching is proposed as a means of improving the learning culture of medicine. By fostering trusting teacher-learner relationships, learners are encouraged to embrace feedback and make the most of failure. This paper posits that a cultural shift is necessary to fully harness the potential of coaching in graduate medical education. We introduce the deliberately developmental organization framework, a conceptual model focusing on three core dimensions: developmental communities, developmental aspirations, and developmental practices. These dimensions broaden the scope of coaching interactions. Implementing this organizational change within graduate medical education might be challenging, yet we argue that embracing deliberately developmental principles can embed coaching into everyday interactions and foster a culture in which discussing failure to maximize learning becomes acceptable. By applying the dimensions of developmental communities, aspirations, and practices, we present a six-principle roadmap towards transforming graduate medical education training programs into deliberately developmental organizations.

Flattening Hierarchical Structures to Empower Women Trainee Leaders on Social Media Teams

We share our experience empowering women trainees and leadership through a flattened hierarchical social media team structure with supporting evidence from measurable outcomes.

You are in charge now: exploration of educational relationships between anaesthetic trainees and their supervising specialists

Background

Much of the education during anaesthesia training occurs in the workplace where trainees work under the close supervision of a more senior anaesthetist. Trainee anaesthetists are exposed to multiple supervisors with whom they form educational and supervisory relationships over the course of their training. Surprisingly little research has been conducted to explore the factors behind the development and maintenance of these relationships. This study explores the process of how education occurs in the workplace by examining the relationship from the perspective of both trainees and specialists.

Methods

This is an exploratory qualitative study. Eight trainee and 10 specialist anaesthetists participated in an individual semi-structured interview. The data were analysed thematically by each of the authors to generate themes.

Results

Six themes were identified in the analysis: (1) sizing up; (2) negotiated autonomy; (3) working closely together; (4) workplace practices; (5) education being valued; and (6) gender. A conceptual model to illustrate the relationships between the six themes was developed.

Conclusions

Supervisory relationships were viewed positively by participants despite impediments such as lack of continuity and busy clinical environments. But there were tensions, particularly in balancing trainee autonomy with patient safety. A nuanced 'sizing up' process, with negotiation of autonomy, was described by both supervisors and trainees. Our findings may support supervisory relationships to reach this ideal more effectively.

The role of feedback in supporting trainees who underperform in clinical environments

Introduction

Underperformance in clinical environments can be costly and emotional for all stakeholders. Feedback is an important pedagogical strategy for working with underperformance - both formal and informal strategies can make a difference. Feedback is a typical feature of remediation programs, and yet there is little consensus on how feedback should unfold in the context of underperformance.

Methods

This narrative review synthesises literature at the intersections of feedback and underperformance in clinical environments where service, learning and safety need to be considered. We do so with a critical eye towards generating insights for working with underperformance in the clinical environment.

Synthesis and discussion

There are compounding and multi-level factors that contribute to underperformance and subsequent failure. This complexity overwrites simplistic notions of 'earned' failure through individual traits and deficit. Working with such complexity requires feedback that goes beyond educator input or 'telling'. When we shift beyond feedback as input to process, we recognise that these processes are fundamentally relational, where trust and safety are necessary for trainees to share their weaknesses and doubts. Emotions are always present and they signal action. Feedback literacy might help us consider how to engage trainees with feedback so that they take an active (autonomous) role in developing their evaluative judgements. Finally, feedback cultures can be influential and take effort to shift if at all. A key mechanism running through all these considerations of feedback is enabling internal motivation, and creating conditions for trainees to feel relatedness, competence and autonomy. Broadening our perceptions of feedback, beyond telling, might help create environments for learning to flourish.

How progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships: a mixed method study

The combination of measuring performance and giving feedback creates tension between formative and summative purposes of progress evaluations and can be challenging for supervisors. There are conflicting perspectives and evidence on the effects supervisor-trainee relationships have on assessing performance. The aim of this study was to learn how progress evaluations are used in postgraduate education with longitudinal supervisor-trainee relationships. Progress evaluations in a two-year community-pharmacy specialization program were studied with a mixed-method approach. An adapted version of the Canadian Medical Education Directives for Specialists (CanMEDS) framework was used. Validity of the performance evaluation scores of 342 trainees was analyzed using repeated measures ANOVA. Semi-structured interviews were held with fifteen supervisors to investigate their response processes, the utility of the progress evaluations, and the influence of supervisor-trainee relationships. Time and CanMEDS roles affected the three-monthly progress evaluation scores. Interviews revealed that supervisors varied in their response processes. They were more committed to stimulating development than to scoring actual performance. Progress evaluations were utilized to discuss and give feedback on trainee development and to add structure to the learning process. A positive supervisor-trainee relationship was seen as the foundation for feedback and supervisors preferred the roles of educator, mentor, and coach over the role of assessor. We found that progress evaluations are a good method for directing feedback in longitudinal supervisor-trainee relationships. The reliability of scoring performance was low. We recommend progress evaluations to be independent of formal assessments in order to minimize roles-conflicts of supervisors.

Teacher, Gatekeeper, or Team Member: supervisor positioning in programmatic assessment

Competency-based assessment is undergoing an evolution with the popularisation of programmatic assessment. Fundamental to programmatic assessment are the attributes and buy-in of the people participating in the system. Our previous research revealed unspoken, yet influential, cultural and relationship dynamics that interact with programmatic assessment to influence success. Pulling at this thread, we conducted secondary analysis of focus groups and interviews (n = 44 supervisors) using the critical lens of Positioning Theory to explore how workplace supervisors experienced and perceived their positioning within programmatic assessment. We found that supervisors positioned themselves in two of three ways. First, supervisors universally positioned themselves as a Teacher, describing an inherent duty to educate students. Enactment of this position was dichotomous, with some supervisors ascribing a passive and disempowered position onto students while others empowered students by cultivating an egalitarian teaching relationship. Second, two mutually exclusive positions were described-either Gatekeeper or Team Member. Supervisors positioning themselves as Gatekeepers had a duty to protect the community and were vigilant to the detection of inadequate student performance. Programmatic assessment challenged this positioning by reorientating supervisor rights and duties which diminished their perceived authority and led to frustration and resistance. In contrast, Team Members enacted a right to make a valuable contribution to programmatic assessment and felt liberated from the burden of assessment, enabling them to assent power shifts towards students and the university. Identifying supervisor positions revealed how programmatic assessment challenged traditional structures and ideologies, impeding success, and provides insights into supporting supervisors in programmatic assessment.