Myths and legends in workplace-based assessment; it’s all about perspective?

Assessing trainee's performance using reported observations: Perceptions of nurse meta-assessors

BACKGROUND

Educational and health care organizations who prepare meta-assessors to fulfill their role in the assessment of trainees' performance based on reported observations have little literature to rely on. While the assessment of trainees' performance based on reported observations has been operationalized, we have yet to understand the elements that can affect its quality fully. Closing this gap in the literature will provide valuable insight that could inform the implementation and quality monitoring of the assessment of trainees' performance based on reported observations.

OBJECTIVES

The purpose of this study was to explore the elements to consider in the assessment of trainees' performance based on reported observations from the perspectives of meta-assessors.

METHODS

Design, Settings, Participants, data collection and analysis. The authors adopted Sandelowski's qualitative descriptive approach to interview nurse meta-assessors from two nursing programs. A semi-structured interview guide was used to document the elements to consider in the assessment of nursing trainees' performance based on reported observations, and a survey was used to collect sociodemographic data. The authors conducted a thematic analysis of the interview transcripts.

RESULTS

Thirteen meta-assessors participated in the study. Three core themes were identified: (1) meta-assessors' appropriation of their perceived assessment roles and activities, (2) team climate of information sharing, and (3) challenges associated with the assessment of trainees' performance based on reported observations. Each theme is comprised of several sub themes.

CONCLUSIONS

To optimize the quality of the assessment of the trainee's performance based on reported observations and ratings, HPE programs might consider how to clarify better the meta-assessor's roles and activities, as well as how interventions could be created to promote a climate of information sharing and to address the challenges identified. This work will guide educational and health care organizations for better preparation and support for meta-assessors and preceptors.

How workplace-based assessments guide learning in postgraduate education: A scoping review

INTRODUCTION

Competency-based medical education (CBME) led to the widespread adoption of workplace-based assessment (WBA) with the promise of achieving assessment for learning. Despite this, studies have illustrated tensions between the summative and formative role of WBA which undermine learning goals. Models of workplace-based learning (WBL) provide insight, however, these models excluded WBA. This scoping review synthesizes the primary literature addressing the role of WBA to guide learning in postgraduate medical education, with the goal of identifying gaps to address in future studies.

METHODS

The search was applied to OVID Medline, Web of Science, ERIC and CINAHL databases, articles up to September 2020 were included. Titles and abstracts were screened by two reviewers, followed by a full text review. Two members independently extracted and analysed quantitative and qualitative data using a descriptive-analytic technique rooted in Billett's four premises of WBL. Themes were synthesized and discussed until consensus.

RESULTS

All 33 papers focused on the perception of learning through WBA. The majority applied qualitative methodology (70%), and 12 studies (36%) made explicit reference to theory. Aligning with Billett's first premise, results reinforce that learning always occurs in the workplace. WBA helped guide learning goals and enhanced feedback frequency and specificity. Billett's remaining premises provided an important lens to understand how tensions that existed in WBL have been exacerbated with frequent WBA. As individuals engage in both work and WBA, they are slowly transforming the workplace. Culture and context frame individual experiences and the perceived authenticity of WBA. Finally, individuals will have different goals, and learn different things, from the same experience.

CONCLUSION

Analysing WBA literature through the lens of WBL theory allows us to reframe previously described tensions. We propose that future studies attend to learning theory, and demonstrate alignment with philosophical position, to advance our understanding of assessment-for-learning in the workplace.

Study protocol: content and perceived educational utility of different modalities of clinical teaching visit (CTV) workplace-based assessments within Australian general practice vocational training: a cross-sectional study

INTRODUCTION

Clinical teaching visits (CTVs) are formative workplace-based assessments that involve a senior general practitioner (GP) observing a clinical practice session of a general practice registrar (specialist vocational GP trainee). These visits constitute a key part of Australian GP training. Despite being mandatory and resource-intensive, there is a paucity of evidence regarding the content and educational utility of CTVs. This study aims to establish the content and educational utility of CTVs across varying practice settings within Australia, as perceived by registrars and their assessors ('CT visitors'). In addition, this study aims to establish registrar, CT visitor and practice factors associated with CTV content and perceived CTV utility ratings.

METHODS AND ANALYSIS

This study will collect data prospectively using online questionnaires completed soon after incident CTVs. Participants will be registrars and CT visitors of CTVs conducted from March 2020 to January 2021. The setting is three Regional Training Organisations across four Australian states and territories (encompassing 37% of Australian GP registrars).Outcome factors will be a number of specified CTV content elements occurring during the CTV as well as participants' perceptions of CTV utility, which will be analysed using univariate and multivariable regression.

ETHICS AND DISSEMINATION

Ethics approval has been granted by the University of Newcastle Human Research Ethics Committee, approval number H-2020-0037. Study findings are planned to be disseminated via conference presentation, peer-reviewed journals, educational practice translational workshops and the GP Synergy research subwebsite.

Performance in practice; exploring trainer and trainee experiences of user-designed formative assessment tools

INTRODUCTION

General Practice training in Ireland currently has various methods of formative assessment and feedback delivered to trainees. In 2018 the Irish College of General Practitioners commissioned the generation of two new user-designed formative feedback tools that would allow trainee feedback to drive learning. These tools became known as the Performance in Practice (PiP) tools.

AIMS

To explore the experiences of General Practice (GP) trainers and trainees having completed a pilot of using the PiP tools for 4 months.

METHODS

An explorative phenomenological approach was taken to understand the experiences of trainers and trainees. One to one interviews were conducted, and the transcripts analysed for themes and sub-theme via Template analysis.

RESULTS

User experiences focused on two main areas; educational value and acceptability. In relation to educational value, the PiP tools were seen as an improvement over established forms of formative feedback, as they were centred around the curriculum and therefore reflected the unique multifaceted requirements of an independently practising GP. Acceptability primarily focused around data governance and structures, as well as practical issues such as ease of software use.

CONCLUSIONS

Overall, the experience of using the PiP tools was positive for both trainers and trainees. Future plans to further explore implementation of the PiP tools have been significantly informed by this research.