Do role-players affect the outcome of a high-stakes postgraduate OSCE, in terms of candidate sex or ethnicity? Results from an analysis of the 52,702 anonymised case scores from one year of the MRCGP clinical skills assessment

Unveiling Disparities: Exploring Differential Attainment in Postgraduate Training Within Clinical Oncology

AIMS

Differential attainment (DA) in post graduate medical training is a recognised challenge and refers to unexplained variation across groups when split by several protected characteristics. The Royal College of Radiology is committed to fostering diversity, inclusivity, and equality with the goal of narrowing existing gaps and improving training outcomes.

MATERIALS AND METHODS

This was a mixed methods study aiming to understand the causes of DA with view to helping the RCR develop strategies to address this. A cross-sectional survey was completed by 140 clinical oncology trainees in September 2022. Trainees and trainers (17 and 6 respectively) from across England, Scotland, Wales and Northern Ireland, took part in focus group and interviews from August to December 2022. Quantitative and qualitative data merged and interpreted.

RESULT

Results showed international medical graduates and trainees from ethnic minority backgrounds were more likely to encounter challenges. The qualitative findings were used to identify three themes through which these problems could be framed. The trainee as a "space invader," the hidden curriculum of clinical oncology training and the process of navigating and tacking the training journey.

CONCLUSION

Differential attainment is the product of a complex interplay between the trainee, trainer, and the training environment. Therefore, interventions must be tailored to different people and contexts. At a national level, the RCR can adopt general policies to promote this such as mentorship programmes, protected time for supervision and cultural competency training. Efficacy of proposed interventions for trial and their impact on DA should be evaluated to drive evidence-based changes.

The MRCGP Clinical Skills Assessment: an integrative review of evidence

Background Successful completion of all three components of the Membership of the Royal College of General Practitioners UK (MRCGP) is required for a doctor to practise independently as a general practitioner (GP). One component, the Clinical Skills Assessment (CSA), is a high fidelity, high stakes examination designed to provide an objective external assessment of clinical skills. Aim The aim of this integrative review was to identify, critically appraise and synthesise published empirical evidence on the CSA to identify areas for further inquiry or future collaboration. Design and setting An integrative review of empirical literature focused on the MRCGP CSA. Method A search strategy was developed to conduct an integrative review of empirical published literature. The quality of studies was assessed using the Mixed Methods Appraisal Tool (MMAT). Results 11 of the 31 records reviewed were included. Studies were heterogeneous looking at the use of simulated patients, child actors and mock CSAs. Variables contributing to performance on the CSA, with a particular emphasis on the International Medical Graduate, were identified. Conclusion A small pool of published evidence regarding the CSA exists, focused on factors influencing CSA performance. Future research could explore the effectiveness of interventions to improve performance on the CSA and use robust measures to consider candidates' experience of the CSA.

The sights and insights of examiners in objective structured clinical examinations

PURPOSE

The objective structured clinical examination (OSCE) is considered to be one of the most robust methods of clinical assessment. One of its strengths lies in its ability to minimise the effects of examiner bias due to the standardisation of items and tasks for each candidate. However, OSCE examiners' assessment scores are influenced by several factors that may jeopardise the assumed objectivity of OSCEs. To better understand this phenomenon, the current review aims to determine and describe important sources of examiner bias and the factors affecting examiners' assessments.

METHODS

We performed a narrative review of the medical literature using Medline. All articles meeting the selection criteria were reviewed, with salient points extracted and synthesised into a clear and comprehensive summary of the knowledge in this area.

RESULTS

OSCE examiners' assessment scores are influenced by factors belonging to 4 different domains: examination context, examinee characteristics, examinee-examiner interactions, and examiner characteristics. These domains are composed of several factors including halo, hawk/dove and OSCE contrast effects; the examiner's gender and ethnicity; training; lifetime experience in assessing; leadership and familiarity with students; station type; and site effects.

CONCLUSION

Several factors may influence the presumed objectivity of examiners' assessments, and these factors need to be addressed to ensure the objectivity of OSCEs. We offer insights into directions for future research to better understand and address the phenomenon of examiner bias.

Perceived causes of differential attainment in UK postgraduate medical training: a national qualitative study

OBJECTIVES

Explore trainee doctors' experiences of postgraduate training and perceptions of fairness in relation to ethnicity and country of primary medical qualification.

DESIGN

Qualitative semistructured focus group and interview study.

SETTING

Postgraduate training in England (London, Yorkshire and Humber, Kent Surrey and Sussex) and Wales.

PARTICIPANTS

137 participants (96 trainees, 41 trainers) were purposively sampled from a framework comprising: doctors from all stages of training in general practice, medicine, obstetrics and gynaecology, psychiatry, radiology, surgery or foundation, in 4 geographical areas, from white and black and minority ethnic (BME) backgrounds, who qualified in the UK and abroad.

RESULTS

Most trainees described difficult experiences, but BME UK graduates (UKGs) and international medical graduates (IMGs) could face additional difficulties that affected their learning and performance. Relationships with senior doctors were crucial to learning but bias was perceived to make these relationships more problematic for BME UKGs and IMGs. IMGs also had to deal with cultural differences and lack of trust from seniors, often looking to IMG peers for support instead. Workplace-based assessment and recruitment were considered vulnerable to bias whereas examinations were typically considered more rigorous. In a system where success in recruitment and assessments determines where in the country you can get a job, and where work-life balance is often poor, UK BME and international graduates in our sample were more likely to face separation from family and support outside of work, and reported more stress, anxiety or burnout that hindered their learning and performance. A culture in which difficulties are a sign of weakness made seeking support and additional training stigmatising.

CONCLUSIONS

BME UKGs and IMGs can face additional difficulties in training which may impede learning and performance. Non-stigmatising interventions should focus on trainee-trainer relationships at work and organisational changes to improve trainees' ability to seek social support outside work.