Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study

Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS Part A

Successful completion of the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination is mandatory for surgical trainees entering higher specialist training in the United Kingdom. Despite its international reputation, and the value placed on the examination in surgical training, there has been little evidence of its predictive validity until recently. In this review, we present a summary of findings of four recent Intercollegiate studies assessing the predictive validity of the MRCS Part A (written) examination. Data from all four studies showed statistically significant positive correlations between the MRCS Part A and other written examinations taken by surgical trainees over the course of their education. The studies summarised in this review provide compelling evidence for the predictive validity of this gatekeeping examination. This review will be of interest to trainees, training institutions and the Royal Colleges given the value placed on the examination by surgical training programmes.

Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS part B

The Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) is a high-stakes postgraduate examination taken by thousands of surgical trainees worldwide every year. The MRCS is a challenging assessment, highly regarded by surgical training programmes and valued as a gatekeeper to the surgical profession. The examination is taken at considerable personal, social and financial cost to surgical trainees, and failure has significant implications for career progression. Given the value placed on MRCS, it must be a reliable and valid assessment of the knowledge and skills of early-career surgeons. Our first article 'Establishing the Predictive Validity of the Intercollegiate Membership of the Royal Colleges of Surgeons Written Examination: MRCS Part A' discussed the principles of assessment reliability and validity and outlined the mounting evidence supporting the predictive validity of the MRCS Part A (the multiple-choice questionnaire component of the examination). This, the second article in the series discusses six recently published studies investigating the predictive validity of the MRCS Part B (the clinical component of the examination). All national longitudinal cohort studies reviewed have demonstrated significant correlations between MRCS Part B and other assessments taken during the UK surgical training pathway, supporting the predictive validity of MRCS Part B. This review will be of interest to trainees, trainers and Royal Colleges given the value placed on the examination by surgical training programmes.

The educational value of situational judgement tests (SJTs) when used during undergraduate medical training: A systematic review and narrative synthesis

INTRODUCTION

Situational judgement tests (SJTs) are a recognised assessment method for admission into medical school, selection into postgraduate training programs, and postgraduate competency assessment. More recently, however, SJTs have been used during undergraduate medical training (UMT). This systematic review identifies, describes, and appraises the evidence for SJTs in UMT to determine educational associations and outcomes.

METHODS

MEDLINE, EMBASE, ERIC, PsycINFO, SCOPUS, Web of Science, and grey literature were searched for original research studies evaluating SJTs implemented within UMT to 1 November 2022. Studies reporting evaluation outcomes were included. Narrative data syntheses were undertaken. Risk of Bias was appraised using the Quality in Prognosis Studies tool.

RESULTS

24 studies were included. National database-derived SJTs (n = 14) assessed against professionalism, postgraduate attainment, construct of medical degree, medical school admissions scores, personality attributes, and declaration of disability. In-house derived SJTs (n = 10) assessed against professionalism, clinical skills, and personality attributes. Most evidence evaluated and reported inverse SJT associations with professionalism and were moderate risk of bias.

CONCLUSION

SJTs may have utility for developing professional behaviours in medical students. However, further research testing SJT robustness, standard setting methodologies, and prospectively evaluating SJTs against objective outcome measures within the context of UMT is warranted.

Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study

Objective

A recent independent review on diversity and inclusivity highlighted concerns that barriers to surgical career progression exist for some groups of individuals and not others. Group-level differences in performance at the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examinations have been identified but are yet to be investigated. We aimed to characterise the relationship between sociodemographic differences and performance at MRCS.

Design

Retrospective cohort study.

Setting

Secondary care.

Participants

All UK MRCS candidates attempting Part A (n = 5780) and Part B (n = 2600) between 2013 and 2019 with linked sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk).

Main outcome measures

Chi-square tests established univariate associations with MRCS performance. Multiple logistic regression identified independent predictors of success, adjusted for medical school performance.

Results

Statistically significant differences in MRCS pass rates were found according to gender, ethnicity, age, graduate status, educational background and socioeconomic status (all p < 0.05). After adjusting for prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were independent predictors of MRCS Part A success and being a non-graduate (OR 1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51, 95% CI 1.08–2.10) were independent predictors of Part B success. Black and minority ethnic groups were significantly less likely to pass MRCS Part B at their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR 0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White candidates.

Conclusions

There is significant group-level differential attainment at MRCS, likely to represent the accumulation of privilege and disadvantage experienced by individuals throughout their education and training. Those leading surgical education now have a responsibility to identify and address the causes of these attainment differences.

Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study

OBJECTIVES

The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values.

DESIGN AND PARTICIPANTS

A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database.

METHODS

We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt.

RESULTS

MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05).

CONCLUSIONS

There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.

Course Review: Doctors Academy Basic Sciences and Clinical Application for the Membership of the Royal College of Surgeons Part A Course

The Membership of the Royal College of Surgeons (MRCS part A and B) is a mandatory examination that is required by all trainees to enter the surgical speciality training programme in the United Kingdom. Students and clinicians alike often find it a struggle to succeed in passing these exams given the breadth and depth of knowledge required across a spectrum of surgical sciences. There are several factors contributing to success, including ethnic background, number of attempts, and medical school performance. Studies have also shown that a marker of success for Part B was influenced by the Part A performance. During the COVID-19 peak, the MRCS Part A written exam has had its format changed to an online exam reflecting the policy surrounding social distancing. Due to this, the pass mark has become more difficult to attain compared to traditional face-to-face settings. As such, many trainees are finding it more difficult to succeed in the exam, leading to more emphasis on more thorough preparation for exams. This can be in the form of courses, textbooks, and using more of the online question banks.

The Basic Sciences and Clinical Application for MRCS A Course is organised by the Doctors Academy group and details an intensive and interactive three-day revision course held intermittently throughout the year in Cardiff, Wales multiple times a year. Delivered by an expert faculty, it aims to provide maximum preparation in the lead up to the MRCS Part A exam with emphasis on retaining high-yield knowledge and application of key exam revision techniques. We reviewed the post-course surveys from 2019 to 2021 and found that the majority of attendees found the course content to be excellent and relevant to the exams. These questions were a mandatory part of the course and reflect a variety of training grades from foundation to core training/speciality training (CT1/2 to ST1/2). Overall, candidates found that this course aided their exam preparation immensely and contributed to passing.

MRCS Performance by OMFS trainees: An update and call to action

Completion of the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination is a prerequisite for completion of specialty surgical training in UK. The aim of this study was to compare MRCS performance of OMFS trainees to trainees in other specialities over the past 13 years. Differential attainment (DA) was noted in MRCS Part A (MCQ) pass rates. There was no statistically significant difference in MRCS Part B pass rates. The reasons for this DA are currently unknown and require further investigation. It has been suggested that this may be due at least in part to competing time, family and financial obligations which increase over time.