1
|
Ellis R, Al-Tawarah Y, Brennan PA, Lee AJ, Hines J, Scrimgeour DS, Cleland J. The impact of disability on recruitment to higher surgical specialty training: A retrospective cohort study. Surgeon 2024:S1479-666X(24)00072-6. [PMID: 39084924 DOI: 10.1016/j.surge.2024.07.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2024] [Revised: 07/12/2024] [Accepted: 07/13/2024] [Indexed: 08/02/2024]
Abstract
BACKGROUND UK examining bodies are required to eliminate discrimination against people with protected characteristics. To achieve this in surgery, differential attainment (DA) in assessments used as gatekeepers to career progression must be ruled out. This study investigated the impact of disability status on the likelihood of success at national selection for Higher Surgical Training (HST). METHODS A retrospective cohort study of all UK graduates in the UKMED database (https://www.ukmed.ac.uk) who underwent selection for HST (ST3) from 2012 to 2019 (n = 2875). Univariate analysis identified differences in success rates at first-application. Logistic regression models identified whether disability was a predictor of success after adjusting for sociodemographic factors and prior MRCS performance. RESULTS There was no significant difference in success rates between candidates with and without disabilities (all p > 0.05) for any surgical specialty. Disability status was not a statistically significant predictor of success. Female candidates were 25 % more likely to be successful (OR 1.25 [95%CI 1.05 to 1.49]) and Non-White candidates were 20 % less likely to be successful (OR 0.80 [95%CI 0.68 to 0.96]). Candidates who passed MRCS Part A and Part B at the first attempt were 49 % (OR 1.49 [95%CI 1.25 to 1.77]) and 90 % (OR 1.90 [95%CI 1.58 to 2.28]) more likely to be successful. CONCLUSION No significant difference was found in the likelihood of being successful at HST selection for any surgical specialty between applicants with and without disabilities, regardless of type of disability. DA was identified between other sociodemographic groups which requires further exploration.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom.
| | - Yasin Al-Tawarah
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom.
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, Aberdeen, United Kingdom.
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom.
| | - John Hines
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, United Kingdom.
| | - Duncan Sg Scrimgeour
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, United Kingdom.
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.
| |
Collapse
|
2
|
Sidhu NS, Fleming S. Re-examining single-moment-in-time high-stakes examinations in specialist training: A critical narrative review. MEDICAL TEACHER 2024; 46:528-536. [PMID: 37740944 DOI: 10.1080/0142159x.2023.2260081] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/25/2023]
Abstract
In this critical narrative review, we challenge the belief that single-moment-in-time high-stakes examinations (SMITHSEx) are an essential component of contemporary specialist training. We explore the arguments both for and against SMITHSEx, examine potential alternatives, and discuss the barriers to change.SMITHSEx are viewed as the "gold standard" assessment of competence but focus excessively on knowledge assessment rather than capturing essential competencies required for safe and competent workplace performance. Contrary to popular belief, regulatory bodies do not mandate SMITHSEx in specialist training. Though acting as significant drivers of learning and professional identity formation, these attributes are not exclusive to SMITHSEx.Skills such as crisis management, procedural skills, professionalism, communication, collaboration, lifelong learning, reflection on practice, and judgement are often overlooked by SMITHSEx. Their inherent design raises questions about the validity and objectivity of SMITHSEx as a measure of workplace competence. They have a detrimental impact on trainee well-being, contributing to burnout and differential attainment.Alternatives to SMITHSEx include continuous low-stakes assessments throughout training, ongoing evaluation of competence in the workplace, and competency-based medical education (CBME) concepts. These aim to provide a more comprehensive and context-specific assessment of trainees' competence while also improving trainee welfare.Specialist training colleges should evolve from exam providers to holistic education sources. Assessments should emphasise essential practical knowledge over trivia, align with clinical practice, aid learning, and be part of a diverse toolkit. Eliminating SMITHSEx from specialist training will foster a competency-based approach, benefiting future medical professionals' well-being and success.
Collapse
Affiliation(s)
- Navdeep S Sidhu
- Department of Anaesthesiology, School of Medicine, University of Auckland, Auckland, New Zealand
- Department of Anaesthesia and Perioperative Medicine, North Shore Hospital, Auckland, New Zealand
| | - Simon Fleming
- Department of Hand Surgery, Royal North Shore Hospital, Sydney, New South Wales, Australia
| |
Collapse
|
3
|
Allen P, Jessup B, Kirschbaum M, Khanal S, Baker-Smith V, Graham B, Barnett T. Preparing for success in final summative medical specialist examinations: The case for RACE. BMC MEDICAL EDUCATION 2023; 23:918. [PMID: 38053089 DOI: 10.1186/s12909-023-04920-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Accepted: 11/30/2023] [Indexed: 12/07/2023]
Abstract
BACKGROUND Failure rates on medical specialist final summative examinations in Australia are high, regardless of speciality. Examination failure can have detrimental psycho-social, financial and job security effects on the trainee, while delays in completion of training adversely impacts workforce growth and health outcomes for the community. The study aimed to explore the preparation factors that contribute to ophthalmology trainee success in their final summative examination. METHODS Semi-structured in-depth interviews were conducted with 29 participants via telephone or Zoom with ophthalmology trainees and Fellows. To be eligible, interviewees had to have sat the Royal Australian and New Zealand College of Ophthalmologists Advanced Clinical Examination (RACE) within the past five years or were providing supervision to trainees preparing for RACE. Interviews were audio-recorded, transcribed and thematically analysed. RESULTS Examination success was underpinned by six themes relating to preparation: (i) 'Those who fail to plan, plan to fail', which related to development and adherence to a study plan; (ii) 'It takes a village' encompassed trainees establishing and activating personal and professional supports; (iii) 'Get to know your opponent', which encompassed developing an understanding of the examination construct, format and requirements; (iv) 'There is no substitute for hard work', which related to intensive study over a period of 12-18 months; (v) 'Keep pace with the herd', which referred to benchmarking preparation efforts and progress against peers; and (vi) 'Don't jump the gun', which related to ensuring readiness to sit. CONCLUSIONS Maximising medical specialist examination pass rates is in the best interest of trainees, training Colleges, health care systems and communities. Recognising and facilitating preparation approaches that foster success in final summative examinations are the collective responsibility of trainees, specialist training Colleges, training networks and health systems. Trainees need to plan for examination success, be self-determined to commit to intensive study over an extended time period and be realistic about their readiness to sit.
Collapse
Affiliation(s)
- Penelope Allen
- Rural Clinical School, College of Health and Medicine, University of Tasmania, Burnie, TAS, Australia
| | - Belinda Jessup
- Centre for Rural Health, College of Health and Medicine, University of Tasmania, Locked Bag 1322, Launceston, TAS, 7250, Australia
| | - Melissa Kirschbaum
- Centre for Rural Health, College of Health and Medicine, University of Tasmania, Locked Bag 1322, Launceston, TAS, 7250, Australia.
| | - Santosh Khanal
- The Royal Australian and New Zealand College of Ophthalmologists, Sydney, NSW, Australia
| | - Victoria Baker-Smith
- The Royal Australian and New Zealand College of Ophthalmologists, Sydney, NSW, Australia
| | - Barnabas Graham
- The Royal Australian and New Zealand College of Ophthalmologists, Sydney, NSW, Australia
| | - Tony Barnett
- Centre for Rural Health, College of Health and Medicine, University of Tasmania, Locked Bag 1322, Launceston, TAS, 7250, Australia
| |
Collapse
|
4
|
Ellis R, Cleland J, Scrimgeour DS, Lee AJ, Hines J, Brennan PA. Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS Part A. Surgeon 2023; 21:323-330. [PMID: 37544852 DOI: 10.1016/j.surge.2023.07.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/08/2023]
Abstract
Successful completion of the Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination is mandatory for surgical trainees entering higher specialist training in the United Kingdom. Despite its international reputation, and the value placed on the examination in surgical training, there has been little evidence of its predictive validity until recently. In this review, we present a summary of findings of four recent Intercollegiate studies assessing the predictive validity of the MRCS Part A (written) examination. Data from all four studies showed statistically significant positive correlations between the MRCS Part A and other written examinations taken by surgical trainees over the course of their education. The studies summarised in this review provide compelling evidence for the predictive validity of this gatekeeping examination. This review will be of interest to trainees, training institutions and the Royal Colleges given the value placed on the examination by surgical training programmes.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB25 2ZD, United Kingdom; Urology Department, Nottingham University Hospitals, Nottingham, United Kingdom.
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB25 2ZD, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, United Kingdom.
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom.
| | - John Hines
- Urology Department, University College Hospital, London, W1G 8PH, United Kingdom.
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, United Kingdom.
| |
Collapse
|
5
|
Ellis R, Cleland J, Scrimgeour DS, Lee AJ, Hines J, Brennan PA. Establishing the predictive validity of the intercollegiate membership of the Royal Colleges of surgeons written examination: MRCS part B. Surgeon 2023; 21:278-284. [PMID: 37517979 DOI: 10.1016/j.surge.2023.07.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 07/12/2023] [Accepted: 07/13/2023] [Indexed: 08/01/2023]
Abstract
The Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) is a high-stakes postgraduate examination taken by thousands of surgical trainees worldwide every year. The MRCS is a challenging assessment, highly regarded by surgical training programmes and valued as a gatekeeper to the surgical profession. The examination is taken at considerable personal, social and financial cost to surgical trainees, and failure has significant implications for career progression. Given the value placed on MRCS, it must be a reliable and valid assessment of the knowledge and skills of early-career surgeons. Our first article 'Establishing the Predictive Validity of the Intercollegiate Membership of the Royal Colleges of Surgeons Written Examination: MRCS Part A' discussed the principles of assessment reliability and validity and outlined the mounting evidence supporting the predictive validity of the MRCS Part A (the multiple-choice questionnaire component of the examination). This, the second article in the series discusses six recently published studies investigating the predictive validity of the MRCS Part B (the clinical component of the examination). All national longitudinal cohort studies reviewed have demonstrated significant correlations between MRCS Part B and other assessments taken during the UK surgical training pathway, supporting the predictive validity of MRCS Part B. This review will be of interest to trainees, trainers and Royal Colleges given the value placed on the examination by surgical training programmes.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; Urology Department, Nottingham University Hospitals, Nottingham, United Kingdom.
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore.
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, United Kingdom.
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, United Kingdom.
| | - John Hines
- University College Hospital London, United Kingdom.
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, United Kingdom.
| |
Collapse
|
6
|
You Y, Wang W, Cleland J. Does medical education reform change who is selected? A national cross-sectional survey from China. BMJ Open 2023; 13:e070239. [PMID: 37567746 PMCID: PMC10423783 DOI: 10.1136/bmjopen-2022-070239] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/17/2022] [Accepted: 07/19/2023] [Indexed: 08/13/2023] Open
Abstract
OBJECTIVES Approximately 10 years ago, China introduced an education plan to improve the overall quality of medical education and to better serve the population's health needs. Many medical schools were then recognised and financed by China's Ministry of Education to develop and operationalise new pilot programmes (PPs) aligned with this plan. These ran in parallel with the traditional programmes (TPs). One way to achieve the plan's first aim, improving the quality of medical education, is to select academically stronger candidates. We, thus, examined and compared who were selected into PPs and TPs. DESIGN Cross-sectional study. SETTING Data were collected from 123 medical schools across China via the 2021 China Medical Student Survey. PARTICIPANTS Participants were undergraduate clinical medicine students across all year groups. PRIMARY AND SECONDARY OUTCOME MEASURES Medical school selection was via the National College Entrance Examination (NCEE). Medical students' NCEE performance and their sociodemographics were used as the primary and secondary outcome measures. Mann-Whitney or χ2 tests were used to compare the means between educational programmes (PPs vs TPs) and various selection outcomes. Multilevel mixed-effects regressions were employed to account for school idiosyncratic selection results. RESULTS Of the 204 817 respondents, 194 163 (94.8%) were in a TP and 10 654 (5.2%) a PP. PP respondents (median=75.2, IQR=69.5-78.8) had significantly higher NCEE scores than their TP counterparts (median=73.9, IQR=68.5-78.7). Holding constant their NCEE score, PP respondents were significantly more likely to come from urban areas, not be first-generation college students, and have parents with higher occupational status and income. CONCLUSIONS Assuming quality can be indicated by prior academic achievement at the point of selection, PPs achieved this mission. However, doing so limited medical students' diversity. This may be unhelpful in achieving the Education Plan's goal to better serve China's health needs.
Collapse
Affiliation(s)
- You You
- National Center for Health Professions Education Development, Peking University, Beijing, People's Republic of China
- Institute of Economics of Education, Peking University, Beijing, People's Republic of China
| | - Weimin Wang
- National Center for Health Professions Education Development, Peking University, Beijing, People's Republic of China
- Health Science Center, Peking University, Beijing, People's Republic of China
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| |
Collapse
|
7
|
Downie S, Cherry J, Dunn J, Harding T, Eastwood D, Gill S, Johnson S. The role of Gender in Operative Autonomy in orthopaedic Surgical Trainees (GOAST). Bone Joint J 2023; 105-B:821-832. [PMID: 37399113 DOI: 10.1302/0301-620x.105b7.bjj-2023-0132.r2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 07/05/2023]
Abstract
Aims Global literature suggests that female surgical trainees have lower rates of independent operating (operative autonomy) than their male counterparts. The objective of this study was to identify any association between gender and lead/independent operating in speciality orthopaedic trainees within the UK national training programme. Methods This was a retrospective case-control study using electronic surgical logbook data from 2009 to 2021 for 274 UK orthopaedic trainees. Total operative numbers and level of supervision were compared between male and female trainees, with correction for less than full-time training (LTFT), prior experience, and time out during training (OOP). The primary outcome was the percentage of cases undertaken as lead surgeon (supervised and unsupervised) by UK orthopaedic trainees by gender. Results All participants gave permission for their data to be used. In total, 274 UK orthopaedic trainees submitted data (65% men (n = 177) and 33% women (n = 91)), with a total of 285,915 surgical procedures logged over 1,364 trainee-years. Males were lead surgeon (under supervision) on 3% more cases than females (61% (115,948/189,378) to 58% (50,285/86,375), respectively; p < 0.001), and independent operator (unsupervised) on 1% more cases. A similar trend of higher operative numbers in male trainees was seen for senior (ST6 to 8) trainees (+5% and +1%; p < 0.001), those with no time OOP (+6% and +8%; p < 0.001), and those with orthopaedic experience prior to orthopaedic specialty training (+7% and +3% for lead surgeon and independent operator, respectively; p < 0.001). The gender difference was less marked for those on LTFT training, those who took time OOP, and those with no prior orthopaedic experience. Conclusion This study showed that males perform 3% more cases as the lead surgeon than females during UK orthopaedic training (p < 0.001). This may be due to differences in how cases are recorded, but must engender further research to ensure that all surgeons are treated equitably during their training.
Collapse
Affiliation(s)
- Samantha Downie
- Department of Trauma & Orthopaedics, Ninewells Hospital & Medical School, Dundee, UK
| | | | - Jennifer Dunn
- Department of Trauma & Orthopaedics, Ninewells Hospital & Medical School, Dundee, UK
| | - Thomas Harding
- Department of Trauma & Orthopaedics, Ninewells Hospital & Medical School, Dundee, UK
| | | | - Sarah Gill
- Department of Orthopaedic Surgery, Queen Elizabeth University Hospital Glasgow, Glasgow, UK
| | - Simon Johnson
- Department of Trauma & Orthopaedics, Ninewells Hospital & Medical School, Dundee, UK
| |
Collapse
|
8
|
Ellis R, Cleland J, Lee AJ, Scrimgeour DSG, Brennan PA. A cross-sectional study examining MRCS performance by core surgical training location. MEDICAL TEACHER 2022; 44:388-393. [PMID: 34727832 DOI: 10.1080/0142159x.2021.1995599] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
BACKGROUND In the UK, core surgical training (CST) is the first specialty experience that early-career surgeons receive but training differs significantly across CST deaneries. To identify the impact these differences have on trainee performance, we assessed whether success at the Membership of the Royal College of Surgeons (MRCS) examinations is associated with CST deanery. METHODS A retrospective cohort study of UK trainees in CST who attempted MRCS between 2014 and 2020 (n = 1104). Chi-squared tests examined associations between locality and first-attempt MRCS performance. Multivariate logistic regression models identified the likelihood of MRCS success depending on CST deanery. RESULTS MRCS Part A and Part B pass rates were associated with CST deanery (p < 0.001 and p = 0.013, respectively). Candidates that trained in Thames Valley (Odds Ratio [OR] 2.52 (95% Confidence Interval [CI] 1.00-6.42), North Central and East London (OR 2.37 [95% CI 1.04-5.40]) or South London (OR 2.36 [95% CI 1.09-5.10]) were each more than twice as likely to pass MRCS Part A at first attempt. Trainees from North Central and East London were more than ten times more likely to pass MRCS Part B at first attempt (OR 10.59 [95% CI 1.23-51.00]). However, 68% of candidates attempted Part A prior to CST and 48% attempted Part B before or during the first year of CST. CONCLUSION MRCS performance is associated with CST deanery; however, many candidates passed the exam with little or any CST experience suggesting that some deaneries attract high academic performers. MRCS performance is therefore not a suitable marker of CST training quality.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | | | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| |
Collapse
|
9
|
Ellis R, Brennan PA, Lee AJ, Scrimgeour DS, Cleland J. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study. J R Soc Med 2022; 115:257-272. [PMID: 35171739 PMCID: PMC9234885 DOI: 10.1177/01410768221079018] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Objective A recent independent review on diversity and inclusivity highlighted concerns
that barriers to surgical career progression exist for some groups of
individuals and not others. Group-level differences in performance at the
Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS)
examinations have been identified but are yet to be investigated. We aimed
to characterise the relationship between sociodemographic differences and
performance at MRCS. Design Retrospective cohort study. Setting Secondary care. Participants All UK MRCS candidates attempting Part A (n = 5780) and Part
B (n = 2600) between 2013 and 2019 with linked
sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk). Main outcome measures Chi-square tests established univariate associations with MRCS performance.
Multiple logistic regression identified independent predictors of success,
adjusted for medical school performance. Results Statistically significant differences in MRCS pass rates were found according
to gender, ethnicity, age, graduate status, educational background and
socioeconomic status (all p < 0.05). After adjusting for
prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence
interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were
independent predictors of MRCS Part A success and being a non-graduate (OR
1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51,
95% CI 1.08–2.10) were independent predictors of Part B success. Black and
minority ethnic groups were significantly less likely to pass MRCS Part B at
their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR
0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White
candidates. Conclusions There is significant group-level differential attainment at MRCS, likely to
represent the accumulation of privilege and disadvantage experienced by
individuals throughout their education and training. Those leading surgical
education now have a responsibility to identify and address the causes of
these attainment differences.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Urology Department, Nottingham University Hospitals, Nottingham, NG5 1PB, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, UK
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB24 3FX, UK
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, UK
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, 308232, Singapore
| |
Collapse
|
10
|
Ellis R, Brennan PA, Scrimgeour DSG, Lee AJ, Cleland J. Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study. BMJ Open 2022; 12:e054616. [PMID: 34987044 PMCID: PMC8734024 DOI: 10.1136/bmjopen-2021-054616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/19/2021] [Accepted: 12/02/2021] [Indexed: 12/03/2022] Open
Abstract
OBJECTIVES The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values. DESIGN AND PARTICIPANTS A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database. METHODS We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt. RESULTS MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05). CONCLUSIONS There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.
Collapse
Affiliation(s)
- Ricky Ellis
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| | - Duncan S G Scrimgeour
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, UK
| | - Amanda J Lee
- Medical Statistics Team, University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
11
|
Ellis R, Scrimgeour DSG, Brennan PA, Lee AJ, Cleland J. Does performance at medical school predict success at the Intercollegiate Membership of the Royal College of Surgeons (MRCS) examination? A retrospective cohort study. BMJ Open 2021; 11:e046615. [PMID: 34400449 PMCID: PMC8370550 DOI: 10.1136/bmjopen-2020-046615] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
BACKGROUND Identifying predictors of success in postgraduate examinations can help guide the career choices of medical students and may aid early identification of trainees requiring extra support to progress in specialty training. We assessed whether performance on the educational performance measurement (EPM) and situational judgement test (SJT) used for selection into foundation training predicted success at the Membership of the Royal College of Surgeons (MRCS) examination. METHODS This was a longitudinal, cohort study using data from the UK Medical Education Database (https://www.ukmed.ac.uk). UK medical graduates who had attempted Part A (n=2585) and Part B (n=755) of the MRCS between 2014 and 2017 were included. χ2 and independent t-tests were used to examine the relationship between medical school performance and sociodemographic factors with first-attempt success at MRCS Part A and B. Multivariate logistic regression was employed to identify independent predictors of MRCS performance. RESULTS The odds of passing MRCS increased by 55% for Part A (OR 1.55 (95% CI 1.48 to 1.61)) and 23% for Part B (1.23 (1.14 to 1.32)) for every additional EPM decile point gained. For every point awarded for additional degrees in the EPM, candidates were 20% more likely to pass MRCS Part A (1.20 (1.13 to 1.29)) and 17% more likely to pass Part B (1.17 (1.04 to 1.33)). For every point awarded for publications in the EPM, candidates were 14% more likely to pass MRCS Part A (1.14 (1.01 to 1.28)). SJT score was not a statistically significant independent predictor of MRCS success. CONCLUSION This study has demonstrated the EPM's independent predictive power and found that medical school performance deciles are the most significant measure of predicting later success in the MRCS. These findings can be used by medical schools, training boards and workforce planners to inform evidence-based and contemporary selection and assessment strategies.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Department of Urology, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Duncan S G Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| | - Amanda J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
12
|
Ellis R, Cleland J, Scrimgeour D, Lee AJ, Brennan PA. The impact of disability on performance in a high-stakes postgraduate surgical examination: a retrospective cohort study. J R Soc Med 2021; 115:58-68. [PMID: 34269623 DOI: 10.1177/01410768211032573] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
OBJECTIVE Despite rising numbers of doctors in the workforce with disabilities, little is known about the impact of disabilities on postgraduate performance. To ensure all groups are treated fairly in surgical training, it is essential to know whether any attainment differences exist in markers of surgical performance. To address this gap, we assessed the impact of disabilities on performance on the Intercollegiate Membership of the Royal College of Surgeons examination (MRCS). DESIGN Retrospective cohort study. SETTING Secondary care. PARTICIPANTS All UK MRCS candidates attempting Part A (n = 9600) and Part B (n = 4560) between 2007 and 2017 with linked disability data in the UK Medical Education Database (https://www.ukmed.ac.uk) were included. MAIN OUTCOME MEASURES Chi-square tests and correlation coefficients established univariate associations with MRCS performance, while multiple logistic regressions identified independent predictors of success. RESULTS Though MRCS Part B pass rates were similar (p = 0.339), candidates with registered disabilities had significantly lower first-attempt Part A pass rates (46.3% vs. 59.8%, p < 0.001). Candidates with disabilities also performed less well in examinations taken throughout school and medical school, and after adjusting for prior academic performance and sociodemographic predictors of success, logistic regression found that candidates with disabilities were no less likely to pass MRCS than their peers (odds ratio 1.04, 95% confidence interval 0.66 to 1.62). No significant variation was found in MRCS performance between type of disability or degree of limitations caused by disability (p > 0.05). CONCLUSION Although candidates with registered disabilities performed less well in formal, written examinations, our data indicate that they are as likely to pass MRCS at first attempt as their peers who achieved similar grades at high school and medical school. In order to enable equity in career progression, further work is needed to investigate the causes of attainment differences in early career assessments.
Collapse
Affiliation(s)
- R Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen AB25 2ZD, UK.,Urology Department, Nottingham University Hospitals NHS Trust, Nottingham NG5 1PB, UK
| | - J Cleland
- 371018Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308232, Singapore
| | - Dsg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen AB25 2ZD, UK.,Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen AB25 2ZN, UK
| | - A J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, UK
| | - P A Brennan
- Department of Maxillo-Facial Surgery, 112006Queen Alexandra Hospital, Portsmouth, PO6 3LY, UK
| |
Collapse
|
13
|
Ellis R, Cleland J, Scrimgeour DSG, Lee AJ, Brennan PA. A cross-sectional study examining the association between MRCS performance and surgeons receiving sanctions against their medical registration. Surgeon 2021; 20:211-215. [PMID: 34030984 DOI: 10.1016/j.surge.2021.04.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 03/20/2021] [Accepted: 04/06/2021] [Indexed: 11/18/2022]
Abstract
BACKGROUND Fitness to practice (FtP) investigations by the General Medical Council (GMC) safeguard patients and maintain the integrity of the medical profession. The likelihood of FtP sanctions is influenced by specialty and socio-demographic factors and can be predicted by performance at postgraduate examinations. This is the first study to characterise the prevalence of FtP sanctions in early-career surgeons and to examine the association with performance at the Membership of the Royal College of Surgeons (MRCS) examination. METHODS All UK graduates who attempted MRCS between September 2007-January 2020 were matched to the GMC list of registered medical practitioners. Clinicians who had active FtP sanctions between 28th August 2018 and 28th August 2020 were identified. Data were anonymised by RCS England prior to analysis. RESULTS Of 11,660 candidates who attempted MRCS within the study period, only 31 (0.3%) had FtP sanctions between 2018 and 2020. Of these, 12 had active conditions on registration, seven had undertakings and 14 had warnings. There was no statistically significant difference in MRCS performance in either Parts A or B of the examination for those with and those free from FtP sanctions (P > 0.05). CONCLUSIONS In this, the largest study of MRCS candidates to date, the prevalence of active FtP sanctions in early-career surgeons was 0.3%, significantly lower than the prevalence of sanctions across more experienced UK surgeons (0.9%). These data highlight early-career surgeons as a low-risk group for disciplinary action and should reassure patients and medical professionals of the rarity of FtP sanctions.
Collapse
Affiliation(s)
- R Ellis
- Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom; Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom.
| | - J Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - D S G Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom; Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, United Kingdom
| | - A J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, AB25 2ZD, United Kingdom
| | - P A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, United Kingdom
| |
Collapse
|
14
|
Mapara R, Munro C, Ajekigbe M, McLachlan G. Sexism, racism, and homophobia at the Royal College of Surgeons of England. BMJ 2021; 373:n998. [PMID: 33879524 DOI: 10.1136/bmj.n998] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
|
15
|
Ellis R, Brennan P, Scrimgeour DS, Lee AJ, Cleland J. Performance at medical school selection correlates with success in Part A of the intercollegiate Membership of the Royal College of Surgeons (MRCS) examination. Postgrad Med J 2021; 98:e19. [PMID: 33692157 DOI: 10.1136/postgradmedj-2021-139748] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 02/15/2021] [Accepted: 02/20/2021] [Indexed: 11/04/2022]
Abstract
Medical schools in the UK typically use prior academic attainment and an admissions test (University Clinical Aptitude Test (UCAT), Biomedical Admissions Test (BMAT) or the Graduate Medical School Admissions Test (GAMSAT)) to help select applicants for interview. To justify their use, more information is needed about the predictive validity of these tests. Thus, we investigated the relationship between performance in admissions tests and the Membership of the Royal College of Surgeons (MRCS) examination.The UKMED database (https://www.ukmed.ac.uk) was used to access medical school selection data for all UK graduates who attempted MRCS Part A (n=11 570) and Part B (n=5690) between 2007 and 2019. Univariate and multivariate logistic regression models identified independent predictors of MRCS success. Pearson correlation coefficients examined the linear relationship between test scores and MRCS performance.Successful MRCS Part A candidates scored higher in A-Levels, UCAT, BMAT and GAMSAT (p<0.05). No significant differences were observed for MRCS Part B. All admissions tests were found to independently predict MRCS Part A performance after adjusting for prior academic attainment (A-Level performance) (p<0.05). Admission test scores demonstrated statistically significant correlations with MRCS Part A performance (p<0.001).The utility of admissions tests is clear with respect to helping medical schools select from large numbers of applicants for a limited number of places. Additionally, these tests appear to offer incremental value above A-Level performance alone. We expect this data to guide medical schools' use of admissions test scores in their selection process.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK .,Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Peter Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, Portsmouth, UK
| | - Duncan Sg Scrimgeour
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, Aberdeen, UK
| | - Amanda J Lee
- Department of Medical Statistics, Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research & Scholarship Unit, Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
16
|
Nomura O, Onishi H, Park YS, Michihata N, Kobayashi T, Kaneko K, Yoshikawa T, Ishiguro A. Predictors of performance on the pediatric board certification examination. BMC MEDICAL EDUCATION 2021; 21:122. [PMID: 33618691 PMCID: PMC7898761 DOI: 10.1186/s12909-021-02515-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Accepted: 01/26/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Examining the predictors of summative assessment performance is important for improving educational programs and structuring appropriate learning environments for trainees. However, predictors of certification examination performance in pediatric postgraduate education have not been comprehensively investigated in Japan. METHODS The Pediatric Board Examination database in Japan, which includes 1578 postgraduate trainees from 2015 to 2016, was analyzed. The examinations included multiple-choice questions (MCQs), case summary reports, and an interview, and the predictors for each of these components were investigated by multiple regression analysis. RESULTS The number of examination attempts and the training duration were significant negative predictors of the scores for the MCQ, case summary, and interview. Employment at a community hospital or private university hospital were negative predictors of the MCQ and case summary score, respectively. Female sex and the number of academic presentations positively predicted the case summary and interview scores. The number of research publications was a positive predictor of the MCQ score, and employment at a community hospital was a positive predictor of the case summary score. CONCLUSION This study found that delayed and repeated examination taking were negative predictors, while the scholarly activity of trainees was a positive predictor, of pediatric board certification examination performance.
Collapse
Affiliation(s)
- Osamu Nomura
- Department of Emergency and Disaster Medicine, Hirosaki University, 5 Zaifu, Hirosaki, Aomori, 036-8216, Japan.
- Center for Postgraduate Education and Training, National Center for Child Health and Development, 2-10-1 Okura, Setagaya-ku, Tokyo, Japan.
| | - Hirotaka Onishi
- International Research Center for Medical Education, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Yoon Soo Park
- Harvard Medical School, Harvard University, 25 Shattuck St, Boston, MA, USA
- Massachusetts General Hospital, 55 Fruit Street Bartlett (BAR-2R-202), Boston, MA, USA
| | - Nobuaki Michihata
- Department of Health Services Research, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, Japan
| | - Tohru Kobayashi
- Department of Management and Strategy, Clinical Research Center, National Center for Child Health and Development, 2-10-1 Okura, Setagaya-ku, Tokyo, Japan
| | - Kazunari Kaneko
- Department of Pediatrics, Kansai Medical University, 2-5-1 Shin-machi, Hirakata, Osaka, Japan
| | - Tetsushi Yoshikawa
- Department of Pediatrics, Fujita Health University, 1-98 Dengakugakubo, Kutsukake-cho, Toyoake, Aichi, Japan
| | - Akira Ishiguro
- Center for Postgraduate Education and Training, National Center for Child Health and Development, 2-10-1 Okura, Setagaya-ku, Tokyo, Japan
| |
Collapse
|
17
|
Brennan P, Scrimgeour D, Shakib K. Editorial: exploring evidence-based options for reducing the duration of UK OMFS training. Br J Oral Maxillofac Surg 2020; 58:1216-1218. [DOI: 10.1016/j.bjoms.2020.01.013] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2019] [Accepted: 01/02/2020] [Indexed: 11/16/2022]
|
18
|
Singh RP. Should we recruit trainees into the Oral and Maxillofacial Surgery pathway at the beginning of the second degree to ensure the long term viability of the specialty? Br J Oral Maxillofac Surg 2020; 58:1353-1354. [PMID: 33208287 DOI: 10.1016/j.bjoms.2020.10.013] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 10/07/2020] [Indexed: 12/13/2022]
Abstract
There has been a sustained decline in the number of trainees applying for entry into the Oral and Maxillofacial Surgery (OMFS) training program, which has sparked further debate on the issue of OMFS training and the future of the specialty in the United Kingdom. Here I discuss the option of recruiting the trainees into the program at the beginning of their second degree to streamline the training, and to improve the recruitment drive to ensure long term viability of the specialty.
Collapse
Affiliation(s)
- Rabindra P Singh
- Department of Oral and Maxillofacial Surgery, University Hospital Southampton, Tremona Road, Southampton, SO16 6YD, United Kingdom.
| |
Collapse
|