1
|
Sidhu NS, Fleming S. Re-examining single-moment-in-time high-stakes examinations in specialist training: A critical narrative review. MEDICAL TEACHER 2024; 46:528-536. [PMID: 37740944 DOI: 10.1080/0142159x.2023.2260081] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/25/2023]
Abstract
In this critical narrative review, we challenge the belief that single-moment-in-time high-stakes examinations (SMITHSEx) are an essential component of contemporary specialist training. We explore the arguments both for and against SMITHSEx, examine potential alternatives, and discuss the barriers to change.SMITHSEx are viewed as the "gold standard" assessment of competence but focus excessively on knowledge assessment rather than capturing essential competencies required for safe and competent workplace performance. Contrary to popular belief, regulatory bodies do not mandate SMITHSEx in specialist training. Though acting as significant drivers of learning and professional identity formation, these attributes are not exclusive to SMITHSEx.Skills such as crisis management, procedural skills, professionalism, communication, collaboration, lifelong learning, reflection on practice, and judgement are often overlooked by SMITHSEx. Their inherent design raises questions about the validity and objectivity of SMITHSEx as a measure of workplace competence. They have a detrimental impact on trainee well-being, contributing to burnout and differential attainment.Alternatives to SMITHSEx include continuous low-stakes assessments throughout training, ongoing evaluation of competence in the workplace, and competency-based medical education (CBME) concepts. These aim to provide a more comprehensive and context-specific assessment of trainees' competence while also improving trainee welfare.Specialist training colleges should evolve from exam providers to holistic education sources. Assessments should emphasise essential practical knowledge over trivia, align with clinical practice, aid learning, and be part of a diverse toolkit. Eliminating SMITHSEx from specialist training will foster a competency-based approach, benefiting future medical professionals' well-being and success.
Collapse
Affiliation(s)
- Navdeep S Sidhu
- Department of Anaesthesiology, School of Medicine, University of Auckland, Auckland, New Zealand
- Department of Anaesthesia and Perioperative Medicine, North Shore Hospital, Auckland, New Zealand
| | - Simon Fleming
- Department of Hand Surgery, Royal North Shore Hospital, Sydney, New South Wales, Australia
| |
Collapse
|
2
|
Pinder RJ, Bury F, Sathyamoorthy G, Majeed A, Rao M. Differential attainment in specialty training recruitment in the UK: an observational analysis of the impact of psychometric testing assessment in Public Health postgraduate selection. BMJ Open 2023; 13:e069738. [PMID: 36894198 PMCID: PMC10008157 DOI: 10.1136/bmjopen-2022-069738] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/11/2023] Open
Abstract
OBJECTIVES To determine how current psychometric testing approaches used in selection of postgraduate training in UK Public Health are associated with socioeconomic and sociocultural background of applicants (including ethnicity). DESIGN Observational study using contemporaneous data collected during recruitment and psychometric test scores. SETTING Assessment centre of UK national Public Health recruitment for postgraduate Public Health training. The assessment centre element of selection comprises three psychometric assessments: Rust Advanced Numerical Reasoning, Watson-Glaser Critical Thinking Assessment II and Public Health situational judgement test. PARTICIPANTS 629 applicants completed the assessment centre in 2021. 219 (34.8%) were UK medical graduates, 73 (116%) were international medical graduates and 337 (53.6%) were from backgrounds other than medicine. MAIN OUTCOME MEASURE Multivariable-adjusted progression statistics in the form of adjusted OR (aOR), accounting for age, sex, ethnicity, professional background and surrogate measures of familial socioeconomic and sociocultural status. RESULTS 357 (56.8%) candidates passed all three psychometric tests. Candidate characteristics negatively associated with progression were black ethnicity (aOR 0.19, 0.08 to 0.44), Asian ethnicity (aOR 0.35, 0.16 to 0.71) and coming from a non-UK medical graduate background (aOR 0.05, 0.03 to 0.12); similar differential attainment was observed in each of the psychometric tests. Even within the UK-trained medical cohort, candidates from white British backgrounds were more likely to progress than those from ethnic minorities (89.2% vs 75.0%, p=0.003). CONCLUSION Although perceived to mitigate the risks of conscious and unconscious bias in selection to medical postgraduate training, these psychometric tests demonstrate unexplained variation that suggests differential attainment. Other specialties should enhance their data collection to evaluate the impact of differential attainment on current selection processes and take forward opportunities to mitigate differential attainment where possible.
Collapse
Affiliation(s)
- Richard J Pinder
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
- NIHR Northwest London Applied Research Collaboration, Imperial College London, London, UK
| | - Fran Bury
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
| | - Ganesh Sathyamoorthy
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
- NIHR Northwest London Applied Research Collaboration, Imperial College London, London, UK
- Ethnicity and Health Unit, Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
| | - Azeem Majeed
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
- NIHR Northwest London Applied Research Collaboration, Imperial College London, London, UK
| | - Mala Rao
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
- NIHR Northwest London Applied Research Collaboration, Imperial College London, London, UK
- Ethnicity and Health Unit, Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
| |
Collapse
|
3
|
Ellis R, Brennan PA, Lee AJ, Scrimgeour DS, Cleland J. Differential attainment at MRCS according to gender, ethnicity, age and socioeconomic factors: a retrospective cohort study. J R Soc Med 2022; 115:257-272. [PMID: 35171739 PMCID: PMC9234885 DOI: 10.1177/01410768221079018] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Objective A recent independent review on diversity and inclusivity highlighted concerns
that barriers to surgical career progression exist for some groups of
individuals and not others. Group-level differences in performance at the
Intercollegiate Membership of the Royal Colleges of Surgeons (MRCS)
examinations have been identified but are yet to be investigated. We aimed
to characterise the relationship between sociodemographic differences and
performance at MRCS. Design Retrospective cohort study. Setting Secondary care. Participants All UK MRCS candidates attempting Part A (n = 5780) and Part
B (n = 2600) between 2013 and 2019 with linked
sociodemographic data in the UK Medical Education Database (https://www.ukmed.ac.uk). Main outcome measures Chi-square tests established univariate associations with MRCS performance.
Multiple logistic regression identified independent predictors of success,
adjusted for medical school performance. Results Statistically significant differences in MRCS pass rates were found according
to gender, ethnicity, age, graduate status, educational background and
socioeconomic status (all p < 0.05). After adjusting for
prior academic attainment, being male (odds ratio [OR] 2.34, 95% confidence
interval [CI] 1.87–2.92) or a non-graduate (OR 1.98, 95% CI 1.44–2.74) were
independent predictors of MRCS Part A success and being a non-graduate (OR
1.77, 95% CI 1.15–2.71) and having attended a fee-paying school (OR 1.51,
95% CI 1.08–2.10) were independent predictors of Part B success. Black and
minority ethnic groups were significantly less likely to pass MRCS Part B at
their first attempt (OR 0.41, 95% CI 0.18–0.92 for Black candidates and OR
0.49, 95% CI 0.35–0.69 for Asian candidates) compared to White
candidates. Conclusions There is significant group-level differential attainment at MRCS, likely to
represent the accumulation of privilege and disadvantage experienced by
individuals throughout their education and training. Those leading surgical
education now have a responsibility to identify and address the causes of
these attainment differences.
Collapse
Affiliation(s)
- Ricky Ellis
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Urology Department, Nottingham University Hospitals, Nottingham, NG5 1PB, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, PO6 3LY, UK
| | - Amanda J Lee
- Medical Statistics Team, Institute of Applied Health Sciences, University of Aberdeen, AB24 3FX, UK
| | - Duncan Sg Scrimgeour
- Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, AB24 3FX, UK.,Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, AB25 2ZN, UK
| | - Jennifer Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, 308232, Singapore
| |
Collapse
|
4
|
Ellis R, Brennan PA, Scrimgeour DSG, Lee AJ, Cleland J. Does performance at the intercollegiate Membership of the Royal Colleges of Surgeons (MRCS) examination vary according to UK medical school and course type? A retrospective cohort study. BMJ Open 2022; 12:e054616. [PMID: 34987044 PMCID: PMC8734024 DOI: 10.1136/bmjopen-2021-054616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/19/2021] [Accepted: 12/02/2021] [Indexed: 12/03/2022] Open
Abstract
OBJECTIVES The knowledge, skills and behaviours required of new UK medical graduates are the same but how these are achieved differs given medical schools vary in their mission, curricula and pedagogy. Medical school differences seem to influence performance on postgraduate assessments. To date, the relationship between medical schools, course types and performance at the Membership of the Royal Colleges of Surgeons examination (MRCS) has not been investigated. Understanding this relationship is vital to achieving alignment across undergraduate and postgraduate training, learning and assessment values. DESIGN AND PARTICIPANTS A retrospective longitudinal cohort study of UK medical graduates who attempted MRCS Part A (n=9730) and MRCS Part B (n=4645) between 2007 and 2017, using individual-level linked sociodemographic and prior academic attainment data from the UK Medical Education Database. METHODS We studied MRCS performance across all UK medical schools and examined relationships between potential predictors and MRCS performance using χ2 analysis. Multivariate logistic regression models identified independent predictors of MRCS success at first attempt. RESULTS MRCS pass rates differed significantly between individual medical schools (p<0.001) but not after adjusting for prior A-Level performance. Candidates from courses other than those described as problem-based learning (PBL) were 53% more likely to pass MRCS Part A (OR 1.53 (95% CI 1.25 to 1.87) and 54% more likely to pass Part B (OR 1.54 (1.05 to 2.25)) at first attempt after adjusting for prior academic performance. Attending a Standard-Entry 5-year medicine programme, having no prior degree and attending a Russell Group university were independent predictors of MRCS success in regression models (p<0.05). CONCLUSIONS There are significant differences in MRCS performance between medical schools. However, this variation is largely due to individual factors such as academic ability, rather than medical school factors. This study also highlights group level attainment differences that warrant further investigation to ensure equity within medical training.
Collapse
Affiliation(s)
- Ricky Ellis
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Urology Department, Nottingham University Hospitals NHS Trust, Nottingham, UK
| | - Peter A Brennan
- Department of Maxillo-Facial Surgery, Queen Alexandra Hospital, Portsmouth, UK
| | - Duncan S G Scrimgeour
- University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
- Department of Colorectal Surgery, Aberdeen Royal Infirmary, Aberdeen, UK
| | - Amanda J Lee
- Medical Statistics Team, University of Aberdeen Institute of Applied Health Sciences, Aberdeen, UK
| | - Jennifer Cleland
- Medical Education Research and Scholarship Unit (MERSU), Lee Kong Chian School of Medicine, Singapore
| |
Collapse
|
5
|
Blackham R, Hamdorf J. Critical aspects in developing curriculum-based assessment for emerging surgical procedures. INTERNATIONAL JOURNAL OF MEDICAL EDUCATION 2021; 12:264-266. [PMID: 34942600 PMCID: PMC8995009 DOI: 10.5116/ijme.61ba.07c6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/12/2021] [Accepted: 12/15/2021] [Indexed: 06/14/2023]
|
6
|
Chu K, Sathanandan S. The virtual Clinical Assessment of Skills and Competence: the impact and challenges of a digitised final examination. BJPsych Bull 2021; 47:110-115. [PMID: 34937596 PMCID: PMC10063989 DOI: 10.1192/bjb.2021.112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
The COVID-19 pandemic has affected how clinical examinations are conducted, resulting in the Royal College of Psychiatrists delivering the Clinical Assessment of Skills and Competence virtually. Although this pragmatic step has allowed for progression of training, it has come at the cost of a significantly altered examination experience. This article aims to explore the fairness of such an examination, the difference in trainee experience, and the use of telemedicine to consider what might be lost as well as gained at a time when medical education and delivery of healthcare are moving toward the digitised frontier.
Collapse
Affiliation(s)
- Kenny Chu
- St Pancras Hospital, Camden and Islington NHS Foundation Trust, UK
| | | |
Collapse
|
7
|
Silkens MEWM, Sarker SJ, Medisauskaite A. Uncovering trends in training progression for a national cohort of psychiatry trainees: discrete-time survival analysis. BJPsych Open 2021; 7:e120. [PMID: 34184624 PMCID: PMC8269924 DOI: 10.1192/bjo.2021.958] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND The global rise in mental health issues calls for a strong psychiatry workforce. Yet, psychiatry training worldwide is facing recruitment challenges, causing unfilled consultant posts and possibly threatening the quality of patient care. An in-depth understanding of trainees' progression through training is warranted to explore what happens to recruited trainees during training. AIMS To uncover current trends in psychiatry trainees' progression through training in the UK. METHOD This national retrospective cohort study with data from the UK Medical Education Database used discrete-time survival analysis to analyse training progression for those trainees who started their core psychiatry post in 2012-2017 (2820 trainees; 59.6% female, 67.6% UK graduates (UKGs)). The impact of sociodemographic characteristics on training progression were also investigated. RESULTS The overall probability of completing training in 6 years (minimum years required to complete psychiatry training in the UK) was 17.2% (ranging from 4.8% for non-UKG females to 29% for UKG males). The probability to not progress was highest (57.1%) from core to specialty training. For UKGs, trainees from ethnicities other than White, trainees with a disability, and trainees who had experienced childhood social deprivation (measured as entitlement to free school meals) had a significantly (P ≤ 0.02) lower probability of completing training in 6 years. CONCLUSIONS Less than one in five psychiatry trainees are likely to complete training in 6 years and this probability varies across groups of doctors. Completing psychiatry training in 6 years is, therefore, the exception rather than the norm and this has important implications for trainees, those planning psychiatry workforces or responsible for psychiatry training.
Collapse
Affiliation(s)
- Milou E W M Silkens
- Research Department of Medical Education, UCL Medical School, University College London, UK
| | - Shah-Jalal Sarker
- Research Department of Medical Education, UCL Medical School, University College London, UK; and UCL Queen Square Institute of Neurology, School of Life & Medical Sciences, University College London, UK
| | - Asta Medisauskaite
- Research Department of Medical Education, UCL Medical School, University College London, UK
| |
Collapse
|