The mediators of minority ethnic underperformance in final medical school examinations

Construct-level predictive validity of educational attainment and intellectual aptitude tests in medical student selection: meta-regression of six UK longitudinal studies

BACKGROUND

Measures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.

METHODS

Construct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.

RESULTS

Meta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.

CONCLUSIONS

Educational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

Gender, ethnicity and graduate status, and junior doctors' self-reported preparedness for clinical practice: national questionnaire surveys

OBJECTIVES

Medical schools need to ensure that graduates feel well prepared for their first medical job. Our objective was to report on differences in junior doctors' self-reported preparedness for work according to gender, ethnicity and graduate status.

DESIGN

Postal and electronic questionnaires.

SETTING

UK.

PARTICIPANTS

Medical graduates of 2008 and 2009, from all UK medical schools, one year after graduation.

MAIN OUTCOME MEASURES

The main outcome measure was the doctors' level of agreement with the statement that 'My experience at medical school prepared me well for the jobs I have undertaken so far', to which respondents were asked to reply on a scale from 'strongly agree' to 'strongly disagree'.

RESULTS

Women were slightly less likely than men to agree that they felt well prepared for work (50% of women agreed or strongly agreed vs. 54% of men), independently of medical school, ethnicity, graduate entry status and intercalated degree status, although they were no more likely than men to regard lack of preparedness as having been a problem for them. Adjusting for the other subgroup differences, non-white respondents were less likely to report feeling well prepared than white (44% vs. 54%), and were more likely to indicate that lack of preparedness was a problem (30% non-white vs. 24% white). There were also some gender and ethnic differences in preparedness for specific areas of work.

CONCLUSIONS

The identified gender and ethnic differences need to be further explored to determine whether they are due to differences in self-confidence or in actual preparedness.

The hidden medical school: a longitudinal study of how social networks form, and how they relate to academic performance

BACKGROUND

UK medical schools typically have over 300 students per year, making it impossible for students to know all the others well.

AIMS

This longitudinal cohort study measured the formation of medical student social networks and their relationship to grades.

METHOD

In November 2009, 215/317 (68%) Year 2 UCL medical students reported their friendships with others in their year, by questionnaire. Multiple regression assessed the relationship between friendships, exam results and background variables (obtained from student records), with permutation testing to assess statistical significance.

RESULTS

Students of the same sex, the same ethnic group, and in the same tutor and small groups (to which they were randomly assigned at the start of medical school) were socially closer. Taking into account absolute difference in Year 1 grades, Year 2 pairs who were socially closer in November 2009 had more similar May 2010 grades. Individual student variables did not predict similarity in 2010 grades after taking friendships into account.

CONCLUSIONS

The results suggest that medical students chose friends of the same sex and ethnic group as themselves; but random allocation of students to tutor groups also influenced friendships. Most importantly, friendships related to subsequent exam performance, suggesting friendship may influence learning.