Views of doctors of varying disciplines on HPAT-Ireland as a selection tool for medicine

Preparing for selection success: Socio-demographic differences in opportunities and obstacles

INTRODUCTION

There is a growing drive to improve the heterogeneity of medical school cohorts. However, those from lower socio-economic groups remain under-represented. Understanding the methods used by applicants to prepare for medical school selection, and the challenges they face with respect to opportunities and access, may provide important insights to this lack of diversity. This research considered the influence of socio-demographic background on preparation for medical school selection and on the successful offer of a place.

METHODS

All eligible applicants to a large UK medical school were invited to complete an anonymous online survey (international and mature applicants excluded), detailing demographic data and experiences of preparing for selection and challenges faced (n = 1885). Selection outcomes across all UK medical schools were examined for consenting students (n = 955). Univariate and multivariate analyses explored the associations of preparatory activities and demographic data with subsequent offer of a place at a UK medical school.

RESULTS

The survey response rate was 66.4%. Clinical work experience (hospital or general practice), fee-based courses for admissions tests and school preparation courses for interview were activities significantly associated with the offer of a place (P < .05). Those attending independent (private) schools most frequently reported school support and fee-based courses to support preparation (P < .01). Applicants from state non-selective (SNS) schools and lower socio-economic groups more frequently reported challenges in accessing fee-based support, school interview courses and clinical work experience (P ≤ .02).

DISCUSSION

Clinical work experience, commercial courses for admissions tests and school-based support for interview represent areas of preparation that are associated with success. However, they also represent areas that are more challenging to access for demographic groups traditionally under-represented in medicine. Addressing complex issues of fairness in highly selective higher education settings can appear an insurmountable task. These preparatory activities represent key areas for applicants, schools and institutions to explore and address.

Testing for medical school selection: What are prospective doctors' experiences and perceptions of the GAMSAT and what are the consequences of testing?

Written tests for selection into medicine have demonstrated reliability and there is accumulating evidence regarding their validity, but we know little about the broader impacts or consequences of medical school selection tests from the perspectives of key stakeholders. In this first Australian study of its kind, we use consequential validity as a theoretical lens to examine how medical school students and applicants view and experience the Graduate Medical Schools Admission Test (GAMSAT), and the consequences of testing. Participants (n = 447) were recruited from five graduate-entry medical schools across Australia and a publicly available online test preparation forum. An online survey was used to gather demographic information, and quantitative and qualitative data. Quantitative data were analysed via descriptive statistics and qualitative data were thematically analysed. The findings showed there was a considerable financial burden associated with preparing for and sitting the GAMSAT and moderate agreement regarding the GAMSAT as a fair selection method. The main unintended consequences of using the GAMSAT as a selection tool included barriers related to test affordability and language, and socialisation into the hidden curriculum of medicine. Selection tools such as the GAMSAT have some limitations when the goals are to support equitable participation in medicine and professional identity development. Our study highlights the value interpretive and theoretically-informed research in contributing to the evidence base on medical school selection.

A systematic review of stakeholder views of selection methods for medical schools admission

BACKGROUND

The purpose of this paper is to systematically review the literature with respect to stakeholder views of selection methods for medical school admissions.

METHODS

An electronic search of nine databases was conducted between January 2000-July 2014. Two reviewers independently assessed all titles (n = 1017) and retained abstracts (n = 233) for relevance. Methodological quality of quantitative papers was assessed using the MERSQI instrument. The overall quality of evidence in this field was low. Evidence was synthesised in a narrative review.

RESULTS

Applicants support interviews, and multiple mini interviews (MMIs). There is emerging evidence that situational judgement tests (SJTs) and selection centres (SCs) are also well regarded, but aptitude tests less so. Selectors endorse the use of interviews in general and in particular MMIs judging them to be fair, relevant and appropriate, with emerging evidence of similarly positive reactions to SCs. Aptitude tests and academic records were valued in decisions of whom to call to interview. Medical students prefer interviews based selection to cognitive aptitude tests. They are unconvinced about the transparency and veracity of written applications. Perceptions of organisational justice, which describe views of fairness in organisational processes, appear to be highly influential on stakeholders' views of the acceptability of selection methods. In particular procedural justice (perceived fairness of selection tools in terms of job relevance and characteristics of the test) and distributive justice (perceived fairness of selection outcomes in terms of equal opportunity and equity), appear to be important considerations when deciding on acceptability of selection methods. There were significant gaps with respect to both key stakeholder groups and the range of selection tools assessed.

CONCLUSIONS

Notwithstanding the observed limitations in the quality of research in this field, there appears to be broad concordance of views on the various selection methods, across the diverse stakeholders groups. This review highlights the need for better standards, more appropriate methodologies and for broadening the scope of stakeholder research.

The construct validity of HPAT-Ireland for the selection of medical students: unresolved issues and future research implications

Aptitude tests are widely used in selection. However, despite certain advantages their use remains controversial. This paper aims to critically appraise five sources of evidence for the construct validity of the Health Professions Admission Test (HPAT)-Ireland, an aptitude test used for selecting undergraduate medical students. The objectives are to identify gaps in the evidence, draw comparisons with other aptitude tests and outline future research directions. Our appraisal of the literature found that stakeholder feedback indicates that there is reasonable evidence for test content validity for two of the three sections of HPAT-Ireland. By contrast the Non-Verbal Reasoning section is widely criticised as having limited relevance to medical school performance and future clinical practice. In terms of concurrent validity there is a significant small to medium, negative correlation with school exit examinations, but not consistently so across all studies (r = -0.18, -0.28, 0.017). Likewise predictive validity studies vary, from negative to moderate strength correlations with examination performance during early years at medical school. Five studies indicate that HPAT-Ireland is supported in principle by the majority of stakeholders. While one consequence of its introduction is that successful applicants are now coming from more diverse academic backgrounds, there is no evidence that the socio-economic background of medical school entrants has been altered significantly. Negative perceptions of unfairness relating to gender, coaching and socio-economics remain. The evidence to date suggests that while there are slight gender differences, initially favouring males, these vary year on year. In conclusion, the attitudes towards, and performance of, HPAT-Ireland is not unlike that of other aptitude tests widely used internationally. The main justifications for its introduction have been achieved, in that Ireland no longer relies exclusively on a single measure of academic record for selection to medical school. However a number of areas require further research and exploration.