The economic costs of selecting medical students: An Australian case study

Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153

Selection is the first assessment of medical education and training. Medical schools must select from a pool of academically successful applicants and ensure that the way in which they choose future clinicians is robust, defensible, fair to all who apply and cost-effective. However, there is no comprehensive and evidence-informed guide to help those tasked with setting up or rejuvenating their local selection process. To address this gap, our guide draws on the latest research, international case studies and consideration of common dilemmas to provide practical guidance for designing, implementing and evaluating an effective medical school selection system. We draw on a model from the field of instructional design to frame the many different activities involved in doing so: the ADDIE model. ADDIE provides a systematic framework of Analysis (of the outcomes to be achieved by the selection process, and the barriers and facilitators to achieving these), Design (what tools and content are needed so the goals of selection are achieved), Development (what materials and resources are needed and available), Implementation (plan [including piloting], do study and adjust) and Evaluation (quality assurance is embedded throughout but the last step involves extensive evaluation of the entire process and its outcomes).HIGHLIGHTSRobust, defensible and fair selection into medical school is essential. This guide systematically covers the processes required to achieve this, from needs analysis through design, development and implementation, to evaluation of the success of a selection process.

"Important but risky": attitudes of global thought leaders towards cost and value research in health professions education

Studies of cost and value can inform educational decision making, yet our understanding of the barriers to such research is incomplete. To address this gap, our aim was to explore the attitudes of global thought leaders in HPE towards cost and value research. This was a qualitative virtual interview study underpinned by social constructionism. In telephone or videoconference interviews in 2018-2019, we asked global healthcare professional thought leaders their views regarding HPE cost and value research, outstanding research questions in this area and why addressing these questions was important. Analysis was inductive and thematic, and incorporated review and comments from the original interviewees (member checking). We interviewed 11 thought leaders, nine of whom gave later feedback on our data interpretation (member checking). We identified four themes: Cost research is really important but potentially risky (quantifying and reporting costs provides evidence for decision-making but could lead to increased accountability and loss of autonomy); I don't have the knowledge and skills (lack of economic literacy); it's not what I went into education research to do (professional identity); and it's difficult to generate generalizable findings (the importance of context). This study contributes to a wider conversation in the literature about cost and value research by bringing in the views of global HPE thought leaders. Our findings provide insight to inform how best to engage and empower educators and researchers in the processes of asking and answering meaningful, acceptable and relevant cost and value questions in HPE.

Recruitment and retention of volunteer multiple mini interview interviewers: Understanding their motivations

BACKGROUND

Without volunteer interviewers, many universities would not be able to run multiple mini interviews (MMIs) due to the prohibitive cost of paying interviewers. Despite the opportunity cost borne by volunteers, many interviewers participate in multiple MMI sessions per year and volunteer year after year. There is surprisingly little research into what motivates interviewers to volunteer as MMI interviewers. This research aims to explore both what motivates individuals to volunteer to interview in MMIs for undergraduate medical selection and what adds and detracts value from their participation.

METHODS

We applied a qualitative sequential two-phase design consisting of open-ended survey questions, followed by semi-structured interviews to explore interviewers' motivators in more depth. The survey data on motivators and the six functions from the volunteer functions inventory (VFI) informed interview data collection and provided a lens through which to examine MMI interviewer motivations. Content analysis was used to analyse the survey data. Framework analysis was used to analyse the interview data.

RESULTS

The survey was completed by 108 interviewers (50% response rate), and 19 semi-structured interviews were conducted (54% response rate). From the content analysis, the time commitment of involvement was the biggest detractor identified by participants. Through the framework analysis, five overarching motivators were developed: (i) acting on values, (ii) gaining understanding, (iii) gaining personal satisfaction and gratification, (iv) shaping the future workforce and (v) having social interaction. These mirrored five of the six functions proposed in the VFI.

CONCLUSIONS

There are a range of motivating factors that influenced the participants' decision to volunteer as an interviewer for MMIs. Some motivations were for the benefit of others, some were self-serving, and some a combination of both. Universities should utilise these motivating factors to aid in targeted recruitment of volunteer interviewers.

Unpacking economic programme theory for supervision training: Preliminary steps towards realist economic evaluation

INTRODUCTION

Cost studies are increasingly popular given resource constraints. While scholars stress the context-dependent nature of cost, and the importance of theory, cost studies remain context-blind and atheoretical. However, realist economic evaluation (REE) privileges context and the testing/refinement of economic programme theory. This preliminary REE serves to test and refine economic programme theory for supervision training programmes of different durations to better inform future programme design/implementation.

METHODS

Our preliminary REE unpacked how short (half-day) and extended (12 week) supervision training programmes in Victoria, Australia, produced costs and outcomes. We employed mixed methods: qualitative realist and quantitative cost methods. Economically optimised programme models were developed guided by identified cost-sensitive mechanisms and contexts.

RESULTS

As part of identified context-mechanism-outcome configurations (CMOCs) for both training programmes, we found a wider diversity of positive outcomes but greater costs for the extended programme (11 outcomes; AU$3069/learner) compared with the short programme (7 outcomes; $385/learner). We identified four shared cost-sensitive mechanisms for both programmes (training duration, learner protected time, learner engagement, and facilitator competence) and one shared cost-sensitive context (learners' supervisory experience). We developed two economically optimised programme models: (1) short programme for experienced supervisors facilitated by senior educators ($406/learner); and (2) extended programme for inexperienced supervisors facilitated by junior educators ($2104/learner).

DISCUSSION

Our initial economic programme theory was somewhat supported, refuted and refined. Results were partly consistent with previous research, but also extended it through unpacking cost-sensitive mechanisms and contexts. Although our preliminary REE fills a pressing gap in the methodology literature, conducting REE was challenging given our desire to integrate economic and realist analyses fully, and remain faithful to realist principles. Attention to training duration and experience levels of the facilitator-learner dyad may help to balance the cost and outcomes of training programmes.

Digital Divide in Online Education During the COVID-19 Pandemic: A Cosmetic Course From the View of the Regional Socioeconomic Distribution

Objectives: During the pandemic, quarantine has led to the lockdown of many physical educational institutions. Thus, massive open online courses (MOOCs) have become a more common choice for participants. MOOCs are often flagged as supplemental methods to educational disparities caused by regional socioeconomic distribution. However, dissenters argue that MOOCs can exacerbate the digital divide. This study aimed to compare the participants' performance before and after the outbreak of COVID-19, analyze the impact of the epidemic on online education of cosmetic dermatology from the view of the regional socioeconomic distribution, and investigate whether MOOCs exacerbate the digital divide in the COVID-19 epidemic.

Methods: The study was conducted in participants of the MOOC course Appreciation and Analysis of Cosmetics from January 2018 to December 2020. Based on the platform data and official socioeconomic statistics, correlation of multivariate analysis was used to determine the factors related to the number of total participants. A panel regression model and stepwise least squares regression analysis (STEPLS) were employed to further analyze the relationship between GDP, population, number of college students and number of total participants in different years in the eastern, central and western regions of China.

Results: The number of total participants in 2020 surged 82.02% compared with that in 2019. Completion rates were generally stable in 2018 and 2019 before the COVID-19 pandemic and significantly decreased in 2020 after the outbreak of the pandemic. GDP was the most important socioeconomic factor that determined the total number of participants and it was positively related to the total number of participants before and after the outbreak of the pandemic. The number of college students was unrelated to the total number of participants before the epidemic, and after the outbreak of COVID-19 in 2020, the number became positively related in all regions of China.

Conclusions: This study shows that the epidemic pushes more people to choose MOOCs to study cosmetic dermatology, and online education could exacerbate rather than reduce disparities that are related to regional and socioeconomic status in the cosmetic field in the COVID-19 pandemic.

An investigation of the predictive validity of selection tools on performance in physiotherapy training in Australia

OBJECTIVES

Despite a large body of research on selection in medical education, very little is conducted in other health professions. This study investigated the predictive validity of multiple selection tools on academic and clinical performance outcomes of undergraduate physiotherapy students.

DESIGN

A retrospective observational study.

SETTING

Undergraduate physiotherapy program in Australia.

PARTICIPANTS

497 undergraduate physiotherapy students across seven entry cohorts. Including students directly from secondary school (n=381) and with prior tertiary study (n=116).

MAIN OUTCOME MEASURES

Academic performance as measured by written examinations. Clinical performance, measured by Objective Structured Clinical Examinations (OSCEs) during on-campus units and the Assessment of Physiotherapy Practice (APP) for off-campus clinical placements. Predictor variables included selection tools (academic achievement, interview, aptitude test) and demographic variables (age, gender).

RESULTS

Selection interview was a positive predictor of OSCEs and final year clinical performance in direct school leaver participants. Academic achievement scores from selection positively predicted written examinations scores.

CONCLUSION

Clinical and academic performance were predicted by tools measuring different domains at selection. Assessing broadly across academic and non-academic domains at selection can be valuable in identifying applicants who will be able to meet the range of outcomes for course completion and subsequent registration in the physiotherapy profession.

The impact of Multiple Mini Interviews on the attrition and academic outcomes of midwifery students

BACKGROUND

Admission to the Bachelor of Midwifery (BMid) in Australia has traditionally been based on academic ranking. The BMid is a high demand course offered to a limited number of students and therefore choosing applicants who complete the degree is important. Multiple Mini Interviews (MMIs) are used to assess non-cognitive skills and select students into healthcare degrees. One university in Australia has introduced MMIs as part of the application process for the BMid.

AIM

Compare attrition rates and Grade Point Average (GPA) scores between students admitted into the BMid using both academic ranking and MMIs, to those admitted on academic ranking alone.

METHODS

A basic convergent mixed methods design, using an explanatory unidirectional framework to integrate data. Attrition rates, GPA, and multiple mini interview scores (2013-2019), were linked and compared for before and after the use of MMI's. Focus groups with students, interviewers, and hospital-based educators, explored stakeholder experiences. Open-ended questions from an applicant survey were added to the qualitative data set, which was analysed thematically.

FINDINGS

Students who enrolled via the MMI's had significantly lower attrition rates than those enrolled before MMI's were introduced. GPA scores were significantly higher for students who enrolled via the MMI's. Integration of data found MMI's identified students passionate to undertake midwifery, and that success at the interviews increased students' confidence to successfully complete their studies.

CONCLUSION

MMI's as part of the entry process into the BMid enabled identification of applicants more likely to remain in the course and succeed in their studies.

Spending Wisely: The Role of Cost and Value Research in the Pursuit of Advancing Anatomical Sciences Education

Studies of "cost and value" in anatomical sciences education examine not only what works, but at what cost, thus evaluating the inputs and outputs of education. This research provides insights into how to use available resources (e.g., academic time, budgets, infrastructure) as a mechanism to obtaining the maximum outcomes available. The purpose of this viewpoint article is to expand on the application of cost and value concepts to anatomical sciences education, contextualizing these concepts through a deeper dive into the more costly educational approaches of human donor dissection. In doing so, both questions and opportunities are raised for the discipline of anatomical sciences going forward. Educational decisions, inclusive of cost and value appraisals, consider the range of outcomes for which the activity is designed to achieve, and the activity's integration with the philosophy of the educational program it is contributing to; these decisions, thus, evaluate more than just cost alone. Healthcare students' engagement with human donor dissection pedagogy offers an array of reported non-economic benefits, including non-traditional discipline-independent skill (NDIS) development (e.g., professionalism, teamwork skills). These skills are often harder to measure, but are no less important to the final pedagogical decision-making process. The goal of cost and value research is to create an evidence-base toward education that delivers maximum value for a given spend. Anatomy educators, researchers, and decision makers who embrace cost and value dialogue, and interpret and apply findings from studies of educational costs, are best positioned to improve the educational value for their learners and provide effective outputs for all stakeholders.

"You can't always get what you want…": economic thinking, constrained optimization and health professions education

Every choice we make in health professions education has a cost, whether it be financial or otherwise; by choosing one action (e.g., integrating more simulation, studying more for a summative examination) we lose the opportunity to take an alternative action (e.g., freeing up time for other teaching, leisure time). Economics significantly shapes the way we behave and think as educators and learners and so there is increasing interest in using economic ways of thinking and approaches to examine and understand how choices are made, the influence of constraints and boundaries in educational decision making, and how costs are felt. Thus, in this article, we provide a brief historical overview of modern economics, to illustrate how the core concepts of economics-scarcity (and desirability), rationality, and optimization-developed over time. We explain the important concept of bounded rationality, which explains how individual, meso-factors and contextual factors influence decision making. We then consider the opportunities that these concepts afford for health professions education and research. We conclude by proposing that embracing economic thinking opens up new questions and new ways of approaching old questions which can add knowledge about how choice is enacted in contemporary health professions education.