The predictive validity of Multiple Mini Interviews (MMIs) in nursing and midwifery programmes: Year three findings from a cross-discipline cohort study

Interviewer bias when using multiple mini-interviews in selecting student nurses in a Chinese setting

BACKGROUND

Interviewer effects may cause unfairness in assessments in multiple mini-interviews (MMIs). Due to cultural differences, the bias factors of interviewers may vary between the East and the West. MMIs are a relatively new type of assessment setting in China and few studies have been conducted to examine the interviewer effects of MMIs in this context.

OBJECTIVES

We adopted a multi-faceted Rasch measurement (MFRM) to measure interviewer effects in assessments in Hong Kong.

METHODS

Data were collected from a nursing school in Hong Kong. There were 431 candidates and 12 interviewers engaged in a six-station MMI setting. The scores collected from the interviews were analyzed in terms of 1) interviewer stringency/leniency, 2) candidate gender, 3) interview time, and 4) rating category in the station. The Student's t-statistic values were calculated to investigate the marking tendencies of individual interviewers.

RESULTS

The research findings suggest that interviewers differ in their degree of stringency/leniency, but the number of candidates examined by each interviewer does not affect interviewer stringency/leniency in terms of the interviewer's assessment. There is not sufficient evidence indicating that candidate gender and interview time are bias factors affecting assessment score in this study. Among the six rating categories examined, honesty/integrity is the most stringent category, while self-awareness is the most lenient category. Interview bias from individuals was identified. When we consider the interview scores given by individual interviewers, it is evident that some interviewers may have been biased toward a certain gender or rating categories.

CONCLUSIONS

MMIs are useful when selecting student nurses in a Chinese setting. However, interviewer bias may exist. We used an MFRM to better understand interviewer bias across various dimensions. The present study contributes to the development and use of MMIs in non-Western countries and can be used as a reference to extend this research to other locations.

Reliability and validity of new online selection tests for midwifery students

BACKGROUND AND OBJECTIVE

Newly created midwifery bachelor programmes attract an increasing number of applicants. Hence, educators have moved away from traditional non-standardised selection methods and started developing new tools for the admission of students. Researchers have expressed concerns around the validity of such selection measures and their ability to identify the appropriate candidates. The objective of this study was to develop and implement selection procedures for a new midwifery Bachelor of Science degree in Hamburg, Germany, and assess their psychometric properties.

DESIGN

This is a mixed-methods longitudinal study on the reliability, validity and acceptability of newly developed selection criteria.

SETTING

The newly established midwifery bachelor of science in Hamburg, Germany.

PARTICIPANTS

Upon completion of their online application, all midwifery applicants were invited by email to participate in our research project and thereby informed of its purpose and voluntary nature of their participation. The total number of candidates who took the test was 366, of which 309 agreed to take part in this study. Sixty-five applicants were eventually admitted to the midwifery Bachelor programme, of which 59 were included in the study.

MEASUREMENTS AND FINDINGS

We developed two tests for cognitive ability (HAM-Mid I and II) and adapted one test for personal and professional characteristics (Casper - Computer-based Assessment for Sampling Personal Characteristics). Due to the Covid pandemic, we implemented the tests online instead of on site as initially planned. HAM-Mid I had the lowest (Cronbach's Alpha = 0.38) whereas Casper had the highest internal consistency (Cronbach's Alpha = 0.77) of all three tests. Age, previous academic achievement and native tongue were significantly associated with applicants' performance on HAM-Mid II and Casper admission tests. HAM-Mid II was associated with students' performance in the first year midwifery exam (r = 0.31, p < 0.05). Overall evaluation of HAM-Mid II and Casper was good while HAM-Mid I received a more negative feedback.

KEY CONCLUSIONS

Evidence on the reliability and predictive validity of the newly developed tests suggests that the applied admission criteria are appropriate for an objective selection of bachelor midwifery students.

IMPLICATIONS FOR PRACTICE

The interests of university and clinical stakeholders of a midwifery bachelor degree can be represented through the development of tests for the assessment of cognitive ability and personal and professional characteristics.

The impact of Multiple Mini Interviews on the attrition and academic outcomes of midwifery students

BACKGROUND

Admission to the Bachelor of Midwifery (BMid) in Australia has traditionally been based on academic ranking. The BMid is a high demand course offered to a limited number of students and therefore choosing applicants who complete the degree is important. Multiple Mini Interviews (MMIs) are used to assess non-cognitive skills and select students into healthcare degrees. One university in Australia has introduced MMIs as part of the application process for the BMid.

AIM

Compare attrition rates and Grade Point Average (GPA) scores between students admitted into the BMid using both academic ranking and MMIs, to those admitted on academic ranking alone.

METHODS

A basic convergent mixed methods design, using an explanatory unidirectional framework to integrate data. Attrition rates, GPA, and multiple mini interview scores (2013-2019), were linked and compared for before and after the use of MMI's. Focus groups with students, interviewers, and hospital-based educators, explored stakeholder experiences. Open-ended questions from an applicant survey were added to the qualitative data set, which was analysed thematically.

FINDINGS

Students who enrolled via the MMI's had significantly lower attrition rates than those enrolled before MMI's were introduced. GPA scores were significantly higher for students who enrolled via the MMI's. Integration of data found MMI's identified students passionate to undertake midwifery, and that success at the interviews increased students' confidence to successfully complete their studies.

CONCLUSION

MMI's as part of the entry process into the BMid enabled identification of applicants more likely to remain in the course and succeed in their studies.

Personal qualities needed by undergraduate nursing students for a successful work integrated learning (WIL) experience

BACKGROUND

International research has shown that during work integrated learning (WIL) undergraduate nursing students may experience stress, self-doubt, and coping issues, however personal qualities these students bring to their WIL journey or those needed to succeed during WIL remain unknown.

OBJECTIVE

To describe the views of students midway through their Bachelor of Nursing program regarding personal qualities they bring to and need for WIL success.

DESIGN

This article reports on a descriptive study that used an electronic questionnaire to collect the data.

SETTINGS AND PARTICIPANTS

Bachelor of nursing students (n = 421), who were studying fulltime or part time equivalent, at a regional university in Queensland Australia, were invited to complete the anonymous electronic questionnaire that was developed by the research team following an extensive review of literature. Inclusion criteria required participants to have completed at least one WIL experience and be midway through their three-year Bachelor of Nursing program.

METHODS

Data were collected during January and February 2017. The electronic questionnaire consisted of demographic, forced choice questions, and open-ended questions. This paper presents the qualitative analysis of responses to two of the open-ended questions that focused on personal strengths and work integrated learning experiences. Data were analysed using the inductive content analysis approach.

RESULTS

Six personal qualities were identified including: enthusiasm to succeed, commitment to learning, self-motivation, confidence, effective communication skills, and compassion.

CONCLUSIONS

Pre-entry screening of applicants to bachelor level nursing programs and the delivery of learning opportunities that encourages both reflection and debriefing may assist in describing personal qualities needed to succeed in WIL.

An exploration of the predictive validity of selection criteria on progress outcomes for pre-registration nursing programmes-A systematic review

AIM

To identify the selection methods currently being used for pre-registration nursing programmes and to assess the predictive power that these methods have on students' success.

BACKGROUND

Research into selection methods in nursing education is beginning to emerge, yet it is unclear which methods are most predictive of students' success.

DESIGN

A systematic review of the literature.

METHODS

A systematic search of ten electronic databases: CINAHL, MEDLINE Ovid, EMBASE, PROQUEST Health and Medical, PROQUEST Education, COCHRANE Library, Web of Science, ASSIA, SCOPUS and PROSPERO was conducted. The results were expanded by the handsearching of journals, reference lists and grey literature. The PRISMA statement guided the review. Studies published in English between January 2008-March 2020 were eligible for inclusion, and quality assessment was undertaken using the CASP Checklist for Cohort Studies.

RESULTS

Twenty-five studies met the criteria for inclusion. A range of selection methods was identified including prior academic achievement, admissions tests, interviews, emotional intelligence tests, personal statements and previous healthcare experience. Prior academic achievement and admissions tests appear to be the selection methods most predictive of student success. The evidence surrounding other selection methods such as interviews and personal statements is less conclusive.

CONCLUSION

Selecting individuals with the appropriate knowledge, interpersonal skills and personal qualities needed to complete an undergraduate nursing programme is an important part of the role of nurse educators. This review shows that a wide variety of selection methods are used across different institutions, some of which are more effective than others in predicting student outcomes.

RELEVANCE TO CLINICAL PRACTICE

Further research is required to justify the continued use of some commonly used selection methods for undergraduate nursing programmes. Selection models that combine various types of selection criteria with predictive power appear to increase the probability of selecting students that will have successful outcomes.