1
|
Callwood A, Harris J, Gillam L, Roberts S, Kubacki A, Christidis A, Tiffin PA. Cross-sectional evaluation of an asynchronous multiple mini-interview (MMI) in selection to health professions training programmes with 10 principles for fairness built-in. BMJ Open 2023; 13:e074440. [PMID: 37907294 PMCID: PMC10618971 DOI: 10.1136/bmjopen-2023-074440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 09/18/2023] [Indexed: 11/02/2023] Open
Abstract
OBJECTIVES We aimed to explore the psychometric properties of the first known online asynchronous multiple mini-interview (MMI) designed for fairness with subgroup analyses by key characteristics, usability and acceptability. DESIGN Cross-discipline multimethod evaluation. SETTING One UK University. PARTICIPANTS Applicants to nursing, midwifery and paramedic science undergraduate programmes during 2021-2022. PRIMARY, SECONDARY OUTCOME MEASURES Psychometric properties (internal consistency, construct validity, dimensionality) were assessed using Cronbach's alpha (α), parallel analysis (PA), Schmid-Leiman transformation and ordinal confirmatory factor analysis (CFA). Usability and acceptability were evaluated using descriptive statistics and conventional content analysis. METHODS The system was configured in a seven question 4 min MMI. Applicants' videorecorded their answers which were later assessed by interviewers and scores summed. Applicants and interviewers completed online evaluation questionnaires. RESULTS Performance data from 712 applicants determined good-excellent reliability for the asynchronous MMI (mean α 0.72) with similar results across subgroups (gender, age, disability/support needs, UK/non-UK). PA and factor analysis results suggested there were seven factors relating to the MMI questions with an underlying general factor that explained the variance in observed candidate responses. A CFA testing a seven-factor hierarchical model showed an excellent fit to the data (Confirmatory Fit Index=0.99), Tucker Lewis Index=0.99, root mean square error (RMSE) =0.034). Applicants (n=210) viewed the flexibility, relaxed environment and cost savings advantageous. Interviewers (n=65) reported the system to be intuitive, flexible with >70% time saved compared with face-to-face interviews. Reduced personal communication was cited as the principal disadvantage. CONCLUSIONS We found that the asynchronous MMI was reliable, time-efficient, fair and acceptable and building fairness in was lost-cost. These novel, insights are applicable across health professions selection internationally informing the future configuration of online interviews to ensure workforces represent the societies they serve.
Collapse
Affiliation(s)
- Alison Callwood
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Jenny Harris
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Lee Gillam
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK
| | - Sarah Roberts
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | | | - Angelos Christidis
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK
| | | |
Collapse
|
2
|
Bangerter A, Mayor E, Muralidhar S, Kleinlogel EP, Gatica‐Perez D, Schmid Mast M. Automatic identification of storytelling responses to past‐behavior interview questions via machine learning. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2023. [DOI: 10.1111/ijsa.12428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
|
3
|
Köchling A, Wehner MC. Better explaining the benefits why AI? Analyzing the impact of explaining the benefits of AI‐supported selection on applicant responses. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2022. [DOI: 10.1111/ijsa.12412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
|
4
|
Roulin N, Wong O, Langer M, Bourdage JS. Is more always better? How preparation time and re-recording opportunities impact fairness, anxiety, impression management, and performance in asynchronous video interviews. EUROPEAN JOURNAL OF WORK AND ORGANIZATIONAL PSYCHOLOGY 2022. [DOI: 10.1080/1359432x.2022.2156862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Affiliation(s)
- Nicolas Roulin
- Department of Psychology, Saint Mary’s University, Halifax, Canada
| | - Odelia Wong
- Department of Psychology, Saint Mary’s University, Halifax, Canada
| | - Markus Langer
- Department of Psychology, University of Saarland, Saarbrücken, Germany
| | | |
Collapse
|
5
|
Basch JM, Melchers KG, Büttner JC. Preselection in the digital age: A comparison of perceptions of asynchronous video interviews with online tests and online application documents in a simulation context. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2022. [DOI: 10.1111/ijsa.12403] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Affiliation(s)
- Johannes M. Basch
- Abteilung Arbeits‐ und Organisationspsychologie Universität Ulm Ulm Germany
| | - Klaus G. Melchers
- Abteilung Arbeits‐ und Organisationspsychologie Universität Ulm Ulm Germany
| | - Julia C. Büttner
- Abteilung Arbeits‐ und Organisationspsychologie Universität Ulm Ulm Germany
| |
Collapse
|
6
|
Bill B, Melchers KG. Thou Shalt not Lie! Exploring and testing countermeasures against faking intentions and faking in selection interviews. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2022. [DOI: 10.1111/ijsa.12402] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Affiliation(s)
- Benedikt Bill
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| | - Klaus G. Melchers
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| |
Collapse
|
7
|
A Weighted Bonferroni-OWA Operator Based Cumulative Belief Degree Approach to Personnel Selection Based on Automated Video Interview Assessment Data. MATHEMATICS 2022. [DOI: 10.3390/math10091582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Asynchronous Video Interviewing (AVI) is considered one of the most recent and promising innovations in the recruitment process. Using AVI in combination with AI-based technologies enables recruiters/employers to automate many of the tasks that are typically required for screening, assessing, and selecting candidates. In fact, the automated assessment and selection process is a complex and uncertain problem involving highly subjective, multiple interrelated criteria. In order to address these issues, an effective and practical approach is proposed that is able to transform, weight, combine, and rank automated AVI assessments obtained through AI technologies and machine learning. The suggested approach combines Cumulative Belief Structures with the Weighted Bonferroni-OWA operator, which allows (i) aggregating assessment scores obtained in different forms and scales; (ii) incorporating interrelationships between criteria into the analysis (iii) considering accuracies of the learning algorithms as weights of criteria; and (iv) weighting criteria objectively. The proposed approach ensures a completely data-driven and efficient approach to the personnel selection process. To justify the effectiveness and applicability of the suggested approach, an example case is presented in which the new approach is compared to classical MCDM techniques.
Collapse
|
8
|
Gonzalez MF, Liu W, Shirase L, Tomczak DL, Lobbe CE, Justenhoven R, Martin NR. Allying with AI? Reactions toward human-based, AI/ML-based, and augmented hiring processes. COMPUTERS IN HUMAN BEHAVIOR 2022. [DOI: 10.1016/j.chb.2022.107179] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
|
9
|
Dunlop PD, Holtrop D, Wee S. How asynchronous video interviews are used in practice: A study of an Australian‐based AVI vendor. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2022. [DOI: 10.1111/ijsa.12372] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Affiliation(s)
- Patrick D. Dunlop
- Future of Work Institute, Faculty of Business and Law Curtin University Bentley Western Australia Australia
| | - Djurre Holtrop
- Department of Social Psychology, Tilburg School of Social and Behavioral Sciences Tilburg University Warandelaan The Netherlands
| | - Serena Wee
- School of Psychological Science The University of Western Australia Crawley Western Australia Australia
| |
Collapse
|
10
|
Köchling A, Wehner MC, Warkocz J. Can I show my skills? Affective responses to artificial intelligence in the recruitment process. REVIEW OF MANAGERIAL SCIENCE 2022. [DOI: 10.1007/s11846-021-00514-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
AbstractCompanies increasingly use artificial intelligence (AI) and algorithmic decision-making (ADM) for their recruitment and selection process for cost and efficiency reasons. However, there are concerns about the applicant’s affective response to AI systems in recruitment, and knowledge about the affective responses to the selection process is still limited, especially when AI supports different selection process stages (i.e., preselection, telephone interview, and video interview). Drawing on the affective response model, we propose that affective responses (i.e., opportunity to perform, emotional creepiness) mediate the relationships between an increasing AI-based selection process and organizational attractiveness. In particular, by using a scenario-based between-subject design with German employees (N = 160), we investigate whether and how AI-support during a complete recruitment process diminishes the opportunity to perform and increases emotional creepiness during the process. Moreover, we examine the influence of opportunity to perform and emotional creepiness on organizational attractiveness. We found that AI-support at later stages of the selection process (i.e., telephone and video interview) decreased the opportunity to perform and increased emotional creepiness. In turn, the opportunity to perform and emotional creepiness mediated the association of AI-support in telephone/video interviews on organizational attractiveness. However, we did not find negative affective responses to AI-support earlier stage of the selection process (i.e., during preselection). As we offer evidence for possible adverse reactions to the usage of AI in selection processes, this study provides important practical and theoretical implications.
Collapse
|
11
|
Ebadi S, Bashiri S. Psychological Benefits and Challenges of Ph.D. Entrance Exam Virtual Interviews During COVID-19 Pandemic: Does Gender Play a Role? Front Psychol 2021; 12:800715. [PMID: 34899548 PMCID: PMC8654781 DOI: 10.3389/fpsyg.2021.800715] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2021] [Accepted: 11/02/2021] [Indexed: 11/18/2022] Open
Abstract
This study aimed to investigate the reflections of Iranian students on Ph.D. entrance exam interviews held virtually nationwide during the Pandemic. Two hundred thirty-one Ph.D. applicants were invited to fill out an anonymous online survey designed in Google Forms, 36 out of whom volunteered to participate in follow-up semi-structured interviews. Two ANOVA measures were used to investigate the possible influences of gender and prior virtual interview experience on the applicants’ perceptions. Thematic analysis was also adopted to explore the participants’ attitudes and perceptions toward virtual interviews (VI). Quantitative findings showed that Iranian Ph.D. applicants perceived VI as a flexible and helpful procedure that provided them with satisfactory interpersonal treatment. Still, they did not favor the procedure’s perceived behavioral control and ability to communicate information to perform procedural justice. Moreover, neither their gender nor their prior experience of the virtual interview was a predictor of their perspectives’ discrepancies. The thematic analysis of the qualitative data revealed that despite having some cognitive, personal, and financial benefits, VI posed some technical, communicative, and personal challenges. This study provides implications for universities and applicants who will use VI for different purposes in higher education.
Collapse
Affiliation(s)
- Saman Ebadi
- Faculty of Humanities, Razi University, Kermanshah, Iran
| | - Saba Bashiri
- Faculty of Humanities, Razi University, Kermanshah, Iran
| |
Collapse
|
12
|
Repelled at first sight? Expectations and intentions of job-seekers reading about AI selection in job advertisements. COMPUTERS IN HUMAN BEHAVIOR 2021. [DOI: 10.1016/j.chb.2021.106931] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
13
|
Arseneault R, Roulin N. A theoretical model of cross‐cultural impression management in employment interviews. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2021. [DOI: 10.1111/ijsa.12348] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Affiliation(s)
- Rene Arseneault
- Department of Management University of Regina Regina Saskatchewan Canada
| | - Nicolas Roulin
- Department of Psychology Saint Mary's University Halifax Nova Scotia Canada
| |
Collapse
|
14
|
Basch JM, Brenner F, Melchers KG, Krumm S, Dräger L, Herzer H, Schuwerk E. A good thing takes time: The role of preparation time in asynchronous video interviews. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2021. [DOI: 10.1111/ijsa.12341] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Affiliation(s)
- Johannes M. Basch
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| | - Falko Brenner
- Abteilung Psychologische Diagnostik, Differentielle und Persönlichkeitspsychologie Freie Universität Berlin Berlin Germany
| | - Klaus G. Melchers
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| | - Stefan Krumm
- Abteilung Psychologische Diagnostik, Differentielle und Persönlichkeitspsychologie Freie Universität Berlin Berlin Germany
| | - Luise Dräger
- Lehrstuhl für Personalwirtschaft und Business Governance Martin‐Luther‐Universität Halle Wittenberg Halle Germany
| | - Helen Herzer
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| | - Evelyn Schuwerk
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| |
Collapse
|
15
|
“I” feel(s) left out: The importance of information and communication technology in personnel selection research. INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY-PERSPECTIVES ON SCIENCE AND PRACTICE 2021. [DOI: 10.1017/iop.2021.79] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
16
|
Basch JM, Melchers KG. The use of technology‐mediated interviews and their perception from the organization's point of view. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2021. [DOI: 10.1111/ijsa.12339] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Johannes M. Basch
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| | - Klaus G. Melchers
- Abteilung Arbeits‐ und Organisationspsychologie, Institut für Psychologie und Pädagogik Universität Ulm Ulm Germany
| |
Collapse
|
17
|
Constantin KL, Powell DM, McCarthy JM. Expanding conceptual understanding of interview anxiety and performance: Integrating cognitive, behavioral, and physiological features. INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT 2021. [DOI: 10.1111/ijsa.12326] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Affiliation(s)
| | | | - Julie M. McCarthy
- Department of Management University of Toronto Scarborough Toronto ON Canada
| |
Collapse
|
18
|
Melchers KG, Petrig A, Basch JM, Sauer J. A Comparison of Conventional and Technology-Mediated Selection Interviews With Regard to Interviewees' Performance, Perceptions, Strain, and Anxiety. Front Psychol 2021; 11:603632. [PMID: 33510679 PMCID: PMC7835329 DOI: 10.3389/fpsyg.2020.603632] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2020] [Accepted: 12/11/2020] [Indexed: 12/02/2022] Open
Abstract
Organizations increasingly use technology-mediated interviews. However, only limited research is available concerning the comparability of different interview media and most of the available studies stem from a time when technology-mediated interviews were less common than in the present time. In an experiment using simulated selection interviews, we compared traditional face-to-face (FTF) interviews with telephone and videoconference interviews to determine whether ratings of interviewees’ performance, their perceptions of the interview, or their strain and anxiety are affected by the type of interview. Before participating in the actual interview, participants had a more positive view of FTF interviews compared to technology-mediated interviews. However, fairness perceptions did not differ anymore after the interview. Furthermore, there were no differences between the three interview media concerning psychological and physiological indicators of strain or interview anxiety. Nevertheless, ratings of interviewees’ performance were lower in the technology-mediated interviews than in FTF interviews. Thus, differences between different interview media can still be found nowadays even though most applicants are much more familiar with technology-mediated communication than in the past. The results show that organizations should take this into account and therefore avoid using different interview media when they interview different applicants for the same job opening.
Collapse
Affiliation(s)
- Klaus G Melchers
- Institut für Psychologie und Pädagogik, Universität Ulm, Ulm, Germany
| | | | - Johannes M Basch
- Institut für Psychologie und Pädagogik, Universität Ulm, Ulm, Germany
| | - Juergen Sauer
- Department of Psychology, University of Fribourg, Fribourg, Switzerland
| |
Collapse
|