Predicting undergraduate OSCE performance using traditional and construct-driven situational judgment tests at admission

Situational Judgement Tests (SJTs) are popular to screen for social skills during undergraduate medical admission as they have been shown to predict relevant study outcomes. Two different types of SJTs can be distinguished: Traditional SJTs, which measure general effective behavior, and construct-driven SJTs which are designed to measure specific constructs. To date, there has been no comparison of the predictive validity of these two types of SJTs in medical admission. With the present research, we examine whether the HAM-SJT, a traditional SJT, and the CD-SJT, a construct-driven SJT with an agentic and a communal scale, administered during undergraduate medical admission can predict OSCE (i.e., objective structured clinical examination) performance in a low-stakes (nLS = 159) and a high-stakes (nHS = 160) sample of medical students. Results showed a moderate positive relation between the communal scale of the CD-SJT and performance in OSCE stations with trained patients in the high-stakes sample (r =.20, p =.009). This SJT had also an incremental value in predicting the OSCE performance above and beyond GPA (i.e., grade point average), a science test (i.e., HAM-Nat), and gender (ß = 0.18, 95% CI [0.03; 0.33], p =.020). That is, individuals who chose more communal behavioral responses in the SJT were rated more favorably in interactions with trained patients in the OSCE. A comparable correlation coefficient was observed for the HAM-SJT when controlling for range restriction due to admission (rraw = 0.14 vs. rcontrolled = 0.20). Our research provides a first indication for the predictive validity of construct-driven SJTs in high-stakes undergraduate medical admission.

Development of a scale to measure selection, optimization, compensation (SOC) strategy in late middle-aged women: a methodological study

PURPOSE

Selection-optimization-compensation (SOC) models have been proposed and applied to various populations to examine successful aging from a multidimensional perspective. This study aimed to develop a scale to measure SOC strategy among late middle-aged women (aged 50 to 64 years) and to test its validity and reliability.

METHODS

Preliminary items were developed through a literature review and interviews. Overall, 32 preliminary items were confirmed via two rounds of expert content validity analysis and a pilot survey. Data were collected from 299 late middle-aged women and analyzed using IBM SPSS/PC+ version 27.0. Construct validity, criterion validity, and reliability tests were conducted.

RESULTS

The SOC strategy scale, reflecting the characteristics of late middle-aged women and developed through exploratory factor analysis, comprised 19 items across four factors: goal-oriented selection, compensation for loss, outcome optimization, and ability-based optimization. The scale explained 66.9% of the variance in total factors, with a Cronbach's α of .95. Statistically significant correlations with the reference scale (r=.30, p<.001) were observed.

CONCLUSION

The developed scale demonstrated high validity and reliability, thus representing a viable instrument for measuring SOC strategy among late middle-aged women. Using this scale to assess the use of SOC approaches in these women can improve our understanding of the aging process and help establish supportive programs for their aging journeys.

Challenges in Conducting Empirical Epidemiological Research with Truck and Bus Drivers in Diverse Settings in North America

Over 6.5 million commercial vehicle drivers were operating a large truck or bus in the United States in 2020. This career often has high stress and long working hours, with few opportunities for physical activity. Previous research has linked these factors to adverse health conditions. Adverse health conditions affect not only the professional drivers' wellbeing but potentially also commercial motor vehicle (CMV) operators' safe driving ability and public safety for others sharing the roadway. The prevalence of health conditions with high impact on roadway safety in North American CMV drivers necessitates empirical epidemiological research to better understand and improve driver health. The paper presents four challenges in conducting epidemiological research with truck and bus drivers in North America and potential resolutions identified in past and current research. These challenges include (1) the correlation between driving performance, driving experience, and driver demographic factors; (2) the impact of medical treatment status on the relationship between health conditions and driver risk; (3) capturing accurate data in self-report data collection methods; and (4) reaching the CMV population for research. These challenges are common and influential in epidemiological research of this population, as drivers face severe health issues, health-related federal regulations, and the impact of vehicle operation on the safety of themselves and others using the roadways.

Assessing distinguishable social skills in medical admission: does construct-driven development solve validity issues of situational judgment tests?

BACKGROUND

Social skills are important for future physicians and are therefore increasingly considered in selection processes. One economic assessment method from which different social skills can be inferred are Situational Judgment Tests (SJTs) in which applicants are asked to rate behavioral responses in context-relevant situations. However, traditional SJTs have so far failed to distinctively measure specified constructs. To address this shortcoming in the medical admission context, we applied a construct-driven approach of SJT development in which test development was deductively guided by agency and communion as target constructs.

METHOD

The final version of the construct-driven SJT includes 15 items per construct with three behavioral responses. Medical school applicants (N = 1527) completed the construct-driven SJT, a traditional SJT, and an aptitude test under high-stakes condition as part of their application. A subsample (N = 575) participated in a subsequent voluntary online study with self-report measures of personality and past behavior.

RESULTS

The proposed two-factor structure and internal consistency of the construct-driven SJT was confirmed. Communal SJT scores were positively associated with self-reported communal personality and communal behavior, yet effects were smaller than expected. Findings for agentic SJT scores were mixed with positive small associations to self-reported agentic personality scores and agentic behavior but unexpected negative relations to communal self-reported measures.

CONCLUSIONS

Results suggest that construct-driven SJTs might overcome validity limitations of traditional SJTs, although their implementation is challenging. Despite first indicators of validity, future research needs to address practical points of application in high-stakes settings, inclusion of other constructs, and especially prediction of actual behavior before the application of construct-driven SJTs for selection purposes in medical admission can be recommended.

The Lazy or Dishonest Respondent: Detection and Prevention

Self-report measures are characterized as being susceptible to threats associated with deliberate dissimulation or response distortion (i.e., social desirability responding) and careless responding. Careless responding typically arises in low-stakes settings (e.g., participating in a study for course credit) where some respondents are not motivated to respond in a conscientious manner to the items. In contrast, in high-stakes assessments (e.g., prehire assessments), because of the outcomes associated with their responses, respondents are motivated to present themselves in as favorable a light as possible and, thus, may respond dishonestly in an effort to accomplish this objective. In this article, we draw a distinction between the lazy respondent, which we associate with careless responding, and the dishonest respondent, which we associate with response distortion. We then seek to answer the following questions for both careless responding and response distortion: ( a) What is it? ( b) Why is it a problem or concern? ( c) Why do people engage in it? ( d) How pervasive is it? ( e) Can and how is it prevented or mitigated? ( f) How is it detected? ( g) What does one do when one detects it? We conclude with a discussion of suggested future research directions and some practical guidelines for practitioners and researchers.