The Practical Effects of Measurement Invariance: Gender Invariance in Two Big Five Personality Measures

Measurement Invariance of the HEXACO-100 Across Gender Groups: A Three-Sample Study

We used exploratory structural equation modeling to examine gender-based measurement invariance (MI) in the HEXACO-100 across three samples that varied in terms of age (undergraduate students in Study 1, working adults in Studies 2 and 3) and testing context (research context in Studies 1 and 2, high-stakes selection context in Study 3). Across three studies, we consistently found support for configural and metric invariance but not scalar invariance. However, the effect size measures of non-invariance were generally small. That said, in the Emotionality scale, for the same latent score, females scored higher than males due to measurement non-invariance (between 0.26 and 0.48 standard deviation units). Thus, the observed mean gender differences overestimated the true mean gender differences. The current study provides detailed evidence regarding gender-based MI of HEXACO personality scales. More generally, it provides insight regarding the effect that measurement artifacts can have on understanding psychological gender differences at the latent level.

Review of the psychometric properties of the Patient Health Questionnaire-9 (PHQ-9) Spanish version in a sample of Puerto Rican workers

Background

This study aimed to examine the internal structure and assess the psychometric properties of the Patient Health Questionnaire (PHQ-9) in a Puerto Rican sample of workers. This instrument is a nine-item questionnaire, which was conceptualized as a unidimensional structure; however, there are mixed results regarding this internal structure. This measure is used in the occupational health psychology context in organizations in Puerto Rico; nevertheless, there is little evidence of its psychometric properties with samples of workers.

Materials and methods

A total of 955 samples from two different study samples were used in this cross-sectional study design in which the PHQ-9 was used. We conducted confirmatory factor analysis, bifactor analysis, and random intercept item factor analysis to examine the internal structure of the PHQ-9. Moreover, a two-factor model was examined by randomly assigning items to the two factors. Measurement invariance across sex and the relationship with other constructs were examined.

Results

The best-fitted model was the bifactor model followed by the random intercept item factor. The five sets of two-factor models with items randomly assigned obtained acceptable and similar fit indices regardless of the items.

Conclusion

The results suggest that the PHQ-9 appears to be a reliable and valid instrument to measure depression. The more parsimonious interpretation of its scores, for now, is a unidimensional structure. Comparison across sex appears to be useful in occupational health psychology research settings since the results suggest that the PHQ-9 is invariant regarding this variable.

Individual differences in adolescent and young adult daily mobility patterns and their relationships to big five personality traits: a behavioral genetic analysis

Youth behavior changes and their relationships to personality have generally been investigated using self-report studies, which are subject to reporting biases and confounding variables. Supplementing these with objective measures, like GPS location data, and twin-based research designs, which help control for confounding genetic and environmental influences, may allow for more rigorous, causally informative research on adolescent behavior patterns. To investigate this possibility, this study aimed to (1) investigate whether behavior changes during the transition from adolescence to emerging adulthood are evident in changing mobility patterns, (2) estimate the influence of adolescent personality on mobility patterns, and (3) estimate genetic and environmental influences on mobility, personality, and the relationship between them. Twins aged Fourteen to twenty-two (N=709, 55% female) provided a baseline personality measure, the Big Five Inventory, and multiple years of smartphone GPS data from June 2016 - December 2019. Mobility, as measured by daily locations visited and distance travelled, was found via mixed effects models to increase during adolescence before declining slightly in emerging adulthood. Mobility was positively associated with Extraversion and Conscientiousness (r of 0.17 - 0.25, r of 0.10 - 0.16) and negatively with Openness (r of -0.11 - -0.13). ACE models found large genetic (A = 0.56 - 0.81) and small-moderate environmental (C of 0.12 - 0.28, E of 0.07 - 0.15) influences on mobility. A and E influences were highly shared across mobility measures (rg = 0.70, re= 0.58). Associations between mobility and personality were partially explained by mutual genetic influences (rg of -0.27 - 0.53). Results show that as autonomy increases during adolescence and emerging adulthood, we see corresponding increases in youth mobility. Furthermore, the heritability of mobility patterns and their relationship to personality demonstrate that mobility patterns are informative, psychologically meaningful behaviors worthy of continued interest in psychology.

Structural validity and classification performance of the Italian Short Negative Acts Questionnaire: A Structural Equation Modeling approach for building ROC curves

We investigated the structural (internal) validity and classification performance of the Italian Short Negative Acts Questionnaire (SNAQ), a 9-item self-report instrument assessing bullying at work. Consistent with recent attention of researchers to control measurement error in predictive models (Jacobucci & Grimm, Perspectives on Psychological Science, 15(3), 809–816 2020), classification performance was investigated through a proposed novel procedure that uses Structural Equation Modeling for building ROC curves. Participants included 357 workers (females = 50.4%) from various sectors. Our results showed that (a) the Italian SNAQ demonstrates adequate levels of structural validity; (b) its classification performance (in terms of self-labeled bullying) is outstanding; and (c) the ROC curves estimated by means of Structural Equation Modeling outperform those estimated with classical observed-variable approaches. In conclusion, we provided further evidence regarding the good psychometric properties of the Italian SNAQ and we also offered a novel approach for estimating ROC curves that does not neglect the issue of measurement quality.

Investigating Measurement Invariance of the Psychological Entitlement Scale – Grandiose-Based and Vulnerable-Based

Abstract. Entitlement has attracted interest across various social science disciplines due to its broad connection to selfish decision-making outcomes and mental health. Although unidimensional entitlement scales have been widely used, these scales conflate vulnerable- and grandiose-based entitlement forms. The Psychological Entitlement Scale – Grandiose-Based and Vulnerable-Based (PES-G/V) was recently devised to measure these entitlement forms. Prior work has supported the structure and construct validity of the PES-G/V, but no research has addressed the measurement invariance (MI) of the PES-G/V. Hence, we examined MI in relation to gender, two popular sampling frames in psychology studies (US MTurk participants and US college participants), and age. Results supported scalar MI across levels of each of the grouping variables. In sum, the structural properties of the PES-G/V seemed robust to the group distinctions.

A Conceptual Framework for Investigating and Mitigating Machine-Learning Measurement Bias (MLMB) in Psychological Assessment

Given significant concerns about fairness and bias in the use of artificial intelligence (AI) and machine learning (ML) for psychological assessment, we provide a conceptual framework for investigating and mitigating machine-learning measurement bias (MLMB) from a psychometric perspective. MLMB is defined as differential functioning of the trained ML model between subgroups. MLMB manifests empirically when a trained ML model produces different predicted score levels for different subgroups (e.g., race, gender) despite them having the same ground-truth levels for the underlying construct of interest (e.g., personality) and/or when the model yields differential predictive accuracies across the subgroups. Because the development of ML models involves both data and algorithms, both biased data and algorithm-training bias are potential sources of MLMB. Data bias can occur in the form of nonequivalence between subgroups in the ground truth, platform-based construct, behavioral expression, and/or feature computing. Algorithm-training bias can occur when algorithms are developed with nonequivalence in the relation between extracted features and ground truth (i.e., algorithm features are differentially used, weighted, or transformed between subgroups). We explain how these potential sources of bias may manifest during ML model development and share initial ideas for mitigating them, including recognizing that new statistical and algorithmic procedures need to be developed. We also discuss how this framework clarifies MLMB but does not reduce the complexity of the issue.

Is use of the general system justification scale across countries justified? Testing its measurement equivalence

System justification is a widely researched topic in social and political psychology. One major measurement instrument in system justification research is the General System Justification Scale (G-SJS). This scale has been used, among others, for comparisons across social groups in different countries. Such comparisons rely on the assumption that the scale is measurement equivalent. However, this assumption has never been comprehensively tested. Thus, the present two studies assessed the measurement equivalence of the G-SJS following classic measurement equivalence guidelines (i.e., multigroup confirmatory factor analyses) in Study 1 and using a new method for comparing larger numbers of groups in Study 2 (i.e., alignment optimization). In Study 1, we analysed the measurement equivalence in Great Britain (n = 444), Germany (n = 454), and France (n = 463). In Study 2, we used a publicly available dataset consisting of 66 samples from 30 countries (N = 13,495) to again assess the measurement equivalence of the scale. Results indicated (partial) metric equivalence, but not scalar equivalence in both studies. Overall, the studies indicate that mean comparisons across the examined countries are not warranted with the current form of the G-SJS. The scale needs to be revised for valid cross-country comparisons of means.

Big Five personality trait differences between students from different majors aspiring to the teaching profession

Person-Environment fit theories claim that students choose their academic path according to their personality. In this regard, teacher candidates are of special interest. On the one hand, they all make the same choice to enroll in a teacher education program. On the other hand, they make different choices with respect to the subjects they are going to teach. If the Person-Environment fit approach also applies to the selection regarding teacher candidates’ subject areas, teacher candidates from different majors might have different personality traits and as a result, different starting conditions for becoming a successful teacher. Such differences need to be taken into account by teacher education in order to create programs that allow teacher candidates from different majors to equally succeed. Therefore, the current study investigates to what extent personality group differences across majors occur within the population of teacher candidates. Using data from a large-scale study, the Big Five personality traits of 1735 female and 565 male teacher candidates were analyzed, with teacher candidates compared to male (n = 1122) and female (n = 1570) students who studied the same major but who did not intend to become teachers. Unlike previous studies, academic majors were not grouped into few broad categories, but eight different majors were distinguished. The results indicate that teacher candidates are more extraverted than their non-teaching counterparts. In addition, personality trait differences between teacher candidates from different majors could be observed. The results are discussed as they relate to the recruitment and training of future teachers.

Registered Report: Testing Ideological Asymmetries in Measurement Invariance

People with different ideological identities differ in their values, personality, affect, and psychological motivations. These differences are observed on measures of practical and clinical importance and these differences are the central node tying together theories about the psychology of political ideology; however, they rest on a critical untested assumption: The measures are invariant across ideological groups. Here, we test this assumption across 28 constructs in data from the United States and the Netherlands. Measures are not invariant across ideological divisions. At the same time, estimates of ideological similarities and differences are largely similar before and after correcting for measurement noninvariance. This may give us increased confidence in the results from this research area, while simultaneously highlighting that some instance of noninvariance did change conclusions and that individual items are not always comparable across political groups.

Using the 16PF to Test the Differentiation of Personality by Intelligence Hypothesis

The differentiation of personality by intelligence hypothesis suggests that there will be greater individual differences in personality traits for those individuals who are more intelligent. Conversely, less intelligent individuals will be more similar to each other in their personality traits. The hypothesis was tested with a large sample of managerial job candidates who completed an omnibus personality measure with 16 scales and five intelligence measures (used to generate an intelligence g-factor). Based on the g-factor composite, the sample was split using the median to conduct factor analyses within each half. A five-factor model was tested for both the lower and higher intelligence halves and were found to have configural invariance but not metric or scalar invariance. In general, the results provide little support for the differentiation hypothesis as there was no clear and consistent pattern of lower inter-scale correlations for the more intelligent individuals.