Method Effects on an Adaptation of the Rosenberg Self-Esteem Scale in Greek and the Role of Personality Traits

Detecting non-content-based response styles in survey data: An application of mixture factor analysis

It is common for some participants in self-report surveys to be careless, inattentive, or lacking in effort. Data quality can be severely compromised by responses that are not based on item content (non-content-based [nCB] responses), leading to strong biases in the results of data analysis and misinterpretation of individual scores. In this study, we propose a specification of factor mixture analysis (FMA) to detect nCB responses. We investigated the usefulness and effectiveness of the FMA model in detecting nCB responses using both simulated data (Study 1) and real data (Study 2). In the first study, FMA showed reasonably robust sensitivity (.60 to .86) and excellent specificity (.96 to .99) on mixed-worded scales, suggesting that FMA had superior properties as a screening tool under different sample conditions. However, FMA performance was poor on scales composed of only positive items because of the difficulty in distinguishing acquiescent patterns from valid responses representing high levels of the trait. In Study 2 (real data), FMA detected a minority of cases (6.5%) with highly anomalous response patterns. Removing these cases resulted in a large increase in the fit of the unidimensional model and a substantial reduction in spurious multidimensionality.

Associations between childhood adversity, psychiatric symptoms, and self-esteem outcomes in adolescents and young adults: An experience sampling study

OBJECTIVES

Self-esteem and self-esteem stability are important factors during adolescence and young adulthood that can be negatively impacted by childhood adversity and psychiatric symptoms. We examined whether childhood adversity and psychiatric symptoms are associated with decreased global self-esteem as well as increased self-esteem instability as measured with experience sampling method. In addition, we examined if childhood adversity moderates the association between psychiatric symptoms and self-esteem outcomes.

METHODS

Our study consisted of 788 adolescents and young adults who were part of a twin pair. The twin structure was not of interest to the current study. Mean age was 16.8 (SD = 2.38, range: 14-25), 42% was male. We used a multilevel modeling approach to examine our hypotheses to account for the presence of twins in the data set.

RESULTS

Childhood adversity and psychiatric symptoms were negatively associated with global self-esteem (respectively standardized β = -.18, SE = 0.04, p < .0001 and standardized β = -.45, SE = 0.04, p < .0001), with a larger effect for psychiatric symptoms. Similarly, both were associated with increased self-esteem instability (respectively standardized β = .076, SE = 0.025, p = .002 and standardized β = .11, SE = 0.021, p < .0001). In addition, interactions between childhood adversity and psychiatric symptoms on both global self-esteem (standardized β = .06, SE = 0.01, p < .0001) and self-esteem instability (standardized β = -.002, SE = 0.0006, p = .001) were found, showing that the negative association of psychiatric symptoms with self-esteem outcomes is less pronounced in young people with higher levels of childhood adversity, or formulated differently, is more pronounced in young people with little or no exposure to childhood adversity.

CONCLUSION

Global self-esteem and self-esteem instability in young people are influenced by both current psychiatric symptomatology and exposure to childhood adversity. Those with more psychiatric symptoms show worse self-esteem and higher self-esteem instability, which is moderated by childhood adversity. For young people with high childhood adversity levels lower self-esteem and higher self-esteem instability are less influenced by reductions in psychiatric symptoms.

An Empirical Identification Issue of the Bifactor Item Response Theory Model

Using the bifactor item response theory model to analyze data arising from educational and psychological studies has gained popularity over the years. Unfortunately, using this model in practice comes with challenges. One such challenge is an empirical identification issue that is seldom discussed in the literature, and its impact on the estimates of the bifactor model's parameters has not been demonstrated. This issue occurs when an item's discriminations on the general and specific dimensions are approximately equal (i.e., the within-item discriminations are similar in strength), leading to difficulties in obtaining unique estimates for those discriminations. We conducted three simulation studies to demonstrate that within-item discriminations being similar in strength creates problems in estimation stability. The results suggest that a large sample could alleviate but not resolve the problems, at least when considering sample sizes up to 4,000. When the discriminations within items were made clearly different, the estimates of these discriminations were more consistent across the data replicates than that observed when the discriminations within the items were similar. The results also show that the similarity of an item's discriminatory magnitudes on different dimensions has direct implications on the sample size needed in order to consistently obtain accurate parameter estimates. Although our goal was to provide evidence of the empirical identification issue, the study further reveals that the extent of similarity of within-item discriminations, the magnitude of discriminations, and how well the items are targeted to the respondents also play factors in the estimation of the bifactor model's parameters.

Wording Effects in Assessment: Missing the Trees for the Forest

This article examines wording effects when positive and negative worded items are included in psychological assessment. Wordings effects have been analyzed in the literature using statistical approaches based on population homogeneity assumptions (i.e. CFA, SEM), commonly adopting the bifactor model to separate trait variance and wording effects. This article presents an alternative approach by explicitly modeling population heterogeneity through a latent profile model, based on the idea that a subset of individuals exhibits wording effects. This kind of mixture model allows simultaneously to classify respondents, substantively characterize the differences in their response profiles, and report respondents' results in a comparable manner. Using the Rosenberg's self-esteem scale data from the LISS Panel (N = 6,762) in three studies, we identify a subgroup of participants who respond differentially according to item-wording and examine the impact of its responses in the estimation of the RSES measurement model, in terms of global and individual fit, under one-factor and bifactor models.The results of these analyses support the interpretation of wording effects in terms of a theoretically-proposed differential pattern of response to positively and negatively worded items, introducing a valuable tool for examining the artifactual or substantive interpretations of such wording effects.

Response tendencies due to item wording using eye-tracking methodology accounting for individual differences and item characteristics

The discrepancy of the scores on responses to negatively and positively worded items has led to hypotheses of inattention, confusion, difficulty, and differential processing of negatively worded items. The present study, utilizing eye-tracking methodology, aimed to fill an explanatory gap regarding response behavior, providing observations on the item-level response process. It experimentally examined characteristics of the items (wording type, self-relevance) and characteristics of the respondents (neuroticism, verbal abilities, and mood) for their impact on response outcomes. A sample of 87 university students completed a computerized version of a questionnaire with items presented in four alternative wording types: positive, negative, negated positive, and negated negative; half of the items referred to attitudes toward the self and the other half to attitudes toward others. Participants' eye movements during item completion were recorded with the Gazepoint-GP3-HD desk-mounted eye tracker. In linear mixed effects models, wording type and self-relevance were found to relate to response time, time of viewing and revisits to the body of the items and the response options, indicating that there were effects at the stages of comprehension and selection of response. Neuroticism was associated with differential item responses, suggesting a role in later levels of the response process, the retrieval, judgment, and response selection stages. Eye-tracking measures can enhance the examination of response tendencies with regards to item content, item wording, and person characteristics.

Using Constrained Factor Mixture Analysis to Validate Mixed-Worded Psychological Scales: The Case of the Rosenberg Self-Esteem Scale in the Dominican Republic

A common method to collect information in the behavioral and health sciences is the self-report. However, the validity of self-reports is frequently threatened by response biases, particularly those associated with inconsistent responses to positively and negatively worded items of the same dimension, known as wording effects. Modeling strategies based on confirmatory factor analysis have traditionally been used to account for this response bias, but they have recently become under scrutiny due to their incorrect assumption of population homogeneity, inability to recover uncontaminated person scores or preserve structural validities, and their inherent ambiguity. Recently, two constrained factor mixture analysis (FMA) models have been proposed by Arias et al. (2020) and Steinmann et al. (2021) that can be used to identify and screen inconsistent response profiles. While these methods have shown promise, tests of their performance have been limited and they have not been directly compared. Thus the objective of the current study was to assess and compare their performance with data from the Dominican Republic of the Rosenberg Self-Esteem Scale (N = 632). Additionally, as this scale had not yet been studied for this population, another objective was to show how using constrained FMAs could help in the validation of mixed-worded scales. The results indicated that removing the inconsistent respondents identified by both FMAs (≈8%) reduced the amount of wording effects in the database. However, whereas the Steinmann et al. method only cleaned the data partially, the Arias et al. (2020) method was able to remove the great majority of the wording effects variance. Based on the screened data with the Arias et al. method, we evaluated the psychometric properties of the RSES for the Dominican population, and the results indicated that the scores had good validity and reliability properties. Given these findings, we recommend that researchers incorporate constrained FMAs into their toolbox and consider using them to screen out inconsistent respondents to mixed-worded scales.

From Bi-Dimensionality to Uni-Dimensionality in Self-Report Questionnaires

Abstract. The common factor model – by far the most widely used model for factor analysis – assumes equal item intercepts across respondents. Due to idiosyncratic ways of understanding and answering items of a questionnaire, this assumption is often violated, leading to an underestimation of model fit. Maydeu-Olivares and Coffman (2006) suggested the introduction of a random intercept into the model to address this concern. The present study applies this method to six established instruments (measuring depression, procrastination, optimism, self-esteem, core self-evaluations, and self-regulation) with ambiguous factor structures, using data from representative general population samples. In testing and comparing three alternative factor models (one-factor model, two-factor model, and one-factor model with a random intercept) and analyzing differential correlational patterns with an external criterion, we empirically demonstrate the random intercept model’s merit, and clarify the factor structure for the above-mentioned questionnaires. In sum, we recommend the random intercept model for cases in which acquiescence is suspected to affect response behavior.

Factorial Structure and Nomological Network of the Inventory of Callous-Unemotional Traits Accounting for Item Keying Variance

The factorial structure of the Inventory of Callous Unemotional Traits (ICU) is currently under dispute. The present study aims to test the factorial structure of a Greek adaptation of the ICU by considering item keying variance and examining alternative theoretical and empirically derived models. Additionally, it aims to investigate the nomological network of the ICU subscales, after controlling for item keying variance. The sample consisted of 1536 Greek-Cypriot adolescents, who completed a battery of questionnaires, including the ICU. Results showed that the consideration of item keying variance improved the overall fit of all the examined models and led to significant changes in the predictive validity of the subscales, while method factors presented distinct patterns of associations with external variables. Overall, results suggest that ICU is contaminated by item keying variance, which can be filtered out to provide clinically useful insight into the factorial structure of the ICU.

Adapting and validating the Rosenberg Self-Esteem Scale for elderly Spanish population

OBJECTIVES

This study aims to adapt and validate the most common measure of self-esteem, the Rosenberg Self-Esteem Scale (RSES), in the elderly Spanish population based on the initial one-factor model proposed by the author of the scale.

DESIGN

The factorial validity of the scale was tested using confirmatory factor analysis.

SETTING

The study was carried out in the city of Valencia (Spain).

PARTICIPANTS

A total of 231 elderly people with a mean age 72.68 (SD=8.55).

MEASUREMENTS

The participants completed the questionnaire RSES for the validation process, sociodemographic data and Mini-Mental State Examination.

RESULTS

Confirmatory factor analysis with a five-item structure for the one-factor structure showed good fit indexes (Chi square [5] = 217.20, p < .05; CFI = .965; GFI = .980; RMSEA = .070 [90% confidence interval of RMSEA, .022-.087]), and reliability, as internal consistency, measure with Cronbach's alpha was .732.

CONCLUSION

The adaptation of the RSES showed a unifactorial structure with good internal consistency. This reduced adaptation/version of the scale may facilitate clinical practice and be useful in research in older people.

Investigating Response Heterogeneity in the Context of Positively and Negatively Worded Items by Using Factor Mixture Modeling

Factor mixture modeling was used to investigate potential response incongruity between positively and negatively worded items. Survey respondents ( N = 591) answered questions about job satisfaction and dissatisfaction. Results revealed two classes of respondent: a majority class, who generally do not have problems answering positively and negatively worded items, and a minority class, who have serious trouble with negatively worded items. With the exclusion of the minority class, job satisfaction and dissatisfaction were found to be essentially unidimensional rather than bidimensional as previous research had suggested. These results challenge previous findings regarding the bidimensionality of job satisfaction and cast doubt on the widespread research practice of assuming population homogeneity in survey responses. A detailed flowchart illustrating the analytic procedure and an Mplus syntax program are provided so that researchers can conduct similar investigations on constructs of interest.

The Structure of the Rosenberg Self-Esteem Scale

Abstract. The Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965 ) intends to measure a single dominant factor representing global self-esteem. However, several studies have identified some form of multidimensionality for the RSES. Therefore, we examined the factor structure of the RSES with a fixed-effects meta-analytic structural equation modeling approach including 113 independent samples (N = 140,671). A confirmatory bifactor model with specific factors for positively and negatively worded items and a general self-esteem factor fitted best. However, the general factor captured most of the explained common variance in the RSES, whereas the specific factors accounted for less than 15%. The general factor loadings were invariant across samples from the United States and other highly individualistic countries, but lower for less individualistic countries. Thus, although the RSES essentially represents a unidimensional scale, cross-cultural comparisons might not be justified because the cultural background of the respondents affects the interpretation of the items.

Cognitive Abilities Explain Wording Effects in the Rosenberg Self-Esteem Scale

There is consensus that the 10 items of the Rosenberg Self-Esteem Scale (RSES) reflect wording effects resulting from positively and negatively keyed items. The present study examined the effects of cognitive abilities on the factor structure of the RSES with a novel, nonparametric latent variable technique called local structural equation models. In a nationally representative German large-scale assessment including 12,437 students competing measurement models for the RSES were compared: a bifactor model with a common factor and a specific factor for all negatively worded items had an optimal fit. Local structural equation models showed that the unidimensionality of the scale increased with higher levels of reading competence and reasoning, while the proportion of variance attributed to the negatively keyed items declined. Wording effects on the factor structure of the RSES seem to represent a response style artifact associated with cognitive abilities.

Translation, Adaptation, and Validation of the Need for Cognition Scale–Short Form in the Greek Language for Secondary School Students

The purpose of the present study was to adapt and validate the Need for Cognition Scale–Short Form (NfC-SF) in the Greek language. A multistep process was followed, including (a) the translation and adaptation of the questionnaire, (b) a reliability analysis of the instrument’s items in combination with an exploratory factor analysis with 177 secondary school students, and (c) a confirmatory factor analysis for defining the underlying structure of the scale, using a sample of 532 secondary school students. The statistical analyses validated a 14-item version of the NfC-SF for measuring the cognitive motivation of secondary school, Greek-speaking students. The present study also extends previous research about the underlying structure of the NfC by suggesting that method effects should be considered in measurement models for improving scale validity.

Is the Bifactor Model a Better Model or Is It Just Better at Modeling Implausible Responses? Application of Iteratively Reweighted Least Squares to the Rosenberg Self-Esteem Scale

Although the structure of the Rosenberg Self-Esteem Scale (RSES) has been exhaustively evaluated, questions regarding dimensionality and direction of wording effects continue to be debated. To shed new light on these issues, we ask (a) for what percentage of individuals is a unidimensional model adequate, (b) what additional percentage of individuals can be modeled with multidimensional specifications, and (c) what percentage of individuals respond so inconsistently that they cannot be well modeled? To estimate these percentages, we applied iteratively reweighted least squares (IRLS) to examine the structure of the RSES in a large, publicly available data set. A distance measure, d_s, reflecting a distance between a response pattern and an estimated model, was used for case weighting. We found that a bifactor model provided the best overall model fit, with one general factor and two wording-related group factors. However, on the basis of d_r values, a distance measure based on individual residuals, we concluded that approximately 86% of cases were adequately modeled through a unidimensional structure, and only an additional 3% required a bifactor model. Roughly 11% of cases were judged as "unmodelable" due to their significant residuals in all models considered. Finally, analysis of d_s revealed that some, but not all, of the superior fit of the bifactor model is owed to that model's ability to better accommodate implausible and possibly invalid response patterns, and not necessarily because it better accounts for the effects of direction of wording.