On the Structure of Affect

Abstract. The Positive and Negative Affect Schedule (PANAS; Watson et al., 1988 ) is a popular self-report questionnaire that is administered all over the world. Though originally developed to measure two independent factors, different models have been proposed in the literature. Comparisons among alternative models as well as analyses concerning their robustness in cross-national research have left an inconclusive picture. Therefore, the present study evaluates the dimensionality of the PANAS and differences between English and translated versions of the PANAS using a meta-analytic structural equation modeling approach. Correlation matrices from 57 independent samples ( N = 54,043) were pooled across subsamples. For both English and non-English samples, a correlated two-factor model including correlated uniquenesses provided the best fit. However, measurement invariance analyses indicated differences in factor loadings between subsamples. Thus, cross-national application of the PANAS might only be justified if measurement equivalence was explicitly tested for the countries at hand.

Limited evidence for the effect of red color on cognitive performance: A meta-analysis

Red color supposedly affects cognitive functioning in achievement situations and impairs test performance. Although this has been shown for different cognitive domains in different populations and cultural contexts, recent studies including close replications failed to corroborate this effect. Reported here is a random-effects meta-analysis of 67 effect sizes (38 samples) that compared test performance after viewing red or a control color. For anagram tests and knowledge tests no significant difference between color conditions was found (Cohen's d of -0.06 and -0.04); for reasoning tests the pooled effect of d = -0.34, 95% CI [-0.61, -0.06] indicated significantly lower scores in the red condition. The cumulative meta-analysis revealed substantially larger effects in initial studies as compared to subsequent research. After correcting for publication bias no evidential value for an effect of red color on intellectual performance was available. The review casts doubt on the existence of a robust color-priming effect in achievement situations.

Degrees of Freedom in Multigroup Confirmatory Factor Analyses

Abstract. Measurement invaraiance is a key concept in psychological assessment and a fundamental prerequisite for meaningful comparisons across groups. In the prevalent approach, multigroup confirmatory factor analysis (MGCFA), specific measurement parameters are constrained to equality across groups. The degrees of freedom ( df) for these models readily follow from the hypothesized measurement model and the invariance constraints. In light of research questioning the soundness of statistical reporting in psychology, we examined how often reported df match with the df recalcualted based on information given in the publications. More specifically, we reviewed 128 studies from six leading peer-reviewed journals focusing on psychological assessment and recalculated the df for 302 measurement invariance testing procedures. Overall, about a quarter of all articles included at least one discrepancy with metric and scalar invariance being more frequently affected. We discuss moderators of these discrepancies and identify typical pitfalls in measurement invariance testing. Moreover, we provide example syntax for different methods of scaling latent variables and introduce a tool that allows for the recalculation of df in common MGCFA models to improve the statistical soundness of invariance testing in psychological research.

A Meta-Analysis of Test Scores in Proctored and Unproctored Ability Assessments

Abstract. Unproctored, web-based assessments are frequently compromised by a lack of control over the participants’ test-taking behavior. It is likely that participants cheat if personal consequences are high. This meta-analysis summarizes findings on context effects in unproctored and proctored ability assessments and examines mean score differences and correlations between both assessment contexts. As potential moderators, we consider (a) the perceived consequences of the assessment, (b) countermeasures against cheating, (c) the susceptibility to cheating of the measure itself, and (d) the use of different test media. For standardized mean differences, a three-level random-effects meta-analysis based on 109 effect sizes from 49 studies (total N = 100,434) identified a pooled effect of Δ = 0.20, 95% CI [0.10, 0.31], indicating higher scores in unproctored assessments. Moderator analyses revealed significantly smaller effects for measures that are difficult to research on the Internet. These results demonstrate that unproctored ability assessments are biased by cheating. Unproctored assessments may be most suitable for tasks that are difficult to search on the Internet.

Processing the Word Red and Intellectual Performance: Four Replication Attempts

Colors convey meaning and can impair intellectual performance in achievement situations. Even the processing of color words can exert similar detrimental effects. In four experiments, we tried to replicate previous findings regarding the processing of the word “red” (as compared to a control color) on cognitive test scores. Experiments 1 and 2 (Ns = 69 and 104) are direct replications of Lichtenfeld, Maier, Elliot, and Pekrun (2009). Both experiments failed to uncover a red color effect on verbal reasoning scores among high school students and undergraduates (Cohen’s d = 0.04 and –0.23). Experiments 3 and 4 (N = 103 and 1,149) failed to identify an effect of processing red on general knowledge test scores (Cohen’sd = 0.19) and 0.01) among undergraduates and adults. Together, these results do not corroborate the assumption that processing the word red impairs intellectual performance.

Are Social Media Ruining Our Lives? A Review of Meta-Analytic Evidence

A growing number of studies have examined the psychological corollaries of using social networking sites (SNSs) such as Facebook, Instagram, or Twitter (often called social media). The interdisciplinary research area and conflicting evidence from primary studies complicate the assessment of current scholarly knowledge in this field of high public attention. We review meta-analytic evidence on three hotly debated topics regarding the effects of SNSs: well-being, academic achievement, and narcissism. Meta-analyses from different laboratories draw a rather equivocal picture. They show small associations in the r = .10 range between the intensity of SNS use and loneliness, self-esteem, life satisfaction, or self-reported depression, and somewhat stronger links to a thin body ideal and higher social capital. There is no indication for potential devastating effects of social media on school achievement; social media use and school grades are unrelated for adolescents. The meta-analyses revealed small to moderate associations between narcissism and SNS use. In sum, meta-analytic evidence is not in support of dramatic claims relating social media use to mischief.

Development of a Shortened Version of the Latent and Manifest Benefits of Work (LAMB) Scale

Abstract. Paid work offers latent benefits (e.g., social contact, time structure) that go beyond mere access to financial resources in predicting individuals’ psychological well-being. Despite the importance of the concept for organizational research and practice, available instruments measuring these latent benefits suffer either from psychometric deficiencies or from a scale length that makes integrating them into large-scale work/life surveys cumbersome. Thus, the current two studies ( N = 1,054 and N = 677) report on the development of the Short Latent and Manifest Benefits of Work scale (LAMB-S; cf. Muller, Creed, Waters, & Machin, 2005 ). The new 18-item instrument showed a clear factor structure, appropriate external validities, and even slight improvements in content and criterion validity for some subscales. Overall, the LAMB-S represents an economical instrument with satisfactory psychometric properties, making it an attractive alternative in situations where participant time is limited.

Blood lead levels and cognitive functioning: A meta-analysis

OBJECTIVE

The neurotoxicity of lead is well-known and even low exposure levels potentially impact neurocognitive abilities. This metaanalysis aimed to investigate associations between neurocognitive performance and lead exposure in adults and further assess potential effect thresholds.

MATERIAL AND METHODS

Articles indexed in Medline published until January 2017 reporting associations between lead exposure, blood lead, cognitive abilities and sensomotoric performance were included. The unbiased, standardized mean differences g between lead exposure and control groups extracted from the primary studies were pooled using a three-level, random-effects meta-analytic model with a restricted maximum likelihood estimator. Moderator analyses were conducted using weighted, mixedeffects regression analyses.

RESULTS

We identified 22 articles (n=3,849 participants, mean age 39.94±7.87years) published between 1976 and 2014 reporting lead exposure effects on cognitive and sensomotoric parameters (verbal abilities, visuospatial abilities, memory, attention, psychomotor function). On average, blood lead concentrations were 21.09±6.44 μg/dl higher in exposed than in control subjects. After exclusion of outliers, the random-effects three-level meta-analysis identified a significant (p<.001) pooled mean difference between exposure and control groups. Except for a smaller effect in the digit symbol test (p<.05), lead exposure did not result in different outcomes across the examined cognitive measures. Based on a marginally significant (p=.06) effect of difference in exposure levels, a blood lead increase of 10 μg/dl translated into a decline in cognitive abilities of Hedges g=.09.

CONCLUSIONS

Neurocognitive performance in adults with occupational or environmental lead exposure was significant impaired with regard to the examined parameters. However, further studies are needed for the determination of effect thresholds and reversibility.

Exploring the myth of the chubby gamer: A meta-analysis on sedentary video gaming and body mass

RATIONALE

High body mass and obesity are frequently linked to the use of sedentary media, like television (TV) or non-active video games. Empirical evidence regarding video gaming, however, has been mixed, and theoretical considerations explaining a relationship between general screen time and body mass may not generalize to non-active video gaming.

OBJECTIVE

The current meta-analysis had two main goals. First, we wanted to provide an estimate of the average effect size of the relationship between sedentary video gaming and body mass. In doing so we acknowledged several context variables to gauge the stability of the average effect. Second, to provide additional evidence on processes, we tested the displacement effect of physical activity by video gaming time with the help of a meta-analytic structural equation model (MASEM).

METHOD

Published and unpublished studies were identified through keyword searches in different databases and references in relevant reports were inspected for further studies. We present a random-effects, three-level meta-analysis based on 20 studies (total N = 38,097) with 32 effect sizes.

RESULTS

The analyses revealed a small positive relationship between non-active video game use and body mass, ρˆ=.09, 95% CI [0.03, 0.14], indicating that they shared less than 1% in variance. The studies showed significant heterogeneity, Q (31) = 593.03, p < .001, I² = 95.13. Moderator analyses revealed that the relationship was more pronounced for adults, ρˆ=.22, 95% CI [0.04, 0.40], as compared to adolescents, ρˆ=.01, 95% CI [-0.21, 0.23], or children, ρˆ=.09, 95% CI [-0.07, 0.25]. Meta-analytic structural equation modeling found little evidence for a displacement of physical activity through time spent on video gaming.

CONCLUSION

These results do not corroborate the assumption of a strong link between video gaming and body mass as respective associations are small and primarily observed among adults.

The Longitudinal Measurement of Reasoning Abilities in Students With Special Educational Needs

Students with special educational needs in the area of learning (SEN-L) have learning disabilities that can lead to academic difficulties in regular schools. In Germany, these students are frequently enrolled in special schools providing specific training and support for these students. Because of their cognitive difficulties, it is unclear whether standard achievement tests that are typically administered in educational large-scale assessments (LSA) are suitable of students with SEN-L. The present study evaluated the psychometric properties of a short instrument for the assessment of reasoning abilities that was administered as part of a longitudinal LSA to German students from special schools (N = 324) and basic secondary schools (N = 338) twice within 6 years. Item response modeling demonstrated an essentially unidimensional scale for both school types. Few items exhibited systematic differential item functioning (DIF) between students with and without SEN-L, allowing for valid cross-group comparisons. However, change analyses across the two time points needed to account for longitudinal DIF among students with SEN-L. Overall, the cognitive test allowed for a valid measurement of reasoning abilities in students with SEN-L and comparative analyses regarding students without SEN-L. These results demonstrate the feasibility of incorporating students with SEN-L into educational LSAs.

Many Labs 2: Investigating Variation in Replicability Across Samples and Settings

We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation in effect magnitudes across samples and settings. Each protocol was administered to approximately half of 125 samples that comprised 15,305 participants from 36 countries and territories. Using the conventional criterion of statistical significance ( p < .05), we found that 15 (54%) of the replications provided evidence of a statistically significant effect in the same direction as the original finding. With a strict significance criterion ( p < .0001), 14 (50%) of the replications still provided such evidence, a reflection of the extremely high-powered design. Seven (25%) of the replications yielded effect sizes larger than the original ones, and 21 (75%) yielded effect sizes smaller than the original ones. The median comparable Cohen’s ds were 0.60 for the original findings and 0.15 for the replications. The effect sizes were small (< 0.20) in 16 of the replications (57%), and 9 effects (32%) were in the direction opposite the direction of the original effect. Across settings, the Q statistic indicated significant heterogeneity in 11 (39%) of the replication effects, and most of those were among the findings with the largest overall effect sizes; only 1 effect that was near zero in the aggregate showed significant heterogeneity according to this measure. Only 1 effect had a tau value greater than .20, an indication of moderate heterogeneity. Eight others had tau values near or slightly above .10, an indication of slight heterogeneity. Moderation tests indicated that very little heterogeneity was attributable to the order in which the tasks were performed or whether the tasks were administered in lab versus online. Exploratory comparisons revealed little heterogeneity between Western, educated, industrialized, rich, and democratic (WEIRD) cultures and less WEIRD cultures (i.e., cultures with relatively high and low WEIRDness scores, respectively). Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied.

Modeling competence development in the presence of selection bias

A major challenge for representative longitudinal studies is panel attrition, because some respondents refuse to continue participating across all measurement waves. Depending on the nature of this selection process, statistical inferences based on the observed sample can be biased. Therefore, statistical analyses need to consider a missing-data mechanism. Because each missing-data model hinges on frequently untestable assumptions, sensitivity analyses are indispensable to gauging the robustness of statistical inferences. This article highlights contemporary approaches for applied researchers to acknowledge missing data in longitudinal, multilevel modeling and shows how sensitivity analyses can guide their interpretation. Using a representative sample of N = 13,417 German students, the development of mathematical competence across three years was examined by contrasting seven missing-data models, including listwise deletion, full-information maximum likelihood estimation, inverse probability weighting, multiple imputation, selection models, and pattern mixture models. These analyses identified strong selection effects related to various individual and context factors. Comparative analyses revealed that inverse probability weighting performed rather poorly in growth curve modeling. Moreover, school-specific effects should be acknowledged in missing-data models for educational data. Finally, we demonstrated how sensitivity analyses can be used to gauge the robustness of the identified effects.

The structure of the General Health Questionnaire (GHQ-12): two meta-analytic factor analyses

The General Health Questionnaire (GHQ-12) is a popular measure of psychological distress. Despite its widespread use, an ongoing controversy pertains to its internal structure. Although the GHQ-12 was originally constructed to capture a unitary construct, empirical studies identified different factor structures. Therefore, this study examined the dimensionality of the GHQ-12 in two independent meta-analyses. The first meta-analysis used summary data published in 38 primary studies (total N = 76,473). Meta-analytic exploratory factor analyses identified two factors formed by negatively and positively worded items. The second meta-analysis included individual responses of 410,640 participants from 84 independent samples. Meta-analytic confirmatory factor analyses corroborated the two-dimensional structure of the GHQ-12. However, bifactor modelling showed that most of the variance was explained by a general factor. Therefore, subscale scores reflected rather limited unique variance. Overall, the two meta-analyses demonstrated that the GHQ-12 is essentially unidimensional. It is not recommended to use and interpret subscale scores because they primarily reflect general mental health rather than distinct constructs.

The Structure of the Rosenberg Self-Esteem Scale

Abstract. The Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965 ) intends to measure a single dominant factor representing global self-esteem. However, several studies have identified some form of multidimensionality for the RSES. Therefore, we examined the factor structure of the RSES with a fixed-effects meta-analytic structural equation modeling approach including 113 independent samples (N = 140,671). A confirmatory bifactor model with specific factors for positively and negatively worded items and a general self-esteem factor fitted best. However, the general factor captured most of the explained common variance in the RSES, whereas the specific factors accounted for less than 15%. The general factor loadings were invariant across samples from the United States and other highly individualistic countries, but lower for less individualistic countries. Thus, although the RSES essentially represents a unidimensional scale, cross-cultural comparisons might not be justified because the cultural background of the respondents affects the interpretation of the items.

Cognitive Abilities Explain Wording Effects in the Rosenberg Self-Esteem Scale

There is consensus that the 10 items of the Rosenberg Self-Esteem Scale (RSES) reflect wording effects resulting from positively and negatively keyed items. The present study examined the effects of cognitive abilities on the factor structure of the RSES with a novel, nonparametric latent variable technique called local structural equation models. In a nationally representative German large-scale assessment including 12,437 students competing measurement models for the RSES were compared: a bifactor model with a common factor and a specific factor for all negatively worded items had an optimal fit. Local structural equation models showed that the unidimensionality of the scale increased with higher levels of reading competence and reasoning, while the proportion of variance attributed to the negatively keyed items declined. Wording effects on the factor structure of the RSES seem to represent a response style artifact associated with cognitive abilities.

Opinion leadership types or continuous opinion leadership traits?

Opinion leadership is typically conceptualised as a continuous personality trait. However, many authors adhere to the view of qualitatively different opinion leadership types and apply arbitrary criteria to split continuous trait scores into two groups (i.e., opinion leaders vs. non-leaders). The present study is the first to empirically evaluate this approach. A sample of N = 3812 adults (67% women) was administered a validated opinion leadership scale. Finite mixture models examined whether the latent trait distribution can be represented by a set of discrete trait levels that reflected distinct opinion leadership types. The results did not give support to a discrete typology that distinguished leaders from non-leaders. Rather, opinion leadership was best characterised as a continuous trait.

The Measurement of Variability and Change in Life Satisfaction

Abstract. In many large-scale surveys, life satisfaction is measured with a single item. Little is known about how sensitive single-item instruments react to changes across time or in the face of critical life events. The present study draws on a Dutch sample of N = 4,034 participants who were administered three different single items and an established multi-item instrument at four annual surveys within 6 years. An indicator-specific latent state-trait growth model was employed to estimate the agreement in change coefficients for the different instruments. The results indicated that single items estimated a similar developmental decline in life satisfaction across the life course as multi-item scales, but they were less sensitive to the loss of paid work. Interindividual differences in intraindividual change were better captured with multiple items. Moreover, single items reflected occasion-specific variance to a larger degree than longer scales. In addition, occasion-specific effects were larger among younger respondents as compared to older age groups. Therefore, single items appear to represent valid measures for the estimation of the average change in life satisfaction within a sample. However, they might exhibit a low power to detect moderating influences on the estimated change trajectories and capture less trait variance than multi-item scales.

Narcissism and Social Networking Behavior: A Meta-Analysis

OBJECTIVE

The increasing popularity of social networking sites (SNS) such as Facebook and Twitter has given rise to speculations that the intensity of using these platforms is associated with narcissistic tendencies. However, recent research on this issue has been all but conclusive.

METHOD

We present a three-level, random effects meta-analysis including 289 effect sizes from 57 studies (total N = 25,631) on the association between trait narcissism and social networking behavior.

RESULTS

The meta-analysis identified a small to moderate effect of ρ = .17 (τ = .11), 95% CI [.13, .21], for grandiose narcissism that replicated across different social networking platforms, respondent characteristics, and time. Moderator analyses revealed pronounced cultural differences, with stronger associations in power-distant cultures. Moreover, social networking behaviors geared toward self-presentation and the number of SNS friends exhibited stronger effects than usage durations.

CONCLUSIONS

Overall, the study not only supported but also refined the notion of a relationship between engaging in social networking sites and narcissistic personality traits.

Human Capital and Reemployment Success: The Role of Cognitive Abilities and Personality

Involuntary periods of unemployment represent major negative experiences for many individuals. Therefore, it is important to identify factors determining the speed job seekers are able to find new employment. The present study focused on cognitive and non-cognitive abilities of job seekers that determine their reemployment success. A sample of German adults (N = 1366) reported on their employment histories over the course of six years and provided measures on their fluid and crystallized intelligence, mathematical and reading competence, and the Big Five of personality. Proportional hazard regression analyses modeled the conditional probability of finding a new job at a given time dependent on the cognitive and personality scores. The results showed that fluid and crystallized intelligence as well as reading competence increased the probability of reemployment. Moreover, emotionally stable job seekers had higher odds of finding new employment. Other personality traits of the Big Five were less relevant for reemployment success. Finally, crystallized intelligence and emotional stability exhibited unique predictive power after controlling for the other traits and showed incremental effects with regard to age, education, and job type. These findings highlight that stable individual differences have a systematic, albeit rather small, effect on unemployment durations.