Lord's paradox in a continuous setting and a regression artifact in numerical cognition research

Prospective effects of mindfulness on anxiety and depressive symptoms may be spurious: Simulated reanalysis of a meta-analytic cross-lagged panel analysis

A recent meta-analysis claimed decreasing prospective effects of acting with awareness and non-reacting, two facets of dispositional mindfulness, on subsequent anxiety and depressive symptoms. However, the meta-analytic cross-lagged effects were estimated while adjusting for a prior measurement of the outcome variable and it is known that such adjusted cross-lagged effects may be spurious due to correlations with residuals and regression to the mean. We fitted competing models on simulations of the same meta-analytic data and found that prospective effects of the mindfulness facets on anxiety and depressive symptoms probably were spurious. It is important for researchers to be aware of limitations of adjusted cross-lagged effects, meta-analytically estimated or not, in order not to overinterpret findings.

A formal model accounting for measurement reliability shows attenuated effect of higher education on intelligence in longitudinal data

The effect of higher education on intelligence has been examined using longitudinal data. Typically, these studies reveal a positive effect, approximately 1 IQ point per year of higher education, particularly when pre-education intelligence is considered as a covariate in the analyses. However, such covariate adjustment is known to yield positively biased results if the covariate has measurement errors and is correlated with the predictor. Simultaneously, a negative bias may emerge if the intelligence measure after higher education has non-classical measurement errors as in data from the 1970 British Cohort Study that were used in a previous study of the effect of higher education. In response, we have devised an estimation method that used iterated simulations to account for both classical measurement errors in the covariate and non-classical errors in the dependent variable. Upon applying this method in a reanalysis of the data from the 1970 British Cohort Study, we find that the estimated effect of higher education diminishes to 0.4 IQ points per year. Additionally, our findings suggest that the impact of higher education is somewhat more pronounced in the initial 2 years of higher education, aligning with the notion of diminishing marginal cognitive benefits.

Spurious prospective effects between general and domain-specific self-esteem: A reanalysis of a meta-analysis of longitudinal studies

A recent meta-analysis, of 38 studies with data from 43 independent samples (total N = 24,668), claimed evidence for positive reciprocal prospective effects, and hence for both top-down and bottom-up processes, between general and domain-specific self-esteem. However, the meta-analytic cross-lagged effects were estimated while adjusting for a prior measurement of the outcome variable and it is known that such adjusted cross-lagged effects may be spurious due to correlations with residuals and regression to the mean. In the present reanalyses, we found all of the prospective effects to be spurious. Consequently, claims about increasing prospective effects and top-down and bottom-up processes between general and domain-specific self-esteem can be questioned. It is important for researchers to be aware of the limitations of cross-lagged panel analyses, and of analyses of correlational data in general, in order not to overinterpret findings.

Predictors of homework engagement in group CBT for social anxiety: client beliefs about homework, its consequences, group cohesion, and working alliance

OBJECTIVE

Group cognitive behaviour therapy (CBT) for social anxiety disorder (SAD) is effective, but little data exist on generic relational components of the therapeutic process, such as group cohesion and therapy alliance, and central CBT-specific components such as homework engagement, beliefs, and perceived consequences. The aim of this study was to investigate the relationships between homework, group cohesion, and working alliance during group CBT for social anxiety disorder.

METHOD

Participants (N = 105) with SAD engaged in 12 sessions of group CBT. Measures of homework, working alliance, and group cohesion were completed at multiple points throughout treatment. Random-intercept cross-lagged panel models were used to evaluate the prospective relationships between measures.

RESULTS

Prospective relationships between the homework outcomes did not vary throughout the treatment period, with the only significant relationships seen between the random intercepts ("trait" levels). Homework beliefs were a significant negative predictor of future group cohesion, but only in mid- to late-treatment. Homework consequences and working alliance were significantly and positively predictive of each other throughout therapy.

CONCLUSION

Early homework engagement is associated with higher engagement throughout therapy. Working alliance and homework engagement are important to bolster early in group CBT.Trial registration: Australian New Zealand Clinical Trials Registry identifier: ACTRN12616000579493..

Prospective associations between social support and posttraumatic stress disorder may be spurious: A re-meta-analysis of longitudinal studies and a simulation

A recent meta-analysis concluded to have found proof for both a social causation model, according to which social support protects against posttraumatic stress disorder (PTSD), and a social selection model, which claims that PTSD erodes social support. However, the prospective cross-lagged effects were estimated while adjusting for a prior measurement of the outcome and this method is vulnerable for spurious findings due to correlations with residuals and regression toward the mean. The present re-analyses of the meta-analytic effects indicated that depending on used model one can choose to claim that social support has either a decreasing, an increasing, or no prospective effect on subsequent change in PTSD symptom severity, and vice versa. Therefore, claims over and above a negative cross-sectional correlation between social support and PTSD, including the social causation and social selection models, can be questioned. The findings were validated by analyses of simulated data, which indicated that prospective effects were not necessary for the observed meta-analytic associations.

Distorted meta-analytic findings on peer influence: A reanalysis

In a recent meta-analysis, Giletta et al. (2021) [1] found a positive effect of peers' behavior at time 1 on target youths' behavior at time 2 while adjusting for target youths' behavior at time 1 and claimed to have quantified peer influence. However, it is established that controlled cross-lagged effects could be due to correlations with measurement errors and reversion in the direction of the mean rather than due to true decreasing or increasing effects. Here, in a reanalysis of the same meta-analytic data as used by Giletta et al., we found that peer influence, as operationalized by Giletta et al., may have been distorted (i.e. spurious). We do not claim that peer influence does not exist, but it may be hard, maybe not even possible, to prove by analyses of observational data that it does exist. Difficulties to prove causal effects by analyses of observational data is common for all areas of research and not specific for research on peer influence.

Unmasking artifactual links: A reanalysis reveals No direct causal relationship between self-esteem and quality of social relations

A meta-analysis conducted by Harris and Orth (2020) found positive prospective cross-lagged effects between quality of social relations and self-esteem in included longitudinal studies. Harris and Orth concluded that the link between self-esteem and quality of social relations is reciprocal and characterized by a positive feedback loop. However, meta-analytic effects were estimated while controlling for a prior measurement of the outcome and such effects are known to be susceptible to artifactual (i.e. spurious) effects due to correlations with measurement errors and reversion to mediocrity. We reanalyzed the same data and found paradoxical effects indicating, simultaneously, both increasing and decreasing effects between self-esteem and social relations. These findings suggest that prospective effects between self-esteem and quality of social relations are artifactual rather than due to a true reciprocal effect. Thus, these findings have important theoretical implications and challenge both the risk regulation model, which posits that self-esteem has a causal effect on quality of social relations, and the sociometer theory, which claims that quality of relations is the cause and self-esteem the effect. The present results prompt further investigation into the underlying mechanisms driving these artifactual associations. Additionally, the study highlights the importance of considering methodological limitations in future meta-analyses to improve the accuracy of causal inferences.

Gender-Sexuality Alliance Advisors' Self-Efficacy to Address Transgender Issues: An Interpersonal Protective Factor for Transgender Student Depression

Purpose: Our purpose was to assess the association between Gender-Sexuality Alliances (GSAs) advisors' self-efficacy to address transgender issues and their students' depressive symptoms, by students' gender identity (i.e., transgender vs. cisgender). We predict that higher advisor self-efficacy will be associated with decreases in student depressive symptoms for transgender students, though not necessarily for cisgender students. Methods: Data come from surveys of student members (n = 366) and advisors (n = 58) of 38 purposively sampled GSAs in Massachusetts high schools, in 2016-2017 and 2017-2018. We used a linear mixed-effects model to assess the association between advisor self-efficacy to address transgender issues and student change in Center for Epidemiological Studies Depression-10 scores between the beginning and end of the school year by gender identity, adjusting for student covariates. Results: Students were 10-20 years old (mean = 15, standard deviation [SD] = 1.4); 28% were transgender, 28% were students of color, and 86% were lesbian, gay, bisexual, or queer/questioning or other non-heterosexual identity. The GSA advisor self-efficacy scores ranged from 13 to 25 with a mean of 20.4 (SD = 3.0). Greater advisor self-efficacy to address transgender issues was associated with a decrease in depressive symptoms for transgender students (estimate = -0.47, p = 0.01), but not for cisgender students. Conclusions: GSA advisor self-efficacy to address transgender issues could be protective for transgender student depressive symptoms. Thus, increasing advisor self-efficacy to address transgender issues may help decrease depressive symptomatology for transgender youth, and intervention work in this area is needed to bolster this claim.

Dangers of including outcome at baseline as a covariate in latent change score models: Results from simulations and empirical re-analyses

Latent change score modeling is a type of structural equation modeling used for estimating change over time. Often change is regressed on the initial value of the outcome variable. However, similarly to other regression analyses, this procedure may be susceptible to regression to the mean. The present study employed simulations as well as re-analyses of previously published data, claimed to indicate reciprocal promoting effects of vocabulary and matrix reasoning on each other's longitudinal development. Both in simulations and empirical re-analyses, when adjusting for initial value on the outcome, latent change score modeling tended to indicate an effect of a predictor on the change in an outcome even when no change had taken place. Furthermore, analyses tended to indicate a paradoxical effect on change both forward and backward in time. We conclude that results from latent change score modeling are susceptible to regression to the mean when adjusting for the initial value on the outcome. Researchers are recommended not to regress change on the initial value included in the calculation of the change score when employing latent change score modeling but, instead, to define this parameter as a covariance.

Target parameters and bias in non-causal change-score analyses with measurement errors

In studies where the outcome is a change-score, it is often debated whether or not the analysis should adjust for the baseline score. When the aim is to make causal inference, it has been argued that the two analyses (adjusted vs. unadjusted) target different causal parameters, which may both be relevant. However, these arguments are not applicable when the aim is to make predictions rather than to estimate causal effects. When the scores are measured with error, there have been attempts to quantify the bias resulting from adjustment for the (mis-)measured baseline score or lack thereof. However, these bias results have been derived under an unrealistically simple model, and assuming that the target parameter is the unadjusted (for the true baseline score) association, thus dismissing the adjusted association as a possibly relevant target parameter. In this paper we address these limitations. We argue that, even if the aim is to make predictions, there are two possibly relevant target parameters; one adjusted for the baseline score and one unadjusted. We consider both the simple case when there are no measurement errors, and the more complex case when the scores are measured with error. For the latter case, we consider a more realistic model than previous authors. Under this model we derive analytic expressions for the biases that arise when adjusting or not adjusting for the (mis-)measured baseline score, with respect to the two possible target parameters. Finally, we use these expressions to discuss when adjustment is warranted in change-score analyses.

Questioning the vulnerability model: Prospective associations between low self-esteem and subsequent depression ratings may be spurious

BACKGROUND

According to the vulnerability model, low self-esteem makes people more depressed. Support for the vulnerability model comes almost exclusively from analyses using cross-lagged panel models, showing a negative effect of initial self-esteem on subsequent depression ratings when adjusting for initial depression. However, it is well known that such adjusted effects are susceptible to regression toward the mean.

METHODS

Data from four waves of measurements in five different samples (total N = 2703) were analyzed with two different cross-lagged panel models, two different random intercept cross-lagged panel models, and two different latent change score models, predicting change forwards as well as backwards in time.

RESULTS

High initial self-esteem predicted both decreased and increased depression ratings between measurements and an increase in self-esteem between measurements predicted a concurrent decrease in depression ratings.

LIMITATIONS

Only data from two western countries, Switzerland and USA, were analyzed. Whether the main finding, that a prospective effect of self-esteem on subsequent depression ratings might be spurious, applies to other countries and cultures remains an open question.

CONCLUSIONS

Due to the incongruent results, any causal effect of self-esteem on depression ratings, and thus the vulnerability model as such, cannot be corroborated by the data and models analyzed here. Instead, we propose, tentatively, that prospective associations between self-esteem and depression ratings may be spurious due to a combination of reasons, including regression toward the mean. The indication that depression might not be affected by measures to improve individuals' self-esteem is of clinical relevance.

Pitfalls in Post Hoc Analyses of Population Receptive Field Data

Data binning involves grouping observations into bins and calculating bin-wise summary statistics. It can cope with overplotting and noise, making it a versatile tool for comparing many observations. However, data binning goes awry if the same observations are used for binning (selection) and contrasting (selective analysis). This creates circularity, biasing noise components and resulting in artifactual changes in the form of regression towards the mean. Importantly, these artifactual changes are a statistical necessity. Here, we use (null) simulations and empirical repeat data to expose this flaw in the scope of post hoc analyses of population receptive field data. In doing so, we reveal that the type of data analysis, data properties, and circular data cleaning are factors shaping the appearance of such artifactual changes. We furthermore highlight that circular data cleaning and circular sorting of change scores are selection practices that result in artifactual changes even without circular data binning. These pitfalls might have led to erroneous claims about changes in population receptive fields in previous work and can be mitigated by using independent data for selection purposes. Our evaluations highlight the urgency for us researchers to make the validation of analysis pipelines standard practice.

Regression to the mean in latent change score models: an example involving breastfeeding and intelligence

Background

Latent change score models are often used to study change over time in observational data. However, latent change score models may be susceptible to regression to the mean. Earlier observational studies have identified a positive association between breastfeeding and child intelligence, even when adjusting for maternal intelligence.

Method

In the present study, we investigate regression to the mean in the case of breastfeeding and intelligence of children. We used latent change score modeling to analyze intergenerational change in intelligence, both from mothers to children and backward from children to mothers, in the 1979 National Longitudinal Survey of Youth (NLSY79) dataset (N = 6283).

Results

When analyzing change from mothers to children, breastfeeding was found to have a positive association with intergenerational change in intelligence, whereas when analyzing backward change from children to mothers, a negative association was found.

Conclusions

These discrepant findings highlight a hidden flexibility in the analytical space and call into question the reliability of earlier studies of breastfeeding and intelligence using observational data.

Clinical and parental predictors of emotion regulation following cognitive behaviour therapy in children with autism

LAY ABSTRACT

Children with autism commonly experience difficulty controlling their emotions. Although existing treatments are successful in teaching critical emotion regulation skills, not all children improve. It is important to identify the factors that influence treatment response to be able to reach more children. This study aimed to identify child and parent characteristics that predict treatment response in a 10-week cognitive behaviour therapy treatment for children with autism, 8-12 years of age, and their parents. We found that youth who started the treatment with higher verbal abilities, who were more anxious in social situations, and had parents who were more anxious, were more likely to improve in learning new emotion regulation skills. We also found that children who had more physical discomforts or complaints before starting the treatment were less likely to improve in their negative expressions of emotion. Our study suggests that it is important for clinicians to promote active involvement and learning by avoiding complex language and to use more visual materials to supplement the learning process, and make sure that sessions are sensitive to the individual needs of participants.

Do Textbooks Matter for Reading Comprehension? A Study in Flemish Primary Education

This study assessed whether textbooks affect academic performance and engagement in reading comprehension in primary education in Flanders (Belgium). The data of the Progress in International Reading Literacy Study 2016 and a reassessment of this study in 2018 were used to describe students’ learning progress in reading comprehension and evolution in engagement between the fourth and sixth grade. The sample consisted of 3051 students in 98 schools. The averages of students’ learning progress and engagement were compared for five textbooks by using multilevel autoregression model and multilevel change score models. Contrasts between textbooks in average learning progress and engagement were also estimated. To control for differences between student populations that are educated with the different textbooks, we controlled for student’s socioeconomic status, language and initial academic performance in fourth grade at the student- and school-level. The main hypotheses were that textbooks affect learning progress and reading engagement. This was based on the literature and prior (mainly) cross-sectional research which describe textbooks as playing an important role in the curriculum that is taught to students on a daily basis. The results of both models showed that textbooks do not affect student’s average learning progress in reading comprehension and evolution in engagement between the fourth grade and sixth grade in Flanders. Hence, the hypotheses were rejected.

Predicting the Effect of a Predictor When Controlling for Baseline

The present simulation study indicates that a method where the regression effect of a predictor (X) on an outcome at follow-up (Y1) is calculated while adjusting for the outcome at baseline (Y0) can give spurious findings, especially when there is a strong correlation between X and Y0 and when the test-retest correlation between Y0 and Y1 is relatively weak. Researchers wishing to avoid spurious findings and Type 1 errors should be aware of this phenomenon and are recommended to verify found effects by an unadjusted effect of X on the Y1-Y0 difference.

Five-year change scores in old age for six neuropsychological tests and normative data for the Useful Field of View (UFOV) test: The influence of physical health

INTRODUCTION

Neuropsychological assessment of cognitive change over time is often conducted in clinical settings, but whether neuropsychological change scores are influenced by physical health has, as far as we know, not been examined previously.

METHOD

In a sample of 153 older Swedish adults (age range, 72-86 years), we evaluated the influence of common age-related diseases, terminal decline pathology, age, education, and gender, to provide (a) preliminary test-specific regression weights and 90% confidence intervals to assess significant change in performance after five years on tests of visual scanning, mental shifting, visual spatial ability, memory, reaction time, and selective attention, and (b) normative data for the Useful Field of View test (UFOV) from a single testing occasion.

RESULTS

Multiple regression analyses showed that test-retest changes were affected by physical health for mental shifting, visual spatial ability, memory, and reaction time, by age for mental shifting and visual reaction time, by education for visual spatial ability, and by Age × Education for auditory reaction time. Gender did not affect any of the change scores. The overall average of variance explained was 2.5%: up to 8.1% for physical health, 4.4% for age, and 3.6% for education. The UFOV scores were mostly influenced by age, but also by physical health and education.

CONCLUSIONS

The findings indicate that considering the influence of health on normative change scores in old age in addition to demographic factors leads to more accurate predictions of whether true change has occurred.

Using Logistic Regression to Predict Onset and Recovery With Tau Equivalency

Many studies analyze the effect of a predictor X on the onset or recovery of an outcome Y, for example some kind of disorder. The findings from this simulation study indicate that such effects can be found even if there are no changes in individuals’ true scores on the outcome, i.e., with tau equivalency, given some degree of positive test-retest correlation of the outcome and a correlation between the predictor and the outcome at baseline. Researchers predicting onset/recovery should be aware of this fact and in order not to draw hasty conclusions to control for what can be expected from these correlations alone.

Associations between loneliness and perceived social support and outcomes of mental health problems: a systematic review

BACKGROUND

The adverse effects of loneliness and of poor perceived social support on physical health and mortality are established, but no systematic synthesis is available of their relationship with the outcomes of mental health problems over time. In this systematic review, we aim to examine the evidence on whether loneliness and closely related concepts predict poor outcomes among adults with mental health problems.

METHODS

We searched six databases and reference lists for longitudinal quantitative studies that examined the relationship between baseline measures of loneliness and poor perceived social support and outcomes at follow up. Thirty-four eligible papers were retrieved. Due to heterogeneity among included studies in clinical populations, predictor measures and outcomes, a narrative synthesis was conducted.

RESULTS

We found substantial evidence from prospective studies that people with depression who perceive their social support as poorer have worse outcomes in terms of symptoms, recovery and social functioning. Loneliness has been investigated much less than perceived social support, but there is some evidence that greater loneliness predicts poorer depression outcome. There is also some preliminary evidence of associations between perceived social support and outcomes in schizophrenia, bipolar disorder and anxiety disorders.

CONCLUSIONS

Loneliness and quality of social support in depression are potential targets for development and testing of interventions, while for other conditions further evidence is needed regarding relationships with outcomes.

Adjustment to retirement: effects of resource change on physical and psychological well-being

A 1-year longitudinal study was conducted to examine the effects of retirement resources on the physical and psychological well-being of Hong Kong Chinese retirees during the transition to retirement. This study consisted of two assessments: Time 1 was conducted 6 months before retirement, while Time 2 was implemented 6 months after retirement. Personal resources and physical and psychological well-being were measured in the two assessments. The final sample contained 128 retirees who completed both assessments. Compared with Time 1, the retirees reported fewer financial resources at Time 2. Change in cognitive resources was significantly predictive of the changes in physical functioning, life satisfaction, psychological well-being, and psychological distress during the transition period. The findings of this longitudinal study reveal that in addition to financial, physical, and social resources that have often been emphasized in the past literature, mental resources also play an important role in positive adjustment to retirement. Future retirement planning programs are recommended to include modules for strengthening cognitive, emotional, and motivational resources of retired persons.