A Meta-Meta-Analysis: Empirical Review of Statistical Power, Type I Error Rates, Effect Sizes, and Model Selection of Meta-Analyses Published in Psychology

Exploring the impact of socially assistive robots on health and wellbeing across the lifespan: An umbrella review and meta-analysis

BACKGROUND

Socially assistive robots offer an alternate source of connection for interventions within health and social care amidst a landscape of technological advancement and reduced staff capacity. There is a need to summarise the available systematic reviews on the health and wellbeing impacts to evaluate effectiveness, explore potential moderators and mediators, and identify recommendations for future research and practice.

OBJECTIVE

To explore the effect of socially assistive robots within health and social care on psychosocial, behavioural, and physiological health and wellbeing outcomes across the lifespan (PROSPERO registration number: CRD42023423862).

DESIGN

An umbrella review utilising meta-analysis, narrative synthesis, and vote counting by direction of effect.

METHODS

14 databases were searched (ProQuest Health Research Premium collection, Scopus, PubMed, Web of Science, ASM Digital Library, IEEE Xplore, Cochrane Reviews, and EPISTEMONIKOS) from 2005 to May 4, 2023. Systematic reviews including the effects of socially assistive robots on health outcomes were included and a pooled meta-analysis, vote counting by direction of effect, and narrative synthesis were applied. The second version of A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) was applied to assess quality of included reviews.

RESULTS

35 reviews were identified, most focusing on older adults with or without dementia (n = 24). Pooled meta-analysis indicated no effect of socially assistive robots on quality of life (standard mean difference (SMD) = 0.43), anxiety (SMD = -0.02), or depression (SMD = 0.21), although vote counting identified significant improvements in social interaction, mood, positive affect, loneliness, stress, and pain across the lifespan, and narrative synthesis identified an improvement in anxiety in children. However, some reviews reported no significant difference between the effects of socially assistive robots and a plush toy, and there was no effect of socially assistive robots on psychiatric outcomes including agitation, neuropsychiatric symptoms, and medication use.

DISCUSSION

Socially assistive robots show promise for improving non-psychiatric outcomes such as loneliness, positive affect, stress, and pain, but exert no effect on psychiatric outcomes such as depression and agitation. The main mechanism of effect within group settings appeared to be the stimulation of social interaction with other humans. Limitations include the low quality and high amount of overlap between included reviews.

CONCLUSION

Socially assistive robots may help to improve loneliness, social interaction, and positive affect in older adults, decrease anxiety and distress in children, and improve mood, stress, and reduce pain across the lifespan. However, before recommendations for socially assistive robots can be made, a cost-effectiveness analysis of socially assistive robots to improve mood across the lifespan, and a quantitative analysis of the effects on pain, anxiety, and distress in children are required.

One research question, two meta-analyses, three conclusions: Commentary on "A systematic review with meta-analysis of Cognitive Bias Modification interventions for anger and aggression"

Recently two independent meta-analyses on the efficacy of Cognitive Bias Modification of Interpretation (CBM-I) to reduce aggressive behavior came to different conclusions: Ciesinski et al. (2023) concluded that "CBM demonstrates efficacy for the treatment of aggressive behavior" (Abstract), whereas our research team concluded that "findings show limited support for the efficacy of CBM-I to reduce aggressive behavior" (AlMoghrabi et al., 2023, Discussion). How can similar meta-analyses reach such different conclusions? In this commentary, we raise awareness concerning how 1) seemingly identical research questions can be based on meaningfully different definitions of the intervention and outcomes; 2) intervention efficacy conclusions can depend on outcome assessment type; and 3) the interpretation of underpowered moderator analyses should not depend on statistical significance. We end our commentary with a third, more nuanced conclusion that can reconcile the two disparate conclusions: that current CBM-I is an effective experimental manipulation to modify interpretation biases, but not an effective stand-alone treatment to reduce aggressive behavior.

Developmental psychologists should adopt citizen science to improve generalization and reproducibility

Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large numbers of subjects from a wide range of demographics with a large, randomly-sampled stimulus set, and using a variety of experimental parameters. Because few studies accomplish any of this, meta-scientists predict that findings will frequently fail to replicate or generalize. We argue that to be more robust and replicable, developmental psychology needs to find a mechanism for collecting data at greater scale and from more diverse populations. Luckily, this mechanism already exists: Citizen science, in which large numbers of uncompensated volunteers provide data. While best-known for its contributions to astronomy and ecology, citizen science has also produced major findings in neuroscience and psychology, and increasingly in developmental psychology. We provide examples, address practical challenges, discuss limitations, and compare to other methods of obtaining large datasets. Ultimately, we argue that the range of studies where it makes sense *not* to use citizen science is steadily dwindling.

Effectiveness of cognitive behavioural therapy-based interventions for maternal perinatal depression: a systematic review and meta-analysis

BACKGROUND

Depression during the perinatal period (during pregnancy and the year after childbirth) is common and associated with a range of negative effects for mothers, infants, family members, and wider society. Although existing evidence suggests cognitive behavioral therapy (CBT) based interventions are effective for perinatal depression, less is known about the effect of CBT-based interventions on important secondary outcomes, and a number of potential clinical and methodological moderators have not been examined.

METHODS

A systematic review and meta-analysis primarily examined the effectiveness of CBT-based interventions for perinatal depression on symptoms of depression. Secondary aims examined the effectiveness of CBT-based interventions for perinatal depression on symptoms of anxiety, stress, parenting, perceived social support, and perceived parental competence; and explored clinical and methodological moderators potentially associated with effectiveness. A systematic search of electronic databases and other sources was performed up to November 2021. We included randomized controlled trials comparing CBT-based interventions for perinatal depression with control conditions allowing for the isolation of the effects of CBT.

RESULTS

In total, 31 studies (5291 participants) were included in the systematic review and 26 studies (4658 participants) were included in the meta-analysis. The overall effect size was medium (hedges g = - 0.53 [95% CI - 0.65 to - 0.40]); with high heterogeneity. Significant effects were also found for anxiety, individual stress, and perceived social support, however few studies examined secondary outcomes. Subgroup analysis identified type of control, type of CBT, and type of health professional as significant moderators of the main effect (symptoms of depression). Some concerns of risk of bias were present in the majority of studies and one study had a high risk of bias.

CONCLUSIONS

CBT-based interventions for depression during the perinatal period appear effective, however results should be interpreted with caution given high levels of heterogeneity and low quality of included studies. There is a need to further investigate possibly important clinical moderators of effect, including the type of health professional delivering interventions. Further, results indicate a need to establish a minimum core data set to improve the consistency of secondary outcome collection across trials and to design and conduct trials with longer-term follow-up periods.

TRIAL REGISTRATION

CRD42020152254 .

Infants' preference for speech is stable across the first year of life: Meta-analytic evidence

Previous work suggested that humans' sophisticated speech perception abilities stem from an early capacity to pay attention to speech in the auditory environment. What are the roots of this early preference? We assess the extent to which it is due to it being a vocal sound, a natural sound, and a familiar sound through a meta-analytic approach, classifying experiments as a function of whether they used native or foreign speech and whether the competitor, against which preference is tested, was vocal or non-vocal, natural or artificial. We also tested for the effect of age. Synthesizing data from 791 infants across 39 experiments, we found a medium effect size, confirming at the scale of the literature that infants reliably prefer speech over other sounds. This preference was not significantly moderated by the language used, vocal quality, or naturalness of the competitor, nor by infant age. The current body of evidence appears most compatible with the hypothesis that speech is preferred consistently as such and not just due to its vocal, natural, or familiar nature. We discuss limitations of the extant body of work on speech preference, including evidence consistent with a publication bias and low representation of certain stimuli types and ages.

Predicting implicit and explicit exercise identity from descriptive social norms regarding exercise

OBJECTIVES

Exercise identity may promote exercise maintenance. However, less is known about factors that affect exercise identity. Whether descriptive social norms are potential intervention targets for identity development was evaluated.

DESIGN

A cross-sectional design using data from the Attitudes, Identities, and Individual Differences (AIID) study was employed - with additional cases collected to increase sample size and power - to evaluate whether descriptive social norms regarding exercise are related to implicit and explicit exercise identities.

MAIN OUTCOME MEASURES

Participants completed measures of proximal and distal descriptive social norms regarding exercise, explicit and implicit exercise identity, physical activity behavior, and demographics. Multiple regression was used to assess whether social norms regarding exercise predict exercise identities.

RESULTS

Only proximal descriptive social norms were significantly associated with explicit exercise identity, whereas neither proximal nor distal descriptive social norms were associated with implicit exercise identity. The slopes for explicit and implicit identity differed when predicted by distal (but not proximal) descriptive social norms.

CONCLUSIONS

Proximal descriptive social norms may be associated with explicit exercise identity and may be a worthy intervention targeting alongside identity to influence change in exercise behavior. More research is needed to further understand these relationships.

Comparing methods for handling missing covariates in meta-regression

Meta-analysts often encounter missing covariate values when estimating meta-regression models. In practice, ad hoc approaches involving data deletion have been widely used. The current study investigates the performance of different methods for handling missing covariates in meta-regression, including complete-case analysis (CCA), shifting-case analysis (SCA), multiple imputation (MI), and full information maximum likelihood (FIML), assuming missing at random mechanism. According to the simulation results, we advocate the use of MI and FIML than CCA and SCA approaches in practice. In addition, we cautiously note the challenges and potential advantages of using MI in the meta-analysis context.

Systematic review and meta-analysis on the etiology of bacterial pneumonia in children in sub-Saharan Africa

Introduction

Before the introduction of vaccination to protect children from pneumonia, Streptococcus pneumoniae and Haemophilus influenzae type B (HiB) were the most frequent aetiological agents causing bacterial pneumonia in children under five years old. However, the etiology of childhood pneumonia appears to be changing and nonvaccine- type S. pneumoniae, non-typeable H. influenzae, and Staphylococcus aureus are becoming more relevant.

Objective

We conducted a systematic review aimed at identifying the common causes of bacterial pneumonia in children in sub-Saharan Africa.

Methods

We searched PubMed, Web of Science and African Index Medicus and included primary studies conducted since January 2010 that reported on the bacterial causes of pneumonia in children under five from sub-Saharan Africa. We extracted data items (about the study setting, pneumonia diagnosis, sampling, microbiological methods, and etiological agents) as well as study quality indicators.

Results

Streptococcus pneumoniae was the most common bacteria in blood cultures from children with pneumonia (8%, 95% CI: 4-14%), and H. influenzae was second (3%, 95% CI: 1-17%). Children’s nasopharynx commonly contained S. pneumoniae (66%), Moraxella catarrhalis (62%), and H. influenzae (44%).

Conclusion

S. pneumoniae and H. influenzae cause bacterial pneumonia in sub-Saharan African children. Our review also highlights the prevalence of potentially pathogenic bacteria in the nasopharynx of children under five and calls for more research into how nasopharyngeal colonization causes pneumonia.

A meta-analysis of conditioned fear generalization in anxiety-related disorders

Generalization of conditioned fear is adaptive in some situations but maladaptive when fear excessively generalizes to innocuous stimuli with incidental resemblance to a genuine threat cue. Recently, empirical interest in fear generalization as a transdiagnostic explanatory mechanism underlying anxiety-related disorders has accelerated. As there are now several studies of fear generalization across multiple types of anxiety-related disorders, the authors conducted a meta-analysis of studies reporting behavioral measures (subjective ratings and psychophysiological indices) of fear generalization in anxiety-related disorder vs. comparison groups. We conducted systematic searches of electronic databases (conducted from January-October 2020) for fear generalization studies involving anxiety-related disorder groups or subclinical analog groups. A total of 300 records were full-text screened and two unpublished datasets were obtained, yielding 16 studies reporting behavioral fear generalization measures. Random-effects meta-analytic models and meta-regressions were applied to the identified data. Fear generalization was significantly heightened in anxiety-related disorder participants (N = 439) relative to comparison participants (N = 428). We did not identify any significant clinical, sample, or methodological moderators. Heightened fear generalization is quantitatively supported as distinguishing anxiety-related disorder groups from comparison groups. Evidence suggests this effect is transdiagnostic, relatively robust to experimental or sample parameters, and that generalization paradigms are a well-supported framework for neurobehavioral investigations of learning and emotion in anxiety-related disorders. We discuss these findings in the context of prior fear conditioning meta-analyses, past neuroimaging investigations of fear generalization in anxiety-related disorders, and future directions and challenges for the field.

Brexit and Trump: Which Theory of Social Stasis and Social Change Copes Best With the New Populism?

Why do voters seek to change the political landscape or to retain it? System justification theory (SJT) proposes that a separate system motive to preserve the existing order drives support for the status-quo, and that this motivation operates independently from personal and collective interests. But how does this explanation apply to recent populist shifts in the political order such as Brexit and the emergence of Donald Trump? While the system motive may seem useful in understanding why the usual progressives (Remain/Clinton voters) may want to stick with an established order, it seems insufficient to explain why the more conservative voters (Brexit/Trump voters) would want to upend the establishment. Thus, we compared SJT’s system motive explanation for the system attitudes of voters on both sides of the political divide to an alternative explanation drawn from the newer social identity model of system attitudes (SIMSA). According to SIMSA, the difficulty in explaining the system attitudes of Brexit/Trump and Remain/Clinton voters from SJT’s system motive standpoint can be resolved by focusing instead on the collective interests that both camps seek to satisfy with their votes. We examined these explanations in two studies conducted soon after Brexit (N = 313) and Trump’s election (N = 289) in 2016, with results providing more support for SIMSA than for SJT.

The Undoing Effect of Positive Emotions: A Meta-Analytic Review

The undoing hypothesis proposes that positive emotions serve to undo sympathetic arousal related to negative emotions and stress. However, a recent qualitative review challenged the undoing effect by presenting conflicting results. To address this issue quantitatively, we conducted a meta-analytic review of 16 studies ( N = 1,220; 72 effect sizes) measuring sympathetic recovery during elicited positive emotions and neutral conditions. Findings indicated that in most cases, positive emotions did not speed sympathetic recovery compared to neutral conditions. However, when a composite index of cardiovascular reactivity was used, undoing effects were evident. Our findings suggest the need for further work on the functions of positive emotions.

Determining an Evidence Base for Particular Fields of Educational Practice: A Systematic Review of Meta-Analyses on Effective Mathematics and Science Teaching

The call for evidence-based practice in education emphasizes the need for research to provide evidence for particular fields of educational practice. With this systematic literature review we summarize and analyze aggregated effectiveness information from 41 meta-analyses published between 2004 and 2019 to inform evidence-based practice in a particular field. In line with target specifications in education that are provided for a certain school subject and educational level, we developed and adopted a selection heuristic for filtering aggregated effect sizes specific to both science and mathematics education and the secondary student population. The results include 78 context-specific aggregated effect sizes based on data from over one million students. The findings encompass a multitude of different teaching strategies, most of which offer a measurable advantage to alternatives. Findings demonstrate that context-specific effect size information may often differ from more general effect size information on teaching effectiveness and adherence to quality standards varies in sampled meta-analyses. Thus, although meta-analytic research has strongly developed over the last few years, providing context-specific and high-quality evidence still needs to be a focus in the field of secondary mathematics and science teaching and beyond.

Autonomic Nervous System Activity During Positive Emotions: A Meta-Analytic Review

Autonomic nervous system (ANS) activity is a fundamental component of emotional responding. It is not clear, however, whether positive emotional states are associated with differential ANS reactivity. To address this issue, we conducted a meta-analytic review of 120 articles (686 effect sizes, total N = 6,546), measuring ANS activity during 11 elicited positive emotions, namely amusement, attachment love, awe, contentment, craving, excitement, gratitude, joy, nurturant love, pride, and sexual desire. We identified a widely dispersed collection of studies. Univariate results indicated that positive emotions produce no or weak and highly variable increases in ANS reactivity. However, the limitations of work to date – which we discuss – mean that our conclusions should be treated as empirically grounded hypotheses that future research should validate.

Emotions in Intergroup Contact: Incidental and Integral Emotions' Effects on Interethnic Bias Are Moderated by Emotion Applicability and Subjective Agency

This research draws from three distinct lines of research on the link between emotions and intergroup bias as springboard to integrative, new hypotheses. Past research suggests that emotions extrinsic to the outgroup (or “incidental”), and intrinsic to the outgroup (or “integral”), produce valence-congruent effects on intergroup bias when relevant or “applicable” to the outgroup (e.g., incidental/integral anger and ethnic outgroups). These emotions produce valence incongruent effects when irrelevant or “non-applicable” to the outgroup (e.g., incidental/integral sadness and happiness, and ethnic outgroups). Internally valid and ecologically sound tests of these contrasting effects are missing; hence we examined them experimentally in meaningful settings of interethnic contact. To this end, we hybridized established research paradigms in mood and intergroup contact research; this approach enabled us to use same materials and induction methods to instigate incidental and integral emotions in a single research design. In Experiment 1, White Australian students (N = 93) in in vivo real face-to-face contact with an ethnic tutor in their classroom displayed less interethnic bias when incidentally sad (vs. happy) or integrally happy (vs. sad). In Experiment 2, White American males' (N = 492) anti-Arab bias displayed divergent effects under incidental vs. integral (non-applicable) sadness/happiness and similar effects under incidental vs. integral (applicable) anger. The role of perceptions of agency in the emotion-inducing situation is also explored, tested, and explained drawing from mainstream emotion theory. As expected, integral and incidental applicable emotions caused valence congruent effects, at the opposite sides of the subjective agency spectrum, by encouraging the generalization of dislike from the outgroup contact partner to the outgroup as a whole. On the other hand, incidental-non-applicable emotions caused valence-incongruent effects on bias, under high agency conditions, by encouraging (non-partner-centered) heuristic processing. Because of the improved methodology, these effects can be regarded as genuine and not the byproduct of methodological artifacts. This theory-driven and empirically sound analysis of the interplay between emotion source, emotion applicability and subjective agency in intergroup contact can increase the precision of emotion-based bias reduction strategies by deepening understanding of the emotion conditions that lead to intergroup bias attenuation vs. exacerbation.

Pooling resources to enhance rigour in psychophysiological research: Insights from open science approaches to meta-analysis

Recent years have witnessed calls for increased rigour and credibility in the cognitive and behavioural sciences, including psychophysiology. Many procedures exist to increase rigour, and among the most important is the need to increase statistical power. Achieving sufficient statistical power, however, is a considerable challenge for resource intensive methodologies, particularly for between-subjects designs. Meta-analysis is one potential solution; yet, the validity of such quantitative review is limited by potential bias in both the primary literature and in meta-analysis itself. Here, we provide a non-technical overview and evaluation of open science methods that could be adopted to increase the transparency of novel meta-analyses. We also contrast post hoc statistical procedures that can be used to correct for publication bias in the primary literature. We suggest that traditional meta-analyses, as applied in ERP research, are exploratory in nature, providing a range of plausible effect sizes without necessarily having the ability to confirm (or disconfirm) existing hypotheses. To complement traditional approaches, we detail how prospective meta-analyses, combined with multisite collaboration, could be used to conduct statistically powerful, confirmatory ERP research.

Risk factors associated with disease aggravation among 126 hospitalized patients with COVID-19 in different places in China: A retrospective observational study

Coronavirus disease 2019 (COVID-19) has rapidly spread on a global scale. Therefore, it is urgent to identify risk factors that could be associated with severe type of COVID-19 from common type.For this retrospective study, we recruited patients with COVID-19 in Wuhan and Zhoukou. Patients were classified into a severe group and common group based on guidelines after admission. Clinical manifestations and laboratory tests were compared, and univariate binary logistic regression and multivariate regression analyses were applied to assess potential risk factors.A total of 126 patients were recruited from January 23 to March 23, 2020. Ninety cases were identified as the common type and 36 as the severe type. The average age in the severe group was significantly older than that in the common group (P = .008). Patients with severe COVID-19 exhibited higher proportions of dyspnea (P = .001), weakness (P = .023), and diarrhea (P = .046). Moreover, there were more patients with hypertension (P = .01) or coinfection (P = .001) in the severe group than in the common group. Additionally, severe COVID-19 was associated with increased neutrophil counts (P < .001), C-reactive protein (P < .001), procalcitonin (P = .024) and decreased lymphocyte counts (P = .001), hemoglobin (P < .001), total protein (TP) (P < .001), and albumin (ALB) (P < .001). Based on logistic regression analysis, dyspnea (P < .001), TP (P = .042), and ALB (P = .003) were independent risk factors for severe disease.Patients with lower TP, ALB, and dyspnea should be carefully monitored, and early intervention should be implemented to prevent the development of severe disease.

Methodological quality of meta-analyses indexed in PsycINFO: leads for enhancements: a meta-epidemiological study

OBJECTIVES

Meta-analyses (MAs) are often used because they are lauded to provide robust evidence that synthesises information from multiple studies. However, the validity of MA conclusions relies on the procedural rigour applied by the authors. Therefore, this meta-research study aims to characterise the methodological quality and meta-analytic practices of MAs indexed in PsycINFO.

DESIGN

A meta-epidemiological study.

PARTICIPANTS

We evaluated a random sample of 206 MAs indexed in the PsycINFO database in 2016.

PRIMARY AND SECONDARY OUTCOMES

Two authors independently extracted the methodological characteristics of all MAs and checked their quality according to the 16 items of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR2) tool for MA critical appraisal. Moreover, we investigated the effect of mentioning Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) on the methodological quality of MAs.

RESULTS

According to AMSTAR2 criteria, 95% of the 206 MAs were rated as critically low quality. Statistical methods were appropriate and publication bias was well evaluated in 87% and 70% of the MAs, respectively. However, much improvement is needed in data collection and analysis: only 11% of MAs published a research protocol, 44% had a comprehensive literature search strategy, 37% assessed and 29% interpreted the risk of bias in the individual included studies, and 11% presented a list of excluded studies. Interestingly, the explicit mentioning of PRISMA suggested a positive influence on the methodological quality of MAs.

CONCLUSION

The methodological quality of MAs in our sample was critically low according to the AMSTAR2 criteria. Some efforts to tremendously improve the methodological quality of MAs could increase their robustness and reliability.

Methodological quality of meta-analyses indexed in PsycINFO: leads for enhancements: a meta-epidemiological study

OBJECTIVES

Meta-analyses (MAs) are often used because they are lauded to provide robust evidence that synthesises information from multiple studies. However, the validity of MA conclusions relies on the procedural rigour applied by the authors. Therefore, this meta-research study aims to characterise the methodological quality and meta-analytic practices of MAs indexed in PsycINFO.

DESIGN

A meta-epidemiological study.

PARTICIPANTS

We evaluated a random sample of 206 MAs indexed in the PsycINFO database in 2016.

PRIMARY AND SECONDARY OUTCOMES

Two authors independently extracted the methodological characteristics of all MAs and checked their quality according to the 16 items of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR2) tool for MA critical appraisal. Moreover, we investigated the effect of mentioning Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) on the methodological quality of MAs.

RESULTS

According to AMSTAR2 criteria, 95% of the 206 MAs were rated as critically low quality. Statistical methods were appropriate and publication bias was well evaluated in 87% and 70% of the MAs, respectively. However, much improvement is needed in data collection and analysis: only 11% of MAs published a research protocol, 44% had a comprehensive literature search strategy, 37% assessed and 29% interpreted the risk of bias in the individual included studies, and 11% presented a list of excluded studies. Interestingly, the explicit mentioning of PRISMA suggested a positive influence on the methodological quality of MAs.

CONCLUSION

The methodological quality of MAs in our sample was critically low according to the AMSTAR2 criteria. Some efforts to tremendously improve the methodological quality of MAs could increase their robustness and reliability.

The influence of nonnormality from primary studies on the standardized mean difference in meta-analysis

In this study we investigated the influence of data nonnormality in the primary studies on meta-analysis of the standardized mean difference (SMD) for a two-independent-group design. The bias, mean squared error, and confidence interval coverage probability of the mean effect sizes under different types of population distributions were compared. Also, the performance of the Q test was examined. The results showed that oppositely skewed distributions (i.e., distributions skewed in different directions) showed poor performance for point and interval estimates of mean effect sizes in meta-analysis, especially when the tails were pointing toward each other. The previously found adverse impacts due to nonnormality in primary studies do not disappear when primary studies with nonnormal data are meta-analyzed, even when the average sample size and number of studies are large. The results also showed that, when the tails were pointing toward each other, the Type I error rates of the Q test were inflated. We suggest that the impact of violating the assumption of normality should not be ignored in meta-analysis.

Whose Issue Is It Anyway? The Effects of Leader Gender and Equality Message Framing on Men's and Women's Mobilization Toward Workplace Gender Equality

Social psychologists have not fully investigated the role of leadership in mobilizing widespread support for social change, particularly gender equality. The burden of achieving gender equality is typically placed on women (particularly female leaders) - the main targets of such inequality. Traditional approaches frame workplace gender equality as either a women's issue, which limits men's (non-target's) involvement in the movement, or a meritocratic non-issue that exists due to women's (target's) tendency to pursue less intensive careers. In contrast to such work focusing on women's experiences as targets of discrimination or men's role in preserving inequality, we propose a solidarity-based approach that positions men and women as agents of change. This approach relies on two processes: leadership processes - particularly leadership as a form of influence based on shared identities among leaders and followers (e.g., their gender group); and political solidarity as a way to mobilize the silent majority (men) to work as allies beside a minority (women) and embrace equality as a common cause for both groups. In two experiments (Ns = 338, 336) we studied how leader gender and message framing affect men's and women's support for equality by contrasting a solidarity-based framing of gender equality as a common cause for men and women, with a women's issue frame (Experiment 1) or a meritocratic frame (Experiment 2). The statement was attributed to a male or female leader (Experiments 1-2) or, additionally, to a government agency (Experiment 1). Women reported higher sense of common cause (Experiment 2) and collective action intentions than men (Experiments 1-2), and higher intentions under common cause compared to meritocracy frames (Experiment 2). Interestingly, male leaders invoked higher sense of common cause and collective action intentions for both men and women regardless of framing (Experiment 2). Irrespective of leader gender however, as predicted common cause framing boosted perceived leader prototypicality, legitimacy, and influence across the board (Experiments 1-2). Yet this was qualified by women (compared to men) rating leaders as more legitimate and influential under common cause compared to meritocracy framing (Experiment 2). Women's reactions to equality messages, and the intersection of leadership and solidarity toward equality are discussed.

The Neurocognitive Profile of Attention-Deficit/Hyperactivity Disorder: A Review of Meta-Analyses

Objective

Numerous meta-analyses have summarized studies comparing the neurocognitive performance of individuals with attention-deficit/hyperactivity disorder (ADHD) to that of healthy controls.

Method

The present study is a systematic review and quantitative summary of those meta-analyses that aimed to determine the extent to which individuals with ADHD differ cognitively from typically developing controls.

Results

Of 253 standardized mean differences (SMDs) drawn from 34 meta-analyses, 244 (96%) were positive, indicating better neurocognitive performance in the control group than the ADHD group. The mean effect size was .45 (SD = .27). Unweighted means of SMDs for neurocognitive domains ranged from .35 (set shifting) to .54 (working memory). When weighted by the number of studies aggregated, they ranged from .35 (set shifting) to .66 (reaction time variability). Neurocognitive domains with mean effects over .50 included working memory (.54), reaction time variability (.53), response inhibition (.52), intelligence/achievement (.51), and planning/organization (.51). When weighted by number of aggregated studies, the domains with mean effects over .50 were reaction time variability (.66), intelligence/achievement (.60), vigilance (.56), working memory (.54), and response inhibition (.52). Age moderated the relationship between ADHD diagnosis and neurocognitive functioning, with greater between-groups differences among children and adults than among adolescents. Funding also moderated this relationship: meta-analyses that received drug funding found larger effect sizes than those without drug funding.

Conclusions

The evidence suggests that ADHD is associated with substantial deficits across a variety of neurocognitive domains. This is the most in-depth review of the neurocognitive functioning of people with ADHD to date.

Music and substance use: A meta-analytic review

While previous research has documented a relation between music and substance use among consumers, to date, there are no meta-analytic reviews of the literature, making our meta-analysis the first in this area. Results from 31 studies, yielding a total of 330,652, indicated that music had a significant effect on substance use, with both music format and genre being significant contributors. The effect of music on substance use also varied by substance type. In addition, participant biological sex and location of data collection were found to moderate the effect of music on substance use. Theoretical implications are discussed along with directions for future research.

Attachment buffers the physiological impact of social exclusion

Attachment systems facilitate coping with stress, with previous studies demonstrating attachment figures diminishing subjective, behavioral and neural responses to social pain. Yet little is known about the physiological mechanisms governing this benefit in the context of social exclusion. This study investigated the impact of attachment (vs non-attachment) priming on affective and cardiovascular responses to social exclusion induced by the computerized “Cyberball” ball-tossing game, and the moderating influence of individual differences in attachment style, rejection sensitivity and self-construal. No significant change in high frequency heart rate variability (HF-HRV)–an index of parasympathetic activity and cardiovagal balance–was observed across the time course in the attachment priming condition, whereas the non-attachment condition showed significant fluctuation in HF-HRV–increasing during Cyberball and decreasing relative to baseline during recovery. Moreover, the benefit afforded by attachment priming on was enhanced amongst participants with lower rejection sensitivity and higher collectivistic self-construal, and those with higher anxious attachment style in the non-attachment prime group showed a trend towards increased HF-HRV during the Cyberball. Results are consistent with Social Baseline Theory, which argues that social proximity–particularly from attachment figures–protects against the metabolic costs associated with strong reactions to stress, including the preservation of cardiovagal homeostasis in this instance. Social attachments may provide an important mechanism to increase adaptive responding to the distressing experience of social exclusion.

Meta-analyses of individual versus group interventions for pre-school children with autism spectrum disorder (ASD)

There is little evidence regarding the effects of individual and group intervention for children with autism spectrum disorder (ASD) on important outcomes. We performed meta-analyses using a random effects model to investigate the effectiveness of the individual and group intervention studies and to compare the effectiveness of these two types if possible. The main analysis which excluded studies at a high risk of bias (Analysis I) included 14 randomised controlled trials targeting children with ASD≤6 years of age (594 children). The results suggested that both individual and group interventions showed significant effects compared to the control condition on "reciprocity of social interaction towards others" (standard mean difference[SMD] [95%confidence interval{CI}] = 0.59[0.25, 0.93], p = 0.16; 0.45[0.02, 0.88], p = 0.39, respectively). Only individual interventions showed significant effects compared to the control condition on "parental synchrony" (SMD [95%CI] = 0.99 [0.70, 1.29], p<0.01). Our results showed no significant differences between individual and group interventions in effects on "autism general symptoms" (no study available for group intervention), "developmental quotient" (no study available for group intervention), "expressive language" (p = 0.56), "receptive language" (p = 0.29), "reciprocity of social interaction towards others" (p = 0.62), or "adaptive behaviour" (p = 0.43). We also performed sensitivity analyses including studies that had been excluded due to being at a high risk of potential bias (Analysis II). The results suggested that "reciprocity of social interactions towards others" showed significant effects for individual intervention compared to the control condition (0.50[0.31,0.69], p<0.001) but not for group intervention (0.23[-0.33, 0.78], p = 0.42). Individual intervention also showed significant effects on "parental synchrony" (0.98[0.30,1.66], p = 0.005) in the sensitivity analysis. The results also suggested no significant difference on all the outcomes between the individual and group interventions. We also reanalysed the data using cluster-robust standard errors as sensitivity analyses (Analysis III). Analysis III showed no significant effects in the intervention condition compared to the control condition on all the outcomes for both individual and group interventions. When Analysis II was reanalysed using cluster-robust standard errors (Analysis IV), individual interventions showed significant effects compared to the control condition on "reciprocity of social interaction towards others" and "parental synchrony" (mean estimate[95%CI], robust standard error, p = 0.50[0.20, 0.81], 0.13, 0.006; and 1.06[0.08, 2.05], 0.42, 0.04, respectively), and none of the outcomes showed significant effects under the intervention condition compared to the control condition for group interventions. The discrepancies in the results between the main analysis (Analysis I) and the sensitivity analyses (Analyses II, III, and IV) may be due to the small number of included studies. Since the outcome of "reciprocity of social interaction towards others" can be a dependent variable that is usually measured in a context-bound setting with the child's parent, we cannot conclude that individual interventions for pre-school children with ASD have significant effects on generalised skills for engaging in reciprocal interactions with others, even if the interventions have significant effects on the outcome. However, the outcomes of "reciprocity of social interaction towards others" may be promising targets for both individual and group interventions involving pre-school children with ASD. "Parental synchrony" may also be a promising target for individual interventions.

TRIAL REGISTRATION

(CRD42011001349).

Longitudinal Impact of a Randomized Clinical Trial to Improve Family Function, Reduce Maternal Stress and Improve Child Outcomes in Families of Children with ADHD

Objective Evaluate the efficacy of a 12 month nursing case-management intervention over a period of 18 months, 6 months after the end of intervention, for families of children attention deficit hyperactivity disorder (ADHD). Methods Mother and child dyads were enrolled to participate in a randomized controlled clinical trial. Children were 4-18 years old. Data were collected at baseline, 6, 12, and 18 months or 6 months after the termination of direct intervention. Longitudinal analyses, using generalized estimating equations, were conducted to assess change in study outcomes relating to family function, maternal stress, and child behavior over the 18 month period. Results Compared to control families, some family function outcomes were moderately improved in the intervention group. In particular, intervention families demonstrated substantial improvement in implementing family behavior controls (p value = 0.038) and improvement in family satisfaction (not statistically significant p = 0.062). Although there was improvement in the overall family function measure there was not a statistically significant difference between groups. Maternal stress and child behavior outcomes were not significantly different between control and intervention groups by the end of the intervention. Conclusions for Practice Addressing ADHD is complex and requires the assessment of comorbidities that might exacerbate negative behavior. Our findings support the latest American Academy of Pediatrics guidelines to use behavioral therapy as the first line of treatment in young children. Nursing case-management interventions that provide direct family education and improve family function, especially with respect to providing structure and behavior control, may complement and facilitate behavioral therapy for treatment of ADHD and improving child behavior.

On the Multilevel Nature of Meta-Analysis: A Tutorial, Comparison of Software Programs, and Discussion of Analytic Choices

The term "multilevel meta-analysis" is encountered not only in applied research studies, but in multilevel resources comparing traditional meta-analysis to multilevel meta-analysis. In this tutorial, we argue that the term "multilevel meta-analysis" is redundant since all meta-analysis can be formulated as a special kind of multilevel model. To clarify the multilevel nature of meta-analysis the four standard meta-analytic models are presented using multilevel equations and fit to an example data set using four software programs: two specific to meta-analysis (metafor in R and SPSS macros) and two specific to multilevel modeling (PROC MIXED in SAS and HLM). The same parameter estimates are obtained across programs underscoring that all meta-analyses are multilevel in nature. Despite the equivalent results, not all software programs are alike and differences are noted in the output provided and estimators available. This tutorial also recasts distinctions made in the literature between traditional and multilevel meta-analysis as differences between meta-analytic choices, not between meta-analytic models, and provides guidance to inform choices in estimators, significance tests, moderator analyses, and modeling sequence. The extent to which the software programs allow flexibility with respect to these decisions is noted, with metafor emerging as the most favorable program reviewed.

A systematic review and meta-analysis of comprehensive interventions for pre-school children with autism spectrum disorder (ASD)

BACKGROUND

There has an increasing number of published trials on psychosocial intervention programmes for pre-school children with autism spectrum disorder (ASD). To achieve better quality of unbiased evidence for the effectiveness of ASD interventions, it is necessary to conduct a comprehensive review that covers studies with adequate quality standards, such as randomised controlled trials (RCTs), and different types of intervention In this study, we categorize interventions for ASD as behavioural, social-communication focused, and multimodal developmental based on Howlin's classification of early interventions for children with ASD. The aim of this study was to compare these three models and investigate the strengths and weaknesses of each type of intervention and to identify the approaches that contribute to a successful outcome for children with autism.

METHODS

We performed a systematic review and meta-analysis. We included RCTs targeting children with ASD 6 years old or younger. A random effects model was used to present the effect estimate for the outcomes. This study also performed combined meta-analyses of all the three models to investigate the overall effectiveness of the intervention programmes.

RESULTS

32 randomized controlled studies were found to be eligible for inclusion. The synthesized data included 594 children from 14 RCTs. There was no statistically significant difference in the effects on autism general symptoms between the social-communication-focused model and the multimodal developmental model (p = 0.83). The results suggest that there is evidence of an effect on 'reciprocity of social interaction towards others' (standard mean difference [95% confidential interval] = 0.53[0.29,0.78], p<0.01) and 'parental synchrony' (SMD = 0.99[0.70,1.29], p<0.01).

CONCLUSION

The small number of studies included in the present study limited the ability to make inferences when comparing the three models and investigating the strengths and weaknesses of each type of intervention with respect to important outcomes. Since the outcome of 'reciprocity of social interaction towards others' could be a dependent variable that might be context-bound to interactions with the child's parent, we cannot conclude the interventions for pre-school children with ASD have significant effects on a generalized skill to engage in reciprocal interactions with others. However, the outcomes of 'reciprocity of social interaction towards others' and 'parental synchrony' may be promising targets for interventions involving pre-school children with ASD.

TRIAL REGISTRATION

Prospero CRD42011001349.

From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology

The gold standard for an empirical science is the replicability of its research results. But the estimated average replicability rate of key-effects that top-tier psychology journals report falls between 36 and 39% (objective vs. subjective rate; Open Science Collaboration, 2015). So the standard mode of applying null-hypothesis significance testing (NHST) fails to adequately separate stable from random effects. Therefore, NHST does not fully convince as a statistical inference strategy. We argue that the replicability crisis is "home-made" because more sophisticated strategies can deliver results the successful replication of which is sufficiently probable. Thus, we can overcome the replicability crisis by integrating empirical results into genuine research programs. Instead of continuing to narrowly evaluate only the stability of data against random fluctuations (discovery context), such programs evaluate rival hypotheses against stable data (justification context).

A Meta-Analysis on Antecedents and Outcomes of Detachment from Work

Detachment from work has been proposed as an important non-work experience helping employees to recover from work demands. This meta-analysis (86 publications, k = 91 independent study samples, N = 38,124 employees) examined core antecedents and outcomes of detachment in employee samples. With regard to outcomes, results indicated average positive correlations between detachment and self-reported mental (i.e., less exhaustion, higher life satisfaction, more well-being, better sleep) and physical (i.e., lower physical discomfort) health, state well-being (i.e., less fatigue, higher positive affect, more intensive state of recovery), and task performance (small to medium sized effects). However, average relationships between detachment and physiological stress indicators and work motivation were not significant while associations with contextual performance and creativity were significant, but negative. Concerning work characteristics, as expected, job demands were negatively related and job resources were positively related to detachment (small sized effects). Further, analyses revealed that person characteristics such as negative affectivity/neuroticism (small sized effect) and heavy work investment (medium sized effect) were negatively related to detachment whereas detachment and demographic variables (i.e., age and gender) were not related. Moreover, we found a medium sized average negative relationship between engagement in work-related activities during non-work time and detachment. For most of the examined relationships heterogeneity of effect sizes was moderate to high. We identified study design, samples' gender distribution, and affective valence of work-related thoughts as moderators for some of these aforementioned relationships. The results of this meta-analysis point to detachment as a non-work (recovery) experience that is influenced by work-related and personal characteristics which in turn is relevant for a range of employee outcomes.

Illness representations, coping, and illness outcomes in people with cancer: a systematic review and meta-analysis

OBJECTIVE

Cancer is associated with negative health and emotional outcomes in those affected by it, suggesting the need to better understand the psychosocial determinants of illness outcomes and coping. The common sense model is the leading psychological model of self-regulation in the face of illness and assumes that subjective illness representations explain how people attempt to cope with illness. This systematic review and meta-analysis examines the associations of the common sense model's illness representation dimensions with health and coping outcomes in people with cancer.

METHODS

A systematic literature search located 54 studies fulfilling the inclusion criteria, with 38 providing sufficient data for meta-analysis. A narrative review of the remaining studies was also conducted.

RESULTS

Random-effects models revealed small to moderate effect sizes (Fisher Z) for the relations between illness representations and coping behaviors (in particular between control perceptions, problem-focused coping, and cognitive reappraisal) and moderate to large effect sizes between illness representations and illness outcomes (in particular between identity, consequences, emotional representations, and psychological distress). The narrative review of studies with insufficient data provided similar results.

CONCLUSIONS

The results indicate how illness representations relate to illness outcomes in people with cancer. However, more high-quality studies are needed to examine causal effects of illness representations on coping and outcomes. High heterogeneity indicates potential moderators of the relationships between illness representations and health and coping outcomes, including diagnostic, prognostic, and treatment-related variables. This review can inform the design of interventions to improve coping strategies and mental health outcomes in people with cancer.

Can We Increase Psychological Well-Being? The Effects of Interventions on Psychological Well-Being: A Meta-Analysis of Randomized Controlled Trials

BACKGROUND

There is a rapidly growing interest in psychological well-being (PWB) as outcome of interventions. Ryff developed theory-based indicators of PWB that are consistent with a eudaimonic perspective of happiness. Numerous interventions have been developed with the aim to increase PWB. However, the effects on PWB measured as coherent outcome have not been examined across studies yet. This meta-analysis of randomized controlled trials of behavioral interventions aims to answer the question whether it is possible to enhance PWB.

METHODS

A systematic literature search was performed in PsycINFO, Cochrane and Web of Science. To be included, studies had to be randomized controlled trials of behavioral interventions with psychological well-being as primary or secondary outcome measure, measured with either Ryff's Psychological Well-Being Scales or the Mental Health Continuum-Short Form. The meta-analysis was performed using a random effects model. From the 2,298 articles found, 27 met the inclusion criteria. The included studies involved 3,579 participants.

RESULTS

We found a moderate effect (Cohen's d = 0.44; z = 5.62; p < .001). Heterogeneity between the studies was large (Q (26) = 134.12; p < .001; I2 = 80.62). At follow-up after two to ten months, a small but still significant effect size of 0.22 was found. There was no clear indication of publication bias. Interventions were more effective in clinical groups and when they were delivered individually. Effects were larger in studies of lower quality.

CONCLUSIONS

It appears to be possible to improve PWB with behavioral interventions. The results are promising for the further development and implementation of interventions to promote PWB. Delivering interventions face-to-face seems to be the most promising option. We recommend to keep including clinical groups in the research of psychological well-being. Heterogeneity is a limitation of the study and there is need for more high-quality studies.

Effectiveness of Cognitive Behavioural Self-Help for the Treatment of Depression and Anxiety in People with Long-Term Physical Health Conditions: a Systematic Review and Meta-Analysis of Randomised Controlled Trials

BACKGROUND

Depression and anxiety are prevalent comorbidities in people with long-term physical health conditions; however, there is limited access to evidence-based treatments for comorbid mental health difficulties.

PURPOSE

This study is a meta-analysis examining the effectiveness of cognitive behavioural self-help for physical symptoms, depression and anxiety in people with long-term conditions.

METHODS

This study involves a systematic search of electronic databases supplemented by expert contact, reference and citation checking and grey literature.

RESULTS

The meta-analysis yielded a small effect size for 11 studies reporting primary outcomes of depression (g = -0.20) and 8 studies anxiety (g = -0.21) with a large effect size (g = -1.14) for 1 study examining physical health symptoms. There were no significant moderators of the main effect.

CONCLUSIONS

Limited evidence supports cognitive behavioural self-help for depression, anxiety and physical symptoms in people with long-term conditions. Small effect sizes for depression and anxiety may result from failure to recruit participants with clinical levels of these difficulties at baseline.

The Influence of Judgment Calls on Meta-Analytic Findings

Previous research has suggested that judgment calls (i.e., methodological choices made in the process of conducting a meta-analysis) have a strong influence on meta-analytic findings and question their robustness. However, prior research applies case study comparison or reanalysis of a few meta-analyses with a focus on a few selected judgment calls. These studies neglect the fact that different judgment calls are related to each other and simultaneously influence the outcomes of a meta-analysis, and that meta-analytic findings can vary due to non-judgment call differences between meta-analyses (e.g., variations of effects over time). The current study analyzes the influence of 13 judgment calls in 176 meta-analyses in marketing research by applying a multivariate, multilevel meta-meta-analysis. The analysis considers simultaneous influences from different judgment calls on meta-analytic effect sizes and controls for alternative explanations based on non-judgment call differences between meta-analyses. The findings suggest that judgment calls have only a minor influence on meta-analytic findings, whereas non-judgment call differences between meta-analyses are more likely to explain differences in meta-analytic findings. The findings support the robustness of meta-analytic results and conclusions.

Metaanalyse–praktische Schritte und Entscheidungen im Umsetzungsprozess

In diesem Beitrag wird der Durchführungsprozess metaanalytischer Techniken nach Hunter und Schmidt (2004) Schritt für Schritt beschrieben. In Form eines Tutoriums geben wir evidenzbasierte Empfehlungen, verweisen auf relevante Quellen und Hilfsmittel und bewerten alternative Vorgehensweisen in Bezug auf ihre Güte und Akzeptanz. Wir wenden uns an Forschende, die eine Metaanalyse mit der Besonderheit der Artefaktkorrekturen nach Hunter und Schmidt durchführen wollen und geben Anregungen für die Verbreitung metaanalytischer Ergebnisse sowohl im Wissenschaftskontext als auch im Austausch mit Praktikern.

The use of meta-analytic statistical significance testing

Meta-analysis multiplicity, the concept of conducting multiple tests of statistical significance within one review, is an underdeveloped literature. We address this issue by considering how Type I errors can impact meta-analytic results, suggest how statistical power may be affected through the use of multiplicity corrections, and propose how meta-analysts should analyze multiple tests of statistical significance. The context for this study is a meta-review of meta-analyses published in two leading review journals in education and psychology. Our review of 130 meta-analyses revealed a strong reliance on statistical significance testing without consideration of Type I errors or the use of multiplicity corrections. In order to provide valid conclusions, meta-analysts must consider these issues prior to conducting the review.

A Meta-View of Multivariate Statistical Inference Methods in European Psychology Journals

We investigated the extent and nature of multivariate statistical inferential procedures used in eight European psychology journals covering a range of content (i.e., clinical, social, health, personality, organizational, developmental, educational, and cognitive). Multivariate methods included those found in popular texts that focused on prediction, group difference, and advanced modeling: multiple regression, logistic regression, analysis of covariance, multivariate analysis of variance, factor or principal component analysis, structural equation modeling, multilevel modeling, and other methods. Results revealed that an average of 57% of the articles from these eight journals involved multivariate analyses with a third using multiple regression, 17% using structural modeling, and the remaining methods collectively comprising about 50% of the analyses. The most frequently occurring inferential procedures involved prediction weights, dichotomous p values, figures with data, and significance tests with very few articles involving confidence intervals, statistical mediation, longitudinal analyses, power analysis, or meta-analysis. Contributions, limitations and future directions are discussed.