Current practices in meta-regression in psychology, education, and medicine

Protocol: Machine learning for selecting moderators in meta-analysis: A systematic review of methods and their applications, and an evaluation using data on tutoring interventions

Objectives This is the protocol for a Campbell systematic review. The objectives are as follows: The first objective is to find and describe machine and statistical learning (ML) methods designed for moderator meta-analysis. The second objective is to find and describe applications of such ML methods in moderator meta-analyses of health, medical, and social science interventions. These two parts of the meta-review will primarily involve a systematic review and will be conducted according to guidelines specified by the Campbell Collaboration (MECCIR guidelines). The outcomes will be a list of ML methods that are designed for moderator meta-analysis (first objective), and a description of how (some of) these methods have been applied in the health, medical, and social sciences (second objective). The third objective is to examine how the ML methods identified in the meta-review can help researchers formulate new hypotheses or select among existing ones, and compare the identified methods to one another and to regular meta-regression methods for moderator analysis. To compare the performance of different moderator meta-analysis methods, we will apply the methods to data on tutoring interventions from two systematic reviews of interventions to improve academic achievement for students with or at risk-of academic difficulties, and to an independent test sample of tutoring studies published after the search period in the two reviews.

Conducting power analysis for meta-analysis with dependent effect sizes: Common guidelines and an introduction to the POMADE R package

Sample size and statistical power are important factors to consider when planning a research synthesis. Power analysis methods have been developed for fixed effect or random effects models, but until recently these methods were limited to simple data structures with a single, independent effect per study. Recent work has provided power approximation formulas for meta-analyses involving studies with multiple, dependent effect size estimates, which are common in syntheses of social science research. Prior work focused on developing and validating the approximations but did not address the practice challenges encountered in applying them for purposes of planning a synthesis involving dependent effect sizes. We aim to facilitate the application of these recent developments by providing practical guidance on how to conduct power analysis for planning a meta-analysis of dependent effect sizes and by introducing a new R package, POMADE, designed for this purpose. We present a comprehensive overview of resources for finding information about the study design features and model parameters needed to conduct power analysis, along with detailed worked examples using the POMADE package. For presenting power analysis findings, we emphasize graphical tools that can depict power under a range of plausible assumptions and introduce a novel plot, the traffic light power plot, for conveying the degree of certainty in one's assumptions.

Surgical treatment of cystic pituitary adenomas: literature-based definitions and postoperative outcomes

BACKGROUND AND OBJECTIVES

To survey the applied definitions of 'cystic' among pituitary adenomas and evaluate whether postoperative outcomes differ relative to non-cystic counterparts.

METHODS

A literature search and meta-analysis was performed using PRISMA guidelines. Studies were eligible if novel data were reported regarding the applied definition of 'cystic' and postoperative outcomes among cases of surgically treated pituitary adenomas. Data were pooled with random effects meta-analysis models into cohorts based on the applied definition of 'cystic'. Categorical meta-regressions were used to investigate differences between cohorts. Among studies comparing cystic and non-cystic pituitary adenomas, meta-analysis models were applied to determine the Odds Ratio [95% Confidence Interval]. Statistical analyses were performed using Comprehensive Meta-Analysis (CMA, 4.0), with a priori significance defined as P < 0.05.

RESULTS

Ten studies were eligible yielding 283 patients with cystic pituitary adenomas. The definitions of 'cystic' mainly varied between the visual appearance of cystic components on preoperative magnetic resonance imaging and a volumetric definition requiring 50% or greater of tumor volume exhibiting cystic components. Tumor diameter was seldom reported with an associated standard deviation/error, limiting meta-analyses. When the data were pooled in accordance with the definition applied, there were no significant differences in the rates of gross total resection (P = 0.830), endocrinologic remission (P = 0.563), and tumor recurrence (P = 0.320). Meta-analyses on studies comparing cystic versus non-cystic pituitary adenomas indicated no significant difference in the rates of gross total resection (P = 0.729), endocrinologic remission (P = 0.857), and tumor recurrence (P = 0.465).

CONCLUSION

Despite some individual studies describing a significant influence of pituitary adenoma texture on postoperative outcomes, meta-analyses revealed no such differences between cystic and non-cystic pituitary adenomas. This discrepancy may be explained in part by the inconsistent definition of 'cystic' and between-group differences in tumor size. A notion of a field-standard definition of 'cystic' among pituitary adenomas should be established to facilitate inter-study comparisons.

Reading Interventions for Students in Grades 3-12 With Significant Word Reading Difficulties

Most students with reading difficulties struggle to read words. We examined intervention effects for students with significant word reading difficulties (SWRD; standard score of 80 on at least one pretest measure of word reading), which includes individuals with or at risk for dyslexia. We investigated: (a) What are the effects of reading interventions for students in Grades 3-12 with SWRD? and (b) What intervention features (i.e., instructional components and elements of dosage) are related to improved reading outcomes for the target population? A meta-analysis of 22 studies and 208 effect sizes revealed a statistically significant, positive, mean effect (g = 0.14, standard error [SE] = 0.04, p = .01, 95% confidence interval [CI]: [0.04, 0.23]) of interventions for the target population. Subset analyses revealed positive, statistically significant intervention effects on measures of pseudoword reading (g = 0.38, SE = 0.07, p = .0003, 95% CI [0.21, 0.54]) and pseudoword reading fluency (g = 0.29, SE = 0.09, p = .010, 95% CI [0.09, 0.49]). Moderator analyses yielded statistically significant, positive effects associated with increased total hours of intervention, β = 0.003, SE = 0.0009, t(8.31) = 3.58, p = .007. Overall, findings indicate a need for interventions that improve generalized real-world reading for the target population.

Agreement Between Maximal Lactate Steady State and Critical Power in Different Sports: A Systematic Review and Bayesian's Meta-Regression

ABSTRACT

Borszcz, FK, de Aguiar, RA, Costa, VP, Denadai, BS, and de Lucas, RD. Agreement between maximal lactate steady state and critical power in different sports: A systematic review and Bayesian's meta-regression. J Strength Cond Res 38(6): e320-e339, 2024-This study aimed to systematically review the literature and perform a meta-regression to determine the level of agreement between maximal lactate steady state (MLSS) and critical power (CP). Considered eligible to include were peer-reviewed and "gray literature" studies in English, Spanish, and Portuguese languages in cyclical exercises. The last search was made on March 24, 2022, on PubMed, ScienceDirect, SciELO, and Google Scholar. The study's quality was evaluated using 4 criteria adapted from the COSMIN tool. The level of agreement was examined by 2 separate meta-regressions modeled under Bayesian's methods, the first for the mean differences and the second for the SD of differences. The searches yielded 455 studies, of which 36 studies were included. Quality scale revealed detailed methods and small samples used and that some studies lacked inclusion/exclusion criteria reporting. For MLSS and CP comparison, likely (i.e., coefficients with high probabilities) covariates that change the mean difference were the MLSS time frame and delta criteria of blood lactate concentration, MLSS number and duration of pauses, CP longest predictive trial duration, CP type of predictive trials, CP model fitting parameters, and exercise modality. Covariates for SD of the differences were the subject's maximal oxygen uptake, CP's longest predictive trial duration, and exercise modality. Traditional MLSS protocol and CP from 2- to 15-minute trials do not reflect equivalent exercise intensity levels; the proximity between MLSS and CP measures can differ depending on test design, and both MLSS and CP have inherent limitations. Therefore, comparisons between them should always consider these aspects.

The effects of aftercare/resettlement services on crime and violence in children and youth: A systematic review

Background

High rates of youth re-offending indicate that young custody-leavers face challenges when reintegrating into their communities. Aftercare and resettlement programs can occur pre-, during, and post-release and generally provide multiple forms of support services to address youths' transitional needs.

Objectives

The present review examines (1) the impact of youth aftercare/resettlement programs on crime-related outcomes, (2) how treatment effect is moderated by participant, program, and study characteristics, (3) whether some types of interventions are more effective than others, (4) barriers/facilitators to effective program implementation, (5) the theory of change underlying resettlement interventions, and (6) available research on intervention cost.

Search Methods

A comprehensive set of keywords and synonyms was combined in a Boolean search across 26 electronic databases. Multiple gray literature sources were also searched, including 23 journals, 4 meeting archives, 11 organization websites, 3 open access journal websites, and the CVs of 8 well-known researchers in the field. The search was completed in January 2023.

Selection Criteria

For objectives 1-3, studies were included if they utilized a randomized controlled design or quasi-experimental comparison group design in which participants were matched on at least some baseline variables and included at least one quantitative individual-measure of crime. For objective 4, included studies presented process evaluations of aftercare/reentry programs, clearly stated their research goals, and used qualitative methods in an appropriate way to answer the stated research question. For objectives 5 and 6, no specific methods were required; any study meeting the criteria for objectives 1-4 which presented findings on theory of change or cost data were included. For all outcomes, only studies conducted in a westernized country, and published after 1991 in English, French, or German were considered.

Data Collection and Analysis

Two coders conducted primary data extraction for the included studies. Data were entered into a Microsoft Excel database. After data extraction, the two coders validated the coding by cross-checking the database with each research report. Discrepancies between coders were discussed until consensus was reached. Where consensus could not be reached, a third coder was consulted. Study risk of bias was addressed using the ROBINS-I (Sterne et al., 2016), ROB-2 (Higgins et al., 2019), and the critical appraisal skills programme (CASP, 2018). Objectives 1-3 were addressed by synthesizing quantitative outcomes from rigorous impact evaluations of aftercare interventions using random effects models and meta-regression. Thematic and narrative analysis was conducted to address objectives 4-6.

Results

The search resulted in 15 impact studies, representing 4,718 participants across 21 program sites, and 35 effect sizes. The 21 impact evaluations were rated as having either low/moderate bias (k = 11) or serious bias (k = 10). The synthesis of 15 impact studies found no significant effects for arrest (k = 14; OR = 1.044, 95% prediction interval [0.527, 2.075], t = 0.335) or incarceration (k = 8, OR = 0.806, 95% prediction interval [2.203, 1.433], t = -1.674). A significant pooled effect was found for conviction (k = 13, OR = 1.209, 95% prediction interval [1.000, 1.462], t = 2.256), but results were highly sensitive to the inclusion of specific studies. No meaningful pattern of results emerged in moderator analyses with respect to study, sample, program component, or program delivery characteristics. The 19 process studies were rated as either high quality (k = 12) or moderate quality (k = 7). Thematic synthesis of the process evaluations revealed 15 themes related to the strengths/challenges of program implementation. The assessment of program cost (k = 7) determined a lack of data within the literature, preventing any summative analysis.

Authors' Conclusions

Current evidence is promising with respect to conviction outcomes but overall does not find that aftercare/resettlement interventions have a reliably positive impact on crime-related outcomes for young people who have offended. High variability across outcomes and reported data resulted in small sample sizes per outcome and limited moderator analyses. Multiple challenges for program implementation exist; additional rigorous research is sorely needed to further investigate the nuances of the program effects.

The prevalence and risk of depression in aged COVID-19 survivors: a bibliometric and meta-analysis

To explore depression prevalence and related risk factors among elderly coronavirus disease 2019 (COVID-19) survivors, while also evaluating research characteristics. We searched Web of Science, PubMed, Embase, Scopus, CNKI and Wanfang Data for studies that reported COVID-19 and depression in older adults. 'Bibliometrix' facilitated bibliometric analysis and information visualisation. Random-effects models merged depression prevalence and relevant risks. Publication bias and its impact were examined using funnel plots, Begg's test, Egger's linear regression, and trim-and-fill method. Meta-regression, bubble plots, and Baujat plots probed heterogeneity. Sensitivity analysis applied the leave-one-out method. The study is registered with PROSPERO, CRD42023417706. The bibliometric analysis comprised 138 studies. Publication frequency peaked in the US, China, and Italy, reflecting significant growth. The meta-analysis comprised 43 studies. Elderly COVID-19 patients exhibit 28.33% depression prevalence (95% CI: 21.24-35.97). Severe cases (43.91%, 95% CI: 32.28-55.88) experienced higher depression prevalence than mild cases (16.45%, 95% CI: 11.92-21.50). Sex had no depression prevalence impact based on bubble plots. Notably, depression risk did not significantly differ between elderly and young COVID-19 patients (odds ratio (OR) = 1.1808, 95% CI: 0.7323-1.9038). However, COVID-19 infection emerged as a substantial elderly depression risk factor (OR = 1.8521, 95% CI: 1.2877-2.6639). Sensitivity analysis confirmed result robustness. Elderly COVID-19 survivors are likely to develop depression symptoms with regional variations. Severe cases are associated with heightened depression prevalence. COVID-19 infection stands out as a key elderly depression risk factor, while sex does not influence prevalence. The field's expansion necessitates sustained collaboration and extensive research endeavours.

Inferior Petrosal Sinus Sampling Tumor Lateralization and the Surgical Treatment of Cushing Disease: A Meta-Analysis and Systematic Review

OBJECTIVE

To determine whether accurate inferior petrosal sinus sampling (IPSS) tumor lateralization is associated with improved clinical outcomes following the surgical treatment of Cushing disease.

METHODS

The presented study was performed in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. Data regarding patient demographics, IPSS tumor lateralization, and postoperative endocrinologic outcomes were abstracted and pooled with random effects meta-analysis models. Additional meta-regression models were used to examine the association between the accuracy of IPSS tumor lateralization and postoperative outcomes (recurrence/persistence or remission/cure). Statistical analyses were performed using the Comprehensive Meta-Analysis software (significance of P < 0.05).

RESULTS

Seventeen eligible articles were identified, yielding data on 461 patients. Within average follow-up duration (∼59 months), the rate of correct IPSS tumor lateralization was 69% [95% confidence interval: 61%, 76%], and the rate of postoperative remission/cure was 78% [67%, 86%]. Preoperative IPSS tumor lateralization was concordant with magnetic resonance imaging lateralization for 53% of patients [40%, 66%]. There was no significant association between the rate of correct IPSS tumor lateralization and postoperative remission/cure among study-level data (P = 0.735). Additionally, there was no association among subgroup analyses for studies using stimulatory agents during IPSS (corticotropin-releasing hormone or desmopressin, P = 0.635), nor among subgroup analyses for adult (P = 0.363) and pediatric (P = 0.931) patients.

CONCLUSIONS

Limited data suggest that the rate of correct IPSS tumor lateralization may not be positively associated with postoperative remission or cure in patients with Cushing disease. These findings bring into question the utility of IPSS tumor lateralization in the context of preoperative planning and surgical approach rather than confirming a pituitary source.

Time-Perception Deficits in Attention-Deficit/Hyperactivity Disorder: A Systematic Review and Meta-Analysis

The present meta-analysis quantified the deficit in time perception in Attention-Deficit/Hyperactivity Disorder (ADHD) throughout the lifespan and examined potential moderators of this deficit. Our sample of 824 effect sizes showed a mean g of 0.688 that was moderated by the age of the sample and working memory. Separate moderator analyses for samples below or above the age of 18 showed that the link with working memory only applied to the samples below the age of 18, whereas an effect of ADHD subtype only applied to samples 18 and above. The discussion highlights the implications for remediation and avenues for future research.

Accuracy and precision of fixed and random effects in meta-analyses of randomized control trials for continuous outcomes

Meta-analyses of treatment effects in randomized control trials are often faced with the problem of missing information required to calculate effect sizes and their sampling variances. Particularly, correlations between pre- and posttest scores are frequently not available. As an ad-hoc solution, researchers impute a constant value for the missing correlation. As an alternative, we propose adopting a multivariate meta-regression approach that models independent group effect sizes and accounts for the dependency structure using robust variance estimation or three-level modeling. A comprehensive simulation study mimicking realistic conditions of meta-analyses in clinical and educational psychology suggested that imputing a fixed correlation 0.8 or adopting a multivariate meta-regression with robust variance estimation work well for estimating the pooled effect but lead to slightly distorted between-study heterogeneity estimates. In contrast, three-level meta-regressions resulted in largely unbiased fixed effects but more inconsistent prediction intervals. Based on these results recommendations for meta-analytic practice and future meta-analytic developments are provided.

Cross-national analyses require additional controls to account for the non-independence of nations

Cross-national analyses test hypotheses about the drivers of variation in national outcomes. However, since nations are connected in various ways, such as via spatial proximity and shared cultural ancestry, cross-national analyses often violate assumptions of non-independence, inflating false positive rates. Here, we show that, despite being recognised as an important statistical pitfall for over 200 years, cross-national research in economics and psychology still does not sufficiently account for non-independence. In a review of the 100 highest-cited cross-national studies of economic development and values, we find that controls for non-independence are rare. When studies do control for non-independence, our simulations suggest that most commonly used methods are insufficient for reducing false positives in non-independent data. In reanalyses of twelve previous cross-national correlations, half of the estimates are compatible with no association after controlling for non-independence using global proximity matrices. We urge social scientists to sufficiently control for non-independence in cross-national research.

Multilevel multivariate meta-analysis made easy: An introduction to MLMVmeta

The basic random effects meta-analytic model is overwhelmingly dominant in psychological research. Indeed, it is typically employed even when more complex multilevel multivariate meta-analytic models are warranted. In this paper, we aim to help overcome challenges so that multilevel multivariate meta-analytic models will be more often employed in practice. We do so by introducing MLMVmeta-an easy-to-use web application that implements multilevel multivariate meta-analytic methodology that is both specially tailored to contemporary psychological research and easily estimable, interpretable, and parsimonious-and illustrating it across three case studies. The three case studies demonstrate the more accurate and extensive results that can be obtained via multilevel multivariate meta-analytic models. Further, they sequentially build in complexity featuring increasing numbers of experimental factors and conditions, dependent variables, and levels; this in turn necessitates increasingly complex model specifications that also sequentially build upon one another.

Comparing Surgical Approaches for Craniopharyngioma Resection Among Adults and Children: A Meta-analysis and Systematic Review

BACKGROUND

Craniopharyngioma treatment often necessitates surgical resection. Conventional approaches, such as transcranial and transsphenoidal approaches, are predominantly used. However, a recently adopted supraorbital approach may be more efficacious. Thus, this meta-analysis and systematic review aimed to compare the efficacy of the transcranial, transsphenoidal, and supraorbital approaches for resection and treatment of craniopharyngiomas.

METHODS

This study was performed following PRISMA guidelines. Analyses were performed according to study design: (1) analyses for studies comparing several surgical approaches; and (2) analyses of all included studies. Random effects meta-analysis models were used to pool odds ratios among studies comparing several approaches. Similarly, categorical meta-regression models were used to examine the effect of surgical approach as a covariate of outcome data for all studies. Statistics were performed using Comprehensive Meta-Analysis software (CMA 3.3, Biostat, Englewood, NJ) (significance set at P < 0.05).

RESULTS

Patients were well-matched for age, sex, and preoperative comorbidities between groups stratified by surgical approach. Analyses including 22 studies that compared several approaches demonstrated that the transsphenoidal group had a greater occurrence of postoperative visual improvement (P < 0.0001), lesser occurrence of visual deterioration (P < 0.0001), and lower tumor recurrence rate (P = 0.015) compared with the transcranial group. Only 2 studies compared the supraorbital approach to another approach, limiting analyses. Analyses including all studies demonstrated that the supraorbital group did not differ to either the transcranial or transsphenoidal group for any examined variables.

CONCLUSIONS

The present study suggests that the transsphenoidal approach is associated with improved clinical outcomes for craniopharyngioma resection. There are limited data regarding the supraorbital approach, warranting future investigation.

Heterogeneous heterogeneity by default: Testing categorical moderators in mixed-effects meta-analysis

Categorical moderators are often included in mixed-effects meta-analysis to explain heterogeneity in effect sizes. An assumption in tests of categorical moderator effects is that of a constant between-study variance across all levels of the moderator. Although it rarely receives serious thought, there can be statistical ramifications to upholding this assumption. We propose that researchers should instead default to assuming unequal between-study variances when analysing categorical moderators. To achieve this, we suggest using a mixed-effects location-scale model (MELSM) to allow group-specific estimates for the between-study variance. In two extensive simulation studies, we show that in terms of Type I error and statistical power, little is lost by using the MELSM for moderator tests, but there can be serious costs when an equal variance mixed-effects model (MEM) is used. Most notably, in scenarios with balanced sample sizes or equal between-study variance, the Type I error and power rates are nearly identical between the MEM and the MELSM. On the other hand, with imbalanced sample sizes and unequal variances, the Type I error rate under the MEM can be grossly inflated or overly conservative, whereas the MELSM does comparatively well in controlling the Type I error across the majority of cases. A notable exception where the MELSM did not clearly outperform the MEM was in the case of few studies (e.g., 5). With respect to power, the MELSM had similar or higher power than the MEM in conditions where the latter produced non-inflated Type 1 error rates. Together, our results support the idea that assuming unequal between-study variances is preferred as a default strategy when testing categorical moderators.

A meta-analysis of internet-based cognitive behavioral therapy for military and veteran populations

BACKGROUND

Military and veteran populations are unique in their trauma exposures, rates of mental illness and comorbidities, and response to treatments. While reviews have suggested that internet-based Cognitive Behavioral Therapy (iCBT) can be useful for treating mental health conditions, the extent to which they may be appropriate for military and veteran populations remain unclear. The goals of the current meta-analysis are to: (1) substantiate the effects of iCBT for military and veteran populations, (2) evaluate its effectiveness compared to control conditions, and (3) examine potential factors that may influence their effectiveness.

METHODS

This review was completed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting and Cochrane review guidelines. The literature search was conducted using PsycInfo, Medline, Embase, and Proquest Dissertation & Theses on June 4, 2021 with no date restriction. Inclusion criteria included studies that: (1) were restricted to adult military or veteran populations, (2) incorporated iCBT as the primary treatment, and (3) evaluated mental health outcomes. Exclusion criteria included: (1) literature reviews, (2) qualitative studies, (3) study protocols, (4) studies that did not include a clinical/analogue population, and (5) studies with no measure of change on outcome variables. Two independent screeners reviewed studies for eligibility. Data was pooled and analyzed using random-effects and mixed-effects models. Study data information were extracted as the main outcomes, including study condition, sample size, and pre- and post-treatment means, standard deviations for all assessed outcomes, and target outcome. Predictor information were also extracted, and included demographics information, the types of outcomes measured, concurrent treatment, dropout rate, format, length, and delivery of intervention.

RESULTS

A total of 20 studies and 91 samples of data were included in the meta-analysis. The pooled effect size showed a small but meaningful effect for iCBT, g = 0.54, SE = 0.04, 95% CI (0.45, 0.62), Z = 12.32, p < .001. These effects were heterogenous across samples, (I2 = 87.96), Q(90) = 747.62, p < .001. Predictor analyses found length of intervention and concurrent treatment to influence study variance within sampled studies, p < .05. Evaluation of iCBT on primary outcomes indicated a small but meaningful effect for PTSD and depression, while effects of iCBT on secondary outcomes found similar results with depression, p < .001.

CONCLUSIONS

Findings from the meta-analysis lend support for the use of iCBT with military and veteran populations. Conditions under which iCBT may be optimized are discussed.

Meta-analyzing individual participant data from studies with complex survey designs: A tutorial on using the two-stage approach for data from educational large-scale assessments

Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends. Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles. All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies.

Comparing methods for handling missing covariates in meta-regression

Meta-analysts often encounter missing covariate values when estimating meta-regression models. In practice, ad hoc approaches involving data deletion have been widely used. The current study investigates the performance of different methods for handling missing covariates in meta-regression, including complete-case analysis (CCA), shifting-case analysis (SCA), multiple imputation (MI), and full information maximum likelihood (FIML), assuming missing at random mechanism. According to the simulation results, we advocate the use of MI and FIML than CCA and SCA approaches in practice. In addition, we cautiously note the challenges and potential advantages of using MI in the meta-analysis context.

The correlation of leukocyte-specific protein 1 (LSP1) rs3817198(T>C) polymorphism with breast cancer: A meta-analysis

BACKGROUND

Multiple studies have investigated the correlation of single nucleotide polymorphisms (SNPs) in leukocyte-specific protein 1 (LSP1) with susceptibility to breast cancer (BC) and have yielded inconsistent conclusions, particularly rs3817198(T > C). Consequently, we performed a meta-analysis to estimate this relationship more comprehensively.

METHODS

Four databases were utilized to locate eligible publications: PubMed, Embase, Web of Science, and China National Knowledge Infrastructure. This meta-analysis included 14 studies, including 22 reports of 33194 cases and 36661 controls. The relationship of rs3817198 polymorphism with breast cancer was estimated using odds ratios (ORs) with 95% confidence intervals (CIs). The LSP1 co-expression network was constructed by STRING, and Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) enrichment analyses were performed using DAVIDE. Download TCGA breast cancer mRNA-seq data and analyze the relationship between LSP1 expression and breast cancer chemotherapy sensitivity.

RESULTS

The results indicated that rs3817198(T > C) was positively correlated to with breast malignancy (dominant model: OR = 1.11, 95%CI = 1.06-1.17; recessive model: OR = 1.10, 95%CI = 1.04-1.15; heterozygous model: OR = 1.09, 95%CI = 1.04-1.15; homozygous model: OR = 1.18, 95%CI = 1.09-1.28; additive model: OR = 1.09, 95%CI = 1.05-1.13), among Caucasians and Asians. However, rs3817198(T > C) may reduce the risk of breast carcinoma in Africans. Rs3817198(T > C) might result in breast carcinoma in individuals with BRCA1 and BRCA2 variants and can contribute to estrogen receptor (ER)-positive breast carcinoma. The expression of LSP1 was inversely correlated with the IC50 of doxorubicin (P = 8.91e-15, Cor = -0.23), 5-fluorouracil (P = 1.18e-22, Cor = -0.29), and cisplatin (P = 1.35e-42, Cor = -0.40).

CONCLUSION

Our study identified that LSP1 rs3817198 polymorphism might result in breast malignancy, particularly among Caucasians and Asians, but lower breast cancer susceptibility in African populations. The expression of LSP1 was negatively correlated with the IC50 of doxorubicin, 5-fluorouracil, and cisplatin.

Location-scale models for meta-analysis

Heterogeneity is commonplace in meta-analysis. When heterogeneity is found, researchers often aim to identify predictors that account for at least part of such heterogeneity by using mixed-effects meta-regression models. Another potentially relevant goal is to focus on the amount of heterogeneity as a function of one or more predictors, but this cannot be examined with standard random- and mixed-effects models, which assume a constant (i.e., homoscedastic) value for the heterogeneity variance component across studies. In this paper, we describe a location-scale model for meta-analysis as an extension of the standard random- and mixed-effects models that not only allows an examination of whether predictors are related to the size of the outcomes (i.e., their location), but also the amount of heterogeneity (i.e., their scale). We present estimation methods for such a location-scale model through maximum and restricted maximum likelihood approaches, as well as methods for inference and suggestions for visualization. We also provide an implementation via the metafor package for R that makes this model readily available to researchers. Location-scale models can provide a useful tool to researchers interested in heterogeneity in meta-analysis, with the potential to enhance the scope of research questions in the field of evidence synthesis.

Systematic Review and Meta-Analysis of the Effectiveness of Teacher Delivered Interventions for Externalizing Behaviors

This systematic review and meta-analysis explores the effectiveness of teacher interventions supporting children with externalizing behaviors based on teacher and child outcomes. A systematic search was conducted using 5 electronic databases. From 5714 papers, 31 papers that included interventions delivered directly to teachers and aimed to benefit either teachers and/or children with externalizing behaviors were included. The review focused on qualified teachers working with children aged 2-13. The results of the current meta-analysis revealed a positive effect of teacher intervention on teacher and child outcomes, including the increased use of teacher-appropriate strategies, as well as significant and moderate improvements in teacher-child closeness, and small reductions in teacher-child conflict. For child outcomes, the interventions reduced externalizing behavior problems and ADHD symptoms and enhanced prosocial behavior. Only one fully blinded analysis for conduct problems was possible and revealed a moderate but significant reduction in favor of intervention. These findings provide evidence to support the role of teacher interventions for both teachers and children with externalizing behaviors. Future research should include more PBLIND measurements so that MPROX findings can be confirmed. More research should be done to evaluate the influence of teacher interventions on teachers' well-being.

Impulsivity, aggression, and suicidality relationship in adults: A systematic review and meta-analysis

BACKGROUND

The aim of the study was to crystallize the nature of relationships between impulsivity, aggression, and suicidality. We reviewed studies of adults with published, psychometric measures of impulsivity and aggression, and measures of suicidality.

METHODS

Our primary data source was Web of Science (from inception to 31st December 2021). Quality of articles was assessed using a Joanna Briggs Appraisal Tool and publication bias using Trim and Fill. We synthesised results using random effects meta-analyses and explored moderation by measure of impulsivity, aggression, and suicidality, and population.

FINDINGS

77 studies were included in our analysis. We found weak positive relationships between impulsivity (r = 0·19 [95% CI: 0·15-0·22]), aggression (0·23 ([0·17, 0·29]), and impulsive aggression (0·16 [0·1, 0·22]) with suicidality. Heterogeneity was significant and only partially explained by moderators. Limitations included the lack of studies which assess impulsivity or aggression proximal to suicidality.

INTERPRETATION

Given small effect sizes and significant heterogeneity, the study suggests that additional studies are needed in the field to analyze the relation between impulsivity and aggression with suicidality.

FUNDING

None.

ClusterWild Bootstrapping to Handle Dependent Effect Sizes in Meta-Analysis with a Small Number of Studies

The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiplecontrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small-sample correction methods, particularly for multiplecontrast hypothesis tests.We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests. This article is protected by copyright. All rights reserved.

The relationship between immune and cognitive dysfunction in mood and psychotic disorder: a systematic review and a meta-analysis

BACKGROUND

In psychotic and mood disorders, immune alterations are hypothesized to underlie cognitive symptoms, as they have been associated with elevated blood levels of inflammatory cytokines, kynurenine metabolites, and markers of microglial activation. The current meta-analysis synthesizes all available clinical evidence on the associations between immunomarkers (IMs) and cognition in these psychiatric illnesses.

METHODS

Pubmed, Web of Science, and Psycinfo were searched for peer-reviewed studies on schizophrenia spectrum disorder (SZ), bipolar disorder (BD), or major depressive disorder (MDD) including an association analysis between at least one baseline neuropsychological outcome measure (NP) and one IM (PROSPERO ID:CRD42021278371). Quality assessment was performed using BIOCROSS. Correlation meta-analyses, and random effect models, were conducted in Comprehensive Meta-Analysis version 3 investigating the association between eight cognitive domains and pro-inflammatory and anti-inflammatory indices (PII and AII) as well as individual IM.

RESULTS

Seventy-five studies (n = 29,104) revealed global cognitive performance (GCP) to be very weakly associated to PII (r = -0.076; p = 0.003; I² = 77.4) or AII (r = 0.067; p = 0.334; I² = 38.0) in the combined patient sample. Very weak associations between blood-based immune markers and global or domain-specific GCP were found, either combined or stratified by diagnostic subgroup (GCP x PII: SZ: r = -0.036, p = 0.370, I² = 70.4; BD: r = -0.095, p = 0.013, I² = 44.0; MDD: r = -0.133, p = 0.040, I² = 83.5). We found evidence of publication bias.

DISCUSSION

There is evidence of only a weak association between blood-based immune markers and cognition in mood and psychotic disorders. Significant publication and reporting biases were observed and most likely underlie the inflation of such associations in individual studies.

Facilitating Open Science Practices for Research Syntheses: PreregRS Guides Preregistration

Researchers increasingly engage in adopting open science practices in the field of research syntheses, such as preregistration. Preregistration is a central open science practice in empirical research to enhance transparency in the research process and it gains steady adoption in the context of conducting research synthesis. From an interdisciplinary perspective, frameworks and particularly templates are lacking which support researchers preparing a preregistration. To this end, we introduce preregRS, a template to guide researchers across disciplines through the process of preregistering research syntheses. We utilized an R Markdown template file to provide a framework that structures the process of preparing a preregistration. Researchers can write up the preregistration using the template file similar to filling out a form, with the template providing additional hints and further information for the decisions along the framework. We integrated the R Markdown template in an R package for easy installation and use, but also provide a browser-based option for users granting low-barrier access. PreregRS constitutes a first step to facilitate and support preregistration with research syntheses for all disciplines. It further adds to establishing open science practices in conducting research syntheses. This article is protected by copyright. All rights reserved.

A Reliability Generalization Meta-analysis of the Padua Inventory-Revised (PI-R)

Background/Objective: The Padua Inventory-Revised (PI-R) is a widely applied instrument to measure obsessive-compulsive symptoms in clinical and nonclinical samples. We conducted a reliability generalization meta-analysis on the PI-R. Method: An exhaustive literature search yielded 118 empirical studies that had applied the PI-R, from which 30 studies (33 samples) reported an original reliability estimate. Results: Assuming a random-effects model, the average internal consistency reliability (Cronbach's alpha) was .92 (95% CI [.91, .93]) for the total scores, and ranged from .74 to .89 for the subscales. Assuming mixed-effects models, moderator analyses showed a positive statistically significant association between the standard deviation of the total scores and the reliability coefficients (p = .002; R² = .38). Conclusions: In terms of reliability, the PI-R scale was found to be adequate for both research and clinical purposes, although exhibiting large heterogeneity across studies. Future empirical studies using the PI-R should be required to provide at least one reliability estimate based on their own data.

Neuropsychological functioning in adult anorexia nervosa: A meta-analysis

Several studies have conceptualized neuropsychological dysfunction as part of the core pathology and defining behaviors seen in the eating disorder anorexia nervosa (AN). The aim of the current review was to synthesize the differences in neuropsychological test performance between individuals with AN and healthy controls, quantify and explain their heterogeneity. The search and screening procedures resulted in fifty studies that comprised 186 neuropsychological test results. Utilizing random-effects meta-analyses, the results revealed evidence for significant, moderate underperformance in people with AN in overall neuropsychological functioning (g¯ = -0.43, 95 % CI [-0.50, -0.36]). Weighted mean effect sizes ranged from g¯ = -0.53 for visuospatial abilities to g¯ = -0.10 for planning. Study and participant characteristics, including body mass index (BMI) and age, had significant moderator effects, especially on executive function, memory, and visuospatial abilities. The findings from the current study provide an extensive and comprehensive overview of the possible impairments in neuropsychological functioning in adult patients diagnosed with AN.

Tryptophan Catabolites in Bipolar Disorder: A Meta-Analysis

Objective

Tryptophan catabolites (TRYCATs) are implicated in the pathophysiology of mood disorders by mediating immune-inflammation and neurodegenerative processes. We performed a meta-analysis of TRYCAT levels in bipolar disorder (BD) patients compared to healthy controls.

Methods

A systematic literature search in seven electronic databases (PubMed, Embase, Web of Science, Cochrane, Emcare, PsycINFO, Academic Search Premier) was conducted on TRYCAT levels in cerebrospinal fluid or peripheral blood according to the PRISMA statement. A minimum of three studies per TRYCAT was required for inclusion. Standardized mean differences (SMD) were computed using random effect models. Subgroup analyses were performed for BD patients in a different mood state (depressed, manic). The methodological quality of the studies was rated using the modified Newcastle-Ottawa Quality assessment Scale.

Results

Twenty-one eligible studies were identified. Peripheral levels of tryptophan (SMD = -0.44; p < 0.001), kynurenine (SMD = - 0.3; p = 0.001) and kynurenic acid (SMD = -.45; p = < 0.001) were lower in BD patients versus healthy controls. In the only three eligible studies investigating TRP in cerebrospinal fluid, tryptophan was not significantly different between BD and healthy controls. The methodological quality of the studies was moderate. Subgroup analyses revealed no significant difference in TRP and KYN values between manic and depressed BD patients, but these results were based on a limited number of studies.

Conclusion

The TRYCAT pathway appears to be downregulated in BD patients. There is a need for more and high-quality studies of peripheral and central TRYCAT levels, preferably using longitudinal designs.

Using information-theoretic approaches for model selection in meta-analysis

Meta‐regression can be used to examine the association between effect size estimates and the characteristics of the studies included in a meta‐analysis using regression‐type methods. By searching for those characteristics (i.e., moderators) that are related to the effect sizes, we seek to identify a model that represents the best approximation to the underlying data generating mechanism. Model selection via testing, either through a series of univariate models or a model including all moderators, is the most commonly used approach for this purpose. Here, we describe alternative model selection methods based on information criteria, multimodel inference, and relative variable importance. We demonstrate their application using an illustrative example and present results from a simulation study to compare the performance of the various model selection methods for identifying the true model across a wide variety of conditions. Whether information‐theoretic approaches can also be used not only in combination with maximum likelihood (ML) but also restricted maximum likelihood (REML) estimation was also examined. The results indicate that the conventional methods for model selection may be outperformed by information‐theoretic approaches. The latter are more often among the set of best methods across all of the conditions simulated and can have higher probabilities for identifying the true model under particular scenarios. Moreover, their performance based on REML estimation was either very similar to that from ML estimation or at times even better depending on how exactly the REML likelihood was computed. These results suggest that alternative model selection methods should be more widely applied in meta‐regression.

Meta-analysis with Robust Variance Estimation: Expanding the Range of Working Models

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the exact form of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multilevel and multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer potential benefits in terms of better capturing the types of data structures that occur in practice and, under some circumstances, improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the "metafor" and "clubSandwich" packages for R), illustrate the proposed approach in a meta-analysis of randomized trials on the effects of brief alcohol interventions for adolescents and young adults, and report findings from a simulation study evaluating the performance of the new methods.

Heterogeneity of Research Results: A New Perspective From Which to Assess and Promote Progress in Psychological Science

Heterogeneity emerges when multiple close or conceptual replications on the same subject produce results that vary more than expected from the sampling error. Here we argue that unexplained heterogeneity reflects a lack of coherence between the concepts applied and data observed and therefore a lack of understanding of the subject matter. Typical levels of heterogeneity thus offer a useful but neglected perspective on the levels of understanding achieved in psychological science. Focusing on continuous outcome variables, we surveyed heterogeneity in 150 meta-analyses from cognitive, organizational, and social psychology and 57 multiple close replications. Heterogeneity proved to be very high in meta-analyses, with powerful moderators being conspicuously absent. Population effects in the average meta-analysis vary from small to very large for reasons that are typically not understood. In contrast, heterogeneity was moderate in close replications. A newly identified relationship between heterogeneity and effect size allowed us to make predictions about expected heterogeneity levels. We discuss important implications for the formulation and evaluation of theories in psychology. On the basis of insights from the history and philosophy of science, we argue that the reduction of heterogeneity is important for progress in psychology and its practical applications, and we suggest changes to our collective research practice toward this end.

Exploratory Analyses for Missing Data in Meta-Analyses and Meta-Regression: A Tutorial

OBJECTIVES

In this tutorial, we examine methods for exploring missingness in a dataset in ways that can help to identify the sources and extent of missingness, as well as clarify gaps in evidence.

METHODS

Using raw data from a meta-analysis of substance abuse interventions, we demonstrate the use of exploratory missingness analysis (EMA) including techniques for numerical summaries and visual displays of missing data.

RESULTS

These techniques examine the patterns of missing covariates in meta-analysis data and the relationships among variables with missing data and observed variables including the effect size. The case study shows complex relationships among missingness and other potential covariates in meta-regression, highlighting gaps in the evidence base.

CONCLUSION

Meta-analysts could often benefit by employing some form of EMA as they encounter missing data.

A Systematic Overview of Meta-Analyses on Socioeconomic Status, Cognitive Ability, and Achievement: The Need to Focus on Specific Pathways

Meta-analyses on the relation between socioeconomic status (SES) and performance on measures of cognitive ability and achievement arrive at the same general conclusion of a small to medium association. Advancements in methods make possible for meta-analyses to examine specific pathways linking SES to cognitive ability and achievement, as well as the moderators of these pathways. In this study, we conducted a systematic overview of meta-analyses on SES to address three research questions: 1) what is the direction and overall strength of association between SES and performance on measures of cognitive ability and achievement, and how precise are the effect sizes reported? 2) to what extent have meta-analyses examined moderation by components of SES, age, sex, and race/ethnicity? and 3) to what extent have meta-analyses examined mechanisms linking SES to cognitive ability and achievement? We conducted a systematic search using online archives (i.e., PsycINFO, ERIC, PubMed, Sociological Abstracts, and Web of Science), searching issues in Psychological Bulletin and Review of Educational Research, and examining references and citations. We identified 14 meta-analyses published between 1982 and 2019. These meta-analyses consistently reported positive associations of small to medium magnitude, indicating that SES is a meaningful contributor to the development of cognitive ability and achievement. Fewer meta-analyses reported evidence of moderation by age, sex, and race/ethnicity. None of the meta-analyses directly examined mechanisms, but provided evidence of possible mechanisms for future research. We suggest that meta-analyses can increase their contribution to future research, interventions, and policy by narrowing their focus on specific pathways.

Comprehensive meta-analysis of resilience interventions

BACKGROUND/RATIONALE

There is no current consensus on operational definitions of resilience. Instead, researchers often debate the optimal approach to understanding resilience, while continuing to explore ways to enhance and/or promote its qualities in various populations. The goal of the current meta-analysis is to substantiate existing evidence examining the promotion of resilience through various interventions. Particular emphasis was placed upon the factors that contribute to variability across interventions, such as age, gender, duration of intervention, intervention approaches and risk exposure of targeted population.

METHOD

The literature search was conducted on May 28, 2019. Search terms included "resilience intervention" OR "promoting resilience" OR "promoting resiliency" OR "resilience-based intervention". A total of 268 studies, with 1584 independent samples, were included in the meta-analysis. In addition to overall efficacy, outcome-based analyses were conducted for intervention outcomes based on action, biophysical, coping, emotion, resilience, symptoms, and well-being. Finally, moderators of age, gender, length of intervention, intervention approach, intervention target, and the level of risk exposure of the sampled population were examined as moderators.

RESULTS

The multi-level meta-analysis indicated that resilience-promoting interventions yielded a small, but statistically significant overall effect, Hedges's g = 0.48 (SE = 0.04, 95% CI = [0.40, 0.56]. The variability in study effect sizes within and between studies was significant, p < .001, with many falling short of the threshold for practical significance.

DISCUSSION

Findings lend some support for the overall efficacy of resilience interventions. However, empirical results should be cautiously interpreted in tandem with their theoretical relevance and potential advancements to the construct of resilience. Variabilities across findings reflect the current ambiguities surrounding the conceptualization and operationalization of resilience. Directions for future research on resilience as well as practical considerations are discussed.

Blood-based kynurenine pathway alterations in schizophrenia spectrum disorders: A meta-analysis

INTRODUCTION

The kynurenine pathway (KP) has been proposed as indirect link between systemic immune responses and clinical symptom development in schizophrenia spectrum disorders (SSD). Empirical evidence for such immune-related KP shifts in SSD has however resulted in divergent findings.

METHODS

We conducted a systematic literature search in PubMed. Thirty papers (total number of patients n = 1506; controls: n = 1432) reported on peripheral concentrations of KP metabolites in SSD patients versus controls. Six KP metabolites were included in a meta-analysis, with secondary analysis of covariate and subgroup effects of patients' symptomatic state, age and duration of illness.

RESULTS

Tryptophan (SMD: -0.30; p = .003) and Xanthurenic Acid (SMD: -0.80; p < .001) were significantly decreased in SSD compared to controls, while Quinolinic Acid (SMD: -0.40; p = .08) and Kynurenic Acid (SMD: -0.39; p = .04) were only significantly decreased in patients with acute or highly symptomatic illness. Finally, in relatively older patient cohorts Kynurenine (SMD: -0.31; p = .02) and Kynurenic Acid (SMD: -0.40; p = .002) were found to be decreased.

CONCLUSION

A partial downregulation of the KP is observed in SSD patients, in particular during acute symptomatic states and in older age, effects that were independent from each other. In contrast, younger and stable or remitted patients display limited to no KP metabolite abnormalities. The current meta-analysis illustrates the dynamic nature of KP abnormalities. It should be noted that all included studies investigated peripheral KP metabolites, which do not necessarily reflect central KP metabolite abnormalities in schizophrenic patients.

Enriching Meta-Analytic Models of Summary Data: A Thought Experiment and Case Study

Meta-analysis typically involves the analysis of summary data (e.g., means, standard deviations, and sample sizes) from a set of studies via a statistical model that is a special case of a hierarchical (or multilevel) model. Unfortunately, the common summary-data approach to meta-analysis used in psychological research is often employed in settings where the complexity of the data warrants alternative approaches. In this article, we propose a thought experiment that can lead meta-analysts to move away from the common summary-data approach to meta-analysis and toward richer and more appropriate summary-data approaches when the complexity of the data warrants it. Specifically, we propose that it can be extremely fruitful for meta-analysts to act as if they possess the individual-level data from the studies and consider what model specifications they might fit even when they possess only summary data. This thought experiment is justified because (a) the analysis of the individual-level data from the studies via a hierarchical model is considered the “gold standard” for meta-analysis and (b) for a wide variety of cases common in meta-analysis, the summary-data and individual-level-data approaches are, by a principle known as statistical sufficiency, equivalent when the underlying models are appropriately specified. We illustrate the value of our thought experiment via a case study that evolves across five parts that cover a wide variety of data settings common in meta-analysis.