Accumulating evidence across studies: Consistent methods protect against false findings produced by p-hacking

Much empirical science involves evaluating alternative explanations for the obtained data. For example, given certain assumptions underlying a statistical test, a "significant" result generally refers to implausibility of a null (zero) effect in the population producing the obtained study data. However, methodological work on various versions of p-hacking (i.e., using different analysis strategies until a "significant" result is produced) questions whether significant p-values might often reflect false findings. Indeed, initial simulations of single studies showed that the potential for finding "significant" but false findings might be much higher than the nominal .05 value when various analysis flexibilities are undertaken. In many settings, however, research articles report multiple studies using consistent methods across the studies, where those consistent methods would constrain the flexibilities used to produce high false-finding rates for simulations of single studies. Thus, we conducted simulations of study sets. These simulations show that consistent methods across studies (i.e., consistent in terms of which measures are analyzed, which conditions are included, and whether and how covariates are included) dramatically reduce the potential for flexible research practices (p-hacking) to produce consistent sets of significant results across studies. For p-hacking to produce even modest probabilities of a consistent set of studies would require (a) a large amount of selectivity in study reporting and (b) severe (and quite intentional) versions of p-hacking. With no more than modest selective reporting and with consistent methods across studies, p-hacking does not provide a plausible explanation for consistent empirical results across studies, especially as the size of the reported study set increases. In addition, the simulations show that p-hacking can produce high rates of false findings for single studies with very large samples. In contrast, a series of methodologically-consistent studies (even with much smaller samples) is much less vulnerable to the forms of p-hacking examined in the simulations.

Not Twisted, Just Kinky: Replication and Structural Invariance of Attachment, Personality, and Well-Being Among BDSM Practitioners

BDSM is a range of diverse sexual practices. Stigma regarding BDSM is associated with dysfunctional personalities, insecure attachment styles, or damaged well-being. Previous studies have shown contrary evidence to these views. However, the replicability of these findings remains understudied. This study conducts a close replication to examine personality, attachment, rejection sensitivity, and well-being differences between BDSM practitioners and non-practitioners. To address previous limitations, this study provides a highly powered sample of a new population (Spanish, N = 1,907), assessing effect sizes and the impact of LGTBIQA+ individuals and employing an alternative BDSM role classification. Additionally, we examined attachment styles, personality, and well-being differences among BDSM practitioners. As predicted, BDSM practitioners showed higher levels of secure attachment, conscientiousness, openness, and well-being while also lower levels of insecure attachments, rejection sensitivity, neuroticism, and agreeableness, countering the stigma. Gender, sexual orientations, and experience with BDSM showed explanatory potential. The associations between attachment, personality, and well-being were consistent across both BDSM practitioners and non-practitioners, as well as across various BDSM roles. BDSM practitioners share the same psychological structure as non-practitioners but also show more functional profiles. Thus, de-stigmatizing BDSM populations is reinforced and recommended. Limitations and implications for applied and research audiences are discussed.

Awareness of embodiment enhances enjoyment and engages sensorimotor cortices

Whether in performing arts, sporting, or everyday contexts, when we watch others move, we tend to enjoy bodies moving in synchrony. Our enjoyment of body movements is further enhanced by our own prior experience with performing those movements, or our 'embodied experience'. The relationships between movement synchrony and enjoyment, as well as embodied experience and movement enjoyment, are well known. The interaction between enjoyment of movements, synchrony, and embodiment is less well understood, and may be central for developing new approaches for enriching social interaction. To examine the interplay between movement enjoyment, synchrony, and embodiment, we asked participants to copy another person's movements as accurately as possible, thereby gaining embodied experience of movement sequences. Participants then viewed other dyads performing the same or different sequences synchronously, and we assessed participants' recognition of having performed these sequences, as well as their enjoyment of each movement sequence. We used functional near-infrared spectroscopy to measure cortical activation over frontotemporal sensorimotor regions while participants performed and viewed movements. We found that enjoyment was greatest when participants had mirrored the sequence and recognised it, suggesting that awareness of embodiment may be central to enjoyment of synchronous movements. Exploratory analyses of relationships between cortical activation and enjoyment and recognition implicated the sensorimotor cortices, which subserve action observation and aesthetic processing. These findings hold implications for clinical research and therapies seeking to foster successful social interaction.

Treatment of Eating Disorders: Current Status, Challenges, and Future Directions

Specific psychological treatments have demonstrated efficacy and represent the first-line approaches recommended for anorexia nervosa, bulimia nervosa, and binge-eating disorder. Unfortunately, many patients, particularly those with anorexia nervosa, do not derive sufficient benefit from existing treatments, and better or alternative treatments for eating disorders are needed. Less progress has been made in developing pharmacologic options for eating disorders. No medications approved for anorexia nervosa exist, and only one each exists for bulimia nervosa and for binge-eating disorder; available data indicate that most patients fail to benefit from available medications. Longer and combined treatments have generally not enhanced outcomes. This review presents emerging findings from more complex and clinically relevant adaptive treatment designs, as they offer some clinical guidance and may serve as models for future enhanced treatment research.

Addressing researcher fraud: retrospective, real-time, and preventive strategies-including legal points and data management that prevents fraud

Researcher fraud is often easy and enticing in academic research, with little risk of detection. Cases of extensive fraud continue to occur. The amount of fraud that goes undetected is unknown and may be substantial. Three strategies for addressing researcher fraud are (a) retrospective investigations after allegations of fraud have been made, (b) sting operations that provide conclusive evidence of fraud as it occurs, and (c) data management practices that prevent the occurrence of fraud. Institutional and regulatory efforts to address researcher fraud have focused almost exclusively on the retrospective strategy. The retrospective approach is subject to controversy due to the limitations of post-hoc evidence in science, the difficulty in establishing who actually committed the fraud in some cases, the application of a legal standard of evidence that is much lower than the usual standards of evidence in science, and the lack of legal expertise by scientists investigating fraud. The retrospective strategy may be reliably effective primarily in cases of extensive, careless fraud. Sting operations can overcome these limitations and controversies, but are not feasible in many situations. Data management practices that are effective at preventing researcher fraud and unintentional errors are well-established in clinical trials regulated by government agencies, but appear to be largely unknown or unimplemented in most academic research. Established data management practices include: archiving secure copies of the raw data, audit trails, restricted access to the data and data collection processes, software validation, quality control checks, blinding, preregistration of data processing and analysis programs, and research audits that directly address fraud. Current discussions about data management in academic research focus on sharing data with little attention to practices that prevent intentional and unintentional errors. A designation or badge such as error-controlled data management could be established to indicate research that was conducted with data management practices that effectively address intentional and unintentional errors.

Field testing the transferability of behavioural science knowledge on promoting vaccinations

As behavioural science is increasingly adopted by organizations, there is a growing need to assess the robustness and transferability of empirical findings. Here, we investigate the transferability of insights from various sources of behavioural science knowledge to field settings. Across three pre-registered randomized controlled trials (RCTs, N = 314,824) involving a critical policy domain-COVID-19 booster uptake-we field tested text-based interventions that either increased vaccinations in prior field work (RCT1, NCT05586204), elevated vaccination intentions in an online study (RCT2, NCT05586178) or were favoured by scientists and non-experts (RCT3, NCT05586165). Despite repeated exposure to COVID-19 vaccination messaging in our population, reminders and psychological ownership language increased booster uptake, replicating prior findings. However, strategies deemed effective by prediction or intention surveys, such as encouraging the bundling of COVID-19 boosters and flu shots or addressing misconceptions, yielded no detectable benefits over simple reminders. These findings underscore the importance of testing interventions' transferability to real-world settings.

"When" Versus "Whether" Gender/Sex Differences: Insights From Psychological Research on Negotiation, Risk-Taking, and Leadership

We present a conceptual framework of situational moderators of gender/sex effects in negotiation, risk-taking, and leadership-three masculine-stereotypic domains associated with gender/sex gaps in pay and authority. We propose that greater situational ambiguity and higher relevance and salience of gender/sex increase the likelihood of gender/sex-linked behaviors in these domains. We argue that greater ambiguity increases the extent to which actors and audiences must search inwardly (e.g., mental schema, past experience) or outwardly (e.g., social norms) for cues on how to behave or evaluate a situation and thereby widens the door for gender/sex-linked influences. Correspondingly, we propose that gender/sex effects on behavior and evaluations in these domains will be more likely when gender/sex is more relevant and salient to the setting or task. We propose further that these two situational moderators may work jointly or interactively to influence the likelihood of gender/sex effects in negotiation, risk-taking, and leadership. We conclude by discussing applications of our conceptual framework to psychological science and its translation to practice, including directions for future research.

I understand your pain but I do not feel it: lower affective empathy in response to others' social pain in narcissism

Introduction

While the relationship between narcissism and empathy has been well-researched, studies have paid less attention to empathic accuracy, i.e., appreciating the precise strength of another person's emotions, and self-other distinction, in terms of the disparity between affective ratings for self and other in response to emotive stimuli. Furthermore, empathic responses may vary depending on whether the pain is physical or social.

Methods

We investigated empathic accuracy, affective empathy, and the distinction between pain, emotion and intensity ratings for self and other, in high (n = 44) and low (n = 43) narcissism groups (HNG and LNG, respectively) selected from 611 students, in response to both types of pain. Participants watched six videos where targets expressed genuine experiences of physical and social pain, and rated the perceived affect and pain experienced by the person in the video and their own empathic emotional responses.

Results and discussion

The HNG displayed lower affective empathy and empathic accuracy than the LNG for both pain types. Within the HNG there was higher empathic accuracy for social vs. physical pain, despite reduced affective empathy for social pain, in contrast to the LNG. In addition to this paradox, the HNG demonstrated greater differences between ratings for the self and for target others than the LNG, suggesting that narcissism is associated with higher self-other distinction in response to viewing other people describing social pain.

The State of Open Science Practices in Psychometric Studies of Suicide: A Systematic Review

The adoption of open science practices (OSPs) is crucial for promoting transparency and robustness in research. We conducted a systematic review to assess the frequency and trends of OSPs in psychometric studies focusing on measures of suicidal thoughts and behavior. We analyzed publications from two international databases, examining the use of OSPs such as open access publication, preregistration, provision of open materials, and data sharing. Our findings indicate a lack of adherence to OSPs in psychometric studies of suicide. The majority of manuscripts were published under restricted access, and preregistrations were not utilized. The provision of open materials and data was rare, with limited access to instruments and analysis scripts. Open access versions (preprints/postprints) were scarce. The low adoption of OSPs in psychometric studies of suicide calls for urgent action. Embracing a culture of open science will enhance transparency, reproducibility, and the impact of research in suicide prevention efforts.

The transparency of quantitative empirical legal research published in highly ranked law journals (2018-2020): an observational study

Background

Scientists are increasingly concerned with making their work easy to verify and build upon. Associated practices include sharing data, materials, and analytic scripts, and preregistering protocols. This shift towards increased transparency and rigor has been referred to as a "credibility revolution." The credibility of empirical legal research has been questioned in the past due to its distinctive peer review system and because the legal background of its researchers means that many often are not trained in study design or statistics. Still, there has been no systematic study of transparency and credibility-related characteristics of published empirical legal research.

Methods

To fill this gap and provide an estimate of current practices that can be tracked as the field evolves, we assessed 300 empirical articles from highly ranked law journals including both faculty-edited journals and student-edited journals.

Results

We found high levels of article accessibility (86%, 95% CI = [82%, 90%]), especially among student-edited journals (100%). Few articles stated that a study's data are available (19%, 95% CI = [15%, 23%]). Statements of preregistration (3%, 95% CI = [1%, 5%]) and availability of analytic scripts (6%, 95% CI = [4%, 9%]) were very uncommon. (i.e., they collected new data using the study's reported methods, but found results inconsistent or not as strong as the original).

Conclusion

We suggest that empirical legal researchers and the journals that publish their work cultivate norms and practices to encourage research credibility. Our estimates may be revisited to track the field's progress in the coming years.

Methodological integrity assessment in the mobile paradigm literature: A lesson for understanding opportunistic use of researcher degrees of freedom in psychology

The mobile paradigm has played a fundamental role in memory development research. One key characteristic of the mobile paradigm literature is that across decades, researchers have faithfully followed a particular methodological protocol with its own unique definitions of learning and memory. To investigate the extent to which these methodological choices affected the results, the literature (77 publications and 505 statistical tests) was evaluated for four frequently encountered research biases. The results suggested that research using the paradigm was conducted with scientific rigor. However, methodological choices along with unique operational definitions of learning and memory accounted for more than half of the findings. Thus, the literature has been contaminated by methodological artifacts due to the opportunistic use of researcher degrees of freedom.

Best practices for cleaning eye movement data in reading research

One challenge that comes with studying eye movement behavior is deciding how to clean the eye movement data (e.g., fixation durations) before conducting analyses. Reading researchers must decide which data cleaning methods they will use and which thresholds they will set to remove eye movements that are not reflective of lexical processing. The purpose of this project was to determine what data cleaning methods are typically used and if there are any consequences of using different data cleaning methods. In the first study, an analysis of 192 recently published articles indicated that there is inconsistency in the reporting and application of data cleaning methods. In the second study, three different data cleaning methods were applied based on the literature analysis in the first study. Analyses were conducted to determine the impact of different data cleaning methods on three commonly studied effects in reading research (frequency, predictability, and length). Overall, standardized estimates decreased for each effect when more data were removed; however, removing more data also resulted in decreased variance. As a result, effects remained significant with each data cleaning method, and simulated power remained high for both a moderate and small sample size. Effect sizes remained consistent for most effects but decreased for the length effect as more data were removed. Seven suggestions are provided that are based on open science practices with the intention of helping researchers, reviewers, and the field as a whole.

Construct validity evidence reporting practices for the Reading the Mind in the Eyes Test: A systematic scoping review

The Reading the Mind in the Eyes Test (RMET) is one of the most influential measures of social cognitive ability, and it has been used extensively in clinical populations. However, questions have been raised about the validity of RMET scores. We conducted a systematic scoping review of the validity evidence reported in studies that administered the RMET (n = 1461; of which 804 included at least one clinical sample) with a focus on six key dimensions: internal consistency, test-retest reliability, factor structure, convergent validity, discriminant validity, and known group validity. Strikingly, 63% of these studies failed to provide validity evidence from any of these six categories. Moreover, when evidence was reported, it frequently failed to meet widely accepted validity standards. Overall, our results suggest a troubling conclusion: the validity of RMET scores (and the research findings based on them) are largely unsubstantiated and uninterpretable. More broadly, this project demonstrates how unaddressed measurement issues can undermine a voluminous psychological literature.

Publication bias in psychology: A closer look at the correlation between sample size and effect size

Previously observed negative correlations between sample size and effect size (n-ES correlation) in psychological research have been interpreted as evidence for publication bias and related undesirable biases. Here, we present two studies aimed at better understanding to what extent negative n-ES correlations reflect such biases or might be explained by unproblematic adjustments of sample size to expected effect sizes. In Study 1, we analysed n-ES correlations in 150 meta-analyses from cognitive, organizational, and social psychology and in 57 multiple replications, which are free from relevant biases. In Study 2, we used a random sample of 160 psychology papers to compare the n-ES correlation for effects that are central to these papers and effects selected at random from these papers. n-ES correlations proved inconspicuous in meta-analyses. In line with previous research, they do not suggest that publication bias and related biases have a strong impact on meta-analyses in psychology. A much higher n-ES correlation emerged for publications' focal effects. To what extent this should be attributed to publication bias and related biases remains unclear.

Model Specification Searches in Structural Equation Modeling Using Bee Swarm Optimization

Metaheuristics are optimization algorithms that efficiently solve a variety of complex combinatorial problems. In psychological research, metaheuristics have been applied in short-scale construction and model specification search. In the present study, we propose a bee swarm optimization (BSO) algorithm to explore the structure underlying a psychological measurement instrument. The algorithm assigns items to an unknown number of nested factors in a confirmatory bifactor model, while simultaneously selecting items for the final scale. To achieve this, the algorithm follows the biological template of bees' foraging behavior: Scout bees explore new food sources, whereas onlooker bees search in the vicinity of previously explored, promising food sources. Analogously, scout bees in BSO introduce major changes to a model specification (e.g., adding or removing a specific factor), whereas onlooker bees only make minor changes (e.g., adding an item to a factor or swapping items between specific factors). Through this division of labor in an artificial bee colony, the algorithm aims to strike a balance between two opposing strategies diversification (or exploration) versus intensification (or exploitation). We demonstrate the usefulness of the algorithm to find the underlying structure in two empirical data sets (Holzinger-Swineford and short dark triad questionnaire, SDQ3). Furthermore, we illustrate the influence of relevant hyperparameters such as the number of bees in the hive, the percentage of scouts to onlookers, and the number of top solutions to be followed. Finally, useful applications of the new algorithm are discussed, as well as limitations and possible future research opportunities.

The role of research design in the reproducibility of L1 and L2 language networks: A review of bilingual neuroimaging meta-analyses

Meta-analyses are a method by which to increase the statistical power and generalizability of neuroimaging findings. In the neurolinguistics literature, meta-analyses have the potential to substantiate hypotheses about L1 and L2 processing networks and to reveal differences between the two that may escape detection in individual studies. Why then is there so little consensus between the reported findings of even the most recently published and most highly powered meta-analyses? Limitations in the literature, such as the absence of a common method to define and measure descriptive categories (e.g., proficiency level, degree of language exposure, age of acquisition, etc.) are often cited. An equally plausible explanation lies in the technical details of how individual meta-analyses are conducted. This paper provides a review of recent meta-analyses, with a discussion of their methodological choices and the possible effect those choices may have on the reported findings.

How Can People Become Happier? A Systematic Review of Preregistered Experiments

Can happiness be reliably increased? Thousands of studies speak to this question. However, many of them were conducted during a period in which researchers commonly "p-hacked," creating uncertainty about how many discoveries might be false positives. To prevent p-hacking, happiness researchers increasingly preregister their studies, committing to analysis plans before analyzing data. We conducted a systematic literature search to identify preregistered experiments testing strategies for increasing happiness. We found surprisingly little support for many widely recommended strategies (e.g., performing random acts of kindness). However, our review suggests that other strategies-such as being more sociable-may reliably promote happiness. We also found strong evidence that governments and organizations can improve happiness by providing underprivileged individuals with financial support. We conclude that happiness research stands on the brink of an exciting new era, in which modern best practices will be applied to develop theoretically grounded strategies that can produce lasting gains in life satisfaction.

Translating the Transdiagnostic: Aligning Assessment Practices With Research Advances

Researchers and clinicians working within the Diagnostic and Statistical Manual of Mental Disorders: Fifth Edition, Text Rev (DSM-5-TR) framework face a difficult question: what does it mean to have an evidence-based assessment of a nonevidence-based diagnostic construct? Alternative nosological approaches conceptualize psychopathology as (a) hierarchical, allowing researchers to move between levels of description and (b) dimensional, eliminating artificial dichotomies between disorders and the dichotomy between mental illness and mental well-being. In this article, we provide an overview of ongoing efforts to develop validated measures of transdiagnostic nosologies (i.e., the Hierarchical Taxonomy of Psychopathology; HiTOP) with applications for measurement-based care. However, descriptive models like HiTOP, which summarize patterns of covariation among psychopathology symptoms, do not address dynamic processes underlying the problems associated with psychopathology. Ambulatory assessment, well-suited to examine such dynamic processes, has also developed rapidly in recent decades. Thus, the goal of the current article is twofold. First, we provide a brief overview of developments in constructing valid measures of the HiTOP model as well as developments in ambulatory assessment practices. Second, we outline how these parallel developments can be integrated to advance measurement-based treatment. We end with a discussion of some major challenges for future research to address to integrate advances more fully in transdiagnostic and ambulatory assessment practices.

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses

Meta-analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) "For a given severity of internal bias across studies and of publication bias, how much could the results change?"; and (2) "For a given severity of publication bias, how severe would internal bias have to be, hypothetically, to attenuate the results to the null or by a given amount?" These methods consider the average internal bias across studies, obviating specifying the bias in each study individually. The analyst can assume that internal bias affects all studies, or alternatively that it only affects a known subset (e.g., nonrandomized studies). The internal bias can be of unknown origin or, for certain types of bias in causal estimates, can be bounded analytically. The analyst can specify the severity of publication bias or, alternatively, consider a "worst-case" form of publication bias. Robust estimation methods accommodate non-normal effects, small meta-analyses, and clustered estimates. As we illustrate by re-analyzing published meta-analyses, the methods can provide insights that are not captured by simply considering each bias in turn. An R package implementing the methods is available (multibiasmeta).

Open Science Considerations for Descriptive Research in Developmental Science

Descriptive developmental research seeks to document, describe, and analyze the conditions under which infants and children live and learn. Here, we articulate how open-science practices can be incorporated into descriptive research to increase its transparency, reliability, and replicability. To date, most open-science practices have been oriented toward experimental rather than descriptive studies, and it can be confusing to figure out how to translate open-science practices (e.g., preregistration) for research that is more descriptive in nature. We discuss a number of unique considerations for descriptive developmental research, taking inspiration from existing open-science practices and providing examples from recent and ongoing studies. By embracing a scientific culture where descriptive research and open science coexist productively, developmental science will be better positioned to generate comprehensive theories of development and understand variability in development across communities and cultures.

Outcomes of conservative treatment of acromial and scapular spine stress fracture post reverse shoulder arthroplasty: a systematic review with meta-analysis

INTRODUCTION

Acromial and scapular spine stress fractures are common complications post reverse shoulder arthroplasty. A systematic review with meta-analysis was conducted to evaluate the clinical outcomes of conservatively treated acromial and scapular spine stress fractures post reverse shoulder arthroplasty in comparison to nonfracture control.

METHODS

Embase, MEDLINE, and Web of Science were searched on September 9, 2022, using various terms related to fracture, acromion, scapula, and reverse shoulder arthroplasty. Meta-analysis using a random effects model was performed on common outcome scores. Risk of bias was assessed using the Joanna Briggs Institute tools for case-controlled studies. Sensitivity analysis was performed for imputed standard deviations and studies with <20 participants.

RESULTS

Thirteen studies with a total of 339 fracture and 3142 control patients were included in the systematic review. Meta-analysis was performed on 12 studies assessing active forward flexion, abduction, external rotation, Constant-Murley score, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form, pain score, Simple Shoulder Test, and Single Assessment Numeric Evaluation. Fracture patients experienced significantly poorer functional outcome compared with control patients when the fractures were managed conservatively. Levy III fractures tend to experience worse outcomes. Pain score did not demonstrate a statistically significant difference between the fracture and nonfracture cohort.

DISCUSSION

Currently, literature regarding surgical management is lacking, partially because of a dearth of high-quality literature but also hindered by a lack of standardized techniques. This review was limited by inclusion of studies of low evidence, small sample sizes, and inconsistency in outcome measurement and follow-up period. Overall, patients with acromial and scapular spine stress fractures treated conservatively are expected to have reduced shoulder function compared to nonfracture patients, with the spine of scapula fracture patients suffering the poorest outcomes. Future studies should aim to use the same sets of outcomes parameters to assess the patients at fixed time intervals and report outcomes stratified by Levy classification. This will enable interinstitutional collaboration and pooling of results.

Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues

This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.

A Metascientific Review of the Evidential Value of Acceptance and Commitment Therapy for Depression

In the past three-and-a-half decades, nearly 500 randomized controlled trials (RCTs) have examined Acceptance and Commitment Therapy (ACT) for a range of health problems, including depression. However, emerging concerns regarding the replicability of scientific findings across psychology and mental health treatment outcome research highlight a need to re-examine the strength of evidence for treatment efficacy. Therefore, we conducted a metascientific review of the evidential value of ACT in treating depression. Whereas reporting accuracy was generally high across all trials, we found important differences in evidential value metrics corresponding to the types of control conditions used. RCTs of ACT compared to weaker controls (e.g., no treatment, waitlist) were well-powered, with sample sizes appropriate for detecting plausible effect sizes. They typically yielded stronger Bayesian evidence for (and larger posterior estimates of) ACT efficacy, though there was some evidence of significance inflation among these effects. RCTs of ACT against stronger controls (e.g., other psychotherapies), meanwhile, were poorly powered, designed to detect implausibly large effect sizes, and yielded ambiguous-if not contradicting-Bayesian evidence and estimates of efficacy. Although our review supports a view of ACT as efficacious for treating depression compared to weaker controls, future RCTs must provide more transparent reporting with larger groups of participants to properly assess the difference between ACT and competitor treatments such as behavioral activation and other forms of cognitive behavioral therapy. Clinicians and health organizations should reassess the use of ACT for depression if costs and resources are higher than for other efficacious treatments. Clinical trials contributing effects to our synthesis can be found at https://osf.io/qky35.

Social Psychology of and for World-Making

ACADEMIC ABSTRACT

Social psychology's disconnect from the vital and urgent questions of people's lived experiences reveals limitations in the current paradigm. We draw on a related perspective in social psychology1-the sociocultural approach-and argue how this perspective can be elaborated to consider not only social psychology as a historical science but also social psychology of and for world-making. This conceptualization can make sense of key theoretical and methodological challenges faced by contemporary social psychology. As such, we describe the ontology, epistemology, ethics, and methods of social psychology of and for world-making. We illustrate our framework with concrete examples from social psychology. We argue that reconceptualizing social psychology in terms of world-making can make it more humble yet also more relevant, reconnecting it with the pressing issues of our time.

PUBLIC ABSTRACT

We propose that social psychology should focus on "world-making" in two senses. First, people are future-oriented and often are guided more by what could be than what is. Second, social psychology can contribute to this future orientation by supporting people's world-making and also critically reflecting on the role of social psychological research in world-making. We unpack the philosophical assumptions, methodological procedures, and ethical considerations that underpin a social psychology of and for world-making. Social psychological research, whether it is intended or not, contributes to the societies and cultures in which we live, and thus it cannot be a passive bystander of world-making. By embracing social psychology of and for world-making and facing up to the contemporary societal challenges upon which our collective future depends will make social psychology more humble but also more relevant.

Are we all implicit puritans? New evidence that work and sex are intuitively moralized in both traditional and non-traditional cultures

Contradicting our earlier claims of American moral exceptionalism, recent self-replication evidence from our laboratory indicates that implicit puritanism characterizes the judgments of people across cultures. Implicit cultural evolution may lag behind explicit change, such that differences between traditional and non-traditional cultures are greater at a deliberative than an intuitive level. Not too deep down, perhaps we are all implicit puritans.

A systematic review of the strength of evidence for the most commonly recommended happiness strategies in mainstream media

We conducted a systematic review of the evidence underlying some of the most widely recommended strategies for increasing happiness. By coding media articles on happiness, we first identified the five most commonly recommended strategies: expressing gratitude, enhancing sociability, exercising, practising mindfulness/meditation and increasing nature exposure. Next, we conducted a systematic search of the published scientific literature. We identified well-powered, pre-registered experiments testing the effects of these strategies on any aspect of subjective wellbeing (that is, positive affect, negative affect and life satisfaction) in non-clinical samples. A total of 57 studies were included. Our review suggests that a strong scientific foundation is lacking for some of the most commonly recommended happiness strategies. As the effectiveness of these strategies remains an open question, there is an urgent need for well-powered, pre-registered studies investigating strategies for promoting happiness.

Extraordinary Claims in the Literature on High-Intensity Interval Training (HIIT): I. Bonafide Scientific Revolution or a Looming Crisis of Replication and Credibility?

The literature on high-intensity interval training (HIIT) contains claims that, if true, could revolutionize the science and practice of exercise. This critical analysis examines two varieties of claims: (i) HIIT is effective in improving various indices of fitness and health, and (ii) HIIT is as effective as more time-consuming moderate-intensity continuous exercise. Using data from two recent systematic reviews as working examples, we show that studies in both categories exhibit considerable weaknesses when judged through the prism of fundamental statistical principles. Predominantly, small-to-medium effects are investigated in severely underpowered studies, thus greatly increasing the risk of both type I and type II errors of statistical inference. Studies in the first category combine the volatility of estimates associated with small samples with numerous dependent variables analyzed without consideration of the inflation of the type I error rate. Studies in the second category inappropriately use the p > 0.05 criterion from small studies to support claims of 'similar' or 'comparable' effects. It is concluded that the situation in the HIIT literature is reminiscent of the research climate that led to the replication crisis in psychology. As in psychology, this could be an opportunity to reform statistical practices in exercise science.

Age and professional experience as determinants of the utilization of psychoneuroimmunological research in clinical practice: An exploratory study

The immune system is affected by psychosocial stimuli and plays a major role in the development of various diseases. Psychoneuroimmunology (PNI)-based interventions may positively influence the disease course; however, the impact of PNI research findings on clinical practice differs depending on the medical specialties involved. A comprehensive overview of the use of PNI research findings in clinical practice is currently lacking. This exploratory study aimed to provide insight into the dissemination of PNI research findings and their practical applications among clinical practitioners. Data was collected from 50 physicians using an ad hoc online questionnaire. We invited participants to take part in our online survey via an article in the DocCheck Newsletter, a German-language newsletter for physicians. Bivariate nonparametric correlation analysis (Spearman correlation) were used to explore the relationship between independent variables (age, sex, medical specialty, professional experience, and clinical environment) and dependent variables (six questionnaire items concerned with awareness, relevance, and utilization of PNI concepts). While 46% of respondents believed that PNI research findings were relevant to patient treatment, only 22% used PNI-based interventions as part of their therapeutic regimen. Furthermore, 90% of participants could not refer their patients to therapists offering PNI-based interventions. Moderately positive correlations were identified between the increasing age (rs = .48, P < .001) and increasing amount of professional experience (rs = .34, P = .02) of study participants and awareness of the theoretical foundations of PNI research. Although there is some awareness of PNI among medical practitioners, there appears to be a clear barrier inhibiting the implementation of research findings in current treatment practices. Therefore, it is necessary to examine the impact of increasing age and professional experience on the utilization of PNI-based interventions in patient care.

Common, uncommon, and novel applications of random forest in psychological research

Recent reform efforts have pushed toward a better understanding of the distinction between exploratory and confirmatory research, and appropriate use of each. As some utilize more exploratory tools, it may be tempting to employ multiple linear regression models. In this paper, we advocate for the use of random forest (RF) models. RF is able to obtain better predictive performance than traditional regression, while also inherently protecting against overfitting as well as detecting nonlinear effects and interactions among predictors. Given the advantages of RF compared to other statistical procedures, it is a tool commonly used within a plethora of industries, including stock trading, banking, pharmaceuticals, and patient healthcare planning. However, we find RF is used within the field of psychology comparatively less frequently. In the current paper, we advocate for RF as an important statistical tool within the context of behavioral and psychological research. In hopes of increasing the use of RF in the field of psychology, we provide information pertaining to the limitations one might confront in using RF and how to overcome such limitations. Moreover, we discuss various methods for how to optimally utilize RF with psychological data, such as nonparametric modeling, interaction and nonlinearity detection, variable selection, prediction and classification modeling, and assessing parameters of Monte Carlo simulations. Throughout, we illustrate the use of RF with visualization strategies, aimed to make RF models more comprehensible and intuitive.

Reproducibility in Neuroimaging Analysis: Challenges and Solutions

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.

Preregistration: Practical Considerations for Speech, Language, and Hearing Research

PURPOSE

In the last decade, psychology and other sciences have implemented numerous reforms to improve the robustness of our research, many of which are based on increasing transparency throughout the research process. Among these reforms is the practice of preregistration, in which researchers create a time-stamped and uneditable document before data collection that describes the methods of the study, how the data will be analyzed, the sample size, and many other decisions. The current article highlights the benefits of preregistration with a focus on the specific issues that speech, language, and hearing researchers are likely to encounter, and additionally provides a tutorial for writing preregistrations.

CONCLUSIONS

Although rates of preregistration have increased dramatically in recent years, the practice is still relatively uncommon in research on speech, language, and hearing. Low rates of adoption may be driven by a lack of understanding of the benefits of preregistration (either generally or for our discipline in particular) or uncertainty about how to proceed if it becomes necessary to deviate from the preregistered plan. Alternatively, researchers may see the benefits of preregistration but not know where to start, and gathering this information from a wide variety of sources is arduous and time consuming. This tutorial addresses each of these potential roadblocks to preregistration and equips readers with tools to facilitate writing preregistrations for research on speech, language, and hearing.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.21644843.

Open Science Practices in Communication Sciences and Disorders: A Survey

PURPOSE

Open science is a collection of practices that seek to improve the accessibility, transparency, and replicability of science. Although these practices have garnered interest in related fields, it remains unclear whether open science practices have been adopted in the field of communication sciences and disorders (CSD). This study aimed to survey the knowledge, implementation, and perceived benefits and barriers of open science practices in CSD.

METHOD

An online survey was disseminated to researchers in the United States actively engaged in CSD research. Four-core open science practices were examined: preregistration, self-archiving, gold open access, and open data. Data were analyzed using descriptive statistics and regression models.

RESULTS

Two hundred twenty-two participants met the inclusion criteria. Most participants were doctoral students (38%) or assistant professors (24%) at R1 institutions (58%). Participants reported low knowledge of preregistration and gold open access. There was, however, a high level of desire to learn more for all practices. Implementation of open science practices was also low, most notably for preregistration, gold open access, and open data (< 25%). Predictors of knowledge and participation, as well as perceived barriers to implementation, are discussed.

CONCLUSION

Although participation in open science appears low in the field of CSD, participants expressed a strong desire to learn more in order to engage in these practices in the future. Supplemental Material and Open Science Form: https://doi.org/10.23641/asha.21569040.

Open Science in eating disorders: Using current evidence to inspire a plan for increasing the transparency of our research

OBJECTIVE

There is increasing consensus that open science practices improve the transparency and quality of clinical science. However, several barriers impede the implementation of these practices at the individual, institutional, and field levels; understanding and addressing these barriers is critical to promoting targeted efforts in increasing effective uptake of open science.

METHODS

Within this research forum, we drew from publicly available online information sources to identify initial characterizations of researchers engaged in several types of open science practices in the field of eating disorders. We use these observations to discuss potential barriers and recommendations for next steps in the promotion of these practices.

RESULTS

Data from online open science repositories suggest that individuals using these publishing approaches with pre-prints and articles with eating-disorder-relevant content are predominantly non-male gender identifying, early to mid-career stage, and are more likely to be European-, United States-, or Canada-based.

DISCUSSION

We outline recommendations for tangible ways that the eating disorder field can support broad, increased uptake of open science practices, including supporting initiatives to increase knowledge and correct misconceptions; and prioritizing the development and accessibility of open science resources.

PUBLIC SIGNIFICANCE STATEMENT

The use of open science practices has the potential to increase the transparency and quality of clinical science. This Forum uses publicly sourced online data to characterize researchers engaged in open science practices in the field of eating disorders. These observations provide an important framework from which to discuss potential barriers to open science and recommendations for next steps in the promotion of these practices.

A manifesto for applying behavioural science

Recent years have seen a rapid increase in the use of behavioural science to address the priorities of public and private sector actors. There is now a vibrant ecosystem of practitioners, teams and academics building on each other's findings across the globe. Their focus on robust evaluation means we know that this work has had an impact on important issues such as antimicrobial resistance, educational attainment and climate change. However, several critiques have also emerged; taken together, they suggest that applied behavioural science needs to evolve further over its next decade. This manifesto for the future of applied behavioural science looks at the challenges facing the field and sets out ten proposals to address them. Meeting these challenges will mean that behavioural science is better equipped to help to build policies, products and services on stronger empirical foundations-and thereby address the world's crucial challenges.

Parental conditional regard: A meta-analysis

INTRODUCTION

Parental conditional regard involves parents giving or withdrawing affection and approval, depending on children's and adolescents' compliance with parental expectations, to shape behaviors and traits. Research grounded in self-determination theory suggests parental conditional regard harms psychological development. Using self-determination theory as a theoretical foundation for investigating outcomes associated with parental conditional regard, the present study consolidated meta-analytic associations between parental conditional regard and four theoretically important individual difference correlates: introjected self-regulation, contingent self-esteem, depressive symptoms, and relatedness.

METHODS

Following PRISMA guidelines, a systematic literature search was conducted using the PsycINFO, ProQuest, and EBSCO databases for English-language, peer-reviewed published studies and unpublished studies. Eligible studies reported an association between parental conditional regard and the four theoretically derived correlates or another correlate of interest in pre-adolescent children, adolescents, or young adults. The results were based on a random-effects model for meta-analyses and the Q statistic for moderator analyses.

RESULTS

Across 31 samples in total, greater parental conditional regard was significantly associated with more introjected regulation (r = .33), contingent self-esteem (r = .29), and level of depressive symptoms (r = .22); and less relatedness (r = -.24). Moderator results for parental conditional regard type found parental conditional regard's association with introjected regulation was significantly stronger for studies measuring giving regard (parental conditional positive regard) than withdrawing regard (parental conditional negative regard). The association of parental conditional regard with depressive symptoms was significantly stronger for studies measuring parental conditional negative regard than parental conditional positive regard.

CONCLUSIONS

The meta-analytic results provide theoretical and empirical support for the connections between self-determination and the impact of parental conditional regard.

Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project

The low reproducibility rate in social sciences has produced hesitation among researchers in accepting published findings at their face value. Despite the advent of initiatives to increase transparency in research reporting, the field is still lacking tools to verify the credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research and allow their peers to verify this credibility. We demonstrate the application of these methods in a multi-laboratory replication of Bem's Experiment 1 (Bem 2011 J. Pers. Soc. Psychol. 100, 407-425. (doi:10.1037/a0021524)) on extrasensory perception (ESP), which was co-designed by a consensus panel including both proponents and opponents of Bem's original hypothesis. In the study we applied direct data deposition in combination with born-open data and real-time research reports to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video-documented trial sessions to ascertain as-intended protocol delivery, and external research auditors to monitor research integrity. We found 49.89% successful guesses, while Bem reported 53.07% success rate, with the chance level being 50%. Thus, Bem's findings were not replicated in our study. In the paper, we discuss the implementation, feasibility and perceived usefulness of the credibility-enhancing methodologies used throughout the project.

Expectations of efficient actions bias social perception: a pre-registered online replication

Humans take a teleological stance when observing others' actions, interpreting them as intentional and goal directed. In predictive processing accounts of social perception, this teleological stance would be mediated by a perceptual prediction of an ideal energy-efficient reference trajectory with which a rational actor would achieve their goals within the current environmental constraints. Hudson and colleagues (2018 Proc. R. Soc. B 285, 20180638. (doi:10.1098/rspb.2018.0638)) tested this hypothesis in a series of experiments in which participants reported the perceived disappearance points of hands reaching for objects. They found that these judgements were biased towards the expected efficient reference trajectories. Observed straight reaches were reported higher when an obstacle needed to be overcome than if the path was clear. By contrast, unnecessarily high reaches over empty space were perceptually flattened. Moreover, these perceptual biases increased the more the environmental constraints and expected action trajectories were explicitly processed. These findings provide an important advance to our understanding of the mechanisms underlying social perception. The current replication tests the robustness of these findings and whether they uphold in an online setting.

Predictive Models of Life Satisfaction in Older People: A Machine Learning Approach

Studies of life satisfaction in older adults have been conducted extensively through empirical research, questionnaires, and theoretical analysis, with the majority of these studies basing their analyses on simple linear relationships between variables. However, most real-life relationships are complex and cannot be approximated with simple correlations. Here, we first investigate predictors correlated with life satisfaction in older adults. Then, machine learning is used to generate several predictive models based on a large sample of older adults (age ≥ 50 years; n = 34,630) from the RAND Health and Retirement Study. Results show that subjective social status, positive emotions, and negative emotions are the most critical predictors of life satisfaction. The Support Vector Regression (SVR) model exhibited the highest prediction accuracy for life satisfaction in older individuals among several models, including Multiple Linear Regression (MLR), Ridge Regression (RR), Least Absolute Shrinkage and Selection Operator Regression (LASSO), K Nearest Neighbors (KNN), and Decision Tree Regression (DT) models. Although the KNN and DT models exhibited better model fitting than MLR, RR, and LASSO, their performances were poor in terms of model validation and model generalization. These results indicate that machine learning is superior to simple correlations for understanding life satisfaction among older adults.

The Emotional Rewards of Prosocial Spending Are Robust and Replicable in Large Samples

Past studies show that spending money on other people— prosocial spending—increases a person’s happiness. However, foundational research on this topic was conducted prior to psychology’s credibility revolution (or “replication crisis”), so it is essential to ask whether the evidence supporting this claim is robust and replicable. Here, we consider all 15 published preregistered experiments on prosocial spending to evaluate whether there is causal evidence for the idea that spending money on other people promotes happiness. Although the evidence appears somewhat mixed, we argue that the emotional benefits of prosocial spending are robust and replicable in large samples. These benefits are particularly likely when people have some choice about whether or how to give and when they understand how their generosity makes a difference. This review provides renewed support for the idea that prosocial spending promotes happiness and offers a template for revisiting phenomena that were established prior to the credibility revolution.

May a witness challenge the conviction? (Some) Confirmation bias in legal experts

Criminal investigations and trials are always guided by assumptions such that a suspect or defendant has committed a certain crime. Research on confirmation bias suggests that such prior assumptions bias subsequent information processing, fostering a confirmation of those assumptions. Biased information processing, in turn, would pose a severe threat to legal decision-making. Since previous evidence regarding professional decision makers is sparse and inconsistent, the present paper investigated whether legal experts showed evidence of confirmation bias. Specifically, we provided case materials pointing to a certain suspect and investigated evaluations of subsequent eyewitness evidence as a function of whether it was consistent or inconsistent with the initial suspicion. Although identical, witnessing conditions were rated as significantly poorer for inconsistent (vs. consistent) statements, thus, indicating confirmation bias. The effects were rather small, but this finding did not hinge on professional training, as another study with (law) students suggested. We argue that even small effects may threaten fair judgments and that our findings likely underestimate real-world effects.

Understanding the Motives for Terrorism-Does it Have an Effect on Psychological Reactions? A Replication and Extension

The collective communication model of terrorism (CCMT) proposes that understanding terrorists' motives influences appraisal (threat perception and emotional well-being) and reaction to terrorism (intention to retaliate). Fischer et al. (2011) presented evidence from two experiments for the assumption that understanding motives of terrorism influences appraisal. The present preregistered experiment aimed to replicate their second experiment, validate the measures they used, and also test the second proposition of the CCMT. Ensuring sufficient power for multiple tests and the given effect size, we collected data from 188 participants. The findings by Fischer et al. (2011) were partly replicated, but the comparison of the original effect sizes and the effect sizes from the replication attempt does not provide convincing evidence for the hypothesis that understanding the motives for terrorism reduces the perceived threat or negative emotional impact of acts of terrorism. Correlations with other risk-perception measures call into question the validity of the items used to assess perceived threat. Results suggest that understanding the motives for terrorism may influence whether the targeted populations want to retaliate.

The Artificial Intelligence System for the Generation of Sports Education Guidance Model and Physical Fitness Evaluation Under Deep Learning

In recent years, China's achievements in artificial intelligence (AI) have attracted the attention of the world, and AI technology has penetrated into all walks of life. In particular, the in-depth integration of AI technology with sports education guidance and physical fitness evaluation has achieved very significant progress and results, which has improved the quality of life of people and provided more high-quality, customized, and personalized health management services for human beings. This study aimed to explore the application model of deep learning in sports education and guidance and in the analysis of the residents' physical fitness, so as to formulate a personalized and intelligent exercise program. The residents of A and B units are selected as the research object to evaluate the physical fitness. Subsequently, the self-designed questionnaire is used to survey the chronic disease online, and the acquired data are put into a deep learning model for the analysis to obtain the physique scoring results and exercise guidance. According to the results of physical fitness evaluation, the proportion of overweight was the highest (40.4%), followed by fatty liver (24.3%) and hyperlipidemia (20.4%), showing high incidence in people aged 41-50 years. The highest incidence of female gynecological diseases was gout (23.0%) and hyperlipidemia (20.6%). After exercise therapy, the scores were excellent and good. Conclusions: The database SQL Server 2005 was a platform for storing all kinds of data and knowledge-based rule information. The user's access service was provided by the remote server via the browser. Therefore, building a rule-based reasoning mechanism can realize physical test data collection, physical fitness evaluation, and information management for improving physical fitness.

Accelerating Psychological Science With Metastudies: A Demonstration Using the Risky-Choice Framing Effect

A metastudy is a set of many tiny studies (microstudies) created from a much larger collection of possibilities. Metastudies can yield many of the benefits of time-consuming replications and meta-analyses but more efficiently and with greater attention to generalizability and the causal effects of moderators. Statistical precision and power are higher than in studies with the same total sample size but with fewer conditions and more participants per condition. In this article, we describe metastudies and their benefits, demonstrate how to conduct a metastudy using the well-known risky-choice framing effect as an example, and report on the generalizability of that effect. In three metastudies (total N = 2,338), the framing effect generalized well across most of the potential moderators tested, as was expected. Surprisingly, however, the effect was up to twice as large when the certain option was replaced with a slightly risky option; prospect theory predicts the opposite, and fuzzy-trace theory predicts no difference. Metastudies provide a relatively quick and not-so-painful way of examining an effect's generalizability without waiting for a meta-analysis. Both individual labs and multilab networks are encouraged to shift from traditional studies to metastudies.

Lessons from a catalogue of 6674 brain recordings

It is now possible for scientists to publicly catalogue all the data they have ever collected on one phenomenon. For a decade, we have been measuring a brain response to visual symmetry called the sustained posterior negativity (SPN). Here we report how we have made a total of 6674 individual SPNs from 2215 participants publicly available, along with data extraction and visualization tools (https://osf.io/2sncj/). We also report how re-analysis of the SPN catalogue has shed light on aspects of the scientific process, such as statistical power and publication bias, and revealed new scientific insights.