Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature

Preregistration in practice: A comparison of preregistered and non-preregistered studies in psychology

Preregistration has gained traction as one of the most promising solutions to improve the replicability of scientific effects. In this project, we compared 193 psychology studies that earned a Preregistration Challenge prize or preregistration badge to 193 related studies that were not preregistered. In contrast to our theoretical expectations and prior research, we did not find that preregistered studies had a lower proportion of positive results (Hypothesis 1), smaller effect sizes (Hypothesis 2), or fewer statistical errors (Hypothesis 3) than non-preregistered studies. Supporting our Hypotheses 4 and 5, we found that preregistered studies more often contained power analyses and typically had larger sample sizes than non-preregistered studies. Finally, concerns about the publishability and impact of preregistered studies seem unwarranted, as preregistered studies did not take longer to publish and scored better on several impact measures. Overall, our data indicate that preregistration has beneficial effects in the realm of statistical power and impact, but we did not find robust evidence that preregistration prevents p-hacking and HARKing (Hypothesizing After the Results are Known).

Power to Detect What? Considerations for Planning and Evaluating Sample Size

ACADEMIC ABSTRACT

In the wake of the replication crisis, social and personality psychologists have increased attention to power analysis and the adequacy of sample sizes. In this article, we analyze current controversies in this area, including choosing effect sizes, why and whether power analyses should be conducted on already-collected data, how to mitigate the negative effects of sample size criteria on specific kinds of research, and which power criterion to use. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings), given the limitations of interest-based minimums or field-wide effect sizes. We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology.

PUBLIC ABSTRACT

Recently, social-personality psychology has been criticized for basing some of its conclusions on studies with low numbers of participants. As a result, power analysis, a mathematical way to ensure that a study has enough participants to reliably "detect" a given size of psychological effect, has become popular. This article describes power analysis and discusses some controversies about it, including how researchers should derive assumptions about effect size, and how the requirements of power analysis can be applied without harming research on hard-to-reach and marginalized communities. For novel research questions, we advocate that researchers base sample sizes on effects that are likely to be cost-effective for other people to implement (in applied settings) or to study (in basic research settings). We discuss two alternatives to power analysis, precision analysis and sequential analysis, and end with recommendations for improving the practices of researchers, reviewers, and journal editors in social-personality psychology.

Subjective logic as a complementary tool to meta-analysis to explicitly address second-order uncertainty in research findings: A case from infant studies

Any experiment brings about results and conclusions that necessarily have a component of uncertainty. Many factors influence the degree of this uncertainty, yet they can be overlooked when drawing conclusions from a body of research. Here, we showcase how subjective logic could be employed as a complementary tool to meta-analysis to incorporate the chosen sources of uncertainty into the answer that researchers seek to provide to their research question. We illustrate this approach by focusing on a body of research already meta-analyzed, whose overall aim was to assess if human infants prefer prosocial agents over antisocial agents. We show how each finding can be encoded as a subjective opinion, and how findings can be aggregated to produce an answer that explicitly incorporates uncertainty. We argue that a core feature and strength of this approach is its transparency in the process of factoring in uncertainty and reasoning about research findings. Subjective logic promises to be a powerful complementary tool to incorporate uncertainty explicitly and transparently in the evaluation of research.

Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis

Translation of spinal cord injury (SCI) therapeutics from pre-clinical animal studies into human studies is challenged by effect size variability, irreproducibility, and misalignment of evidence used by pre-clinical versus clinical literature. Clinical literature values reproducibility, with the highest grade evidence (class 1) consisting of meta-analysis demonstrating large therapeutic efficacy replicating across multiple studies. Conversely, pre-clinical literature values novelty over replication and lacks rigorous meta-analyses to assess reproducibility of effect sizes across multiple articles. Here, we applied modified clinical meta-analysis methods to pre-clinical studies, comparing effect sizes extracted from published literature to raw data on individual animals from these same studies. Literature-extracted data (LED) from numerical and graphical outcomes reported in publications were compared with individual animal data (IAD) deposited in a federally supported repository of SCI data. The animal groups from the IAD were matched with the same cohorts in the LED for a direct comparison. We applied random-effects meta-analysis to evaluate predictors of neuroconversion in LED versus IAD. We included publications with common injury models (contusive injuries) and standardized end-points (open field assessments). The extraction of data from 25 published articles yielded n = 1841 subjects, whereas IAD from these same articles included n = 2441 subjects. We observed differences in the number of experimental groups and animals per group, insufficient reporting of dropout animals, and missing information on experimental details. Meta-analysis revealed differences in effect sizes across LED versus IAD stratifications, for instance, severe injuries had the largest effect size in LED (standardized mean difference [SMD = 4.92]), but mild injuries had the largest effect size in IAD (SMD = 6.06). Publications with smaller sample sizes yielded larger effect sizes, while studies with larger sample sizes had smaller effects. The results demonstrate the feasibility of combining IAD analysis with traditional LED meta-analysis to assess effect size reproducibility in SCI.

Enabling analytical power calculations for multilevel models with autocorrelated errors through deriving and approximating the precision matrix

To unravel how within-person psychological processes fluctuate in daily life, and how these processes differ between persons, intensive longitudinal (IL) designs in which participants are repeatedly measured, have become popular. Commonly used statistical models for those designs are multilevel models with autocorrelated errors. Substantive hypotheses of interest are then typically investigated via statistical hypotheses tests for model parameters of interest. An important question in the design of such IL studies concerns the determination of the number of participants and the number of measurements per person needed to achieve sufficient statistical power for those statistical tests. Recent advances in computational methods and software have enabled the computation of statistical power using Monte Carlo simulations. However, this approach is computationally intensive and therefore quite restrictive. To ease power computations, we derive simple-to-use analytical formulas for multilevel models with AR(1) within-person errors. Analytic expressions for a model family are obtained via asymptotic approximations of all sample statistics in the precision matrix of the fixed effects. To validate this analytical approach to power computation, we compare it to the simulation-based approach via a series of Monte Carlo simulations. We find comparable performances making the analytic approach a useful tool for researchers that can drastically save them time and resources.

From E-Commerce to the Metaverse: A Neuroscientific Analysis of Digital Consumer Behavior

The growing interest in consumer behavior in the digital environment is leading scholars and companies to focus on consumer behavior and choices on digital platforms, such as the metaverse. On this immersive digital shopping platform, consumer neuroscience provides an optimal opportunity to explore consumers' emotions and cognitions. In this study, neuroscience techniques (EEG, SC, BVP) were used to compare emotional and cognitive aspects of shopping between metaverse and traditional e-commerce platforms. Participants were asked to purchase the same product once on a metaverse platform (Second Life, SL) and once via an e-commerce website (EC). After each task, questionnaires were administered to measure perceived enjoyment, informativeness, ease of use, cognitive effort, and flow. Statistical analyses were conducted to examine differences between SL and EC at the neurophysiological and self-report levels, as well as between different stages of the purchase process. The results show that SL elicits greater cognitive engagement than EC, but it is also more mentally demanding, with a higher workload and more memorization, and fails to elicit a strong positive emotional response, leading to a poorer shopping experience. These findings provide insights not only for digital-related consumer research but also for companies to improve their metaverse shopping experience. Before investing in the platform or creating a digital retail space, companies should thoroughly analyze it, focusing on how to enhance users' cognition and emotions, ultimately promoting a better consumer experience. Despite its limitations, this pilot study sheds light on the emotional and cognitive aspects of metaverse shopping and suggests potential for further research with a consumer neuroscience approach in the metaverse field.

Estimating the total variance explained by whole-brain imaging for zero-inflated outcomes

There is a dearth of statistical models that adequately capture the total signal attributed to whole-brain imaging features. The total signal is often widely distributed across the brain, with individual imaging features exhibiting small effect sizes for predicting neurobehavioral phenotypes. The challenge of capturing the total signal is compounded by the distribution of neurobehavioral data, particularly responses to psychological questionnaires, which often feature zero-inflated, highly skewed outcomes. To close this gap, we have developed a novel Variational Bayes algorithm that characterizes the total signal captured by whole-brain imaging features for zero-inflated outcomes. Our zero-inflated variance (ZIV) estimator estimates the fraction of variance explained (FVE) and the proportion of non-null effects (PNN) from large-scale imaging data. In simulations, ZIV demonstrates superior performance over other linear models. When applied to data from the Adolescent Brain Cognitive DevelopmentSM (ABCD) Study, we found that whole-brain imaging features contribute to a larger FVE for externalizing behaviors compared to internalizing behaviors. Moreover, focusing on features contributing to the PNN, ZIV estimator localized key neurocircuitry associated with neurobehavioral traits. To the best of our knowledge, the ZIV estimator is the first specialized method for analyzing zero-inflated neuroimaging data, enhancing future studies on brain-behavior relationships and improving the understanding of neurobehavioral disorders.

Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task

Empirical studies reporting low test-retest reliability of individual blood oxygen-level dependent (BOLD) signal estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMRI. Over the last decade, several individual studies have reported that modeling decisions, such as smoothing, motion correction and contrast selection, may improve estimates of test-retest reliability of BOLD signal estimates. However, it remains an empirical question whether certain analytic decisions consistently improve individual and group level reliability estimates in an fMRI task across multiple large, independent samples. This study used three independent samples (Ns: 60, 81, 119) that collected the same task (Monetary Incentive Delay task) across two runs and two sessions to evaluate the effects of analytic decisions on the individual (intraclass correlation coefficient [ICC(3,1)]) and group (Jaccard/Spearman rho) reliability estimates of BOLD activity of task fMRI data. The analytic decisions in this study vary across four categories: smoothing kernel (five options), motion correction (four options), task parameterizing (three options) and task contrasts (four options), totaling 240 different pipeline permutations. Across all 240 pipelines, the median ICC estimates are consistently low, with a maximum median ICC estimate of .43 - .55 across the three samples. The analytic decisions with the greatest impact on the median ICC and group similarity estimates are the Implicit Baseline contrast, Cue Model parameterization and a larger smoothing kernel. Using an Implicit Baseline in a contrast condition meaningfully increased group similarity and ICC estimates as compared to using the Neutral cue. This effect was largest for the Cue Model parameterization; however, improvements in reliability came at the cost of interpretability. This study illustrates that estimates of reliability in the MID task are consistently low and variable at small samples, and a higher test-retest reliability may not always improve interpretability of the estimated BOLD signal.

No association between error-related ERPs and trait anxiety in a nonclinical sample: Convergence across analytical methods including mass-univariate statistics

Enhanced error monitoring, as indexed by increased amplitude of the error-related negativity (ERN) event-related potential (ERP) component, has been suggested to reflect a vulnerability neuro-marker of anxiety disorders. Another error-related ERP component is the error positivity (Pe), which reflects late-stage error processing. The associations between heightened ERN and Pe amplitudes and anxiety levels in the nonclinical population have been inconsistent. In this preregistered study, we examined the association between anxiety, ERN, and Pe, using different analytical methods (mass-univariate analyses, MUAs and conventional analyses), self-reported anxiety scales (STAI and STICSA), and trial numbers (all correct trials and equal numbers of correct and error trials). In a sample of 82 healthy adults, both conventional and MUAs demonstrated a robust enhancement of the ERN and Pe to errors relative to the correct-response ERPs. However, the mass-univariate approach additionally unveiled a wider array of electrodes and a longer effect duration for this error enhancement. Across the analytic methods, the results showed a lack of consistent correlation between trait anxiety and error-related ERPs. Findings were not modulated by trial numbers, analyses, or anxiety scales. The present results suggest a lack of enhancement of error monitoring by anxious traits in individuals with subclinical anxiety and those with clinical anxiety but without a clinical diagnosis. Importantly, the absence of such correlation questions the validity of the ERN as a neural marker for anxiety disorders. Future studies that investigate neuro-markers of anxiety may explore alternative task designs and employ robust statistics to provide a more comprehensive understanding of anxiety vulnerability.

Research Integrity definitions and challenges

Research integrity is guided by a set of principles to ensure research reliability and rigor. It serves as a pillar to uphold society's trust in science and foster scientific progress. However, over the past 2 decades, a surge in research integrity concerns, including fraudulent research, reproducibility challenges, and questionable practices, has raised critical questions about the reliability of scientific outputs, particularly in biomedical research. In the biomedical sciences, any breaches in research integrity could potentially lead to a domino effect impacting patient care, medical interventions, and the broader implementation of healthcare policies. Addressing these breaches requires measures such as rigorous research methods, transparent reporting, and changing the research culture. Institutional support through clear guidelines, robust training, and mentorship is crucial to fostering a culture of research integrity. However, structural and institutional factors, including research incentives and recognition systems, play an important role in research behavior. Therefore, promoting research integrity demands a collective effort from all stakeholders to maintain public trust in the scientific community and ensure the reliability of science. Here we discuss some definitions and principles, the implications for biomedical sciences, and propose actionable steps to foster research integrity.

Network-based artificial intelligence approaches for advancing personalized psychiatry

Psychiatric disorders have a complex biological underpinning likely involving an interplay of genetic and environmental risk contributions. Substantial efforts are being made to use artificial intelligence approaches to integrate features within and across data types to increase our etiological understanding and advance personalized psychiatry. Network science offers a conceptual framework for exploring the often complex relationships across different levels of biological organization, from cellular mechanistic to brain-functional and phenotypic networks. Utilizing such network information effectively as part of artificial intelligence approaches is a promising route toward a more in-depth understanding of illness biology, the deciphering of patient heterogeneity, and the identification of signatures that may be sufficiently predictive to be clinically useful. Here, we present examples of how network information has been used as part of artificial intelligence within psychiatry and beyond and outline future perspectives on how personalized psychiatry approaches may profit from a closer integration of psychiatric research, artificial intelligence development, and network science.

Psychological and physiological effects of soundscapes: A systematic review of 25 experiments in the English and Chinese literature

The aim of this systematic review was to conduct a comprehensive and rigorous investigation of both psychological and physiological responses to, and audio-visual interactions with, soundscapes to present an overview of the current status and to provide suggestions for future research. Our literature search focused on empirical and quantitative studies of journal articles and gray literature in English and Chinese. This systematic review excluded literature related to pure music, religious sounds, humanistic sounds, historical sounds, medical research, and differences in materials used. The Joanna Briggs Institute's Checklist for Randomized Controlled Trials was used to assess the risk of bias in the included studies. Twenty-five studies were included, involving 1950 participants. The major findings of this systematic review were that: (1) there were significant associations between the psychological and physiological responses; (2) the audio-visual interaction affected the psychological and physiological responses; and (3) because of the high risk of bias of the included studies, interpretation of their findings should be cautious. Nevertheless, given that this systematic review has a higher level of evidence than a single study and the synthesized evidence identified in this review is aligned with the results of other studies, the studies reviewed herein together provide consistent evidence. Replications are important in empirical research to build trustworthy results. Future research should focus on the psychological responses of pleasantness, preference, tranquility, the eight semantic dimensions (ISO 12913-2:2018), and the 11 pairs of adjectives describing the soundscape (Ba et al., 2023) and the physiological responses of heart rate variability and salivary, and follow the CONSORT guidelines to improve the research quality. An integration of sensory modalities, environmental factors, contextual indicators, temporal data, demographic variables, socio-cultural factors, and psychological and physiological responses may provide deeper insights into how people experience and understand the acoustic environment in context.

A short-interval longitudinal study of associations between psychological distress and hippocampal grey matter in early adolescence

This study of Australian adolescents (N = 88, 12-13-years-old) investigated the relationship between hippocampal grey matter volume (GMV) and self-reported psychological distress (K10) at four timepoints, across 12 months. Participants were divided into two groups; those who had K10 scores between 10 and 15 for all four timepoints were categorised as "low distress" (i.e., control group; n = 38), while participants who had K10 scores of 16 or higher at least once over the year were categorised as "moderate-high distress" (n = 50). Associations were tested by GEE fitting of GMV and K10 measures at the same time point, and in the preceding and subsequent timepoints. Analyses revealed smaller preceding left GMV and larger preceding right GMV were associated with higher subsequent K10 scores in the "moderate-high distress" group. This was not observed in the control group. In contrast, the control group showed significant co-occurring associations (i.e., at the same TP) between GMV and K10 scores. The "moderate-high distress" group experienced greater variability in distress. These results suggest that GMV development in early adolescence is differently associated with psychological distress for those who experience "moderate-high distress" at some point over the year, compared to controls. These findings offer a novel way to utilise short-interval, multiple time-point longitudinal data to explore changes in volume and experience of psychological distress in early adolescents. The results suggest hippocampal volume in early adolescence may be linked to fluctuations in psychological distress.

Campbell's Law Explains the Replication Crisis: Pre-Registration Badges Are History Repeating

Campbell's Law explains the replication crisis. In brief, useful tools such as hypotheses, p-values, and multi-study designs came to be viewed as indicators of strong science, and thus goals in and of themselves. Consequently, their use became distorted in unanticipated ways (e.g., hypothesizing after results were known [HARKing], p-Hacking, misuses of researcher degrees of freedom), and fragile findings proliferated. Pre-registration mandates are positioned as an antidote. However, I argue that such efforts, perhaps best exemplified by pre-registration badges (PRBs), are history repeating: Another useful tool has been converted into an indicator of strong science and a goal in and of itself. This, too, will distort its use and harm psychological science in unanticipated ways. For example, there is already evidence that papers seeking PRBs routinely violate the rules and spirit of pre-registration. I suggest that pre-registration mandates will (a) discourage optimal scientific practice, (b) exacerbate the file drawer problem, (c) encourage pre-registering after results are known (PRARKing), and (d) create false trust in fragile findings. I conclude that multiple design features can help support replicability (e.g., adequate sample size, valid measurement, robustness checks, pre-registration), none should be canonized, replication is the only arbiter of replicability, and the most important solution is sociocultural: to foster a field that reveres and reinforces robust science-just as we once revered and reinforced flashy but fragile science.

Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach

Surface facial electromyography (EMG) is commonly used to detect emotions from subtle facial expressions. Although there are established procedures for collecting EMG data and some aspects of their processing, there is little agreement among researchers about the optimal way to process the EMG signal, so that the study-unrelated variability (noise) is removed, and the emotion-related variability is best detected. The aim of the current paper was to establish an optimal processing pipeline for EMG data for identifying emotional expressions in facial muscles. We identified the most common processing steps from existing literature and created 72 processing pipelines that represented all the different processing choices. We applied these pipelines to a previously published dataset from a facial mimicry experiment, where 100 adult participants observed happy and sad facial expressions, whilst the activity of their facial muscles, zygomaticus major and corrugator supercilii, was recorded with EMG. We used a resampling approach and subsets of the original data to investigate the effect and robustness of different processing choices on the performance of a logistic regression model that predicted the mimicked emotion (happy/sad) from the EMG signal. In addition, we used a random forest model to identify the most important processing steps for the sensitivity of the logistic regression model. Three processing steps were found to be most impactful: baseline correction, standardisation within muscles, and standardisation within subjects. The chosen feature of interest and the signal averaging had little influence on the sensitivity to the effect. We recommend an optimal processing pipeline, share our code and data, and provide a step-by-step walkthrough for researchers.

Contamination-Focussed Vignettes as an Analogue of Infectious Pandemics: An Experimental Validation using the State Disgust and Anxiety Responses in OCD

Despite infectious pandemics proving particularly detrimental to those with Obsessive-Compulsive Disorder (OCD), the investigation of analogous experimental paradigms is lacking. To address this gap, we conducted two studies employing vignettes that depicted contamination-related situations commonly experienced during a pandemic (e.g., Coughing into hands and failing to use hand sanitizer). We manipulated the salience of these vignettes across three levels: high contamination, low contamination, and a neutral control condition. Our examination of state anxiety and disgust responses in all participants revealed the successful manipulation of the vignettes' impact. Specifically, individuals with more severe OCD symptoms reported significantly higher levels of state disgust and anxiety for both high and low contamination vignettes, in contrast to the group with lower symptom severity. No significant differences were observed in the neutral vignette condition between the high- and low-scoring groups. Interestingly, for those with higher OCD symptoms, high salience contamination-focused vignettes resulted in similarly elevated state disgust and anxiety, regardless of whether the vignettes were situated in public (Study 1) or domestic (Study 2) settings. This suggests that the heightened sensitivity to contamination-related scenarios observed in individuals with OCD symptoms in the present study is not confined to a specific context. These findings support the use of contamination-focused vignettes as analogues for studying infectious pandemics and provide valuable insights into OCD models, interventions, and future research.

Deep learning models reveal replicable, generalizable, and behaviorally relevant sex differences in human functional brain organization

Sex plays a crucial role in human brain development, aging, and the manifestation of psychiatric and neurological disorders. However, our understanding of sex differences in human functional brain organization and their behavioral consequences has been hindered by inconsistent findings and a lack of replication. Here, we address these challenges using a spatiotemporal deep neural network (stDNN) model to uncover latent functional brain dynamics that distinguish male and female brains. Our stDNN model accurately differentiated male and female brains, demonstrating consistently high cross-validation accuracy (>90%), replicability, and generalizability across multisession data from the same individuals and three independent cohorts (N ~ 1,500 young adults aged 20 to 35). Explainable AI (XAI) analysis revealed that brain features associated with the default mode network, striatum, and limbic network consistently exhibited significant sex differences (effect sizes > 1.5) across sessions and independent cohorts. Furthermore, XAI-derived brain features accurately predicted sex-specific cognitive profiles, a finding that was also independently replicated. Our results demonstrate that sex differences in functional brain dynamics are not only highly replicable and generalizable but also behaviorally relevant, challenging the notion of a continuum in male-female brain organization. Our findings underscore the crucial role of sex as a biological determinant in human brain organization, have significant implications for developing personalized sex-specific biomarkers in psychiatric and neurological disorders, and provide innovative AI-based computational tools for future research.

EEGManyPipelines: A Large-scale, Grassroots Multi-analyst Study of Electroencephalography Analysis Practices in the Wild

The ongoing reproducibility crisis in psychology and cognitive neuroscience has sparked increasing calls to re-evaluate and reshape scientific culture and practices. Heeding those calls, we have recently launched the EEGManyPipelines project as a means to assess the robustness of EEG research in naturalistic conditions and experiment with an alternative model of conducting scientific research. One hundred sixty-eight analyst teams, encompassing 396 individual researchers from 37 countries, independently analyzed the same unpublished, representative EEG data set to test the same set of predefined hypotheses and then provided their analysis pipelines and reported outcomes. Here, we lay out how large-scale scientific projects can be set up in a grassroots, community-driven manner without a central organizing laboratory. We explain our recruitment strategy, our guidance for analysts, the eventual outputs of this project, and how it might have a lasting impact on the field.

Exploring a formal approach to selecting studies for replication: A feasibility study in social neuroscience

Replication of published results is crucial for ensuring the robustness and self-correction of research, yet replications are scarce in many fields. Replicating researchers will therefore often have to decide which of several relevant candidates to target for replication. Formal strategies for efficient study selection have been proposed, but none have been explored for practical feasibility - a prerequisite for validation. Here we move one step closer to efficient replication study selection by exploring the feasibility of a particular selection strategy that estimates replication value as a function of citation impact and sample size (Isager, van 't Veer, & Lakens, 2021). We tested our strategy on a sample of fMRI studies in social neuroscience. We first report our efforts to generate a representative candidate set of replication targets. We then explore the feasibility and reliability of estimating replication value for the targets in our set, resulting in a dataset of 1358 studies ranked on their value of prioritising them for replication. In addition, we carefully examine possible measures, test auxiliary assumptions, and identify boundary conditions of measuring value and uncertainty. We end our report by discussing how future validation studies might be designed. Our study demonstrates the importance of investigating how to implement study selection strategies in practice. Our sample and study design can be extended to explore the feasibility of other formal study selection strategies that have been proposed.

fNIRS a novel neuroimaging tool to investigate olfaction, olfactory imagery, and crossmodal interactions: a systematic review

Olfaction is understudied in neuroimaging research compared to other senses, but there is growing evidence of its therapeutic benefits on mood and well-being. Olfactory imagery can provide similar health benefits as olfactory interventions. Harnessing crossmodal visual-olfactory interactions can facilitate olfactory imagery. Understanding and employing these cross-modal interactions between visual and olfactory stimuli could aid in the research and applications of olfaction and olfactory imagery interventions for health and wellbeing. This review examines current knowledge, debates, and research on olfaction, olfactive imagery, and crossmodal visual-olfactory integration. A total of 56 papers, identified using the PRISMA method, were evaluated to identify key brain regions, research themes and methods used to determine the suitability of fNIRS as a tool for studying these topics. The review identified fNIRS-compatible protocols and brain regions within the fNIRS recording depth of approximately 1.5 cm associated with olfactory imagery and crossmodal visual-olfactory integration. Commonly cited regions include the orbitofrontal cortex, inferior frontal gyrus and dorsolateral prefrontal cortex. The findings of this review indicate that fNIRS would be a suitable tool for research into these processes. Additionally, fNIRS suitability for use in naturalistic settings may lead to the development of new research approaches with greater ecological validity compared to existing neuroimaging techniques.

Alpha/beta oscillations reveal cognitive and affective brain states associated with role taking in a dyadic cooperative game

Social cooperation often requires taking different roles in order to reach a shared goal. By defining individual tasks, these roles dictate processing demands of the collaborators. The main aim of the present study was to examine the hypothesis that induced alpha and lower beta oscillations provide insights into affective and cognitive brain states during social cooperation. Toward this end, an experimental game was used in which participants had to navigate a Pacman figure through a maze by sending and receiving information about the correct moving direction. Supporting our hypotheses, individual roles taken by the collaborators during gameplay were associated with significant changes in alpha and lower beta power. Furthermore, effects were similar when participants played the Pacman Game with human or computer partners. Findings are discussed from the perspective of the information-via-desynchronization hypothesis proposing that alpha and lower beta power decreases reflect states of enhanced cortical information representation. Overall, experimental games are a useful tool for extending basic research on brain oscillations to the domain of naturalistic social interaction as emphasized by the second-person neuroscience perspective.

Better Accuracy for Better Science . . . Through Random Conclusions

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.

Redefining effect size interpretations for psychotherapy RCTs in depression

INTRODUCTION

Effect sizes are often used to interpret the magnitude of a result and in power calculations when planning research studies. However, as effect size interpretations are context-dependent, Jacob Cohen's suggested guidelines for what represents a small, medium, and large effect are unlikely to be suitable for a diverse range of research populations and interventions. Our objective here is to determine empirically-derived effect size thresholds associated with psychotherapy randomized controlled trials (RCTs) in depression by calculating the effect size distribution.

METHODS

We extracted effect sizes from 366 RCTs provided by the systematic review of Cuijpers and colleagues (2020) on psychotherapy for depressive disorders across all age groups. The 50th percentile effect size, as this represents a medium effect size, and the 25th (small) and 75th (large) percentile effect sizes were calculated to determine empirically-derived effect size thresholds.

RESULTS

After adjusting for publication bias, 0.27, 0.53, and 0.86 represent small, medium, and large effect sizes, respectively, for psychotherapy treatment for depressive disorders.

DISCUSSION

The effect size distribution for psychotherapy treatment of depression indicates that observed effect size thresholds are larger than Cohen's suggested effect size thresholds (0.2, 0.5, and 0.8). These results have implications for the interpretation of study effects and the planning of future studies via power analyses, which often use effect size thresholds.

Developmental psychologists should adopt citizen science to improve generalization and reproducibility

Widespread failures of replication and generalization are, ironically, a scientific triumph, in that they confirm the fundamental metascientific theory that underlies our field. Generalizable and replicable findings require testing large numbers of subjects from a wide range of demographics with a large, randomly-sampled stimulus set, and using a variety of experimental parameters. Because few studies accomplish any of this, meta-scientists predict that findings will frequently fail to replicate or generalize. We argue that to be more robust and replicable, developmental psychology needs to find a mechanism for collecting data at greater scale and from more diverse populations. Luckily, this mechanism already exists: Citizen science, in which large numbers of uncompensated volunteers provide data. While best-known for its contributions to astronomy and ecology, citizen science has also produced major findings in neuroscience and psychology, and increasingly in developmental psychology. We provide examples, address practical challenges, discuss limitations, and compare to other methods of obtaining large datasets. Ultimately, we argue that the range of studies where it makes sense *not* to use citizen science is steadily dwindling.

Beyond the language network: Associations between reading, receptive vocabulary, and grey matter volume in 10-year-olds

Most research on the neurostructural basis of language abilities in children stems from small samples and surface-based measures. To complement and expand the existent knowledge, we investigated associations between grey matter volume and language performance in a large sample of 9-to-11-year-old children, using data from the Adolescent Brain Cognitive Development (ABCD) Study (N = 1865) and an alternative measure of grey matter morphology. We estimated whole-brain grey matter volume for one half of the sample (N = 939) and tested for correlations with scores on a picture vocabulary and a letter and word reading test, with and without factoring in general intelligence and total grey matter volume as additional covariates. The initial analyses yielded correlations between grey matter in the right occipital fusiform gyrus, the right lingual gyrus, and the cerebellum for both vocabulary and reading. Employing the significant clusters from the first analyses as regions of interest in the second half of the cohort (N = 926) in correlational and multiple regression analyses suggests the cluster in the right occipital fusiform and lingual gyri to be most robust. Overall, the amount of variance explained by grey matter volume is limited and factoring in additional covariates paints an inconsistent picture. The present findings reinforce existent doubt with respect to explaining individual differences in reading and vocabulary performance based on unique contributions of macrostructural brain features.

"Losses disguised as wins" in electronic gambling machines contribute to win overestimation in a large online sample

Introduction

Losses disguised as wins (LDWs) are a salient type of losing outcome common to electronic gambling machines (EGMs). These events occur when a gambling payout is less than the amount wagered (i.e., a net loss) but is nonetheless accompanied by the sounds and animations that accompany genuine wins. Previous lab-based studies have reported that participants tend to overestimate genuine wins when LDWs are present. This study reports an independent replication of these findings in a large online sample that included a substantial number of individuals reporting high-risk gambling and frequent EGM users.

Methods

This online study recruited a sample of 940 participants who were randomly assigned to view one of two brief videos. Each video displayed a short period of simulated online slot machine gambling and included 2 genuine wins and either 3 or 0 LDWs. Participants were asked to estimate the number of times a win occurred that was more than the amount bet. Participants also completed the Problem Gambling Severity Index.

Results

The mean estimated number of genuine wins was significantly larger for the condition displaying LDWs, 3.02 [95% CI = 2.82, 3.21] than the control condition, 2.14 [1.98, 2.30], t(887.66) = 6.78, d = 0.44, p <.001.

Conclusions

We replicated the LDW-triggered win overestimation effect previously reported in lab-based experiments that have recruited smaller samples. This effect was robust in both low-risk and high-risk groups, indicating that even experienced gamblers remain susceptible. Exploratory modelling suggested only a minority of individuals were uninfluenced by LDWs.

The influence of learning history on anterograde interference

Classically interpreted as a competition between opposite memories (A vs B), anterograde interference (AI) also emerges in the absence of competing memories (A vs A), suggesting that mechanisms other than those involved in memory competition contribute to AI. To investigate this, we tested the hypothesis that extending motor practice would enhance a first memory, but come at the cost of reduced learning capabilities when subsequently exposed to a second learning session of the same task. Based on converging biological evidence, AI was expected to depend upon the degree of extended practice of the initial exposure. During a first Session, four conditions were carried out where participants (n = 24) adapted to a gradually introduced -20° visual deviation while the extent of the initial exposure was manipulated by varying the duration or type of the performance asymptote. Specifically, the performance asymptote at -20° was either Short (40 trials), Moderate (160 trials), Long (320 trials), or absent due to continuously changing perturbations around the mean of -20° (Jagged; 160 trials). After a 2-min interval, participants re-adapted to the same (-20°) visual deviation, which was meant to probe the effect of extended practice in the first Session on the learning capabilities of a second identical memory (A vs A). The results first confirmed that the duration of exposure in the first Session enhanced immediate aftereffects in the Moderate, Long, and Jagged conditions as compared to the Short condition, suggesting that extended practice enhanced retention of the first memory. When comparing the second Session to the first one, results revealed a different pattern of re-adaptation depending on the duration of initial exposure: in the Short condition, there was evidence for facilitated re-adaptation and similar aftereffects. However, in the Moderate, Long and Jagged conditions, re-adaptation was similar and aftereffects were impaired, suggestive of AI. This suggests that extended practice initially enhances memory formation, but comes at the cost of reduced subsequent learning capabilities. One possibility is that AI occurs because extended practice induces the emergence of network-specific homeostatic constraints, which limit subsequent neuroplastic and learning capabilities in the same neural network.

Assessing the Evidential Value of Mental Fatigue and Exercise Research

It has often been reported that mental exertion, presumably leading to mental fatigue, can negatively affect exercise performance; however, recent findings have questioned the strength of the effect. To further complicate this issue, an overlooked problem might be the presence of publication bias in studies using underpowered designs, which is known to inflate false positive report probability and effect size estimates. Altogether, the presence of bias is likely to reduce the evidential value of the published literature on this topic, although it is unknown to what extent. The purpose of the current work was to assess the evidential value of studies published to date on the effect of mental exertion on exercise performance by assessing the presence of publication bias and the observed statistical power achieved by these studies. A traditional meta-analysis revealed a Cohen's dz effect size of - 0.54, 95% CI [- 0.68, - 0.40], p < .001. However, when we applied methods for estimating and correcting for publication bias (based on funnel plot asymmetry and observed p-values), we found that the bias-corrected effect size became negligible with most of publication-bias methods and decreased to - 0.36 in the more optimistic of all the scenarios. A robust Bayesian meta-analysis found strong evidence in favor of publication bias, BFpb > 1000, and inconclusive evidence in favor of the effect, adjusted dz = 0.01, 95% CrI [- 0.46, 0.37], BF10 = 0.90. Furthermore, the median observed statistical power assuming the unadjusted meta-analytic effect size (i.e., - 0.54) as the true effect size was 39% (min = 19%, max = 96%), indicating that, on average, these studies only had a 39% chance of observing a significant result if the true effect was Cohen's dz = - 0.54. If the more optimistic adjusted effect size (- 0.36) was assumed as the true effect, the median statistical power was just 20%. We conclude that the current literature is a useful case study for illustrating the dangers of conducting underpowered studies to detect the effect size of interest.

Elevated levels of mixed-hand preference in dyslexia: Meta-analyses of 68 studies

Since almost a hundred years, psychologists have investigated the link between hand preference and dyslexia. We present a meta-analysis to determine whether there is indeed an increase in atypical hand preference in dyslexia. We included studies used in two previous meta-analyses (Bishop, 1990; Eglinton & Annett, 1994) as well as studies identified through PubMed MEDLINE, PsycInfo, Google Scholar, and Web of Science up to August 2022. K = 68 studies (n = 4660 individuals with dyslexia; n = 40845 controls) were entered into three random effects meta-analyses using the odds ratio as the effect size (non-right-handers; left-handers; mixed-handers vs. total). Evidence of elevated levels of atypical hand preference in dyslexia emerged that were especially pronounced for mixed-hand preference (OR = 1.57), although this category was underdefined. Differences in (direction or degree) of hand skill or degree of hand preference could not be assessed as no pertinent studies were located. Our findings allow for robust conclusions only for a relationship of mixed-hand preference with dyslexia.

Drawing causal inferences from randomized controlled trials in psychotherapy research

OBJECTIVE

Randomized control trials (RCTs) have played a critical role in psychotherapy research. This article discusses RCTs in the context of the criteria for drawing causal inferences in psychotherapy and intervention research more generally. The article also highlights underused variations of RCTs and how they not only establish causal relations but also address critical questions that can improve our intervention portfolio and patient care.

CONCLUSION

Random assignment is discussed in terms of what it can and cannot do in relation to drawing conclusions about the effects of interventions. Finally, RCTs are examined in the context of multiple questions that can guide therapy research, improve patient care, and develop treatments that reach people in need of psychological services.

Are most published research findings false? Trends in statistical power, publication selection bias, and the false discovery rate in psychology (1975-2017)

The validity of scientific findings may be challenged by the replicability crisis (or cases of fraud), which may result not only in a loss of trust within society but may also lead to wrong or even harmful policy or medical decisions. The question is: how reliable are scientific results that are reported as statistically significant, and how does this reliability develop over time? Based on 35,515 papers in psychology published between 1975 and 2017 containing 487,996 test values, this article empirically examines the statistical power, publication bias, and p-hacking, as well as the false discovery rate. Assuming constant true effects, the statistical power was found to be lower than the suggested 80% except for large underlying true effects (d = 0.8) and increased only slightly over time. Also, publication bias and p-hacking were found to be substantial. The share of false discoveries among all significant results was estimated at 17.7%, assuming a proportion θ = 50% of all hypotheses being true and assuming that p-hacking is the only mechanism generating a higher proportion of just significant results compared to just nonsignificant results. As the analyses rely on multiple assumptions that cannot be tested, alternative scenarios were laid out, again resulting in the rather optimistic result that although research results may suffer from low statistical power and publication selection bias, most of the results reported as statistically significant may contain substantial results, rather than statistical artifacts.

The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices: Five years later

Changes in statistical practices and reporting have been documented by Giofrè et al. PLOS ONE 12(4), e0175583 (2017), who investigated ten statistical and open practices in two high-ranking journals (Psychological Science [PS] and Journal of Experimental Psychology-General [JEPG]): null hypothesis significance testing; confidence or credible intervals; meta-analysis of the results of multiple experiments; confidence interval interpretation; effect size interpretation; sample size determination; data exclusion; data availability; materials availability; and preregistered design and analysis plan. The investigation was based on an analysis of all papers published in these journals between 2013 and 2015. The aim of the present study was to follow up changes in both PS and JEPG in subsequent years, from 2016 to 2020, adding code availability as a further open practice. We found improvement in most practices, with some exceptions (i.e., confidence interval interpretation and meta-analysis). Despite these positive changes, our results indicate a need for further improvements in statistical practices and adoption of open practices.

Multivariate prediction of cognitive performance from the sleep electroencephalogram

Human cognitive performance is a key function whose biological foundations have been partially revealed by genetic and brain imaging studies. The sleep electroencephalogram (EEG) is tightly linked to structural and functional features of the central nervous system and serves as another promising biomarker. We used data from MrOS, a large cohort of older men and cross-validated regularized regression to link sleep EEG features to cognitive performance in cross-sectional analyses. In independent validation samples 2.5-10% of variance in cognitive performance can be accounted for by sleep EEG features, depending on the covariates used. Demographic characteristics account for more covariance between sleep EEG and cognition than health variables, and consequently reduce this association by a greater degree, but even with the strictest covariate sets a statistically significant association is present. Sigma power in NREM and beta power in REM sleep were associated with better cognitive performance, while theta power in REM sleep was associated with worse performance, with no substantial effect of coherence and other sleep EEG metrics. Our findings show that cognitive performance is associated with the sleep EEG (r = 0.283), with the strongest effect ascribed to spindle-frequency activity. This association becomes weaker after adjusting for demographic (r = 0.186) and health variables (r = 0.155), but its resilience to covariate inclusion suggest that it also partially reflects trait-like differences in cognitive ability.

The Best-Case Heuristic: Relative Optimism in Relationships, Politics, and a Global Health Pandemic

In four experiments covering three different life domains, participants made future predictions in what they considered the most realistic scenario, an optimistic best-case scenario, or a pessimistic worst-case scenario (N = 2,900 Americans). Consistent with a best-case heuristic, participants made "realistic" predictions that were much closer to their best-case scenario than to their worst-case scenario. We found the same best-case asymmetry in health-related predictions during the COVID-19 pandemic, for romantic relationships, and a future presidential election. In a fully between-subject design (Experiment 4), realistic and best-case predictions were practically identical, and they were naturally made faster than the worst-case predictions. At least in the current study domains, the findings suggest that people generate "realistic" predictions by leaning toward their best-case scenario and largely ignoring their worst-case scenario. Although political conservatism was correlated with lower covid-related risk perception and lower support of early public-health interventions, the best-case prediction heuristic was ideologically symmetric.

Improving the stability of bivariate correlations using informative Bayesian priors: a Monte Carlo simulation study

Objective

Much of psychological research has suffered from small sample sizes and low statistical power, resulting in unstable parameter estimates. The Bayesian approach offers a promising solution by incorporating prior knowledge into statistical models, which may lead to improved stability compared to a frequentist approach.

Methods

Simulated data from four populations with known bivariate correlations (ρ = 0.1, 0.2, 0.3, 0.4) was used to estimate the sample correlation as samples were sequentially added from the population, from n = 10 to n = 500. The impact of three different, subjectively defined prior distributions (weakly, moderately, and highly informative) was investigated and compared to a frequentist model.

Results

The results show that bivariate correlation estimates are unstable, and that the risk of obtaining an estimate that is exaggerated or in the wrong direction is relatively high, for sample sizes for below 100, and considerably so for sample sizes below 50. However, this instability can be constrained by informative Bayesian priors.

Conclusion

Informative Bayesian priors have the potential to significantly reduce sample size requirements and help ensure that obtained estimates are in line with realistic expectations. The combined stabilizing and regularizing effect of a weakly informative prior is particularly useful when conducting research with small samples. The impact of more informative Bayesian priors depends on one's threshold for probability and whether one's goal is to obtain an estimate merely in the correct direction, or to obtain a high precision estimate whose associated interval falls within a narrow range. Implications for sample size requirements and directions for future research are discussed.

The Brain Disorders Debate, Chekhov, and Mental Health Humanities

The contemporary brain disorders debate echoes a century-long conflict between two different approaches to mental suffering: one that relies on natural sciences and another drawing from the arts and humanities. We review contemporary neuroimaging studies and find that neither side has won. The study of mental differences needs both the sciences and the arts and humanities. To help develop an approach mindful of both, we turn to physician-writer Anton Chekhov's story "A Nervous Breakdown." We review the value of the arts and humanities as a coequal partner with natural sciences in the creation of a robust mental health humanities.

Exploring the multiverse: the impact of researchers' analytic decisions on relationships between depression and inflammatory markers

In recent years, a replication crisis in psychiatry has led to a growing focus on the impact of researchers' analytic decisions on the results from studies. Multiverse analyses involve examining results across a wide array of possible analytic decisions (e.g., log-transforming variables, number of covariates, or treatment of outliers) and identifying if study results are robust to researchers' analytic decisions. Studies have begun to use multiverse analysis for well-studied relationships that have some heterogeneity in results/conclusions across studies.We examine the well-studied relationship between peripheral inflammatory markers (PIMs; e.g., white blood cell count (WBC) and C-reactive protein (CRP)) and depression severity in the large NHANES dataset (n = 25,962). Specification curve analyses tested the impact of 9 common analytic decisions (comprising of 58,000+ possible combinations) on the association of PIMs and depression severity. Relationships of PIMs and total depression severity are robust to analytic decisions (based on tests of inference jointly examining effect sizes and p-values). However, moderate/large differences are noted in effect sizes based on analytic decisions and the majority of analyses do not result in significant findings, with the percentage of analyses with statistically significant results being 46.1% for WBC and 43.8% for CRP. For associations of PIMs with specific symptoms of depression, some associations (e.g., sleep, appetite) in males (but not females) were robust to analytic decisions. We discuss how multiverse analyses can be used to guide research and also the need for authors, reviewers, and editors to incorporate multiverse analyses to enhance replicability of research findings.

Estimating the Total Variance Explained by Whole-Brain Imaging for Zero-inflated Outcomes

Zero-inflated outcomes are very common in behavioral data, particularly for responses to psychological questionnaires. Modeling these challenging distributions is further exacerbated by the absence of established statistical models capable of characterizing total signals attributed to whole-brain imaging features, making the accurate assessment of brain-behavior relationships particularly formidable. Given this critical need, we have developed a novel variational Bayes algorithm that characterizes the total signal captured by whole-brain imaging features for zero-inflated outcomes . Our zero-inflated variance (ZIV) estimator robustly estimates the fraction of variance explained (FVE) and the proportion of non-null effects from large-scale imaging data. In simulations, ZIV outperformed other linear prediction algorithms. Applying ZIV to data from one of the largest neuroimaging studies, the Adolescent Brain Cognitive Development SM (ABCD) Study, we found that whole-brain imaging features have a larger FVE for externalizing compared to internalizing behavior. We also demonstrate that the ZIV estimator, especially applied to focal sub-scales, can localize key neurocircuitry associated with human behavior.

How the Complexity of Psychological Processes Reframes the Issue of Reproducibility in Psychological Science

In the past decade, various recommendations have been published to enhance the methodological rigor and publication standards in psychological science. However, adhering to these recommendations may have limited impact on the reproducibility of causal effects as long as psychological phenomena continue to be viewed as decomposable into separate and additive statistical structures of causal relationships. In this article, we show that (a) psychological phenomena are patterns emerging from nondecomposable and nonisolable complex processes that obey idiosyncratic nonlinear dynamics, (b) these processual features jeopardize the chances of standard reproducibility of statistical results, and (c) these features call on researchers to reconsider what can and should be reproduced, that is, the psychological processes per se, and the signatures of their complexity and dynamics. Accordingly, we argue for a greater consideration of process causality of psychological phenomena reflected by key properties of complex dynamical systems (CDSs). This implies developing and testing formal models of psychological dynamics, which can be implemented by computer simulation. The scope of the CDS paradigm and its convergences with other paradigms are discussed regarding the reproducibility issue. Ironically, the CDS approach could account for both reproducibility and nonreproducibility of the statistical effects usually sought in mainstream psychological science.

Enhancing precision in human neuroscience

Human neuroscience has always been pushing the boundary of what is measurable. During the last decade, concerns about statistical power and replicability - in science in general, but also specifically in human neuroscience - have fueled an extensive debate. One important insight from this discourse is the need for larger samples, which naturally increases statistical power. An alternative is to increase the precision of measurements, which is the focus of this review. This option is often overlooked, even though statistical power benefits from increasing precision as much as from increasing sample size. Nonetheless, precision has always been at the heart of good scientific practice in human neuroscience, with researchers relying on lab traditions or rules of thumb to ensure sufficient precision for their studies. In this review, we encourage a more systematic approach to precision. We start by introducing measurement precision and its importance for well-powered studies in human neuroscience. Then, determinants for precision in a range of neuroscientific methods (MRI, M/EEG, EDA, Eye-Tracking, and Endocrinology) are elaborated. We end by discussing how a more systematic evaluation of precision and the application of respective insights can lead to an increase in reproducibility in human neuroscience.

The ups and downs of bilingualism: A review of the literature on executive control using event-related potentials

Whether bilingualism enhances executive control (EC) is controversial. This article reviews 24 studies on the bilingual EC effect using event-related potentials (ERPs). It evaluates the evidence based on considerations of neural efficiency, different EC theories, and accounts regarding the locus of the bilingual effect. The review finds some evidence for a positive bilingual impact. This is more consistent for the P3 and response-locked ERPs. Moreover, when considering each component independently, evidence primarily supports a monitoring and secondarily an inhibition locus. Additionally, an N2/ERN (error-related negativity) dissociation (no bilingual N2 effect but positive ERN impact, evident as smaller ERN), coupled with the P3 results, suggest that monitoring may not be the (only) locus of a bilingual effect but (an)other post-monitoring mechanism(s). Attention disengagement also receives some support. Finally, results across studies are largely consistent with the Bilingualism Anterior to Posterior and Subcortical Shift model (BAPSS): Bilingual effects, when found, often manifest as shorter latencies, larger components or wider amplitude effects during earlier (N2, P3) but smaller components or narrower effects during later processing (stimulus-locked negativities and response-locked components). However, this evidence is not unequivocal. Many bilingual-monolingual comparisons reveal null or some suggest negative or opposite to prediction bilingual effects. Second, the scant evidence about which bilingual experiences impact EC is, generally, unclear, while some evidence indicates negative effects. Third, BAPSS is often not confirmed when multiple components are examined within subjects. Finally, this literature is challenged by confounds and small samples. Further research is required to conclude a positive bilingual effect on EC in ERPs.

NIDM-Terms: community-based terminology management for improved neuroimaging dataset descriptions and query

The biomedical research community is motivated to share and reuse data from studies and projects by funding agencies and publishers. Effectively combining and reusing neuroimaging data from publicly available datasets, requires the capability to query across datasets in order to identify cohorts that match both neuroimaging and clinical/behavioral data criteria. Critical barriers to operationalizing such queries include, in part, the broad use of undefined study variables with limited or no annotations that make it difficult to understand the data available without significant interaction with the original authors. Using the Brain Imaging Data Structure (BIDS) to organize neuroimaging data has made querying across studies for specific image types possible at scale. However, in BIDS, beyond file naming and tightly controlled imaging directory structures, there are very few constraints on ancillary variable naming/meaning or experiment-specific metadata. In this work, we present NIDM-Terms, a set of user-friendly terminology management tools and associated software to better manage individual lab terminologies and help with annotating BIDS datasets. Using these tools to annotate BIDS data with a Neuroimaging Data Model (NIDM) semantic web representation, enables queries across datasets to identify cohorts with specific neuroimaging and clinical/behavioral measurements. This manuscript describes the overall informatics structures and demonstrates the use of tools to annotate BIDS datasets to perform integrated cross-cohort queries.

Social and asocial learning in zebrafish are encoded by a shared brain network that is differentially modulated by local activation

Group living animals use social and asocial cues to predict the presence of reward or punishment in the environment through associative learning. The degree to which social and asocial learning share the same mechanisms is still a matter of debate. We have used a classical conditioning paradigm in zebrafish, in which a social (fish image) or an asocial (circle image) conditioned stimulus (CS) have been paired with an unconditioned stimulus (US=food), and we have used the expression of the immediate early gene c-fos to map the neural circuits associated with each learning type. Our results show that the learning performance is similar to social and asocial CSs. However, the brain regions activated in each learning type are distinct and a community analysis of brain network data reveals segregated functional submodules, which seem to be associated with different cognitive functions involved in the learning tasks. These results suggest that, despite localized differences in brain activity between social and asocial learning, they share a common learning module and social learning also recruits a specific social stimulus integration module. Therefore, our results support the occurrence of a common general-purpose learning module, that is differentially modulated by localized activation in social and asocial learning.

A multi-lab experimental assessment reveals that replicability can be improved by using empirical estimates of genotype-by-lab interaction

The utility of mouse and rat studies critically depends on their replicability in other laboratories. A widely advocated approach to improving replicability is through the rigorous control of predefined animal or experimental conditions, known as standardization. However, this approach limits the generalizability of the findings to only to the standardized conditions and is a potential cause rather than solution to what has been called a replicability crisis. Alternative strategies include estimating the heterogeneity of effects across laboratories, either through designs that vary testing conditions, or by direct statistical analysis of laboratory variation. We previously evaluated our statistical approach for estimating the interlaboratory replicability of a single laboratory discovery. Those results, however, were from a well-coordinated, multi-lab phenotyping study and did not extend to the more realistic setting in which laboratories are operating independently of each other. Here, we sought to test our statistical approach as a realistic prospective experiment, in mice, using 152 results from 5 independent published studies deposited in the Mouse Phenome Database (MPD). In independent replication experiments at 3 laboratories, we found that 53 of the results were replicable, so the other 99 were considered non-replicable. Of the 99 non-replicable results, 59 were statistically significant (at 0.05) in their original single-lab analysis, putting the probability that a single-lab statistical discovery was made even though it is non-replicable, at 59.6%. We then introduced the dimensionless "Genotype-by-Laboratory" (GxL) factor-the ratio between the standard deviations of the GxL interaction and the standard deviation within groups. Using the GxL factor reduced the number of single-lab statistical discoveries and alongside reduced the probability of a non-replicable result to be discovered in the single lab to 12.1%. Such reduction naturally leads to reduced power to make replicable discoveries, but this reduction was small (from 87% to 66%), indicating the small price paid for the large improvement in replicability. Tools and data needed for the above GxL adjustment are publicly available at the MPD and will become increasingly useful as the range of assays and testing conditions in this resource increases.

Publication bias impacts on effect size, statistical power, and magnitude (Type M) and sign (Type S) errors in ecology and evolutionary biology

Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the 'replication crisis'. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power. This registered report assesses the prevalence and severity of small-study (i.e., smaller studies reporting larger effect sizes) and decline effects (i.e., effect sizes decreasing over time) across ecology and evolutionary biology using 87 meta-analyses comprising 4,250 primary studies and 17,638 effect sizes. Further, we estimate how publication bias might distort the estimation of effect sizes, statistical power, and errors in magnitude (Type M or exaggeration ratio) and sign (Type S). We show strong evidence for the pervasiveness of both small-study and decline effects in ecology and evolution. There was widespread prevalence of publication bias that resulted in meta-analytic means being over-estimated by (at least) 0.12 standard deviations. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4). Notably, publication bias reduced power from 23% to 15% and increased type M error rates from 2.7 to 4.4 because it creates a non-random sample of effect size evidence. The sign errors of effect sizes (Type S error) increased from 5% to 8% because of publication bias. Our research provides clear evidence that many published ecological and evolutionary findings are inflated. Our results highlight the importance of designing high-power empirical studies (e.g., via collaborative team science), promoting and encouraging replication studies, testing and correcting for publication bias in meta-analyses, and adopting open and transparent research practices, such as (pre)registration, data- and code-sharing, and transparent reporting.

The Garden of Forking Paths; An Evaluation of Joseph's 'A Reevaluation of the 1990 "Minnesota Study of Twins Reared Apart" IQ Study'

Joseph has written what purports to be a refutation of studies of Twins Reared-Apart (TRAs) with a singular focus on the Minnesota Study of Twins Reared-Apart (MISTRA). I show, in detail, that (a) his criticisms of previous TRA studies depend on sources that were discredited prior to MISTRA, as they all failed the test of replicability, (b) the list of biases he uses to invalidate MISTRA do not support his arguments, (c) the accusations of questionable research practices are unsubstantiated, (d) his claim that MISTRA should be evaluated in the context of psychology's replication crisis is refuted. The TRA studies are constructive replications. Like many other scholars, past and present, he has been misled by the variation introduced by small samples (sampling error) and the distortion created by walking in the garden of forking paths. His endeavor is a concatenation of elision and erroneous statistical/scientific reasoning.

Time is of the essence: past selves are not prioritized even when selective discrimination costs are controlled for

The perceptual Self-Prioritization effect (SPE) refers to an advantage in attending to stimuli associated with the self relative to those associated with another individual. In the perceptual matching task, arbitrary pairings between oneself and other persons, and a geometric shape need to be learned. Apart from the SPE, this task also produces high matching performance for a close other. While cognitive representations of past selves are sometimes viewed as resembling that of an intimate other, and while there is some evidence that other types of psychological closeness modulate the SPE, it remains unclear whether such prioritization effects extend to past selves. Previous experiments on this topic required participants to distinguish between different points in time within the same task, raising the possibility that potential past self-prioritization was masked by task difficulty. In our experiment, we addressed this potential confound by presenting N = 118 participants with a simpler version of the matching task. We re-investigated self-prioritization in perceptual matching under conditions of mental time travel to the past. In line with previous evidence, we found clear prioritization of present selves, which was evident in response times, accuracies and the efficiency of practice. Performance was consistently poorest for the past self, indicating not only a lack of privileged processing, but rather a relative de-prioritization. Performance was not improved by either increased proximity of the time period in question, nor by experimenter-induced re-imagining of the self. Our results do not support a perceptual prioritization of past selves.

Evolutionarily conserved role of oxytocin in social fear contagion in zebrafish

Emotional contagion is the most ancestral form of empathy. We tested to what extent the proximate mechanisms of emotional contagion are evolutionarily conserved by assessing the role of oxytocin, known to regulate empathic behaviors in mammals, in social fear contagion in zebrafish. Using oxytocin and oxytocin receptor mutants, we show that oxytocin is both necessary and sufficient for observer zebrafish to imitate the distressed behavior of conspecific demonstrators. The brain regions associated with emotional contagion in zebrafish are homologous to those involved in the same process in rodents (e.g., striatum, lateral septum), receiving direct projections from oxytocinergic neurons located in the pre-optic area. Together, our results support an evolutionary conserved role for oxytocin as a key regulator of basic empathic behaviors across vertebrates.

Acetaminophen changes Mu rhythm power related to pain empathy

Empathy is an intricate ability that entails the subjective feeling and understanding of emotions someone else may be experiencing. Acetaminophen, the active ingredient found in Tylenol, is among the most common pain medications consumed. There is new evidence, however, that suggests this common analgesic may also dampen empathic processes. However, no previous study has investigated the effect acetaminophen may have on pain empathy or mu power during a pain empathy task. Therefore, participants were randomly assigned to either an experimental (acetaminophen) or control (sugar) group in a double-blinded experimental research design aimed to measure mu power (using EEG) and behavioral responses to painful and non-painful images. Participants in the experimental group were administered 1000 mg of acetaminophen, and it was verified that participants were unaware which group they were assigned. We found that mu suppression was greater in the acetaminophen group, which was strongest at electrode C3. Additionally, mu power differences between painful and non-painful images were related to trait empathy, and mu power during the painful images were positively correlated with empathy scores. Results from this study suggest that in addition to reducing physical pain, acetaminophen may also change the brain response when perceiving others in pain. The implications of these findings could possibly lead to changes in how we prescribe and administer this common drug.