Progress toward openness, transparency, and reproducibility in cognitive neuroscience

Topological cluster statistic (TCS): Toward structural connectivity-guided fMRI cluster enhancement

Functional magnetic resonance imaging (fMRI) studies most commonly use cluster-based inference to detect local changes in brain activity. Insufficient statistical power and disproportionate false-positive rates reportedly hinder optimal inference. We propose a structural connectivity-guided clustering framework, called topological cluster statistic (TCS), that enhances sensitivity by leveraging white matter anatomical connectivity information. TCS harnesses multimodal information from diffusion tractography and functional imaging to improve task fMRI activation inference. Compared to conventional approaches, TCS consistently improves power over a wide range of effects. This improvement results in a 10%-50% increase in local sensitivity with the greatest gains for medium-sized effects. TCS additionally enables inspection of underlying anatomical networks and thus uncovers knowledge regarding the anatomical underpinnings of brain activation. This novel approach is made available in the PALM software to facilitate usability. Given the increasing recognition that activation reflects widespread, coordinated processes, TCS provides a way to integrate the known structure underlying widespread activations into neuroimaging analyses moving forward.

The reverberation of implementation errors in a neuroimaging meta-analytic software package: A citation analysis to a technical report on GingerALE

GingerALE, a widely used neuroimaging meta-analysis software package, contained errors in earlier versions that were later corrected. The technical report "Implementation errors in the GingerALE Software: description and recommendations" by Eickhoff et al. (2017) documented these errors and their corresponding fixes. In the current study, the papers that cited the GingerALE technical report were analyzed to identify the reasons for these citations. In August 2023, a search through Web of Science Core Collection identified 158 papers that cited the GingerALE technical report. These papers were manually examined to extract the citation statements and code the citation reasons into 12 categories. The analysis revealed that the most frequent reason for citing the report was to justify the use of a specific statistical threshold, followed by a simple acknowledgement of using GingerALE, acknowledging the impact of the errors in earlier versions of GingerALE on prior studies or the lack of effect on current results, and justifying the number of experiments in a meta-analysis. A small number of reasons related to non-GingerALE software, matters not related to activation likelihood estimation (ALE), or statements not mentioned in the GingerALE technical report.

Hemispheric Asymmetry in TMS-Induced Effects on Spatial Attention: A Meta-Analysis

Hemispheric asymmetry is a fundamental principle in the functional architecture of the brain. It plays an important role in attention research where right hemisphere dominance is core to many attention theories. Lesion studies seem to confirm such hemispheric dominance with patients being more likely to develop left hemineglect after right hemispheric stroke than vice versa. However, the underlying concept of hemispheric dominance is still not entirely clear. Brain stimulation studies using transcranial magnetic stimulation (TMS) might be able to illuminate this concept. To examine the putative hemispheric asymmetry in spatial attention, we conducted a meta-analysis of studies applying inhibitory TMS protocols to the left or right posterior parietal cortices (PPC), assessing effects on attention biases with the landmark and line bisection task. A total of 18 studies including 222 participants from 1994 to February 2022 were identified. The analysis revealed a significant shift of the perceived midpoint towards the ipsilateral hemifield after right PPC suppression (Cohen's d = 0.52), but no significant effect after left PPC suppression (Cohen's d = 0.26), suggesting a hemispheric asymmetry even though the subgroup difference does not reach significance (p = .06). A complementary Bayesian meta-analysis revealed a high probability of at least a medium effect size after right PPC disruption versus a low probability after left PPC disruption. This is the first quantitative meta-analysis supporting right hemisphere-specific TMS-induced spatial attention deficits, mimicking hemineglect in healthy participants. We discuss the result in the light of prominent attention theories, ultimately concluding how difficult it remains to differentiate between these theories based on attentional bias scores alone.

Overlooked poor-quality patient samples in sequencing data impair reproducibility of published clinically relevant datasets

BACKGROUND

Reproducibility is a major concern in biomedical studies, and existing publication guidelines do not solve the problem. Batch effects and quality imbalances between groups of biological samples are major factors hampering reproducibility. Yet, the latter is rarely considered in the scientific literature.

RESULTS

Our analysis uses 40 clinically relevant RNA-seq datasets to quantify the impact of quality imbalance between groups of samples on the reproducibility of gene expression studies. High-quality imbalance is frequent (14 datasets; 35%), and hundreds of quality markers are present in more than 50% of the datasets. Enrichment analysis suggests common stress-driven effects among the low-quality samples and highlights a complementary role of transcription factors and miRNAs to regulate stress response. Preliminary ChIP-seq results show similar trends. Quality imbalance has an impact on the number of differential genes derived by comparing control to disease samples (the higher the imbalance, the higher the number of genes), on the proportion of quality markers in top differential genes (the higher the imbalance, the higher the proportion; up to 22%) and on the proportion of known disease genes in top differential genes (the higher the imbalance, the lower the proportion). We show that removing outliers based on their quality score improves the resulting downstream analysis.

CONCLUSIONS

Thanks to a stringent selection of well-designed datasets, we demonstrate that quality imbalance between groups of samples can significantly reduce the relevance of differential genes, consequently reducing reproducibility between studies. Appropriate experimental design and analysis methods can substantially reduce the problem.

Transparency and reproducibility in the Adolescent Brain Cognitive Development (ABCD) study

BACKGROUND

Transparency can build trust in the scientific process, but scientific findings can be undermined by poor and obscure data use and reporting practices. The purpose of this work is to report how data from the Adolescent Brain Cognitive Development (ABCD) Study has been used to date, and to provide practical recommendations on how to improve the transparency and reproducibility of findings.

METHODS

Articles published from 2017 to 2023 that used ABCD Study data were reviewed using more than 30 data extraction items to gather information on data use practices. Total frequencies were reported for each extraction item, along with computation of a Level of Completeness (LOC) score that represented overall endorsement of extraction items. Univariate linear regression models were used to examine the correlation between LOC scores and individual extraction items. Post hoc analysis included examination of whether LOC scores were correlated with the logged 2-year journal impact factor.

RESULTS

There were 549 full-length articles included in the main analysis. Analytic scripts were shared in 30 % of full-length articles. The number of participants excluded due to missing data was reported in 60 % of articles, and information on missing data for individual variables (e.g., household income) was provided in 38 % of articles. A table describing the analytic sample was included in 83 % of articles. A race and/or ethnicity variable was included in 78 % of reviewed articles, while its inclusion was justified in only 41 % of these articles. LOC scores were highly correlated with extraction items related to examination of missing data. A bottom 10 % of LOC score was significantly correlated with a lower logged journal impact factor when compared to the top 10 % of LOC scores (β=-0.77, 95 % -1.02, -0.51; p-value < 0.0001).

CONCLUSION

These findings highlight opportunities for improvement in future papers using ABCD Study data to readily adapt analytic practices for better transparency and reproducibility efforts. A list of recommendations is provided to facilitate adherence in future research.

Distributed Sensitivity to Syntax and Semantics throughout the Language Network

Human language is expressive because it is compositional: The meaning of a sentence (semantics) can be inferred from its structure (syntax). It is commonly believed that language syntax and semantics are processed by distinct brain regions. Here, we revisit this claim using precision fMRI methods to capture separation or overlap of function in the brains of individual participants. Contrary to prior claims, we find distributed sensitivity to both syntax and semantics throughout a broad frontotemporal brain network. Our results join a growing body of evidence for an integrated network for language in the human brain within which internal specialization is primarily a matter of degree rather than kind, in contrast with influential proposals that advocate distinct specialization of different brain areas for different types of linguistic functions.

Transparency and Reproducibility in the Adolescent Brain Cognitive Development (ABCD) Study

Background

Transparency can build trust in the scientific process, but scientific findings can be undermined by poor and obscure data use and reporting practices. The purpose of this work is to report how data from the Adolescent Brain Cognitive Development (ABCD) Study has been used to date, and to provide practical recommendations on how to improve the transparency and reproducibility of findings.

Methods

Articles published from 2017 to 2023 that used ABCD Study data were reviewed using more than 30 data extraction items to gather information on data use practices. Total frequencies were reported for each extraction item, along with computation of a Level of Completeness (LOC) score that represented overall endorsement of extraction items. Univariate linear regression models were used to examine the correlation between LOC scores and individual extraction items. Post hoc analysis included examination of whether LOC scores were correlated with the logged 2-year journal impact factor.

Results

There were 549 full-length articles included in the main analysis. Analytic scripts were shared in 30% of full-length articles. The number of participants excluded due to missing data was reported in 60% of articles, and information on missing data for individual variables (e.g., household income) was provided in 38% of articles. A table describing the analytic sample was included in 83% of articles. A race and/or ethnicity variable was included in 78% of reviewed articles, while its inclusion was justified in only 41% of these articles. LOC scores were highly correlated with extraction items related to examination of missing data. A bottom 10% of LOC score was significantly correlated with a lower logged journal impact factor when compared to the top 10% of LOC scores (β=-0.77, 95% -1.02, -0.51; p-value < 0.0001).

Conclusion

These findings highlight opportunities for improvement in future papers using ABCD Study data to readily adapt analytic practices for better transparency and reproducibility efforts. A list of recommendations is provided to facilitate adherence in future research.

Increasing Value and Reducing Waste of Research on Neurofeedback Effects in Post-traumatic Stress Disorder: A State-of-the-Art-Review

Post-Traumatic Stress Disorder (PTSD) is often considered challenging to treat due to factors that contribute to its complexity. In the last decade, more attention has been paid to non-pharmacological or non-psychological therapies for PTSD, including neurofeedback (NFB). NFB is a promising non-invasive technique targeting specific brainwave patterns associated with psychiatric symptomatology. By learning to regulate brain activity in a closed-loop paradigm, individuals can improve their functionality while reducing symptom severity. However, owing to its lax regulation and heterogeneous legal status across different countries, the degree to which it has scientific support as a psychiatric treatment remains controversial. In this state-of-the-art review, we searched PubMed, Cochrane Central, Web of Science, Scopus, and MEDLINE and identified meta-analyses and systematic reviews exploring the efficacy of NFB for PTSD. We included seven systematic reviews, out of which three included meta-analyses (32 studies and 669 participants) that targeted NFB as an intervention while addressing a single condition-PTSD. We used the MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 and the criteria described by Cristea and Naudet (Behav Res Therapy 123:103479, 2019, https://doi.org/10.1016/j.brat.2019.103479 ) to identify sources of research waste and increasing value in biomedical research. The seven assessed reviews had an overall extremely poor quality score (5 critically low, one low, one moderate, and none high) and multiple sources of waste while opening opportunities for increasing value in the NFB literature. Our research shows that it remains unclear whether NFB training is significantly beneficial in treating PTSD. The quality of the investigated literature is low and maintains a persistent uncertainty over numerous points, which are highly important for deciding whether an intervention has clinical efficacy. Just as importantly, none of the reviews we appraised explored the statistical power, referred to open data of the included studies, or adjusted their pooled effect sizes for publication bias and risk of bias. Based on the obtained results, we identified some recurrent sources of waste (such as a lack of research decisions based on sound questions or using an appropriate methodology in a fully transparent, unbiased, and useable manner) and proposed some directions for increasing value (homogeneity and consensus) in designing and reporting research on NFB interventions in PTSD.

A research agenda for understanding how social inequality is linked to brain structure and function

Consistent evidence documents powerful effects of social inequality on health, well-being and academic achievement. Yet research on whether social inequality may also be linked to brain structure and function has, until recently, been rare. Here we describe three methodological approaches that can be used to study this question-single site, single study; multi-site, single study; and spatial meta-analysis. We review empirical work that, using these approaches, has observed associations between neural outcomes and structural measures of social inequality-including structural stigma, community-level prejudice, gender inequality, neighbourhood disadvantage and the generosity of the social safety net for low-income families. We evaluate the relative strengths and limitations of these approaches, discuss ethical considerations and outline directions for future research. In doing so, we advocate for a paradigm shift in cognitive neuroscience that explicitly incorporates upstream structural and contextual factors, which we argue holds promise for uncovering the neural correlates of social inequality.

Few temporally distributed brain connectivity states predict human cognitive abilities

Human functional brain connectivity can be temporally decomposed into states of high and low cofluctuation, defined as coactivation of brain regions over time. Rare states of particularly high cofluctuation have been shown to reflect fundamentals of intrinsic functional network architecture and to be highly subject-specific. However, it is unclear whether such network-defining states also contribute to individual variations in cognitive abilities - which strongly rely on the interactions among distributed brain regions. By introducing CMEP, a new eigenvector-based prediction framework, we show that as few as 16 temporally separated time frames (< 1.5% of 10min resting-state fMRI) can significantly predict individual differences in intelligence (N = 263, p < .001). Against previous expectations, individual's network-defining time frames of particularly high cofluctuation do not predict intelligence. Multiple functional brain networks contribute to the prediction, and all results replicate in an independent sample (N = 831). Our results suggest that although fundamentals of person-specific functional connectomes can be derived from few time frames of highest connectivity, temporally distributed information is necessary to extract information about cognitive abilities. This information is not restricted to specific connectivity states, like network-defining high-cofluctuation states, but rather reflected across the entire length of the brain connectivity time series.

A shared neural basis underlying psychiatric comorbidity

Recent studies proposed a general psychopathology factor underlying common comorbidities among psychiatric disorders. However, its neurobiological mechanisms and generalizability remain elusive. In this study, we used a large longitudinal neuroimaging cohort from adolescence to young adulthood (IMAGEN) to define a neuropsychopathological (NP) factor across externalizing and internalizing symptoms using multitask connectomes. We demonstrate that this NP factor might represent a unified, genetically determined, delayed development of the prefrontal cortex that further leads to poor executive function. We also show this NP factor to be reproducible in multiple developmental periods, from preadolescence to early adulthood, and generalizable to the resting-state connectome and clinical samples (the ADHD-200 Sample and the Stratify Project). In conclusion, we identify a reproducible and general neural basis underlying symptoms of multiple mental health disorders, bridging multidimensional evidence from behavioral, neuroimaging and genetic substrates. These findings may help to develop new therapeutic interventions for psychiatric comorbidities.

Shedding light on functional near-infrared spectroscopy and open science practices

Open science practices work to increase methodological rigor, transparency, and replicability of published findings. We aim to reflect on what the functional near-infrared spectroscopy (fNIRS) community has done to promote open science practices in fNIRS research and set goals to accomplish over the next 10 years.

Reproducibility and Scientific Integrity of Big Data Research in Urban Public Health and Digital Epidemiology: A Call to Action

The emergence of big data science presents a unique opportunity to improve public-health research practices. Because working with big data is inherently complex, big data research must be clear and transparent to avoid reproducibility issues and positively impact population health. Timely implementation of solution-focused approaches is critical as new data sources and methods take root in public-health research, including urban public health and digital epidemiology. This commentary highlights methodological and analytic approaches that can reduce research waste and improve the reproducibility and replicability of big data research in public health. The recommendations described in this commentary, including a focus on practices, publication norms, and education, are neither exhaustive nor unique to big data, but, nonetheless, implementing them can broadly improve public-health research. Clearly defined and openly shared guidelines will not only improve the quality of current research practices but also initiate change at multiple levels: the individual level, the institutional level, and the international level.

Reproducing FSL's fMRI data analysis via Nipype: Relevance, challenges, and solutions

The "replication crisis" in neuroscientific research has led to calls for improving reproducibility. In traditional neuroscience analyses, irreproducibility may occur as a result of issues across various stages of the methodological process. For example, different operating systems, different software packages, and even different versions of the same package can lead to variable results. Nipype, an open-source Python project, integrates different neuroimaging software packages uniformly to improve the reproducibility of neuroimaging analyses. Nipype has the advantage over traditional software packages (e.g., FSL, ANFI, SPM, etc.) by (1) providing comprehensive software development frameworks and usage information, (2) improving computational efficiency, (3) facilitating reproducibility through sufficient details, and (4) easing the steep learning curve. Despite the rich tutorials it has provided, the Nipype community lacks a standard three-level GLM tutorial for FSL. Using the classical Flanker task dataset, we first precisely reproduce a three-level GLM analysis with FSL via Nipype. Next, we point out some undocumented discrepancies between Nipype and FSL functions that led to substantial differences in results. Finally, we provide revised Nipype code in re-executable notebooks that assure result invariability between FSL and Nipype. Our analyses, notebooks, and operating software specifications (e.g., docker build files) are available on the Open Science Framework platform.

Blueprints for measuring natural behavior

Until recently laboratory tasks for studying behavior were highly artificial, simplified, and designed without consideration for the environmental or social context. Although such an approach offers good control over behavior, it does not allow for researching either voluntary responses or individual differences. Importantly for neuroscience studies, the activity of the neural circuits involved in producing unnatural, artificial behavior is variable and hard to predict. In addition, different ensembles may be activated depending on the strategy the animal adopts to deal with the spurious problem. Thus, artificial and simplified tasks based on responses, which do not occur spontaneously entail problems with modeling behavioral impairments and underlying brain deficits. To develop valid models of human disorders we need to test spontaneous behaviors consistently engaging well-defined, evolutionarily conserved neuronal circuits. Such research focuses on behavioral patterns relevant for surviving and thriving under varying environmental conditions, which also enable high reproducibility across different testing settings.

Group-level inference of information-based measures for the analyses of cognitive brain networks from neurophysiological data

The reproducibility crisis in neuroimaging and in particular in the case of underpowered studies has introduced doubts on our ability to reproduce, replicate and generalize findings. As a response, we have seen the emergence of suggested guidelines and principles for neuroscientists known as Good Scientific Practice for conducting more reliable research. Still, every study remains almost unique in its combination of analytical and statistical approaches. While it is understandable considering the diversity of designs and brain data recording, it also represents a striking point against reproducibility. Here, we propose a non-parametric permutation-based statistical framework, primarily designed for neurophysiological data, in order to perform group-level inferences on non-negative measures of information encompassing metrics from information-theory, machine-learning or measures of distances. The framework supports both fixed- and random-effect models to adapt to inter-individuals and inter-sessions variability. Using numerical simulations, we compared the accuracy in ground-truth retrieving of both group models, such as test- and cluster-wise corrections for multiple comparisons. We then reproduced and extended existing results using both spatially uniform MEG and non-uniform intracranial neurophysiological data. We showed how the framework can be used to extract stereotypical task- and behavior-related effects across the population covering scales from the local level of brain regions, inter-areal functional connectivity to measures summarizing network properties. We also present an open-source Python toolbox called Frites¹ that includes the proposed statistical pipeline using information-theoretic metrics such as single-trial functional connectivity estimations for the extraction of cognitive brain networks. Taken together, we believe that this framework deserves careful attention as its robustness and flexibility could be the starting point toward the uniformization of statistical approaches.

Diversifying Participation: The Rarity of Reporting Racial Demographics in Neuroimaging Research

Functional neuroimaging has been instrumental to the field of cognitive neuroscience; however, its increasing prevalence has evoked conversations concerning limitations associated with reproducibility and bias. Prevailing racial, cultural, and socioeconomic biases in scientific research perpetuate demographic homogeneity in participation, contributing to failed replicability and generalizability and driving inaccurate representations of neurological normalcy. The current report employs systematic and exploratory search methods to investigate ongoing practices surrounding participant recruitment and documentation. The systematic search found that only 20 out of the 536 articles collected reported the race and ethnicity demographics of their participants, exposing a dearth of race and ethnicity demographics reporting in neuroimaging research. These results drive our recommendations for increased transparency and diversity surrounding research participation.

Open multimodal iEEG-fMRI dataset from naturalistic stimulation with a short audiovisual film

Intracranial human recordings are a valuable and rare resource of information about the brain. Making such data publicly available not only helps tackle reproducibility issues in science, it helps make more use of these valuable data. This is especially true for data collected using naturalistic tasks. Here, we describe a dataset collected from a large group of human subjects while they watched a short audiovisual film. The dataset has several unique features. First, it includes a large amount of intracranial electroencephalography (iEEG) data (51 participants, age range of 5-55 years, who all performed the same task). Second, it includes functional magnetic resonance imaging (fMRI) recordings (30 participants, age range of 7-47) during the same task. Eighteen participants performed both iEEG and fMRI versions of the task, non-simultaneously. Third, the data were acquired using a rich audiovisual stimulus, for which we provide detailed speech and video annotations. This dataset can be used to study neural mechanisms of multimodal perception and language comprehension, and similarity of neural signals across brain recording modalities.

Connectome spatial smoothing (CSS): Concepts, methods, and evaluation

Structural connectomes are increasingly mapped at high spatial resolutions comprising many hundreds-if not thousands-of network nodes. However, high-resolution connectomes are particularly susceptible to image registration misalignment, tractography artifacts, and noise, all of which can lead to reductions in connectome accuracy and test-retest reliability. We investigate a network analogue of image smoothing to address these key challenges. Connectome Spatial Smoothing (CSS) involves jointly applying a carefully chosen smoothing kernel to the two endpoints of each tractography streamline, yielding a spatially smoothed connectivity matrix. We develop computationally efficient methods to perform CSS using a matrix congruence transformation and evaluate a range of different smoothing kernel choices on CSS performance. We find that smoothing substantially improves the identifiability, sensitivity, and test-retest reliability of high-resolution connectivity maps, though at a cost of increasing storage burden. For atlas-based connectomes (i.e. low-resolution connectivity maps), we show that CSS marginally improves the statistical power to detect associations between connectivity and cognitive performance, particularly for connectomes mapped using probabilistic tractography. CSS was also found to enable more reliable statistical inference compared to connectomes without any smoothing. We provide recommendations for optimal smoothing kernel parameters for connectomes mapped using both deterministic and probabilistic tractography. We conclude that spatial smoothing is particularly important for the reliability of high-resolution connectomes, but can also provide benefits at lower parcellation resolutions. We hope that our work enables computationally efficient integration of spatial smoothing into established structural connectome mapping pipelines.

Dynamics of cumulative advantage and threats to equity in open science: a scoping review

Open Science holds the promise to make scientific endeavours more inclusive, participatory, understandable, accessible and re-usable for large audiences. However, making processes open will not per se drive wide reuse or participation unless also accompanied by the capacity (in terms of knowledge, skills, financial resources, technological readiness and motivation) to do so. These capacities vary considerably across regions, institutions and demographics. Those advantaged by such factors will remain potentially privileged, putting Open Science's agenda of inclusivity at risk of propagating conditions of 'cumulative advantage'. With this paper, we systematically scope existing research addressing the question: 'What evidence and discourse exists in the literature about the ways in which dynamics and structures of inequality could persist or be exacerbated in the transition to Open Science, across disciplines, regions and demographics?' Aiming to synthesize findings, identify gaps in the literature and inform future research and policy, our results identify threats to equity associated with all aspects of Open Science, including Open Access, Open and FAIR Data, Open Methods, Open Evaluation, Citizen Science, as well as its interfaces with society, industry and policy. Key threats include: stratifications of publishing due to the exclusionary nature of the author-pays model of Open Access; potential widening of the digital divide due to the infrastructure-dependent, highly situated nature of open data practices; risks of diminishing qualitative methodologies as 'reproducibility' becomes synonymous with quality; new risks of bias and exclusion in means of transparent evaluation; and crucial asymmetries in the Open Science relationships with industry and the public, which privileges the former and fails to fully include the latter.

The new normal? Redaction bias in biomedical science

A concerning amount of biomedical research is not reproducible. Unreliable results impede empirical progress in medical science, ultimately putting patients at risk. Many proximal causes of this irreproducibility have been identified, a major one being inappropriate statistical methods and analytical choices by investigators. Within this, we formally quantify the impact of inappropriate redaction beyond a threshold value in biomedical science. This is effectively truncation of a dataset by removing extreme data points, and we elucidate its potential to accidentally or deliberately engineer a spurious result in significance testing. We demonstrate that the removal of a surprisingly small number of data points can be used to dramatically alter a result. It is unknown how often redaction bias occurs in the broader literature, but given the risk of distortion to the literature involved, we suggest that it must be studiously avoided, and mitigated with approaches to counteract any potential malign effects to the research quality of medical science.

Conceptual expansion via novel metaphor processing: An ERP replication and extension study examining individual differences in creativity

The aims of the present ERP study were twofold. First, to determine whether a previous study on creative cognition could be replicated, and second, to extend these findings by examining individual differences in creativity. Conceptual expansion, a capacity that is central to creativity, was induced via the processing of novel metaphors. Brain activity patterns in relation to these were compared to the processing of literal and nonsense phrases. The previous findings were replicated in that the N400, known for its sensitivity to semantic anomalies, indexed the originality of the phrases, while a post-N400 late component (LC), which is linked to semantic integration processes, indexed the appropriateness of the phrases. Moreover, only the LC was significantly sensitive to individual differences in creativity in the processing of these phrases. Differences at the level of semantic integration processes as well as the structure of knowledge organization are thereby implicated in individual differences in creativity.

Can Open Science be a Tool to Dismantle Claims of Hardwired Brain Sex Differences? Opportunities and Challenges for Feminist Researchers

Feminist scholars have long been concerned with claims of hardwired brain sex differences emanating from neuroscience and evolutionary psychology. Past criticisms of these claims have rightfully questioned the impact of this research on gender equality, pointing out how findings can be used to vindicate gender stereotypes. In this article, we appraise the brain sex differences literature through the lens of open science, a movement aimed at improving the robustness and reliability of science. In this discussion, we offer a feminist evaluation of the strategies (e.g., pre-registration, data sharing, and accountability) provided by open science, and we question whether these may be the novel and disruptive tools needed to dismantle claims about hardwired brain sex differences. We suggest that open science strategies can be useful in challenging some of these claims, and we note that promising initiatives are already being developed in neuroscience and allied fields. We end by acknowledging the distinct challenges that feminist researchers wishing to engage in open science face, particularly in the context of limited diversity. We conclude that open science presents considerable opportunity for feminist researchers, and that it will be crucial for feminists to be involved in shaping the future of this movement.

Continuous-time deconvolutional regression for psycholinguistic modeling

The influence of stimuli in psycholinguistic experiments diffuses across time because the human response to language is not instantaneous. The linear models typically used to analyze psycholinguistic data are unable to account for this phenomenon due to strong temporal independence assumptions, while existing deconvolutional methods for estimating diffuse temporal structure model time discretely and therefore cannot be directly applied to natural language stimuli where events (words) have variable duration. In light of evidence that continuous-time deconvolutional regression (CDR) can address these issues (Shain & Schuler, 2018), this article motivates the use of CDR for many experimental settings, exposits some of its mathematical properties, and empirically evaluates the influence of various experimental confounds (noise, multicollinearity, and impulse response misspecification), hyperparameter settings, and response types (behavioral and fMRI). Results show that CDR (1) yields highly consistent estimates across a variety of hyperparameter configurations, (2) faithfully recovers the data-generating model on synthetic data, even under adverse training conditions, and (3) outperforms widely-used statistical approaches when applied to naturalistic reading and fMRI data. In addition, procedures for testing scientific hypotheses using CDR are defined and demonstrated, and empirically-motivated best-practices for CDR modeling are proposed. Results support the use of CDR for analyzing psycholinguistic time series, especially in a naturalistic experimental paradigm.

Can inhibition deficit hypothesis account for age-related differences in semantic fluency? Converging evidence from Stroop color and word test and an ERP flanker task

The inhibition deficit hypothesis (IDH) proposed that individual differences in inhibitory control is an underlying reason for age-related language decline. This study examined whether the hypothesis holds within the domain of lexico-semantic retrieval. Sixty-six older adults aged 60-79 were tested in a semantic fluency task comprising 16 categories; each response was classified as automatic or controlled. Also, Stroop color and word test and an ERP flanker task were employed to yield both behavioral and neural measures of inhibitory control. Mixed-effects modelling revealed that the number of controlled (but not automatic) responses was negatively associated with age. This interaction could be partially accounted for by the behavioral Stroop inhibition score and two neural measures from the ERP flanker task (P2 and Pc amplitudes). These results not only provide converging evidence supporting the IDH, but also demonstrate the involvement of specific inhibitory control components, including attentional control and performance monitoring.

Bridging the Brain and Data Sciences

Brain scientists are now capable of collecting more data in a single experiment than researchers a generation ago might have collected over an entire career. Indeed, the brain itself seems to thirst for more and more data. Such digital information not only comprises individual studies but is also increasingly shared and made openly available for secondary, confirmatory, and/or combined analyses. Numerous web resources now exist containing data across spatiotemporal scales. Data processing workflow technologies running via cloud-enabled computing infrastructures allow for large-scale processing. Such a move toward greater openness is fundamentally changing how brain science results are communicated and linked to available raw data and processed results. Ethical, professional, and motivational issues challenge the whole-scale commitment to data-driven neuroscience. Nevertheless, fueled by government investments into primary brain data collection coupled with increased sharing and community pressure challenging the dominant publishing model, large-scale brain and data science is here to stay.

Falsification and consciousness

The search for a scientific theory of consciousness should result in theories that are falsifiable. However, here we show that falsification is especially problematic for theories of consciousness. We formally describe the standard experimental setup for testing these theories. Based on a theory's application to some physical system, such as the brain, testing requires comparing a theory's predicted experience (given some internal observables of the system like brain imaging data) with an inferred experience (using report or behavior). If there is a mismatch between inference and prediction, a theory is falsified. We show that if inference and prediction are independent, it follows that any minimally informative theory of consciousness is automatically falsified. This is deeply problematic since the field's reliance on report or behavior to infer conscious experiences implies such independence, so this fragility affects many contemporary theories of consciousness. Furthermore, we show that if inference and prediction are strictly dependent, it follows that a theory is unfalsifiable. This affects theories which claim consciousness to be determined by report or behavior. Finally, we explore possible ways out of this dilemma.

Point estimates, Simpson's paradox, and nonergodicity in biological sciences

Modern biomedical, behavioral and psychological inference about cause-effect relationships respects an ergodic assumption, that is, that mean response of representative samples allow predictions about individual members of those samples. Recent empirical evidence in all of the same fields indicates systematic violations of the ergodic assumption. Indeed, violation of ergodicity in biomedical, behavioral and psychological causes is precisely the inspiration behind our research inquiry. Here, we review the long term costs to scientific progress in these domains and a practical way forward. Specifically, we advocate using statistical measures that can themselves encode the degree and type of nonergodicity in measurements. Taking such steps will lead to a paradigm shift, allowing researchers to investigate the nonstationary, far-from-equilibrium processes that characterize the creativity and emergence of biological and psychological behavior.

Data Triangulation in Consumer Neuroscience: Integrating Functional Neuroimaging With Meta-Analyses, Psychometrics, and Behavioral Data

This article reviews a wide range of functional magnetic resonance imaging (fMRI) studies conducted in the field of consumer neuroscience to (1) highlight common interpretative approaches of neuroimaging data (i.e., forward inference and reverse inference), (2) discuss potential interpretative issues associated with these approaches, and (3) provide a framework that employs a multi-method approach aimed to possibly raise the explanatory power and, thus, the validity of functional neuroimaging research in consumer neuroscience. Based on this framework, we argue that the validity of fMRI studies can be improved by the triangulation of (1) careful design of neuroimaging studies and analyses of data, (2) meta-analyses, and (3) the integration of psychometric and behavioral data with neuroimaging data. Guidelines on when and how to employ triangulation methods on neuroimaging data are included. Moreover, we also included discussions on practices and research directions that validate fMRI studies in consumer neuroscience beyond data triangulation.

A Neglected Topic in Neuroscience: Replicability of fMRI Results With Specific Reference to ANOREXIA NERVOSA

Functional magnetic resonance imaging (fMRI) studies report impaired functional correlates of cognition and emotion in mental disorders. The validity of preexisting studies needs to be confirmed through replication studies, which there is a lack of. So far, most replication studies have been conducted on non-patients (NP) and primarily investigated cognitive and motor tasks. To fill this gap, we conducted the first fMRI replication study to investigate brain function using disease-related food stimuli in patients with anorexia nervosa (AN). Using fMRI, we investigated 31 AN patients and 27 NP for increased amygdala and reduced midcingulate activation when viewing food and non-food stimuli, as reported by the original study (11AN, 11NP; Joos et al., 2011). Similar to the previous study, we observed in the within group comparisons (food>non-food) a frontoinsular activation for both groups. Although in AN the recorded activation clustered more prominently and extended into the cingulate cortex. In the between-group comparisons, the increased amygdala and reduced midcingulate activation could not be replicated. Instead, AN showed a higher activation of the cingulate cortices, the pre-/postcentral gyrus and the inferior parietal lobe. Unlike in the initial study, no significant differences between NP>AN could be observed. The inconsistency of results and the non-replication of the study could have several reasons, such as high inter-individual variance of functional correlates of emotion processing, as well as intra-individual variances and the smaller group size of the initial study. These results underline the importance of replication for assessing the reliability and validity of results from fMRI research.

Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices

As concern over the use of questionable research practices (QRPs) in academic science has increased over the last couple of decades, some reforms have been implemented and many others have been debated and recommended. While many of these proposals have merit, efforts to improve scientific practices are more likely to succeed when they are responsive to the prevailing views and concerns of scientists themselves. To date, there have been few efforts to solicit wide-ranging input from researchers on the topic of needed reforms. This article is a qualitative report of responses from federally funded scientists to the question of what should be done to address the problem of QRPs in their disciplines. Overall, participants were concerned about how institutional and career-oriented incentives encourage the use of QRPs. Compared to previous recommendations, participants had surprisingly little confidence in the ability of ethics training to improve research integrity.

Rigor and reproducibility in rodent behavioral research

Behavioral neuroscience research incorporates the identical high level of meticulous methodologies and exacting attention to detail as all other scientific disciplines. To achieve maximal rigor and reproducibility of findings, well-trained investigators employ a variety of established best practices. Here we explicate some of the requirements for rigorous experimental design and accurate data analysis in conducting mouse and rat behavioral tests. Novel object recognition is used as an example of a cognitive assay which has been conducted successfully with a range of methods, all based on common principles of appropriate procedures, controls, and statistics. Directors of Rodent Core facilities within Intellectual and Developmental Disabilities Research Centers contribute key aspects of their own novel object recognition protocols, offering insights into essential similarities and less-critical differences. Literature cited in this review article will lead the interested reader to source papers that provide step-by-step protocols which illustrate optimized methods for many standard rodent behavioral assays. Adhering to best practices in behavioral neuroscience will enhance the value of animal models for the multiple goals of understanding biological mechanisms, evaluating consequences of genetic mutations, and discovering efficacious therapeutics.

Direct replication of task-dependent neural activation patterns during sadness introspection in two independent adolescent samples

Functional neuroimaging results need to replicate to inform sound models of human social cognition and its neural correlates. Introspection, the capacity to reflect on one's thoughts and feelings, is one process required for normative social cognition and emotional functioning. Engaging in introspection draws on a network of brain regions including medial prefrontal cortex (mPFC), posterior cingulate cortex (PCC), middle temporal gyri (MTG), and temporoparietal junction (TPJ). Maturation of these regions during adolescence mirrors the behavioral advances seen in adolescent social cognition, but the neural correlates of introspection in adolescence need to replicate to confirm their generalizability and role as a possible mechanism. The current study investigated whether reflecting upon one's own feelings of sadness would activate and replicate similar brain regions in two independent samples of adolescents. Participants included 156 adolescents (50% female) from the California Families Project and 119 adolescent girls from the Pittsburgh Girls Study of Emotion. All participants completed the Emotion Regulation Questionnaire (ERQ) and underwent a functional magnetic resonance imaging scan while completing the same facial emotion‐processing task at age 16–17 years. Both samples showed similar whole‐brain activation patterns when engaged in sadness introspection and when judging a nonemotional facial feature. Whole‐brain activation was unrelated to ERQ scores in both samples. Neural responsivity to task manipulations replicated in regions recruited for socio‐emotional (mPFC, PCC, MTG, TPJ) and attention (dorsolateral PFC, precentral gyri, superior occipital gyrus, superior parietal lobule) processing. These findings demonstrate robust replication of neural engagement during sadness introspection in two independent adolescent samples.

Why Study the History of Neuroscience?

The history of neuroscience is the memory of the discipline and this memory depends on the study of the present traces of the past; the things left behind: artifacts, equipment, written documents, data books, photographs, memoirs, etc. History, in all of its definitions, is an integral part of neuroscience and I have used examples from the literature and my personal experience to illustrate the importance of the different aspects of history in neuroscience. Each time we talk about the brain, do an experiment, or write a research article, we are involved in history. Each published experiment becomes a historical document; it relies on past research (the "Introduction" section), procedures developed in the past ("Methods" section) and as soon as new data are published, they become history and become embedded into the history of the discipline ("Discussion" section). In order to be transparent and able to be replicated, each experiment requires its own historical archive. Studying history means researching books, documents and objects in libraries, archives, and museums. It means looking at data books, letters and memos, talking to scientists, and reading biographies and autobiographies. History can be made relevant by integrating historical documents into classes and by using historical websites. Finally, conducting historical research can be interesting, entertaining, and can lead to travel to out-of-the-way and exotic places and meeting interesting people.

National Neuroinformatics Framework for Canadian Consortium on Neurodegeneration in Aging (CCNA)

The Canadian Institutes for Health Research (CIHR) launched the "International Collaborative Research Strategy for Alzheimer's Disease" as a signature initiative, focusing on Alzheimer's Disease (AD) and related neurodegenerative disorders (NDDs). The Canadian Consortium for Neurodegeneration and Aging (CCNA) was subsequently established to coordinate and strengthen Canadian research on AD and NDDs. To facilitate this research, CCNA uses LORIS, a modular data management system that integrates acquisition, storage, curation, and dissemination across multiple modalities. Through an unprecedented national collaboration studying various groups of dementia-related diagnoses, CCNA aims to investigate and develop proactive treatment strategies to improve disease prognosis and quality of life of those affected. However, this constitutes a unique technical undertaking, as heterogeneous data collected from sites across Canada must be uniformly organized, stored, and processed in a consistent manner. Currently clinical, neuropsychological, imaging, genomic, and biospecimen data for 509 CCNA subjects have been uploaded to LORIS. In addition, data validation is handled through a number of quality control (QC) measures such as double data entry (DDE), conflict flagging and resolution, imaging protocol checks, and visual imaging quality validation. Site coordinators are also notified of incidental findings found in MRI reads or biosample analyses. Data is then disseminated to CCNA researchers via a web-based Data-Querying Tool (DQT). This paper will detail the wide array of capabilities handled by LORIS for CCNA, aiming to provide the necessary neuroinformatic infrastructure for this nation-wide investigation of healthy and diseased aging.

O que (não) há de “complexo” no comportamento? Behaviorismo radical, self, insight e linguagem

Resumo Uma crítica comum encontrada em manuais e livros didáticos de psicologia é que a análise do comportamento não seria capaz de explicar fenômenos psicológicos complexos. Estes seriam melhor abordados por explicações cognitivistas baseadas em mecanismos internos ao organismo. Este ensaio tem como objetivo avaliar a pertinência dessa crítica à luz de exemplos da literatura analítico-comportamental. A partir da análise de pesquisas que tratam de formação de self, insight e linguagem, argumenta-se que a “complexidade” foi importada para os laboratórios de análise do comportamento, assim como floresceu em diversas linhas de pesquisa de tradição behaviorista radical. Em adição, são discutidos cinco significados possíveis dados à “complexidade” extraídos da literatura consultada. Conclui-se que não há significado útil do termo e que, por essa razão, talvez seja pertinente abandoná-lo como critério de classificação de comportamentos. Como consequência, “comportamento complexo” seria simplesmente “comportamento” e nada mais.

Questionable research practices in student final theses - Prevalence, attitudes, and the role of the supervisor's perceived attitudes

Although questionable research practices (QRPs) and p-hacking have received attention in recent years, little research has focused on their prevalence and acceptance in students. Students are the researchers of the future and will represent the field in the future. Therefore, they should not be learning to use and accept QRPs, which would reduce their ability to produce and evaluate meaningful research. 207 psychology students and fresh graduates provided self-report data on the prevalence and predictors of QRPs. Attitudes towards QRPs, belief that significant results constitute better science or lead to better grades, motivation, and stress levels were predictors. Furthermore, we assessed perceived supervisor attitudes towards QRPs as an important predictive factor. The results were in line with estimates of QRP prevalence from academia. The best predictor of QRP use was students' QRP attitudes. Perceived supervisor attitudes exerted both a direct and indirect effect via student attitudes. Motivation to write a good thesis was a protective factor, whereas stress had no effect. Students in this sample did not subscribe to beliefs that significant results were better for science or their grades. Such beliefs further did not impact QRP attitudes or use in this sample. Finally, students engaged in more QRPs pertaining to reporting and analysis than those pertaining to study design. We conclude that supervisors have an important function in shaping students' attitudes towards QRPs and can improve their research practices by motivating them well. Furthermore, this research provides some impetus towards identifying predictors of QRP use in academia.

Practical Solutions for Sharing Data and Materials From Psychological Research

Widespread sharing of data and materials (including displays and text- and video-based descriptions of experimental procedures) will improve the reproducibility of psychological science and accelerate the pace of discovery. In this article, we discuss some of the challenges to open sharing and offer practical solutions for researchers who wish to share more of the products-and process-of their research. Many of these solutions were devised by the Databrary.org data library for storing and sharing video, audio, and other forms of sensitive or personally identifiable data. We also discuss ways in which researchers can make shared data and materials easier for others to find and reuse. Widely adopted, these solutions and practices will increase transparency and speed progress in psychological science.