Data quality over data quantity in computational cognitive neuroscience

A registered report of a two-site study of variations of the flanker task: ERN experimental effects and data quality

Error-related negativity is a widely used measure of error monitoring, and many projects are independently moving ERN recorded during a flanker task toward standardization, optimization, and eventual clinical application. However, each project uses a different version of the flanker task and tacitly assumes ERN is functionally equivalent across each version. The routine neglect of a rigorous test of this assumption undermines efforts to integrate ERN findings across tasks, optimize and standardize ERN assessment, and widely apply ERN in clinical trials. The purpose of this registered report was to determine whether ERN shows similar experimental effects (correct vs. error trials) and data quality (intraindividual variability) during three commonly used versions of a flanker task. ERN was recorded from 172 participants during three versions of a flanker task across two study sites. ERN scores showed numerical differences between tasks, raising questions about the comparability of ERN findings across studies and tasks. Although ERN scores from all three versions of the flanker task yielded high data quality and internal consistency, one version did outperform the other two in terms of the size of experimental effects and the data quality. Exploratory analyses of the error positivity (Pe) provided tentative support for the other two versions of the task over the paradigm that appeared optimal for ERN. The present study provides a roadmap for how to statistically compare psychometric characteristics of ERP scores across paradigms and gives preliminary recommendations for flanker tasks to use for ERN- and Pe-focused studies.

Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology

Psychophysiological research is an inherently complex undertaking due to the nature of the data, and its analysis is characterized by many decision points that shape the final dataset and a study's findings. These decisions create a "multiverse" of possible outcomes, and each decision from study conceptualization to statistical analysis can lead to different results and interpretations. This review describes the concept of multiverse analyses, a methodological approach designed to understand the impact of different decisions on the robustness of a study's findings and interpretation. The emphasis is on transparently showcasing different reasonable approaches for constructing a final dataset and on highlighting the influence of various decision points, from experimental design to data processing and outcome selection. For example, the choice of an experimental task can significantly impact event-related brain potential (ERP) scores or skin conductance responses (SCRs), and different tasks might elicit unique variances in each measure. This review underscores the importance of transparently embracing the flexibility inherent in psychophysiological research and the potential consequences of not understanding the fragility or robustness of experimental findings. By navigating the intricate terrain of the psychophysiological multiverse, this review serves as an introduction, helping researchers to make informed decisions, improve the collective understanding of psychophysiological findings, and push the boundaries of the field.

RELAX part 2: A fully automated EEG data cleaning algorithm that is applicable to Event-Related-Potentials

OBJECTIVE

Electroencephalography (EEG) is often used to examine neural activity time-locked to stimuli presentation, referred to as Event-Related Potentials (ERP). However, EEG is influenced by non-neural artifacts, which can confound ERP comparisons. Artifact cleaning reduces artifacts, but often requires time-consuming manual decisions. Most automated methods filter frequencies <1 Hz out of the data, so are not recommended for ERPs (which contain frequencies <1 Hz). Our aim was to test the RELAX (Reduction of Electroencephalographic Artifacts) pre-processing pipeline for use on ERP data.

METHODS

The cleaning performance of multiple versions of RELAX were compared to four commonly used EEG cleaning pipelines across both artifact cleaning metrics and the amount of variance in ERPs explained by different conditions in a Go-Nogo task. Results RELAX with Multi-channel Wiener Filtering (MWF) and wavelet-enhanced independent component analysis applied to artifacts identified with ICLabel (wICA_ICLabel) cleaned data most effectively and produced amongst the most dependable ERP estimates. RELAX with wICA_ICLabel only or MWF_only may detect effects better for some ERPs.

CONCLUSIONS

RELAX shows high artifact cleaning performance even when data is high-pass filtered at 0.25 Hz (applicable to ERP analyses).

SIGNIFICANCE

RELAX is easy to implement via EEGLAB in MATLAB and freely available on GitHub. Given its performance and objectivity we recommend RELAX to improve artifact cleaning and consistency across ERP research.

Introducing RELAX: An automated pre-processing pipeline for cleaning EEG data - Part 1: Algorithm and application to oscillations

OBJECTIVE

Electroencephalographic (EEG) data are often contaminated with non-neural artifacts which can confound experimental results. Current artifact cleaning approaches often require costly manual input. Our aim was to provide a fully automated EEG cleaning pipeline that addresses all artifact types and improves measurement of EEG outcomes METHODS: We developed RELAX (the Reduction of Electroencephalographic Artifacts). RELAX cleans continuous data using Multi-channel Wiener filtering [MWF] and/or wavelet enhanced independent component analysis [wICA] applied to artifacts identified by ICLabel [wICA_ICLabel]). Several versions of RELAX were compared using three datasets (N = 213, 60 and 23 respectively) against six commonly used pipelines across a range of artifact cleaning metrics, including measures of remaining blink and muscle activity, and the variance explained by experimental manipulations after cleaning.

RESULTS

RELAX with MWF and wICA_ICLabel showed amongst the best performance at cleaning blink and muscle artifacts while preserving neural signal. RELAX with wICA_ICLabel only may perform better at differentiating alpha oscillations between working memory conditions.

CONCLUSIONS

RELAX provides automated, objective and high-performing EEG cleaning, is easy to use, and freely available on GitHub.

SIGNIFICANCE

We recommend RELAX for data cleaning across EEG studies to reduce artifact confounds, improve outcome measurement and improve inter-study consistency.

Mapping gene by early life stress interactions on child subcortical brain structures: A genome-wide prospective study

Background

Although it is well-established that both genetics and the environment influence brain development, they are typically examined separately. Here, we aimed to prospectively investigate the interactive effects of genetic variants-from a genome-wide approach-and early life stress (ELS) on child subcortical brain structures, and their association with subsequent mental health problems.

Method

Primary analyses were conducted using data from the Generation R Study (N = 2257), including genotype and cumulative prenatal and postnatal ELS scores (encompassing life events, contextual risk, parental risk, interpersonal risk, direct victimisation). Neuroimaging data were collected at age 10 years, including intracranial and subcortical brain volumes (accumbens, amygdala, caudate, hippocampus, pallidum, putamen, thalamus). Genome-wide association and genome-wide-by-environment interaction analyses (GWEIS, run separately for prenatal/postnatal ELS) were conducted for eight brain outcomes (i.e., 24 genome-wide analyses) in the Generation R Study (discovery). Polygenic scores (PGS) using the resulting weights were calculated in an independent (target) cohort (adolescent brain cognitive development Study; N = 10,751), to validate associations with corresponding subcortical volumes and examine links to later mother-reported internalising and externalising problems.

Results

One GWEIS-prenatal stress locus was associated with caudate volume (rs139505895, mapping onto PRSS12 and NDST3) and two GWEIS-postnatal stress loci with the accumbens (rs2397823 and rs3130008, mapping onto CUTA, SYNGAP1, and TABP). Functional annotation revealed that these genes play a role in neuronal plasticity and synaptic function, and have been implicated in neuro-developmental phenotypes, for example, intellectual disability, autism, and schizophrenia. None of these associations survived a more stringent correction for multiple testing across all analysis sets. In the validation sample, all PGSgenotype were associated with their respective brain volumes, but no PGSGxE associated with any subcortical volume. None of the PGS associated with internalising or externalising problems.

Conclusions

This study lends novel suggestive insights into gene-environment interplay on the developing brain as well as pointing to promising candidate loci for future replication and mechanistic studies.

The data-processing multiverse of event-related potentials (ERPs): A roadmap for the optimization and standardization of ERP processing and reduction pipelines

In studies of event-related brain potentials (ERPs), numerous decisions about data processing are required to extract ERP scores from continuous data. Unfortunately, the systematic impact of these choices on the data quality and psychometric reliability of ERP scores or even ERP scores themselves is virtually unknown, which is a barrier to the standardization of ERPs. The aim of the present study was to optimize processing pipelines for the error-related negativity (ERN) and error positivity (Pe) by considering a multiverse of data processing choices. A multiverse analysis of a data processing pipeline examines the impact of a large set of different reasonable choices to determine the robustness of effects, such as the impact of different decisions on between-trial standard deviations (i.e., data quality) and between-condition differences (i.e., experimental effects). ERN and Pe data from 298 healthy young adults were used to determine the impact of different methodological choices on data quality and experimental effects (correct vs. error trials) at several key stages: highpass filtering, lowpass filtering, ocular artifact correction, reference, baseline adjustment, scoring sensors, and measurement procedure. This multiverse analysis yielded 3,456 ERN scores and 576 Pe scores per person. An optimized pipeline for ERN included a 15 Hz lowpass filter, ICA-based ocular artifact correction, and a region of interest (ROI) approach to scoring. For Pe, the optimized pipeline included a 0.10 Hz highpass filter, 30 Hz lowpass filter, regression-based ocular artifact correction, a -200 to 0 ms baseline adjustment window, and an ROI approach to scoring. The multiverse approach can be used to optimize pipelines for eventual standardization, which would support efforts toward establishing normative ERP databases. The proposed process of analyzing the data-processing multiverse of ERP scores paves the way for better refinement, identification, and selection of data processing parameters, ultimately improving the precision and utility of ERPs.

Cognitive flexibility and N2/P3 event-related brain potentials

Task switching is often considered for evaluating limitations of cognitive flexibility. Switch costs are behavioural indices of limited cognitive flexibility, and switch costs may be decomposable into stimulus- and response-related fractions, as conjectured by the domain hypothesis of cognitive flexibility. According to the domain hypothesis, there exist separable stimulus- and response-related neural networks for cognitive flexibility, which should be discernible as distinct event-related potentials (ERPs). The present card-matching study allowed isolating stimulus- and response-related switch costs, while measuring ERPs evoked by task cues and target stimuli with a focus on the target-locked N2/P3 complex. Behavioural data revealed that both stimulus-task and response-task bindings contribute to switch costs. Cue-locked ERPs yielded larger anterior negativity/posterior positivity in response to switch cues compared to repeat cues. Target-locked ERPs revealed separable ERP correlates of stimulus- and response-related switch costs. P3 waveforms with fronto-central scalp distributions emerged as a corollary of stimulus-related switch costs. Fronto-centrally distributed N2 waveforms occurred when stimulus-task and response-task bindings contributed jointly to switch costs. The reported N2/P3 ERP data are commensurate with the domain hypothesis according to which there exist separable stimulus- and response-related neural networks for cognitive flexibility.

Investigating the effects of ongoing-task bias on prospective memory

Event-based prospective memory (PM) refers to the cognitive processes required to perform a planned action upon encountering a future event. Event-based PM studies engage participants in an ongoing task (e.g., lexical decision-making) with an instruction to make an alternative PM response to certain items (e.g., items containing "tor"). The Prospective Memory Decision Control (PMDC) model, which provides a quantitative process account of ongoing-task and PM decisions, proposes that PM and ongoing-task processes compete in a race to threshold. We use PMDC to test whether, as proposed by the Delay Theory of PM costs, PM can be improved by biasing decision-making against a specific ongoing-task choice, so that the PM process is more likely to win the race. We manipulated bias in a lexical decision task with an accompanying PM intention. In one condition, a bias was induced against deciding items were words, and in another, a bias was induced against deciding items were non-words. The bias manipulation had little effect on PM accuracy but did affect the types of ongoing-task responses made on missed PM trials. PMDC fit the observed data well and verified that the bias manipulation had the intended effect on ongoing-task processes. Furthermore, although simulations from PMDC could produce an improvement in PM accuracy due to ongoing-task bias, this required implausible parameter values. These results illustrate the importance of understanding event-based PM in terms of a comprehensive model of the processes that interact to determine all aspects of task performance.

Model-based assessment and neural correlates of spatial memory deficits in mild cognitive impairment

Mild cognitive impairment (MCI) is characterized by subjective and objective memory impairments within the context of generally intact everyday functioning. Such memory deficits are typically thought to arise from medial temporal lobe dysfunction; however, differences in memory task performance can arise from a variety of altered processes (e.g., strategy adjustments) rather than, or in addition to, "pure" memory deficits. To address this problem, we applied the linear ballistic accumulator (LBA: Brown and Heathcote, 2008) model to data from individuals with MCI (n = 18) and healthy older adults (HOA; n = 16) who performed an object-location association memory retrieval task during functional magnetic resonance imaging (fMRI). The primary goals were to 1) assess between-group differences in model parameters indexing processes of interest (memory sensitivity, accumulation speed, caution and time spent on peripheral perceptual and motor processes) and 2) determine whether differences in model-based metrics were consistent with fMRI data. The LBA provided evidence that, relative to the HOA group, those with MCI displayed lower sensitivity (i.e., difficulty discriminating targets from lures), suggestive of memory impairment, and displayed higher evidence accumulation speed and greater caution, suggestive of increased arousal and strategic changes in this group, although these changes had little impact on MCI-related accuracy differences. Consistent with these findings, fMRI revealed reduced activation in brain regions previously linked to evidence accumulation and to the implementation of caution reductions in the MCI group. Findings suggest that multiple cognitive mechanisms differ during memory retrieval in MCI, and that these mechanisms may explain neuroimaging alterations outside of the medial temporal lobes.

Modeling the effects of methylphenidate on interference and evidence accumulation processes using the conflict linear ballistic accumulator

RATIONALE

Although methylphenidate and other stimulants have been demonstrated to improve task performance across a variety of domains, a computationally rigorous account of how these drugs alter cognitive processing remains elusive. Recent applications of mathematical models of cognitive processing and electrophysiological methods to this question have suggested that stimulants improve the integrity of evidence accumulation processes for relevant choices, potentially through catecholaminergic modulation of neural signal-to-noise ratios. However, this nascent line of work has thus far been limited to simple perceptual tasks and has largely omitted more complex conflict paradigms that contain experimental manipulations of specific top-down interference resolution processes.

OBJECTIVES AND METHODS

To address this gap, this study applied the conflict linear ballistic accumulator (LBA), a newly proposed model designed for conflict tasks, to data from healthy adults who performed the Multi-Source Interference Task (MSIT) after acute methylphenidate or placebo challenge.

RESULTS

Model-based analyses revealed that methylphenidate improved performance by reducing individuals' response thresholds and by enhancing evidence accumulation processes across all task conditions, either by improving the quality of evidence or by reducing variability in accumulation processes. In contrast, the drug did not reduce bottom-up interference or selectively facilitate top-down interference resolution processes probed by the experimental conflict manipulation.

CONCLUSIONS

Enhancement of evidence accumulation is a biologically plausible and task-general mechanism of stimulant effects on cognition. Moreover, the assumption that methylphenidate's effects on behavior are only visible with complex executive tasks may be misguided.

The lexical semantics of adjective-noun phrases in the human brain

As a person reads, the brain performs complex operations to create higher order semantic representations from individual words. While these steps are effortless for competent readers, we are only beginning to understand how the brain performs these actions. Here, we explore lexical semantics using magnetoencephalography (MEG) recordings of people reading adjective-noun phrases presented one word at a time. We track the neural representation of single word representations over time, through different brain regions. Our results reveal two novel findings: (a) a neural representation of the adjective is present during noun presentation, but this representation is different from that observed during adjective presentation and (b) the neural representation of adjective semantics observed during adjective reading is reactivated after phrase reading, with remarkable consistency. We also note that while the semantic representation of the adjective during the reading of the adjective is very distributed, the later representations are concentrated largely to temporal and frontal areas previously associated with composition. Taken together, these results paint a picture of information flow in the brain as phrases are read and understood.

Strategic attention and decision control support prospective memory in a complex dual-task environment

Human performance in complex multiple-task environments depends critically on the interplay between cognitive control and cognitive capacity. In this paper we propose a tractable computational model of how cognitive control and capacity influence the speed and accuracy of decisions made in the event-based prospective memory (PM) paradigm, and in doing so test a new quantitative formulation that measures two distinct components of cognitive capacity (gain and focus) that apply generally to choices among two or more options. Consistent with prior work, individuals used proactive control (increased ongoing task thresholds under PM load) and reactive control (inhibited ongoing task accumulation rates to PM items) to support PM performance. Individuals used cognitive gain to increase the amount of resources allocated to the ongoing task under time pressure and PM load. However, when demands exceeded the capacity limit, resources were reallocated (shared) between ongoing task and PM processes. Extending previous work, individuals used cognitive focus to control the quality of processing for the ongoing and PM tasks based on the particular demand and payoff structure of the environment (e.g., higher focus for higher priority tasks; lower focus under high time pressure and with PM load). Our model provides the first detailed quantitative understanding of cognitive gain and focus as they apply to evidence accumulation models, which - along with cognitive control mechanisms - support decision-making in complex multiple-task environments.

The neural determinants of age-related changes in fluid intelligence: a pre-registered, longitudinal analysis in UK Biobank

Background: Fluid intelligence declines with advancing age, starting in early adulthood. Within-subject declines in fluid intelligence are highly correlated with contemporaneous declines in the ability to live and function independently. To support healthy aging, the mechanisms underlying these declines need to be better understood. Methods: In this pre-registered analysis, we applied latent growth curve modelling to investigate the neural determinants of longitudinal changes in fluid intelligence across three time points in 185,317 individuals (N=9,719 two waves, N=870 three waves) from the UK Biobank (age range: 39-73 years). Results: We found a weak but significant effect of cross-sectional age on the mean fluid intelligence score, such that older individuals scored slightly lower. However, the mean longitudinal slope was positive, rather than negative, suggesting improvement across testing occasions. Despite the considerable sample size, the slope variance was non-significant, suggesting no reliable individual differences in change over time. This null-result is likely due to the nature of the cognitive test used. In a subset of individuals, we found that white matter microstructure (N=8839, as indexed by fractional anisotropy) and grey-matter volume (N=9931) in pre-defined regions-of-interest accounted for complementary and unique variance in mean fluid intelligence scores. The strongest effects were such that higher grey matter volume in the frontal pole and greater white matter microstructure in the posterior thalamic radiations were associated with higher fluid intelligence scores. Conclusions: In a large preregistered analysis, we demonstrate a weak but significant negative association between age and fluid intelligence. However, we did not observe plausible longitudinal patterns, instead observing a weak increase across testing occasions, and no significant individual differences in rates of change, likely due to the suboptimal task design. Finally, we find support for our preregistered expectation that white- and grey matter make separate contributions to individual differences in fluid intelligence beyond age.

The neural determinants of age-related changes in fluid intelligence: a pre-registered, longitudinal analysis in UK Biobank

Background: Fluid intelligence declines with advancing age, starting in early adulthood. Within-subject declines in fluid intelligence are highly correlated with contemporaneous declines in the ability to live and function independently. To support healthy aging, the mechanisms underlying these declines need to be better understood. Methods: In this pre-registered analysis, we applied latent growth curve modelling to investigate the neural determinants of longitudinal changes in fluid intelligence across three time points in 185,317 individuals (N=9,719 two waves, N=870 three waves) from the UK Biobank (age range: 39-73 years). Results: We found a weak but significant effect of cross-sectional age on the mean fluid intelligence score, such that older individuals scored slightly lower. However, the mean longitudinal slope was positive, rather than negative, suggesting improvement across testing occasions. Despite the considerable sample size, the slope variance was non-significant, suggesting no reliable individual differences in change over time. This null-result is likely due to the nature of the cognitive test used. In a subset of individuals, we found that white matter microstructure (N=8839, as indexed by fractional anisotropy) and grey-matter volume (N=9931) in pre-defined regions-of-interest accounted for complementary and unique variance in mean fluid intelligence scores. The strongest effects were such that higher grey matter volume in the frontal pole and greater white matter microstructure in the posterior thalamic radiations were associated with higher fluid intelligence scores. Conclusions: In a large preregistered analysis, we demonstrate a weak but significant negative association between age and fluid intelligence. However, we did not observe plausible longitudinal patterns, instead observing a weak increase across testing occasions, and no significant individual differences in rates of change, likely due to the suboptimal task design. Finally, we find support for our preregistered expectation that white- and grey matter make separate contributions to individual differences in fluid intelligence beyond age.