From ERPs to MVPA Using the Amsterdam Decoding and Modeling Toolbox (ADAM)

Decoding in the Fourth Dimension: Classification of Temporal Patterns and Their Generalization Across Locations

Neuroimaging research has increasingly used decoding techniques, in which multivariate statistical methods identify patterns in neural data that allow the classification of experimental conditions or participant groups. Typically, the features used for decoding are spatial in nature, including voxel patterns and electrode locations. However, the strength of many neurophysiological recording techniques such as electroencephalography or magnetoencephalography is in their rich temporal, rather than spatial, content. The present report introduces the time-GAL toolbox, which implements a decoding method based on time information in electrophysiological recordings. The toolbox first quantifies the decodable information contained in neural time series. This information is then used in a subsequent step, generalization across location (GAL), which characterizes the relationship between sensor locations based on their ability to cross-decode. Two datasets are used to demonstrate the usage of the toolbox, involving (1) event-related potentials in response to affective pictures and (2) steady-state visual evoked potentials in response to aversively conditioned grating stimuli. In both cases, experimental conditions were successfully decoded based on the temporal features contained in the neural time series. Spatial cross-decoding occurred in regions known to be involved in visual and affective processing. We conclude that the approach implemented in the time-GAL toolbox holds promise for analyzing neural time series from a wide range of paradigms and measurement domains providing an assumption-free method to quantifying differences in temporal patterns of neural information processing and whether these patterns are shared across sensor locations.

Acoustic Exaggeration Enhances Speech Discrimination in Young Autistic Children

Child-directed speech (CDS), which amplifies acoustic and social features of speech during interactions with young children, promotes typical phonetic and language development. In autism, both behavioral and brain data indicate reduced sensitivity to human speech, which predicts absent, decreased, or atypical benefits of exaggerated speech signals such as CDS. This study investigates the impact of exaggerated fundamental frequency (F0) and voice-onset time on the neural processing of speech sounds in 22 Chinese-speaking autistic children aged 2-7 years old with a history of speech delays, compared with 25 typically developing (TD) peers. Electroencephalography (EEG) data were collected during passive listening to exaggerated and non-exaggerated syllables. A time-resolved multivariate pattern analysis (MVPA) was used to evaluate the potential effects of acoustic exaggeration on syllable discrimination in terms of neural decoding accuracy. For non-exaggerated syllables, neither the autism nor the TD group achieved above-chance decoding accuracy. In contrast, for exaggerated syllables, both groups achieved above-chance decoding, indicating significant syllable discrimination, with no difference in accuracy between the autism and TD groups. However, the temporal generalization patterns in the MVPA results revealed distinct neural mechanisms supporting syllable discrimination between the groups. Although the TD group demonstrated a left-hemisphere advantage for decoding and generalization, the autism group displayed similar decoding patterns between hemispheres. These findings highlight the potential of selective acoustic exaggeration to support speech learning in autistic children, underscoring the importance of tailored, sensory-based interventions.

Metacontrol instructions lead to adult-like event segmentation in adolescents

Event segmentation, which involves dividing continuous information into meaningful units, changes as children develop into adolescents. Adolescents tend to segment events more coarsely than adults. This study explores whether adolescents could adjust their segmentation style to resemble that of adults when provided with explicit metacontrol-related instructions. We compared event segmentation in two adolescent groups and one adult group, while simultaneously recording EEG data. One adolescent group was instructed to perform segmentation as finely as possible, whereas the other adolescent group and adults received no specific instructions on segmentation granularity. EEG data were analyzed using multivariate pattern analysis and source reconstruction. The findings revealed that adolescents given fine-grained instructions adjusted their segmentation probability closer to adult levels, although they did not fully match adults in processing multiple simultaneous changes. Neurophysiological results indicated that adolescents with fine-grained instructions exhibited neural decoding performance more similar to adults. Increased activity in the inferior frontal gyrus in these adolescents compared to adults related to this. The results suggest that adolescents with fine-grained instructions demonstrated more persistent cognitive control and enhanced top-down attention than their peers and adults. The study shows that adolescent cognitive processes can be shifted toward adult-like performance through instructions.

Revealing the different levels of action monitoring in visuomotor transformation task: Evidence from decomposition of cortical potentials

This study investigates the cortical correlates of motor response control and monitoring, using the Theory of Event Coding (TEC) as a framework to investigate signals related to low-level sensory processing of motor reafference and high-level response monitoring, including verification of response outcomes with the internal model. We used a visuomotor paradigm with two targets at different distances from the participant. For the recorded movement-related cortical potentials (MRCPs), we analyzed their different components and assessed the movement phases during which they are active. Residual iteration decomposition (RIDE) and multivariate pattern analysis (MVPA) were used for this analysis. Using RIDE, we separated MRCPs into signals related to different parallel processes of visuomotor transformation: stimulus processing (S-cluster), motor response preparation and execution (R-cluster), and intermediate processes (C-cluster). We revealed sequential activation in the R-cluster, with execution-related negative components and positive contralateral peaks reflecting reafference processing. We also identified the motor post-imperative negative variation within the R-cluster, highlighting the response outcome evaluation process included in the action file. Our findings extend the understanding of C-cluster signals, typically associated with stimulus-response mapping, by demonstrating C-activation from the preparatory stages through to response termination, highlighting its participation in action monitoring. In addition, we highlighted the ability of MVPA to identify movement-related attribute encoding: where statistical analysis showed independence of stimulus processing activity from movement distance, MVPA revealed distance-related differences in the S-cluster within a time window aligned with the lateralized readiness potential (LRP). This highlights the importance of integrating RIDE and MVPA to uncover the intricate neural dynamics of motor control, sensory integration, and response monitoring.

High-intensity physiological activation disrupts the neural signatures of conflict processing

Physiological activation fluctuates throughout the day. Previous studies have shown that during periods of reduced activation, cognitive control remains resilient due to neural compensatory mechanisms. In this study, we investigate the effects of high physiological activation on both behavioural and neural markers of cognitive control. We hypothesize that while behavioural measures of cognitive control would remain intact during periods of high activation, there would be observable changes in neural correlates. In our electroencephalography study, we manipulate levels of physiological activation through physical exercise. Although we observe no significant impact on behavioural measures of cognitive conflict, both univariate and multivariate time-frequency markers prove unreliable under conditions of high activation. Moreover, we observe no modulation of whole-brain connectivity measures by physiological activation. We suggest that this dissociation between behavioural and neural measures indicates that the human cognitive control system remains resilient even at high activation, possibly due to underlying neural compensatory mechanisms.

Neural signatures of visual awareness independent of postperceptual processing

What are the neural processes associated with perceptual awareness that are distinct from preconscious sensory encoding and postperceptual processes such as reporting an experience? Using electroencephalography and a no-report visual masking paradigm, we manipulated stimulus visibility by varying the time between stimuli and masks in linear steps (17, 33, 50, 67, and 83 ms). Awareness increased nonlinearly, with stimuli never seen at the two shortest intervals, always seen at the two longest, and 50% seen at the intermediate interval. Separate report and no-report conditions were used to isolate awareness from task performance. Our results revealed a neural signal closely linked to perceptual awareness, independent of the task: a fronto-central event-related potential that we refer to as the N2 (~250 to 300 ms). Earlier event-related potential signals reflected the linear manipulation of stimulus strength, while later signals like P3b and temporal generalization of decoding were tied to task performance, appearing only in the report condition. Taken together, these findings inform current debates regarding theories of consciousness and offer new avenues for exploring the neural mechanisms supporting conscious processing.

Minimal exposure durations reveal visual processing priorities for different stimulus attributes

Human vision can detect a single photon, but the minimal exposure required to extract meaning from stimulation remains unknown. This requirement cannot be characterised by stimulus energy, because the system is differentially sensitive to attributes defined by configuration rather than physical amplitude. Determining minimal exposure durations required for processing various stimulus attributes can thus reveal the system's priorities. Using a tachistoscope enabling arbitrarily brief displays, we establish minimal durations for processing human faces, a stimulus category whose perception is associated with several well-characterised behavioural and neural markers. Neural and psychophysical measures show a sequence of distinct minimal exposures for stimulation detection, object-level detection, face-specific processing, and emotion-specific processing. Resolving ongoing debates, face orientation affects minimal exposure but emotional expression does not. Awareness emerges with detection, showing no evidence of subliminal perception. These findings inform theories of visual processing and awareness, elucidating the information to which the visual system is attuned.

Delayed modulation of alpha band activity increases response inhibition deficits in adolescents with AD(H)D

Deficiencies in inhibitory control are one of the hallmarks of attention-deficit-(hyperactivity) disorder (AD(H)D). Response inhibition demands can become increased through additional conflicts, namely when already integrated representations of perception-action associations have to be updated. Yet, the neural mechanisms of how such conflicts worsen response inhibition in AD(H)D are unknown, but, if identified, could help to better understand the complex nature of AD(H)D-associated impulsivity. We investigated both behavioral performance and EEG activity in the theta and alpha band of adolescents (10-18 years of age) with AD(H)D (n = 28) compared to neurotypical (NT) controls (n = 33) in a conflict-modulated Go/Nogo paradigm. We used multivariate pattern analysis (MVPA) and EEG-beamforming to examine how changes in representational content are coded by oscillatory activity and to delineate the cortical structures involved in it. The presented behavioral and neurophysiological data show that adolescents with AD(H)D are more strongly affected by increased response inhibition demands through additional conflicts than NT controls. Precisely, AD(H)D participants showed higher false alarm rates than NT controls in both, non-overlapping and overlapping Nogo trials, but performed even worse in the latter. This is likely due to an inefficient updating of representations related to delayed modulations of alpha band activity in the ventral stream and orbitofrontal regions. Theta band activity is also modulated by conflict but was not differentially affected in the two groups. By this, the present study provides novel insights into underlying neurophysiological mechanisms of the complex nature of response inhibition deficits in adolescents with AD(H)D, stressing the importance to examine the interplay of theta and alpha band activity more closely to better understand inhibitory control deficits in AD(H)D.

Decoding cognition in neurodevelopmental, psychiatric and neurological conditions with multivariate pattern analysis of EEG data

Multivariate pattern analysis (MVPA) of electroencephalographic (EEG) data represents a revolutionary approach to investigate how the brain encodes information. By considering complex interactions among spatio-temporal features at the individual level, MVPA overcomes the limitations of univariate techniques, which often fail to account for the significant inter- and intra-individual neural variability. This is particularly relevant when studying clinical populations, and therefore MVPA of EEG data has recently started to be employed as a tool to study cognition in brain disorders. Here, we review the insights offered by this methodology in the study of anomalous patterns of neural activity in conditions such as autism, ADHD, schizophrenia, dyslexia, neurological and neurodegenerative disorders, within different cognitive domains (perception, attention, memory, consciousness). Despite potential drawbacks that should be attentively addressed, these studies reveal a peculiar sensitivity of MVPA in unveiling dysfunctional and compensatory neurocognitive dynamics of information processing, which often remain blind to traditional univariate approaches. Such higher sensitivity in characterizing individual neurocognitive profiles can provide unique opportunities to optimise assessment and promote personalised interventions.

Attentional demands in the visual field modulate audiovisual interactions in the temporal domain

Attention and crossmodal interactions are closely linked through a complex interplay at different stages of sensory processing. Within the context of motion perception, previous research revealed that attentional demands alter audiovisual interactions in the temporal domain. In the present study, we aimed to understand the neurophysiological correlates of these attentional modulations. We utilized an audiovisual motion paradigm that elicits auditory time interval effects on perceived visual speed. The audiovisual interactions in the temporal domain were quantified by changes in perceived visual speed across different auditory time intervals. We manipulated attentional demands in the visual field by having a secondary task on a stationary object (i.e., single- vs. dual-task conditions). When the attentional demands were high (i.e., dual-task condition), there was a significant decrease in the effects of auditory time interval on perceived visual speed, suggesting a reduction in audiovisual interactions. Moreover, we found significant differences in both early and late neural activities elicited by visual stimuli across task conditions (single vs. dual), reflecting an overall increase in attentional demands in the visual field. Consistent with the changes in perceived visual speed, the audiovisual interactions in neural signals declined in the late positive component range. Compared with the findings from previous studies using different paradigms, our findings support the view that attentional modulations of crossmodal interactions are not unitary and depend on task-specific components. They also have important implications for motion processing and speed estimation in daily life situations where sensory relevance and attentional demands constantly change.

Decoding the task specificity of post-error adjustments: Features and determinants

Errors typically trigger post-error adjustments aimed at improving subsequent reactions within a single task, but little work has focused on whether these adjustments are task-general or task-specific across different tasks. We collected behavioral and electrophysiological (EEG) data when participants performed a psychological refractory period paradigm. This paradigm required them to complete Task 1 and Task 2 separated by a variable stimulus onset asynchrony (SOA). Behaviorally, post-error slowing and post-error accuracy exhibited task-general features at short SOAs but some task-specific features at long SOAs. EEG results manifest that task-general adjustments had a short-lived effect, whereas task-specific adjustments were long-lasting. Moreover, error awareness specifically conduced to the improvement of subsequent sensory processing and behavior performance in Task 1 (the task where errors occurred). These findings demonstrate that post-error adjustments rely on both transient, task-general interference and longer-lasting, task-specific control mechanisms simultaneously, with error awareness playing a crucial role in determining these mechanisms. We further discuss the contribution of central resources to the task specificity of post-error adjustments.

Domain specificity of error monitoring: An ERP study in young and older adults

Metacognition refers to the ability to monitor and control one's cognitive processes, which plays an important role in decision-making throughout the lifespan. It is still debated whether metacognitive abilities decline with age. Neuroimaging evidence suggests that metacognition is served by domain-specific mechanisms. These domains may differentially decline with increasing age. The current investigates whether the error-related negativity (ERN) and the error positivity (Pe) which reflect error detection and error awareness, respectively, differ across perceptual and memory domains in young and older adults. In total, 38 young adults and 37 older adults completed a classic Flanker Task (perceptual) and an adapted memory-based version. No difference in ERN amplitude was found between young and older adults and across domains. Perceptual ERN peaked earlier than Memory ERN. Memory ΔERN was larger than Perceptual ΔERN. Pe was smaller in older adults and ΔPe was larger for perceptual than memory flanker. Memory Pe peaked earlier in young as compared to older adults. Multivariate analyses of whole scalp data supported cross-domain differences. During the task, ERN decreased in young but not in older adults. Memory Pe decreased in young adults but increased in older adults while no significant change in perceptual Pe was found. The study's findings suggest that neural correlates of error monitoring differ across cognitive domains. Moreover, it was shown that error awareness declines in old age but its within-task dynamics vary across cognitive domains. Possible mechanisms underlying metacognition impairments in aging are discussed.

Attenuated conflict self-referential information facilitating conflict resolution

Self-referential information can reduce the congruency effect by acting as a signal to enhance cognitive control. However, it cannot be denied that self-referential information can attract and hold attention. To investigate this issue, the study used a revised Stroop task and recorded behavioral and electrophysiological data from thirty-three participants. We combined event-related potential (ERP) and multivariate pattern analysis (MVPA) to examine the neural correlates of self-referential processing and conflict processing. In the behavioral results, self-referential information reduced the congruency effect. Specifically, self-reference stimuli elicited smaller N2 amplitude than non-self-reference stimuli, indicating that self-referential information was promptly identified and reduced top-down cognitive resource consumption. Self-referential information could be reliably decoded from ERP signals in the early-to-mid stage. Moreover, self-reference conditions exhibited earlier congruency decoding than non-self-reference conditions, facilitating conflict monitoring. In the late stage, under the incongruent condition, self-reference stimuli elicited smaller sustained potential amplitude than non-self-reference stimuli, indicating that cognitive control in the self-reference condition required fewer cognitive resources for conflict resolution. Together, these findings revealed that self-referential information was identified and facilitated conflict monitoring, leading to more effective conflict resolution.

An early effect of the parafoveal preview on post-saccadic processing of English words

A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.

Common and unique neurophysiological signatures for the stopping and revising of actions reveal the temporal dynamics of inhibitory control

Inhibitory control is a crucial cognitive-control ability for behavioral flexibility that has been extensively investigated through action-stopping tasks. Multiple neurophysiological features have been proposed to represent 'signatures' of inhibitory control during action-stopping, though the processes signified by these signatures are still controversially discussed. The present study aimed to disentangle these processes by comparing simple stopping situations with those in which additional action revisions were needed. Three experiments in female and male humans were performed to characterize the neurophysiological dynamics involved in action-stopping and - changing, with hypotheses derived from recently developed two-stage 'pause-then-cancel' models of inhibitory control. Both stopping and revising an action triggered an early broad 'pause'-process, marked by frontal EEG β-bursts and non-selective suppression of corticospinal excitability. However, partial-EMG responses showed that motor activity was only partially inhibited by this 'pause', and that this activity can be further modulated during action-revision. In line with two-stage models of inhibitory control, subsequent frontocentral EEG activity after this initial 'pause' selectively scaled depending on the required action revisions, with more activity observed for more complex revisions. This demonstrates the presence of a selective, effector-specific 'retune' phase as the second process involved in action-stopping and -revision. Together, these findings show that inhibitory control is implemented over an extended period of time and in at least two phases. We are further able to align the most commonly proposed neurophysiological signatures to these phases and show that they are differentially modulated by the complexity of action-revision.

Choice overload interferes with early processing and necessitates late compensation: Evidence from electroencephalogram

Having a multitude of choices can be advantageous, yet an abundance of options can be detrimental to the decision-making process. Based on existing research, the present study combined electroencephalogram and self-reported methodologies to investigate the neural mechanisms underlying the phenomenon of choice overload. Behavioural data suggested that an increase in the number of options led to negative evaluations and avoidance of choice tendencies, even in the absence of time pressure. Event-related potential results indicated that the large choice set interfered with the early visual process, as evidenced by the small P1 amplitude, and failed to attract more attentional resources in the early stage, as evidenced by the small amplitude of P2 and N2. However, the LPC amplitude was increased in the late stage, suggesting greater investment of attentional resources and higher emotional arousal. Multivariate pattern analysis revealed that the difference between small and large choice set began at around 120 ms, and the early and late stages were characterised by opposite activation patterns. This suggested that too many options interfered with early processing and necessitate continued processing at a later stage. In summary, both behavioural and event-related potential (ERP) results confirm the choice overload effect, and it was observed that individuals tend to subjectively exaggerate the choice overload effect.

The neural dynamics of conflict adaptation induced by conflict observation: Evidence from univariate and multivariate analysis

Conflict adaptation can be expressed as greater performance (shorter response time and lower error rate) after incongruent trials when compared to congruent trials. It has been observed in designs that minimize confounding factors, i.e., feature integration, contingency learning, and temporal learning. Our current study aimed to further elucidate the temporal evolution mechanisms of conflict adaptation. To address this issue, the current study employed a combination of behavioral, univariate, and multivariate analysis (MVPA) methods in a modified color-word Stroop task, where half of the trials required button presses (DO trials), and the other half only required observation (LOOK trials). Both behavioral and the ERP results (N450 and SP) in the LOOK-DO transition trials revealed significant conflict adaptation without feature integration, contingency learning, and temporal learning, providing support for the conflict monitoring theory. Furthermore, during the LOOK trials, significant Stroop effect in the N450 and SP components were observed, indicating that conflict monitoring occurred at the stimulus level and triggered reactive control adjustments. The MVPA results decoded the congruent-incongruent and incongruent-incongruent conditions during the conflict adjustment phase but not during the conflict monitoring phase, emphasizing the unique contribution of conflict adjustment to conflict adaptation. The current research findings provided more compelling supporting evidence for the conflict monitoring theory, while also indicating that future studies should employ the present design to elucidate the specific processes of conflict adaptation.

Adaptive adjustment after conflict with group opinion: evidence from neural electrophysiology

Individuals inherently seek social consensus when making decisions or judgments. Previous studies have consistently indicated that dissenting group opinions are perceived as social conflict that demands attitude adjustment. However, the neurocognitive processes of attitude adjustment are unclear. In this electrophysiological study, participants were recruited to perform a face attractiveness judgment task. After forming their own judgment of a face, participants were informed of a purported group judgment (either consistent or inconsistent with their judgment), and then, critically, the same face was presented again. The neural responses to the second presented faces were measured. The second presented faces evoked a larger late positive potential after conflict with group opinions than those that did not conflict, suggesting that more motivated attention was allocated to stimulus. Moreover, faces elicited greater midfrontal theta (4-7 Hz) power after conflict with group opinions than after consistency with group opinions, suggesting that cognitive control was initiated to support attitude adjustment. Furthermore, the mixed-effects model revealed that single-trial theta power predicted behavioral change in the Conflict condition, but not in the No-Conflict condition. These findings provide novel insights into the neurocognitive processes underlying attitude adjustment, which is crucial to behavioral change during conformity.

Prolonged neural encoding of visual information in autism

Autism spectrum disorder (ASD) is associated with a hyper-focused visual attentional style, impacting higher-order social and affective domains. The understanding of such peculiarity can benefit from the use of multivariate pattern analysis (MVPA) of high-resolution electroencephalography (EEG) data, which has proved to be a powerful technique to investigate the hidden neural dynamics orchestrating sensory and cognitive processes. Here, we recorded EEG in typically developing (TD) children and in children with ASD during a visuo-spatial attentional task where attention was exogenously captured by a small (zoom-in) or large (zoom-out) cue in the visual field before the appearance of a target at different eccentricities. MVPA was performed both in the cue-locked period, to reveal potential differences in the modulation of the attentional focus, and in the target-locked period, to reveal potential cascade effects on stimulus processing. Cue-locked MVPA revealed that while in the TD group the pattern of neural activity contained information about the cue mainly before the target appearance, the ASD group showed a temporally sustained and topographically diffuse significant decoding of the cue neural response even after the target onset, suggesting a delayed extinction of cue-related neural activity. Crucially, this delayed extinction positively correlated with behavioral measures of attentional hyperfocusing. Results of target-locked MVPA were coherent with a hyper-focused attentional profile, highlighting an earlier and stronger decoding of target neural responses in small cue trials in the ASD group. The present findings document a spatially and temporally overrepresented encoding of visual information in ASD, which can constitute one of the main reasons behind their peculiar cognitive style.

Identification and effective connections of core networks in patients with temporal lobe epilepsy and cognitive impairment: Granger causality analysis and multivariate pattern analysis

OBJECTIVE

This study aimed to explore effective connectivity (EC) of the core networks in cognition impairment associated with temporal lobe epilepsy (CI-TLE) by applying resting state and Granger causality analysis (REST-GCA). The specific brain regions that played a critical role in classification were assessed using multivariate pattern analysis (MVPA).

METHODS

Thirty-two patients with CI-TLE and 29 healthy controls who were matched based on age and gender underwent functional magnetic resonance imaging (fMRI).

RESULTS

REST-GCA revealed that patients with CI-TLE displayed decreased GC values in the following brain areas: from the posterior cingulate cortex (PCC) to the left fusiform gyrus (lFFG) and the right parahippocampal gyrus (rPPG); from the right dorsal prefrontal cortex (rDPFC) to the left superior parietal lobule (lSPL); from the left amygdala (lAG) to the PCC. Inhibitory EC was observed from the rDPFC to the PCC compared to HCs. The GC values increased from the right dorsal prefrontal cingulate cortex (rdACC) to the PCC and from the right dorsal forebrain insula (rDAI) to the right middle temporal gyrus (rMTG) in the CI-TLE patients. MVPA showed that the classification yielded an accuracy of 81.91% (78.12%, specificity =85.71%).

CONCLUSION

Our observations indicated that the abnormal EC between the frontal and parietal regions might be associated with the pathophysiological mechanism of CI-TLE. These results also indicated that EC might be play a role as a potential discriminative pattern to detect CI-TLE in patients.

Temporal characteristics of emotional conflict processing: the modulation role of attachment styles

Theoretical account of attachment proposed that individual differences in adult attachment styles play a key role in adjusting balance between affective evaluation and cognitive control. Yet, little is known about the temporal characteristics of emotional conflict processing modulated by attachment styles. Accordingly, the present study used event-related potentials (ERP) and multivariate pattern analysis (MVPA) combined with an emotional face-word Stroop task to investigate the temporal dynamics of attachment-related cognitive-affective patterns in emotional conflict processing. The ERP results demonstrated multiple-process of emotional conflict modulated by attachment styles. In early sensory processing, positive faces captured avoidant attachment individuals' attention as reflected in greater P1, while the same situation led to greater N170 in secure and anxious individuals. Crucially, impairment in conflict-monitoring function was found in anxious individuals as reflected by the absence of interference effect on N450, leading to impaired ability of inhibitory control as indicated by decreased slow potential. In contrast, avoidant individuals showed greater slow potential for inhibiting emotional interference. Furthermore, MVPA revealed that the corresponding time window for conflict monitoring was found for emotional distractors decoding rather than congruency decoding in the anxious attachment group. Convergent results from ERPs and MVPA indicated that the deficits in emotional conflict monitoring and resolution among anxious individuals might be due to the excessive approach to emotional distractors, as they habitually use emotional evaluation rather than cognitive control. In summary, the present study provides electrophysiological evidence that attachment styles modulated emotional conflict processing, which highlights the contribution of attachment to social information processing.

The role of distractors in rapid serial visual presentation reveals the mechanism of attentional blink by EEG-based univariate and multivariate analyses

Attentional blink pertains to the performance of participants with a severe decline in identifying the second target presented after the first target reported correctly within 200-500 ms in a rapid serial visual presentation. The current study was conducted to investigate the neural mechanism of the effect of the distractor (D1) that immediately follows first target to attentional blink by altering whether D1 was substituted with a blank with electroencephalography recording. The results showed that D1 interfered with the attentional enhancement and working memory encoding in both single-target rapid serial visual presentation task and dual-target rapid serial visual presentation task, which were mainly manifested in delayed and attenuated P3a and diminished P3b of first target. Single-trial analysis indicated that first target and second target will compete with each other for working memory encoding resources in short lag, but not in the long lag. In addition, D1 interfered with the working memory encoding of second target under short lag rather than long lag in the dual-target rapid serial visual presentation task. These results suggested that attentional blink can be attributed to the limited working memory encoding resource, whereas the amount of available resources is subject to modulation by attention. The D1 hinders the attention enhancement of first target, thereby exacerbating attentional blink.

Suppressive Control of Incentive Salience in Real-World Human Vision

Reward-related activity in the dopaminergic midbrain is thought to guide animal behavior, in part by boosting the perceptual and attentional processing of reward-predictive environmental stimuli. In line with this incentive salience hypothesis, studies of human visual search have shown that simple synthetic stimuli, such as lines, shapes, or Gabor patches, capture attention to their location when they are characterized by reward-associated visual features, such as color. In the real world, however, we commonly search for members of a category of visually heterogeneous objects, such as people, cars, or trees, where category examples do not share low-level features. Is attention captured to examples of a reward-associated real-world object category? Here, we have human participants search for targets in photographs of city and landscapes that contain task-irrelevant examples of a reward-associated category. We use the temporal precision of EEG machine learning and ERPs to show that these distractors acquire incentive salience and draw attention, but do not capture it. Instead, we find evidence of rapid, stimulus-triggered attentional suppression, such that the neural encoding of these objects is degraded relative to neutral objects. Humans appear able to suppress the incentive salience of reward-associated objects when they know these objects will be irrelevant, supporting the rapid deployment of attention to other objects that might be more useful. Incentive salience is thought to underlie key behaviors in eating disorders and addiction, among other conditions, and the kind of suppression identified here likely plays a role in mediating the attentional biases that emerge in these circumstances.Significance Statement Like other animals, humans are prone to notice and interact with environmental objects that have proven rewarding in earlier experience. However, it is common that such objects have no immediate strategic use and are therefore distracting. Do these reward-associated real-world objects capture our attention, despite our strategic efforts otherwise? Or are we able to strategically control the impulse to notice them? Here we use machine learning classification of human electrical brain activity to show that we can establish strategic control over the salience of naturalistic reward-associated objects. These objects draw our attention, but do not necessarily capture it, and this kind of control may play an important role in mediating conditions like eating disorder and addiction.

The neural dynamics of familiarity-dependent face identity representation

Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.

Vicarious touch: Overlapping neural patterns between seeing and feeling touch

Simulation theories propose that vicarious touch arises when seeing someone else being touched triggers corresponding representations of being touched. Prior electroencephalography (EEG) findings show that seeing touch modulates both early and late somatosensory responses (measured with or without direct tactile stimulation). Functional Magnetic Resonance Imaging (fMRI) studies have shown that seeing touch increases somatosensory cortical activation. These findings have been taken to suggest that when we see someone being touched, we simulate that touch in our sensory systems. The somatosensory overlap when seeing and feeling touch differs between individuals, potentially underpinning variation in vicarious touch experiences. Increases in amplitude (EEG) or cerebral blood flow response (fMRI), however, are limited in that they cannot test for the information contained in the neural signal: seeing touch may not activate the same information as feeling touch. Here, we use time-resolved multivariate pattern analysis on whole-brain EEG data from people with and without vicarious touch experiences to test whether seen touch evokes overlapping neural representations with the first-hand experience of touch. Participants felt touch to the fingers (tactile trials) or watched carefully matched videos of touch to another person's fingers (visual trials). In both groups, EEG was sufficiently sensitive to allow decoding of touch location (little finger vs. thumb) on tactile trials. However, only in individuals who reported feeling touch when watching videos of touch could a classifier trained on tactile trials distinguish touch location on visual trials. This demonstrates that, for people who experience vicarious touch, there is overlap in the information about touch location held in the neural patterns when seeing and feeling touch. The timecourse of this overlap implies that seeing touch evokes similar representations to later stages of tactile processing. Therefore, while simulation may underlie vicarious tactile sensations, our findings suggest this involves an abstracted representation of directly felt touch.

EEG Reveals Alterations in Motor Imagery in People With Amnestic Mild Cognitive Impairment

OBJECTIVES

Motor imagery has been used to investigate the cognitive mechanism of motor control. Although behavioral and electrophysiological changes in motor imagery in people with amnestic mild cognitive impairment (aMCI) have been reported, deficits in different types of imagery remain unclear. To explore this question, we used electroencephalography (EEG) to study neural correlates of visual imagery (VI) and kinesthetic imagery (KI) and their relationship to cognitive function in people with aMCI.

METHODS

A hand laterality judgment task was used to induce implicit motor imagery in 29 people with aMCI and 40 healthy controls during EEG recording. Mass univariate and multivariate EEG analysis was applied to explore group differences in a data-driven manner.

RESULTS

Modulation of stimuli orientation to event-related potential (ERP) amplitudes differed significantly between groups at 2 clusters located in the posterior-parietal and frontal areas. Multivariate decoding revealed sufficient representation of VI-related orientation features in both groups. Relative to healthy controls, the aMCI group lacked accurate representation of KI-related biomechanical features, suggesting deficits in automatic activation of KI strategy. Electrophysiological correlates were associated with episodic memory, visuospatial function, and executive function. Higher decoding accuracy of biomechanical features predicted better executive function via longer response time in the imagery task in the aMCI group.

DISCUSSION

These findings reveal electrophysiological correlates related to motor imagery deficits in aMCI, including local ERP amplitudes and large-scale activity patterns. Alterations in EEG activity are related to cognitive function in multiple domains, including episodic memory, suggesting the potential of these EEG indices as biomarkers of cognitive impairment.

Decoding the content of working memory in school-aged children

Developmental improvements in working memory (WM) maintenance predict many real-world outcomes, including educational attainment. It is thus critical to understand which WM mechanisms support these behavioral improvements, and how WM maintenance strategies might change through development. One challenge is that specific WM neural mechanisms cannot easily be measured behaviorally, especially in a child population. However, new multivariate decoding techniques have been designed, primarily in adult populations, that can sensitively decode the contents of WM. The goal of this study was to deploy multivariate decoding techniques known to decode memory representations in adults to decode the contents of WM in children. We created a simple computerized WM game for children, in which children maintained different categories of information (visual, spatial or verbal). We collected electroencephalography (EEG) data from 20 children (7-12-year-olds) while they played the game. Using Multivariate Pattern Analysis (MVPA) on children's EEG signals, we reliably decoded the category of the maintained information during the sensory and maintenance period. Across exploratory reliability and validity analyses, we examined the robustness of these results when trained on less data, and how these patterns generalized within individuals throughout the testing session. Furthermore, these results matched theory-based predictions of WM across individuals and across ages. Our proof-of-concept study proposes a direct and age-appropriate potential alternative to exclusively behavioral WM maintenance measures in children. Our study demonstrates the utility of MVPA to measure and track the uninstructed representational content of children's WM. Future research could use our technique to investigate children's WM maintenance and strategies.

Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Our results illustrate how the widely-used Accuracy (Acc) metric, which measures the overall proportion of successful predictions, yields misleadingly high performances, as class imbalance increases. Because Acc weights the per-class ratios of correct predictions proportionally to class size, it largely disregards the performance on the minority class. A binary classification model that learns to systematically vote for the majority class will yield an artificially high decoding accuracy that directly reflects the imbalance between the two classes, rather than any genuine generalizable ability to discriminate between them. We show that other evaluation metrics such as the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC), and the less common Balanced Accuracy (BAcc) metric - defined as the arithmetic mean between sensitivity and specificity, provide more reliable performance evaluations for imbalanced data. Our findings also highlight the robustness of Random Forest (RF), and the benefits of using stratified cross-validation and hyperprameter optimization to tackle data imbalance. Critically, for neuroscience ML applications that seek to minimize overall classification error, we recommend the routine use of BAcc, which in the specific case of balanced data is equivalent to using standard Acc, and readily extends to multi-class settings. Importantly, we present a list of recommendations for dealing with imbalanced data, as well as open-source code to allow the neuroscience community to replicate and extend our observations and explore alternative approaches to coping with imbalanced data.

Contingent negative variation to tactile stimuli - differences in anticipatory and preparatory processes between participants with and without blindness

People who are blind demonstrate remarkable abilities within the spared senses and compensatory enhancement of cognitive skills, underscored by substantial plastic reorganization in relevant neural areas. However, little is known about whether people with blindness form top-down models of the world on short timescales more efficiently to guide goal-oriented behavior. This electroencephalography study investigates this hypothesis at the neurophysiological level, focusing on contingent negative variation (CNV) as a marker of anticipatory and preparatory processes prior to expected events. In sum, 20 participants with blindness and 27 sighted participants completed a classic CNV task and a memory CNV task, both containing tactile stimuli to exploit the expertise of the former group. Although the reaction times in the classic CNV task did not differ between groups, participants who are blind reached higher performance rates in the memory task. This superior performance co-occurred with a distinct neurophysiological profile, relative to controls: greater late CNV amplitudes over central areas, suggesting enhanced stimulus expectancy and motor preparation prior to key events. Controls, in contrast, recruited more frontal sites, consistent with inefficient sensory-aligned control. We conclude that in more demanding cognitive contexts exploiting the spared senses, people with blindness efficiently generate task-relevant internal models to facilitate behavior.

Classification of age groups and task conditions provides additional evidence for differences in electrophysiological correlates of inhibitory control across the lifespan

The aim of this study was to extend previous findings on selective attention over a lifetime using machine learning procedures. By decoding group membership and stimulus type, we aimed to study differences in the neural representation of inhibitory control across age groups at a single-trial level. We re-analyzed data from 211 subjects from six age groups between 8 and 83 years of age. Based on single-trial EEG recordings during a flanker task, we used support vector machines to predict the age group as well as to determine the presented stimulus type (i.e., congruent, or incongruent stimulus). The classification of group membership was highly above chance level (accuracy: 55%, chance level: 17%). Early EEG responses were found to play an important role, and a grouped pattern of classification performance emerged corresponding to age structure. There was a clear cluster of individuals after retirement, i.e., misclassifications mostly occurred within this cluster. The stimulus type could be classified above chance level in ~ 95% of subjects. We identified time windows relevant for classification performance that are discussed in the context of early visual attention and conflict processing. In children and older adults, a high variability and latency of these time windows were found. We were able to demonstrate differences in neuronal dynamics at the level of individual trials. Our analysis was sensitive to mapping gross changes, e.g., at retirement age, and to differentiating components of visual attention across age groups, adding value for the diagnosis of cognitive status across the lifespan. Overall, the results highlight the use of machine learning in the study of brain activity over a lifetime.

Studying memory processes at different levels with simultaneous depth and surface EEG recordings

Investigating cognitive brain functions using non-invasive electrophysiology can be challenging due to the particularities of the task-related EEG activity, the depth of the activated brain areas, and the extent of the networks involved. Stereoelectroencephalographic (SEEG) investigations in patients with drug-resistant epilepsy offer an extraordinary opportunity to validate information derived from non-invasive recordings at macro-scales. The SEEG approach can provide brain activity with high spatial specificity during tasks that target specific cognitive processes (e.g., memory). Full validation is possible only when performing simultaneous scalp SEEG recordings, which allows recording signals in the exact same brain state. This is the approach we have taken in 12 subjects performing a visual memory task that requires the recognition of previously viewed objects. The intracranial signals on 965 contact pairs have been compared to 391 simultaneously recorded scalp signals at a regional and whole-brain level, using multivariate pattern analysis. The results show that the task conditions are best captured by intracranial sensors, despite the limited spatial coverage of SEEG electrodes, compared to the whole-brain non-invasive recordings. Applying beamformer source reconstruction or independent component analysis does not result in an improvement of the multivariate task decoding performance using surface sensor data. By analyzing a joint scalp and SEEG dataset, we investigated whether the two types of signals carry complementary information that might improve the machine-learning classifier performance. This joint analysis revealed that the results are driven by the modality exhibiting best individual performance, namely SEEG.

Neural mechanisms of expert persuasion on willingness to pay for sugar

Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.

Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.

Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.

Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

Neurophysiological principles of inhibitory control processes during cognitive flexibility

Inhibitory control plays an indispensable role in cognitive flexibility. Nevertheless, the neurophysiological principles underlying this are incompletely understood. This owes to the fact that the representational dynamics, as coded in oscillatory neural activity of different frequency bands has not been considered until now-despite being of conceptual relevance. Moreover, it is unclear in how far distinct functional neuroanatomical regions are concomitantly involved in the processing of representational dynamics. We examine these questions using a combination of EEG methods. We show that theta-band activity plays an essential role for inhibitory control processes during cognitive flexibility across informational aspects coded in distinct fractions of the neurophysiological signal. It is shown that posterior parietal structures and the inferior parietal cortex seem to be the most important cortical region for inhibitory control processes during cognitive flexibility. Theta-band activity plays an essential role in processes of retrieving the previously inhibited representations related to the current task during cognitive flexibility. The representational content relevant for inhibitory processes during cognitive flexibility is coded in the theta frequency band. We outline how the observed neural mechanisms inform recent overarching cognitive frameworks on how flexible action control is accomplished.

On the Role of Stimulus-Response Context in Inhibitory Control in Alcohol Use Disorder

The behavioral and neural dynamics of response inhibition deficits in alcohol use disorder (AUD) are still largely unclear, despite them possibly being key to the mechanistic understanding of the disorder. Our study investigated the effect of automatic vs. controlled processing during response inhibition in participants with mild-to-moderate AUD and matched healthy controls. For this, a Simon Nogo task was combined with EEG signal decomposition, multivariate pattern analysis (MVPA), and source localization methods. The final sample comprised n = 59 (32♂) AUD participants and n = 64 (28♂) control participants. Compared with the control group, AUD participants showed overall better response inhibition performance. Furthermore, the AUD group was less influenced by the modulatory effect of automatic vs. controlled processes during response inhibition (i.e., had a smaller Simon Nogo effect). The neurophysiological data revealed that the reduced Simon Nogo effect in the AUD group was associated with reduced activation differences between congruent and incongruent Nogo trials in the inferior and middle frontal gyrus. Notably, the drinking frequency (but not the number of AUD criteria we had used to distinguish groups) predicted the extent of the Simon Nogo effect. We suggest that the counterintuitive advantage of participants with mild-to-moderate AUD over those in the control group could be explained by the allostatic model of drinking effects.

Decoding continuous variables from event-related potential (ERP) data with linear support vector regression using the Decision Decoding Toolbox (DDTBOX)

Multivariate classification analysis for event-related potential (ERP) data is a powerful tool for predicting cognitive variables. However, classification is often restricted to categorical variables and under-utilises continuous data, such as response times, response force, or subjective ratings. An alternative approach is support vector regression (SVR), which uses single-trial data to predict continuous variables of interest. In this tutorial-style paper, we demonstrate how SVR is implemented in the Decision Decoding Toolbox (DDTBOX). To illustrate in more detail how results depend on specific toolbox settings and data features, we report results from two simulation studies resembling real EEG data, and one real ERP-data set, in which we predicted continuous variables across a range of analysis parameters. Across all studies, we demonstrate that SVR is effective for analysis windows ranging from 2 to 100 ms, and relatively unaffected by temporal averaging. Prediction is still successful when only a small number of channels encode true information, and the analysis is robust to temporal jittering of the relevant information in the signal. Our results show that SVR as implemented in DDTBOX can reliably predict continuous, more nuanced variables, which may not be well-captured by classification analysis. In sum, we demonstrate that linear SVR is a powerful tool for the investigation of single-trial EEG data in relation to continuous variables, and we provide practical guidance for users.

Evidence for independent representational contents in inhibitory control subprocesses associated with frontoparietal cortices

Inhibitory control processes have intensively been studied in cognitive science for the past decades. Even though the neural dynamics underlying these processes are increasingly better understood, a critical open question is how the representational dynamics of the inhibitory control processes are modulated when engaging in response inhibition in a relatively automatic or a controlled mode. Against the background of an overarching theory of perception-action integration, we combine temporal and spatial EEG signal decomposition methods with multivariate pattern analysis and source localization to obtain fine-grained insights into the neural dynamics of the representational content of response inhibition. For this purpose, we used a sample of N = 40 healthy adult participants. The behavioural data suggest that response inhibition was better in a more controlled than a more automated response execution mode. Regarding neural dynamics, effects of response inhibition modes relied on a concomitant coding of stimulus-related information and rules of how stimulus information is related to the appropriate motor programme. Crucially, these fractions of information, which are encoded at the same time in the neurophysiological signal, are based on two independent spatial neurophysiological activity patterns, also showing differences in the temporal stability of the representational content. Source localizations revealed that the precuneus and inferior parietal cortex regions are more relevant than prefrontal areas for the representation of stimulus-response selection codes. We provide a blueprint how a concatenation of EEG signal analysis methods, capturing distinct aspects of neural dynamics, can be connected to cognitive science theory on the importance of representations in action control.

A ventral stream-prefrontal cortex processing cascade enables working memory gating dynamics

The representation of incoming information, goals and the flexible processing of these are required for cognitive control. Efficient mechanisms are needed to decide when it is important that novel information enters working memory (WM) and when these WM 'gates' have to be closed. Compared to neural foundations of maintaining information in WM, considerably less is known about what neural mechanisms underlie the representational dynamics during WM gating. Using different EEG analysis methods, we trace the path of mental representations along the human cortex during WM gate opening and closing. We show temporally nested representational dynamics during WM gate opening and closing depending on multiple independent neural activity profiles. These activity profiles are attributable to a ventral stream-prefrontal cortex processing cascade. The representational dynamics start in the ventral stream during WM gate opening and WM gate closing before prefrontal cortical regions are modulated. A regional specific activity profile is shown within the prefrontal cortex depending on whether WM gates are opened or closed, matching overarching concepts of prefrontal cortex functions. The study closes an essential conceptual gap detailing the neural dynamics underlying how mental representations drive the WM gate to open or close to enable WM functions such as updating and maintenance.

Alpha Oscillations Track Content-Specific Working Memory Capacity

Although the neural basis of working memory (WM) capacity is often studied by exploiting interindividual differences, capacity may also differ across memory materials within a given individual. Here, we exploit the content dependence of WM capacity as a novel approach to investigate the oscillatory correlates of WM capacity, focusing on posterior 9-12 Hz alpha activity during retention. We recorded scalp electroencephalography (EEG) while male and female human participants performed WM tasks with varying memory loads (two vs. four items) and materials (English letters vs. regular shapes vs. abstract shapes). First, behavioral data confirmed that memory capacity was fundamentally content dependent; capacity for abstract shapes plateaued at around two, whereas the participants could remember more letters and regular shapes. Critically, content-specific capacity was paralleled in the degree of attenuation of EEG-alpha activity that plateaued in a similar content-specific manner. Although we observed greater alpha attenuation for higher loads for all materials, we found larger load effects for letters and regular shapes than for abstract shapes, which is consistent with our behavioral data showing a lower capacity plateau for abstract shapes. Moreover, when only considering two-item trials, alpha attenuation was greater for abstract shapes where two items were close to the capacity plateau than for other materials. Multivariate decoding of alpha activity patterns reinforced these findings. Finally, for each material, load effects on capacity (K) and alpha attenuation were correlated across individuals. Our results demonstrate that alpha oscillations track memory capacity in a content-specific manner and track not just the number of items but also their complexity.SIGNIFICANCE STATEMENT WM is limited in its capacity. We show that capacity is not fixed for an individual but is rather memory-content dependent. Moreover, we used this as a novel approach to investigate the neural basis of WM capacity with EEG. We found that both behavioral capacity estimates and neural oscillations in the alpha band varied with memory loads and materials. The critical finding is a capacity plateau of approximately two items only for the more complex materials, accompanied by a similar plateau in the EEG alpha attenuation. The load effects on capacity and alpha attenuation were furthermore correlated across individuals for each of the materials. Our results demonstrate that alpha oscillations track the content-specific nature of WM capacity.

Cognitive science theory-driven pharmacology elucidates the neurobiological basis of perception-motor integration

An efficient integration of sensory and motor processes is crucial to goal-directed behavior. Despite this high relevance, and although cognitive theories provide clear conceptual frameworks, the neurobiological basis of these processes remains insufficiently understood. In a double-blind, randomized placebo-controlled pharmacological study, we examine the relevance of catecholamines for perception-motor integration processes. Using EEG data, we perform an in-depth analysis of the underlying neurophysiological mechanisms, focusing on sensorimotor integration processes during response inhibition. We show that the catecholaminergic system affects sensorimotor integration during response inhibition by modulating the stability of the representational content. Importantly, catecholamine levels do not affect the stability of all aspects of information processing during sensorimotor integration, but rather-as suggested by cognitive theory-of specific codes in the neurophysiological signal. Particularly fronto-parietal cortical regions are associated with the identified mechanisms. The study shows how cognitive science theory-driven pharmacology can shed light on the neurobiological basis of perception-motor integration and how catecholamines affect specific information codes relevant to cognitive control.

Decoding the Specificity of Post-Error Adjustments Using EEG-Based Multivariate Pattern Analysis

Errors can elicit post-error adjustments that serve to optimize performance by preventing further errors. An essential but unsolved issue is that whether post-error adjustments are domain-general or domain-specific, which was investigated in the present study through eliciting different types of errors. Behavioral and electrophysiological data were recorded when male and female subjects performed the Eriksen flanker task. For this study, we examined the aforementioned issue by combining event-related potential and multivariate pattern analysis. The results indicated that post-error slowing, error-related negativity, and error positivity were comparable between congruent and incongruent errors, indicating that errors triggered domain-general interference mechanisms, whereas post-error accuracy and late positive potential elicited by incongruent errors were larger than those elicited by congruent errors, exhibiting domain-specific control adjustment mechanisms. Importantly, no successful decoding soon after errors was found between congruent and incongruent errors, but above-chance decoding was observed between these two types of errors with increasing time, which further support that domain-general adjustments occurred in the early stage, whereas domain-specific adjustments appeared in the late stage. Furthermore, brain-behavior correlation results suggested that the late post-error adjustments predicted subsequent behavior performance. Together, this study revealed that early domain-general interference adjustments induced by errors are reflected in error detection and error awareness, which are independent of error types; on the contrary, late domain-specific control adjustments are reflected in attentional adjustments, which are modulated by error types.SIGNIFICANCE STATEMENT To date, clear evidence on the specificity of post-error adjustments is lacking. The present study provides neurophysiological evidence that post-error adjustments simultaneously rely on both domain-general and domain-specific mechanisms. Event-related potential results indicated that domain-general adjustments were accompanied by the interference of error detection and error awareness. In contrast, domain-specific adjustments were associated with attentional adjustments. Multivariate pattern analysis further decoded the two features of post-error adjustments in the early stage matching the time patterns of error-related negativity and error positivity and in the late stage corresponding to the late positive potential. Temporal generalization analysis showed that domain-specific processing appeared stably in late post-error adjustments. Hence, we propose that post-error different stages may determine the specificity of post-error adjustments.

The neural stability of perception-motor representations affects action outcomes and behavioral adaptation

Actions can fail - even though this is well known, little is known about what distinguishes neurophysiological processes preceding errors and correct actions. In this study, relying on the Theory of Event Coding, we test the assumption that only specific aspects of information coded in EEG activity are relevant for understanding processes leading to response errors. We examined N = 69 healthy participants who performed a mental rotation task and combined temporal EEG signal decomposition with multivariate pattern analysis (MVPA) and source localization analyses. We show that fractions of the EEG signal, primarily representing stimulus-response translation (event file) processes and motor response representations, are essential. Stimulus representations were less critical. The source localization results revealed widespread activity modulations in structures including the frontopolar, the middle and superior frontal, the anterior cingulate cortex, the cuneus, the inferior parietal cortex, and the ventral stream regions. These are associated with differential effects of the neural dynamics preceding correct/erroneous responses. The temporal-generalization MVPA showed that event file representations and representations of the motor response were already distinct 200 ms after stimulus presentation and this lasted till around 700 ms. The stability of this representational content was predictive for the magnitude of posterror slowing, which was particularly strong when there was no clear distinction between the neural activity profile of event file representations associated with a correct or an erroneous response. The study provides a detailed analysis of the dynamics leading to an error/correct response in connection to an overarching framework on action control.

What makes somatosensory short-term memory maintenance effective? An EEG study comparing contralateral delay activity between sighted participants and participants who are blind

Somatosensory short-term memory is essential for object recognition, sensorimotor learning, and, especially, Braille reading for people who are blind. This study examined how visual sensory deprivation and a compensatory focus on somatosensory information influences memory processes in this domain. We measured slow cortical negativity developing during short-term tactile memory maintenance (tactile contralateral delay activity, tCDA) in frontal and somatosensory areas while a sample of 24 sighted participants and 22 participants who are blind completed a tactile change-detection task where varying loads of Braille pin patterns served as stimuli. Auditory cues, appearing at varying latencies between sample arrays, could be used to reduce memory demands during maintenance. Participants who are blind (trained Braille readers) outperformed sighted participants behaviorally. In addition, while task-related frontal activation featured in both groups, participants who are blind uniquely showed higher tCDA amplitudes specifically over somatosensory areas. The site specificity of this component's functional relevance in short-term memory maintenance was further supported by somatosensory tCDA amplitudes first correlating across the whole sample with behavioral performance, and secondly showing sensitivity to varying memory load. The results substantiate sensory recruitment models and provide new insights into the effects of visual sensory deprivation on tactile processing. Between-group differences in the interplay between frontal and somatosensory areas during somatosensory maintenance also suggest that efficient maintenance of complex tactile stimuli in short-term memory is primarily facilitated by lateralized activity in somatosensory cortex.

On the Role of Memory Representations in Action Control: Neurophysiological Decoding Reveals the Reactivation of Integrated Stimulus-Response Feature Representations

Efficient response selection is essential to flexible, goal-directed behavior. Prominent theoretical frameworks such as the Theory of Event Coding and Binding and Retrieval in Action Control have provided insights regarding the dynamics of perception-action integration processes. According to Theory of Event Coding and Binding and Retrieval in Action Control, encoded representations of stimulus-response bindings influence later retrieval processes of these bindings. However, this concept still lacks conclusive empirical evidence. In the current study, we applied representational decoding to EEG data. On the behavioral level, the findings replicated binding effects that have been established in previous studies: The task performance was impaired when an event file had to be reconfigured. The EEG-decoding results showed that retrieval processes of stimulus-response bindings could be decoded using the representational content developed after the initial establishment of these stimulus-response bindings. We showed that stimulus-related properties became immediately reactivated when re-encountering the respective stimulus-response association. These reactivations were temporally stable. In contrast, representations of stimulus-response mappings revealed a transient pattern of activity and could not successfully be decoded directly after stimulus-response binding. Information detailing the bindings between stimuli and responses were also retrieved, but only after having been loaded into a memory system. The current study supports the notion that stimulus-response integration and memory processes are intertwined at multiple levels.

Protocol to decode representations from EEG data with intermixed signals using temporal signal decomposition and multivariate pattern-analysis

The electroencephalogram (EEG) is one of the most widely used techniques in cognitive neuroscience. We present a protocol showing how to combine a temporal signal decomposition approach (RIDE, Residue iteration decomposition) with multivariate pattern analysis (MVPA) to obtain insights into the temporal stability of representations coded in distinct informational fractions of the EEG signal. In this protocol, we describe pre-processing of human EEG data, followed by the set-up and use of MATLAB-based toolboxes for RIDE and MVPA analysis.

For complete details on the use and execution of this protocol, please refer to Petruo et al. (2021).

•

A protocol for decoding temporally decomposed EEG signal

•

Steps for Residue iteration decomposition (RIDE) and handling the decomposed data

•

Steps for subsequent multivariate pattern analysis (MVPA) with different toolboxes

•

Recommendations for combined RIDE-MVPA research applications

Publisher’s note: Undertaking any experimental protocol requires adherence to local institutional guidelines for laboratory safety and ethics.

Contributions of representational distinctiveness and stability to memory performance and age differences

Long-standing theories of cognitive aging suggest that memory decline is associated with age-related differences in the way information is neurally represented. Multivariate pattern similarity analyses enabled researchers to take a representational perspective on brain and cognition, and allowed them to study the properties of neural representations that support successful episodic memory. Two representational properties have been identified as crucial for memory performance, namely the distinctiveness and the stability of neural representations. Here, we review studies that used multivariate analysis tools for different neuroimaging techniques to clarify how these representational properties relate to memory performance across adulthood. While most evidence on age differences in neural representations involved stimulus category information , recent studies demonstrated that particularly item-level stability and specificity of activity patterns are linked to memory success and decline during aging. Overall, multivariate methods offer a versatile tool for our understanding of age differences in the neural representations underlying memory.

Banknote authenticity is signalled by rapid neural responses

Authenticating valuable objects is widely assumed to involve protracted scrutiny for detection of reproduction flaws. Yet, accurate authentication of banknotes is possible within one second of viewing, suggesting that rapid neural processes may underpin counterfeit detection. To investigate, we measured event-related brain potentials (ERPs) in response to briefly viewed genuine or forensically recovered counterfeit banknotes presented in a visual oddball counterfeit detection task. Three ERP components, P1, P3, and extended P3, were assessed for each combination of banknote type (genuine, counterfeit) and overt response (“real”, “fake”). P1 amplitude was greater for oddballs, demonstrating that the initial feedforward sweep of visual processing yields the essential information for differentiating genuine from counterfeit. A similar oddball effect was found for P3. The magnitude of this P3 effect was positively correlated with behavioural counterfeit sensitivity, although the corresponding correlation for P1 was not. For the extended P3, amplitude was greatest for correctly detected counterfeits and similarly small for missed counterfeits, incorrectly and correctly categorised genuine banknotes. These results show that authentication of complex stimuli involves a cascade of neural processes that unfolds in under a second, beginning with a very rapid sensory analysis, followed by a later decision stage requiring higher level processing.

Spectral Pattern Similarity Analysis: Tutorial and Application in Developmental Cognitive Neuroscience

The human brain encodes information in neural activation patterns. While standard approaches to analyzing neural data focus on brain (de-)activation (e.g., regarding the location, timing, or magnitude of neural responses), multivariate neural pattern similarity analyses target the informational content represented by neural activity. In adults, a number of representational properties have been identified that are linked to cognitive performance, in particular the stability, distinctiveness, and specificity of neural patterns. However, although growing cognitive abilities across childhood suggest advancements in representational quality, developmental studies still rarely utilize information-based pattern similarity approaches, especially in electroencephalography (EEG) research. Here, we provide a comprehensive methodological introduction and step-by-step tutorial for pattern similarity analysis of spectral (frequency-resolved) EEG data including a publicly available pipeline and sample dataset with data from children and adults. We discuss computation of single-subject pattern similarities and their statistical comparison at the within-person to the between-group level as well as the illustration and interpretation of the results. This tutorial targets both novice and more experienced EEG researchers and aims to facilitate the usage of spectral pattern similarity analyses, making these methodologies more readily accessible for (developmental) cognitive neuroscientists.

Multi-level decoding of task sets in neurophysiological data during cognitive flexibility

Cognitive flexibility is essential to achieve higher level goals. Cognitive theories assume that the activation/deactivation of goals and task rules is central to understand cognitive flexibility. However, how this activation/deactivation dynamic is implemented on a neurophysiological level is unclear. Using EEG-based multivariate pattern analysis (MVPA) methods, we show that activation of relevant information occurs parallel in time at multiple levels in the neurophysiological signal containing aspects of stimulus-related processing, response selection, and motor response execution, and relates to different brain regions. The intensity with which task sets are activated and processed dynamically decreases and increases. The temporal stability of these activations could, however, hardly explain behavioral performance. Instead, task set deactivation processes associated with left orbitofrontal regions and inferior parietal regions selectively acting on motor response task sets are relevant. The study shows how propositions from cognitive theories stressing the importance task set activation/deactivation during cognitive flexibility are implemented on a neurophysiological level.

•

Stimulus-related, motor, and response selection aspects of task set were decoded

•

Activation of task rule information occurs at multiple neurophysiological levels

•

Activation and deactivation of rule sets contributes to cognitive flexibility