Single-trial modeling separates multiple overlapping prediction errors during reward processing in human EEG

Risk-Taking Is Associated with Decreased Subjective Value Signals and Increased Prediction Error Signals in the Hot Columbia Card Task

It remains a pressing concern to understand how neural computations relate to risky decisions. However, most observations of brain-behavior relationships in the risk-taking domain lack a rigorous computational basis or fail to emulate of the dynamic, sequential nature of real-life risky decision-making. Recent advances emphasize the role of neural prediction error (PE) signals. We modeled, according to prospect theory, the choices of n = 43 human participants (33 females, 10 males) performing an EEG version of the hot Columbia Card Task, featuring rounds of sequential decisions between stopping (safe option) and continuing with increasing odds of a high loss (risky option). Single-trial regression EEG analyses yielded a subjective value signal at centroparietal (300-700 ms) and frontocentral (>800 ms) electrodes and in the delta band, as well as PE signals tied to the feedback-related negativity, P3a, and P3b, and in the theta band. Higher risk preference (total number of risky choices) was linked to attenuated subjective value signals but increased PE signals. Higher P3-like activity associated with the most positive PE in each round predicted stopping in the present round but not risk-taking in the subsequent round. Our findings indicate that decreased representation of decision values and increased sensitivity to winning despite low odds (positive PE) facilitate risky choices at the subject level. Strong neural responses when gains are least expected (the most positive PE on each round) adaptively contribute to safer choices at the trial-by-trial level but do not affect risky choice at the round-by-round level.

Prediction-error-dependent processing of immediate and delayed positive feedback

Learning often involves trial-and-error, i.e. repeating behaviours that lead to desired outcomes, and adjusting behaviour when outcomes do not meet our expectations and thus lead to prediction errors (PEs). PEs have been shown to be reflected in the reward positivity (RewP), an event-related potential (ERP) component between 200 and 350 ms after performance feedback which is linked to striatal processing and assessed via electroencephalography (EEG). Here we show that this is also true for delayed feedback processing, for which a critical role of the hippocampus has been suggested. We found a general reduction of the RewP for delayed feedback, but the PE was similarly reflected in the RewP and the later P300 for immediate and delayed positive feedback, while no effect was found for negative feedback. Our results suggest that, despite processing differences between immediate and delayed feedback, positive PEs drive feedback processing and learning irrespective of delay.

How and when social evaluative feedback is processed in the brain: A systematic review on ERP studies

Social evaluative feedback informs the receiver of the other's views, which may contain judgments of personality-related traits and/or the level of likability. Such kinds of social evaluative feedback are of particular importance to humans. Event-related potentials (ERPs) can directly measure where in the processing stream feedback valence, expectancy, or contextual relevance modulate information processing. This review provides an overview and systematization of studies and early, mid-latency, and late ERP effects. Early effects were inconsistently reported for all factors. Feedback valence effects are more consistently reported for specific mid-latency ERPs (Reward Positivity, RewP, and Early Posterior Negativity, EPN) and late positivities (P3 and Late Positive Potential, LPP). Unexpected feedback consistently increased the Feedback Related Negativity (FRN) and, less consistently, decreased P3 amplitudes. Contextual relevance of the sender (e.g., human vs computer sender) or self-relatedness increased mid-latency to late ERPs. Interactions between valence and other factors were less often found, arising during mid-latency stages, where most consistent interactions showed larger EPN and P3 amplitude differences for valent feedback in a more relevant context. The ERP findings highlight that social evaluative feedback is consistently differentiated during mid-latency processing stages. The review discusses the relevance of findings, possible shortcomings of different experimental designs, and open questions. Furthermore, we suggest concrete venues for future research.

Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex

Orbitofrontal cortex (OFC) is classically linked to inhibitory control, emotion regulation, and reward processing. Recent perspectives propose that the OFC also generates predictions about perceptual events, actions, and their outcomes. We tested the role of the OFC in detecting violations of prediction at two levels of abstraction (i.e., hierarchical predictive processing) by studying the event-related potentials (ERPs) of patients with focal OFC lesions (n = 12) and healthy controls (n = 14) while they detected deviant sequences of tones in a local-global paradigm. The structural regularities of the tones were controlled at two hierarchical levels by rules defined at a local (i.e., between tones within sequences) and at a global (i.e., between sequences) level. In OFC patients, ERPs elicited by standard tones were unaffected at both local and global levels compared to controls. However, patients showed an attenuated mismatch negativity (MMN) and P3a to local prediction violation, as well as a diminished MMN followed by a delayed P3a to the combined local and global level prediction violation. The subsequent P3b component to conditions involving violations of prediction at the level of global rules was preserved in the OFC group. Comparable effects were absent in patients with lesions restricted to the lateral PFC, which lends a degree of anatomical specificity to the altered predictive processing resulting from OFC lesion. Overall, the altered magnitudes and time courses of MMN/P3a responses after lesions to the OFC indicate that the neural correlates of detection of auditory regularity violation are impacted at two hierarchical levels of rule abstraction.

The parietal cortex has a causal role in ambiguity computations in humans

Humans often face the challenge of making decisions between ambiguous options. The level of ambiguity in decision-making has been linked to activity in the parietal cortex, but its exact computational role remains elusive. To test the hypothesis that the parietal cortex plays a causal role in computing ambiguous probabilities, we conducted consecutive fMRI and TMS-EEG studies. We found that participants assigned unknown probabilities to objective probabilities, elevating the uncertainty of their decisions. Parietal cortex activity correlated with the objective degree of ambiguity and with a process that underestimates the uncertainty during decision-making. Conversely, the midcingulate cortex (MCC) encodes prediction errors and increases its connectivity with the parietal cortex during outcome processing. Disruption of the parietal activity increased the uncertainty evaluation of the options, decreasing cingulate cortex oscillations during outcome evaluation and lateral frontal oscillations related to value ambiguous probability. These results provide evidence for a causal role of the parietal cortex in computing uncertainty during ambiguous decisions made by humans.

Asymmetric coding of reward prediction errors in human insula and dorsomedial prefrontal cortex

The signed value and unsigned salience of reward prediction errors (RPEs) are critical to understanding reinforcement learning (RL) and cognitive control. Dorsomedial prefrontal cortex (dMPFC) and insula (INS) are key regions for integrating reward and surprise information, but conflicting evidence for both signed and unsigned activity has led to multiple proposals for the nature of RPE representations in these brain areas. Recently developed RL models allow neurons to respond differently to positive and negative RPEs. Here, we use intracranially recorded high frequency activity (HFA) to test whether this flexible asymmetric coding strategy captures RPE coding diversity in human INS and dMPFC. At the region level, we found a bias towards positive RPEs in both areas which paralleled behavioral adaptation. At the local level, we found spatially interleaved neural populations responding to unsigned RPE salience and valence-specific positive and negative RPEs. Furthermore, directional connectivity estimates revealed a leading role of INS in communicating positive and unsigned RPEs to dMPFC. These findings support asymmetric coding across distinct but intermingled neural populations as a core principle of RPE processing and inform theories of the role of dMPFC and INS in RL and cognitive control.

Unraveling the influence of trial-based motivational changes on performance monitoring stages in a flanker task

Performance monitoring (PM) is a vital component of adaptive behavior and known to be influenced by motivation. We examined effects of potential gain (PG) and loss avoidance (LA) on neural correlates of PM at different processing stages, using a task with trial-based changes in these motivational contexts. Findings suggest more attention is allocated to the PG context, with higher amplitudes for respective correlates of stimulus and feedback processing. The PG context favored rapid responses, while the LA context emphasized accurate responses. Lower response thresholds in the PG context after correct responses derived from a drift-diffusion model also indicate a more approach-oriented response style in the PG context. This cognitive shift is mirrored in neural correlates: negative feedback in the PG context elicited a higher feedback-related negativity (FRN) and higher theta power, whereas positive feedback in the LA context elicited higher P3a and P3b amplitudes, as well as higher theta power. There was no effect of motivational context on response-locked brain activity. Given the similar frequency of negative feedback in both contexts, the elevated FRN and theta power in PG trials cannot be attributed to variations in reward prediction error. The observed variations in the FRN indicate that the effect of outcome valence is modulated by motivational salience.

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Expectations are often dynamic: sports fans know that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here we present behavioral and electrophysiological evidence of sub-second changes in expectations using slot machines as a case study. In Study 1, we demonstrate that EEG signal before the slot machine stops varies based on proximity to winning. Study 2 introduces a behavioral paradigm to measure dynamic expectations via betting, and shows that expectation trajectories vary as a function of winning proximity. Notably, these expectation trajectories parallel Study 1's EEG activity. Studies 3 (EEG) and 4 (behavioral) replicate these findings in the loss domain. These four studies provide compelling evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our research opens promising avenues for understanding the dynamic nature of reward expectations and their impact on cognitive processes.

Neural dissociation between reward and salience prediction errors through the lens of optimistic bias

The question of how the brain represents reward prediction errors is central to reinforcement learning and adaptive, goal-directed behavior. Previous studies have revealed prediction error representations in multiple electrophysiological signatures, but it remains elusive whether these electrophysiological correlates underlying prediction errors are sensitive to valence (in a signed form) or to salience (in an unsigned form). One possible reason concerns the loose correspondence between objective probability and subjective prediction resulting from the optimistic bias, that is, the tendency to overestimate the likelihood of encountering positive future events. In the present electroencephalography (EEG) study, we approached this question by directly measuring participants' idiosyncratic, trial-to-trial prediction errors elicited by subjective and objective probabilities across two experiments. We adopted monetary gain and loss feedback in Experiment 1 and positive and negative feedback as communicated by the same zero-value feedback in Experiment 2. We provided electrophysiological evidence in time and time-frequency domains supporting both reward and salience prediction error signals. Moreover, we showed that these electrophysiological signatures were highly flexible and sensitive to an optimistic bias and various forms of salience. Our findings shed new light on multiple presentations of prediction error in the human brain, which differ in format and functional role.

Event-Related Potential Correlates of Learning to Produce Novel Foreign Phonemes

Learning to pronounce a foreign phoneme requires an individual to acquire a motor program that enables the reproduction of the new acoustic target sound. This process is largely based on the use of auditory feedback to detect pronunciation errors to adjust vocalization. While early auditory evoked neural activity underlies automatic detection and adaptation to vocalization errors, little is known about the neural correlates of acquiring novel speech targets. To investigate the neural processes that mediate the learning of foreign phoneme pronunciation, we recorded event-related potentials when participants (N = 19) pronounced native or foreign phonemes. Behavioral results indicated that the participants' pronunciation of the foreign phoneme improved during the experiment. Early auditory responses (N1 and P2 waves, approximately 85-290 ms after the sound onset) revealed no differences between foreign and native phonemes. In contrast, the amplitude of the frontocentrally distributed late slow wave (LSW, 320-440 ms) was modulated by the pronunciation of the foreign phonemes, and the effect changed during the experiment, paralleling the improvement in pronunciation. These results suggest that the LSW may reflect higher-order monitoring processes that signal successful pronunciation and help learn novel phonemes.

Electrophysiological evidence for the effects of pain on the different stages of reward evaluation under a purchasing situation

Pain and reward have crucial roles in determining human behaviors. It is still unclear how pain influences different stages of reward processing. This study aimed to assess the physical pain’s impact on reward processing with event-related potential (ERP) method. In the present study, a flash sale game (reward-seeking task) was carried out, in which the participants were instructed to press a button as soon as possible to obtain the earphone (a reward) after experiencing either electric shock or not and finally evaluated the outcome of their response. High-temporal-resolution electroencephalogram data were simultaneously recorded to reveal the neural mechanism underlying the pain effect. The ERP analyses revealed that pain affected the feedback processing reflected by feedback-related negativity (FRN) and P300. Specifically, participants in the nopain situation exhibited greater FRN discrepancy between success and failure feedbacks relative to that in the pain situation. Moreover, the P300 amplitude was enhanced in the nopain condition compared to the pain condition regardless of the feedback valence. These results demonstrate that the pain reduced the sensitivity to the reward valence at the early stage and weakened the motivational salience at the late stage. Altogether, this study extends the understanding of the effect of pain on reward processing from the temporal perspective under a purchasing situation.

Expectations of immediate and delayed reward differentially affect cognitive task performance

The current study used a modified Monetary Incentive Delay task to examine the neural mechanisms underlying anticipating and receiving an immediate or delayed reward and examined the influence of pursuing these rewards on cognitive task performance. A pre-cue indicating the potential of gaining a monetary reward (immediate-, delayed-, vs. no-reward) was followed by a target stimulus requiring a fast and accurate response. Then, response-contingent feedback was presented indicating whether or not the participant would receive the corresponding reward. Linear mixed-effect models revealed the fastest behavioural responses and the strongest neural activity, as reflected in event-related-potentials and event-related-spectral-perturbation responses, for immediate reward, followed by delayed reward, with the slowest behavioural responses and the weakest neural activities observed in the no-reward condition. Expectations related to the cue-P3 component and the cue-delta activities predicted behavioural performance, especially in the immediate reward condition. Moreover, exploratory analyses revealed that depression moderated the relationship between target-locked neural activity and behavioural performance in the delayed reward condition, with lower neural activity being related to worse behavioural performance amongst participants scoring high on depression. These results indicate that differential value representations formed through delay discounting directly affect neural responses in reward processing and directly influence the effort invested in the current task, which is reflected by behavioural responses and is in agreement with the expected value of control theory.

Disentangling performance-monitoring signals encoded in feedback-related EEG dynamics

The feedback-related negativity (FRN) is a well-established electrophysiological correlate of feedback-processing. However, there is still an ongoing debate whether the FRN is driven by negative or positive reward prediction errors (RPE), valence of feedback, or mere surprise. Our study disentangles independent contributions of valence, surprise, and RPE on the feedback-related neuronal signal including the FRN and P3 components using the statistical power of a sample of N = 992 healthy individuals. The participants performed a modified time-estimation task, while EEG from 64 scalp electrodes was recorded. Our results show that valence coding is present during the FRN with larger amplitudes for negative feedback. The FRN is further modulated by surprise in a valence-dependent way being more positive-going for surprising positive outcomes. The P3 was strongly driven by both global and local surprise, with larger amplitudes for unexpected feedback and local deviants. Behavioral adaptations after feedback and FRN just show small associations. Results support the theory of the FRN as a representation of a signed RPE. Additionally, our data indicates that surprising positive feedback enhances the EEG response in the time window of the P3. These results corroborate previous findings linking the P3 to the evaluation of PEs in decision making and learning tasks.

Electrophysiological foundations of the human default-mode network revealed by intracranial-EEG recordings during resting-state and cognition

Investigations using noninvasive functional magnetic resonance imaging (fMRI) have provided significant insights into the unique functional organization and profound importance of the human default mode network (DMN), yet these methods are limited in their ability to resolve network dynamics across multiple timescales. Electrophysiological techniques are critical to address these challenges, yet few studies have explored the neurophysiological underpinnings of the DMN. Here we investigate the electrophysiological organization of the DMN in a common large-scale network framework consistent with prior fMRI studies. We used intracranial EEG (iEEG) recordings, and evaluated intra- and cross-network interactions during resting-state and its modulation during a cognitive task involving episodic memory formation. Our analysis revealed significantly greater intra-DMN phase iEEG synchronization in the slow-wave (< 4 Hz), while DMN interactions with other brain networks was higher in the beta (12-30 Hz) and gamma (30-80 Hz) bands. Crucially, slow-wave intra-DMN synchronization was observed in the task-free resting-state and during both verbal memory encoding and recall. Compared to resting-state, slow-wave intra-DMN phase synchronization was significantly higher during both memory encoding and recall. Slow-wave intra-DMN phase synchronization increased during successful memory retrieval, highlighting its behavioral relevance. Finally, analysis of nonlinear dynamic causal interactions revealed that the DMN is a causal outflow network during both memory encoding and recall. Our findings identify frequency specific neurophysiological signatures of the DMN which allow it to maintain stability and flexibility, intrinsically and during task-based cognition, provide novel insights into the electrophysiological foundations of the human DMN, and elucidate network mechanisms by which it supports cognition.

Automatic Sensory Predictions: A Review of Predictive Mechanisms in the Brain and Their Link to Conscious Processing

The human brain has the astonishing capacity of integrating streams of sensory information from the environment and forming predictions about future events in an automatic way. Despite being initially developed for visual processing, the bulk of predictive coding research has subsequently focused on auditory processing, with the famous mismatch negativity signal as possibly the most studied signature of a surprise or prediction error (PE) signal. Auditory PEs are present during various consciousness states. Intriguingly, their presence and characteristics have been linked with residual levels of consciousness and return of awareness. In this review we first give an overview of the neural substrates of predictive processes in the auditory modality and their relation to consciousness. Then, we focus on different states of consciousness - wakefulness, sleep, anesthesia, coma, meditation, and hypnosis - and on what mysteries predictive processing has been able to disclose about brain functioning in such states. We review studies investigating how the neural signatures of auditory predictions are modulated by states of reduced or lacking consciousness. As a future outlook, we propose the combination of electrophysiological and computational techniques that will allow investigation of which facets of sensory predictive processes are maintained when consciousness fades away.