Which way do I go? Neural activation in response to feedback and spatial processing in a virtual T-maze

Recording Neural Reward Signals in a Naturalistic Operant Task Using Mobile-EEG and Augmented Reality

The electrophysiological response to rewards recorded during laboratory tasks has been well documented, yet little is known about the neural response patterns in a more naturalistic setting. Here, we combined a mobile-EEG system with an augmented reality headset to record event-related brain potentials (ERPs) while participants engaged in a naturalistic operant task to find rewards. Twenty-five participants were asked to navigate toward a west or east goal location marked by floating orbs, and once participants reached the goal location, the orb would then signify a reward (5 cents) or no-reward (0 cents) outcome. Following the outcome, participants returned to a start location marked by floating purple rings, and once standing in the middle, a 3 s counter signaled the next trial, for a total of 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked ∼230 ms with a maximal at channel FCz (M = -0.695 μV, ±0.23) and was significantly different than zero (p < 0.01). Participants took ∼3.38 s to reach the goal location and exhibited a general lose-shift (68.3% ±3.5) response strategy and posterror slowing. Overall, these novel findings provide support for the idea that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step toward a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.

Oscillatory correlates of threat imminence during virtual navigation

The Predatory Imminence Continuum Theory proposes that defensive behaviors depend on the proximity of a threat. While the neural mechanisms underlying this proposal are well studied in animal models, it remains poorly understood in humans. To address this issue, we recorded EEG from 24 (15 female) young adults engaged in a first-person virtual reality Risk-Reward interaction task. On each trial, participants were placed in a virtual room and presented with either a threat or reward conditioned stimulus (CS) in the same room location (proximal) or different room location (distal). Behaviorally, all participants learned to avoid the threat-CS, with most using the optimal behavior to actively avoid the proximal threat-CS (88% accuracy) and passively avoid the distal threat-CS (69% accuracy). Similarly, participants learned to actively approach the distal reward-CS (82% accuracy) and to remain passive to the proximal reward-CS (72% accuracy). At an electrophysiological level, we observed a general increase in theta power (4-8 Hz) over the right posterior channel P8 across all conditions, with the proximal threat-CS evoking the largest theta response. By contrast, distal cues induced two bursts of gamma (30-60 Hz) power over midline-parietal channel Pz (200 msec post-cue) and right frontal channel Fp2 (300 msec post-cue). Interestingly, the first burst of gamma power was sensitive to the distal threat-CS and the second burst at channel Fp2 was sensitive to the distal reward-CS. Together, these findings demonstrate that oscillatory processes differentiate between the spatial proximity information during threat and reward encoding, likely optimizing the selection of the appropriate behavioral response.

Prediction-error-dependent processing of immediate and delayed positive feedback

Learning often involves trial-and-error, i.e. repeating behaviours that lead to desired outcomes, and adjusting behaviour when outcomes do not meet our expectations and thus lead to prediction errors (PEs). PEs have been shown to be reflected in the reward positivity (RewP), an event-related potential (ERP) component between 200 and 350 ms after performance feedback which is linked to striatal processing and assessed via electroencephalography (EEG). Here we show that this is also true for delayed feedback processing, for which a critical role of the hippocampus has been suggested. We found a general reduction of the RewP for delayed feedback, but the PE was similarly reflected in the RewP and the later P300 for immediate and delayed positive feedback, while no effect was found for negative feedback. Our results suggest that, despite processing differences between immediate and delayed feedback, positive PEs drive feedback processing and learning irrespective of delay.

Implications of depressive mood in OSAHS patients: insights from event-related potential

BACKGROUND

Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a chronic breathing disorder characterized by recurrent upper airway obstruction during sleep. Although previous studies have shown a link between OSAHS and depressive mood, the neurobiological mechanisms underlying mood disorders in OSAHS patients remain poorly understood. This study aims to investigate the emotion processing mechanism in OSAHS patients with depressive mood using event-related potentials (ERPs).

METHODS

Seventy-four OSAHS patients were divided into the depressive mood and non-depressive mood groups according to their Self-rating Depression Scale (SDS) scores. Patients underwent overnight polysomnography and completed various cognitive and emotional questionnaires. The patients were shown facial images displaying positive, neutral, and negative emotions and tasked to identify the emotion category, while their visual evoked potential was simultaneously recorded.

RESULTS

The two groups did not differ significantly in age, BMI, and years of education, but showed significant differences in their slow wave sleep ratio (P = 0.039), ESS (P = 0.006), MMSE (P < 0.001), and MOCA scores (P = 0.043). No significant difference was found in accuracy and response time on emotional face recognition between the two groups. N170 latency in the depressive group was significantly longer than the non-depressive group (P = 0.014 and 0.007) at the bilateral parieto-occipital lobe, while no significant difference in N170 amplitude was found. No significant difference in P300 amplitude or latency between the two groups. Furthermore, N170 amplitude at PO7 was positively correlated with the arousal index and negatively with MOCA scores (both P < 0.01).

CONCLUSION

OSAHS patients with depressive mood exhibit increased N170 latency and impaired facial emotion recognition ability. Special attention towards the depressive mood among OSAHS patients is warranted for its implications for patient care.

Recording neural reward signals in the real-world using mobile-EEG and augmented reality

The electrophysiological response to rewards recorded during laboratory-based tasks has been well documented over the past two decades, yet little is known about the neural response patterns in 'real-world' settings. To address this issue, we combined a mobile-EEG system with an augmented reality headset (which blends high definition "holograms" within the real-world) to record event-related brain potentials (ERP) while participants navigated an operant chamber to find rewards. 25 participants (age = 18-43, Male=6, Female=19) were asked to choose between two floating holograms marking a west or east goal-location in a large room, and once participants reached the goal location, the hologram would turn into a reward (5 cents) or no-reward (0 cents) cue. Following the feedback cue, participants were required to return to a hologram marking the start location, and once standing in it, a 3 second counter hologram would initiate the next trial. This sequence was repeated until participants completed 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked around 235ms post-feedback with a maximal at channel FCz (M=-2.60μV, SD=1.73μV) and was significantly different than zero (p < 0.01). At a behavioral level, participants took approximately 3.38 seconds to reach the goal-location and exhibited a general lose-shift (68.3% ± 3.5) response strategy and were slightly slower to return to the start location following negative feedback (2.43 sec) compared to positive feedback (2.38 sec), evidence of post-error slowing. Overall, these findings provide the first evidence that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step towards a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.

Inhibition of midfrontal theta with transcranial ultrasound explains greater approach versus withdrawal behavior in humans

Recent reviews highlighted low-intensity transcranial focused ultrasound (TUS) as a promising new tool for non-invasive neuromodulation in basic and applied sciences. Our preregistered double-blind within-subjects study (N = 152) utilized TUS targeting the right prefrontal cortex, which, in earlier work, was found to positively enhance self-reported global mood, decrease negative states of self-reported emotional conflict (anxiety/worrying), and modulate related midfrontal functional magnetic resonance imaging activity in affect regulation brain networks. To further explore TUS effects on objective physiological and behavioral variables, we used a virtual T-maze task that has been established in prior studies to measure motivational conflicts regarding whether participants execute approach versus withdrawal behavior (with free-choice responses via continuous joystick movements) while allowing to record related electroencephalographic data such as midfrontal theta activity (MFT). MFT, a reliable marker of conflict representation on a neuronal level, was of particular interest to us since it has repeatedly been shown to explain related behavior, with relatively low MFT typically preceding approach-like risky behavior and relatively high MFT typically preceding withdrawal-like risk aversion. Our central hypothesis is that TUS decreases MFT in T-maze conflict situations and thereby increases approach and reduces withdrawal. Results indicate that TUS led to significant MFT decreases, which significantly explained increases in approach behavior and decreases in withdrawal behavior. This study expands TUS evidence on a physiological and behavioral level with a large sample size of human subjects, suggesting the promise of further research based on this distinct TUS-MFT-behavior link to influence conflict monitoring and its behavioral consequences. Ultimately, this can serve as a foundation for future clinical work to establish TUS interventions for emotional and motivational mental health.

Electrophysiological measures of conflict and reward processing are associated with decisions to engage in physical effort

Anterior cingulate cortex (ACC), a key brain region involved in cognitive control and decision making, is suggested to mediate effort- and value-based decision making, but the specific role of ACC in this process remains debated. Here we used frontal midline theta (FMT) and the reward positivity (RewP) to examine ACC function in a value-based decision making task requiring physical effort. We investigated whether (1) FMT power is sensitive to the difficulty of the decision or to selecting effortful actions, and (2) RewP is sensitive to the subjective value of reward outcomes as a function of effort investment. On each trial, participants chose to execute a low-effort or a high-effort behavior (that required squeezing a hand-dynamometer) to obtain smaller or larger rewards, respectively, while their brainwaves were recorded. We replicated prior findings that tonic FMT increased over the course of the hour-long task, which suggests increased application of control in the face of growing fatigue. RewP amplitude also increased following execution of high-effort compared to low-effort behavior, consistent with increased valuation of reward outcomes by ACC. Although neither phasic nor tonic FMT were associated with decision difficulty or effort selection per se, an exploratory analysis revealed that the interaction of phasic FMT and expected value of choice predicted effort choice. This interaction suggests that phasic FMT increases specifically under situations of decision difficulty when participants ultimately select a high-effort choice. These results point to a unique role for ACC in motivating and persisting at effortful behavior when decision conflict is high.

The potential application of event-related potentials to enhance research on reward processes in eating disorders

OBJECTIVE

Reward-related processes have been posited as key mechanisms underlying the onset and persistence of eating disorders, prompting a growing body of research in this area. Existing studies have primarily utilized self-report, behavioral, and functional magnetic resonance imaging measures to interrogate reward among individuals with eating disorders. However, limitations inherent in each of these methods (e.g., poor temporal resolution) may obscure distinct neurocognitive reward processes, potentially contributing to underdeveloped models of reward dysfunction within eating disorders. The temporal precision of event-related potentials (ERPs), derived from electroencephalography, may thus offer a powerful complementary tool for elucidating the neurocognitive underpinnings of reward. Indeed, a considerable amount of research in other domains of psychopathology (e.g., depression, substance use disorders), as well as studies investigating food reward among non-clinical samples, highlights the utility of ERPs for probing reward processes. However, no study to date has utilized ERPs to directly examine reward functioning in eating disorders.

METHODS

In this paper, we review evidence underscoring the clinical utility of ERP measures of reward, as well as a variety of reward-related tasks that can be used to elicit specific ERP components with demonstrated relevance to reward processing. We then consider the ways in which these tasks/components may be used to help answer a variety of open questions within the eating disorders literature on reward.

RESULTS/DISCUSSION

Given the promise of ERP measures of reward to the field of eating disorders, we ultimately hope to spur and guide research in this currently neglected area.

PUBLIC SIGNIFICANCE

Abnormalities in reward functioning appear to contribute to eating disorders. Event-related potentials (ERPs) offer temporally precise measures of neurocognitive reward processing and thus may be important tools for understanding the relationship between reward and disordered eating. However, research in this area is currently lacking. This paper attempts to facilitate the use of ERPs to study reward among individuals with eating disorders by reviewing the relevant theories and methods.

Group Membership Modulates Fairness Consideration Among Deaf College Students-An Event-Related Potential Study

Group interaction is an essential way of social interaction and plays an important role in our social development. It has been found that when individuals participate in group interactions, the group identity of the interaction partner affects the mental processing and behavioral decision-making of subjects. However, little is known about how deaf college students, who are labeled distinctly different from normal hearing college students, will react when facing proposers from different groups in the ultimatum game (UG) and its time course. In this study, we recruited 29 deaf college students who played the UG in which they received extremely unfair, moderately unfair, or fair offers from either outgroup members (normal hearing college students) or ingroup members (deaf college students), while their brain potentials were recorded. The behavioral results showed that group membership did not impact the acceptance rate of deaf college students. But, event-related potential (ERP) analysis demonstrated an enhanced feedback-related negativity (FRN) elicited by ingroup members compared to outgroup members. Importantly, we found that under fairness conditions, deaf college students induced more positive P2 and P3 facing ingroup members compared to outgroup members. Our results demonstrated that group membership may modulate the performance of deaf college students in the UG and the existence of ingroup bias among deaf college students. This provides some evidence for the fairness characteristics of special populations, so that to improve the educational integration of colleges and universities.

Pain feedback interferes with reward positivity production

The reinforcement learning (RL) theory of the reward positivity (RewP) proposes that RewP indexes a reward prediction error (RPE) signal processed in the anterior cingulate cortex (ACC). According to this theory, RewP is an event-related potential (ERP) that is more positive-going for feedback stimuli that predict better-than-expected outcomes (positive feedback) than for feedback stimuli that predict worse-than-expected outcomes (negative feedback). Despite strong evidence for this hypothesis, findings have been equivocal for tasks involving painful outcomes. We hypothesized that the RewP is modulated by high-level task goals such that outcomes that are congruent with the goals elicit positive RPEs even if their immediate consequences are negative. Accordingly, changes in high-level task goals should modulate RewP amplitude for tasks that involve seeking pain compared to tasks that involve avoiding pain. We recorded the electroencephalogram from participants who were instructed to navigate a virtual T-Maze to find reward-predictive feedback in a reward condition and pain-predictive feedback in a pain condition. We expected more positive-going ERPs to reward feedback in the reward condition and more positive-going ERPs to pain feedback in the pain condition. Despite behavioral results indicating that participants complied with task instructions, contrary to our predictions, we did not find a RewP to pain feedback. We suggest that pain feedback interfered with the effect of high-level task goals on RewP amplitude, which is indicative of conflict at different levels of task hierarchy.

Scalp recorded theta activity is modulated by reward, direction, and speed during virtual navigation in freely moving humans

Theta oscillations (~ 4–12 Hz) are dynamically modulated by speed and direction in freely moving animals. However, due to the paucity of electrophysiological recordings of freely moving humans, this mechanism remains poorly understood. Here, we combined mobile-EEG with fully immersive virtual-reality to investigate theta dynamics in 22 healthy adults (aged 18–29 years old) freely navigating a T-maze to find rewards. Our results revealed three dynamic periods of theta modulation: (1) theta power increases coincided with the participants’ decision-making period; (2) theta power increased for fast and leftward trials as subjects approached the goal location; and (3) feedback onset evoked two phase-locked theta bursts over the right temporal and frontal-midline channels. These results suggest that recording scalp EEG in freely moving humans navigating a simple virtual T-maze can be utilized as a powerful translational model by which to map theta dynamics during “real-life” goal-directed behavior in both health and disease.

Hypersensitivity to negative feedback during dynamic risky-decision making in major depressive disorder: An event-related potential study

BACKGROUND

Patients with major depressive disorder (MDD) exhibit a diminished ability to think or concentrate, indecisiveness, and altered sensitivity to reward and punishment. These impairments can influence complex risk-related decision-making in dynamic environments. The neurophysiological mechanisms mediating MDD effects on decision-making behavior are not well understood.

METHODS

Patients with MDD (N=50) and healthy controls (HC, N=40) were enrolled. They completed a series of psychometric tests. Event-related potentials (ERPs) were recorded during the performance of a well-validated modified version of Balloon Analogue Risk Task (BART).

RESULTS

BART behavior data were similar across the two groups except the MDD patients showed more stability of risk aversion. Neurophysiologically, BART losses generated larger P3 amplitudes than wins, and MDD patients had larger feedback-related negativity (FRN) components than HCs in response to negative feedback (losses). Greater FRN amplitudes in response to losses correlated with higher levels of depressiveness, psychological pain, and anhedonia. A longer FRN latency in MDD patients was associated with more severe suicidal ideation.

LIMITATIONS

The findings are based on cross-sectional data, which are not powerful enough to make causal inferences.

CONCLUSION

MDD patients exhibit enhanced FRNs in the frontocentral region after receiving negative feedback in a risky decision-making task. FRN magnitude is associated with depressive symptom severity. Punishment hypersensitivity may contribute to the maintenance of depressive symptoms in MDD patients, and FRN may be a useful index of such hypersensitivity.

Mobile brain/body imaging of landmark-based navigation with high-density EEG

Coupling behavioral measures and brain imaging in naturalistic, ecological conditions is key to comprehend the neural bases of spatial navigation. This highly integrative function encompasses sensorimotor, cognitive, and executive processes that jointly mediate active exploration and spatial learning. However, most neuroimaging approaches in humans are based on static, motion‐constrained paradigms and they do not account for all these processes, in particular multisensory integration. Following the Mobile Brain/Body Imaging approach, we aimed to explore the cortical correlates of landmark‐based navigation in actively behaving young adults, solving a Y‐maze task in immersive virtual reality. EEG analysis identified a set of brain areas matching state‐of‐the‐art brain imaging literature of landmark‐based navigation. Spatial behavior in mobile conditions additionally involved sensorimotor areas related to motor execution and proprioception usually overlooked in static fMRI paradigms. Expectedly, we located a cortical source in or near the posterior cingulate, in line with the engagement of the retrosplenial complex in spatial reorientation. Consistent with its role in visuo‐spatial processing and coding, we observed an alpha‐power desynchronization while participants gathered visual information. We also hypothesized behavior‐dependent modulations of the cortical signal during navigation. Despite finding few differences between the encoding and retrieval phases of the task, we identified transient time–frequency patterns attributed, for instance, to attentional demand, as reflected in the alpha/gamma range, or memory workload in the delta/theta range. We confirmed that combining mobile high‐density EEG and biometric measures can help unravel the brain structures and the neural modulations subtending ecological landmark‐based navigation.

Feedback-Related Brain Potentials Indicate the Influence of Craving on Decision-Making in Patients with Alcohol Use Disorder: An Experimental Study

INTRODUCTION

Alcohol craving is a key symptom of alcohol use disorder (AUD) and a significant cause of poor treatment outcome and frequent relapse. Craving is supposed to impair executive functions by modulating reward salience and decision-making.

OBJECTIVE

The present study sought to clarify this modulation by scrutinizing reward feedback processing in an experimental decision-making task, which was accomplished by AUD patients in 2 conditions, in the context of induced alcohol craving and in neutral context.

METHODS

AUD inpatients (N = 40) accomplished the Balloon Analog Risk Task, while their EEG was monitored; counterbalanced across conditions, the tasks were preceded either by craving induction by means of imagery and olfactory alcohol cues, or by neutral cues. Decision choice and variability, and event-related potentials (ERPs) prior to (stimulus-preceding negativity [SPN]) and following (P2a) reward feedback upon decisions, and the outcome-related feedback-related negativity (FRN) were compared between conditions and between patients, who experienced high craving upon alcohol cues (N = 18) and those who did not (N = 22).

RESULTS

Upon craving induction (vs. neutral condition), high-craving AUD patients showed less adjustment of decision choice to preceding reward experience and more variable decisions than low-craving AUD patients, together with accentuated reward-associated ERP (SPN and P2a), while outcome-related FRN was not modified by craving.

CONCLUSIONS

Results support orientation to reward in AUD patients, particularly amplified upon experienced craving, which may interfere with (feedback-guided) decision-making even in alcohol-unrelated context. Craving-accentuated ERP indices suggest neuroadaptive changes of cognitive-motivational states upon chronic alcohol abuse. Together with altered reward-related expectancies, this has to be considered in intervention and relapse prevention.

Cardiac defensive reactions and orienting responses correspond to virtual withdrawal behavior choices in a virtual T-maze

Trait or stimulus valence related differences in heart rate change have often been investigated in psychophysiological research. However, the influence of different behavioral responses during motivational contexts on cardiac response patterns has been neglected so far. In this manuscript, HR change during movement via joystick during the negatively valent motivational condition of a virtual T-maze was investigated in two studies. Concerning the behavior, two specific avoidance response types could be identified in previous studies: a backwards withdrawal and an approach to safety. HR change and skin conductance response were hypothesized to be differentially related to these response patterns. For HR changes, the proposed difference was found. Participants facing the safety zone during their avoidance showed cardiac responses associated with the defensive response. In contrast, participants facing the negative entity of the virtual T-maze during their backwards withdrawal yielded cardiac responses associated with orienting reactions. Interestingly, these differences were found independently of the stimuli used in the two paradigms (monster vs. man). These findings lead to the conclusion that the execution of different behavioral responses during motivational conditions is an important factor in analyzing psychophysiological patterns, because they change according to the executed behavior and the linked visual and motivational properties.

Beyond the Motor Cortex: Theta Burst Stimulation of the Anterior Midcingulate Cortex

BACKGROUND

While the facilitatory and inhibitory effects of intermittent theta burst stimulation (iTBS) and continuous TBS (cTBS) protocols have been well documented on motor physiology, the action of TBS protocols on prefrontal functioning remain unclear. Here we asked whether iTBS or cTBS can differentially modulate reward-related signaling in the anterior midcingulate cortex (aMCC).

METHODS

Across 2 experiments, we used a robot-assisted transcranial magnetic stimulation system, combined with electroencephalogram recordings, to investigate the aftereffects of prefrontal iTBS and cTBS on the reward positivity, an electrophysiological signal believed to index sensitivity of the aMCC to rewards. Twenty adults (age, 18-28 years) participated in experiment 1 in which we used a scalp landmark for TBS targeting, and 14 adults (age, 18-28 years) participated in experiment 2, in which we aimed to increase TBS effectiveness by utilizing cortical thickness maps to select individualized dorsal lateral prefrontal cortex targets.

RESULTS

We demonstrated that prefrontal iTBS suppressed reward-related signaling in the aMCC (reduction in reward positivity) and caused a decrease in postfeedback switch choices. cTBS displayed no effect. We replicated and strengthened this effect on the reward positivity by targeting dorsal lateral prefrontal cortex regions displaying maximal cortical thickness.

CONCLUSIONS

While these results are inconsistent with reported TBS effects on motor cortex, the present findings offer a novel transcranial magnetic stimulation targeting approach and normative insights into the magnitude and time course of TBS-induced changes in aMCC excitability. By modulating how the aMCC links value to goal-directed behavior, this research opens an exciting new era of investigative possibilities in the understanding of aMCC function and treatment of aMCC dysfunction.

Neural mechanisms of affective instability and cognitive control in substance use

OBJECTIVE

We explored the impact of affect on cognitive control as this relates to individual differences in affective instability and substance use. Toward this end, we examined how different dimensions of affective instability interact to predict substance misuse and the effect of this on two event-related potential components, the reward positivity and the late positive potential, which are said to reflect the neural mechanisms of reward and emotion processing, respectively.

METHODS

We recorded the ongoing electroencephalogram from undergraduate students as they navigated two T-maze tasks in search of rewards. One of the tasks included neutral, pleasant, and unpleasant pictures from the International Affective Picture System. Participants also completed several questionnaires pertaining to substance use and personality.

RESULTS

A principal components analysis revealed a factor related to affective instability, which we named reactivity. This factor significantly predicted increased substance use. Individuals reporting higher levels of affective reactivity also displayed a larger reward positivity following stimuli with emotional content.

CONCLUSION

The current study uncovered a group of high-risk substance users who were characterized by greater levels of affective reactivity and context-specific increased sensitivity to rewards.

SIGNIFICANCE

These results help to elucidate the complex factors underlying substance use and may facilitate the creation of individually-tailored treatment programs for those struggling with substance use disorders.

Decision- and feedback-related brain potentials reveal risk processing mechanisms in patients with alcohol use disorder

Individuals with alcohol use disorder (AUD) are aware of the risks of alcohol abuse yet continue risky drinking. Research indicates that dysfunctional decision processes and trait variables such as impulsivity contribute to this awareness-behavior discrepancy. The present study focused on decision-related versus feedback-related processes as potential contributors to decision making in AUD by examining the relationship between decision choices and decision- and feedback-related ERP phenomena in the balloon analogue risk task (BART). N = 39 AUD and n = 35 healthy comparison participants (HC) performed the BART modified for EEG assessment. In each of 100 runs, participants made a series of choices about whether to pump up a virtual balloon, which popped pseudorandomly, ending the run. Alternatively, participants ended the run by pressing a "cash-out" button. Each pump not producing a pop provided .05 €; popping resulted in loss of the run's accumulated gain. Groups made similar choices, though AUD responded more slowly. The decision P3 200-400 ms after decision prompt (balloon) was larger in AUD than in HC, and decision P3 enhancement on high-risk trials predicted choices to pump. Feedback-related negativity (FRN) after loss (relative to cash out) feedback was smaller in AUD than in HC, suggesting indifference to negative feedback. In AUD, high impulsivity was associated with risk-modulated decision P3 but not FRN. Results indicate atypical decision- and feedback-related processes that could contribute to difficulties in engaging with daily challenges effectively.

Disentangling the effects of reward value and probability on anticipatory event-related potentials

Optimal decision-making requires humans to predict the value and probability of prospective (rewarding) outcomes. The aim of the present study was to evaluate and dissociate the cortical mechanisms activated by information on an upcoming potentially rewarded target stimulus with varying probabilities. Electro-cortical activity was recorded during a cued Go/NoGo experiment, during which cue letters signaled upcoming target letters to which participants had to respond. The probability of target letter appearance after the cue letter and the amount of money that could be won for correct and fast responses were orthogonally manipulated across four task blocks. As expected, reward availability affected a prefrontally distributed reward-related positivity, and a centrally distributed P300-like event-related potential (ERP). Moreover, a late prefrontally distributed ERP was affected by probability information. These results show that information on value and probability, respectively, activates separate mechanisms in the cortex. These results contribute to a further understanding of the neural underpinnings of normal and abnormal reward processing.

A neurophysiological measure of reward sensitivity and its association with anhedonia in psychiatrically healthy adolescents and young adults

Anhedonia (i.e., the attenuated ability to enjoy pleasurable stimuli) characterizes multiple mood disorders, but its neurophysiological underpinnings are not yet clear. Here, we measured event-related potentials in 116 adolescents and young adults engaged in an asymmetric reinforcement procedure designed to objectively characterize the anhedonic phenotype. In line with previous studies, the behavioral results showed that approximately 35% of the sample did not develop a response bias towards the more frequently rewarded stimuli (a sign of low hedonic capacity). The event-related potentials (ERPs) evoked by the reward feedback stimuli delivered during the task showed that individuals that did not develop a response bias had less cortical positivity at Fz from 224 ms to 316 ms post feedback onset compared to those that developed a response bias during the task. However, further analyses showed that this between groups difference was relatively weak, as it disappeared when we controlled for response-locked ERPs. Furthermore, the response bias observed in the asymmetric reinforcement procedure was not strongly associated with self-reported ratings of hedonic capacity. We conclude that even though the asymmetric reinforcement procedure may be used as a reward sensitivity measure in neurotypical adolescents and young adults, this task may only be able to detect clinically significant levels of anhedonia in this particular population.

Internal cost of spontaneous deception revealed by ERPs and EEG spectral perturbations

Abundant literature has studied the behavioral and neural correlates of deception, but little research has focused on the internal cost of spontaneous deception. In the present study, the event-related potential and event-related spectral perturbations techniques were used to measure the internal cost of spontaneous deception by having participants perform a sender–receiver task in which they decided whether to send deceptive messages to increase their payoff from the task. Several important main findings emerged from this study. We observed a reward positivity (RewP) after senders sent the message, suggesting an integration of reward with associated cost after response in our task. Furthermore, spontaneous deception decreased the amplitude of the RewP and power in the delta and beta bands, whereas it increased the amplitude of power in the theta band, indicating that deception carried an internal cost that devalued individuals’ rewards.

Passively learned spatial navigation cues evoke reinforcement learning reward signals

Since the suggestion by Tolman (1948) that both rodents and humans create cognitive maps during navigation, the specifics of how navigators learn about their environment has been mired in debate. One facet of this debate is whether or not the creation of cognitive maps - also known as allocentric navigation - involves reinforcement learning. Here, we demonstrate a role for reinforcement learning during allocentric navigation using event-related brain potentials (ERPs). In the present experiment, participants navigated in a virtual environment that allowed the use of three different navigation strategies (allocentric, egocentric-response, & egocentric-cue), in which their goal was to locate and remember a hidden platform. Following the navigation phase of the experiment, participants were shown "cue images" representative of the three navigation strategies. Specifically, we examined whether or not these passively learned strategy images elicited a reward positivity - an ERP component associated with reinforcement learning and the anterior cingulate cortex. We found that when allocentric navigators were shown previously learned cues predicting the goal location a reward positivity was elicited. The present findings demonstrate that allocentric navigational cues carry long-term value after navigation and lend support to the claim that reinforcement learning plays a role in the acquisition of allocentric navigation and thus the generation of cognitive maps.

Behavioral preference in sequential decision-making and its association with anxiety

In daily life, people often make consecutive decisions before the ultimate goal is reached (i.e., sequential decision-making). However, this kind of decision-making has been largely overlooked in the literature. The current study investigated whether behavioral preference would change during sequential decisions, and the neural processes underlying the potential changes. For this purpose, we revised the classic balloon analogue risk task and recorded the electroencephalograph (EEG) signals associated with each step of decision-making. Independent component analysis performed on EEG data revealed that four EEG components elicited by periodic feedback in the current step predicted participants' decisions (gamble vs. no gamble) in the next step. In order of time sequence, these components were: bilateral occipital alpha rhythm, bilateral frontal theta rhythm, middle frontal theta rhythm, and bilateral sensorimotor mu rhythm. According to the information flows between these EEG oscillations, we proposed a brain model that describes the temporal dynamics of sequential decision-making. Finally, we found that the tendency to gamble (as well as the power intensity of bilateral frontal theta rhythms) was sensitive to the individual level of trait anxiety in certain steps, which may help understand the role of emotion in decision-making.

Elevated outcome-anticipation and outcome-evaluation ERPs associated with a greater preference for larger-but-delayed rewards

Although waiting for a reward reduces or discounts its value, some people have a stronger tendency to wait for larger rewards and forgo smaller-but-immediate rewards. This ability to delay gratification is captured by individual differences in so-called intertemporal choices in which individuals are asked to choose between larger-but-delayed versus smaller-but-immediate rewards. The current study used event-related potentials (ERPs) to examine whether enhancement in two neurocognitive processes, outcome anticipation and outcome evaluation, modulate individual variability in intertemporal responses. After completing a behavioral intertemporal choice task, 34 participants performed an ERP gambling task. From this ERP task, we separately examined individual differences in outcome anticipation (stimulus-preceding negativity; SPN), early outcome valuation (feedback-related negativity; FRN), and late outcome evaluation (P3). We observed that both elevated outcome-anticipation (SPN) and late outcome-evaluation (P3) neural processes predicted a stronger preference toward larger-but-delayed rewards. No relationship was observed between intertemporal responses and early outcome evaluation (FRN), indicating that the relationship between outcome evaluation and intertemporal responses was specific to the late outcome-evaluation processing stream. Moreover, multiple regression analyses indicated that the SPN and P3 independently modulate individual differences in intertemporal responses, suggesting separate mechanisms underlie the relationship between these two neurocognitive processes and intertemporal responses. Accordingly, we identify two potential neurocognitive modulators of individual variability in intertemporal responses. We discuss the mechanisms underlying these modulators in terms of anticipation-related processing (SPN) and a saliency bias toward gain (compared to loss) outcomes (P3).

Electrophysiological Response to the Informative Value of Feedback Revealed in a Segmented Wisconsin Card Sorting Test

Feedback has two main components. One is valence that indicates the wrong or correct behavior, and the other is the informative value that refers to what we can learn from feedback. Aimed to explore the neural distinction of these two components, we provided participants with a segmented Wisconsin Card Sorting Task, in which they received either positive or negative feedback at different steps. The informative value was manipulated in terms of the order of feedback presentation. The results of event-related potentials time-locked to the feedback presentation confirmed that valence of feedback was processed in a broad epoch, especially in the time window of feedback-related negativity (FRN), reflecting detection of correct or wrong card sorting behavior. In contrast, the informative value of positive and negative feedback was mainly processed in the P300, possibly reflecting information updating or hypothesis revision. These findings provide new evidence that informative values of feedback are processed by cognitive systems that differ from those of feedback valence.

Feedback is the breakfast of champions: the significance of self-controlled formal feedback for autonomous task engagement

With the aim of examining the positive effect of the formal feedback mechanism itself beyond its informational aspect, we engaged participants in the stopwatch task and recorded their electroencephalogram throughout the experiment. This task requires a button press to stop the watch within a given time interval, the completion of which is simultaneously accompanied by adequate information on task performance. In the self-controlled feedback mode, participants could freely choose whether to request formal feedback after completing the task. In another mode, additional feedback was not provided. The 'non-choice' cue was found to elicit a more negative cue-elicited feedback negativity compared with 'choice', suggesting that the opportunity to solicit formal feedback was perceived as more desirable. In addition, a more enhanced stimulus-preceding negativity was observed prior to the task initiation cue in the self-controlled feedback condition, indicating that participants paid more sustained anticipatory attention during task preparation. Taken together, these electrophysiological results suggested an inherent reward within the formal feedback mechanism itself and the significance of self-controlled formal feedback for autonomous task engagement.

Correlation of cue-locked FRN and feedback-locked FRN in the auditory monetary incentive delay task

Reflecting the discrepancy between received and predicted outcomes, the reward prediction error (RPE) plays an important role in learning in a dynamic environment. A number of studies suggested that the feedback-related negativity (FRN) component of an event-related potential, known to be associated with unexpected outcomes, encodes RPEs. While FRN was clearly shown to be sensitive to the probability of outcomes, the effect of outcome magnitude on FRN remains to be further clarified. In studies on the neural underpinnings of reward anticipation and outcome evaluation, a monetary incentive delay (MID) task proved to be particularly useful. We investigated whether feedback-locked FRN and cue-locked dN200 responses recorded during an auditory MID task were sensitive to the probability and magnitude of outcomes. The cue-locked dN200 is associated with the update of information about the magnitude of prospective outcomes. Overall, we showed that feedback-locked FRN was modulated by both the magnitude and the probability of outcomes during an auditory version of MID task, whereas no such effect was found for cue-locked dN200. Furthermore, the cue-locked dN200, which is associated with the update of information about the magnitude of prospective outcomes, correlated with the standard feedback-locked FRN, which is associated with a negative RPE. These results further expand our knowledge on the interplay between the processing of predictive cues that forecast future outcomes and the subsequent revision of these predictions during outcome delivery.

Medial frontal cortex response to unexpected motivationally salient outcomes

The medial frontal cortex (MFC) plays a central role allocating resources to process salient information, in part by responding to prediction errors. While there is some recent debate, the feedback-related negativity (FRN) is thought to index a reward prediction error by signaling outcomes that are worse than expected. A recent study utilizing electric shock provided data inconsistent with these accounts and reported that the omission of both appetitive (money) and aversive outcomes (electric shocks) elicited a medial frontal negativity. These data suggest that the ERPs within this time range support a salience prediction error that responds to unexpected events regardless of valence. To compare the reward and salience prediction error models, we employed a design that delivered both appetitive (monetary) and aversive (noise burst) outcomes. Participants completed a passive S1/S2 prediction design where S1 predicted S2 with 80% accuracy and S2 predicted the outcome with 100% accuracy. We compared both earlier and later ERP responses over the medial frontal cortex to compare the salience and reward prediction hypotheses. Considering both time windows, the ERP response to S2 in the early time window was most positive when S2 signaled that an outcome was unexpectedly delivered and in the later time window, was most negative when an outcome was unexpectedly withheld, regardless of outcome valence. Thus, these results are more consistent with a salience prediction error rather than a reward prediction error.

Electrophysiological Correlates of Cue-Related Processing in a Gambling Task: Early Outcome Evaluation or Outcome Expectation?

Several recent studies have suggested that cues that predict outcomes elicit a feedback-related-like negativity (FRN-like negativity) reflecting initial appraisals of whether desired outcomes are probable. Some other studies, however, have found that the cues that predict outcomes elicited event-related potentials (ERPs) that reflect the expectation to outcomes (e.g., outcome expectations). Given these mixed findings, this study aimed to examine whether the brain activity elicited by predictive cues in a gambling task reflected the initial evaluations of the outcomes, the outcome expectations, or both. We used a gambling task in which the participants were told to guess which of two doors hid a reward. At the beginning of each trial, a cue was presented to inform the participants of how many doors hid a reward. We found that these predictive cues elicited a FRN-like negativity at the frontal sites within around 200–300 ms. However, this negativity did not significantly differ between the cues that fully predicted gains and the cues that fully predicted losses. Furthermore, predictive cues elicited an expectation-related slow wave, and cues that predicted gains with a 50% probability elicited a larger expectation-related slow wave than cues that fully predicted gains or losses. Our results suggest that cues predicting outcomes reflect outcome expectations rather than initial evaluations of the forthcoming outcomes.

Who Deserves My Trust? Cue-Elicited Feedback Negativity Tracks Reputation Learning in Repeated Social Interactions

Trust and trustworthiness contribute to reciprocal behavior and social relationship development. To make better decisions, people need to evaluate others’ trustworthiness. They often assess this kind of reputation by learning through repeated social interactions. The present event-related potential (ERP) study explored the reputation learning process in a repeated trust game where subjects made multi-round decisions of investment to different partners. We found that subjects gradually learned to discriminate trustworthy partners from untrustworthy ones based on how often their partners reciprocated the investment, which was indicated by their own investment decisions. Besides, electrophysiological data showed that the faces of the untrustworthy partners induced larger feedback negativity (FN) amplitude than those of the trustworthy partners, but only in the late phase of the game. The ERP results corresponded with the behavioral pattern and revealed that the learned trustworthiness differentiation was coded by the cue-elicited FN component. Consistent with previous research, our findings suggest that the anterior cue-elicited FN reflects the reputation appraisal and tracks the reputation learning process in social interactions.

Temporal dynamics of object location processing in allocentric reference frame

The spatial location of objects is processed in egocentric and allocentric reference frames, the early temporal dynamics of which have remained relatively unexplored. Previous experiments focused on ERP components related only to egocentric navigation. Thus, we designed a virtual reality experiment to see whether allocentric reference frame-related ERP modulations can also be registered. Participants collected reward objects at the end of the west and east alleys of a cross maze, and their ERPs to the feedback objects were measured. Participants made turn choices from either the south or the north alley randomly in each trial. In this way, we were able to discern place and response coding of object location. Behavioral results indicated a strong preference for using the allocentric reference frame and a preference for choosing the rewarded place in the next trial, suggesting that participants developed probabilistic expectations between places and rewards. We also found that the amplitude of the P1 was sensitive to the allocentric place of the reward object, independent of its value. We did not find evidence for egocentric response learning. These results show that early ERPs are sensitive to the location of objects during navigation in an allocentric reference frame.

Neural mechanisms of reward processing associated with depression-related personality traits

OBJECTIVE

Although impaired reward processing in depression has been well-documented, the exact nature of that deficit remains poorly understood. To investigate the link between depression and the neural mechanisms of reward processing, we examined individual differences in personality.

METHODS

We recorded the electroencephalogram from healthy college students engaged in a probabilistic reinforcement learning task. Participants also completed several personality questionnaires that assessed traits related to reward sensitivity, motivation, and depression. We examined whether behavioral measures of reward learning and event-related potential components related to outcome processing and reward anticipation-namely, the cue and feedback-related reward positivity (RewP) and the stimulus preceding negativity (SPN)-would link these personality traits to depression.

RESULTS

Participants who scored high in reward sensitivity produced a relatively larger feedback-RewP. By contrast, participants who scored high in depression learned the contingencies for infrequently rewarded cue-response combinations relatively poorly, exhibited a larger SPN, and produced a smaller feedback-RewP, especially to outcomes following cue-response combinations that were frequently rewarded.

CONCLUSION

These results point to a primary deficit in reward valuation in individuals who score high in depression, with secondary consequences that impact reward learning and anticipation.

SIGNIFICANCE

Despite recent evidence arguing for an anticipatory deficit in depression, impaired reward valuation as a primary deficit should be further examined in clinical samples.

Behavioural and neural modulation of win-stay but not lose-shift strategies as a function of outcome value in Rock, Paper, Scissors

Competitive environments in which individuals compete for mutually-exclusive outcomes require rational decision making in order to maximize gains but often result in poor quality heuristics. Reasons for the greater reliance on lose-shift relative to win-stay behaviour shown in previous studies were explored using the game of Rock, Paper, Scissors and by manipulating the value of winning and losing. Decision-making following a loss was characterized as relatively fast and relatively inflexible both in terms of the failure to modulate the magnitude of lose-shift strategy and the lack of significant neural modulation. In contrast, decision-making following a win was characterized as relatively slow and relatively flexible both in terms of a behavioural increase in the magnitude of win-stay strategy and a neural modulation of feedback-related negativity (FRN) and stimulus-preceding negativity (SPN) following outcome value modulation. The win-stay/lose-shift heuristic appears not to be a unified mechanism, with the former relying on System 2 processes and the latter relying on System 1 processes. Our ability to play rationally appears more likely when the outcome is positive and when the value of wins are low, highlighting how vulnerable we can be when trying to succeed during competition.

Reward positivity: Reward prediction error or salience prediction error?

The reward positivity is a component of the human ERP elicited by feedback stimuli in trial-and-error learning and guessing tasks. A prominent theory holds that the reward positivity reflects a reward prediction error signal that is sensitive to outcome valence, being larger for unexpected positive events relative to unexpected negative events (Holroyd & Coles, 2002). Although the theory has found substantial empirical support, most of these studies have utilized either monetary or performance feedback to test the hypothesis. However, in apparent contradiction to the theory, a recent study found that unexpected physical punishments also elicit the reward positivity (Talmi, Atkinson, & El-Deredy, 2013). The authors of this report argued that the reward positivity reflects a salience prediction error rather than a reward prediction error. To investigate this finding further, in the present study participants navigated a virtual T maze and received feedback on each trial under two conditions. In a reward condition, the feedback indicated that they would either receive a monetary reward or not and in a punishment condition the feedback indicated that they would receive a small shock or not. We found that the feedback stimuli elicited a typical reward positivity in the reward condition and an apparently delayed reward positivity in the punishment condition. Importantly, this signal was more positive to the stimuli that predicted the omission of a possible punishment relative to stimuli that predicted a forthcoming punishment, which is inconsistent with the salience hypothesis.

Reward Sensitivity of ACC as an Intermediate Phenotype between DRD4-521T and Substance Misuse

The development and expression of the midbrain dopamine system is determined in part by genetic factors that vary across individuals such that dopamine-related genes are partly responsible for addiction vulnerability. However, a complete account of how dopamine-related genes predispose individuals to drug addiction remains to be developed. Adopting an intermediate phenotype approach, we investigated whether reward-related electrophysiological activity of ACC—a cortical region said to utilize dopamine reward signals to learn the value of extended, context-specific sequences of goal-directed behaviors—mediates the influence of multiple dopamine-related functional polymorphisms over substance use. We used structural equation modeling to examine whether two related electrophysiological phenomena associated with the control and reinforcement learning functions of ACC—theta power and the reward positivity—mediated the relationship between the degree of substance misuse and genetic polymorphisms that regulate dopamine processing in frontal cortex. Substance use data were collected from 812 undergraduate students. One hundred ninety-six returned on a subsequent day to participate in an electrophysiological experiment and to provide saliva samples for DNA analysis. We found that these electrophysiological signals mediated a relationship between the DRD4-521T dopamine receptor genotype and substance misuse. Our results provide a theoretical framework that bridges the gap between genes and behavior in drug addiction and illustrate how future interventions might be individually tailored for specific genetic and neurocognitive profiles.

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study

The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson’s disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160ms both patient groups had reduced ERP amplitude compared to controls. Later at 250ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson’s. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.

Valence-separated representation of reward prediction error in feedback-related negativity and positivity

Feedback-related negativity (FRN) is an event-related brain potential (ERP) component elicited by errors and negative outcomes. Previous studies proposed that FRN reflects the activity of a general error-processing system that incorporates reward prediction error (RPE). However, other studies reported inconsistent results on this issue - namely, that FRN only reflects the valence of feedback and that the magnitude of RPE is reflected by the other ERP component called P300. The present study focused on the relationship between the FRN amplitude and RPE. ERPs were recorded during a reversal learning task performed by the participants, and a computational model was used to estimate trial-by-trial RPEs, which we correlated with the ERPs. The results indicated that FRN and P300 reflected the magnitude of RPE in negative outcomes and positive outcomes, respectively. In addition, the correlation between RPE and the P300 amplitude was stronger than the correlation between RPE and the FRN amplitude. These differences in the correlation between ERP and RPE components may explain the inconsistent results reported by previous studies; the asymmetry in the correlations might make it difficult to detect the effect of the RPE magnitude on the FRN and makes it appear that the FRN only reflects the valence of feedback.

Reward feedback stimuli elicit high-beta EEG oscillations in human dorsolateral prefrontal cortex

Reward-related feedback stimuli have been observed to elicit a burst of power in the beta frequency range over frontal areas of the human scalp. Recent discussions have suggested possible neural sources for this activity but there is a paucity of empirical evidence on the question. Here we recorded EEG from participants while they navigated a virtual T-maze to find monetary rewards. Consistent with previous studies, we found that the reward feedback stimuli elicited an increase in beta power (20-30 Hz) over a right-frontal area of the scalp. Source analysis indicated that this signal was produced in the right dorsolateral prefrontal cortex (DLPFC). These findings align with previous observations of reward-related beta oscillations in the DLPFC in non-human primates. We speculate that increased power in the beta frequency range following reward receipt reflects the activation of task-related neural assemblies that encode the stimulus-response mapping in working memory.

Anticipation in stuttering: A theoretical model of the nature of stutter prediction

The fact that some people who stutter have the ability to anticipate a stuttering moment is essential for several theories of stuttering and important for maximum effectiveness of many currently used treatment techniques. The "anticipation effect," however, is poorly understood despite much investigation into this phenomenon. In the present paper, we combine (1) behavioral evidence from the stuttering-anticipation literature, (2) speech production models, and (3) models of error detection to propose a theoretical model of anticipation. Integrating evidence from theories such as Damasio's Somatic Marker Hypothesis, Levelt's Perceptual Monitoring Theory, Guenther's The Directions Into Velocities of Articulators (DIVA) model, Postma's Covert Repair Hypothesis, among others, our central thesis is that the anticipation of a stuttering moment occurs as an outcome of the interactions between previous learning experiences (i.e., learnt associations between stuttered utterances and any self-experienced or environmental consequence) and error monitoring. Possible neurological mechanisms involved in generating conscious anticipation are also discussed, along with directions for future research.

EDUCATIONAL OBJECTIVES

The reader will be able to: (a) describe historical theories that explain how PWS may learn to anticipate stuttering; (b) state some traditional sources of evidence of anticipation in stuttering; (c) describe how PWS may be sensitive to the detection of a stuttering; (d) state some of the neural correlates that may underlie anticipation in stuttering; and (e) describe some of the possible utilities of incorporating anticipation into stuttering interventions.

Rightward-biased hemodynamic response of the parahippocampal system during virtual navigation

Phase reset of parahippocampal electrophysiological oscillations in the theta frequency range is said to contribute to item encoding and retrieval during spatial navigation. Although well-studied in non-human animals, this mechanism is poorly understood in humans. Previously we found that feedback stimuli presented in a virtual maze environment elicited a burst of theta power over right-posterior areas of the human scalp, and that the power and phase angle of these oscillations were greater following right turns compared to left turns in the maze. Here we investigated the source of this effect with functional magnetic resonance imaging. Consistent with our predictions, we found that 1) feedback encountered in the maze task activated right parahippocampal cortex (PHC), 2) right PHC was more activated by rewards following right turns compared to left turns in the maze, and 3) the rightward-biased activation was more pronounced in individuals who displayed good spatial abilities. These findings support our previous electrophysiological findings and highlight, in humans, a role for PHC theta oscillations in encoding salient information for the purpose of spatial navigation.

Impaired reward processing by anterior cingulate cortex in children with attention deficit hyperactivity disorder

Decades of research have examined the neurocognitive mechanisms of cognitive control, but the motivational factors underlying task selection and performance remain to be elucidated. We recently proposed that anterior cingulate cortex (ACC) utilizes reward prediction error signals carried by the midbrain dopamine system to learn the value of tasks according to the principles of hierarchical reinforcement learning. According to this position, disruption of the ACC-dopamine interface can disrupt the selection and execution of extended, task-related behaviors. To investigate this issue, we recorded the event-related brain potential (ERP) from children with attention deficit hyperactivity disorder (ADHD), which is strongly associated with ACC-dopamine dysfunction, and from typically developing children while they navigated a simple "virtual T-maze" to find rewards. Depending on the condition, the feedback stimuli on each trial indicated that the children earned or failed to earn either money or points. We found that the reward positivity, an ERP component proposed to index the impact of dopamine-related reward signals on ACC, was significantly larger with money feedback than with points feedback for the children with ADHD, but not for the typically developing children. These results suggest that disruption of the ACC-dopamine interface may underlie the impairments in motivational control observed in childhood ADHD.

Goal-directed EEG activity evoked by discriminative stimuli in reinforcement learning

In reinforcement learning (RL), discriminative stimuli (S) allow agents to anticipate the value of a future outcome, and the response that will produce that outcome. We examined this processing by recording EEG locked to S during RL. Incentive value of outcomes and predictive value of S were manipulated, allowing us to discriminate between outcome-related and response-related activity. S predicting the correct response differed from nonpredictive S in the P2. S paired with high-value outcomes differed from those paired with low-value outcomes in a frontocentral positivity and in the P3b. A slow negativity then distinguished between predictive and nonpredictive S. These results suggest that, first, attention prioritizes detection of informative S. Activation of mental representations of these informative S then retrieves representations of outcomes, which in turn retrieve representations of responses that previously produced those outcomes.

Developmental changes in the reward positivity: an electrophysiological trajectory of reward processing

Children and adolescents learn to regulate their behavior by utilizing feedback from the environment but exactly how this ability develops remains unclear. To investigate this question, we recorded the event-related brain potential (ERP) from children (8-13 years), adolescents (14-17 years) and young adults (18-23 years) while they navigated a "virtual maze" in pursuit of monetary rewards. The amplitude of the reward positivity, an ERP component elicited by feedback stimuli, was evaluated for each age group. A current theory suggests the reward positivity is produced by the impact of reinforcement learning signals carried by the midbrain dopamine system on anterior cingulate cortex, which utilizes the signals to learn and execute extended behaviors. We found that the three groups produced a reward positivity of comparable size despite relatively longer ERP component latencies for the children, suggesting that the reward processing system reaches maturity early in development. We propose that early development of the midbrain dopamine system facilitates the development of extended goal-directed behaviors in anterior cingulate cortex.

Navigating complex decision spaces: Problems and paradigms in sequential choice

To behave adaptively, we must learn from the consequences of our actions. Doing so is difficult when the consequences of an action follow a delay. This introduces the problem of temporal credit assignment. When feedback follows a sequence of decisions, how should the individual assign credit to the intermediate actions that comprise the sequence? Research in reinforcement learning provides 2 general solutions to this problem: model-free reinforcement learning and model-based reinforcement learning. In this review, we examine connections between stimulus-response and cognitive learning theories, habitual and goal-directed control, and model-free and model-based reinforcement learning. We then consider a range of problems related to temporal credit assignment. These include second-order conditioning and secondary reinforcers, latent learning and detour behavior, partially observable Markov decision processes, actions with distributed outcomes, and hierarchical learning. We ask whether humans and animals, when faced with these problems, behave in a manner consistent with reinforcement learning techniques. Throughout, we seek to identify neural substrates of model-free and model-based reinforcement learning. The former class of techniques is understood in terms of the neurotransmitter dopamine and its effects in the basal ganglia. The latter is understood in terms of a distributed network of regions including the prefrontal cortex, medial temporal lobes, cerebellum, and basal ganglia. Not only do reinforcement learning techniques have a natural interpretation in terms of human and animal behavior but they also provide a useful framework for understanding neural reward valuation and action selection.