Electrophysiological responses of medial prefrontal cortex to feedback at different levels of hierarchy

Task-level value affects trial-level reward processing

Despite disagreement about how anterior cingulate cortex (ACC) supports decision making, a recent hypothesis suggests that activity in this region is best understood in the context of a task or series of tasks. One important task-level variable is average reward because it is both a known driver of effortful behaviour and an important determiner of the tasks in which we choose to engage. Here we asked how average task value affects reward-related ACC activity. To answer this question, we measured a reward-related signal said to be generated in ACC called the reward positivity (RewP) while participants gambled in three tasks of differing average value. The RewP was reduced in the high-value task, an effect that was not explainable by either reward magnitude or outcome expectancy. This result suggests that ACC does not evaluate outcomes and cues in isolation, but in the context of the value of the current task.

Pain feedback interferes with reward positivity production

The reinforcement learning (RL) theory of the reward positivity (RewP) proposes that RewP indexes a reward prediction error (RPE) signal processed in the anterior cingulate cortex (ACC). According to this theory, RewP is an event-related potential (ERP) that is more positive-going for feedback stimuli that predict better-than-expected outcomes (positive feedback) than for feedback stimuli that predict worse-than-expected outcomes (negative feedback). Despite strong evidence for this hypothesis, findings have been equivocal for tasks involving painful outcomes. We hypothesized that the RewP is modulated by high-level task goals such that outcomes that are congruent with the goals elicit positive RPEs even if their immediate consequences are negative. Accordingly, changes in high-level task goals should modulate RewP amplitude for tasks that involve seeking pain compared to tasks that involve avoiding pain. We recorded the electroencephalogram from participants who were instructed to navigate a virtual T-Maze to find reward-predictive feedback in a reward condition and pain-predictive feedback in a pain condition. We expected more positive-going ERPs to reward feedback in the reward condition and more positive-going ERPs to pain feedback in the pain condition. Despite behavioral results indicating that participants complied with task instructions, contrary to our predictions, we did not find a RewP to pain feedback. We suggest that pain feedback interfered with the effect of high-level task goals on RewP amplitude, which is indicative of conflict at different levels of task hierarchy.

The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex

Despite continual debate for the past 30 years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. However, recent computational modeling work has provided insight into this question. Here we review computational models that illustrate three core principles of ACC function, related to hierarchy, world models, and cost. We also discuss four constraints on the neural implementation of these principles, related to modularity, binding, encoding, and learning and regulation. These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning (HMB-HRL), which instantiates a mechanism motivating the execution of high-level plans.