Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior

Is Internet Pornography Causing Sexual Dysfunctions? A Review with Clinical Reports

Traditional factors that once explained men's sexual difficulties appear insufficient to account for the sharp rise in erectile dysfunction, delayed ejaculation, decreased sexual satisfaction, and diminished libido during partnered sex in men under 40. This review (1) considers data from multiple domains, e.g., clinical, biological (addiction/urology), psychological (sexual conditioning), sociological; and (2) presents a series of clinical reports, all with the aim of proposing a possible direction for future research of this phenomenon. Alterations to the brain's motivational system are explored as a possible etiology underlying pornography-related sexual dysfunctions. This review also considers evidence that Internet pornography's unique properties (limitless novelty, potential for easy escalation to more extreme material, video format, etc.) may be potent enough to condition sexual arousal to aspects of Internet pornography use that do not readily transition to real-life partners, such that sex with desired partners may not register as meeting expectations and arousal declines. Clinical reports suggest that terminating Internet pornography use is sometimes sufficient to reverse negative effects, underscoring the need for extensive investigation using methodologies that have subjects remove the variable of Internet pornography use. In the interim, a simple diagnostic protocol for assessing patients with porn-induced sexual dysfunction is put forth.

Age differences in learning emerge from an insufficient representation of uncertainty in older adults

Healthy aging can lead to impairments in learning that affect many laboratory and real-life tasks. These tasks often involve the acquisition of dynamic contingencies, which requires adjusting the rate of learning to environmental statistics. For example, learning rate should increase when expectations are uncertain (uncertainty), outcomes are surprising (surprise) or contingencies are more likely to change (hazard rate). In this study, we combine computational modelling with an age-comparative behavioural study to test whether age-related learning deficits emerge from a failure to optimize learning according to the three factors mentioned above. Our results suggest that learning deficits observed in healthy older adults are driven by a diminished capacity to represent and use uncertainty to guide learning. These findings provide insight into age-related cognitive changes and demonstrate how learning deficits can emerge from a failure to accurately assess how much should be learned.

The ability to learn decreases with old age especially in a dynamically changing environment, however the precise nature of this decline is not understood. Nassar and colleagues report that older adults show a reduced ability to learn from uncertain outcomes compared to younger adults.

Performance monitoring during a minimal group manipulation

The on-going (self-)monitoring of our behaviour is inextricably intertwined with the surrounding social context. In this study, we investigated whether a minimal group paradigm assigning individuals to arbitrary group categories is powerful enough to induce changes in behavioural, psychophysiological and event-related potential correlates of performance monitoring. Following arbitrary group assignment based on ostensible task performance and a group identification task, 22 volunteers performed a flanker-task during both in-group and out-group contexts, while electroencephalography was performed. More errors were committed in the out-group compared with the in-group context. Error-related negativity amplitudes were larger for in-group compared with out-group errors. However, subsequent processing reflected in late Pe amplitudes and stimulus-driven conflict reflected in N2 amplitudes were not affected by the group context. Heart rate deceleration (during both correct and incorrect trials) tended to be more pronounced during the out-group compared with the in-group context. This surprising observation was corroborated by subjective ratings of performance satisfaction, in which participants reported higher satisfaction with their out-group performance. This study identified specific stimulus evaluation processes to be affected by a minimal group manipulation and demonstrated thereby transient top-down effects of a social context manipulation on performance monitoring.

Neural dissociations between meaningful and mere inconsistency in impression updating

Recent neuroimaging work has identified a network of regions that work in concert to update impressions of other people, particularly in response to inconsistent behavior. However, the specific functional contributions of these regions to the updating process remain unclear. Using fMRI, we tested whether increases in activity triggered by inconsistent behavior reflect changes in the stored representations of other people in response to behavioral inconsistency, or merely a response to the inconsistency itself. Participants encountered a series of individuals whose behavior either changed in an attributionally meaningful fashion or was merely inconsistent with the immediately preceding behavior. We observed that left ventrolateral prefrontal cortex (vlPFC) and left inferior frontal gyrus (IFG) were preferentially recruited in response to unexpected, immoral behavior, whereas a separate set of regions (including dorsal anterior cingulate cortex, posterior cingulate cortex and temporoparietal junction/inferior parietal lobule) was preferentially recruited in response to more mundane inconsistencies in behavior. These results shed light on the distributed systems supporting impression updating. Specifically, while many regions supporting updating may primarily respond to moment-to-moment changes in behavior, a subset of regions (e.g. vlPFC and IFG) may contribute to updating person representations in response to trait-relevant changes in behavior.

A dual but asymmetric role of the dorsal anterior cingulate cortex in response inhibition and switching from a non-salient to salient action

Response inhibition and salience detection are among the most studied psychological constructs of cognitive control. Despite a growing body of work, how inhibition and salience processing interact and engage regional brain activations remains unclear. Here, we examined this issue in a stop signal task (SST), where a prepotent response needs to be inhibited to allow an alternative, less dominant response. Sixteen adult individuals performed two versions of the SST each with 25% (SST25) and 75% (SST75) of stop trials. We posited that greater regional activations to the infrequent trial type in each condition (i.e., to stop as compared to go trials in SST25 and to go as compared to stop trials in SST75) support salience detection. Further, successful inhibition in stop trials requires attention to the stop signal to trigger motor inhibition, and the stop signal reaction time (SSRT) has been used to index the efficiency of motor response inhibition. Therefore, greater regional activations to stop as compared to go success trials in association with the stop signal reaction time (SSRT) serve to expedite response inhibition. In support of an interactive role, the dorsal anterior cingulate cortex (dACC) increases activation to salience detection in both SST25 and SST75, but only mediates response inhibition in SST75. Thus, infrequency response in the dACC supports motor inhibition only when stopping has become a routine. In contrast, although the evidence is less robust, the pre-supplementary motor area (pre-SMA) increases activity to the infrequent stimulus and supports inhibition in both SST25 and SST75. These findings clarify a unique role of the dACC and add to the literature that distinguishes dACC and pre-SMA functions in cognitive control.

Neural circuitry involved in quitting after repeated failures: role of the cingulate and temporal parietal junction

The more times people fail the more likely they are to give up, however little is known about the neural mechanisms underlying this impact of repeated failure on decision making. Here we have used a visual shape discrimination task with computer-controlled feedback combined with functional magnetic resonance imaging (fMRI) to investigate the neural circuits involved. The behavioral task confirmed that the more times subjects experienced failure the more likely they were to give up, with three successive failures being the key threshold and the majority of subjects reaching the point where they decided to quit and try a new stimulus set after three or four failures. The fMRI analysis revealed activity changes in frontal, parietal, temporal, limbic and striatal regions, especially anterior cingulate cortex (ACC), posterior cingulate cortex (PCC) and temporal parietal junction (TPJ) associated with the number of previous failures experienced. Furthermore, their parameter estimates were predictive of subjects’ quitting rate. Thus, subjects reach the point where they decide to quit after three/four failures and this is associated with differential changes in brain regions involved in error monitoring and reward which regulate both failure detection and changes in decision-making strategy.

Dorsal Anterior Cingulate Cortex: A Bottom-Up View

The dorsal anterior cingulate cortex (dACC) has attracted great interest from neuroscientists because it is associated with so many important cognitive functions. Despite, or perhaps because of, its rich functional repertoire, we lack a single comprehensive view of its function. Most research has approached this puzzle from the top down, using aggregate measures such as neuroimaging. We provide a view from the bottom up, with a focus on single-unit responses and anatomy. We summarize the strengths and weaknesses of the three major approaches to characterizing the dACC: as a monitor, as a controller, and as an economic structure. We argue that neurons in the dACC are specialized for representing contexts, or task-state variables relevant for behavior, and strategies, or aspects of future plans. We propose that dACC neurons link contexts with strategies by integrating diverse task-relevant information to create a rich representation of task space and exert high-level and abstract control over decision and action.

Rule learning enhances structural plasticity of long-range axons in frontal cortex

Rules encompass cue-action-outcome associations used to guide decisions and strategies in a specific context. Subregions of the frontal cortex including the orbitofrontal cortex (OFC) and dorsomedial prefrontal cortex (dmPFC) are implicated in rule learning, although changes in structural connectivity underlying rule learning are poorly understood. We imaged OFC axonal projections to dmPFC during training in a multiple choice foraging task and used a reinforcement learning model to quantify explore–exploit strategy use and prediction error magnitude. Here we show that rule training, but not experience of reward alone, enhances OFC bouton plasticity. Baseline bouton density and gains during training correlate with rule exploitation, while bouton loss correlates with exploration and scales with the magnitude of experienced prediction errors. We conclude that rule learning sculpts frontal cortex interconnectivity and adjusts a thermostat for the explore–exploit balance.

The orbitofrontal cortex is associated with foraging behaviour yet the structural changes underlying such rule-based learning remain unclear. Here, the authors imaged OFC axons throughout a digging-based odour discrimination task and found correlations between the rate of bouton turnover and the behavioural strategies of individual mice.

The effects of methylphenidate on cerebral responses to conflict anticipation and unsigned prediction error in a stop-signal task

To adapt flexibly to a rapidly changing environment, humans must anticipate conflict and respond to surprising, unexpected events. To this end, the brain estimates upcoming conflict on the basis of prior experience and computes unsigned prediction error (UPE). Although much work implicates catecholamines in cognitive control, little is known about how pharmacological manipulation of catecholamines affects the neural processes underlying conflict anticipation and UPE computation. We addressed this issue by imaging 24 healthy young adults who received a 45 mg oral dose of methylphenidate (MPH) and 62 matched controls who did not receive MPH prior to performing the stop-signal task. We used a Bayesian Dynamic Belief Model to make trial-by-trial estimates of conflict and UPE during task performance. Replicating previous research, the control group showed anticipation-related activation in the presupplementary motor area and deactivation in the ventromedial prefrontal cortex and parahippocampal gyrus, as well as UPE-related activations in the dorsal anterior cingulate, insula, and inferior parietal lobule. In group comparison, MPH increased anticipation activity in the bilateral caudate head and decreased UPE activity in each of the aforementioned regions. These findings highlight distinct effects of catecholamines on the neural mechanisms underlying conflict anticipation and UPE, signals critical to learning and adaptive behavior.

Transcranial Stimulation over Frontopolar Cortex Elucidates the Choice Attributes and Neural Mechanisms Used to Resolve Exploration-Exploitation Trade-Offs

Optimal behavior requires striking a balance between exploiting tried-and-true options or exploring new possibilities. Neuroimaging studies have identified different brain regions in humans where neural activity is correlated with exploratory or exploitative behavior, but it is unclear whether this activity directly implements these choices or simply reflects a byproduct of the behavior. Moreover, it remains unknown whether arbitrating between exploration and exploitation can be influenced with exogenous methods, such as brain stimulation. In our study, we addressed these questions by selectively upregulating and downregulating neuronal excitability with anodal or cathodal transcranial direct current stimulation over right frontopolar cortex during a reward-learning task. This caused participants to make slower, more exploratory or faster, more exploitative decisions, respectively. Bayesian computational modeling revealed that stimulation affected how much participants took both expected and obtained rewards into account when choosing to exploit or explore: Cathodal stimulation resulted in an increased focus on the option expected to yield the highest payout, whereas anodal stimulation led to choices that were less influenced by anticipated payoff magnitudes and were more driven by recent negative reward prediction errors. These findings suggest that exploration is triggered by a neural mechanism that is sensitive to prior less-than-expected choice outcomes and thus pushes people to seek out alternative courses of action. Together, our findings establish a parsimonious neurobiological mechanism that causes exploration and exploitation, and they provide new insights into the choice features used by this mechanism to direct decision-making.

Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system

Recent experiments have shown that animals and humans have a remarkable ability to adapt their learning rate according to the volatility of the environment. Yet the neural mechanism responsible for such adaptive learning has remained unclear. To fill this gap, we investigated a biophysically inspired, metaplastic synaptic model within the context of a well-studied decision-making network, in which synapses can change their rate of plasticity in addition to their efficacy according to a reward-based learning rule. We found that our model, which assumes that synaptic plasticity is guided by a novel surprise detection system, captures a wide range of key experimental findings and performs as well as a Bayes optimal model, with remarkably little parameter tuning. Our results further demonstrate the computational power of synaptic plasticity, and provide insights into the circuit-level computation which underlies adaptive decision-making.

Interareal Spike-Train Correlations of Anterior Cingulate and Dorsal Prefrontal Cortex during Attention Shifts

The anterior cingulate cortex (ACC) and prefrontal cortex (PFC) are believed to coactivate during goal-directed behavior to identify, select, and monitor relevant sensory information. Here, we tested whether coactivation of neurons across macaque ACC and PFC would be evident at the level of pairwise neuronal correlations during stimulus selection in a spatial attention task. We found that firing correlations emerged shortly after an attention cue, were evident for 50-200 ms time windows, were strongest for neuron pairs in area 24 (ACC) and areas 8 and 9 (dorsal PFC), and were independent of overall firing rate modulations. For a subset of cell pairs from ACC and dorsal PFC, the observed functional spike-train connectivity carried information about the direction of the attention shift. Reliable firing correlations were evident across area boundaries for neurons with broad spike waveforms (putative excitatory neurons) as well as for pairs of putative excitatory neurons and neurons with narrow spike waveforms (putative interneurons). These findings reveal that stimulus selection is accompanied by slow time scale firing correlations across those ACC/PFC subfields implicated to control and monitor attention. This functional coupling was informative about which stimulus was selected and thus indexed possibly the exchange of task-relevant information. We speculate that interareal, transient firing correlations reflect the transient coordination of larger, reciprocally interacting brain networks at a characteristic 50-200 ms time scale. Significance statement: Our manuscript identifies interareal spike-train correlations between primate anterior cingulate and dorsal prefrontal cortex during a period where attentional stimulus selection is likely controlled by these very same circuits. Interareal correlations emerged during the covert attention shift to one of two peripheral stimuli, proceeded on a slow 50-200 ms time scale, and occurred between putative pyramidal and putative interneurons. Spike-train correlations emerged particularly for cell pairs tuned to similar contralateral target locations, thus indexing the interareal coordination of attention-relevant information. These findings characterize a possible way by which prefrontal and anterior cingulate cortex circuits implement their control functions through coordinated firing when macaque monkeys select and monitor relevant stimuli for goal-directed behaviors.

Relationships between Pupil Diameter and Neuronal Activity in the Locus Coeruleus, Colliculi, and Cingulate Cortex

Changes in pupil diameter that reflect effort and other cognitive factors are often interpreted in terms of the activity of norepinephrine-containing neurons in the brainstem nucleus locus coeruleus (LC), but there is little direct evidence for such a relationship. Here, we show that LC activation reliably anticipates changes in pupil diameter that either fluctuate naturally or are driven by external events during near fixation, as in many psychophysical tasks. This relationship occurs on as fine a temporal and spatial scale as single spikes from single units. However, this relationship is not specific to the LC. Similar relationships, albeit with delayed timing and different reliabilities across sites, are evident in the inferior and superior colliculus and anterior and posterior cingulate cortex. Because these regions are interconnected with the LC, the results suggest that non-luminance-mediated changes in pupil diameter might reflect LC-mediated coordination of neuronal activity throughout some parts of the brain.

The Role of the Parietal Cortex in the Representation of Task-Reward Associations

Rewards obtained from specific behaviors can and do change across time. To adapt to such conditions, humans need to represent and update associations between behaviors and their outcomes. Much previous work focused on how rewards affect the processing of specific tasks. However, abstract associations between multiple potential behaviors and multiple rewards are an important basis for adaptation as well. In this experiment, we directly investigated which brain areas represent associations between multiple tasks and rewards, using time-resolved multivariate pattern analysis of functional magnetic resonance imaging data. Importantly, we were able to dissociate neural signals reflecting task-reward associations from those related to task preparation and reward expectation processes, variables that were often correlated in previous research. We hypothesized that brain regions involved in processing tasks and/or rewards will be involved in processing associations between them. Candidate areas included the dorsal anterior cingulate cortex, which is involved in associating simple actions and rewards, and the parietal cortex, which has been shown to represent task rules and action values. Our results indicate that local spatial activation patterns in the inferior parietal cortex indeed represent task-reward associations. Interestingly, the parietal cortex flexibly changes its content of representation within trials. It first represents task-reward associations, later switching to process tasks and rewards directly. These findings highlight the importance of the inferior parietal cortex in associating behaviors with their outcomes and further show that it can flexibly reconfigure its function within single trials. Significance statement: Rewards obtained from specific behaviors rarely remain constant over time. To adapt to changing conditions, humans need to continuously update and represent the current association between behavior and its outcomes. However, little is known about the neural representation of behavior-outcome associations. Here, we used multivariate pattern analysis of functional magnetic resonance imaging data to investigate the neural correlates of such associations. Our results demonstrate that the parietal cortex plays a central role in representing associations between multiple behaviors and their outcomes. They further highlight the flexibility of the parietal cortex, because we find it to adapt its function to changing task demands within trials on relatively short timescales.

Neuronal selectivity for spatial positions of offers and choices in five reward regions

When we evaluate an option, how is the neural representation of its value linked to information that identifies it, such as its position in space? We hypothesized that value information and identity cues are not bound together at a particular point but are represented together at the single unit level throughout the entirety of the choice process. We examined neuronal responses in two-option gambling tasks with lateralized and asynchronous presentation of offers in five reward regions: orbitofrontal cortex (OFC, area 13), ventromedial prefrontal cortex (vmPFC, area 14), ventral striatum (VS), dorsal anterior cingulate cortex (dACC), and subgenual anterior cingulate cortex (sgACC, area 25). Neuronal responses in all areas are sensitive to the positions of both offers and of choices. This selectivity is strongest in reward-sensitive neurons, indicating that it is not a property of a specialized subpopulation of cells. We did not find consistent contralateral or any other organization to these responses, indicating that they may be difficult to detect with aggregate measures like neuroimaging or studies of lesion effects. These results suggest that value coding is wed to factors that identify the object throughout the reward system and suggest a possible solution to the binding problem raised by abstract value encoding schemes.

The Roles of Dopamine and Hypocretin in Reward: A Electroencephalographic Study

The proper functioning of the mesolimbic reward system is largely dependent on the neurotransmitter dopamine. Recent evidence suggests that the hypocretin system has significant projections to this reward system. We examined the distinct effects of reduced dopamine or reduced hypocretin levels on reward activity in patients with Parkinson’s disease, dopamine deficient, as well as patients with narcolepsy-cataplexy, hypocretin depleted, and healthy controls. Participants performed a simple game-like task while high-density electroencephalography was recorded. Topography and timing of event-related potentials for both reward cue, and reward feedback was examined across the entire dataset. While response to reward cue was similar in all groups, two distinct time points were found to distinguish patients and controls for reward feedback. Around 160ms both patient groups had reduced ERP amplitude compared to controls. Later at 250ms, both patient groups also showed a clear event-related potential (ERP), which was absent in controls. The initial differences show that both patient groups show a similar, blunted response to reward delivery. The second potential corresponds to the classic feedback-related negativity (FRN) potential which relies on dopamine activity and reflects reward prediction-error signaling. In particular the mismatch between predicted reward and reward subsequently received was significantly higher in PD compared to NC, independent of reward magnitude and valence. The intermediate FRN response in NC highlights the contribution of hypocretin in reward processing, yet also shows that this is not as detrimental to the reward system as in Parkinson’s. Furthermore, the inability to generate accurate predictions in NC may explain why hypocretin deficiency mediates cataplexy triggered by both positive and negative emotions.

Randomised prior feedback modulates neural signals of outcome monitoring

Substantial evidence indicates that decision outcomes are typically evaluated relative to expectations learned from relatively long sequences of previous outcomes. This mechanism is thought to play a key role in general learning and adaptation processes but relatively little is known about the determinants of outcome evaluation when the capacity to learn from series of prior events is difficult or impossible. To investigate this issue, we examined how the feedback-related negativity (FRN) is modulated by information briefly presented before outcome evaluation. The FRN is a brain potential time-locked to the delivery of decision feedback and it is widely thought to be sensitive to prior expectations. We conducted a multi-trial gambling task in which outcomes at each trial were fully randomised to minimise the capacity to learn from long sequences of prior outcomes. Event-related potentials for outcomes (Win/Loss) in the current trial (Outcome_t) were separated according to the type of outcomes that occurred in the preceding two trials (Outcome_t-1 and Outcome_t-2). We found that FRN voltage was more positive during the processing of win feedback when it was preceded by wins at Outcome_t-1 compared to win feedback preceded by losses at Outcome_t-1. However, no influence of preceding outcomes was found on FRN activity relative to the processing of loss feedback. We also found no effects of Outcome_t-2 on FRN amplitude relative to current feedback. Additional analyses indicated that this effect was largest for trials in which participants selected a decision different to the gamble chosen in the previous trial. These findings are inconsistent with models that solely relate the FRN to prediction error computation. Instead, our results suggest that if stable predictions about future events are weak or non-existent, then outcome processing can be determined by affective systems. More specifically, our results indicate that the FRN is likely to reflect the activity of positive affective systems in these contexts. Importantly, our findings indicate that a multifactorial explanation of the nature of the FRN is necessary and such an account must incorporate affective and motivational factors in outcome processing.

•

FRN is sensitive to prior events but does this persist when learning is impossible?

•

Four FRN theories were examined in a task with fully randomised feedback.

•

Previous trial outcome modulated FRN during positive but not negative outcomes.

•

Results consistent with a positive affective modulation interpretation of the FRN.

•

FRN is determined by factors beyond prediction error (e.g. affective/motivational).

Neural correlates of rules and conflict in medial prefrontal cortex during decision and feedback epochs

The ability to properly adjust behavioral responses to cues in a changing environment is crucial for survival. Activity in the medial Prefrontal Cortex (mPFC) is thought to both represent rules to guide behavior as well as detect and resolve conflicts between rules in changing contingencies. However, while lesion and pharmacological studies have supported a crucial role for mPFC in this type of set-shifting, an understanding of how mPFC represents current rules or detects and resolves conflict between different rules is unclear. Here, we directly address the role of rat mPFC in shifting rule based behavioral strategies using a novel behavioral task designed to tease apart neural signatures of rules, conflict and direction. We demonstrate that activity of single neurons in rat mPFC represent distinct rules. Further, we show increased firing on high conflict trials in a separate population of mPFC neurons. Reduced firing in both populations of neurons was associated with poor performance. Moreover, activity in both populations increased and decreased firing during the outcome epoch when reward was and was not delivered on correct and incorrect trials, respectively. In addition, outcome firing was modulated by the current rule and the degree of conflict associated with the previous decision. These results promote a greater understanding of the role that mPFC plays in switching between rules, signaling both rule and conflict to promote improved behavioral performance.

Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision

We frequently need to commit to a choice to achieve our goals; however, the neural processes that keep us motivated in pursuit of delayed goals remain obscure. We examined ensemble responses of neurons in macaque dorsal anterior cingulate cortex (dACC), an area previously implicated in self-control and persistence, in a task that requires commitment to a choice to obtain a reward. After reward receipt, dACC neurons signaled reward amount with characteristic ensemble firing rate patterns; during the delay in anticipation of the reward, ensemble activity smoothly and gradually came to resemble the postreward pattern. On the subset of risky trials, in which a reward was anticipated with 50% certainty, ramping ensemble activity evolved to the pattern associated with the anticipated reward (and not with the anticipated loss) and then, on loss trials, took on an inverted form anticorrelated with the form associated with a win. These findings enrich our knowledge of reward processing in dACC and may have broader implications for our understanding of persistence and self-control.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

The correlation between biochemical abnormalities in frontal white matter, hippocampus and serum thyroid hormone levels in first-episode patients with major depressive disorder

BACKGROUND

Previous neuroimaging studies found evidence of potential brain biochemical abnormalities in patients with major depressive disorder (MDD). Abnormal serum thyroid hormone levels were also found in MDD patients, which may correlated with the abnormal biochemical metabolism of brain. However, they rarely excluded the compounding effects of medication, and brain degeneration. This study sought to investigate the relationship between the biochemical metabolism and the serum thyroid hormone levels in first-episode, treatment-naive, non-late-life patients with MDD.

METHODS

26 first-episode, treatment-naive, non-late-life patients with MDD and 13 healthy controls were enrolled in this study. Participants underwent two-dimensinal multivoxel proton magnetic resonance spectroscopy ((1)H MRS) [repetition time (TR)=1000ms; echo-time (TE)=144ms] at 1.5T to obtain bilateral metabolite levels from the white matter in prefrontal (WMP) lobe, anterior cingulate cortex (ACC), and hippocampus. The ratios of N-acetylaspartate (NAA)/creatine (Cr) and choline containg compounds (Cho)/creatine (Cr) were calculated. Morning serum free triiodothyronine (FT3), free thyroxin (FT4), total triiodothyronine (T3), total thyroxin (T4), and thyroid-stimulating hormone (TSH) were measured before antidepressant treatment.

RESULTS

On the comparison of brain biochemical changes, MDD patients had a significantly lower NAA/Cr ratio in the left WMP, and lower NAA/Cr and Cho/Cr ratios in the right WMP when compared to the controls. There were no significant differences in the metabolite ratios in the bilateral ACC, and hippocampus. On the comparison of serum thyroid hormone levels, MDD patients had a significantly decreased T3 and TSH levels. On the comparison of correlation of brain biochemical changes and serum thyroid hormone levels in patients with MDD, the NAA/Cr ratio in the right WMP was positively correlated with the level of TSH.

CONCLUSION

These findings suggest that biochemical abnormalities and thyroid dysfunction may emerge early in the course of MDD. Dysfunction of neuronal function in the WMP may correlate with the abnormal TSH in patients with MDD, which may be related to the neuropathology of depression.

Neuronal Reward and Decision Signals: From Theories to Data

Rewards are crucial objects that induce learning, approach behavior, choices, and emotions. Whereas emotions are difficult to investigate in animals, the learning function is mediated by neuronal reward prediction error signals which implement basic constructs of reinforcement learning theory. These signals are found in dopamine neurons, which emit a global reward signal to striatum and frontal cortex, and in specific neurons in striatum, amygdala, and frontal cortex projecting to select neuronal populations. The approach and choice functions involve subjective value, which is objectively assessed by behavioral choices eliciting internal, subjective reward preferences. Utility is the formal mathematical characterization of subjective value and a prime decision variable in economic choice theory. It is coded as utility prediction error by phasic dopamine responses. Utility can incorporate various influences, including risk, delay, effort, and social interaction. Appropriate for formal decision mechanisms, rewards are coded as object value, action value, difference value, and chosen value by specific neurons. Although all reward, reinforcement, and decision variables are theoretical constructs, their neuronal signals constitute measurable physical implementations and as such confirm the validity of these concepts. The neuronal reward signals provide guidance for behavior while constraining the free will to act.

Signatures of Value Comparison in Ventral Striatum Neurons

The ventral striatum (VS), like its cortical afferents, is closely associated with processing of rewards, but the relative contributions of striatal and cortical reward systems remains unclear. Most theories posit distinct roles for these structures, despite their similarities. We compared responses of VS neurons to those of ventromedial prefrontal cortex (vmPFC) Area 14 neurons, recorded in a risky choice task. Five major response patterns observed in vmPFC were also observed in VS: (1) offer value encoding, (2) value difference encoding, (3) preferential encoding of chosen relative to unchosen value, (4) a correlation between residual variance in responses and choices, and (5) prominent encoding of outcomes. We did observe some differences as well; in particular, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, the close match between vmPFC and VS suggests that cortex and its striatal targets make overlapping contributions to economic choice.

A study of single neurons in the ventral striatum reveals signatures of value comparison and selection during a risky choice task, suggesting that the cortex and its striatal targets make overlapping contributions to the choice process. Read the accompanying Primer.

The neural calculations underlying reward-based choice are closely associated with a network of brain areas including the ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC). Most theories ascribe distinct roles to these two structures during choice, but these differences have yet to be confirmed at the level of single neurons. We compared responses of VS neurons to those of vmPFC neurons recorded in rhesus macaques choosing between potential gambles for water rewards. We found widespread similarities in the way that VS and vmPFC neurons fire during the choice process. Neurons in both areas encoded the value of the offered gamble, the difference in value between offered gambles, and the gamble outcome. Additionally, both areas showed stronger coding for the chosen gamble than for the unchosen one and predicted choice even when we controlled for offer value. Interestingly, preferential encoding of the chosen option was stronger and started earlier in VS than in vmPFC. Nonetheless, similarities between vmPFC and VS suggest that cortex and its striatal targets make overlapping contributions to reward-based choice.

Neuronal activity in primate dorsal anterior cingulate cortex signals task conflict and predicts adjustments in pupil-linked arousal

Whether driving a car, shopping for food, or paying attention in a classroom of boisterous teenagers, it's often hard to maintain focus on goals in the face of distraction. Brain imaging studies in humans implicate the dorsal anterior cingulate cortex (dACC) in regulating the conflict between goals and distractors. Here we show that single dACC neurons signal conflict between task goals and distractors in the rhesus macaque, particularly for biologically relevant social stimuli. For some neurons, task conflict signals predicted subsequent changes in pupil size-a peripheral index of arousal linked to noradrenergic tone-associated with reduced distractor interference. dACC neurons also responded to errors, and these signals predicted adjustments in pupil size. These findings provide the first neurophysiological endorsement of the hypothesis that dACC regulates conflict, in part, via modulation of pupil-linked processes such as arousal.

Functional Connectivity under Anticipation of Shock: Correlates of Trait Anxious Affect versus Induced Anxiety

Sustained anxiety about potential future negative events is an important feature of anxiety disorders. In this study, we used a novel anticipation of shock paradigm to investigate individual differences in functional connectivity during prolonged threat of shock. We examined the correlates of between-participant differences in trait anxious affect and induced anxiety, where the latter reflects changes in self-reported anxiety resulting from the shock manipulation. Dissociable effects of trait anxious affect and induced anxiety were observed. Participants with high scores on a latent dimension of anxious affect showed less increase in ventromedial pFC-amygdala connectivity between periods of safety and shock anticipation. Meanwhile, lower levels of induced anxiety were linked to greater augmentation of dorsolateral pFC-anterior insula connectivity during shock anticipation. These findings suggest that ventromedial pFC-amygdala and dorsolateral pFC-insula networks might both contribute to regulation of sustained fear responses, with their recruitment varying independently across participants. The former might reflect an evolutionarily old mechanism for reducing fear or anxiety, whereas the latter might reflect a complementary mechanism by which cognitive control can be implemented to diminish fear responses generated due to anticipation of aversive stimuli or events. These two circuits might provide complementary, alternate targets for exploration in future pharmacological and cognitive intervention studies.

Vicarious reinforcement learning signals when instructing others

Reinforcement learning (RL) theory posits that learning is driven by discrepancies between the predicted and actual outcomes of actions (prediction errors [PEs]). In social environments, learning is often guided by similar RL mechanisms. For example, teachers monitor the actions of students and provide feedback to them. This feedback evokes PEs in students that guide their learning. We report the first study that investigates the neural mechanisms that underpin RL signals in the brain of a teacher. Neurons in the anterior cingulate cortex (ACC) signal PEs when learning from the outcomes of one's own actions but also signal information when outcomes are received by others. Does a teacher's ACC signal PEs when monitoring a student's learning? Using fMRI, we studied brain activity in human subjects (teachers) as they taught a confederate (student) action-outcome associations by providing positive or negative feedback. We examined activity time-locked to the students' responses, when teachers infer student predictions and know actual outcomes. We fitted a RL-based computational model to the behavior of the student to characterize their learning, and examined whether a teacher's ACC signals when a student's predictions are wrong. In line with our hypothesis, activity in the teacher's ACC covaried with the PE values in the model. Additionally, activity in the teacher's insula and ventromedial prefrontal cortex covaried with the predicted value according to the student. Our findings highlight that the ACC signals PEs vicariously for others' erroneous predictions, when monitoring and instructing their learning. These results suggest that RL mechanisms, processed vicariously, may underpin and facilitate teaching behaviors.

A critical role for the anterior thalamus in directing attention to task-relevant stimuli

The prefrontal cortex mediates adaption to changing environmental contingencies. The anterior thalamic nuclei, which are closely interconnected with the prefrontal cortex, are important for rodent spatial memory, but their potential role in executive function has received scant attention. The current study examined whether the anterior thalamic nuclei are involved in attentional processes akin to those of prefrontal regions. Remarkably, the results repeatedly revealed attentional properties opposite to those of the prefrontal cortex. Two separate cohorts of rats with anterior thalamic lesions were tested on an attentional set-shifting paradigm that measures not only the ability of stimuli dimensions that reliably predict reinforcement to gain attention ("intradimensional shift"), but also their ability to shift attention to another stimulus dimension when contingencies change ("extradimensional shift"). In stark contrast to the effects of prefrontal damage, anterior thalamic lesions impaired intradimensional shifts but facilitated extradimensional shifts. Anterior thalamic lesion animals were slower to acquire discriminations based on the currently relevant stimulus dimension but acquired discriminations involving previously irrelevant stimulus dimensions more rapidly than controls. Subsequent tests revealed that the critical determinant of whether anterior thalamic lesions facilitate extradimensional shifts is the degree to which the stimulus dimension has been established as an unreliable predictor of reinforcement over preceding trials. This pattern of performance reveals that the anterior thalamic nuclei are vital for attending to those stimuli that are the best predictors of reward. In their absence, unreliable predictors of reward usurp attentional control.

Learning-induced plasticity in medial prefrontal cortex predicts preference malleability

Learning induces plasticity in neuronal networks. As neuronal populations contribute to multiple representations, we reasoned plasticity in one representation might influence others. We used human fMRI repetition suppression to show that plasticity induced by learning another individual’s values impacts upon a value representation for oneself in medial prefrontal cortex (mPFC), a plasticity also evident behaviorally in a preference shift. We show this plasticity is driven by a striatal “prediction error,” signaling the discrepancy between the other’s choice and a subject’s own preferences. Thus, our data highlight that mPFC encodes agent-independent representations of subjective value, such that prediction errors simultaneously update multiple agents’ value representations. As the resulting change in representational similarity predicts interindividual differences in the malleability of subjective preferences, our findings shed mechanistic light on complex human processes such as the powerful influence of social interaction on beliefs and preferences.

•

Learning the values of another causes plasticity in a mPFC value representation

•

This plasticity predicts how much subjects’ own preferences change

•

Plasticity is explained by a striatal surprise signal

•

Value coding in mPFC occurs independently of the agent for whom a decision is made

Learning to minimize efforts versus maximizing rewards: computational principles and neural correlates

The mechanisms of reward maximization have been extensively studied at both the computational and neural levels. By contrast, little is known about how the brain learns to choose the options that minimize action cost. In principle, the brain could have evolved a general mechanism that applies the same learning rule to the different dimensions of choice options. To test this hypothesis, we scanned healthy human volunteers while they performed a probabilistic instrumental learning task that varied in both the physical effort and the monetary outcome associated with choice options. Behavioral data showed that the same computational rule, using prediction errors to update expectations, could account for both reward maximization and effort minimization. However, these learning-related variables were encoded in partially dissociable brain areas. In line with previous findings, the ventromedial prefrontal cortex was found to positively represent expected and actual rewards, regardless of effort. A separate network, encompassing the anterior insula, the dorsal anterior cingulate, and the posterior parietal cortex, correlated positively with expected and actual efforts. These findings suggest that the same computational rule is applied by distinct brain systems, depending on the choice dimension-cost or benefit-that has to be learned.

Feedback-Related Negativity (FRN) and P300 Are Sensitive to Temporal-Order Violation in Transitive Action Representation

Coherent representation of action sequences implies that the logical temporal order of each action can be correctly represented. Violation of this logical order may induce a sort of expectancies disruption of the temporal structure. Thus the present study explored the event-related potential (ERP) effect related to the cortical response to this violation. Action sequence composed by four frames with final congruous or incongruous endings was submitted to 28 subjects. Two distinct ERP effects, feedback-related negativity (FRN), and P300, were found in response to incongruous endings, with also significant increased RTs. The functional significance of these two ERP deflections was related respectively to the perception of an erroneous action outcome as the ending of an illogical sequence (FRN) and to the necessity to updating the relationship action-context by changing the cognitive model which supports the cognitive expectancies (P300). The significant correlation between the RTs and the ERP measures, especially in case of FRN effect, supported this interpretation. Indeed increased cognitive costs are supposed in case of expectancies violations which require further processes of reanalysis of the coherence between the action and the background (the temporal background) where the action was produced. Two different cortical localizations were found for FRN and P300, respectively a more fronto-central (dorsolateral prefrontal cortex) and posterior (superior temporal gyrus) site. The significance of these results for the temporal order effect for action comprehension was discussed.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Context-sensitivity of the feedback-related negativity for zero-value feedback outcomes

The present study investigated whether the same visual stimulus indicating zero-value feedback (€0) elicits feedback-related negativity (FRN) variation, depending on whether the outcomes correspond with expectations or not. Thirty-one volunteers performed a monetary incentive delay (MID) task while EEG was recorded. FRN amplitudes were comparable and more negative when zero-value outcome deviated from expectations than with expected gain or loss, supporting theories emphasising the impact of unexpectedness and salience on FRN amplitudes. Surprisingly, expected zero-value outcomes elicited the most negative FRNs. However, source localisation showed that such outcomes evoked less activation in cingulate areas than unexpected zero-value outcomes. Our study illustrates the context dependency of identical zero-value feedback stimuli. Moreover, the results indicate that the incentive cues in the MID task evoke different reward prediction error signals. These prediction signals differ in FRN amplitude and neuronal sources, and have to be considered in the design and interpretation of future studies.

Functional connectivity mapping of regions associated with self- and other-processing

Neuroscience literature increasingly suggests a conceptual self composed of interacting neural regions, rather than independent local activations, yet such claims have yet to be investigated. We, thus, combined task-dependent meta-analytic connectivity modeling (MACM) with task-independent resting-state (RS) connectivity analysis to delineate the neural network of the self, across both states. Given psychological evidence implicating the self's interdependence on social information, we also delineated the neural network underlying conceptual other-processing. To elucidate the relation between the self-/other-networks and their function, we mined the MACM metadata to generate a cognitive-behavioral profile for an empirically identified region specific to conceptual self, the pregenual anterior cingulate (pACC), and conceptual other, posterior cingulate/precuneus (PCC/PC). Mining of 7,200 published, task-dependent, neuroimaging studies, using healthy human subjects, yielded 193 studies activating the self-related seed and were conjoined with RS connectivity analysis to delineate a differentiated self-network composed of the pACC (seed) and anterior insula, relative to other functional connectivity. Additionally, 106 studies activating the other-related seed were conjoined with RS connectivity analysis to delineate a differentiated other-network of PCC/PC (seed) and angular gyrus/temporoparietal junction, relative to self-functional connectivity. The self-network seed related to emotional conflict resolution and motivational processing, whereas the other-network seed related to socially oriented processing and contextual information integration. Notably, our findings revealed shared RS connectivity between ensuing self-/other-networks within the ventromedial prefrontal cortex and medial orbitofrontal cortex, suggesting self-updating via integration of self-relevant social information. We, therefore, present initial neurobiological evidence corroborating the increasing claims of an intricate self-network, the architecture of which may promote social value processing.

Surprise signals in the supplementary eye field: rectified prediction errors drive exploration-exploitation transitions

Visual search is coordinated adaptively by monitoring and predicting the environment. The supplementary eye field (SEF) plays a role in oculomotor control and outcome evaluation. However, it is not clear whether the SEF is involved in adjusting behavioral modes based on preceding feedback. We hypothesized that the SEF drives exploration-exploitation transitions by generating "surprise signals" or rectified prediction errors, which reflect differences between predicted and actual outcomes. To test this hypothesis, we introduced an oculomotor two-target search task in which monkeys were required to find two valid targets among four identical stimuli. After they detected the valid targets, they exploited their knowledge of target locations to obtain a reward by choosing the two valid targets alternately. Behavioral analysis revealed two distinct types of oculomotor search patterns: exploration and exploitation. We found that two types of SEF neurons represented the surprise signals. The error-surprise neurons showed enhanced activity when the monkey received the first error feedback after the target pair change, and this activity was followed by an exploratory oculomotor search pattern. The correct-surprise neurons showed enhanced activity when the monkey received the first correct feedback after an error trial, and this increased activity was followed by an exploitative, fixed-type search pattern. Our findings suggest that error-surprise neurons are involved in the transition from exploitation to exploration and that correct-surprise neurons are involved in the transition from exploration to exploitation.

Similarities of biochemical abnormalities between major depressive disorder and bipolar depression: a proton magnetic resonance spectroscopy study

BACKGROUND

Depression in the context of bipolar disorder (BD) is often misdiagnosed as major depressive disorder (MDD), leading to mistreatments and poor clinical outcomes for many bipolar patients. Previous neuroimaging studies found mixed results on brain structure, and biochemical metabolism of the two disorders. To eliminate the compounding effects of medication, and aging, this study sought to investigate the brain biochemical changes of treatment-naïve, non-late-life patients with MDD and BD in white matter in prefrontal (WMP) lobe, anterior cingulate cortex (ACC) and hippocampus by using proton magnetic resonance spectroscopy ((1)H-MRS).

METHODS

Three groups of participants were recruited: 26 MDD patients, 20 depressed BD patients, and 13 healthy controls. The multi-voxel (1)H-MRS [repetition time (TR)=1000ms; echo-time (TE)=144ms] was used for the measurement of N-acetylaspartate(NAA), choline containg compounds (Cho), and creatine (Cr) in three brain locations: white matter in prefrontal (WMP) lobe, anterior cingulate cortex (ACC), and hippocampus. Two ratios of NAA/Cr and Cho/Cr as a measure of brain biochemical changes were compared among three experimental groups.

RESULTS

On the comparison of brain biochemical changes, both MDD patients and BD patients showed many similarities compared to the controls. They both had a significantly lower NAA/Cr ratio in the left WMP lobe. There were no significant differences among three experimental groups for Cho/Cr ratio in the WMP lobe, and for the ratios of NAA/Cr and Cho/Cr in the bilateral ACC and hippocampus. The only difference between MDD and BD patients existed for the NAA/Cr ratio in the right WMP lobe. While MDD patients had a significantly lower NAA/Cr ratio than controls, BD patients showed no such differences. On the comparison of correlation of medical variables and brain biochemical changes, all participants demonstrated no significant correlations.

CONCLUSION

Reduced NAA/Cr ratio at the left WMP lobe indicated the dysfunction of neuronal viability in deep white matter, in both MDD and BD patients who shared similarities of brain biochemical abnormalities, which might imply an overlap in neuropathology of depression.

The neural network underlying incentive-based learning: implications for interpreting circuit disruptions in psychiatric disorders

Coupling stimuli and actions with positive or negative outcomes facilitates the selection of appropriate actions. Several brain regions are involved in the development of goal-directed behaviors and habit formation during incentive-based learning. This Review focuses on higher cognitive control of decision making and the cortical and subcortical structures and connections that attribute value to stimuli, associate that value with choices, and select an action plan. Delineating the connectivity between these areas is fundamental for understanding how brain regions work together to evaluate stimuli, develop actions plans, and modify behavior, as well as for elucidating the pathophysiology of psychiatric diseases.

An information theory account of cognitive control

Our ability to efficiently process information and generate appropriate responses depends on the processes collectively called cognitive control. Despite a considerable focus in the literature on the cognitive control of information processing, neural mechanisms underlying control are still unclear, and have not been characterized by considering the quantity of information to be processed. A novel and comprehensive account of cognitive control is proposed using concepts from information theory, which is concerned with communication system analysis and the quantification of information. This account treats the brain as an information-processing entity where cognitive control and its underlying brain networks play a pivotal role in dealing with conditions of uncertainty. This hypothesis and theory article justifies the validity and properties of such an account and relates experimental findings to the frontoparietal network under the framework of information theory.

Anterior cingulate engagement in a foraging context reflects choice difficulty, not foraging value

Previous theories predict that human dorsal anterior cingulate (dACC) should respond to decision difficulty. An alternative theory has been recently advanced that proposes that dACC evolved to represent the value of 'non-default', foraging behavior, calling into question its role in choice difficulty. However, this new theory does not take into account that choosing whether or not to pursue foraging-like behavior can also be more difficult than simply resorting to a default. The results of two neuroimaging experiments show that dACC is only associated with foraging value when foraging value is confounded with choice difficulty; when the two are dissociated, dACC engagement is only explained by choice difficulty, and not the value of foraging. In addition to refuting this new theory, our studies help to formalize a fundamental connection between choice difficulty and foraging-like decisions, while also prescribing a solution for a common pitfall in studies of reward-based decision making.

For better or worse: reward comparison by the ventromedial prefrontal cortex

In this issue of Neuron, Strait et al. (2014) studied how neurons in the monkey ventromedial prefrontal cortex encode value-based decisions. Neurons were commonly influenced by reward magnitude and probability, showed anticorrelation for better and worse options, and covaried with choice independent of value.

Threat-related learning relies on distinct dorsal prefrontal cortex network connectivity

Conditioned changes in the emotional response to threat (e.g. aversive unconditioned stimulus; UCS) are mediated in part by the prefrontal cortex (PFC). Unpredictable threats elicit large emotional responses, while the response is diminished when the threat is predictable. A better understanding of how PFC connectivity to other brain regions varies with threat predictability would provide important insights into the neural processes that mediate conditioned diminution of the emotional response to threat. The present study examined brain connectivity during predictable and unpredictable threat exposure using a fear conditioning paradigm (previously published in Wood et al., 2012) in which unconditioned functional magnetic resonance imaging data were reanalyzed to assess effective connectivity. Granger causality analysis was performed using the time series data from 15 activated regions of interest after hemodynamic deconvolution, to determine regional effective connectivity. In addition, connectivity path weights were correlated with trait anxiety measures to assess the relationship between negative affect and brain connectivity. Results indicate the dorsomedial PFC (dmPFC) serves as a neural hub that influences activity in other brain regions when threats are unpredictable. In contrast, the dorsolateral PFC (dlPFC) serves as a neural hub that influences the activity of other brain regions when threats are predictable. These findings are consistent with the view that the dmPFC coordinates brain activity to take action, perhaps in a reactive manner, when an unpredicted threat is encountered, while the dlPFC coordinates brain regions to take action, in what may be a more proactive manner, to respond to predictable threats. Further, dlPFC connectivity to other brain regions (e.g. ventromedial PFC, amygdala, and insula) varied with negative affect (i.e. trait anxiety) when the UCS was predictable, suggesting that stronger connectivity may be required for emotion regulation in individuals with higher levels of negative affect.

Prefrontal cortical regulation of fear learning

The prefrontal cortex regulates the expression of fear based on previously learned information. Recently, this brain area has emerged as being crucial in the initial formation of fear memories, providing new avenues to study the neurobiology underlying aberrant learning in anxiety disorders. Here we review the circumstances under which the prefrontal cortex is recruited in the formation of memory, highlighting relevant work in laboratory animals and human subjects. We propose that the prefrontal cortex facilitates fear memory through the integration of sensory and emotional signals and through the coordination of memory storage in an amygdala-based network.

Frontal theta as a mechanism for cognitive control

Recent advancements in cognitive neuroscience have afforded a description of neural responses in terms of latent algorithmic operations. However, the adoption of this approach to human scalp electroencephalography (EEG) has been more limited, despite the ability of this methodology to quantify canonical neuronal processes. Here, we provide evidence that theta band activities over the midfrontal cortex appear to reflect a common computation used for realizing the need for cognitive control. Moreover, by virtue of inherent properties of field oscillations, these theta band processes may be used to communicate this need and subsequently implement such control across disparate brain regions. Thus, frontal theta is a compelling candidate mechanism by which emergent processes, such as 'cognitive control', may be biophysically realized.

A general role for medial prefrontal cortex in event prediction

A recent computational neural model of medial prefrontal cortex (mPFC), namely the predicted response-outcome (PRO) model (Alexander and Brown, 2011), suggests that mPFC learns to predict the outcomes of actions. The model accounted for a wide range of data on the mPFC. Nevertheless, numerous recent findings suggest that mPFC may signal predictions and prediction errors even when the predicted outcomes are not contingent on prior actions. Here we show that the existing PRO model can learn to predict outcomes in a general sense, and not only when the outcomes are contingent on actions. A series of simulations show how this generalized PRO model can account for an even broader range of findings in the mPFC, including human ERP, fMRI, and macaque single-unit data. The results suggest that the mPFC learns to predict salient events in general and provides a theoretical framework that links mPFC function to model-based reinforcement learning, Bayesian learning, and theories of cognitive control.