Calculating the cost of acting in frontal cortex

The nexus between decision making and emotion regulation: a review of convergent neurocognitive substrates

Emotional information, such as reward or punishment, gains rapid and often preferential access to neurocognitive resources. This ability to quickly evaluate and integrate emotion-related information is thought to benefit a range of behaviours critical for survival. Conversely, the improper use of, or preoccupation with, emotional information is associated with disruptions in functioning and psychiatric disorders. Optimally, an organism utilizes emotional information when it is significant, and minimizes its influence when it is not. Recently, similar regions of prefrontal cortex have been identified that are associated with regulating both behavioural conflict (motor response selection or inhibition) and affective conflict (emotional representation and awareness). In this review, data will be examined that concerns this convergence between decision making (modulating what we do) and emotion regulation (modulating how we feel) and an informal model will be proposed linking these processes at a neurocognitive level. The studies reviewed collectively support the conclusion that overlapping areas of prefrontal cortex perform similar computations whether the functional objective is to modulate an operant response, or an emotional one. Specifically, the idea is raised that key aspects of decision making and emotion regulation are bound by a common functional objective in which internal representations of conditioned stimuli and reinforcers are modulated to facilitate optimal behaviour or states. Emphasis is placed on dorsomedial, dorsolateral, ventrolateral, and ventromedial regions of prefrontal cortex.

Goal representations and motivational drive in schizophrenia: the role of prefrontal-striatal interactions

The past several years have seen a resurgence of interest in understanding the psychological and neural bases of what are often referred to as "negative symptoms" in schizophrenia. These aspects of schizophrenia include constructs such as asociality, avolition (a reduction in the motivation to initiate or persist in goal-directed behavior), and anhedonia (a reduction in the ability to experience pleasure). We believe that these dimensions of impairment in individuals with schizophrenia reflect difficulties using internal representations of emotional experiences, previous rewards, and motivational goals to drive current and future behavior in a way that would allow them to obtain desired outcomes, a deficit that has major clinical significance in terms of functional capacity. In this article, we review the major components of the systems that link experienced and anticipated rewards with motivated behavior that could potentially be impaired in schizophrenia. We conclude that the existing evidence suggests relatively intact hedonics in schizophrenia, but impairments in some aspects of reinforcement learning, reward prediction, and prediction error processing, consistent with an impairment in "wanting." As of yet, there is only indirect evidence of impairment in anterior cingulate and orbital frontal function that may support value and effort computations. However, there are intriguing hints that individuals with schizophrenia may not be able to use reward information to modulate cognitive control and dorsolateral prefrontal cortex function, suggesting a potentially important role for cortical-striatal interactions in mediating impairment in motivated and goal-directed behavior in schizophrenia.

External incentives and internal states guide goal-directed behavior via the differential recruitment of the nucleus accumbens and the medial prefrontal cortex

Goal-directed behavior is governed by internal physiological states and external incentives present in the environment (e.g. hunger and food). While the role of the mesocorticolimbic dopamine (DA) system in behavior guided by environmental incentives has been well studied, the effect of relevant physiological states on the function of this system is less understood. The current study examined the role of the medial prefrontal cortex (mPFC) and the nucleus accumbens (NAcc) in the kind of food-reinforced behaviors known to be sensitive to the internal state produced by food deprivation conditions. Operant lever-press reinforced on fixed ratio 1 (FR1) and progressive ratio (PR) schedules was tested after temporary inactivation of, or DA receptor blockade in, the prelimbic mPFC or NAcc core of rats with differing levels of food deprivation (0, 12 and 36-h). Food deprivation increased PR breakpoints, as well as the number of lever-presses emitted on the FR1 schedule. Both temporary inactivation and DA blockade of NAcc reduced breakpoints across deprivation conditions, while temporary inactivation and DA blockade of mPFC reduced breakpoints only in food-deprived rats. Neither manipulation of mPFC and NAcc had any effect on behavior reinforced on the FR1 schedule. Thus, mPFC and NAcc were differentially relevant to the behaviors tested-NAcc was recruited when the behavioral cost per reinforcer was rising or high regardless of food deprivation conditions, while mPFC was recruited when food-deprived animals behaved through periods of sparse reinforcement density in order to maximize available gain.

Reconsidering anhedonia in depression: lessons from translational neuroscience

Anhedonia is a core symptom of major depressive disorder (MDD), the neurobiological mechanisms of which remain poorly understood. Despite decades of speculation regarding the role of dopamine (DA) in anhedonic symptoms, empirical evidence has remained elusive, with frequent reports of contradictory findings. In the present review, we argue that this has resulted from an underspecified definition of anhedonia, which has failed to dissociate between consummatory and motivational aspects of reward behavior. Given substantial preclinical evidence that DA is involved primarily in motivational aspects of reward, we suggest that a refined definition of anhedonia that distinguishes between deficits in pleasure and motivation is essential for the purposes of identifying its neurobiological substrates. Moreover, bridging the gap between preclinical and clinical models of anhedonia may require moving away from the conceptualization of anhedonia as a steady-state, mood-like phenomena. Consequently, we introduce the term "decisional anhedonia" to address the influence of anhedonia on reward decision-making. These proposed modifications to the theoretical definition of anhedonia have implications for research, assessment and treatment of MDD.

Conscious perception of errors and its relation to the anterior insula

To detect erroneous action outcomes is necessary for flexible adjustments and therefore a prerequisite of adaptive, goal-directed behavior. While performance monitoring has been studied intensively over two decades and a vast amount of knowledge on its functional neuroanatomy has been gathered, much less is known about conscious error perception, often referred to as error awareness. Here, we review and discuss the conditions under which error awareness occurs, its neural correlates and underlying functional neuroanatomy. We focus specifically on the anterior insula, which has been shown to be (a) reliably activated during performance monitoring and (b) modulated by error awareness. Anterior insular activity appears to be closely related to autonomic responses associated with consciously perceived errors, although the causality and directions of these relationships still needs to be unraveled. We discuss the role of the anterior insula in generating versus perceiving autonomic responses and as a key player in balancing effortful task-related and resting-state activity. We suggest that errors elicit reactions highly reminiscent of an orienting response and may thus induce the autonomic arousal needed to recruit the required mental and physical resources. We discuss the role of norepinephrine activity in eliciting sufficiently strong central and autonomic nervous responses enabling the necessary adaptation as well as conscious error perception.

Recurrent, robust and scalable patterns underlie human approach and avoidance

BACKGROUND

Approach and avoidance behavior provide a means for assessing the rewarding or aversive value of stimuli, and can be quantified by a keypress procedure whereby subjects work to increase (approach), decrease (avoid), or do nothing about time of exposure to a rewarding/aversive stimulus. To investigate whether approach/avoidance behavior might be governed by quantitative principles that meet engineering criteria for lawfulness and that encode known features of reward/aversion function, we evaluated whether keypress responses toward pictures with potential motivational value produced any regular patterns, such as a trade-off between approach and avoidance, or recurrent lawful patterns as observed with prospect theory.

METHODOLOGY/PRINCIPAL FINDINGS

Three sets of experiments employed this task with beautiful face images, a standardized set of affective photographs, and pictures of food during controlled states of hunger and satiety. An iterative modeling approach to data identified multiple law-like patterns, based on variables grounded in the individual. These patterns were consistent across stimulus types, robust to noise, describable by a simple power law, and scalable between individuals and groups. Patterns included: (i) a preference trade-off counterbalancing approach and avoidance, (ii) a value function linking preference intensity to uncertainty about preference, and (iii) a saturation function linking preference intensity to its standard deviation, thereby setting limits to both.

CONCLUSIONS/SIGNIFICANCE

These law-like patterns were compatible with critical features of prospect theory, the matching law, and alliesthesia. Furthermore, they appeared consistent with both mean-variance and expected utility approaches to the assessment of risk. Ordering of responses across categories of stimuli demonstrated three properties thought to be relevant for preference-based choice, suggesting these patterns might be grouped together as a relative preference theory. Since variables in these patterns have been associated with reward circuitry structure and function, they may provide a method for quantitative phenotyping of normative and pathological function (e.g., psychiatric illness).

Linking the cytokine and neurocircuitry hypotheses of depression: a translational framework for discovery and development of novel anti-depressants

Recent studies suggest a model of depression that links the cytokine hypothesis from the field of psychoneuroimmunology with the neurocircuitry hypothesis derived from burgeoning insight into neurophysiological changes observed in depressed patients. According to the neurocircuitry hypothesis of depression, failure of homeostatic synaptic plasticity in cortical-striatal-limbic nodes of a distributed network of neural circuits involving the sub-genual anterior cingulate cortex is responsible for core symptoms of depression: loss of interest or pleasure (anhedonia) and depressed mood (sadness). According to the cytokine hypothesis of depression, inflammatory cytokines act on neural circuits to evoke the behavioral and physiological changes observed in depression. Synthesis of these hypotheses implicates cytokines released during injury, infection, illness, or psychological stress as a cause of dysregulated synaptic plasticity in cortical-striatal-limbic circuits implicated in depression. These neural circuits process affective and reward-based information for optimal cost-benefit decision-making, a function that may link cytokine-evoked changes in synaptic plasticity to translatable measures of specific behavioral impairments observed in depressed patients. This viewpoint outlines evidence linking the cytokine and neurocircuitry hypotheses of depression to offer a translational model of major depressive disorder suitable for novel drug discovery and development.

Action selection: a race model for selected and non-selected actions distinguishes the contribution of premotor and prefrontal areas

Race models have been used to explain perceptual, motor and oculomotor decisions. Here we developed a race model to explain how human subjects select actions when there are no overt rewards and no external cues to specify which action to make. Critically, we were able to estimate the cumulative activity of neuronal decision-units for selected and non-selected actions. We used functional magnetic resonance imaging (fMRI) to test for regional brain activity that correlated with the predictions of this race model. Activity in the pre-SMA, cingulate motor and premotor areas correlated with prospective selection between responses according to the race model. Activity in the lateral prefrontal cortex did not correlate with the race model, even though this area was active during action selection. This activity related to the degree to which individuals switched between alternative actions. Crucially, a follow-up experiment showed that it was not present on the first trial. Taken together, these results suggest that the lateral prefrontal cortex is not the source for the generation of action. It is more likely that it is involved in switching to alternatives or monitoring previous actions. Thus, our experiment shows the power of the race model in distinguishing the contribution of different areas in the selection of action.

How humans integrate the prospects of pain and reward during choice

The maxim "no pain, no gain" summarizes scenarios in which an action leading to reward also entails a cost. Although we know a substantial amount about how the brain represents pain and reward separately, we know little about how they are integrated during goal-directed behavior. Two theoretical models might account for the integration of reward and pain. An additive model specifies that the disutility of costs is summed linearly with the utility of benefits, whereas an interactive model suggests that cost and benefit utilities interact so that the sensitivity to benefits is attenuated as costs become increasingly aversive. Using a novel task that required integration of physical pain and monetary reward, we examined the mechanism underlying cost-benefit integration in humans. We provide evidence in support of an interactive model in behavioral choice. Using functional neuroimaging, we identify a neural signature for this interaction such that, when the consequences of actions embody a mixture of reward and pain, there is an attenuation of a predictive reward signal in both ventral anterior cingulate cortex and ventral striatum. We conclude that these regions subserve integration of action costs and benefits in humans, a finding that suggests a cross-species similarity in neural substrates that implement this function and illuminates mechanisms that underlie altered decision making under aversive conditions.

Cognitive impulsivity in animal models: role of response time and reinforcing rate in delay intolerance with two-choice operant tasks

Impulsivity, a key symptom of ADHD (attention-deficit hyperactivity disorder), is also common in obsessive-compulsive and addictive disorders. There is rising interest in animal models of inhibitory-control impairment. Adolescent rats were tested daily in the intolerance-to-delay (ID) task (session 25 min, timeout 20 s), involving choice between either immediate small amount of food (SS), or larger amount of food after a delay (LL). The mixed 5-HT(1A/7) agonist (8-OH-DPAT, 0 or 0.060 mg/kg i.p.) was administered acutely just before the last three sessions at highest delays. In addition to the classical choice parameter (percent LL preference), the spontaneous waiting (termed response time, RT) occurring between end of a timeout (TO) and next nose-poke was calculated. The pace between consecutive reinforcer deliveries is given by the mean inter-trial interval (mITI, i.e. TO + RT). Hence, the impact of any given delay may be proportional to this pace and be expressed as delay-equivalent odds, i.e. the extent by which delays are multiples of the mITI. Data revealed that RT/mITI increased sharply from around 15 s/35 s to around 30 s/50 s when imposed delay changed from 30 s to 45 s (i.e. odds from 0.91 to 1.06). This suggests that rats adopted a strategy allowing them to keep in pace with perceived reinforcing rate. The increasing delay constraint directly influenced the length of rats' spontaneous waiting (RT) before next decision. For higher delays, with odds >1, rats shifted to a clear-cut SS preference, which is devoid of any exogenous temporal constraint. A challenge with 8-OH-DPAT (0 or 0.060 mg/kg i.p.) decreased impulsive choice but also increased RT. Thus, tapping onto 5-HT(1A/7) receptors slightly enhanced RT/mITI values, possibly reflecting ability of rats to cope with slower reinforcing rates and/or with delay-cancelled reward paces. In summary, delay-induced states of aversion may arise from the innate tendency to rely on a regular rate of reinforcement. Conversely, a drug-enhanced capacity to cope with delay may involve an internal ability to adjust expectancy about such a reinforcing rate.

BDNF, relative preference, and reward circuitry responses to emotional communication

Brain derived neurotrophic factor (BDNF) regulates neural development and synaptic transmission. We have tested the hypothesis that functional variation in the BDNF gene (Val66Met polymorphism, rs6265) affects brain reward circuitry encoding human judgment and decision-making regarding relative preference. We quantified relative preference among faces with emotional expressions (angry, fearful, sad, neutral, and happy) by a keypress procedure performed offline to measure effort traded for viewing time. Keypress-based relative preferences across the ensemble of faces were mirrored significantly by fMRI signal in the orbitofrontal cortex, amygdala, and hippocampus when passively viewing these faces. For these three brain regions, there was also a statistically significant group difference by BDNF genotype in the fMRI responses to the emotional expressions. In comparison with Val/Met heterozygotes, Val/Val individuals preferentially sought exposure to positive emotions (e.g., happy faces) and had stronger regional fMRI activation to aversive stimuli (e.g., angry, fearful, and sad faces). BDNF genotype accounted for approximately 30% of the variance in fMRI signal that mirrors keypress responses to these stimuli. This study demonstrates that functional allelic variation in BDNF modulates human brain circuits processing reward/aversion information and relative preference transactions.

Impulsive choice in hippocampal but not orbitofrontal cortex-lesioned rats on a nonspatial decision-making maze task

Orbitofrontal cortical (OFC) and hippocampal (HPC) lesions in primates and rodents have been associated with impulsive behaviour. We showed previously that OFC- or HPC-lesioned rats chose the immediate low-reward (LR) option in preference to the delayed high-reward (HR) option, where LR and HR were associated with different spatial responses in a uniform grey T-maze. We now report that on a novel nonspatial T-maze task in which the HR and LR options are associated with patterned goal arms (black-and-white stripes vs. gray), OFC-lesioned rats did not show impulsive behaviour, choosing the delayed HR option, and were indistinguishable from controls. In contrast, HPC-lesioned rats exhibited impulsive choice in the nonspatial decision-making task, although they chose the HR option on the majority of trials when there was a 10-s delay associated with both goal arms. The previously reported impairment in OFC-lesioned rats on the spatial version of the intertemporal choice task is unlikely to reflect a general problem with spatial learning, because OFC lesions were without effect on acquisition of the standard reference memory water-maze task and spatial working memory performance (nonmatching-to-place) on the T-maze. The differential effect of OFC lesions on the two versions of the intertemporal choice task may be explained instead in terms of the putative role of OFC in using associative information to represent expected outcomes and generate predictions. The impulsivity in HPC-lesioned rats may reflect impaired temporal information processing, and emphasizes a role for the hippocampus beyond the spatial domain.

Forebrain circuitry involved in effort-related choice: Injections of the GABAA agonist muscimol into ventral pallidum alter response allocation in food-seeking behavior

Organisms often make effort-related choices based upon assessments of motivational value and work requirements. Nucleus accumbens dopamine is a critical component of the brain circuitry regulating work output in reinforcement-seeking behavior. Rats with accumbens dopamine depletions reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and instead they select a less-effortful type of food-seeking behavior. The ventral pallidum is a brain area that receives substantial GABAergic input from nucleus accumbens. It was hypothesized that stimulation of GABA(A) receptors in the ventral pallidum would result in behavioral effects that resemble those produced by interference with accumbens dopamine transmission. The present studies employed a concurrent choice lever pressing/chow intake procedure; with this task, interference with accumbens dopamine transmission shifts choice behavior such that lever pressing for food is decreased but chow intake is increased. In the present experiments, infusions of the GABA(A) agonist muscimol (5.0-10.0 ng) into the ventral pallidum decreased lever pressing for preferred food, but increased consumption of the less preferred chow. In contrast, ventral pallidal infusions of muscimol (10.0 ng) had no significant effect on preference for the palatable food in free-feeding choice tests. Furthermore, injections of muscimol into a control site dorsal to the ventral pallidum produced no significant effects on lever pressing and chow intake. These data indicate that stimulation of GABA receptors in ventral pallidum produces behavioral effects similar to those produced by accumbens dopamine depletions. Ventral pallidum appears to be a component of the brain circuitry regulating response allocation and effort-related choice behavior, and may act to convey information from nucleus accumbens to other parts of this circuitry. This research may have implications for understanding the brain mechanisms involved in energy-related psychiatric dysfunctions such as psychomotor retardation in depression, anergia, and apathy.