Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures

A multi-sample evaluation of the measurement structure and function of the modified monetary incentive delay task in adolescents

Interpreting the neural response elicited during task functional magnetic resonance imaging (fMRI) remains a challenge in neurodevelopmental research. The monetary incentive delay (MID) task is an fMRI reward processing task that is extensively used in the literature. However, modern psychometric tools have not been used to evaluate measurement properties of the MID task fMRI data. The current study uses data for a similar task design across three adolescent samples (N = 346 [Agemean 12.0; 44 % Female]; N = 97 [19.3; 58 %]; N = 112 [20.2; 38 %]) to evaluate multiple measurement properties of fMRI responses on the MID task. Confirmatory factor analysis (CFA) is used to evaluate an a priori theoretical model for the task and its measurement invariance across three samples. Exploratory factor analysis (EFA) is used to identify the data-driven measurement structure across the samples. CFA results suggest that the a priori model is a poor representation of these MID task fMRI data. Across the samples, the data-driven EFA models consistently identify a six-to-seven factor structure with run and bilateral brain region factors. This factor structure is moderately-to-highly congruent across the samples. Altogether, these findings demonstrate a need to evaluate theoretical frameworks for popular fMRI task designs to improve our understanding and interpretation of brain-behavior associations.

Understanding anxiety symptoms as aberrant defensive responding along the threat imminence continuum

Threat-anticipatory defensive responses have evolved to promote survival in a dynamic world. While inherently adaptive, aberrant expression of defensive responses to potential threat could manifest as pathological anxiety, which is prevalent, impairing, and associated with adverse outcomes. Extensive translational neuroscience research indicates that normative defensive responses are organized by threat imminence, such that distinct response patterns are observed in each phase of threat encounter and orchestrated by partially conserved neural circuitry. Anxiety symptoms, such as excessive and pervasive worry, physiological arousal, and avoidance behavior, may reflect aberrant expression of otherwise normative defensive responses, and therefore follow the same imminence-based organization. Here, empirical evidence linking aberrant expression of specific, imminence-dependent defensive responding to distinct anxiety symptoms is reviewed, and plausible contributing neural circuitry is highlighted. Drawing from translational and clinical research, the proposed framework informs our understanding of pathological anxiety by grounding anxiety symptoms in conserved psychobiological mechanisms. Potential implications for research and treatment are discussed.

Threat and Reward Imminence Processing in the Human Brain

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. In addition, the extent to which aversive-related and appetitive-related processing engage distinct or overlapping circuits remains poorly understood. Here, we sought to investigate the dynamics of aversive and appetitive processing while male and female participants engaged in comparable trials involving threat avoidance or reward seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence. For example, in the aversive domain, we predicted that the bed nucleus of the stria terminalis (BST), but not the amygdala, would exhibit anticipatory responses given the role of the former in anxious apprehension. We also predicted that the periaqueductal gray (PAG) would exhibit threat-proximity responses based on its involvement in proximal-threat processes, and that the ventral striatum would exhibit threat-imminence responses given its role in threat escape in rodents. Overall, we uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the BST, PAG, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Whereas the ventral striatum generated anticipatory responses in the proximity of reward as expected, it also exhibited threat-related imminence responses. In fact, across multiple brain regions, we observed a main effect of arousal. In other words, we uncovered extensive temporally evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information regardless of valence, findings further supported by network analysis.SIGNIFICANCE STATEMENT In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. Here, we sought to investigate the dynamics of aversive/appetitive processing while participants engaged in trials involving threat avoidance or reward seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence. We uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the bed nucleus of the stria terminalis, periaqueductal gray, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Overall, we uncovered extensive temporally evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information regardless of valence.

Threat and reward imminence processing in the human brain

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. In addition, the extent to which aversive- and appetitive-related processing engage distinct or overlapping circuits remains poorly understood. Here, we sought to investigate the dynamics of aversive and appetitive processing while male and female participants engaged in comparable trials involving threat-avoidance or reward-seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence . For example, in the aversive domain, we predicted that the bed nucleus of the stria terminalis (BST), but not the amygdala, would exhibit anticipatory responses given the role of the former in anxious apprehension. We also predicted that the periaqueductal gray (PAG) would exhibit threat-proximity responses based on its involvement in proximal-threat processes, and that the ventral striatum would exhibit threat-imminence responses given its role in threat escape in rodents. Overall, we uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the BST, PAG, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Whereas the ventral striatum generated anticipatory responses in the proximity of reward as expected, it also exhibited threat-related imminence responses. In fact, across multiple brain regions, we observed a main effect of arousal. In other words, we uncovered extensive temporally-evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information irrespective of valence, findings further supported by network analysis.

Significance Statement

In the human brain, aversive and appetitive processing have been studied with controlled stimuli in rather static settings. Here, we sought to investigate the dynamics of aversive/appetitive processing while participants engaged in trials involving threat-avoidance or reward-seeking. A central goal was to characterize the temporal evolution of responses during periods of threat or reward imminence . We uncovered imminence-related temporally increasing ("ramping") responses in multiple brain regions, including the bed nucleus of the stria terminalis, periaqueductal gray, and ventral striatum, subcortically, and dorsal anterior insula and anterior midcingulate, cortically. Overall, we uncovered extensive temporally-evolving, imminence-related processing in both the aversive and appetitive domain, suggesting that distributed brain circuits are dynamically engaged during the processing of biologically relevant information irrespective of valence.

The rat Lux Actuating Search Task (LAST) and effects of sleep deprivation on task reversal performance

Sleep deprivation (SD) causes significant deficits in multiple aspects of cognition, including sustained attention and working memory. Investigating the neural processes underpinning these cognitive losses has proven challenging due to the confounds of current animal tasks; many employ appetitive or aversive stimuli to motivate behavior, while others lack task complexity that translates to human studies of executive function. We established the Lux Actuating Search Task (LAST) to circumvent these issues. The LAST is performed in a circular, open-field arena that requires rats to find an unmarked, quasi-randomly positioned target. Constant low-level floor vibrations motivate ambulation, while light intensity (determined by the rodent's proximity to the target destination) provides continuous visual feedback. The task has two paradigms that differ based on the relationship between the light intensity and target proximity: the Low Lux Target (LLT) paradigm and the High Lux Target paradigm (HLT). In this study, on days 1–6, the rats completed nine trials per day on one of the two paradigms. On day 7, the rats were either sleep deprived by gentle handling or were left undisturbed before undertaking the opposite (reversal) paradigm on days 7–9. Our results showed that SD significantly impeded the ability of Long Evans rats to learn the reversal paradigm, as indicated by increased times to target and increased failure percentages compared to rats whose sleep was undisturbed. Rats also showed reduced learning with the HLT paradigm, as the initial task or as the reversal task, likely due to the rodents' photophobia limiting their motivation to navigate toward a bright light, which is required to succeed.

•

A continuous feedback paradigm examining the effects of sleep loss on cognitive flexibility in rats is introduced.

•

Floor vibrations motivate and variable light intensity directs navigation to an unmarked location in an open field arena.

•

The reversal of light intensity cues from light to dark and vice versa is disrupted by sleep deprivation.

Neural signals implicated in the processing of appetitive and aversive events in social and non-social contexts

In 2014, we participated in a special issue of Frontiers examining the neural processing of appetitive and aversive events. Specifically, we reviewed brain areas that contribute to the encoding of prediction errors and value versus salience, attention and motivation. Further, we described how we disambiguated these cognitive processes and their neural substrates by using paradigms that incorporate both appetitive and aversive stimuli. We described a circuit in which the orbitofrontal cortex (OFC) signals expected value and the basolateral amygdala (BLA) encodes the salience and valence of both appetitive and aversive events. This information is integrated by the nucleus accumbens (NAc) and dopaminergic (DA) signaling in order to generate prediction and prediction error signals, which guide decision-making and learning via the dorsal striatum (DS). Lastly, the anterior cingulate cortex (ACC) is monitoring actions and outcomes, and signals the need to engage attentional control in order to optimize behavioral output. Here, we expand upon this framework, and review our recent work in which within-task manipulations of both appetitive and aversive stimuli allow us to uncover the neural processes that contribute to the detection of outcomes delivered to a conspecific and behaviors in social contexts. Specifically, we discuss the involvement of single-unit firing in the ACC and DA signals in the NAc during the processing of appetitive and aversive events in both social and non-social contexts.

Dissociable influences of reward and punishment on adaptive cognitive control

To invest effort into any cognitive task, people must be sufficiently motivated. Whereas prior research has focused primarily on how the cognitive control required to complete these tasks is motivated by the potential rewards for success, it is also known that control investment can be equally motivated by the potential negative consequence for failure. Previous theoretical and experimental work has yet to examine how positive and negative incentives differentially influence the manner and intensity with which people allocate control. Here, we develop and test a normative model of control allocation under conditions of varying positive and negative performance incentives. Our model predicts, and our empirical findings confirm, that rewards for success and punishment for failure should differentially influence adjustments to the evidence accumulation rate versus response threshold, respectively. This dissociation further enabled us to infer how motivated a given person was by the consequences of success versus failure.

Impaired Learning From Errors and Punishments and Maladaptive Avoidance-General Mechanisms Underlying Self-Regulation Disorders?

Self-regulation (SR) is an important human function that relates to quality of life in multiple domains including mental health. Previous studies have found important correlates of low SR including impulsivity and poor emotional regulation; however, underpinnings of low SR are incompletely understood. Individuals low in SR frequently engage in maladaptive behaviors (substance abuse, procrastination, etc.) despite negative consequences. This phenomenon suggests that impaired learning from errors and punishments may be important mechanisms underlying low SR. Consistently, previous studies observed impaired error processing in a wide spectrum of individuals with low SR and impaired learning from errors and punishments in SR-related disorders. We also note a possible role for poor emotional regulation and refer to concepts suggesting that engaging in maladaptive behaviors may serve as short term emotion regulation strategies aimed at avoiding or alleviating negative affect. We speculate on transdiagnostic factors underlying poor SR. We propose that impaired error processing (possibly related to striatal functioning) may prevent subjects with low SR from learning from errors and punishments and thus learning better SR skills or tendencies. Additionally, impaired coping in emotionally challenging situations, possibly related to prefrontal-cortical functioning, may lead to maladaptive avoidance. Moreover, maladaptive behaviors may be reinforced by the temporary decreases in negative affect and rewarding values of behaviors. Given existing knowledge gaps, we call for more extensive research and describe possible directions and challenges for future studies.

Drosophila reward system - A summary of current knowledge

The fruit fly Drosophila melanogaster brain is the most extensively investigated model of a reward system in insects. Drosophila can discriminate between rewarding and punishing environmental stimuli and consequently undergo associative learning. Functional models, especially those modelling mushroom bodies, are constantly being developed using newly discovered information, adding to the complexity of creating a simple model of the reward system. This review aims to clarify whether its reward system also includes a hedonic component. Neurochemical systems that mediate the 'wanting' component of reward in the Drosophila brain are well documented, however, the systems that mediate the pleasure component of reward in mammals, including those involving the endogenous opioid and endocannabinoid systems, are unlikely to be present in insects. The mushroom body components exhibit differential developmental age and different functional processes. We propose a hypothetical hierarchy of the levels of reinforcement processing in response to particular stimuli, and the parallel processes that take place concurrently. The possible presence of activity-silencing and meta-satiety inducing levels in Drosophila should be further investigated.

Neutral Sphingomyelinase is an Affective Valence-Dependent Regulator of Learning and Memory

Sphingolipids and enzymes of the sphingolipid rheostat determine synaptic appearance and signaling in the brain, but sphingolipid contribution to normal behavioral plasticity is little understood. Here we asked how the sphingolipid rheostat contributes to learning and memory of various dimensions. We investigated the role of these lipids in the mechanisms of two different types of memory, such as appetitively and aversively motivated memory, which are considered to be mediated by different neural mechanisms. We found an association between superior performance in short- and long-term appetitively motivated learning and regionally enhanced neutral sphingomyelinase (NSM) activity. An opposite interaction was observed in an aversively motivated task. A valence-dissociating role of NSM in learning was confirmed in mice with genetically reduced NSM activity. This role may be mediated by the NSM control of N-methyl-d-aspartate receptor subunit expression. In a translational approach, we confirmed a positive association of serum NSM activity with long-term appetitively motivated memory in nonhuman primates and in healthy humans. Altogether, these data suggest a new sphingolipid mechanism of de-novo learning and memory, which is based on NSM activity.

Integration of value and action in medial prefrontal neural systems

The rodent medial prefrontal cortex (mPFC) plays a key role in regulating cognition, emotion, and behavior. mPFC neurons are activated in diverse experimental paradigms, raising the questions of whether there are specific task elements or dimensions encoded by mPFC neurons, and whether these encoded parameters are selective to neurons in particular mPFC subregions or networks. Here, we consider the role of mPFC neurons in processing appetitive and aversive cues, outcomes, and related behaviors. mPFC neurons are strongly activated in tasks probing value and outcome-associated actions, but these responses vary across experimental paradigms. Can we identify specific categories of responses (e.g., positive or negative value), or do mPFC neurons exhibit response properties that are too heterogeneous/complex to cluster into distinct conceptual groups? Based on a review of relevant studies, we consider what has been done and what needs to be further explored in order to address these questions.

Unanticipated Stressful and Rewarding Experiences Engage the Same Prefrontal Cortex and Ventral Tegmental Area Neuronal Populations

Brain networks that mediate motivated behavior in the context of aversive and rewarding experiences involve the prefrontal cortex (PFC) and ventral tegmental area (VTA). Neurons in both regions are activated by stress and reward, and by learned cues that predict aversive or appetitive outcomes. Recent studies have proposed that separate neuronal populations and circuits in these regions encode learned aversive versus appetitive contexts. But how about the actual experience? Do the same or different PFC and VTA neurons encode unanticipated aversive and appetitive experiences? To address this, we recorded unit activity and local field potentials (LFPs) in the dorsomedial PFC (dmPFC) and VTA of male rats as they were exposed, in the same recording session, to reward (sucrose) or stress (tail pinch) spaced 1 h apart. As expected, experience-specific neuronal responses were observed. Approximately 15–25% of single units in each region responded by excitation or inhibition to either stress or reward, and only stress increased LFP theta oscillation power in both regions and coherence between regions. But the largest number of responses (29% dmPFC and 30% VTA units) involved dual-valence neurons that responded to both stress and reward exposure. Moreover, the temporal profile of neuronal population activity in dmPFC and VTA as assessed by principal component analysis (PCA) were similar during both types of experiences. These results reveal that aversive and rewarding experiences engage overlapping neuronal populations in the dmPFC and the VTA. These populations may provide a locus of vulnerability for stress-related disorders, which are often associated with anhedonia.

Dopamine signals related to appetitive and aversive events in paradigms that manipulate reward and avoidability

Using environmental cues to acquire good and avoid harmful things is critical for survival. Rewarding and aversive outcomes both drive behavior through reinforcement learning and sometimes occur together in the environment, but it remains unclear how these signals are encoded within the brain and if signals for positive and negative reinforcement are encoded similarly. Recent studies demonstrate that the dopaminergic system and interconnected brain regions process both positive and negative reinforcement necessary for approach and avoidance behaviors, respectively. Here, we review these data with a special focus on behavioral paradigms that manipulate both expected reward and the avoidability of aversive events to reveal neural correlates related to value, prediction error encoding, motivation, and salience.

Altered anticipation and processing of aversive interoceptive experience among women remitted from bulimia nervosa

Bulimia nervosa (BN) is characterized by dysregulated intake of food, which may indicate homeostatic imbalance. Critically important for homeostatic regulation is interoception, or the sensing and processing of body-relevant information. A well-documented link between avoidance of unpleasant body sensations and BN symptoms suggests that aversive interoceptive experiences may be particularly relevant to BN pathophysiology. This study examined whether individuals with a history of BN show aberrant neural processing of aversive interoceptive stimuli. Using a cued inspiratory breathing load paradigm, we compared women remitted from BN (RBN; n = 24; to reduce the confounding effects of active bulimic symptoms) and control women (CW; n = 25). During breathing load anticipation, the RBN group, relative to CW, showed increased activation in mid-insula, superior frontal gyrus, putamen, dorsal anterior cingulate, posterior cingulate, and amygdala. However, over the course of the aversive experience, neural activation in RBN relative to CW showed an aberrant decline in most of these regions. Exploratory analyses indicated that greater activation during breathing load anticipation was associated with past bulimic symptom severity and the duration of symptom remission. An exaggerated anticipatory response and an abnormally decreasing response during aversive homeostatic perturbations may promote hallmark bulimic behaviors-binge eating, dietary restriction, and purging. Our findings support a role for homeostatic instability in BN, and these altered patterns of brain activation may serve as novel targets for pharmacological, neuromodulatory, and behavioral interventions.

Reward anticipation and punishment anticipation are instantiated in the brain via opponent mechanisms

fMRI investigations have examined the extent to which reward and punishment motivation are associated with common or opponent neural systems, but such investigations have been limited by confounding variables and methodological constraints. The present study aimed to address limitations of earlier approaches and more comprehensively evaluate the extent to which neural activation associated with reward and punishment motivation reflects opponent or shared systems. Participants completed a modified monetary incentive delay task, which involved the presentation of a cue followed by a target to which participants were required to make a speeded button press. Using a factorial design, cues indicated whether monetary reward and/or loss (i.e., cues signaled probability of reward, punishment, both, or neither) could be expected depending upon response speed. Neural analyses evaluated evidence of (a) directionally opposing effects by testing for regions of differential activation for reward and punishment anticipation, (b) mutual inhibition by testing for interactive effects of reward and punishment anticipation within a factorial design, and (c) opposing effects on shared outputs via a psychophysiological interaction analysis. Evidence supporting all three criteria for opponent systems was obtained. Collectively, present findings support conceptualizing reward and punishment motivation as opponent forces influencing brain and behavior and indicate that shared activation does not suggest the operation of a common neural mechanism instantiating reward and punishment motivation.

Punishment-related memory-guided attention: Neural dynamics of perceptual modulation

Remembering the outcomes of past experiences allows us to generate future expectations and shape selection in the long-term. A growing number of studies has shown that learned positive reward values impact spatial memory-based attentional biases on perception. However, whether memory-driven attentional biases extend to punishment-related values has received comparatively less attention. Here, we manipulated whether recent spatial contextual memories became associated with successful avoidance of punishment (potential monetary loss). Behavioral and electrophysiological measures were collected from 27 participants during a subsequent memory-based attention task, in which we tested for the effect of punishment avoidance associations. Punishment avoidance significantly amplified effects of spatial contextual memories on visual search processes within natural scenes. Compared to non-associated scenes, contextual memories paired with punishment avoidance lead to faster responses to targets presented at remembered locations. Event-related potentials elicited by target stimuli revealed that acquired motivational value of specific spatial locations, by virtue of their association with past avoidance of punishment, dynamically affected neural signatures of early visual processing (indexed by larger P1 and earlier N1 potentials) and target selection (as indicated by reduced N2pc potentials). The present results extend our understanding of how memory, attention, and punishment-related mechanisms interact to optimize perceptual decision in real world environments.

Potential reward reduces the adverse impact of negative distractor stimuli

Knowledge about interactions between reward and negative processing is rudimentary. Here, we employed functional MRI to probe how potential reward signaled by advance cues alters aversive distractor processing during perception. Behaviorally, the influence of aversive stimuli on task performance was reduced during the reward compared to no-reward condition. In the brain, at the task phase, paralleling the observed behavioral pattern, we observed significant interactions in the anterior insula and dorsal anterior cingulate cortex, such that responses during the negative (vs neutral) condition were reduced during the reward compared to no-reward condition. Notably, negative distractor processing in the amygdala appeared to be independent of the reward manipulation. During the initial cue phase, we observed increased reward-related responses in the ventral striatum/accumbens, which were correlated with behavioral interference scores at the subsequent task phase, revealing that participants with increased reward-related responses exhibited a greater behavioral benefit of reward in reducing the adverse effect of negative images. Furthermore, during processing of reward (vs no-reward) cues, the ventral striatum exhibited stronger functional connectivity with fronto-parietal regions important for attentional control. Together, our findings contribute to the understanding of how potential reward influences attentional control and reduces negative distractor processing in the human brain.

Anatomical and functional organization of the human substantia nigra and its connections

We investigated the anatomical and functional organization of the human substantia nigra (SN) using diffusion and functional MRI data from the Human Connectome Project. We identified a tripartite connectivity-based parcellation of SN with a limbic, cognitive, motor arrangement. The medial SN connects with limbic striatal and cortical regions and encodes value (greater response to monetary wins than losses during fMRI), while the ventral SN connects with associative regions of cortex and striatum and encodes salience (equal response to wins and losses). The lateral SN connects with somatomotor regions of striatum and cortex and also encodes salience. Behavioral measures from delay discounting and flanker tasks supported a role for the value-coding medial SN network in decisional impulsivity, while the salience-coding ventral SN network was associated with motor impulsivity. In sum, there is anatomical and functional heterogeneity of human SN, which underpins value versus salience coding, and impulsive choice versus impulsive action.

A Network Model of the Emotional Brain

Emotion is often understood in terms of a circumscribed set of cortical and subcortical brain regions. I propose, instead, that emotion should be understood in terms of large-scale network interactions spanning the entire neuroaxis. I describe multiple anatomical and functional principles of brain organization that lead to the concept of 'functionally integrated systems', cortical-subcortical systems that anchor the organization of emotion in the brain. The proposal is illustrated by describing the cortex-amygdala integrated system and how it intersects with systems involving the ventral striatum/accumbens, septum, hippocampus, hypothalamus, and brainstem. The important role of the thalamus is also highlighted. Overall, the model clarifies why the impact of emotion is wide-ranging, and how emotion is interlocked with perception, cognition, motivation, and action.

Toward a systems-oriented approach to the role of the extended amygdala in adaptive responding

Research into the structure and function of the basal forebrain macrostructure called the extended amygdala (EA) has recently seen considerable growth. This paper reviews that work, with the objectives of identifying underlying themes and developing a common goal towards which investigators of EA function might work. The paper begins with a brief review of the structure and the ontological and phylogenetic origins of the EA. It continues with a review of research into the role of the EA in both aversive and appetitive states, noting that these two seemingly disparate avenues of research converge on the concept of reinforcement - either negative or positive - of adaptive responding. These reviews lead to a proposal as to where the EA may fit in the organization of the basal forebrain, and an invitation to investigators to place their findings in a unifying conceptual framework of the EA as a collection of neural ensembles that mediate adaptive responding.

Dopaminergic Modulation of Decision Making and Subjective Well-Being

The neuromodulator dopamine has a well established role in reporting appetitive prediction errors that are widely considered in terms of learning. However, across a wide variety of contexts, both phasic and tonic aspects of dopamine are likely to exert more immediate effects that have been less well characterized. Of particular interest is dopamine's influence on economic risk taking and on subjective well-being, a quantity known to be substantially affected by prediction errors resulting from the outcomes of risky choices. By boosting dopamine levels using levodopa (l-DOPA) as human subjects made economic decisions and repeatedly reported their momentary happiness, we show here an effect on both choices and happiness. Boosting dopamine levels increased the number of risky options chosen in trials involving potential gains but not trials involving potential losses. This effect could be better captured as increased Pavlovian approach in an approach-avoidance decision model than as a change in risk preferences within an established prospect theory model. Boosting dopamine also increased happiness resulting from some rewards. Our findings thus identify specific novel influences of dopamine on decision making and emotion that are distinct from its established role in learning.

BOLD subjective value signals exhibit robust range adaptation

Many theories of decision making assume that choice options are assessed along a common subjective value (SV) scale. The neural correlates of SV are widespread and reliable, despite the wide variation in the range of values over which decisions are made (e.g., between goods worth a few dollars, in some cases, or hundreds of dollars, in others). According to adaptive coding theories (Barlow, 1961), an efficient value signal should exhibit range adaptation, such that neural activity maintains a fixed dynamic range, and the slope of the value response varies inversely with the range of values within the local context. Although monkey data have demonstrated range adaptation in single-unit correlates of value (Padoa-Schioppa, 2009; Kobayashi et al., 2010), whether BOLD value signals exhibit similar range adaptation is unknown. To test for this possibility, we presented human participants with choices between a fixed immediate and variable delayed payment options. Across two conditions, the delayed options' SVs spanned either a narrow or wide range. SV-tracking activity emerged in the posterior cingulate, ventral striatum, anterior cingulate, and ventromedial prefrontal cortex. Throughout this network, we observed evidence consistent with the predictions of range adaptation: the SV response slope increased in the narrow versus wide range, with statistically significant slope changes confirmed for the posterior cingulate and ventral striatum. No regions exhibited a reliably increased BOLD activity range in the wide versus narrow condition. Our observations of range adaptation present implications for the interpretation of BOLD SV responses that are measured across different contexts or individuals.

Central role for the insular cortex in mediating conditioned responses to anticipatory cues

Reward-related circuits are fundamental for initiating feeding on the basis of food-predicting cues, whereas gustatory circuits are believed to be involved in the evaluation of food during consumption. However, accumulating evidence challenges such a rigid separation. The insular cortex (IC), an area largely studied in rodents for its role in taste processing, is involved in representing anticipatory cues. Although IC responses to anticipatory cues are well established, the role of IC cue-related activity in mediating feeding behaviors is poorly understood. Here, we examined the involvement of the IC in the expression of cue-triggered food approach in mice trained with a Pavlovian conditioning paradigm. We observed a significant change in neuronal firing during presentation of the cue. Pharmacological silencing of the IC inhibited food port approach. Such a behavior could be recapitulated by temporally selective inactivation during the cue. These findings represent the first evidence, to our knowledge, that cue-evoked neuronal activity in the mouse IC modulates behavioral output, and demonstrate a causal link between cue responses and feeding behaviors.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Behavioral flexibility in rats and mice: contributions of distinct frontocortical regions

Research examining the contribution of genetics to behavior is increasingly focused on higher order behavioral and cognitive processes including the ability to modify behaviors when environmental demands change. The frontal cortices of mammals, including rodents, subserve a diverse set of behavioral and cognitive functions including motor planning, social behavior, evaluation of expected outcomes and working memory, which may be particularly sensitive to genetic factors and interactions with experience (e.g. stress). Behavioral flexibility is a core attribute of these functions. This review orients readers to the current landscape of the literature on the frontocortical bases of behavioral flexibility in rodent laboratory experiments. Studies are divided into three broad categories: reversal learning, inhibitory learning and set-shifting. Functional dissociations within the broader scope of behavioral flexibility are reviewed, followed by discussion of the associations between specific components of frontal cortex and specific aspects of relevant behavioral processes. Finally, the authors identify open questions that need to be addressed to better establish the constituents of frontal cortex underlying behavioral flexibility.

A comparison of neural responses to appetitive and aversive stimuli in humans and other mammals

Distinguishing potentially harmful or beneficial stimuli is necessary for the self-preservation and well-being of all organisms. This assessment requires the ongoing valuation of environmental stimuli. Despite much work on the processing of aversive- and appetitive-related brain signals, it is not clear to what degree these two processes interact across the brain. To help clarify this issue, this report used a cross-species comparative approach in humans (i.e. meta-analysis of imaging data) and other mammals (i.e. targeted review of functional neuroanatomy in rodents and non-human primates). Human meta-analysis results suggest network components that appear selective for appetitive (e.g. ventromedial prefrontal cortex, ventral tegmental area) or aversive (e.g. cingulate/supplementary motor cortex, periaqueductal grey) processing, or that reflect overlapping (e.g. anterior insula, amygdala) or asymmetrical, i.e. apparently lateralized, activity (e.g. orbitofrontal cortex, ventral striatum). However, a closer look at the known value-related mechanisms from the animal literature suggests that all of these macroanatomical regions are involved in the processing of both appetitive and aversive stimuli. Differential spatiotemporal network dynamics may help explain similarities and differences in appetitive- and aversion-related activity.