From prediction error to incentive salience: mesolimbic computation of reward motivation

How foraging works: Uncertainty magnifies food-seeking motivation

Food uncertainty has the effect of invigorating food-related responses. Psychologists have noted that mammals and birds respond more to a conditioned stimulus that unreliably predicts food delivery, and ecologists have shown that animals (especially small passerines) consume and/or hoard more food and can get fatter when access to that resource is unpredictable. Are these phenomena related? We think they are. Psychologists have proposed several mechanistic interpretations, while ecologists have suggested a functional interpretation: The effect of unpredictability on fat reserves and hoarding behavior is an evolutionary strategy acting against the risk of starvation when food is in short supply. Both perspectives are complementary, and we argue that the psychology of incentive motivational processes can shed some light on the causal mechanisms leading animals to seek and consume more food under uncertainty in the wild. Our theoretical approach is in agreement with neuroscientific data relating to the role of dopamine, a neurotransmitter strongly involved in incentive motivation, and its plausibility has received some explanatory and predictive value with respect to Pavlovian phenomena. Overall, we argue that the occasional and unavoidable absence of food rewards has motivational effects (called incentive hope) that facilitate foraging effort. We show that this hypothesis is computationally tenable, leading foragers in an unpredictable environment to consume more food items and to have higher long-term energy storage than foragers in a predictable environment.

Using model fish to study the biological mechanisms of cooperative behaviour: A future for translational research concerning social anxiety disorders?

Human societies demand of its composing members the development of a wide array of social tools and strategies. A notable example is human outstanding ability to cooperate with others, in all its complex forms, depicting the reality of a highly demanding social framework in which humans need to be integrated as to attain physical and mental benefits. Considering the importance of social engagement, it's not entirely unexpected that most psychiatric disorders involve some disruption of normal social behaviour, ranging from an abnormal absence to a significant increase of social functioning. It is however surprising that knowledge on these social anxiety disorders still remains so limited. Here we review the literature focusing on the social and cooperative toolbox of 3 fish model species (cleaner fishes, guppies and zebrafish) which are amenable systems to test for social disorders. We build on current knowledge based on ethological information, arising from studies on cooperative behaviour in cleanerfishes and guppies, while profiting from the advantages of the intense use of zebrafish, to create novel paradigms aiming at the major socio-cognitive modules/dimensions in fish species. This focus may enable the discovery of putative conserved endpoints which are relevant for research into social disorders. We suggest that cross-species, cross-domain, functional and genetic approaches could provide a wider array of information on the neurobiological bases of social and cooperative behaviour, crucial to understanding the neural bases of social disorders and key to finding novel avenues towards treatment.

Human Striatal Response to Reward Anticipation Linked to Hippocampal Glutamate Levels

BACKGROUND

Dysfunctional reward processing is associated with a number of psychiatric disorders, such as addiction and schizophrenia. It is thought that reward is regulated mainly by dopamine transmission in the ventral striatum. Contemporary animal models suggest that striatal dopamine concentrations and associated behaviors are related to glutamatergic functioning in the ventral hippocampus. However, in humans the association between reward-related ventral striatal response and hippocampal glutamate levels is unclear.

METHODS

Nineteen healthy participants were studied using proton magnetic resonance spectroscopy to measure hippocampal glutamate levels, and functional magnetic resonance imaging to assess striatal activation and functional connectivity during performance of a monetary incentive delay task.

RESULTS

We found that ventral striatal activation related to reward processing was correlated with hippocampal glutamate levels. In addition, context-dependent functional coupling was demonstrated between the ventral striatum and both the lingual gyrus and hippocampus during reward anticipation. Elevated hippocampal glutamate levels were inversely related to context-dependent functional connectivity between the ventral striatum and the anterior hippocampus while anticipating reward.

CONCLUSIONS

These findings indicate that human striatal responses to reward are influenced by hippocampal glutamate levels. This may be relevant for psychiatric disorders associated with abnormal reward processing such as addiction and schizophrenia.

The effects of cannabis use on salience attribution: a systematic review

OBJECTIVE

The relationship between cannabis use and the onset of psychosis is well established. Aberrant salience processing is widely thought to underpin many of these symptoms. Literature explicitly investigating the relationship between aberrant salience processing and cannabis use is scarce; with those few studies finding that acute tetrahydrocannabinol (THC) administration (the main psychoactive component of cannabis) can result in abnormal salience processing in healthy cohorts, mirroring that observed in psychosis. Nevertheless, the extent of and mechanisms through which cannabis has a modulatory effect on aberrant salience, following both acute and chronic use, remain unclear.

METHODS

Here, we systematically review recent findings on the effects of cannabis use - either through acute THC administration or in chronic users - on brain regions associated with salience processing (through functional MRI data); and performance in cognitive tasks that could be used as either direct or indirect measures of salience processing. We identified 13 studies either directly or indirectly exploring salience processing. Three types of salience were identified and discussed - incentive/motivational, emotional/affective, and attentional salience.

RESULTS

The results demonstrated an impairment of immediate salience processing, following acute THC administration. Amongst the long-term cannabis users, normal salience performance appeared to be underpinned by abnormal neural processes.

CONCLUSIONS

Overall, the lack of research specifically exploring the effects of cannabis use on salience processing, weaken any conclusions drawn. Additional research explicitly focussed on salience processing and cannabis use is required to advance our understanding of the neurocognitive mechanisms underlying the association between cannabis use and development of psychosis.

Food Addiction and Binge Eating: Lessons Learned from Animal Models

The feeding process is required for basic life, influenced by environment cues and tightly regulated according to demands of the internal milieu by regulatory brain circuits. Although eating behaviour cannot be considered "addictive" under normal circumstances, people can become "addicted" to this behaviour, similarly to how some people are addicted to drugs. The symptoms, cravings and causes of "eating addiction" are remarkably similar to those experienced by drug addicts, and both drug-seeking behaviour as eating addiction share the same neural pathways. However, while the drug addiction process has been highly characterised, eating addiction is a nascent field. In fact, there is still a great controversy over the concept of "food addiction". This review aims to summarize the most relevant animal models of "eating addictive behaviour", emphasising binge eating disorder, that could help us to understand the neurobiological mechanisms hidden under this behaviour, and to improve the psychotherapy and pharmacological treatment in patients suffering from these pathologies.

A Prediction Error-driven Retrieval Procedure for Destabilizing and Rewriting Maladaptive Reward Memories in Hazardous Drinkers

Maladaptive reward memories (MRMs) can become unstable following retrieval under certain conditions, allowing their modification by subsequent new learning. However, robust (well-rehearsed) and chronologically old MRMs, such as those underlying substance use disorders, do not destabilize easily when retrieved. A key determinate of memory destabilization during retrieval is prediction error (PE). We describe a retrieval procedure for alcohol MRMs in hazardous drinkers that specifically aims to maximize the generation of PE and therefore the likelihood of MRM destabilization. The procedure requires explicitly generating the expectancy of alcohol consumption and then violating this expectancy (withholding alcohol) following the presentation of a brief set of prototypical alcohol cue images (retrieval + PE). Control procedures involve presenting the same cue images, but allow alcohol to be consumed, generating minimal PE (retrieval-no PE) or generate PE without retrieval of alcohol MRMs, by presenting orange juice cues (no retrieval + PE). Subsequently, we describe a multisensory disgust-based counterconditioning procedure to probe MRM destabilization by re-writing alcohol cue-reward associations prior to reconsolidation. This procedure pairs alcohol cues with images invoking pathogen disgust and an extremely bitter-tasting solution (denatonium benzoate), generating gustatory disgust. Following retrieval + PE, but not no retrieval + PE or retrieval-no PE, counterconditioning produces evidence of MRM rewriting as indexed by lasting reductions in alcohol cue valuation, attentional capture, and alcohol craving.

Parallels and Overlap: The Integration of Homeostatic Signals by Mesolimbic Dopamine Neurons

Motivated behaviors are often initiated in response to perturbations of homeostasis. Indeed, animals and humans have fundamental drives to procure (appetitive behaviors) and eventually ingest (consummatory behaviors) substances based on deficits in body fluid (e.g., thirst) and energy balance (e.g., hunger). Consumption, in turn, reinforces motivated behavior and is therefore considered rewarding. Over the years, the constructs of homeostatic (within the purview of the hypothalamus) and reward (within the purview of mesolimbic circuitry) have been used to describe need-based vs. need-free consumption. However, many experiments have demonstrated that mesolimbic circuits and "higher-order" brain regions are also profoundly influenced by changes to physiological state, which in turn generate behaviors that are poised to maintain homeostasis. Mesolimbic pathways, particularly dopamine neurons of the ventral tegmental area (VTA) and their projections to nucleus accumbens (NAc), can be robustly modulated by a variety of energy balance signals, including post-ingestive feedback relaying nutrient content and hormonal signals reflecting hunger and satiety. Moreover, physiological states can also impact VTA-NAc responses to non-nutritive rewards, such as drugs of abuse. Coupled with recent evidence showing hypothalamic structures are modulated in anticipation of replenished need, classic boundaries between circuits that convey perturbations in homeostasis and those that drive motivated behavior are being questioned. In the current review, we examine data that have revealed the importance of mesolimbic dopamine neurons and their downstream pathways as a dynamic neurobiological mechanism that provides an interface between physiological state, perturbations to homeostasis, and reward-seeking behaviors.

Ventral Striatal Reactivity in Compulsive Sexual Behaviors

Compulsive Sexual Behaviors (CSB) are a reason to seek treatment. Given this reality, the number of studies on CSB has increased substantially in the last decade and the World Health Organization (WHO) included CSB in its proposal for the upcoming ICD-11. Sixty percent of the neuroimaging studies on CSB published since 2014 aimed to examine similarities and differences between brain mechanisms underlying CSB, gambling disorder, and substance use disorders. One of the crucial brain circuits involved in addiction is the reward system involving the ventral striatum (including nucleus accumbens). There are two distinct theories describing ventral striatal activity in addictions: Incentive Salience Theory (IST) and Reward Deficiency Syndrome (RDS). IST describes increased ventral striatal activations during the anticipation of addiction-related reward, while RDS describes decreased ventral striatal reactivity both during the anticipation of the reward and during the reward processing. Here, we aim to investigate how the findings on ventral striatal reactivity in CSB support each of these two addiction frameworks. For this purpose, we conducted a systematic review of neuroimaging studies on CSB available in Pubmed, EBSCO, and Google Scholar between 2005 and 2018. We found nine relevant research papers. Only four of these studies directly investigated processing of erotic cues and/or rewards and reported findings related to ventral striatum activations. Three of these studies indicate increased ventral striatal reactivity for erotic stimuli, which is consistent with IST and does not support predictions based on RDS. Therefore, the current state of this data suggest that CSB is related to increased ventral striatal reactivity during the anticipation of erotic stimuli.

Neuromodulatory Systems and Their Interactions: A Review of Models, Theories, and Experiments

Neuromodulatory systems, including the noradrenergic, serotonergic, dopaminergic, and cholinergic systems, track environmental signals, such as risks, rewards, novelty, effort, and social cooperation. These systems provide a foundation for cognitive function in higher organisms; attention, emotion, goal-directed behavior, and decision-making derive from the interaction between the neuromodulatory systems and brain areas, such as the amygdala, frontal cortex, hippocampus, and sensory cortices. Given their strong influence on behavior and cognition, these systems also play a key role in disease states and are the primary target of many current treatment strategies. The fact that these systems interact with each other either directly or indirectly, however, makes it difficult to understand how a failure in one or more systems can lead to a particular symptom or pathology. In this review, we explore experimental evidence, as well as focus on computational and theoretical models of neuromodulation. Better understanding of neuromodulatory systems may lead to the development of novel treatment strategies for a number of brain disorders.

Reward deficits in behavioural variant frontotemporal dementia include insensitivity to negative stimuli

During reward processing individuals weigh positive and negative features of a stimulus to determine whether they will pursue or avoid it. Though patients with behavioural variant frontotemporal dementia display changes in their pursuit of rewards, such as food, alcohol, money, and sex, the basis for these shifts is not clearly established. In particular, it is unknown whether patients' behaviour results from excessive focus on rewards, insensitivity to punishment, or to dysfunction in a particular stage of reward processing, such as anticipation, consumption, or action selection. Our goal was to determine the nature of the reward deficit in behavioural variant frontotemporal dementia and its underlying anatomy. We devised a series of tasks involving pleasant, unpleasant, and neutral olfactory stimuli, designed to separate distinct phases of reward processing. In a group of 25 patients with behavioural variant frontotemporal dementia and 21 control subjects, diagnosis by valence interactions revealed that patients with behavioural variant frontotemporal dementia rated unpleasant odours as less aversive than did controls and displayed lower skin conductance responses when anticipating an upcoming aversive odour. Subjective pleasantness ratings and skin conductance responses did not differ between behavioural variant frontotemporal dementia and controls for pleasant or neutral smells. In a task designed to measure the effort subjects would expend to smell or avoid smelling a stimulus, patients with behavioural variant frontotemporal dementia were less motivated, and therefore less successful than control subjects, at avoiding what they preferred not to smell, but had equivalent success at obtaining stimuli they found rewarding. Voxel-based morphometry of patients with behavioural variant frontotemporal dementia revealed that the inability to subjectively differentiate the valence of pleasant and unpleasant odours correlated with atrophy in right ventral mid-insula and right amygdala. High pleasantness ratings of unpleasant stimuli correlated with left dorsal anterior insula and frontal pole atrophy. These findings indicate that insensitivity to negative information may be a key component of the reward-seeking behaviours in behavioural variant frontotemporal dementia, and may relate to degeneration of structures that are involved in representing the emotional salience of sensory information.

Family history of alcoholism and the human brain response to oral sucrose

A heightened hedonic response to sweet tastes has been associated with increased alcohol preference and alcohol consumption in both humans and animals. The principal goal of this study was to examine blood oxygenation level dependent (BOLD) activation to high- and low-concentration sweet solutions in subjects who are either positive (FHP) or negative (FHN) for a family history of alcoholism. Seventy-four non-treatment seeking, community-recruited, healthy volunteers (22.8 ± 1.6 SD years; 43% men) rated a range of sucrose concentrations in a taste test and underwent functional magnetic resonance imaging (fMRI) during oral delivery of water, 0.83 M, and 0.10 M sucrose. Sucrose compared to water produced robust activation in primary gustatory cortex, ventral insula, amygdala, and ventral striatum. FHP subjects displayed greater bilateral amygdala activation than FHN subjects in the low sucrose concentration (0.10 M). In secondary analyses, the right amygdala response to the 0.10 M sucrose was greatest in FHP women. When accounting for group differences in drinks per week, the family history groups remained significantly different in their right amygdala response to 0.10 M sucrose. Our findings suggest that the brain response to oral sucrose differs with a family history of alcoholism, and that this response to a mildly reinforcing primary reward might be an endophenotypic marker of alcoholism risk.

•

Studies in humans and animals have suggested sweet tastes as a trait correlate of alcohol risk.

•

Oral sucrose resulted in robust BOLD activation of gustatory and limbic areas.

•

Amygdala responses to 0.10 M sucrose were greatest in drinkers with family histories of alcoholism.

•

This study is first to suggest endophenotypic brain responses to sucrose in familial alcoholism.

When Habits Are Dangerous: Alcohol Expectancies and Habitual Decision Making Predict Relapse in Alcohol Dependence

BACKGROUND

Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients.

METHODS

Ninety detoxified, medication-free, alcohol-dependent patients and 96 age- and gender-matched control subjects underwent functional magnetic resonance imaging during the two-step task. Alcohol expectancies were measured with the Alcohol Expectancy Questionnaire. Over a follow-up period of 48 weeks, 37 patients remained abstinent and 53 patients relapsed as indicated by the Alcohol Timeline Followback method.

RESULTS

Patients who relapsed displayed reduced medial prefrontal cortex activation during model-based decision making. Furthermore, high alcohol expectancies were associated with low model-based control in relapsers, while the opposite was observed in abstainers and healthy control subjects. However, reduced model-based control per se was not associated with subsequent relapse.

CONCLUSIONS

These findings suggest that poor treatment outcome in alcohol dependence does not simply result from a shift from model-based to model-free control but is instead dependent on the interaction between high drug expectancies and low model-based decision making. Reduced model-based medial prefrontal cortex signatures in those who relapse point to a neural correlate of relapse risk. These observations suggest that therapeutic interventions should target subjective alcohol expectancies.

Influence of ventral tegmental area input on cortico-subcortical networks underlying action control and decision making

It is argued that the mesolimbic system has a more general function in processing all salient events, including and extending beyond rewards. Saliency was defined as an event that is unexpected due to its frequency of occurrence and elicits an attentional-behavioral switch. Using functional magnetic resonance imaging (fMRI), signals were measured in response to the modulation of salience of rewarding and nonrewarding events during a reward-based decision making task, the so called desire-reason dilemma paradigm (DRD). Replicating previous findings, both frequent and infrequent, and therefore salient, reward stimuli elicited reliable activation of the ventral tegmental area (VTA) and ventral striatum (vStr). When immediate reward desiring contradicted the superordinate task-goal, we found an increased activation of the VTA and vStr when the salient reward stimuli were presented compared to the nonsalient reward stimuli, indicating a boosting of activation in these brain regions. Furthermore, we found a significantly increased functional connectivity between the VTA and vStr, confirming the boosting of vStr activation via VTA input. Moreover, saliency per se without a reward association led to an increased activation of brain regions in the mesolimbic reward system as well as the orbitofrontal cortex (OFC), inferior frontal gyrus (IFG), and anterior cingulate cortex (ACC). Finally, findings uncovered multiple increased functional interactions between cortical saliency-processing brain areas and the VTA and vStr underlying detection and processing of salient events and adaptive decision making.

Can Pornography be Addictive? An fMRI Study of Men Seeking Treatment for Problematic Pornography Use

Pornography consumption is highly prevalent, particularly among young adult males. For some individuals, problematic pornography use (PPU) is a reason for seeking treatment. Despite the pervasiveness of pornography, PPU appears under-investigated, including with respect to the underlying neural mechanisms. Using functional magnetic resonance imaging (fMRI), we examined ventral striatal responses to erotic and monetary stimuli, disentangling cue-related 'wanting' from reward-related 'liking' among 28 heterosexual males seeking treatment for PPU and 24 heterosexual males without PPU. Subjects engaged in an incentive delay task in the scanner, in which they received erotic or monetary rewards preceded by predictive cues. Blood-oxygen-level-dependent responses to erotic and monetary cues were analyzed and examined with respect to self-reported data on sexual activity collected over the 2 preceding months. Men with and without PPU differed in their striatal responses to cues predicting erotic pictures but not in their responses to erotic pictures. PPU subjects when compared with control subjects showed increased activation of ventral striatum specifically for cues predicting erotic pictures but not for cues predicting monetary gains. Relative sensitivity to cues predicting erotic pictures vs monetary gains was significantly related to the increased behavioral motivation to view erotic images (suggestive of higher 'wanting'), severity of PPU, amount of pornography use per week, and number of weekly masturbations. Our findings suggest that, similar to what is observed in substance and gambling addictions, the neural and behavioral mechanisms associated with the anticipatory processing of cues specifically predicting erotic rewards relate importantly to clinically relevant features of PPU. These findings suggest that PPU may represent a behavioral addiction and that interventions helpful in targeting behavioral and substance addictions warrant consideration for adaptation and use in helping men with PPU.

Temporal dissociation of salience and prediction error responses to appetitive and aversive taste

The feedback-related negativity (FRN), a frontocentral ERP occurring 200-350 ms after emotionally valued outcomes, has been posited as the neural correlate of reward prediction error, a key component of associative learning. Recent evidence challenged this interpretation and has led to the suggestion that this ERP expresses salience instead. Here, we distinguish between utility prediction error and salience by delivering or withholding hedonistically matched appetitive and aversive tastes, and measure ERPs to cues signaling each taste. We observed a typical FRN (computed as the loss-minus-gain difference wave) to appetitive taste, but a reverse FRN to aversive taste. When tested axiomatically, frontocentral ERPs showed a salience response across tastes, with a particularly early response to outcome delivery, supporting recent propositions of a fast, unsigned, and unspecific response to salient stimuli. ERPs also expressed aversive prediction error peaking at 285 ms, which conformed to the logic of an axiomatic model of prediction error. With stimuli that most resemble those used in animal models, we did not detect any frontocentral ERP signal for utility prediction error, in contrast with dominant views of the functional role of the FRN ERP. We link the animal and human literature and present a challenge for current perspectives on associative learning research using ERPs.

Binge-like sucrose consumption reduces the dendritic length and complexity of principal neurons in the adolescent rat basolateral amygdala

A compelling body of evidence suggests that the worldwide obesity epidemic is underpinned by excessive sugar consumption, typified by the modern western diet. Furthermore, evidence is beginning to emerge of maladaptive changes in the mesolimbic reward pathway of the brain in relation to excess sugar consumption that highlights the importance of examining this neural circuitry in an attempt to understand and subsequently mitigate the associated morbidities with obesity. While the basolateral amygdala (BLA) has been shown to mediate the reinforcing properties of drugs of abuse, it has also been shown to play an important role in affective and motivated behaviours and has been shown to undergo maladaptive changes in response to drugs of abuse and stress. Given the overlap in neural circuitry affected by drugs of abuse and sucrose, we sought to examine the effect of short- and long-term binge-like sucrose consumption on the morphology of the BLA principal neurons using an intermittent-access two-bottle choice paradigm. We used Golgi-Cox staining to impregnate principal neurons from the BLA of short- (4 week) and long-term (12 week) sucrose consuming adolescent rats and compared these to age-matched water controls. Our results indicate possibly maladaptive changes to the dendritic architecture of BLA principal neurons, particularly on apical dendrites following long-term sucrose consumption. Specifically, our results show reduced total dendritic arbor length of BLA principal neurons following short- and long-term sucrose consumption. Additionally, we found that long-term binge-like sucrose consumption caused a significant reduction in the length and complexity of apical dendrites. Taken together, our results highlight the differences between short- and long-term binge-like sucrose consumption on BLA principal neuron morphology and are suggestive of a perturbation in the diverse synaptic inputs to these neurons.

Neural substrates of updating the prediction through prediction error during decision making

Learning of prediction error (PE), including reward PE and risk PE, is crucial for updating the prediction in reinforcement learning (RL). Neurobiological and computational models of RL have reported extensive brain activations related to PE. However, the occurrence of PE does not necessarily predict updating the prediction, e.g., in a probability-known event. Therefore, the brain regions specifically engaged in updating the prediction remain unknown. Here, we conducted two functional magnetic resonance imaging (fMRI) experiments, the probability-unknown Iowa Gambling Task (IGT) and the probability-known risk decision task (RDT). Behavioral analyses confirmed that PEs occurred in both tasks but were only used for updating the prediction in the IGT. By comparing PE-related brain activations between the two tasks, we found that the rostral anterior cingulate cortex/ventral medial prefrontal cortex (rACC/vmPFC) and the posterior cingulate cortex (PCC) activated only during the IGT and were related to both reward and risk PE. Moreover, the responses in the rACC/vmPFC and the PCC were modulated by uncertainty and were associated with reward prediction-related brain regions. Electric brain stimulation over these regions lowered the performance in the IGT but not in the RDT. Our findings of a distributed neural circuit of PE processing suggest that the rACC/vmPFC and the PCC play a key role in updating the prediction through PE processing during decision making.

New Targets for Schizophrenia Treatment beyond the Dopamine Hypothesis

Schizophrenia has been primarily associated with dopamine dysfunction, and treatments have been developed that target the dopamine pathway in the central nervous system. However, accumulating evidence has shown that the core pathophysiology of schizophrenia might involve dysfunction in dopaminergic, glutamatergic, serotonergic, and gamma-aminobutyric acid (GABA) signaling, which may lead to aberrant functioning of interneurons that manifest as cognitive, behavioral, and social dysfunction through altered functioning of a broad range of macro- and microcircuits. The interactions between neurotransmitters can be modeled as nodes and edges by using graph theory, and oxidative balance, immune, and glutamatergic systems may represent multiple nodes interlocking at a central hub; imbalance within any of these nodes might affect the entire system. Therefore, this review attempts to address novel treatment targets beyond the dopamine hypothesis, including glutamate, serotonin, acetylcholine, GABA, and inflammatory cytokines. Furthermore, we outline that these treatment targets can be possibly integrated with novel treatment strategies aimed at different symptoms or phases of the illness. We anticipate that reversing anomalous activity in these novel treatment targets or combinations between these strategies might be beneficial in the treatment of schizophrenia.

Virtual Personalities: Using Computational Modeling to Understand Within-Person Variability

How can the same underlying psychological/neurobiological system result in both stable between-individual differences and high levels of within-individual variability in personality states over time and situations? We argue that both types of variability result from a psychological system based on structured, chronic motivations, where behavior at a specific point in time is a joint function of the current availability of motive affordances in the situation, current motivationally relevant bodily or interoceptive states, and the result of the competition among alternative active motives. Here we present a biologically-based theoretical framework, embodied in two different computational models, that shows how individuals with stable personality characteristics, can nevertheless exhibit considerable within-person variability in personality states across time and situations.

Long-lasting contribution of dopamine in the nucleus accumbens core, but not dorsal lateral striatum, to sign-tracking

The attribution of incentive salience to reward-paired cues is dependent on dopamine release in the nucleus accumbens core (NAcC). These dopamine signals conform to traditional reward-prediction error signals and have been shown to diminish with time. Here we examined whether the diminishing dopamine signal in the NAcC has functional implications for the expression of sign-tracking, a Pavlovian conditioned response indicative of the attribution of incentive salience to reward-paired cues. Food-restricted male Sprague Dawley rats were trained in a Pavlovian paradigm in which an insertable lever predicted delivery of food reward in a nearby food cup. After 7 or 14 training sessions, rats received infusions of saline, the dopamine antagonist flupenthixol, or the GABA agonists baclofen and muscimol into the NAcC or the dorsal lateral striatum (DLS). Dopamine antagonism within the NAcC attenuated sign-tracking, whereas reversible inactivation did not affect sign-tracking but increased non-specific food cup checking behaviors. Neither drug in the DLS affected sign-tracking behavior. Critically, extended training did not alter these effects. Although extended experience with an incentive stimulus may reduce cue-evoked dopamine in the NAcC, this does not remove the dependence on dopamine in this region to promote Pavlovian cue approach nor result in the recruitment of dorsal lateral striatal systems for this behavior. These data support the notion that dopamine within the mesoaccumbal system, but not the nigrostriatal system, contributes critically to incentive motivational processes independent of the length of training.

Imaging the Nonmotor Symptoms in Parkinson's Disease

Parkinson's disease is acknowledged to be a multisystem syndrome, manifesting as a result of multineuropeptide dysfunction, including dopaminergic, cholinergic, serotonergic, and noradrenergic deficits. This multisystem disorder ultimately leads to the presentation of a range of nonmotor symptoms, now appreciated to be an integral part of the disease-specific spectrum of symptoms, often preceding the diagnosis of motor Parkinson's disease. In this chapter, we review the dopaminergic and nondopaminergic basis of these symptoms by exploring the neuroimaging evidence based on several techniques including positron emission tomography, single-photon emission computed tomography molecular imaging, magnetic resonance imaging, functional magnetic resonance imaging, and diffusion tensor imaging. We discuss the role of these neuroimaging techniques in elucidating the underlying pathophysiology of NMS in Parkinson's disease.

Reward Prediction Errors in Drug Addiction and Parkinson's Disease: from Neurophysiology to Neuroimaging

PURPOSE OF REVIEW

Surprises are important sources of learning. Cognitive scientists often refer to surprises as "reward prediction errors," a parameter that captures discrepancies between expectations and actual outcomes. Here, we integrate neurophysiological and functional magnetic resonance imaging (fMRI) results addressing the processing of reward prediction errors and how they might be altered in drug addiction and Parkinson's disease.

RECENT FINDINGS

By increasing phasic dopamine responses, drugs might accentuate prediction error signals, causing increases in fMRI activity in mesolimbic areas in response to drugs. Chronic substance dependence, by contrast, has been linked with compromised dopaminergic function, which might be associated with blunted fMRI responses to pleasant non-drug stimuli in mesocorticolimbic areas. In Parkinson's disease, dopamine replacement therapies seem to induce impairments in learning from negative outcomes. The present review provides a holistic overview of reward prediction errors across different pathologies and might inform future clinical strategies targeting impulsive/compulsive disorders.

Within-session decrement of the emission of licking bursts following reward devaluation in rats licking for sucrose

We previously observed that dopamine D2-like receptor blockade in rats licking for sucrose produced a within-session decrement of the emission of licking bursts similar to the effect of either reward devaluation, or neuroleptics, on operant responding for different rewards, which, accordingly, we interpreted as an extinction-like effect. This implies that exposing animals to reward devaluation would result in a drop of burst number taking place only after the contact with the devalued reward. To test this prediction, we compared the difference in the within-session time course of burst number in response to high (10%) versus low (2%) concentration sucrose solutions, either in a condition of reward devaluation (exposure to 2% after daily 10%), or in a condition which does not involve changes in the reward value (two groups of subjects each repeatedly exposed to only one of the two concentrations). Reward devaluation resulted in a within-session decrement of the burst number, with the response rate dropping only after the contact with the devalued reward, as predicted. This response pattern was reliably observed only in subjects at their first devaluation experience. In contrast, exposure of separate groups of animals to the two different concentrations yielded lower levels of burst number in the low concentration group apparent since the beginning of the session, as previously observed with dopamine D1-like receptor blockade. These results show that the analysis of burst number, but not of burst size, reveals a specific activation pattern in response to reward devaluation, which differs from the pattern observed comparing the response to two different sucrose concentrations in separate groups of subjects, i.e. in a condition not involving reward devaluation. Finally, the characterisation of the experimental measures of the analysis of licking microstructure in behaviourally (and psychologically) meaningful functional terms, might be relevant for the investigation of the mechanisms underlying behavioural activation and the related evaluation processes.

The dopamine hypothesis of bipolar affective disorder: the state of the art and implications for treatment

Bipolar affective disorder is a common neuropsychiatric disorder. Although its neurobiological underpinnings are incompletely understood, the dopamine hypothesis has been a key theory of the pathophysiology of both manic and depressive phases of the illness for over four decades. The increased use of antidopaminergics in the treatment of this disorder and new in vivo neuroimaging and post-mortem studies makes it timely to review this theory. To do this, we conducted a systematic search for post-mortem, pharmacological, functional magnetic resonance and molecular imaging studies of dopamine function in bipolar disorder. Converging findings from pharmacological and imaging studies support the hypothesis that a state of hyperdopaminergia, specifically elevations in D2/3 receptor availability and a hyperactive reward processing network, underlies mania. In bipolar depression imaging studies show increased dopamine transporter levels, but changes in other aspects of dopaminergic function are inconsistent. Puzzlingly, pharmacological evidence shows that both dopamine agonists and antidopaminergics can improve bipolar depressive symptoms and perhaps actions at other receptors may reconcile these findings. Tentatively, this evidence suggests a model where an elevation in striatal D2/3 receptor availability would lead to increased dopaminergic neurotransmission and mania, whilst increased striatal dopamine transporter (DAT) levels would lead to reduced dopaminergic function and depression. Thus, it can be speculated that a failure of dopamine receptor and transporter homoeostasis might underlie the pathophysiology of this disorder. The limitations of this model include its reliance on pharmacological evidence, as these studies could potentially affect other monoamines, and the scarcity of imaging evidence on dopaminergic function. This model, if confirmed, has implications for developing new treatment strategies such as reducing the dopamine synthesis and/or release in mania and DAT blockade in bipolar depression.

Liking, wanting, and the incentive-sensitization theory of addiction

Rewards are both "liked" and "wanted," and those 2 words seem almost interchangeable. However, the brain circuitry that mediates the psychological process of "wanting" a particular reward is dissociable from circuitry that mediates the degree to which it is "liked." Incentive salience or "wanting," a form of motivation, is generated by large and robust neural systems that include mesolimbic dopamine. By comparison, "liking," or the actual pleasurable impact of reward consumption, is mediated by smaller and fragile neural systems, and is not dependent on dopamine. The incentive-sensitization theory posits the essence of drug addiction to be excessive amplification specifically of psychological "wanting," especially triggered by cues, without necessarily an amplification of "liking." This is because of long-lasting changes in dopamine-related motivation systems of susceptible individuals, called "neural sensitization." A quarter-century after its proposal, evidence has continued to grow in support the incentive-sensitization theory. Further, its scope is now expanding to include diverse behavioral addictions and other psychopathologies. (PsycINFO Database Record

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning

Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto’s (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.

An Integrative Perspective on the Role of Dopamine in Schizophrenia

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing. Increased spontaneous phasic dopamine release helps to explain positive symptoms and provides a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with positive symptoms: aberrant learning for neutral cues (assessed with behavioral and autonomic responses), and aberrant, increased activation of the ventral striatum, midbrain, and other limbic regions for neutral cues, neutral outcomes, and neutral prediction errors. Taken together, then, these two disturbances explain many findings in schizophrenia. We review evidence supporting their co-occurrence and consider their differential implications for the treatment of positive and negative symptoms.

Is Addiction a Brain Disease?

Where does normal brain or psychological function end, and pathology begin? The line can be hard to discern, making disease sometimes a tricky word. In addiction, normal 'wanting' processes become distorted and excessive, according to the incentive-sensitization theory. Excessive 'wanting' results from drug-induced neural sensitization changes in underlying brain mesolimbic systems of incentive. 'Brain disease' was never used by the theory, but neural sensitization changes are arguably extreme enough and problematic enough to be called pathological. This implies that 'brain disease' can be a legitimate description of addiction, though caveats are needed to acknowledge roles for choice and active agency by the addict. Finally, arguments over 'brain disease' should be put behind us. Our real challenge is to understand addiction and devise better ways to help. Arguments over descriptive words only distract from that challenge.

The behavioral approach system and augmenting/reducing in auditory event-related potentials during emotional visual stimulation

In the recent Reinforcement Sensitivity Theory Personality Questionnaire (RST-PQ, Corr and Cooper, 2016) the behavioral approach system (BAS) has been conceptualized as multidimensional in which facets of reward interest and reactivity, and goal-drive persistence, are separate from impulsivity. Aim of the present work was to highlight the predictive power of BAS and its facets in differentiating electrocortical responses by using an auditory augmenting/reducing event-related potential (ERP) paradigm during emotional visual stimulation. ERPs were recorded for 5 levels of intensity in 39 women. The RST-PQ was used to measure the total BAS (T-BAS) and its four facets of Goal-Drive Persistence (GDP), Reward Interest (RI), Reward Reactivity (RR), and Impulsivity (IMP). T-BAS and RI, and to a less extent GDP and RR, were significantly associated with higher N1/P2 amplitudes at central sites (C3, Cz, C4) across neutral, positive and negative slides. Similar, but less pronounced relations were found for GDP and RR, but this relation was lacking for Imp facet. In addition, N1/P2 slope at central sites was positively correlated with T-BAS, GDP, RI, RR, but not Imp. Indeed, T-BAS facets failed to maintain a significant correlation with N1/P2 slope, after controlling for T-BAS residual scores, indicating that T-BAS drives these significant correlations. LORETA analysis at 219ms (P2 wave) from tone onset revealed a significant activation of the right inferior parietal lobule (IPL, BA40) and left anterior cingulate gyrus (BA32) in high T-BAS compared to low T-BAS participants. Results are discussed within a revised RST framework differentiating reward components from impulsivity.

Reassessing wanting and liking in the study of mesolimbic influence on food intake

Humans and animals such as rats and mice tend to overconsume calorie-dense foods, a phenomenon that likely contributes to obesity. One often-advanced explanation for why we preferentially consume sweet and fatty foods is that they are more "rewarding" than low-calorie foods. "Reward" has been subdivided into three interdependent psychological processes: hedonia (liking a food), reinforcement (formation of associations among stimuli, actions, and/or the food), and motivation (wanting the food). Research into these processes has focused on the mesolimbic system, which comprises both dopamine neurons in the ventral tegmental area and neurons in their major projection target, the nucleus accumbens. The mesolimbic system and closely connected structures are commonly referred to as the brain's "reward circuit." Implicit in this title is the assumption that "rewarding" experiences are generally the result of activity in this circuit. In this review, I argue that food intake and the preference for calorie-dense foods can be explained without reference to subjective emotions. Furthermore, the contribution of mesolimbic dopamine to food intake and preference may not be a general one of promoting or coordinating behaviors that result in the most reward or caloric intake but may instead be limited to the facilitation of a specific form of neural computation that results in conditioned approach behavior. Studies on the neural mechanisms of caloric intake regulation must address how sensory information about calorie intake affects not just the mesolimbic system but also many other forms of computation that govern other types of food-seeking and food-oriented behaviors.

Terminal Dopamine Release Kinetics in the Accumbens Core and Shell Are Distinctly Altered after Withdrawal from Cocaine Self-Administration

Repeated self-administration of cocaine is associated with impairments in motivated behaviors as well as alterations in both dopamine (DA) release and neural signaling within the nucleus accumbens (NAc). These impairments are present even after several weeks of abstinence from drug taking, suggesting that the self-administration experience induces long-lasting neuroplastic alterations in the mesolimbic DA circuit. To understand these changes at the terminal level, rats were allowed to self-administer either cocaine intravenously (∼1 mg/kg per infusion) or water to a receptacle (control) in 2-h sessions over 14 days, followed by 30 days of enforced abstinence. Fast-scan cyclic voltammetry was used to record real-time DA release in either NAc core or shell after electrical stimulations of the ventral tegmental area (VTA) in freely-moving animals. In controls, the kinetics of DA release in the core and shell strikingly differed, with shell displaying slower release and reuptake rates than core. However, cocaine experience differentially altered these signaling patterns by NAc subregion. In the shell, cocaine rats showed less sensitivity to the dynamic range of applied stimulations than controls. In the core, by contrast, cocaine rats displayed robustly reduced peak DA release given the same stimulation, while also showing slower release and reuptake kinetics. The differential effects of cocaine self-administration on terminal function between core and shell is consistent with a region-specific functional reorganization of the mesolimbic DA system after repeated exposure and may provide an anatomical substrate for altered cognitive function after chronic drug-taking and addiction.

What is abnormal about addiction-related attentional biases?

BACKGROUND

The phenotype of addiction includes prominent attentional biases for drug cues, which play a role in motivating drug-seeking behavior and contribute to relapse. In a separate line of research, arbitrary stimuli have been shown to automatically capture attention when previously associated with reward in non-clinical samples.

METHODS AND RESULTS

Here, I argue that these two attentional biases reflect the same cognitive process. I outline five characteristics that exemplify attentional biases for drug cues: resistant to conflicting goals, robust to extinction, linked to dorsal striatal dopamine and to biases in approach behavior, and can distinguish between individuals with and without a history of drug dependence. I then go on to describe how attentional biases for arbitrary reward-associated stimuli share all of these features, and conclude by arguing that the attentional components of addiction reflect a normal cognitive process that promotes reward-seeking behavior.

Models of neuromodulation for computational psychiatry

Psychiatry faces fundamental challenges: based on a syndrome-based nosology, it presently lacks clinical tests to infer on disease processes that cause symptoms of individual patients and must resort to trial-and-error treatment strategies. These challenges have fueled the recent emergence of a novel field-computational psychiatry-that strives for mathematical models of disease processes at physiological and computational (information processing) levels. This review is motivated by one particular goal of computational psychiatry: the development of 'computational assays' that can be applied to behavioral or neuroimaging data from individual patients and support differential diagnosis and guiding patient-specific treatment. Because the majority of available pharmacotherapeutic approaches in psychiatry target neuromodulatory transmitters, models that infer (patho)physiological and (patho)computational actions of different neuromodulatory transmitters are of central interest for computational psychiatry. This article reviews the (many) outstanding questions on the computational roles of neuromodulators (dopamine, acetylcholine, serotonin, and noradrenaline), outlines available evidence, and discusses promises and pitfalls in translating these findings to clinical applications. WIREs Cogn Sci 2017, 8:e1420. doi: 10.1002/wcs.1420 For further resources related to this article, please visit the WIREs website.

Modelling ADHD: A review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning

Attention deficit hyperactivity disorder (ADHD) is characterized by altered decision-making (DM) and reinforcement learning (RL), for which competing theories propose alternative explanations. Computational modelling contributes to understanding DM and RL by integrating behavioural and neurobiological findings, and could elucidate pathogenic mechanisms behind ADHD. This review of neurobiological theories of ADHD describes predictions for the effect of ADHD on DM and RL as described by the drift-diffusion model of DM (DDM) and a basic RL model. Empirical studies employing these models are also reviewed. While theories often agree on how ADHD should be reflected in model parameters, each theory implies a unique combination of predictions. Empirical studies agree with the theories' assumptions of a lowered DDM drift rate in ADHD, while findings are less conclusive for boundary separation. The few studies employing RL models support a lower choice sensitivity in ADHD, but not an altered learning rate. The discussion outlines research areas for further theoretical refinement in the ADHD field.

Visual Sexual Stimuli-Cue or Reward? A Perspective for Interpreting Brain Imaging Findings on Human Sexual Behaviors

There is an increasing number of neuroimaging studies using visual sexual stimuli (VSS), especially within the emerging field of research on compulsive sexual behaviors (CSB). A central question in this field is whether behaviors such as excessive pornography consumption share common brain mechanisms with widely studied substance and behavioral addictions. Depending on how VSS are conceptualized, different predictions can be formulated within the frameworks of Reinforcement Learning or Incentive Salience Theory, where a crucial distinction is made between conditioned and unconditioned stimuli (related to reward anticipation vs. reward consumption, respectively). Surveying 40 recent human neuroimaging studies we show existing ambiguity about the conceptualization of VSS. Therefore, we feel that it is important to address the question of whether VSS should be considered as conditioned stimuli (cue) or unconditioned stimuli (reward). Here we present our own perspective, which is that in most laboratory settings VSS play a role of reward, as evidenced by: (1) experience of pleasure while watching VSS, possibly accompanied by genital reaction; (2) reward-related brain activity correlated with these pleasurable feelings in response to VSS; (3) a willingness to exert effort to view VSS similarly as for other rewarding stimuli such as money; and (4) conditioning for cues predictive of VSS. We hope that this perspective article will initiate a scientific discussion on this important and overlooked topic and increase attention for appropriate interpretations of results of human neuroimaging studies using VSS.

Neural mechanisms of goal-contingent task disengagement: Response-irrelevant stimuli activate the default mode network

As we experience the world, we must decide not only when and how to act based on input from the environment, but also when to avoid responding in situations where acting could lead to a detrimental outcome. The ability to regulate behavior in this way requires flexible cognitive control, as the same stimulus may call for a response in one context but not in another. In this sense, explicit non-responding can be characterized as an active, goal-directed cognitive process. Little is known about the mechanisms by which a currently active goal state modulates information processing to support the avoidance of undesired responding. In the present study, participants executed or withheld responses to a color target based whether its color matched that of a cue at the beginning of each trial. Behavioral and neural responses to task-irrelevant stimuli appearing as distractors were examined as a function of their relationship to the currently response-relevant color indicated by the cue. We observed a robust pattern in which stimuli possessing the currently response-irrelevant feature activate the default mode network (DMN), which was associated with a behavioral cost on trials in which this stimulus competed with a response-relevant target. Our findings reveal a role for the DMN in goal-directed cognitive control, facilitating active disengagement based on contextually-specific task demands.

Dopamine antagonism does not impair learning of Pavlovian conditioned approach to manipulable or non-manipulable cues but biases responding towards goal tracking

Dopamine's (DA) role in reward-processing is currently discussed as either providing a teaching signal to guide learning or mediating the transfer of incentive salience (i.e. motivational aspects) from unconditioned stimuli (US) to conditioned stimuli (CS). We used a Pavlovian conditioned approach (PCA) procedure to further investigate DAs contribution to these processes. Experiment 1 assessed the acquisition of PCA to a manipulable lever cue for 7days under DA-blockade with Flupenthixol (FLU; 225μg/kg) or Saline (SAL) treatment, followed by 6-days off-drug testing. FLU decreased the number of conditioned responses (CR) during the treatment phase, but cessation of treatment resulted in an immediate increase in CR to levels comparable to SAL controls; notably, CR in FLU-treated rats were restricted to goal tracking behaviour. During continued off-drug testing, rats from the FLU group developed sign tracking with a similar temporal pattern as controls. In experiment 2, acquisition of PCA to a non-manipulable auditory cue was investigated. FLU reduced the number of CR during treatment, and removing DA antagonism resulted in a similar rapid increase of CR as seen in experiment 1. These data complement other reports by demonstrating that, independently from the physical properties of the CS, DA is not required for learning predictive aspects of a CS-US relationship but for the development of behaviour (namely sign tracking) which is based on the motivational aspects of a CS-US relationship.

Activation of Dopamine Receptors in the Nucleus Accumbens Promotes Sucrose-Reinforced Cued Approach Behavior

Dopamine receptor activation in the nucleus accumbens (NAc) promotes vigorous environmentally-cued food-seeking in hungry rats. Rats fed ad libitum, however, respond to fewer food-predictive cues, particularly when the value of food reward is low. Here, we investigated whether this difference could be due to differences in the degree of dopamine receptor activation in the NAc. First, we observed that although rats given ad libitum access to chow in their home cages approached a food receptacle in response to reward-predictive cues, the number of such approaches declined as animals accumulated food rewards. Intriguingly, cued approach to food occurred in clusters, with several cued responses followed by successive non-responses. This pattern suggested that behavior was dictated by transitions between two states, responsive and non-responsive. Injection of D1 or D2 dopamine receptor agonists into the NAc dose-dependently increased cue responding by promoting transitions to the responsive state and by preventing transitions to the non-responsive state. In contrast, antagonists of either D1 or D2 receptors promoted long bouts of non-responding by inducing transitions to the non-responsive state and by preventing transitions to the responsive state. Moreover, locomotor behavior during the inter-trial interval was correlated with the responsive state, and was also increased by dopamine receptor agonists. These results suggest that activation of NAc dopamine receptors plays an important role in regulating the probability of approach to food under conditions of normative satiety.

The functional logic of corticostriatal connections

Unidirectional connections from the cortex to the matrix of the corpus striatum initiate the cortico-basal ganglia (BG)-thalamocortical loop, thought to be important in momentary action selection and in longer-term fine tuning of behavioural repertoire; a discrete set of striatal compartments, striosomes, has the complementary role of registering or anticipating reward that shapes corticostriatal plasticity. Re-entrant signals traversing the cortico-BG loop impact predominantly frontal cortices, conveyed through topographically ordered output channels; by contrast, striatal input signals originate from a far broader span of cortex, and are far more divergent in their termination. The term 'disclosed loop' is introduced to describe this organisation: a closed circuit that is open to outside influence at the initial stage of cortical input. The closed circuit component of corticostriatal afferents is newly dubbed 'operative', as it is proposed to establish the bid for action selection on the part of an incipient cortical action plan; the broader set of converging corticostriatal afferents is described as contextual. A corollary of this proposal is that every unit of the striatal volume, including the long, C-shaped tail of the caudate nucleus, should receive a mandatory component of operative input, and hence include at least one area of BG-recipient cortex amongst the sources of its corticostriatal afferents. Individual operative afferents contact twin classes of GABAergic striatal projection neuron (SPN), distinguished by their neurochemical character, and onward circuitry. This is the basis of the classic direct and indirect pathway model of the cortico-BG loop. Each pathway utilises a serial chain of inhibition, with two such links, or three, providing positive and negative feedback, respectively. Operative co-activation of direct and indirect SPNs is, therefore, pictured to simultaneously promote action, and to restrain it. The balance of this rival activity is determined by the contextual inputs, which summarise the external and internal sensory environment, and the state of ongoing behavioural priorities. Notably, the distributed sources of contextual convergence upon a striatal locus mirror the transcortical network harnessed by the origin of the operative input to that locus, thereby capturing a similar set of contingencies relevant to determining action. The disclosed loop formulation of corticostriatal and subsequent BG loop circuitry, as advanced here, refines the operating rationale of the classic model and allows the integration of more recent anatomical and physiological data, some of which can appear at variance with the classic model. Equally, it provides a lucid functional context for continuing cellular studies of SPN biophysics and mechanisms of synaptic plasticity.

Rats value time differently on equivalent foraging and delay-discounting tasks

All organisms have to consider consequences that vary through time. Theories explaining how animals handle intertemporal choice include delay-discounting models, in which the value of future rewards is discounted by the delay until receipt, and foraging models, which predict that decision-makers maximize rate of reward. We measured the behavior of rats on a 2-option delay-discounting task and a stay/go foraging task that were equivalent for rate of reward and physical demand. Despite the highly shared features of the tasks, rats were willing to wait much longer on the foraging task than on the delay-discounting task. Moreover, choice performance by rats was less optimal in terms of total reward received on the foraging task compared to the delay-discounting task. We applied a suite of intertemporal choice models to the data but found that we needed a novel model incorporating interactions of decision-making systems to successfully explain behavior. Our findings (a) highlight the importance of factors that historically have been seen as irrelevant and (b) indicate the inadequacy of current general theories of intertemporal choice. (PsycINFO Database Record