Conflicted between Goal-Directed and Habitual Control, an fMRI Investigation

“Slips of action” occur in everyday life when we momentarily lose sight of a goal (for example, when in a rush or distracted). Associative models propose that these habitual responses can be activated via a direct stimulus-response (S-R) mechanism, regardless of the current hedonic value of the outcome. The slips-of-action task (SOAT) has been extensively used in both healthy and pathological populations to measure habit tendencies, the likelihood of making erroneous responses for devalued outcomes. Inspection of behavioral performance does not reveal, however, whether the impairments were due to impaired goal-directed control or aberrantly strong habit formation. In the current study, we used functional MRI while human participants performed both the instrumental training and SOAT test phases, to elucidate the relative contributions of these mechanisms to performance on the SOAT. On trials in which conflict arises between competing goal-directed and habitual responses, we observed increased activation across areas including the anterior cingulate cortex, paracingulate gyrus, lateral orbitofrontal cortex (OFC), insula, and inferior frontal gyrus (IFG). Responding for devalued outcomes was related to increased activation in the premotor cortex and cerebellum, implicating these regions in habitual responding. Increased activation in the caudate, dorsolateral prefrontal cortex (dlPFC), and frontal pole during training was associated with better performance during the test phase, indicative of goal-directed action control. These results endorse interpretation of the SOAT in terms of competing goal-directed and habitual mechanisms and highlight that cognitive control processes present an additional bottleneck for successful performance on this task.

Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior

Goal-directed actions are instrumental behaviors whose performance depends on the organism's knowledge of the reinforcing outcome's value. In contrast, habits are instrumental behaviors that are insensitive to the outcome's current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random-interval 30-s schedule in the presence of a discriminative stimulus (S) but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the law of effect) and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Transition to Adulthood as a Joint Parent-Youth Project for Young Persons With Intellectual and Developmental Disabilities

Eight dyads ( N = 16) residing in Western Canada participated in this investigation of how young adults with intellectual and developmental disabilities (IDD) and their parents jointly construct, articulate, and act on goals pertinent to the young adults' transition to adulthood. Using the action-project method to collect and analyze conversations and video recall data, cases were grouped representing the ways goal-directed projects brought relationship ( n = 4), planning ( n = 3) or both ( n = 1) to the foreground as joint projects. Resources internal to the dyad such as emotional resources, and external to the dyad, facilitated formulation and pursuit of projects. Lack of external supports and limited parental knowledge about IDD hindered joint project formulation.

From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

Shifting the balance between goals and habits: Five failures in experimental habit induction

Habits are repetitive behaviors that become ingrained with practice, routine, and repetition. The more we repeat an action, the stronger our habits become. Behavioral and clinical neuroscientists have become increasingly interested in this topic because habits may contribute to aspects of maladaptive human behavior, such as compulsive behavior in psychiatry. Numerous studies have demonstrated that habits can be induced in otherwise healthy rats by simply overtraining stimulus-response behaviors. However, despite growing interest in this topic and its application to psychiatry, a similar body of work in humans is absent. Only a single study has been published in humans that shows the effect of extensive training on habit expression. Here, we report five failed attempts to demonstrate that overtraining instrumental behavior leads to the development of inflexible habits in humans, using variants of four previously published outcome devaluation paradigms. Extensive training did not lead to greater habits in two versions of an avoidance learning task, in an appetitive slips-of-action task, or in two independent attempts to replicate the original demonstration. The finding that these outcome devaluation procedures may be insensitive to duration of stimulus-response training in humans has implications for prior work in psychiatric populations. Specifically, it converges with the suggestion that the failures in outcome devaluation in compulsive individuals reflect dysfunction in goal-directed control, rather than overactive habit learning. We discuss why habits are difficult to experimentally induce in healthy humans, and the implications of this for future research in healthy and disordered populations. (PsycINFO Database Record

The Bilateral Prefronto-striatal Pathway Is Necessary for Learning New Goal-Directed Actions

The acquisition of new goal-directed actions requires the encoding of action-outcome associations. At a neural level, this encoding has been hypothesized to involve a prefronto-striatal circuit extending between the prelimbic cortex (PL) and the posterior dorsomedial striatum (pDMS); however, no research identifying this pathway with any precision has been reported. We started by mapping the prelimbic input to the dorsal and ventral striatum using a combination of retrograde and anterograde tracing with CLARITY and established that PL-pDMS projections share some overlap with projections to the nucleus accumbens core (NAc) in rats. We then tested whether each of these pathways were functionally required for goal-directed learning; we used a pathway-specific dual-virus chemogenetic approach to selectively silence pDMS-projecting or NAc-projecting PL neurons during instrumental training and tested rats for goal-directed action. We found that silencing PL-pDMS projections abolished goal-directed learning, whereas silencing PL-NAc projections left goal-directed learning intact. Finally, we used a three-virus approach to silence bilateral and contralateral pDMS-projecting PL neurons and again blocked goal-directed learning. These results establish that the acquisition of new goal-directed actions depends on the bilateral PL-pDMS pathway driven by intratelencephalic cortical neurons.

The response strategy and the place strategy in a plus-maze have different sensitivities to devaluation of expected outcome

Previous studies have suggested that spatial navigation can be achieved with at least two distinct learning processes, involving either cognitive map‐like representations of the local environment, referred to as the “place strategy”, or simple stimulus‐response (S‐R) associations, the “response strategy”. A similar distinction between cognitive/behavioral processes has been made in the context of non‐spatial, instrumental conditioning, with the definition of two processes concerning the sensitivity of a given behavior to the expected value of its outcome as well as to the response‐outcome contingency (“goal‐directed action” and “S‐R habit”). Here we investigated whether these two versions of dichotomist definitions of learned behavior, one spatial and the other non‐spatial, correspond to each other in a formal way. Specifically, we assessed the goal‐directed nature of two navigational strategies, using a combination of an outcome devaluation procedure and a spatial probe trial frequently used to dissociate the two navigational strategies. In Experiment 1, rats trained in a dual‐solution T‐maze task were subjected to an extinction probe trial from the opposite start arm, with or without prefeeding‐induced devaluation of the expected outcome. We found that a non‐significant preference for the place strategy in the non‐devalued condition was completely reversed after devaluation, such that significantly more animals displayed the use of the response strategy. The result suggests that the place strategy is sensitive to the expected value of the outcome, while the response strategy is not. In Experiment 2, rats with hippocampal lesions showed significant reliance on the response strategy, regardless of whether the expected outcome was devalued or not. The result thus offers further evidence that the response strategy conforms to the definition of an outcome‐insensitive, habitual form of instrumental behavior. These results together attest a formal correspondence between two types of dual‐process accounts of animal learning and behavior.

Reaching Into the Unknown: Actions, Goal Hierarchies, and Explorative Agency

Action is widely characterized as possessing a teleological dimension. The dominant way of describing goal-directed action and agency is in terms of exploitation, i.e., pursuing pre-specified goals using existing strategies. Recent theoretical developments emphasize the place of exploration, i.e., discovering new goals or acquiring new strategies. The exploitation-exploration distinction poses questions with regard to goals and agency: Should exploration, as some authors have suggested, be regarded as acting without a goal? We argue that recognizing the hierarchical nature of goals is crucial in distinguishing the two kinds of activity, because this recognition prevents the claim that exploration is goal-free, while allowing for a homogeneous account of both exploitative and explorative actions. An action typically causes relatively low-level/proximal (i.e., sensorimotor, immediate) and relatively high-level/distal (i.e., in the environment, at a wider timescale) outcomes. In exploitation, one relies on existing associations between low- and high-level states, whereas in exploration one does not have the ability or intention to control high-level/distal states. We argue that explorative action entails the capacity to exercise control within the low-level/proximal states, which enables the pursuit of indeterminate goals at the higher levels of a goal hierarchy, and the possibility of acquiring new goals and reorganization of goal hierarchies. We consider how the dominant models of agency might accommodate this capacity for explorative action.

Observation of Point-Light-Walker Locomotion Induces Motor Resonance When Explicitly Represented; An EEG Source Analysis Study

Understanding human motion, to infer the goal of others' actions, is thought to involve the observer's motor repertoire. One prominent class of actions, the human locomotion, has been object of several studies, all focused on manipulating the shape of degraded human figures like point-light walker (PLW) stimuli, represented as walking on the spot. Nevertheless, since the main goal of the locomotor function is to displace the whole body from one position to the other, these stimuli might not fully represent a goal-directed action and thus might not be able to induce the same motor resonance mechanism expected when observing a natural locomotion. To explore this hypothesis, we recorded the event-related potentials (ERP) of canonical/scrambled and translating/centered PLWs decoding. We individuated a novel ERP component (N2c) over central electrodes, around 435 ms after stimulus onset, for translating compared to centered PLW, only when the canonical shape was preserved. Consistently with our hypothesis, sources analysis associated this component to the activation of trunk and lower legs primary sensory-motor and supplementary motor areas. These results confirm the role of own motor repertoire in processing human action and suggest that ERP can detect the associated motor resonance only when the human figure is explicitly involved in performing a meaningful action.

Prefrontal Corticostriatal Disconnection Blocks the Acquisition of Goal-Directed Action

The acquisition of goal-directed action requires encoding of the association between an action and its specific consequences or outcome. At a neural level, this encoding has been hypothesized to involve a prefrontal corticostriatal circuit involving the projection from the prelimbic cortex (PL) to the posterior dorsomedial striatum (pDMS); however, no direct evidence for this claim has been reported. In a series of experiments, we performed functional disconnection of this pathway using targeted lesions of the anterior corpus callosum to disrupt contralateral corticostriatal projections with asymmetrical lesions of the PL and/or pDMS to block plasticity in this circuit in rats. We first demonstrated that unilaterally blocking the PL input to the pDMS prevented the phosphorylation of extracellular signal-related kinase/mitogen activated protein kinase (pERK/pMAPK) induced by instrumental training. Next, we used a full bilateral disconnection of the PL from the pDMS and assessed goal-directed action using an outcome-devaluation test. Importantly, we found evidence that rats maintaining an ipsilateral and/or contralateral connection between the PL and the pDMS were able to acquire goal-directed actions. In contrast, bilateral PL-pDMS disconnection abolished the acquisition of goal-directed actions. Finally, we used a temporary pharmacological disconnection to disrupt PL inputs to the pDMS by infusing the NMDA antagonist dl-2-amino-5-phosphonopentanoic acid into the pDMS during instrumental training and found that this manipulation also disrupted goal-directed learning. These results establish that, in rats, the acquisition of new goal-directed actions depends on a prefrontal-corticostriatal circuit involving a connection between the PL and the pDMS.SIGNIFICANCE STATEMENT It has been hypothesized that the prelimbic cortex (PL) and posterior dorsomedial striatum (pDMS) in rodents interact in a corticostriatal circuit to mediate goal-directed learning. However, no direct evidence supporting this claim has been reported. Using targeted lesions, we performed functional disconnection of the PL-pDMS pathway to assess its role in goal-directed learning. In the first experiment, we demonstrated that PL input to the pDMS is necessary for instrumental training-induced neuronal activity. Next, we disrupted ipsilateral, contralateral, or bilateral PL-pDMS connections and found that only bilateral PL-pDMS disconnection disrupted the acquisition of goal-directed actions, a finding we replicated in our final study using a pharmacological disconnection procedure.

Sleep Deprivation Promotes Habitual Control over Goal-Directed Control: Behavioral and Neuroimaging Evidence

Sleep is one of the most fundamental processes of life, playing an important role in the regulation of brain function. The long-term lack of sleep can cause memory impairments, declines in learning ability, and executive dysfunction. In the present study, we evaluated the effects of sleep deprivation on instrumental learning behavior, particularly goal-directed and habitual actions in humans, and investigated the underlying neural mechanisms. Healthy college students of either gender were enrolled and randomly divided into sleep deprivation group and sleep control group. fMRI data were collected. We found that one night of sleep deprivation led to greater responsiveness to stimuli that were associated with devalued outcomes in the slips-of-action test, indicating a deficit in the formation of goal-directed control and an overreliance on habits. Furthermore, sleep deprivation had no effect on the expression of acquired goal-directed action. The level of goal-directed action after sleep deprivation was positively correlated with baseline working memory capacity. The neuroimaging data indicated that goal-directed learning mainly recruited the ventromedial PFC (vmPFC), the activation of which was less pronounced during goal-directed learning after sleep deprivation. Activation of the vmPFC during goal-directed learning during training was positively correlated with the level of goal-directed action performance. The present study suggests that people rely predominantly on habits at the expense of goal-directed control after sleep deprivation, and this process involves the vmPFC. These results contribute to a better understanding of the effects of sleep loss on decision-making.SIGNIFICANCE STATEMENT Understanding the cognitive consequences of sleep deprivation has become extremely important over the past half century, given the continued decline in sleep duration in industrialized societies. Our results provide novel evidence that goal-directed action may be particularly vulnerable to sleep loss, and the brain mechanism underlying this effect was explored. Elucidation of the effects of sleep deprivation on decision-making will deepen our understanding of the function of sleep, emphasizing the role of sleep in cognitive impairments and mental health.

The transition to adulthood of young adults with IDD: Parents' joint projects

INTRODUCTION

Parents have found the transition to adulthood for their sons or daughters with intellectual and/or developmental disabilities (IDD) particularly challenging. The literature has not examined how parents work together and with others in face of this transition nor has it highlighted parental goals in this process. This study used a perspective based on joint, goal-direct action to describe the projects that Canadian parents engaged in together and with others relative to this transition.

METHODS

Using the qualitative action-project method, joint projects between parents and with others were identified from their conversations and followed for 6 months.

FINDINGS

Three groups of projects were described: equipping the young adult for adult life, connecting for personal support and managing day-to-day while planning for the future.

CONCLUSIONS

Parents act together and with others relative to the transition to adulthood of their young adult children with IDD. These projects are complex and differ in goals, steps, resources and emotional regulation and motivation.

Consolidation of Goal-Directed Action Depends on MAPK/ERK Signaling in Rodent Prelimbic Cortex

The prelimbic prefrontal cortex (PL) has consistently been found to be necessary for the acquisition of goal-directed actions in rodents. Nevertheless, the specific cellular processes underlying this learning remain unknown. We assessed changes in learning-related expression of mitogen-activated protein kinase/extracellular signal-related kinase (MAPK/ERK1/2) phosphorylation (pERK) in layers 2-3 and 5-6 of the anterior and posterior PL and in the population of neurons projecting to posterior dorsomedial striatum (pDMS), also implicated in goal-directed learning. Rats were given either a single session of training to press a lever for a pellet reward or yoked reward deliveries without instrumental training and assessed 5 or 60 min after training for pERK expression. Relative to yoked training, instrumental training produced an increase in pERK expression in all regions of the PL both at 5 and 60 min, and this was accompanied by an increase in nuclear pERK expression in the posterior PL in rats given instrumental training. pDMS-projecting neurons showed a transient increase in pERK expression in posterior layer 5-6 projection neurons after 5 min, and a delayed increase in anterior layer 2-3 neurons after 60 min, suggesting that ERK expression in the PL is necessary for the consolidation of goal-directed learning. Consistent with this claim, we found that rats trained on two lever press actions for distinct outcomes and then infused with the MEK inhibitor PD98059 into the PL immediately after training failed to acquire specific action-outcome associations, suggesting that persistent pERK signaling in the PL is necessary for goal-directed learning.

SIGNIFICANCE STATEMENT

The prelimbic cortex is implicated in goal-directed learning in rodents; however, it is unclear whether it is involved in the consolidation of this learning, and what cellular processes are involved. We used pERK as a marker of activity-related synaptic plasticity to assess learning-induced changes in distinct layers and neuronal populations of the prelimbic prefrontal cortex (PL). Training produced long-lasting upregulation of pERK throughout the PL and specifically within neurons that project to the pDMS, another region critical for goal-directed learning. Next, we demonstrated that pERK signaling in the PL was necessary for the consolidation of goal-directed learning. Together, these results indicate that instrumental training induces ERK signaling in distinct layers and populations in the PL and this signaling underlies the consolidation of goal-directed learning.

The Impact of Segmental Trunk Support on Posture and Reaching While Sitting in Healthy Adults

The authors investigated postural and arm control in seated reaches while providing trunk support at midribs and pelvic levels in adults. Kinematics and electromyography of the arm and ipsiliateral and contralateral paraspinal muscles were examined before and during reaching. Kinematics remained constant across conditions, but changes were observed in neuromuscular control. With midribs support, the ipsilateral cervical muscle showed either increased anticipatory activity or earlier compensatory muscle responses, suggesting its major role in head stabilization. The baseline activity of bilateral lumbar muscles was enhanced with midribs support, whereas with pelvic support, the activation frequency of paraspinal muscles increased during reaching. The results suggest that segmental trunk support in healthy adults modulates ipsilateral or contralateral paraspinal activity while overall kinematic outputs remain invariant.

Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection

We (Bradfield et al., 2013) have demonstrated previously that parafascicular thalamic nucleus (PF)-controlled neurons in the posterior dorsomedial striatum (pDMS) are critical for interlacing new and existing action-outcome contingencies to control goal-directed action. Based on these findings, it was suggested that animals with a dysfunctional PF-pDMS pathway might suffer a deficit in creating or retrieving internal contexts or "states" on which such information could become conditional. To assess this hypothesis more directly, rats were given a disconnection treatment using contralateral cytotoxic lesions of the PF and pDMS (Group CONTRA) or ipsilateral control lesions (Group IPSI) and trained to press a right and left lever for sucrose and pellet outcomes, after which these contingencies were reversed. The rats were then given an outcome devaluation test (all experiments) and a test of outcome-specific reinstatement (Experiments 1 and 3). We found that devaluation performance was intact for both groups after training of initial contingencies, but impaired for Group CONTRA after reversal. However, performance was restored by additional reversal training. Furthermore, when tested a second time after reversal training, rats in both groups demonstrated responding in accordance with the original contingencies, providing direct evidence of modulation of action selection by state. Finally, we found that external context could substitute for internal state and so could rescue responding in Group CONTRA, but only in the reinstatement test. Together, these findings suggest that animals use internal state information to guide action selection and that this information is modulated by the PF-pDMS pathway.SIGNIFICANCE STATEMENT Individuals with Parkinson's disease dementia often suffer a characteristic deficit in "cognitive flexibility." It has been suggested that neurodegeneration in the pathway between the centromedian/parafascicular thalalmic nucleus (PF) and striatum might underlie such deficits (Smith et al., 2014). In rats, we have similarly observed that a functional disconnection of the PF-posterior dorsomedial striatal pathway produces a specific impairment in the ability to alter goal-directed actions (Bradfield et al., 2013). It was suggested that this impairment could be a result of a deficit in state modulation. Here, we present four experiments that provide evidence for this hypothesis and suggest several ways (e.g., extended practice, providing external cues) in which this state modulation can be rescued.

Pulling habits out of rats: adenosine 2A receptor antagonism in dorsomedial striatum rescues meth-amphetamine-induced deficits in goal-directed action

Addiction is characterized by a persistent loss of behavioral control resulting in insensitivity to negative feedback and abnormal decision-making. Here, we investigated the influence of methamphetamine (METH)-paired contextual cues on decision-making in rats. Choice between goal-directed actions was sensitive to outcome devaluation in a saline-paired context but was impaired in the METH-paired context, a deficit that was also found when negative feedback was provided. Reductions in c-Fos-related immunoreactivity were found in dorsomedial striatum (DMS) but not dorsolateral striatum after exposure to the METH context suggesting this effect reflected a loss specifically in goal-directed control in the METH context. This reduction in c-Fos was localized to non-enkephalin-expressing neurons in the DMS, likely dopamine D1-expressing direct pathway neurons, suggesting a relative change in control by the D1-direct versus D2-indirect pathways originating in the DMS may have been induced by METH-context exposure. To test this suggestion, we infused the adenosine 2A receptor antagonist ZM241385 into the DMS prior to test to reduce activity in D2 neurons relative to D1 neurons in the hope of reducing the inhibitory output from this region of the striatum. We found that this treatment fully restored sensitivity to negative feedback in a test conducted in the METH-paired context. These results suggest that drug exposure alters decision-making by downregulation of the circuitry mediating goal-directed action, an effect that can be ameliorated by acute A_2A receptor inhibition in this circuit.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Role of the Perigenual Anterior Cingulate and Orbitofrontal Cortex in Contingency Learning in the Marmoset

Two learning mechanisms contribute to decision-making: goal-directed actions and the "habit" system, by which action-outcome and stimulus-response associations are formed, respectively. Rodent lesion studies and human neuroimaging have implicated both the medial prefrontal cortex (mPFC) and the orbitofrontal cortex (OFC) in the neural basis of contingency learning, a critical component of goal-directed actions, though some published findings are conflicting. We sought to reconcile the existing literature by comparing the effects of excitotoxic lesions of the perigenual anterior cingulate cortex (pgACC), a region of the mPFC, and OFC on contingency learning in the marmoset monkey using a touchscreen-based paradigm, in which the contingent relationship between one of a pair of actions and its outcome was degraded selectively. Both the pgACC and OFC lesion groups were insensitive to the contingency degradation, whereas the control group demonstrated selectively higher performance of the nondegraded action when compared with the degraded action. These findings suggest the pgACC and OFC are both necessary for normal contingency learning and therefore goal-directed behavior.

Interaction of insular cortex and ventral striatum mediates the effect of incentive memory on choice between goal-directed actions

The anterior insular cortex (IC) and the nucleus accumbens (NAc) core have been separately implicated in the selection and performance of actions based on the incentive value of the instrumental outcome. Here, we examined the role of connections between the IC and the NAc core in the performance of goal-directed actions. Rats were trained on two actions for distinct outcomes, after which one of the two outcomes was devalued by specific satiety immediately before a choice extinction test. We first confirmed the projection from the IC to the NAc core and then disconnected these structures via asymmetrical excitotoxic lesions before training. Contralateral, but not ipsilateral, disconnection of the IC and NAc core disrupted outcome devaluation. We hypothesized that communication between the IC and NAc core is necessary for the retrieval of incentive value at test. To test this, we infused the GABAA agonist muscimol into the IC and the μ-opioid receptor antagonist CTAP into the contralateral NAc before the choice extinction test. As expected, inactivation of the IC in one hemisphere and blocking μ-opioid receptors in the contralateral NAc core abolished outcome-selective devaluation. These results suggest that the IC and NAc core form part of a circuit mediating the retrieval of outcome values and the subsequent choice between goal-directed actions based on those values.

Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer

Outcome-specific Pavlovian-instrumental transfer (PIT) demonstrates the way that reward-related cues influence choice between instrumental actions. The nucleus accumbens shell (NAc-S) contributes critically to this effect, particularly through its output to the rostral medial ventral pallidum (VP-m). Using rats, we investigated in two experiments the role in the PIT effect of the two major outputs of this VP-m region innervated by the NAc-S, the mediodorsal thalamus (MD) and the ventral tegmental area (VTA). First, two retrograde tracers were injected into the MD and VTA to compare the neuronal activity of the two populations of projection neurons in the VP-m during PIT relative to controls. Second, the functional role of the connection between the VP-m and the MD or VTA was assessed using asymmetrical pharmacological manipulations before a PIT test. It was found that, whereas neurons in the VP-m projecting to the MD showed significantly more neuronal activation during PIT than those projecting to the VTA, neuronal activation of these latter neurons correlated with the size of the PIT effect. Disconnection of the two pathways during PIT also revealed different deficits in performance: disrupting the VP-m to MD pathway removed the response biasing effects of reward-related cues, whereas disrupting the VP-m to VTA pathway preserved the response bias but altered the overall rate of responding. The current results therefore suggest that the VP-m exerts distinct effects on the VTA and MD and that these latter structures mediate the motivational and cognitive components of specific PIT, respectively.

Distinct fronto-striatal couplings reveal the double-faced nature of response-outcome relations in instruction-based learning

Higher species commonly learn novel behaviors by evaluating retrospectively whether actions have yielded desirable outcomes. By relying on explicit behavioral instructions, only humans can use an acquisition shortcut that prospectively specifies how to yield intended outcomes under the appropriate stimulus conditions. A recent and largely unexplored hypothesis suggests that striatal areas interact with lateral prefrontal cortex (LPFC) when novel behaviors are learned via explicit instruction, and that regional subspecialization exists for the integration of differential response-outcome contingencies into the current task model. Behaviorally, outcome integration during instruction-based learning has been linked to functionally distinct performance indices. This includes (1) compatibility effects, measured in a postlearning test procedure probing the encoding strength of outcome-response (O-R) associations, and (2) increasing response slowing across learning, putatively indicating active usage of O-R associations for the online control of goal-directed action. In the present fMRI study, we examined correlations between these behavioral indices and the dynamics of fronto-striatal couplings in order to mutually constrain and refine the interpretation of neural and behavioral measures in terms of separable subprocesses during outcome integration. We found that O-R encoding strength correlated with LPFC-putamen coupling, suggesting that the putamen is relevant for the formation of both S-R habits and habit-like O-R associations. By contrast, response slowing as a putative index of active usage of O-R associations correlated with LPFC-caudate coupling. This finding highlights the relevance of the caudate for the online control of goal-directed action also under instruction-based learning conditions, and in turn clarifies the functional relevance of the behavioral slowing effect.

The acquisition of goal-directed actions generates opposing plasticity in direct and indirect pathways in dorsomedial striatum

A cortical-basal ganglia network involving, particularly, the posterior region of dorsomedial striatum (DMS) has been implicated in the acquisition of goal-directed actions; however, no direct evidence of learning-related plasticity in this striatal region has been reported, nor is it known whether, or which, specific cell types are involved in this learning process. The striatum is primarily composed of two classes of spiny projection neurons (SPNs): the striatonigral and striatopallidal SPNs, which express dopamine D1 and D2 receptors, respectively. Here we establish that, in mice, the acquisition of goal-directed actions induced plasticity in both D1- and D2-SPNs specifically in the DMS and, importantly, that these changes were in opposing directions; after learning, AMPA/NMDA ratios were increased in D1-SPNs and reduced in the D2-SPNs in the DMS. Such opposing plasticity could provide the basis for rapidly rebiasing the control of task-specific actions, and its dysregulation could underlie disorders associated with striatal function.

δ-Opioid receptors in the accumbens shell mediate the influence of both excitatory and inhibitory predictions on choice

BACKGROUND AND PURPOSE

Stimuli that predict rewarding events can control choice between future actions, and this control could be mediated by δ-opioid receptors in the nucleus accumbens shell (NAc-S). Stimuli predicting the absence of important events can also guide choice, although it remains unknown whether they do so via changes in an accumbal δ-opioid receptor-related process.

EXPERIMENTAL APPROACH

δ-opioid receptor-eGFP mice were trained to perform two instrumental actions that delivered different food outcomes. Choice between the two actions was then tested in the presence of stimuli paired with either the delivery or the non-delivery of each of the two outcomes. Bilateral infusions of the δ-opioid receptor antagonist naltrindole into the NAc-S were used to determine the role of these receptors at the time of choice and δ-opioid receptor expression in the NAc-S used to assess functional activity.

KEY RESULTS

A stimulus predicting a specific outcome biased choice performance towards the action previously earning that same outcome. In contrast, a stimulus signalling the absence of that outcome biased performance away from the action that delivered that outcome towards actions associated with the absence of that outcome. Both effects were associated with increased δ-opioid receptor expression on the membrane of cholinergic interneurons within the NAc-S. Furthermore, both effects were blocked by naltrindole infused into the NAc-S.

CONCLUSIONS AND IMPLICATIONS

These findings suggest that δ-opioid receptors in the NAc-S were involved in the effects of predictive learning on choice between actions, whether those predictions involve the presence or absence of specific rewarding events.

LINKED ARTICLES

This article is part of a themed section on Opioids: New Pathways to Functional Selectivity. To view the other articles in this section visit http://dx.doi.org/10.1111/bph.2015.172.issue-2.

Seeing what you want to see: priors for one's own actions represent exaggerated expectations of success

People perceive the consequences of their own actions differently to how they perceive other sensory events. A large body of psychology research has shown that people also consistently overrate their own performance relative to others, yet little is known about how these "illusions of superiority" are normally maintained. Here we examined the visual perception of the sensory consequences of self-generated and observed goal-directed actions. Across a series of visuomotor tasks, we found that the perception of the sensory consequences of one's own actions is more biased toward success relative to the perception of observed actions. Using Bayesian models, we show that this bias could be explained by priors that represent exaggerated predictions of success. The degree of exaggeration of priors was unaffected by learning, but was correlated with individual differences in trait optimism. In contrast, when observing these actions, priors represented more accurate predictions of the actual performance. The results suggest that the brain internally represents optimistic predictions for one's own actions. Such exaggerated predictions bind the sensory consequences of our own actions with our intended goal, explaining how it is that when acting we tend to see what we want to see.

Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

The ability to learn contingencies between actions and outcomes in a dynamic environment is critical for flexible, adaptive behavior. Goal-directed actions adapt to changes in action-outcome contingencies as well as to changes in the reward-value of the outcome. When networks involved in reward processing and contingency learning are maladaptive, this fundamental ability can be lost, with detrimental consequences for decision-making. Impaired decision-making is a core feature in a number of psychiatric disorders, ranging from depression to schizophrenia. The argument can be developed, therefore, that seemingly disparate symptoms across psychiatric disorders can be explained by dysfunction within common decision-making circuitry. From this perspective, gaining a better understanding of the neural processes involved in goal-directed action, will allow a comparison of deficits observed across traditional diagnostic boundaries within a unified theoretical framework. This review describes the key processes and neural circuits involved in goal-directed decision-making using evidence from animal studies and human neuroimaging. Select studies are discussed to outline what we currently know about causal judgments regarding actions and their consequences, action-related reward evaluation, and, most importantly, how these processes are integrated in goal-directed learning and performance. Finally, we look at how adaptive decision-making is impaired across a range of psychiatric disorders and how deepening our understanding of this circuitry may offer insights into phenotypes and more targeted interventions.