What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Striatal dopamine release tracks the relationship between actions and their consequences

The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Instant habits versus flexible tenacity: Do implementation intentions accelerate habit formation?

Implementation intentions (strategic "if-then" plans) have been shown to support behaviour change. This may be achieved by mentally forming stimulus-response associations, thereby promoting habit formation. Does this deliberate attempt to instal "strategic automaticity" only offer advantages, or does it also come at the cost of reduced flexibility that characterises learnt habits? To investigate this, we tested healthy, young participants on a computerised instrumental learning task. Critically, we introduced implementation intentions ("if I see stimulus X, then I will respond") versus goal intentions ("for outcome Z, I will respond)" during instrumental acquisition, and subsequently assessed behavioural flexibility in an outcome-revaluation test. In Experiment 1, we conducted a between-subjects manipulation of strategic planning, and in Experiment 2, a within-subject manipulation. We hypothesised that implementation intentions would lead to strong stimulus-response associations and consequently impair performance when the signalled outcome value changed and therefore required a different response, while benefitting performance when the outcome value (and required response) remained the same. We found that implementation intentions supported instrumental learning, but impaired test performance overall (most robustly in Experiment 2), irrespective of whether the signalled outcome value had changed. We argue that this general detrimental effect of implementation intentions on test performance is likely a consequence of their negative effect on stimulus-outcome learning. Our findings warrant caution when applying if-then plans to situations where the agent does not already possess perfect knowledge of behavioural contingencies.While implementation intentions may support efficient and fast behavioural execution, this may come at the expense of behavioural flexibility.

Motor-cognitive coupling is impaired in children with mild or severe forms of developmental coordination disorder

Children with developmental coordination disorder (DCD) show deficits in motor-cognitive coupling. However, it remains unclear whether such deficits depend on the severity of DCD. The aim of this study was to examine cognitive-motor coupling under different levels of inhibitory control in children with severe (s-DCD) or moderate DCD (m-DCD), compared with typically-developing children (TDC). The performance of 29 primary-school children aged 6-12 years with s-DCD (Mage = 9.12 ± 1.56 years), 53 m-DCD (Mage = 8.78 ± 1.67 years), and 201 TDC (Mage = 9.20 ± 1.50 years) was compared on a double jump reaching task (DJRT) paradigm, presented on a large 42-inch touchscreen. The task display had a circular home-base, centred at the bottom of the display, and three target locations at radials of -20°, 0°, and 20°, 40 cm above the home-base circle. For the standard double-jump reaching task (DJRT), children moved their index finger from home-base circle to touch the target stimulus as fast as possible; 20% were jump trials where the target shifted left or right at lift-off. For the anti-jump reaching task (AJRT), 20% of trials required an anti-jump movement, touching the contralateral target location. While no group differences were shown on the DJRT, the DCD group were slower to complete reaching movements than the TDC group on AJRT; on the latter, the two DCD sub-groups were not shown to differ. Results confirm the presence of motor inhibition deficits in DCD which may not be dependent on the motor severity of the disorder.

Attention to inattention: effect of the other's attentional state on visual search performance in chimpanzees

In the present study, we examined the effects of the other's triadic attention to objects on visual search performances in chimpanzees. We found the search-asymmetry-like effect of the other's attentional state; the chimpanzees searched a target object not attended by the other individual more efficiently than that attended (Experiment 1). Additional experiments explored the possibility that the other individual "holding an object but not looking at it" led to expectancy violation (Experiment 2) or the role of nonsocial cues such as the proximity relation between the head and the object (Experiment 3). Still, these accounts alone did not explain this effect. It was also shown that the other's attentional state affected the chimpanzees' performances more readily as the interference effect than the facilitation effect (Experiment 4). Furthermore, the same effect was observed in the visual search for the gaze (head direction) of others (Experiment 5). We obtained the same results using photographs of chimpanzees (Experiment 6). Contrary to the chimpanzees, humans detected the object to which attention was directed more efficiently than vice versa (Experiment 7). The present results may reflect species differences between chimpanzees and humans in processing triadic social attention.

Goal-Directed Action Is Initially Impaired in a hAPP-J20 Mouse Model of Alzheimer's Disease

Cognitive-behavioral testing in preclinical models of Alzheimer's disease has failed to capture deficits in goal-directed action control. Here, we provide the first comprehensive investigation of goal-directed action in a transgenic mouse model of Alzheimer's disease. Specifically, we tested outcome devaluation performance in male and female human amyloid precursor protein (hAPP)-J20 mice. Mice were first trained to press left and right levers for pellet and sucrose outcomes, respectively (counterbalanced), over 4 d. On test, mice were prefed one of the outcomes to satiety and given a choice between levers. Devaluation performance was intact for 36-week-old wild-types of both sexes, who responded more on the valued relative to the devalued lever (Valued > Devalued). By contrast, devaluation was impaired (Valued = Devalued) for J20 mice of both sexes, and for 52-week-old male mice regardless of genotype. After additional lever press training (i.e., 8-d lever pressing in total), devaluation was intact for all mice, demonstrating that the initial deficits were not a result of a nonspecific impairment in reward processing, depression, or locomotor activity in J20 or aging mice. Follow-up analyses revealed that microglial expression in the dorsal CA1 region of the hippocampus was associated with poorer outcome devaluation performance on initial, but not later tests. Together, these data demonstrate that goal-directed action is initially impaired in J20 mice of both sexes and in aging male mice regardless of genotype, and that this impairment is related to neuroinflammation in the dorsal CA1 hippocampal region.

The Neural Bases of Action-Outcome Learning in Humans

From an associative perspective the acquisition of new goal-directed actions requires the encoding of specific action-outcome (AO) associations and, therefore, sensitivity to the validity of an action as a predictor of a specific outcome relative to other events. Although competitive architectures have been proposed within associative learning theory to achieve this kind of identity-based selection, whether and how these architectures are implemented by the brain is still a matter of conjecture. To investigate this issue, we trained human participants to encode various AO associations while undergoing functional neuroimaging (fMRI). We then degraded one AO contingency by increasing the probability of the outcome in the absence of its associated action while keeping other AO contingencies intact. We found that this treatment selectively reduced performance of the degraded action. Furthermore, when a signal predicted the unpaired outcome, performance of the action was restored, suggesting that the degradation effect reflects competition between the action and the context for prediction of the specific outcome. We used a Kalman filter to model the contribution of different causal variables to AO learning and found that activity in the medial prefrontal cortex (mPFC) and the dorsal anterior cingulate cortex (dACC) tracked changes in the association of the action and context, respectively, with regard to the specific outcome. Furthermore, we found the mPFC participated in a network with the striatum and posterior parietal cortex to segregate the influence of the various competing predictors to establish specific AO associations.SIGNIFICANCE STATEMENT Humans and other animals learn the consequences of their actions, allowing them to control their environment in a goal-directed manner. Nevertheless, it is unknown how we parse environmental causes from the effects of our own actions to establish these specific action-outcome (AO) relationships. Here, we show that the brain learns the causal structure of the environment by segregating the unique influence of actions from other causes in the medial prefrontal and anterior cingulate cortices and, through a network of structures, including the caudate nucleus and posterior parietal cortex, establishes the distinct causal relationships from which specific AO associations are formed.

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Thalamic input to motor cortex facilitates goal-directed action initiation

Prompt execution of planned motor action is essential for survival. The interactions between frontal cortical circuits and the basal ganglia are central to goal-oriented action selection and initiation.^1-4 In rodents, the ventromedial thalamic nucleus (VM) is one of the critical nodes that conveys the output of the basal ganglia to the frontal cortical areas including the anterior lateral motor cortex (ALM).^5-9 Recent studies showed the critical role of ALM and its interplay with the motor thalamus in preparing sensory-cued rewarded movements, specifically licking.^10-12 Work in primates suggests that the basal ganglia output to the motor thalamus transmits an urgency or vigor signal,^13-15 which leads to shortened reaction times and faster movement initiation. As yet, little is known about what signals are transmitted from the motor thalamus to the cortex during cued movements and how these signals contribute to movement initiation. In the present study, we employed a tactile-cued licking task in mice while monitoring reaction times of the initial lick. We found that inactivation of ALM delayed the initiation of cued licking. Two-photon Ca²⁺ imaging of VM axons revealed that the majority of the axon terminals in ALM were transiently active during licking. Their activity was predictive of the time of the first lick. Chemogenetic and optogenetic manipulation of VM axons in ALM indicated that VM inputs facilitate the initiation of cue-triggered and impulsive licking in trained mice. Our results suggest that VM thalamocortical inputs increase the probability and vigor of initiating planned motor responses.

Parafascicular Thalamic and Orbitofrontal Cortical Inputs to Striatum Represent States for Goal-Directed Action Selection

Several lines of evidence accrued over the last 5–10 years have converged to suggest that the parafascicular nucleus of the thalamus and the lateral orbitofrontal cortex each represent or contribute to internal state/context representations that guide action selection in partially observable task situations. In rodents, inactivations of each structure have been found to selectively impair performance in paradigms testing goal-directed action selection, but only when that action selection relies on state representations. Electrophysiological evidence has suggested that each structure achieves this function via inputs onto cholinergic interneurons (CINs) in the dorsomedial striatum. Here, we briefly review these studies, then point to anatomical evidence regarding the afferents of each structure and what they suggest about the specific features that each contribute to internal state representations. Finally, we speculate as to whether this role might be achieved interdependently through direct PF→OFC projections, or through the convergence of independent direct orbitofrontal cortex (OFC) and parafascicular nucleus of the thalamus (PF) inputs onto striatal targets.

Explicit Sense of Agency in an Automatic Control Situation: Effects of Goal-Directed Action and the Gradual Emergence of Outcome

Sense of agency (SoA), or the subjective feeling that “I am the agent controlling the object,” is essential for learning and enjoying object manipulation. Recently developed automatic control systems, such as the cruise control systems in autonomous vehicles, require less manual control from the manipulators. It has to date been impossible to completely relieve operators of the need for manual control in many automatic control systems developed for tool-using situations. Therefore, it is important to examine how to maintain SoA (illusorily) during an automatic control situation. We investigated the effects of two typical characteristics of everyday tool-use situations on SoA when braking a moving object with a keypress. These characteristics included the presence of a goal (e.g., in driving situations, the driver steps on the brake pedal to stop the car at an expected position) and the gradual emergence of the outcome (e.g., the driver steps on the brake pedal and the car usually slows down first and then stops). We conducted an experiment in which participants stopped a moving object and then rated their SoA for stopping the object. Participants were explicitly informed that the object would sometimes stop independently of their keypress. Results showed that both characteristics decreased SoA in the manual control situation but increased SoA in the automatic control situation. Thus, these characteristics could be useful for maintaining a sense of agency in automatic control situations.

Apathy is associated with reduced precision of prior beliefs about action outcomes

Apathy is a debilitating syndrome that is associated with reduced goal-directed behavior. Although apathy is common and detrimental to prognosis in many neuropsychiatric diseases, its underlying mechanisms remain controversial. We propose a new model of apathy, in the context of Bayesian theories of brain function, whereby actions require predictions of their outcomes to be held with sufficient precision for "explaining away" differences in sensory inputs. In the active inference model, apathy results from reduced precision of prior beliefs about action outcomes. We tested this hypothesis using a visuomotor task in healthy adults (N = 47), with experimental manipulation of physical effort and financial reward. Bayesian modeling of performance and participants' perception of their performance was used to infer the precision of their priors. We confirmed that the perception of performance was biased toward the target, which was accounted for by relatively precise prior beliefs about action outcomes. These priors were consistently more precise than the corresponding performance distribution, and were scaled to effort and reward. Crucially, prior precision was negatively associated with trait apathy, suggesting that apathetic individuals had less precise prior beliefs about action outcomes. The results support a Bayesian account of apathy that could inform future studies of clinical populations. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The coordinate system of endogenous spatial attention during smooth pursuit

A central question in vision is whether spatial attention is represented in an eye-centered (retinotopic) or world-centered (spatiotopic) reference-frame. Most previous studies on this question focused on how coordinates are modulated across saccades. In the present study, we investigated the reference-frame of attention across smooth pursuit eye-movements using a goal-directed saccade task. In two experiments, participants were asked to pursue a moving target while attending to one or two grating stimuli. On each trial, one stimulus was constant in its retinal position and the other was constant in its spatial position. Upon detection of a slight change in stimulus orientation, participants were asked to stop pursuing and perform a fast saccade toward the modified stimulus. In the focused attention condition, they attended one, predefined, stimulus, and in the divided attention condition they attended both. In Experiment 1 the angle of the orientation change marking the target event was constant across participants and conditions. In Experiment 2, the angle was individually adapted to equate performance across participants and conditions. Findings of the two experiments were consistent and showed that the enhancement of mean visual sensitivity in the focused relative to the divided attention condition was similar in magnitude for both retinotopic and spatiotopic targets. This indicates that during smooth pursuit, endogenous attention was proportionally divided between targets in retinotopic and spatiotopic frames of reference.

The teleological stance: Past, present, and future

We review the support for, and criticisms of, the teleological stance theory, often described as a foundation for goal-directed action understanding early in life. A major point of contention in the literature has been how teleological processes and assumptions of rationality are represented and understood in infancy, and this debate has been largely centered on three paradigms. Visual habituation studies assess infant's abilities to retrospectively assess teleological processes; the presence of such processes is supported by the literature. Rational imitation is a phenomenon that has been questioned both theoretically and empirically, and there is currently little support for this concept in the literature. The involvement of teleological processes in action prediction is unclear. To date, the ontology of teleological processes remains unspecified. To remedy this, we present a new action-based theory of teleological processes (here referred to as the embodied account of teleological processes), based on the development of goal-directed reaching with its origin during the fetal period and continuous development over the first few months of life.

Training With Direct Versus Indirect Spatial Stimulus-Response Compatibility in Combat Sports

The study examined effects of spatial stimulus-response compatibility on response time and response accuracy in 20 novice combat sport athletes. Two equivalent groups, based on initial reaction time measures, were required to perceive and move quickly and accurately in response to an unspecific visual stimulus presented on a large screen during the two types of perceptual training in eight laboratory sessions. One group reacted by moving the fist toward the stimulus location on the target (direct compatibility condition). Another group was required to move the fist away from target in the opposite direction (indirect compatibility condition). Specifically, the indirect compatibility group achieved faster reaction times than the direct compatibility group during the two posttests containing video-projected attacks of the opponents, and in one of the two posttests containing real opponents' attacks. Results seem to reveal higher combat performance against real opponents when athletes trained with an indirect stimulus-response compatibility condition.

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Modeling Dynamic Allocation of Effort in a Sequential Task Using Discounting Models

Most rewards in our lives require effort to obtain them. It is known that effort is seen by humans as carrying an intrinsic disutility which devalues the obtainable reward. Established models for effort discounting account for this by using participant-specific discounting parameters inferred from experiments. These parameters offer only a static glance into the bigger picture of effort exertion. The mechanism underlying the dynamic changes in a participant's willingness to exert effort is still unclear and an active topic of research. Here, we modeled dynamic effort exertion as a consequence of effort- and probability-discounting mechanisms during goal reaching, sequential behavior. To do this, we developed a novel sequential decision-making task in which participants made binary choices to reach a minimum number of points. Importantly, the time points and circumstances of effort allocation were decided by participants according to their own preferences and not imposed directly by the task. Using the computational model to analyze participants' choices, we show that the dynamics of effort exertion arise from a combination of changing task needs and forward planning. In other words, the interplay between a participant's inferred discounting parameters is sufficient to explain the dynamic allocation of effort during goal reaching. Using formal model comparison, we also inferred the forward-planning strategy used by participants. The model allowed us to characterize a participant's effort exertion in terms of only a few parameters. Moreover, the model can be adapted to a number of tasks used in establishing the neural underpinnings of forward-planning behavior and meta-control, allowing for the characterization of behavior in terms of model parameters.

Children's understanding of habitual behaviour

Research into the development of Theory of Mind (ToM) has shown how children from a very early age infer other people's goals. However, human behaviour is sometimes driven not by plans to achieve goals, but by habits, which are formed over long periods of reinforcement. Habitual and goal-directed behaviours are often aligned with one another but can diverge when the optimal behavioural policy changes without being directly reinforced (thus specifically hobbling the habitual learning strategy). Unlike the flexibility of goal-directed behaviour, rigid habits can cause agents to persist in behaviour that is no longer adaptive. In the current study, all children predict agents will tend to behave consistently with their goals, but between the ages of 5 and 10, children showed an increasing understanding of how habits can cause agents to persistently take suboptimal actions. These findings stand out from the typical way the development of social reasoning is examined, which instead focuses on children's increasing appreciation of how others' beliefs or expectations affect how they will act in service of their goals. The current findings show that children also learn that under certain circumstances, people's actions are suboptimal despite potentially 'knowing better.'

Complementary Control over Habits and Behavioral Vigor by Phasic Activity in the Dorsolateral Striatum

Despite clear evidence linking the basal ganglia to the control of outcome insensitivity (i.e., habit) and behavioral vigor (i.e., its behavioral speed/fluidity), it remains unclear whether or how these functions relate to one another. Here, using male Long-Evans rats in response-based and cue-based maze-running tasks, we demonstrate that phasic dorsolateral striatum (DLS) activity occurring at the onset of a learned behavior regulates how vigorous and habitual it is. In a response-based task, brief optogenetic excitation at the onset of runs decreased run duration and the occurrence of deliberative behaviors, whereas midrun stimulation carried little effect. Outcome devaluation showed these runs to be habitual. DLS inhibition at run start did not produce robust effects on behavior until after outcome devaluation. At that time, when the DLS was plausibly most critically required for performance (i.e., habitual), inhibition reduced performance vigor measures and caused a dramatic loss of habitual responding (i.e., animals quit the task). In a second cue-based "beacon" task requiring behavior initiation at the start of the run and again in the middle of the run, DLS excitation at both time points could improve the vigor of runs. Postdevaluation testing showed behavior on the beacon task to be habitual as well. This pattern of results suggests that one role for phasic DLS activity at behavior initiation is to promote the execution of the behavior in a vigorous and habitual fashion by a diverse set of measures.SIGNIFICANCE STATEMENT Our research expands the literature twofold. First, we find that features of a habitual behavior that are typically studied separately (i.e., maze response performance, deliberation movements, running vigor, and outcome insensitivity) are quite closely linked together. Second, efforts have been made to understand "what" the dorsolateral striatum (DLS) does for habitual behavior, and our research provides a key set of results showing "when" it is important (i.e., at behavior initiation). By showing such dramatic control over habits by DLS activity in a phasic time window, plausible real-world applications could involve more informed DLS perturbations to curb intractably problematic habits.

Hierarchical Action Control: Adaptive Collaboration Between Actions and Habits

It is now commonly accepted that instrumental actions can reflect goal-directed control; i.e., they can show sensitivity to changes in the relationship to and the value of their consequences. With overtraining, stress, neurodegeneration, psychiatric conditions, or after exposure to various drugs of abuse, goal-directed control declines and instrumental actions are performed independently of their consequences. Although this latter insensitivity has been argued to reflect the development of habitual control, the lack of a positive definition of habits has rendered this conclusion controversial. Here we consider various alternative definitions of habit, including recent suggestions they reflect chunked action sequences, to derive criteria with which to categorize responses as habitual. We consider various theories regarding the interaction between goal-directed and habitual controllers and propose a collaborative model based on their hierarchical integration. We argue that this model is consistent with the available data, can be instantiated both at an associative level and computationally and generates interesting predictions regarding the influence of this collaborative integration on behavior.

Enhanced Avoidance Habits in Relation to History of Early-Life Stress

The effect of stress on the balance between goal-directed behavior and stimulus-response habits has been demonstrated in a number of studies, but the extent to which stressful events that occur during development affect the balance between these systems later in life is less clear. Here, we examined whether individuals with a history of early-life stress (ELS) show a bias toward avoidance habits on an instrumental learning task as adults. Participants (N = 189 in Experiment 1 and N = 112 in Experiment 2) were undergraduate students at the University of California, Los Angeles. In Experiment 1, we hypothesized that a history of ELS and a longer training phase would be associated with greater avoidance habits. Participants learned to make button-press responses to visual stimuli in order to avoid aversive auditory outcomes. Following a training phase involving extensive practice of the responses, participants were tested for habitual responding using outcome devaluation. After completing the instrumental learning task, participants provided retrospective reports of stressful events they experienced during their first 16 years of life. We did not observe evidence for an effect of the length of training, but we did observe an effect of ELS, with greater stress predicting greater odds of performing the avoidance habit. In Experiment 2, we sought to replicate the effect of ELS observed in Experiment 1, and we also tested whether the presence of distraction during training would increase avoidance habit performance. We replicated the effect of ELS but we did not observe evidence of an effect of distraction. Taken together, these data lend support to the hypothesis that stress occurring during development can have lasting effects on the balance between goal-directed behavior and stimulus-response habits in humans. Enhancement of avoidance habits may help explain the higher levels of negative health outcomes such as heart and liver disease that have been observed in individuals with a history of ELS. Some of the negative health behaviors that contribute to these negative health outcomes, e.g., overeating and substance use, may be performed initially to avoid feelings of distress and then transition to being performed habitually.

Action-based attention in Drosophila melanogaster

The mechanism of action selection is a widely shared fundamental process required by animals to interact with the environment and adapt to it. A key step in this process is the filtering of the "distracting" sensory inputs that may disturb action selection. Because it has been suggested that, in principle, action selection may also be processed by shared circuits in vertebrate and invertebrates, we wondered whether invertebrates show the ability to filter out "distracting" stimuli during a goal-directed action, as seen in vertebrates. In this experiment, action selection was studied in wild-type Drosophila melanogaster by investigating their reaction to the abrupt appearance of a visual distractor during an ongoing locomotor action directed to a visual target. We found that when the distractor was present, flies tended to shift the original trajectory toward it, thus acknowledging its presence, but they did not fully commit to it, suggesting that an inhibition process took place to continue the unfolding of the planned goal-directed action. To some extent flies appeared to take into account and represent motorically the distractor, but they did not engage in a complete change of their initial motor program in favor of the distractor. These results provide interesting insights into the selection-for-action mechanism, in a context requiring action-centered attention, that might have appeared rather early in the course of evolution. NEW & NOTEWORTHY Action selection and maintenance of a goal-directed action require animals to ignore irrelevant "distracting" stimuli that might elicit alternative motor programs. In this study we observed, in Drosophila melanogaster, a top-down mechanism inhibiting the response toward salient stimuli, to accomplish a goal-directed action. These data highlight, for the first time in an invertebrate organism, that the action-based attention shown by higher organisms, such as humans and nonhuman primates, might have an ancestral origin.

Context-Dependence and Context-Invariance in the Neural Coding of Intentional Action

Maintaining intentions over time is fundamental to goal-directed action, and previous research demonstrated that intentions are encoded and maintained in a fronto-parietal network including e.g., the dlPFC and IPS. Yet, intention maintenance is highly challenging in the constantly changing environments we experience every day. While we might have formed an intention under specific conditions, this context can change rapidly and unexpectedly. Some suggested that intentions representations in the fronto-parietal cortex change flexibly when external demands change (context-dependent coding). Others suggested that these representations are encoded in an abstract format that is not affected by changes in external demands (context-invariant coding). Here, I will first outline an analysis approach using multivariate pattern analysis of fMRI data to comprehensively assess the context-dependence / invariance of intention representations in the fronto-parietal cortex. I will then highlight some research following the proposed analysis strategy. Results to date are mixed, showing context-dependence in some, but context-invariance in other cases. In an attempt to synthesize these somewhat divergent results, I will argue that depending on characteristics of the intentions as well as the environment, intentions can either be encoded in a context-dependent or a context-invariant format. This enables us to achieve both stability and flexibility of behavior under constantly changing external demands.

Conflicted between Goal-Directed and Habitual Control, an fMRI Investigation

“Slips of action” occur in everyday life when we momentarily lose sight of a goal (for example, when in a rush or distracted). Associative models propose that these habitual responses can be activated via a direct stimulus-response (S-R) mechanism, regardless of the current hedonic value of the outcome. The slips-of-action task (SOAT) has been extensively used in both healthy and pathological populations to measure habit tendencies, the likelihood of making erroneous responses for devalued outcomes. Inspection of behavioral performance does not reveal, however, whether the impairments were due to impaired goal-directed control or aberrantly strong habit formation. In the current study, we used functional MRI while human participants performed both the instrumental training and SOAT test phases, to elucidate the relative contributions of these mechanisms to performance on the SOAT. On trials in which conflict arises between competing goal-directed and habitual responses, we observed increased activation across areas including the anterior cingulate cortex, paracingulate gyrus, lateral orbitofrontal cortex (OFC), insula, and inferior frontal gyrus (IFG). Responding for devalued outcomes was related to increased activation in the premotor cortex and cerebellum, implicating these regions in habitual responding. Increased activation in the caudate, dorsolateral prefrontal cortex (dlPFC), and frontal pole during training was associated with better performance during the test phase, indicative of goal-directed action control. These results endorse interpretation of the SOAT in terms of competing goal-directed and habitual mechanisms and highlight that cognitive control processes present an additional bottleneck for successful performance on this task.

Stimulus control of actions and habits: A role for reinforcer predictability and attention in the development of habitual behavior

Goal-directed actions are instrumental behaviors whose performance depends on the organism's knowledge of the reinforcing outcome's value. In contrast, habits are instrumental behaviors that are insensitive to the outcome's current value. Although habits in everyday life are typically controlled by stimuli that occasion them, most research has studied habits using free-operant procedures in which no discrete stimuli are present to occasion the response. We therefore studied habit learning when rats were reinforced for lever pressing on a random-interval 30-s schedule in the presence of a discriminative stimulus (S) but not in its absence. In Experiment 1, devaluing the reinforcer with taste aversion conditioning weakened instrumental responding in a 30-s S after 4, 22, and 66 sessions of instrumental training. Even extensive practice thus produced goal-directed action, not habit. Experiments 2 and 3 contrastingly found habit when the duration of S was increased from 30 s to 8 min. Experiment 4 then found habit with the 30-s S when it always contained a reinforcer; goal-directed action was maintained when reinforcers were earned at the same rate but occurred in only 50% of Ss (as in the previous experiments). The results challenge the view that habits are an inevitable consequence of repeated reinforcement (as in the law of effect) and instead suggest that discriminated habits develop when the reinforcer becomes predictable. Under those conditions, organisms may pay less attention to their behavior, much as they pay less attention to signals associated with predicted reinforcers in Pavlovian conditioning. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Transition to Adulthood as a Joint Parent-Youth Project for Young Persons With Intellectual and Developmental Disabilities

Eight dyads ( N = 16) residing in Western Canada participated in this investigation of how young adults with intellectual and developmental disabilities (IDD) and their parents jointly construct, articulate, and act on goals pertinent to the young adults' transition to adulthood. Using the action-project method to collect and analyze conversations and video recall data, cases were grouped representing the ways goal-directed projects brought relationship ( n = 4), planning ( n = 3) or both ( n = 1) to the foreground as joint projects. Resources internal to the dyad such as emotional resources, and external to the dyad, facilitated formulation and pursuit of projects. Lack of external supports and limited parental knowledge about IDD hindered joint project formulation.

From learning to action: the integration of dorsal striatal input and output pathways in instrumental conditioning

Considerable evidence suggests that the learning and performance of instrumental actions depend on activity in basal ganglia circuitry; however, these two functions have generally been considered independently. Whereas research investigating the associative mechanisms underlying instrumental conditioning has identified critical cortical and limbic input pathways to the dorsal striatum, the performance of instrumental actions has largely been attributed to activity in the dorsal striatal output pathways, with direct and indirect pathway projection neurons mediating action initiation, perseveration and cessation. Here, we discuss evidence that the dorsal striatal input and basal ganglia output pathways mediate the learning and performance of instrumental actions, respectively, with the dorsal striatum functioning as a transition point. From this perspective, the issue of how multiple striatal inputs are integrated at the level of the dorsal striatum and converted into relatively restricted outputs becomes one of critical significance for understanding how learning is translated into action. So too does the question of how learning signals are modulated by recent experience. We propose that this occurs through recurrent corticostriatothalamic feedback circuits that serve to integrate performance signals by updating ongoing action-related learning.

Shifting the balance between goals and habits: Five failures in experimental habit induction

Habits are repetitive behaviors that become ingrained with practice, routine, and repetition. The more we repeat an action, the stronger our habits become. Behavioral and clinical neuroscientists have become increasingly interested in this topic because habits may contribute to aspects of maladaptive human behavior, such as compulsive behavior in psychiatry. Numerous studies have demonstrated that habits can be induced in otherwise healthy rats by simply overtraining stimulus-response behaviors. However, despite growing interest in this topic and its application to psychiatry, a similar body of work in humans is absent. Only a single study has been published in humans that shows the effect of extensive training on habit expression. Here, we report five failed attempts to demonstrate that overtraining instrumental behavior leads to the development of inflexible habits in humans, using variants of four previously published outcome devaluation paradigms. Extensive training did not lead to greater habits in two versions of an avoidance learning task, in an appetitive slips-of-action task, or in two independent attempts to replicate the original demonstration. The finding that these outcome devaluation procedures may be insensitive to duration of stimulus-response training in humans has implications for prior work in psychiatric populations. Specifically, it converges with the suggestion that the failures in outcome devaluation in compulsive individuals reflect dysfunction in goal-directed control, rather than overactive habit learning. We discuss why habits are difficult to experimentally induce in healthy humans, and the implications of this for future research in healthy and disordered populations. (PsycINFO Database Record

The Bilateral Prefronto-striatal Pathway Is Necessary for Learning New Goal-Directed Actions

The acquisition of new goal-directed actions requires the encoding of action-outcome associations. At a neural level, this encoding has been hypothesized to involve a prefronto-striatal circuit extending between the prelimbic cortex (PL) and the posterior dorsomedial striatum (pDMS); however, no research identifying this pathway with any precision has been reported. We started by mapping the prelimbic input to the dorsal and ventral striatum using a combination of retrograde and anterograde tracing with CLARITY and established that PL-pDMS projections share some overlap with projections to the nucleus accumbens core (NAc) in rats. We then tested whether each of these pathways were functionally required for goal-directed learning; we used a pathway-specific dual-virus chemogenetic approach to selectively silence pDMS-projecting or NAc-projecting PL neurons during instrumental training and tested rats for goal-directed action. We found that silencing PL-pDMS projections abolished goal-directed learning, whereas silencing PL-NAc projections left goal-directed learning intact. Finally, we used a three-virus approach to silence bilateral and contralateral pDMS-projecting PL neurons and again blocked goal-directed learning. These results establish that the acquisition of new goal-directed actions depends on the bilateral PL-pDMS pathway driven by intratelencephalic cortical neurons.

The response strategy and the place strategy in a plus-maze have different sensitivities to devaluation of expected outcome

Previous studies have suggested that spatial navigation can be achieved with at least two distinct learning processes, involving either cognitive map‐like representations of the local environment, referred to as the “place strategy”, or simple stimulus‐response (S‐R) associations, the “response strategy”. A similar distinction between cognitive/behavioral processes has been made in the context of non‐spatial, instrumental conditioning, with the definition of two processes concerning the sensitivity of a given behavior to the expected value of its outcome as well as to the response‐outcome contingency (“goal‐directed action” and “S‐R habit”). Here we investigated whether these two versions of dichotomist definitions of learned behavior, one spatial and the other non‐spatial, correspond to each other in a formal way. Specifically, we assessed the goal‐directed nature of two navigational strategies, using a combination of an outcome devaluation procedure and a spatial probe trial frequently used to dissociate the two navigational strategies. In Experiment 1, rats trained in a dual‐solution T‐maze task were subjected to an extinction probe trial from the opposite start arm, with or without prefeeding‐induced devaluation of the expected outcome. We found that a non‐significant preference for the place strategy in the non‐devalued condition was completely reversed after devaluation, such that significantly more animals displayed the use of the response strategy. The result suggests that the place strategy is sensitive to the expected value of the outcome, while the response strategy is not. In Experiment 2, rats with hippocampal lesions showed significant reliance on the response strategy, regardless of whether the expected outcome was devalued or not. The result thus offers further evidence that the response strategy conforms to the definition of an outcome‐insensitive, habitual form of instrumental behavior. These results together attest a formal correspondence between two types of dual‐process accounts of animal learning and behavior.

Reaching Into the Unknown: Actions, Goal Hierarchies, and Explorative Agency

Action is widely characterized as possessing a teleological dimension. The dominant way of describing goal-directed action and agency is in terms of exploitation, i.e., pursuing pre-specified goals using existing strategies. Recent theoretical developments emphasize the place of exploration, i.e., discovering new goals or acquiring new strategies. The exploitation-exploration distinction poses questions with regard to goals and agency: Should exploration, as some authors have suggested, be regarded as acting without a goal? We argue that recognizing the hierarchical nature of goals is crucial in distinguishing the two kinds of activity, because this recognition prevents the claim that exploration is goal-free, while allowing for a homogeneous account of both exploitative and explorative actions. An action typically causes relatively low-level/proximal (i.e., sensorimotor, immediate) and relatively high-level/distal (i.e., in the environment, at a wider timescale) outcomes. In exploitation, one relies on existing associations between low- and high-level states, whereas in exploration one does not have the ability or intention to control high-level/distal states. We argue that explorative action entails the capacity to exercise control within the low-level/proximal states, which enables the pursuit of indeterminate goals at the higher levels of a goal hierarchy, and the possibility of acquiring new goals and reorganization of goal hierarchies. We consider how the dominant models of agency might accommodate this capacity for explorative action.

Observation of Point-Light-Walker Locomotion Induces Motor Resonance When Explicitly Represented; An EEG Source Analysis Study

Understanding human motion, to infer the goal of others' actions, is thought to involve the observer's motor repertoire. One prominent class of actions, the human locomotion, has been object of several studies, all focused on manipulating the shape of degraded human figures like point-light walker (PLW) stimuli, represented as walking on the spot. Nevertheless, since the main goal of the locomotor function is to displace the whole body from one position to the other, these stimuli might not fully represent a goal-directed action and thus might not be able to induce the same motor resonance mechanism expected when observing a natural locomotion. To explore this hypothesis, we recorded the event-related potentials (ERP) of canonical/scrambled and translating/centered PLWs decoding. We individuated a novel ERP component (N2c) over central electrodes, around 435 ms after stimulus onset, for translating compared to centered PLW, only when the canonical shape was preserved. Consistently with our hypothesis, sources analysis associated this component to the activation of trunk and lower legs primary sensory-motor and supplementary motor areas. These results confirm the role of own motor repertoire in processing human action and suggest that ERP can detect the associated motor resonance only when the human figure is explicitly involved in performing a meaningful action.

Prefrontal Corticostriatal Disconnection Blocks the Acquisition of Goal-Directed Action

The acquisition of goal-directed action requires encoding of the association between an action and its specific consequences or outcome. At a neural level, this encoding has been hypothesized to involve a prefrontal corticostriatal circuit involving the projection from the prelimbic cortex (PL) to the posterior dorsomedial striatum (pDMS); however, no direct evidence for this claim has been reported. In a series of experiments, we performed functional disconnection of this pathway using targeted lesions of the anterior corpus callosum to disrupt contralateral corticostriatal projections with asymmetrical lesions of the PL and/or pDMS to block plasticity in this circuit in rats. We first demonstrated that unilaterally blocking the PL input to the pDMS prevented the phosphorylation of extracellular signal-related kinase/mitogen activated protein kinase (pERK/pMAPK) induced by instrumental training. Next, we used a full bilateral disconnection of the PL from the pDMS and assessed goal-directed action using an outcome-devaluation test. Importantly, we found evidence that rats maintaining an ipsilateral and/or contralateral connection between the PL and the pDMS were able to acquire goal-directed actions. In contrast, bilateral PL-pDMS disconnection abolished the acquisition of goal-directed actions. Finally, we used a temporary pharmacological disconnection to disrupt PL inputs to the pDMS by infusing the NMDA antagonist dl-2-amino-5-phosphonopentanoic acid into the pDMS during instrumental training and found that this manipulation also disrupted goal-directed learning. These results establish that, in rats, the acquisition of new goal-directed actions depends on a prefrontal-corticostriatal circuit involving a connection between the PL and the pDMS.SIGNIFICANCE STATEMENT It has been hypothesized that the prelimbic cortex (PL) and posterior dorsomedial striatum (pDMS) in rodents interact in a corticostriatal circuit to mediate goal-directed learning. However, no direct evidence supporting this claim has been reported. Using targeted lesions, we performed functional disconnection of the PL-pDMS pathway to assess its role in goal-directed learning. In the first experiment, we demonstrated that PL input to the pDMS is necessary for instrumental training-induced neuronal activity. Next, we disrupted ipsilateral, contralateral, or bilateral PL-pDMS connections and found that only bilateral PL-pDMS disconnection disrupted the acquisition of goal-directed actions, a finding we replicated in our final study using a pharmacological disconnection procedure.

Sleep Deprivation Promotes Habitual Control over Goal-Directed Control: Behavioral and Neuroimaging Evidence

Sleep is one of the most fundamental processes of life, playing an important role in the regulation of brain function. The long-term lack of sleep can cause memory impairments, declines in learning ability, and executive dysfunction. In the present study, we evaluated the effects of sleep deprivation on instrumental learning behavior, particularly goal-directed and habitual actions in humans, and investigated the underlying neural mechanisms. Healthy college students of either gender were enrolled and randomly divided into sleep deprivation group and sleep control group. fMRI data were collected. We found that one night of sleep deprivation led to greater responsiveness to stimuli that were associated with devalued outcomes in the slips-of-action test, indicating a deficit in the formation of goal-directed control and an overreliance on habits. Furthermore, sleep deprivation had no effect on the expression of acquired goal-directed action. The level of goal-directed action after sleep deprivation was positively correlated with baseline working memory capacity. The neuroimaging data indicated that goal-directed learning mainly recruited the ventromedial PFC (vmPFC), the activation of which was less pronounced during goal-directed learning after sleep deprivation. Activation of the vmPFC during goal-directed learning during training was positively correlated with the level of goal-directed action performance. The present study suggests that people rely predominantly on habits at the expense of goal-directed control after sleep deprivation, and this process involves the vmPFC. These results contribute to a better understanding of the effects of sleep loss on decision-making.SIGNIFICANCE STATEMENT Understanding the cognitive consequences of sleep deprivation has become extremely important over the past half century, given the continued decline in sleep duration in industrialized societies. Our results provide novel evidence that goal-directed action may be particularly vulnerable to sleep loss, and the brain mechanism underlying this effect was explored. Elucidation of the effects of sleep deprivation on decision-making will deepen our understanding of the function of sleep, emphasizing the role of sleep in cognitive impairments and mental health.

The transition to adulthood of young adults with IDD: Parents' joint projects

INTRODUCTION

Parents have found the transition to adulthood for their sons or daughters with intellectual and/or developmental disabilities (IDD) particularly challenging. The literature has not examined how parents work together and with others in face of this transition nor has it highlighted parental goals in this process. This study used a perspective based on joint, goal-direct action to describe the projects that Canadian parents engaged in together and with others relative to this transition.

METHODS

Using the qualitative action-project method, joint projects between parents and with others were identified from their conversations and followed for 6 months.

FINDINGS

Three groups of projects were described: equipping the young adult for adult life, connecting for personal support and managing day-to-day while planning for the future.

CONCLUSIONS

Parents act together and with others relative to the transition to adulthood of their young adult children with IDD. These projects are complex and differ in goals, steps, resources and emotional regulation and motivation.

Consolidation of Goal-Directed Action Depends on MAPK/ERK Signaling in Rodent Prelimbic Cortex

The prelimbic prefrontal cortex (PL) has consistently been found to be necessary for the acquisition of goal-directed actions in rodents. Nevertheless, the specific cellular processes underlying this learning remain unknown. We assessed changes in learning-related expression of mitogen-activated protein kinase/extracellular signal-related kinase (MAPK/ERK1/2) phosphorylation (pERK) in layers 2-3 and 5-6 of the anterior and posterior PL and in the population of neurons projecting to posterior dorsomedial striatum (pDMS), also implicated in goal-directed learning. Rats were given either a single session of training to press a lever for a pellet reward or yoked reward deliveries without instrumental training and assessed 5 or 60 min after training for pERK expression. Relative to yoked training, instrumental training produced an increase in pERK expression in all regions of the PL both at 5 and 60 min, and this was accompanied by an increase in nuclear pERK expression in the posterior PL in rats given instrumental training. pDMS-projecting neurons showed a transient increase in pERK expression in posterior layer 5-6 projection neurons after 5 min, and a delayed increase in anterior layer 2-3 neurons after 60 min, suggesting that ERK expression in the PL is necessary for the consolidation of goal-directed learning. Consistent with this claim, we found that rats trained on two lever press actions for distinct outcomes and then infused with the MEK inhibitor PD98059 into the PL immediately after training failed to acquire specific action-outcome associations, suggesting that persistent pERK signaling in the PL is necessary for goal-directed learning.

SIGNIFICANCE STATEMENT

The prelimbic cortex is implicated in goal-directed learning in rodents; however, it is unclear whether it is involved in the consolidation of this learning, and what cellular processes are involved. We used pERK as a marker of activity-related synaptic plasticity to assess learning-induced changes in distinct layers and neuronal populations of the prelimbic prefrontal cortex (PL). Training produced long-lasting upregulation of pERK throughout the PL and specifically within neurons that project to the pDMS, another region critical for goal-directed learning. Next, we demonstrated that pERK signaling in the PL was necessary for the consolidation of goal-directed learning. Together, these results indicate that instrumental training induces ERK signaling in distinct layers and populations in the PL and this signaling underlies the consolidation of goal-directed learning.

The Impact of Segmental Trunk Support on Posture and Reaching While Sitting in Healthy Adults

The authors investigated postural and arm control in seated reaches while providing trunk support at midribs and pelvic levels in adults. Kinematics and electromyography of the arm and ipsiliateral and contralateral paraspinal muscles were examined before and during reaching. Kinematics remained constant across conditions, but changes were observed in neuromuscular control. With midribs support, the ipsilateral cervical muscle showed either increased anticipatory activity or earlier compensatory muscle responses, suggesting its major role in head stabilization. The baseline activity of bilateral lumbar muscles was enhanced with midribs support, whereas with pelvic support, the activation frequency of paraspinal muscles increased during reaching. The results suggest that segmental trunk support in healthy adults modulates ipsilateral or contralateral paraspinal activity while overall kinematic outputs remain invariant.

Thalamic Control of Dorsomedial Striatum Regulates Internal State to Guide Goal-Directed Action Selection

We (Bradfield et al., 2013) have demonstrated previously that parafascicular thalamic nucleus (PF)-controlled neurons in the posterior dorsomedial striatum (pDMS) are critical for interlacing new and existing action-outcome contingencies to control goal-directed action. Based on these findings, it was suggested that animals with a dysfunctional PF-pDMS pathway might suffer a deficit in creating or retrieving internal contexts or "states" on which such information could become conditional. To assess this hypothesis more directly, rats were given a disconnection treatment using contralateral cytotoxic lesions of the PF and pDMS (Group CONTRA) or ipsilateral control lesions (Group IPSI) and trained to press a right and left lever for sucrose and pellet outcomes, after which these contingencies were reversed. The rats were then given an outcome devaluation test (all experiments) and a test of outcome-specific reinstatement (Experiments 1 and 3). We found that devaluation performance was intact for both groups after training of initial contingencies, but impaired for Group CONTRA after reversal. However, performance was restored by additional reversal training. Furthermore, when tested a second time after reversal training, rats in both groups demonstrated responding in accordance with the original contingencies, providing direct evidence of modulation of action selection by state. Finally, we found that external context could substitute for internal state and so could rescue responding in Group CONTRA, but only in the reinstatement test. Together, these findings suggest that animals use internal state information to guide action selection and that this information is modulated by the PF-pDMS pathway.SIGNIFICANCE STATEMENT Individuals with Parkinson's disease dementia often suffer a characteristic deficit in "cognitive flexibility." It has been suggested that neurodegeneration in the pathway between the centromedian/parafascicular thalalmic nucleus (PF) and striatum might underlie such deficits (Smith et al., 2014). In rats, we have similarly observed that a functional disconnection of the PF-posterior dorsomedial striatal pathway produces a specific impairment in the ability to alter goal-directed actions (Bradfield et al., 2013). It was suggested that this impairment could be a result of a deficit in state modulation. Here, we present four experiments that provide evidence for this hypothesis and suggest several ways (e.g., extended practice, providing external cues) in which this state modulation can be rescued.

Pulling habits out of rats: adenosine 2A receptor antagonism in dorsomedial striatum rescues meth-amphetamine-induced deficits in goal-directed action

Addiction is characterized by a persistent loss of behavioral control resulting in insensitivity to negative feedback and abnormal decision-making. Here, we investigated the influence of methamphetamine (METH)-paired contextual cues on decision-making in rats. Choice between goal-directed actions was sensitive to outcome devaluation in a saline-paired context but was impaired in the METH-paired context, a deficit that was also found when negative feedback was provided. Reductions in c-Fos-related immunoreactivity were found in dorsomedial striatum (DMS) but not dorsolateral striatum after exposure to the METH context suggesting this effect reflected a loss specifically in goal-directed control in the METH context. This reduction in c-Fos was localized to non-enkephalin-expressing neurons in the DMS, likely dopamine D1-expressing direct pathway neurons, suggesting a relative change in control by the D1-direct versus D2-indirect pathways originating in the DMS may have been induced by METH-context exposure. To test this suggestion, we infused the adenosine 2A receptor antagonist ZM241385 into the DMS prior to test to reduce activity in D2 neurons relative to D1 neurons in the hope of reducing the inhibitory output from this region of the striatum. We found that this treatment fully restored sensitivity to negative feedback in a test conducted in the METH-paired context. These results suggest that drug exposure alters decision-making by downregulation of the circuitry mediating goal-directed action, an effect that can be ameliorated by acute A_2A receptor inhibition in this circuit.

Striatal Activity and Reward Relativity: Neural Signals Encoding Dynamic Outcome Valuation

The striatum is a key brain region involved in reward processing. Striatal activity has been linked to encoding reward magnitude and integrating diverse reward outcome information. Recent work has supported the involvement of striatum in the valuation of outcomes. The present work extends this idea by examining striatal activity during dynamic shifts in value that include different levels and directions of magnitude disparity. A novel task was used to produce diverse relative reward effects on a chain of instrumental action. Rats (Rattus norvegicus) were trained to respond to cues associated with specific outcomes varying by food pellet magnitude. Animals were exposed to single-outcome sessions followed by mixed-outcome sessions, and neural activity was compared among identical outcome trials from the different behavioral contexts. Results recording striatal activity show that neural responses to different task elements reflect incentive contrast as well as other relative effects that involve generalization between outcomes or possible influences of outcome variety. The activity that was most prevalent was linked to food consumption and post-food consumption periods. Relative encoding was sensitive to magnitude disparity. A within-session analysis showed strong contrast effects that were dependent upon the outcome received in the immediately preceding trial. Significantly higher numbers of responses were found in ventral striatum linked to relative outcome effects. Our results support the idea that relative value can incorporate diverse relationships, including comparisons from specific individual outcomes to general behavioral contexts. The striatum contains these diverse relative processes, possibly enabling both a higher information yield concerning value shifts and a greater behavioral flexibility.

Role of the Perigenual Anterior Cingulate and Orbitofrontal Cortex in Contingency Learning in the Marmoset

Two learning mechanisms contribute to decision-making: goal-directed actions and the "habit" system, by which action-outcome and stimulus-response associations are formed, respectively. Rodent lesion studies and human neuroimaging have implicated both the medial prefrontal cortex (mPFC) and the orbitofrontal cortex (OFC) in the neural basis of contingency learning, a critical component of goal-directed actions, though some published findings are conflicting. We sought to reconcile the existing literature by comparing the effects of excitotoxic lesions of the perigenual anterior cingulate cortex (pgACC), a region of the mPFC, and OFC on contingency learning in the marmoset monkey using a touchscreen-based paradigm, in which the contingent relationship between one of a pair of actions and its outcome was degraded selectively. Both the pgACC and OFC lesion groups were insensitive to the contingency degradation, whereas the control group demonstrated selectively higher performance of the nondegraded action when compared with the degraded action. These findings suggest the pgACC and OFC are both necessary for normal contingency learning and therefore goal-directed behavior.

Interaction of insular cortex and ventral striatum mediates the effect of incentive memory on choice between goal-directed actions

The anterior insular cortex (IC) and the nucleus accumbens (NAc) core have been separately implicated in the selection and performance of actions based on the incentive value of the instrumental outcome. Here, we examined the role of connections between the IC and the NAc core in the performance of goal-directed actions. Rats were trained on two actions for distinct outcomes, after which one of the two outcomes was devalued by specific satiety immediately before a choice extinction test. We first confirmed the projection from the IC to the NAc core and then disconnected these structures via asymmetrical excitotoxic lesions before training. Contralateral, but not ipsilateral, disconnection of the IC and NAc core disrupted outcome devaluation. We hypothesized that communication between the IC and NAc core is necessary for the retrieval of incentive value at test. To test this, we infused the GABAA agonist muscimol into the IC and the μ-opioid receptor antagonist CTAP into the contralateral NAc before the choice extinction test. As expected, inactivation of the IC in one hemisphere and blocking μ-opioid receptors in the contralateral NAc core abolished outcome-selective devaluation. These results suggest that the IC and NAc core form part of a circuit mediating the retrieval of outcome values and the subsequent choice between goal-directed actions based on those values.

Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer

Outcome-specific Pavlovian-instrumental transfer (PIT) demonstrates the way that reward-related cues influence choice between instrumental actions. The nucleus accumbens shell (NAc-S) contributes critically to this effect, particularly through its output to the rostral medial ventral pallidum (VP-m). Using rats, we investigated in two experiments the role in the PIT effect of the two major outputs of this VP-m region innervated by the NAc-S, the mediodorsal thalamus (MD) and the ventral tegmental area (VTA). First, two retrograde tracers were injected into the MD and VTA to compare the neuronal activity of the two populations of projection neurons in the VP-m during PIT relative to controls. Second, the functional role of the connection between the VP-m and the MD or VTA was assessed using asymmetrical pharmacological manipulations before a PIT test. It was found that, whereas neurons in the VP-m projecting to the MD showed significantly more neuronal activation during PIT than those projecting to the VTA, neuronal activation of these latter neurons correlated with the size of the PIT effect. Disconnection of the two pathways during PIT also revealed different deficits in performance: disrupting the VP-m to MD pathway removed the response biasing effects of reward-related cues, whereas disrupting the VP-m to VTA pathway preserved the response bias but altered the overall rate of responding. The current results therefore suggest that the VP-m exerts distinct effects on the VTA and MD and that these latter structures mediate the motivational and cognitive components of specific PIT, respectively.

Distinct fronto-striatal couplings reveal the double-faced nature of response-outcome relations in instruction-based learning

Higher species commonly learn novel behaviors by evaluating retrospectively whether actions have yielded desirable outcomes. By relying on explicit behavioral instructions, only humans can use an acquisition shortcut that prospectively specifies how to yield intended outcomes under the appropriate stimulus conditions. A recent and largely unexplored hypothesis suggests that striatal areas interact with lateral prefrontal cortex (LPFC) when novel behaviors are learned via explicit instruction, and that regional subspecialization exists for the integration of differential response-outcome contingencies into the current task model. Behaviorally, outcome integration during instruction-based learning has been linked to functionally distinct performance indices. This includes (1) compatibility effects, measured in a postlearning test procedure probing the encoding strength of outcome-response (O-R) associations, and (2) increasing response slowing across learning, putatively indicating active usage of O-R associations for the online control of goal-directed action. In the present fMRI study, we examined correlations between these behavioral indices and the dynamics of fronto-striatal couplings in order to mutually constrain and refine the interpretation of neural and behavioral measures in terms of separable subprocesses during outcome integration. We found that O-R encoding strength correlated with LPFC-putamen coupling, suggesting that the putamen is relevant for the formation of both S-R habits and habit-like O-R associations. By contrast, response slowing as a putative index of active usage of O-R associations correlated with LPFC-caudate coupling. This finding highlights the relevance of the caudate for the online control of goal-directed action also under instruction-based learning conditions, and in turn clarifies the functional relevance of the behavioral slowing effect.

The acquisition of goal-directed actions generates opposing plasticity in direct and indirect pathways in dorsomedial striatum

A cortical-basal ganglia network involving, particularly, the posterior region of dorsomedial striatum (DMS) has been implicated in the acquisition of goal-directed actions; however, no direct evidence of learning-related plasticity in this striatal region has been reported, nor is it known whether, or which, specific cell types are involved in this learning process. The striatum is primarily composed of two classes of spiny projection neurons (SPNs): the striatonigral and striatopallidal SPNs, which express dopamine D1 and D2 receptors, respectively. Here we establish that, in mice, the acquisition of goal-directed actions induced plasticity in both D1- and D2-SPNs specifically in the DMS and, importantly, that these changes were in opposing directions; after learning, AMPA/NMDA ratios were increased in D1-SPNs and reduced in the D2-SPNs in the DMS. Such opposing plasticity could provide the basis for rapidly rebiasing the control of task-specific actions, and its dysregulation could underlie disorders associated with striatal function.

δ-Opioid receptors in the accumbens shell mediate the influence of both excitatory and inhibitory predictions on choice

BACKGROUND AND PURPOSE

Stimuli that predict rewarding events can control choice between future actions, and this control could be mediated by δ-opioid receptors in the nucleus accumbens shell (NAc-S). Stimuli predicting the absence of important events can also guide choice, although it remains unknown whether they do so via changes in an accumbal δ-opioid receptor-related process.

EXPERIMENTAL APPROACH

δ-opioid receptor-eGFP mice were trained to perform two instrumental actions that delivered different food outcomes. Choice between the two actions was then tested in the presence of stimuli paired with either the delivery or the non-delivery of each of the two outcomes. Bilateral infusions of the δ-opioid receptor antagonist naltrindole into the NAc-S were used to determine the role of these receptors at the time of choice and δ-opioid receptor expression in the NAc-S used to assess functional activity.

KEY RESULTS

A stimulus predicting a specific outcome biased choice performance towards the action previously earning that same outcome. In contrast, a stimulus signalling the absence of that outcome biased performance away from the action that delivered that outcome towards actions associated with the absence of that outcome. Both effects were associated with increased δ-opioid receptor expression on the membrane of cholinergic interneurons within the NAc-S. Furthermore, both effects were blocked by naltrindole infused into the NAc-S.

CONCLUSIONS AND IMPLICATIONS

These findings suggest that δ-opioid receptors in the NAc-S were involved in the effects of predictive learning on choice between actions, whether those predictions involve the presence or absence of specific rewarding events.

LINKED ARTICLES

This article is part of a themed section on Opioids: New Pathways to Functional Selectivity. To view the other articles in this section visit http://dx.doi.org/10.1111/bph.2015.172.issue-2.

Seeing what you want to see: priors for one's own actions represent exaggerated expectations of success

People perceive the consequences of their own actions differently to how they perceive other sensory events. A large body of psychology research has shown that people also consistently overrate their own performance relative to others, yet little is known about how these "illusions of superiority" are normally maintained. Here we examined the visual perception of the sensory consequences of self-generated and observed goal-directed actions. Across a series of visuomotor tasks, we found that the perception of the sensory consequences of one's own actions is more biased toward success relative to the perception of observed actions. Using Bayesian models, we show that this bias could be explained by priors that represent exaggerated predictions of success. The degree of exaggeration of priors was unaffected by learning, but was correlated with individual differences in trait optimism. In contrast, when observing these actions, priors represented more accurate predictions of the actual performance. The results suggest that the brain internally represents optimistic predictions for one's own actions. Such exaggerated predictions bind the sensory consequences of our own actions with our intended goal, explaining how it is that when acting we tend to see what we want to see.

Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

The ability to learn contingencies between actions and outcomes in a dynamic environment is critical for flexible, adaptive behavior. Goal-directed actions adapt to changes in action-outcome contingencies as well as to changes in the reward-value of the outcome. When networks involved in reward processing and contingency learning are maladaptive, this fundamental ability can be lost, with detrimental consequences for decision-making. Impaired decision-making is a core feature in a number of psychiatric disorders, ranging from depression to schizophrenia. The argument can be developed, therefore, that seemingly disparate symptoms across psychiatric disorders can be explained by dysfunction within common decision-making circuitry. From this perspective, gaining a better understanding of the neural processes involved in goal-directed action, will allow a comparison of deficits observed across traditional diagnostic boundaries within a unified theoretical framework. This review describes the key processes and neural circuits involved in goal-directed decision-making using evidence from animal studies and human neuroimaging. Select studies are discussed to outline what we currently know about causal judgments regarding actions and their consequences, action-related reward evaluation, and, most importantly, how these processes are integrated in goal-directed learning and performance. Finally, we look at how adaptive decision-making is impaired across a range of psychiatric disorders and how deepening our understanding of this circuitry may offer insights into phenotypes and more targeted interventions.

Implication of the anterior commissure in the allocation of attention to action

Our recent target article on the allocation of attention to action (herein called the AAA model; Franz, 2012) considered implicated subcortical processes and networks in people with intact corpus callosum (CC) and people without a CC due to commissurotomy or callosotomy. However, a small error in print—namely that the term “commissurotomy” was printed in place of “callosotomy” in some instances—led us to further explore whether any key functional roles have been attributed to the two primary cortical commissures (the anterior and posterior commissures) which remain intact in people with callosotomy, and if so, whether those would be relevant to our current AAA framework. Although existing evidence is sparse, here we consider the hypothesis that the anterior commissure (AC) is a remnant fiber tract which has been largely replaced with evolution of the CC (and we do not herein discuss the posterior commissure further). Indeed, a dearth of studies is available on the AC, calling the need for further research. Herein, we briefly review literature on the AC in humans and then propose a method that might be worthwhile to pursue in future studies.