Corticostriatal foundations of habits

A role for the dorsolateral striatum in prospective action control

The dorsolateral striatum (DLS) is important for performing actions persistently, even when it becomes suboptimal, reflecting a function that is reflexive and habitual. However, there are also ways in which persistent behaviors can result from a more prospective, planning mode of behavior. To help tease apart these possibilities for DLS function, we trained animals to perform a lever press for reward and then inhibited the DLS in key test phases: as the task shifted from a 1-press to a 3-press rule (upshift), as the task was maintained, as the task shifted back to the one-press rule (downshift), and when rewards came independent of pressing. During DLS inhibition, animals always favored their initially learned strategy to press just once, particularly so during the free-reward period. DLS inhibition surprisingly changed performance speed bidirectionally depending on the task shifts. DLS inhibition thus encouraged habitual behavior, suggesting it could normally help adapt to changing conditions.

Characterizing Human Habits in the Lab

Habits pose a fundamental puzzle for those aiming to understand human behavior. They pervade our everyday lives and dominate some forms of psychopathology but are extremely hard to elicit in the lab. In this Registered Report, we developed novel experimental paradigms grounded in computational models, which suggest that habit strength should be proportional to the frequency of behavior and, in contrast to previous research, independent of value. Specifically, we manipulated how often participants performed responses in two tasks varying action repetition without, or separately from, variations in value. Moreover, we asked how this frequency-based habitization related to value-based operationalizations of habit and self-reported propensities for habitual behavior in real life. We find that choice frequency during training increases habit strength at test and that this form of habit shows little relation to value-based operationalizations of habit. Our findings empirically ground a novel perspective on the constituents of habits and suggest that habits may arise in the absence of external reinforcement. We further find no evidence for an overlap between different experimental approaches to measuring habits and no associations with self-reported real-life habits. Thus, our findings call for a rigorous reassessment of our understanding and measurement of human habitual behavior in the lab.

The neural substrates of how model-based learning affects risk taking: Functional coupling between right cerebellum and left caudate

Higher executive control capacity allows people to appropriately evaluate risk and avoid both excessive risk aversion and excessive risk-taking. The neural mechanisms underlying this relationship between executive function and risk taking are still unknown. We used voxel-based morphometry (VBM) analysis combined with resting-state functional connectivity (rs-FC) to evaluate how one component of executive function, model-based learning, relates to risk taking. We measured individuals' use of the model-based learning system with the two-step task, and risk taking with the Balloon Analogue Risk Task. Behavioral results indicated that risk taking was positively correlated with the model-based weighting parameter ω. The VBM results showed a positive association between model-based learning and gray matter volume in the right cerebellum (RCere) and left inferior parietal lobule (LIPL). Functional connectivity results suggested that the coupling between RCere and the left caudate (LCAU) was correlated with both model-based learning and risk taking. Mediation analysis indicated that RCere-LCAU functional connectivity completely mediated the effect of model-based learning on risk taking. These results indicate that learners who favor model-based strategies also engage in more appropriate risky behaviors through interactions between reward-based learning, error-based learning and executive control subserved by a caudate, cerebellar and parietal network.

The Willpower Paradox: Possible and Impossible Conceptions of Self-Control

Self-control denotes the ability to override current desires to render behavior consistent with long-term goals. A key assumption is that self-control is required when short-term desires are transiently stronger (more preferred) than long-term goals and people would yield to temptation without exerting self-control. We argue that this widely shared conception of self-control raises a fundamental yet rarely discussed conceptual paradox: How is it possible that a person most strongly desires to perform a behavior (e.g., eat chocolate) and at the same time desires to recruit self-control to prevent themselves from doing it? A detailed analysis reveals that three common assumptions about self-control cannot be true simultaneously. To avoid the paradox, any coherent theory of self-control must abandon either the assumption (a) that recruitment of self-control is an intentional process, or (b) that humans are unitary agents, or (c) that self-control consists in overriding the currently strongest desire. We propose a taxonomy of different kinds of self-control processes that helps organize current theories according to which of these assumptions they abandon. We conclude by outlining unresolved questions and future research perspectives raised by different conceptions of self-control and discuss implications for the question of whether self-control can be considered rational.

Cortical glutamate and GABA are related to compulsive behaviour in individuals with obsessive compulsive disorder and healthy controls

There has been little analysis of neurochemical correlates of compulsive behaviour to illuminate its underlying neural mechanisms. We use 7-Tesla proton magnetic resonance spectroscopy (¹H-MRS) to assess the balance of excitatory and inhibitory neurotransmission by measuring glutamate and GABA levels in anterior cingulate cortex (ACC) and supplementary motor area (SMA) of healthy volunteers and participants with Obsessive-Compulsive Disorder (OCD). Within the SMA, trait and clinical measures of compulsive behaviour are related to glutamate levels, whereas a behavioural index of habitual control correlates with the glutamate:GABA ratio. Participants with OCD also show the latter relationship in the ACC while exhibiting elevated glutamate and lower GABA levels in that region. This study highlights SMA mechanisms of habitual control relevant to compulsive behaviour, common to the healthy sub-clinical and OCD populations. The results also demonstrate additional involvement of anterior cingulate in the balance between goal-directed and habitual responding in OCD.

Human habit neural circuitry may be perturbed in eating disorders

Circuit-based mechanisms mediating the development and execution of habitual behaviors involve complex cortical-striatal interactions that have been investigated in animal models and more recently in humans. However, how human brain circuits implicated in habit formation may be perturbed in psychiatric disorders remains unclear. First, we identified the locations of the sensorimotor putamen and associative caudate in the human brain using probabilistic tractography from Human Connectome Project data. We found that multivariate connectivity of the sensorimotor putamen was altered in humans with binge eating disorder and bulimia nervosa and that the degree of alteration correlated with severity of disordered eating behavior. Furthermore, the extent of this circuit aberration correlated with mean diffusivity in the sensorimotor putamen and decreased basal dopamine D2/3 receptor binding potential in the striatum, consistent with previously reported microstructural changes and dopamine signaling mediating habit learning in animal models. Our findings suggest a neural circuit that links habit learning and binge eating behavior in humans, which could, in part, explain the treatment-resistant behavior common to eating disorders and other psychiatric conditions.

Changes in global functional network properties predict individual differences in habit formation

Prior evidence suggests that sensorimotor regions play a crucial role in habit formation. Yet, whether and how their global functional network properties might contribute to a more comprehensive characterization of habit formation still remains unclear. Capitalizing on advances in Elastic Net regression and predictive modeling, we examined whether learning-related functional connectivity alterations distributed across the whole brain could predict individual habit strength. Using the leave-one-subject-out cross-validation strategy, we found that the habit strength score of the novel unseen subjects could be successfully predicted. We further characterized the contribution of both, individual large-scale networks and individual brain regions by calculating their predictive weights. This highlighted the pivotal role of functional connectivity changes involving the sensorimotor network and the cingulo-opercular network in subject-specific habit strength prediction. These results contribute to the understanding the neural basis of human habit formation by demonstrating the importance of global functional network properties especially also for predicting the observable behavioral expression of habits.

Adolescent social isolation shifts the balance of decision-making strategy from goal-directed action to habitual response in adulthood via suppressing the excitatory neurotransmission onto the direct pathway of the dorsomedial striatum

Adverse experience, such as social isolation, during adolescence is one of the major causes of neuropsychiatric disorders that extend from adolescence into adulthood, such as substance addiction, obsessive-compulsive disorder, and eating disorders leading to obesity. A common behavioral feature of these neuropsychiatric disorders is a shift in the balance of decision-making strategy from goal-directed action to habitual response. This study has verified that adolescent social isolation directly shifts the balance of decision-making strategy from goal-directed action to habitual response, and that it cannot be reversed by simple regrouping. This study has further revealed that adolescent social isolation induces a suppression in the excitatory neurotransmission onto the direct-pathway medium spiny neurons of the dorsomedial striatum (DMS), and that chemogenetically compensating this suppression effect shifts the balance of decision-making strategy from habitual response back to goal-directed action. These findings suggest that the plasticity in the DMS causes the shift in the balance of decision-making strategy, which would potentially help to develop a general therapy to treat the various neuropsychiatric disorders caused by adolescent social isolation. Such a study is especially necessary under the circumstances that social distancing and lockdown have caused during times of world-wide, society-wide pandemic.

Statistical learning of target location and distractor location rely on different mechanisms during visual search

More studies have demonstrated that people have the capacity to learn and make use of environmental regularities. This capacity is known as statistical learning (SL). Despite rich empirical findings, it is not clear how the two forms of SL (SL of target location and SL of distractor location) influence visual search and whether they rely on the shared cognitive mechanism. In Experiment 1 and Experiment 2, we manipulated the probability of target location and the probability of distractor location, respectively. The results suggest that attentional guidance (they referred to overt attention) may mainly contribute to the SL effect of the target location and the distractor location, which is in line with the notion of priority mapping. To a small extent, facilitation of response selection may also contribute to the SL effect of the target location but does not contribute to the SL effect of the distractor location. However, the main difference between the two kinds of SL occurred in the early stage (it involved covert attention). Together, our findings indicate that the two forms of SL reflect partly shared and partly independent cognitive mechanisms.

Task history dictates how the dorsolateral striatum controls action strategy and vigor

The dorsolateral striatum (DLS) is linked to the learning and honing of action routines. However, the DLS is also important for performing behaviors that have been successful in the past. The learning function can be thought of as prospective, helping to plan ongoing actions to be efficient and often optimal. The performance function is more retrospective, helping the animal continue to behave in a way that had worked previously. How the DLS manages this all is curious. What happens when a learned behavior becomes sub-optimal due to environment changes. In this case, the prospective function of the DLS would cause animals to (adaptively) learn and plan more optimal actions. In contrast, the retrospective function would cause animals to (maladaptively) favor the old behavior. Here we find that, during a change in learned task rules, DLS inhibition causes animals to adjust less rapidly to the new task (and to behave less vigorously) in a 'maladaptive' way. Yet, when the task is changed back to the initially learned rules, DLS inhibition instead causes a rapid and vigorous adjustment of behavior in an 'adaptive' way. These results show that inhibiting the DLS biases behavior towards initially acquired strategies, implying a more retrospective outlook in action selection when the DLS is offline. Thus, an active DLS could encourage planning and learning action routines more prospectively. Moreover, the DLS control over behavior can appear to be either advantageous/flexible or disadvantageous/inflexible depending on task context, and its control over vigor can change depending on task context.

Significant Statement

Basal ganglia networks aid behavioral learning (a prospective planning function) but also favor the use of old behaviors (a retrospective performance function), making it unclear what happens when learned behaviors become suboptimal. Here we inhibit the dorsolateral striatum (DLS) as animals encounter a change in task rules, and again when they shift back to those learned task rules. DLS inhibition reduces adjustment to new task rules (and reduces behavioral vigor), but it increases adjustment back to the initially learned task rules later (and increases vigor). Thus, in both cases, DLS inhibition favored the use of the initially learned behavioral strategy, which could appear either maladaptive or adaptive. We suggest that the DLS might promote a prospective orientation of action control.

Imbalance between the caudate and putamen connectivity in obsessive-compulsive disorder

•

The imbalance between the caudate and putamen connectivity in OCD patient arises from the abnormal connection of caudate.

•

The abnormal caudate connectivity mainly results from the outward extension of cortico-striato-thalamo-cortical loop.

•

The caudate connectivity of OCD patients is negatively associated with their task-switch performance.

Background

Compulsive behaviors in obsessive–compulsive disorder (OCD) have been suggested to result from an imbalance in cortico-striatal connectivity. However, the nature of this impairment, the relative involvement of different striatal areas, their imbalance in genetically related but unimpaired individuals, and their relationship with cognitive dysfunction in OCD patients, remain unknown.

Methods

In the current study, striatal (i.e., caudate and putamen) whole-brain connectivity was computed in a sample of OCD patients (OCD, n = 62), unaffected first-degree relatives (UFDR, n = 53) and healthy controls (HC, n = 73) by ROI-based resting-state functional magnetic resonance imaging (rs-fMRI). A behavioral task switch paradigm outside of the scanner was also performed to measure cognitive flexibility in OCD patients.

Results

There were significantly increased strengths (Z-transformed Pearson correlation coefficient) in caudate connectivity in OCD patients. A significant correlation between the two types of connectivity strengths in the relevant regions was observed only in the OCD patient group. Furthermore, the caudate connectivity of patients was negatively associated with their task-switch performance.

Conclusions

The imbalance between the caudate and putamen connectivity, arising from the abnormal increase of caudate activity, may serve as a clinical characteristic for obsessive–compulsive disorder.

Imbalance in Functional and Structural Connectivity Underlying Goal-Directed and Habitual Learning Systems in Obsessive-Compulsive Disorder

An imbalance between the goal-directed and habitual learning systems has been proposed to underlie compulsivity in obsessive-compulsive disorder (OCD). In addition, the overall balance between these systems may be influenced by stress hormones. We examined the multimodal networks underlying these dual learning systems. Both functional and structural measures indicated reduced connectivity within the goal-directed subnetwork (FC: P = 0.042; SC-FN: P = 0.014) and reduced connectivity between the goal-directed and habitual subnetworks (FC: P = 0.014; SC-FA: P = 0.052), but no differences within the habitual subnetwork in patients with OCD compared with controls. Path modeling indicated that anatomical connectivity in the goal-directed subnetwork influenced compulsive symptoms (R2 = 0.41), whereas functional connectivity within the habit subnetwork and between goal-directed and habitual subnetworks influenced obsessive symptoms (R2 = 0.63). In addition, the relationship between anatomical connectivity in the goal-directed subnetwork and compulsion was moderated by the stress hormone ACTH (adrenocorticotropic hormone), such that at low levels of ACTH greater connectivity resulted in lower compulsion, but at high levels of ACTH this relationship was reversed. These results provide new insights into the neural correlates of the imbalance between dual learning systems, and their relationship with symptom dimensions in patients with OCD. It may further support the reconceptualization of OCD as "compulsive-obsessive disorder," with a greater focus on the transdiagnostic dimension of compulsivity.

Is probabilistic cuing of visual search an inflexible attentional habit? A meta-analytic review

In studies on probabilistic cuing of visual search, participants search for a target among several distractors and report some feature of the target. In a biased stage the target appears more frequently in one specific area of the search display. Eventually, participants become faster at finding the target in that rich region compared to the sparse region. In some experiments, this stage is followed by an unbiased stage, where the target is evenly located across all regions of the display. Despite this change in the spatial distribution of targets, search speed usually remains faster when the target is located in the previously rich region. The persistence of the bias even when it is no longer advantageous has been taken as evidence that this phenomenon is an attentional habit. The aim of this meta-analysis was to test whether the magnitude of probabilistic cuing decreases from the biased to the unbiased stage. A meta-analysis of 42 studies confirmed that probabilistic cuing during the unbiased stage was roughly half the size of cuing during the biased stage, and this decrease persisted even after correcting for publication bias. Thus, the evidence supporting the claim that probabilistic cuing is an attentional habit might not be as compelling as previously thought.

Lower-level associations in Gilles de la Tourette syndrome: Convergence between hyperbinding of stimulus and response features and procedural hyperfunctioning theories

Gilles de la Tourette syndrome (GTS) can be characterized by enhanced cognitive functions related to creating, modifying and maintaining connections between stimuli and responses (S-R links). Specifically, two areas, procedural sequence learning and, as a novel finding, also event file binding, show converging evidence of hyperfunctioning in GTS. In this review, we describe how these two enhanced functions can be considered as cognitive mechanisms behind habitual behaviour, such as tics in GTS. Moreover, the presence of both procedural sequence learning and event file binding hyperfunctioning in the same disorder can be treated as evidence for their functional connections, even beyond GTS. Importantly though, we argue that hyperfunctioning of event file binding and procedural learning are not interchangeable: they have different time scales, different sensitivities to potential impairment in action sequencing and distinguishable contributions to the cognitive profile of GTS. An integrated theoretical account of hyperbinding and hyperlearning in GTS allows to formulate predictions for the emergence, activation and long-term persistence of tics in GTS.

Depression in the Direct Pathway of the Dorsomedial Striatum Permits the Formation of Habitual Action

In order to achieve optimal outcomes in an ever-changing environment, humans and animals generally manage their action control via either goal-directed action or habitual action. These two action strategies are thought to be encoded in distinct parallel circuits in the dorsal striatum, specifically, the posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS), respectively. The striatum is primarily composed of two subtypes of medium spiny neurons (MSNs): the direct-pathway striatonigral and the indirect-pathway striatopallidal MSNs. MSN-subtype-specific synaptic plasticity in the DMS and the DLS has been revealed to underlie goal-directed action and habitual action, respectively. However, whether any MSN-subtype-specific synaptic plasticity in the DMS is associated with habitual action, and if so, whether the synaptic plasticity affects the formation of habitual action, are not known. This study demonstrates that postsynaptic depression in the excitatory synapses of the direct-pathway striatonigral MSNs in the DMS is formed after habit learning. Moreover, chemogenetically rescuing this depression compromises the acquisition, but not the expression, of habitual action. These findings reveal that an MSN-subtype-specific synaptic plasticity in the DMS affects habitual action and suggest that plasticity in the DMS as well as in the DLS contributes to the formation of habitual action.

White Matter Alterations of the Goal-Directed System in Patients With Obsessive-Compulsive Disorder and Their Unaffected First-Degree Relatives

BACKGROUND

It has been postulated that the neurobiological mechanism responsible for the onset of symptoms of obsessive-compulsive disorder (OCD), especially compulsive behavior, is related to alterations of the goal-directed and habitual learning systems. However, little is known about whether changes in these learning systems co-occur with changes in the white matter structure of patients with OCD and their unaffected first-degree relatives (UFDRs).

METHODS

Diffusion tensor imaging data were acquired from 32 patients with OCD (21 male), 32 UFDRs (16 male), and 32 healthy control subjects (16 male). White matter tracts in the goal-directed and habitual networks were reconstructed with seed-based probabilistic tractography. Partial least squares path modeling was used to measure the covariation between white matter connectivity, psychiatric symptoms, and cognitive flexibility.

RESULTS

Patients with OCD showed reduced connectivity in the fiber tracts within the goal-directed but not within the habitual network compared with healthy control subjects. Using partial least squares path modeling, patients' symptoms were negatively associated with connectivity within the goal-directed but not within the habitual network. Cognitive flexibility was correlated negatively with caudate-dorsolateral prefrontal cortex tracts in patients with OCD. UFDRs also exhibited reduced white matter connectivity in the goal-directed network.

CONCLUSIONS

These findings suggest that the balance of learning systems in OCD may be disrupted, mainly impairing white matter in the goal-directed network. Alterations of the goal-directed network could explain overt symptoms and impaired cognitive flexibility in patients with OCD. Similar alterations in the goal-directed network are present in UFDRs. The impaired goal-directed system may be an endophenotype of OCD.

Impairment in the goal-directed corticostriatal learning system as a biomarker for obsessive-compulsive disorder

BACKGROUND

Compulsive behaviors in obsessive-compulsive disorder (OCD) have been related to impairment within the associative cortical-striatal system connecting the caudate and prefrontal cortex that underlies consciously-controlled goal-directed learning and behavior. However, little is known whether this impairment may serve as a biomarker for vulnerability to OCD.

METHODS

Using resting-state functional magnetic resonance imaging (fMRI), we employed Granger causality analysis (GCA) to measure effective connectivity (EC) in previously validated striatal sub-regions, including the caudate, putamen, and the nucleus accumbens, in 35 OCD patients, 35 unaffected first-degree relatives and 35 matched healthy controls.

RESULTS

Both OCD patients and their first-degree relatives showed greater EC than controls between the left caudate and the orbital frontal cortex (OFC). Both OCD patients and their first-degree relatives showed lower EC than controls between the left caudate and lateral prefrontal cortex. These results are consistent with findings from task-related fMRI studies which found impairment in the goal-directed system in OCD patients.

CONCLUSIONS

The same changes in EC were present in both OCD patients and their unaffected first-degree relatives suggest that impairment in the goal-directed learning system may be a biomarker for OCD.

Left Inferior Frontal Gyrus Integrates Multisensory Information in Category Learning

Humans are able to categorize things they encounter in the world (e.g., a cat) by integrating multisensory information from the auditory and visual modalities with ease and speed. However, how the brain learns multisensory categories remains elusive. The present study used functional magnetic resonance imaging to investigate, for the first time, the neural mechanisms underpinning multisensory information-integration (II) category learning. A sensory-modality-general network, including the left insula, right inferior frontal gyrus (IFG), supplementary motor area, left precentral gyrus, bilateral parietal cortex, and right caudate and globus pallidus, was recruited for II categorization, regardless of whether the information came from a single modality or from multiple modalities. Putamen activity was higher in correct categorization than incorrect categorization. Critically, the left IFG and left body and tail of the caudate were activated in multisensory II categorization but not in unisensory II categorization, which suggests this network plays a specific role in integrating multisensory information during category learning. The present results extend our understanding of the role of the left IFG in multisensory processing from the linguistic domain to a broader role in audiovisual learning.

Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling

A well-established notion in cognitive neuroscience proposes that multiple brain systems contribute to choice behaviour. These include: (1) a model-free system that uses values cached from the outcome history of alternative actions, and (2) a model-based system that considers action outcomes and the transition structure of the environment. The widespread use of this distinction, across a range of applications, renders it important to index their distinct influences with high reliability. Here we consider the two-stage task, widely considered as a gold standard measure for the contribution of model-based and model-free systems to human choice. We tested the internal/temporal stability of measures from this task, including those estimated via an established computational model, as well as an extended model using drift-diffusion. Drift-diffusion modeling suggested that both choice in the first stage, and RTs in the second stage, are directly affected by a model-based/free trade-off parameter. Both parameter recovery and the stability of model-based estimates were poor but improved substantially when both choice and RT were used (compared to choice only), and when more trials (than conventionally used in research practice) were included in our analysis. The findings have implications for interpretation of past and future studies based on the use of the two-stage task, as well as for characterising the contribution of model-based processes to choice behaviour.