Subregional specificity in human striatal habit learning: a meta-analytic review of the fMRI literature

Maturation of striatal dopamine supports the development of habitual behavior through adolescence

Developmental trajectories during the transition from adolescence to adulthood contribute to the establishment of stable, adult forms of operation. Understanding the neural mechanisms underlying this transition is crucial for identifying variability in normal development and the onset of psychiatric disorders, which typically emerge during this time. Habitual behaviors can serve as a model for understanding brain mechanisms underlying the stabilization of adult behavior, while also conferring risk for psychopathologies. Dopaminergic (DA) processes in the basal ganglia are thought to facilitate the formation of habits; however, developmental trajectories of habits and the brain systems supporting them have not been characterized in vivo in developing humans. The current study examined trajectories of habitual behavior from adolescence to adulthood and sought to understand how the maturing striatal DA system may act as a potential mechanism in the process of habit formation. We used data from two longitudinal studies (combined n = 217, 10 - 32 years of age, 1-3 visits each, 320 total sessions) to characterize normative developmental trajectories of basal ganglia tissue iron concentration (a proxy for DA-related neurophysiology) and goal-direct and habitual control behaviors in a two-stage decision-making task. Tissue iron concentrations across the basal ganglia and habitual responding during the two-stage sequential decision-making task both increased with age (all p < 0.001). Importantly, habitual responding was associated with tissue iron concentrations in the putamen (F = 4.34, p = 0.014), such that increases in habitual responding were supported by increases in putamen tissue iron concentration during childhood through late adolescence. Exploratory analyses of further subdivisions of anatomical regions found that this association was specific to the posterior putamen. These results provide novel evidence in humans that habitual behavior continues to mature into adulthood and may be supported by increased specialization of reward systems.

The well-worn route revisited: Striatal and hippocampal system contributions to familiar route navigation

Classic research has shown a division in the neuroanatomical structures that support flexible (e.g., short-cutting) and habitual (e.g., familiar route following) navigational behavior, with hippocampal-caudate systems associated with the former and putamen systems with the latter. There is, however, disagreement about whether the neural structures involved in navigation process particular forms of spatial information, such as associations between constellations of cues forming a cognitive map, versus single landmark-action associations, or alternatively, perform particular reinforcement learning algorithms that allow the use of different spatial strategies, so-called model-based (flexible) or model-free (habitual) forms of learning. We sought to test these theories by asking participants (N = 24) to navigate within a virtual environment through a previously learned, 9-junction route with distinctive landmarks at each junction while undergoing functional magnetic resonance imaging (fMRI). In a series of probe trials, we distinguished knowledge of individual landmark-action associations along the route versus knowledge of the correct sequence of landmark-action associations, either by having absent landmarks, or "out-of-sequence" landmarks. Under a map-based perspective, sequence knowledge would not require hippocampal systems, because there are no constellations of cues available for cognitive map formation. Within a learning-based model, however, responding based on knowledge of sequence would require hippocampal systems because prior context has to be utilized. We found that hippocampal-caudate systems were more active in probes requiring sequence knowledge, supporting the learning-based model. However, we also found greater putamen activation in probes where navigation based purely on sequence memory could be planned, supporting models of putamen function that emphasize its role in action sequencing.

Greater reliance on model-free learning in adolescent anorexia nervosa: An examination of dual-system reinforcement learning

Alterations in learning and decision-making systems are thought to contribute to core features of anorexia nervosa (AN), a psychiatric disorder characterized by persistent dietary restriction and weight loss. Instrumental learning theory identifies a dual-system of habit and goal-directed decision-making, linked to model-free and model-based reinforcement learning algorithms. Difficulty arbitrating between these systems, resulting in an over-reliance on one strategy over the other, has been implicated in compulsivity and extreme goal pursuit, both of which are observed in AN. Characterizing alterations in model-free and model-based systems, and their neural correlates, in AN may clarify mechanisms contributing to symptom heterogeneity (e.g., binge/purge symptoms). This study tested whether adolescents with restricting AN (AN-R; n = 36) and binge/purge AN (AN-BP; n = 20) differentially utilized model-based and model-free learning systems compared to a healthy control group (HC; n = 28) during a Markov two-step decision-making task under conditions of reward and punishment. Associations between model-free and model-based learning and resting-state functional connectivity between neural regions of interest, including orbitofrontal cortex (OFC), nucleus accumbens (NAcc), putamen, and sensory motor cortex (SMC) were examined. AN-R showed higher utilization of model-free learning compared to HC for reward, but attenuated model-free and model-based learning for punishment. In AN-R only, higher model-based learning was associated with stronger OFC-to-left NAcc functional connectivity, regions linked to goal-directed behavior. Greater utilization of model-free learning for reward in AN-R may differentiate this group, particularly during adolescence, and facilitate dietary restriction by prioritizing habitual control in rewarding contexts.

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

Measuring context-response associations that drive habits

People achieve important life outcomes of health, financial security, and productivity by repeating operant behavior. To identify whether such operants reflect goal pursuit or habit, the present research introduces a new paradigm that yields objective measures of learning and controls for the motivations of goal pursuit. In two experiments, participants practiced a sequential task of making sushi and then completed a test of the strength of cue-response (habit) associations in memory. Finally, they repeated the sushi task without instructions while under cognitive load (designed to impede deliberation about goals). As predicted, greater task practice yielded stronger cue-response associations, which in turn promoted task success. Practice did not improve performance by enhancing goal intentions or other task motivations. We conclude that repetition facilitates performance by creating mental associations that automatically activate practiced, habitual responses upon perception of recurring context cues.

Individual Differences in Corticostriatal White-matter Tracts Predict Successful Daily-life Routine Formation

Despite good intentions, people often fail to cross the "intention-behavior gap," especially when goal achievement requires repeated action. To bridge this gap, the formation of automatized routines may be crucial. However, people may differ in the tendency to switch from goal-directed toward habitual control. To shed light on why some people succeed in forming routines while others struggle, the present study related the automatization of a novel, daily routine to individual differences in white-matter connectivity in corticostriatal networks that have been implicated in goal-directed and habitual control. Seventy-seven participants underwent diffusion-weighted imaging and formed the daily routine of taking a (placebo) pill for 3 weeks. Pill intake was measured by electronic pill boxes, and participants filled out a daily online questionnaire on the subjective automaticity of this behavior. Automatization of pill intake was negatively related to striatal (mainly caudate) connectivity with frontal goal-directed and cognitive control regions, namely, ventromedial pFC and anterior cingulate gyrus. Furthermore, daily pill intake was positively related to individual differences in striatal (mainly caudate) connectivity with cognitive control regions, including dorsolateral and anterior pFC. Therefore, strong control networks may be relevant for implementing a new routine but may not benefit its automatization. We also show that habit tendency (assessed with an outcome-devaluation task), conscientiousness, and daily life regularity were positively related to routine automatization. This translational study moves the field of habit research forward by relating self-reported routine automatization to individual differences in performance on an experimental habit measure and to brain connectivity.

NMDA receptor blockade in the dorsolateral striatum impairs consolidation but not retrieval of habit memory

The present study investigated whether N-methyl-d-aspartate (NMDA) receptors in the dorsolateral striatum (DLS) mediate consolidation and retrieval of habit memory. Adult male Long-Evans rats were trained in a response learning version of a water plus-maze task in which rats were reinforced to make a habitual and consistent body-turn response at the maze choice point in order to mount a hidden escape platform. Prior research indicates that acquisition, consolidation, and retrieval in this task requires DLS function. The present study consisted of two experiments. In Experiment 1, rats received intra-DLS post-training injections of the NMDA receptor antagonist 2-amino-5- phosphonopentanoic acid (AP5; 2 µg/side) to examine the role of NMDA receptors in consolidation of habit memory. In Experiment 2, different groups of rats received a single pre-retrieval injection of AP5 in the DLS (AP5; 2 µg/side) during the last day of maze training to examine the potential role of NMDA receptors in retrieval of habit memory. Results indicated that post-training intra-DLS AP5 injections impaired memory consolidation. However, administration of AP5 at the same dose that impaired consolidation had no effect on memory retrieval. The findings are consistent with previous research indicating a role for NMDA receptors in the DLS in memory consolidation, and suggest that NMDA-dependent synaptic activity in the DLS may not be a critical component of habit memory retrieval.

Spontaneous brain activity, graph metrics, and head motion related to prospective post-traumatic stress disorder trauma-focused therapy response

Introduction

Trauma-focused psychotherapy for post-traumatic stress disorder (PTSD) is effective in about half of all patients. Investigating biological systems related to prospective treatment response is important to gain insight in mechanisms predisposing patients for successful intervention. We studied if spontaneous brain activity, brain network characteristics and head motion during the resting state are associated with future treatment success.

Methods

Functional magnetic resonance imaging scans were acquired from 46 veterans with PTSD around the start of treatment. Psychotherapy consisted of trauma-focused cognitive behavioral therapy (tf-CBT), eye movement desensitization and reprocessing (EMDR), or a combination thereof. After intervention, 24 patients were classified as treatment responders and 22 as treatment resistant. Differences between groups in spontaneous brain activity were evaluated using amplitude of low-frequency fluctuations (ALFF), while global and regional brain network characteristics were assessed using a minimum spanning tree (MST) approach. In addition, in-scanner head motion was assessed.

Results

No differences in spontaneous brain activity and global network characteristics were observed between the responder and non-responder group. The right inferior parietal lobule, right putamen and left superior parietal lobule had a more central position in the network in the responder group compared to the non-responder group, while the right dorsolateral prefrontal cortex (DLPFC), right inferior frontal gyrus and left inferior temporal gyrus had a less central position. In addition, responders showed less head motion.

Discussion

These results show that areas involved in executive functioning, attentional and action processes, learning, and visual-object processing, are related to prospective PTSD treatment response in veterans. In addition, these findings suggest that involuntary micromovements may be related to future treatment success.

An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits

The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

Development and Validation of a Daily Habit Scale

Habits are defined as automatic behaviours triggered by cues and performed without awareness. They are difficult to control and mentally efficient, which contrasts with goal-directed behaviour, which is characterised by active thought, high computational effort, and the ability to modify this behaviour in response to a changing environment and contextual demands. Habits are not only defined by the frequency with which a behaviour is performed but represent a complex construct that also includes the strength and automaticity of the habitual behaviour. We report here the development and validation of a Daily Habit Scale (DHS) to assess the frequency, automaticity, and strength of daily habits in healthy individuals. Item reduction based on factor analysis resulted in a scale with 38 items grouped into eight factors explaining 52.91% of the variance. The DHS showed very good internal consistency (Cronbach alpha = 0.738) and test-retest reliability (Intraclass correlation coefficient = 0.892, p<0.001) as well as convergent and divergent reliability compared to other scales measuring habits. We found a significant effect of age, gender, anxiety, and depression on the DHS. Considering certain limitations of the DHS, such as not considering the context of performance of habits, and the absence of certain items, such as transportation use, the results of this study suggest that DHS is a reliable and valid measure of daily habits that can be used by both clinicians and researchers as a measure of daily habits.

Habits and Goals in Human Behavior: Separate but Interacting Systems

People automatically repeat behaviors that were frequently rewarded in the past in a given context. Such repetition is commonly attributed to habit, or associations in memory between a context and a response. Once habits form, contexts directly activate the response in mind. An opposing view is that habitual behaviors depend on goals. However, we show that this view is challenged by the goal independence of habits across the fields of social and health psychology, behavioral neuroscience, animal learning, and computational modeling. It also is challenged by direct tests revealing that habits do not depend on implicit goals. Furthermore, we show that two features of habit memory-rapid activation of specific responses and resistance to change-explain the different conditions under which people act on habit versus persuing goals. Finally, we tested these features with a novel secondary analysis of action-slip data. We found that habitual responses are activated regardless of goals, but they can be performed in concert with goal pursuit.

Neuropsychological spectrum in early PD: Insights from controlled and automatic behavioural regulation

Initial changes in Parkinson's disease (PD) are marked by loss of automatic movements and decline of some cognitive functions. Yet, the exact profile and extent of cognitive impairments in early stages of PD as well as their mechanisms related to automatic motor dysfunction remain unclear. Our objective was to examine the neuropsychological changes in early PD and their association to automatic and controlled modes of behavioural control. Significant relationships between early PD and cognitive dysfunction in set-shifting, abstraction ability/concept formation, processing speed, visuospatial/constructional abilities and verbal-visual memory was found. We also noted that tests with a strong effortful and controlled component were similarly affected as automatic tests by early PD, particularly those testing verbal memory, processing speed and visuospatial/constructional functions. Our findings indicate that initial stages of PD sets constraints over most of the cognitive domains normally assessed and are not easily explained in terms of either automatic or controlled mechanisms, as both appear similarly altered in early PD.

Stress, memory, and implications for major depression

The stress response comprises a phylogenetically conserved set of cognitive, physiological, and behavioral responses that evolved as a survival strategy. In this context, the memory of stressful events would be adaptive as it could avoid re-exposure to an adverse event, otherwise the event would be facilitated in positively stressful or non-distressful conditions. However, the interaction between stress and memory comprises complex responses, some of them which are not yet completely understood, and which depend on several factors such as the memory system that is recruited, the nature and duration of the stressful event, as well as the timing in which this interaction takes place. In this narrative review, we briefly discuss the mechanisms of the stress response, the main memory systems, and its neural correlates. Then, we show how stress, through the action of its biochemical mediators, influences memory systems and mnemonic processes. Finally, we make use of major depressive disorder to explore the possible implications of non-adaptive interactions between stress and memory to psychiatric disorders, as well as possible roles for memory studies in the field of psychiatry.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Upward and downward comparisons across monetary and status domains

The ability to accurately infer one's place with respect to others is crucial for social interactions. Individuals tend to evaluate their own actions and outcomes by comparing themselves to others in either an upward or downward direction. We performed two fMRI meta‐analyses on monetary (n = 39; 1,231 participants) and status (n = 23; 572 participants) social comparisons to examine how domain and the direction of comparison can modulate neural correlates of social hierarchy. Overall, both status and monetary downward comparisons activated regions associated with reward processing (striatum) while upward comparisons yielded loss‐related activity. These findings provide partial support for the common currency hypothesis in that downward and upward comparisons from both monetary and status domains resemble gains and losses, respectively. Furthermore, status upward and monetary downward comparisons revealed concordant orbitofrontal cortical activity, an area associated with evaluating the value of goals and decisions implicated in both lesion and empirical fMRI studies investigating social hierarchy. These findings may offer new insight into how people relate to individuals with higher social status and how these social comparisons deviate across monetary and social status domains.

Spatial migration of human reward processing with functional development: Evidence from quantitative meta-analyses

Functional magnetic resonance imaging (fMRI) studies have shown notable age‐dependent differences in reward processing. We analyzed data from a total of 554 children, 1,059 adolescents, and 1,831 adults from 70 articles. Quantitative meta‐analyses results show that adults engage an extended set of regions that include anterior and posterior cingulate gyri, insula, basal ganglia, and thalamus. Adolescents engage the posterior cingulate and middle frontal gyri as well as the insula and amygdala, whereas children show concordance in right insula and striatal regions almost exclusively. Our data support the notion of reorganization of function over childhood and adolescence and may inform current hypotheses relating to decision‐making across age.

Within the framework of the dual-system model, voluntary action is central to cognition

A new version of the dual-system hypothesis is described. Consistent with earlier models, the improvisational subsystem of the instrumental system, which includes the occipital cortex, inferior temporal cortex, and medial temporal cortex, especially the hippocampus, directs the construction of visual representations of the world and constructs ad-hoc responses to novel targets. The habit system, which includes the occipital cortex; parietal cortex; premotor, supplementary motor, and ventrolateral areas of frontal cortex; and the basal ganglia, especially the caudate nucleus, encodes sequences of actions and generates previously successful actions to familiar targets. However, unlike in previous dual-system models, human cognitive activity involved in task performance is not exclusively associated with one system or the other. Rather, the two systems make it possible for people to learn a variety of skills that draw on the competencies of both systems. The collective effects of these skills define human cognition. So, in contrast with earlier versions of the dual-system hypothesis, which identified the habit system solely with procedural learning and implicit improvements in task performance, the model presented here attributes a direct role in declarative-memory tasks to the habit system. Furthermore, within the model, the computational competencies of the two systems are used to construct purposeful sequences of actions-that is, skills. Human cognition is the result of the performance of these skills. Thus, voluntary action is central to human cognition.

Rethinking avoidance: Toward a balanced approach to avoidance in treating anxiety disorders

Avoidance is typically considered a maladaptive behavioral response to excessive fear and anxiety, leading to the maintenance of anxiety disorders. Exposure is a core element of cognitive-behavioral therapy for anxiety disorders. One important aspect of this treatment is repeated and prolonged exposure to a threat while discouraging patients from using avoidance strategies, such as escape or safety behaviors. We will first revisit the role of avoidance learning in the development and maintenance of anxiety disorders, including important insights from the neuroscience literature. Next, we will consider both the negative and positive aspects of avoidance for therapeutic interventions. Finally, we will explore the application of adaptive avoidance in exposure therapy for anxiety disorders. We will argue that there are occasions when avoidance behaviors can serve as effective coping strategies to enhance the person's perception of control over the environment and the potential threat. We conclude that avoidance behaviors can be a valuable therapeutic element, depending on the function of these behaviors.