Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

Subthalamic control of impulsive actions: insights from deep brain stimulation in Parkinson's disease

Control of actions allows adaptive, goal-directed behaviour. The basal ganglia, including the subthalamic nucleus, are thought to play a central role in dynamically controlling actions through recurrent negative feedback loops with the cerebral cortex. Here, we summarize recent translational studies that used deep brain stimulation to record neural activity from and apply electrical stimulation to the subthalamic nucleus in people with Parkinson's disease. These studies have elucidated spatial, spectral and temporal features of the neural mechanisms underlying the controlled delay of actions in cortico-subthalamic networks and demonstrated their causal effects on behaviour in distinct processing windows. While these mechanisms have been conceptualized as control signals for suppressing impulsive response tendencies in conflict tasks and as decision threshold adjustments in value-based and perceptual decisions, we propose a common framework linking decision-making, cognition and movement. Within this framework, subthalamic deep brain stimulation can lead to suboptimal choices by reducing the time that patients take for deliberation before committing to an action. However, clinical studies have consistently shown that the occurrence of impulse control disorders is reduced, not increased, after subthalamic deep brain stimulation surgery. This apparent contradiction can be reconciled when recognizing the multifaceted nature of impulsivity, its underlying mechanisms and modulation by treatment. While subthalamic deep brain stimulation renders patients susceptible to making decisions without proper forethought, this can be disentangled from effects related to dopamine comprising sensitivity to benefits versus costs, reward delay aversion and learning from outcomes. Alterations in these dopamine-mediated mechanisms are thought to underlie the development of impulse control disorders and can be relatively spared with reduced dopaminergic medication after subthalamic deep brain stimulation. Together, results from studies using deep brain stimulation as an experimental tool have improved our understanding of action control in the human brain and have important implications for treatment of patients with neurological disorders.

Targeting muscarinic receptors for the treatment of alcohol use disorders: Opportunities and hurdles for clinical development

Emerging evidence suggests muscarinic acetylcholine receptors represent novel targets to treat alcohol use disorder. In this review, we draw from literature across medicinal chemistry, molecular biology, addiction and learning/cognition fields to interrogate the proposition for muscarinic receptor ligands in treating various aspects of alcohol use disorder, including cognitive dysfunction, motivation to consume alcohol and relapse. In support of this proposition, we describe cholinergic dysfunction in the pathophysiology of alcohol use disorder at a network level, including alcohol-induced adaptations present in both human post-mortem brains and reverse-translated rodent models. Preclinical behavioural pharmacology implicates specific muscarinic receptors, in particular, M4 and M5 receptors, as potential therapeutic targets worthy of further interrogation. We detail how these receptors can be selectively targeted in vivo by the use of subtype-selective allosteric modulators, a strategy that overcomes the issue of targeting a highly conserved orthosteric site bound by acetylcholine. Finally, we highlight the intense pharma interest in allosteric modulators of muscarinic receptors for other indications that provide an opportunity for repurposing into the alcohol use disorder space and provide some currently unanswered questions as a roadmap for future investigation.

The Neurocircuitry of Substance Use Disorder, Treatment, and Change: A Resource for Clinical Psychiatrists

Substance use disorder (SUD) is common in psychiatric patients and has a negative impact on health and well-being. However, SUD often goes untreated, and there is a need for psychiatrists, of all specialties, to address this pervasive clinical problem. In this review, the authors' goal is to provide a resource that describes treatments for SUD, using neuroscience as a framework. They discuss the effect of pharmacotherapy on craving, intoxication, and withdrawal and its ability to interrupt the cycle of substance use in SUD. The neuroscience of stress is reviewed, including medications targeting neurotransmitter systems activated by alarm and fear. Neuroplasticity and promising treatments that use this mechanism, including ketamine, psilocybin, and transcranial magnetic stimulation (TMS), are discussed. The authors conclude by listing resources and practice guidelines for physicians interested in learning more about treatments for SUD.

Effects of sex and estrous cycle on action-outcome contingencies

Goal-directed and habitual-like behaviors are both necessary to efficiently and effectively navigate the environment. A dysregulation between these behaviors can lead to an overreliance on habitual-like behaviors and may contribute to symptoms experienced in some neuropsychiatric disorders such as substance use disorder. One behavioral task used to evaluate goal-directed and habitual-like behavior is an action-outcome task, contingency degradation, where an action (i.e., lever press) is degraded by decoupling the receipt of a reward from the action. However, little is known about how male and female rats and females across the estrous cycle respond during contingency degradation training and extinction testing. Here, we investigated how the variable of sex and estrous cycle influences contingency degradation training and extinction testing and the correlation between baseline anxiety-like behaviors and performance on contingency degradation extinction testing in adult male and female Long-Evans rats. We found that both males and females learned the contingency degradation task. However, during extinction testing, males respond more to the contingent lever than the non-contingent lever while females do not differ in their responses on the non-contingent and contingent levers. Lower baseline anxiety-like behavior predicted better performance on the contingency degradation test in males, but not females. Next, when we examined performance during extinction testing in females based on their estrous cycle stage on test day, we found that females in the proestrus and estrus stages of the estrous cycle do not differ in their responses on the non-contingent and contingent levers, while females in the metestrus and diestrus stages of the estrous cycle respond more on the contingent lever than the non-contingent lever on the extinction test day, similar to male rats. Our findings indicate that the estrous cycle influences how female rats respond during contingency degradation extinction testing that is dependent on their estrous cycle stage.

Transcription factor 4 expression in the developing non-human primate brain: a comparative analysis with the mouse brain

Transcription factor 4 (TCF4) has been implicated in a range of neuropsychiatric disorders, including major depressive disorder, bipolar disorder, and schizophrenia. Mutations or deletions in TCF4 cause Pitt-Hopkins syndrome (PTHS), a rare neurodevelopmental disorder. A detailed understanding of its spatial expression across the developing brain is necessary for comprehending TCF4 biology and, by extension, to develop effective treatments for TCF4-associated disorders. However, most current knowledge is derived from mouse models, which are invaluable for preclinical studies but may not fully capture the complexities of human neuropsychiatric phenotypes. This study compared TCF4 expression in the developing mouse brain to its regional and cellular expression patterns in normal prenatal, neonatal, and young adult rhesus macaque brains, a species more relevant to human neurodevelopment. While the general developmental expression of TCF4 is largely conserved between macaques and mice, we saw several interspecies differences. Most notably, a distinct layered pattern of TCF4 expression was clear in the developing macaque neocortex but largely absent in the mouse brain. High TCF4 expression was seen in the inner dentate gyrus of adult mice but not in macaques. Conversely, TCF4 expression was higher in the adult macaque striatum compared to the mouse striatum. Further research is needed to show the significance of these interspecies differences. Still, they underscore the importance of integrating rodent and primate studies to comprehensively understand TCF4 function and its implications for human disorders. Moreover, the primate-specific expression patterns of TCF4 will inform genetic and other therapeutic strategies to treat TCF4-associated disorders.

Shared and distinct aberrations in frontal-striatal system functional patterns among patients with irritable bowel syndrome and major depressive disorder

BACKGROUND

Considering the high comorbidity, shared risk factors, and genetic pathways between irritable bowel syndrome (IBS) and major depressive disorder (MDD), we hypothesized that there would be both shared and disorder-specific alterations in brain function.

METHODS

A total of 39 IBS patients, 39 MDD patients, and 40 healthy controls (HCs) were enrolled and matched for sex, age, and educational level. All subjects underwent resting-state functional MRI. The clinical variables of anxiety, depression, gastrointestinal symptoms and alexithymia were recorded. The 12 subregions of the striatum were employed as seeds to assess their functional connectivity (FC) with every voxel throughout the whole brain.

RESULTS

Compared to HC, IBS and MDD patients exhibited aberrant frontal-striatal circuitry. We observed a common decrease in FC between the dorsal striatum and regions of the hippocampus, sensorimotor cortex, and prefrontal cortex (PFC) in both IBS and MDD patients. Patients with IBS exhibited disorder-specific decreases in FC within the striatum, along with reduced connectivity between the ventral striatum and sensorimotor cortex. In contrast, MDD patients showed disorder-specific hyperconnectivity in the medial PFC-limbic system. Receiver operating characteristic curve analysis showed that frontal-striatal FC values could serve as transdiagnostic markers of IBS and MDD. Within the IBS group, striatal connectivity was not only negatively associated with weekly abdominal pain days but also negatively correlated with the levels of anxiety and alexithymia.

CONCLUSIONS

This exploratory analysis indicated that patients with IBS and MDD exhibited both shared and disorder-specific frontal-striatal circuit impairments, potentially explaining both comorbidity and distinct phenotypes.

Ceftriaxone reverses diet-induced deficits in goal-directed control

RATIONALE

Obesity is associated with numerous health risks and ever-increasing rates are a significant global concern. However, despite weight loss attempts many people have difficulty maintaining weight loss. Previous studies in animals have shown that chronic access to an obesogenic diet can disrupt goal-directed behavior, impairing the ability of animals to flexibly adjust food-seeking behavior following changes in the value of earned outcomes. Changes in behavioral control have been linked to disruption of glutamate transmission in the dorsal medial striatum (DMS), a region critical for the acquisition and expression of goal-directed behavior.

OBJECTIVES

The goal of this study was to test whether ceftriaxone, a beta-lactam antibiotic shown elsewhere to upregulate the expression of the glutamate transporter GLT-1, would improve goal-directed control following long-term exposure to an obesogenic diet.

METHODS

Male and female rats were given access to either standard chow or chow plus sweetened condensed milk (SCM) for 6 weeks. Access to SCM was ended and rats received daily injections of either ceftriaxone or saline for 6 days. Rats were then trained to press a lever to earn a novel food reward and, finally, were assessed for sensitivity to outcome devaluation. Histological analyses examined changes to GLT-1 protein levels and morphological changes to astrocytes, within the DMS.

RESULTS

We found that ceftriaxone robustly restored goal-directed behavior in animals following long-term exposure to SCM. While we did not observe changes in protein levels of GLT-1 in the DMS, we observed that SCM induced changes in the morphology of astrocytes in the DMS, and that ceftriaxone mitigated these changes.

CONCLUSIONS

These results demonstrate that long-term access to a SCM diet impairs goal-directed behavior while also altering the morphology of astrocytes in the DMS. Furthermore, these results suggest that ceftriaxone administration can reverse the impairment of goal-directed behavior potentially through its actions on astrocytes in decision-making circuitry.

Coding Dynamics of the Striatal Networks During Learning

The rat dorsomedial (DMS) and dorsolateral striatum (DLS), equivalent to caudate nucleus and putamen in primates, are required for goal-directed and habit behaviour, respectively. However, it is still unclear whether and how this functional dichotomy emerges in the course of learning. In this study, we investigated this issue by recording DMS and DLS single neuron activity in rats performing a continuous spatial alternation task, from the acquisition to optimized performance. We first applied a classical analytical approach to identify task-related activity based on the modifications of single neuron firing rate in relation to specific task events or maze trajectories. We then used an innovative approach based on Hawkes process to reconstruct a directed connectivity graph of simultaneously recorded neurons, that was used to decode animal behavior. This approach enabled us to better unravel the role of DMS and DLS neural networks across learning stages. We showed that DMS and DLS display different task-related activity throughout learning stages, and the proportion of coding neurons over time decreases in the DMS and increases in the DLS. Despite these major differences, the decoding power of both networks increases during learning. These results suggest that DMS and DLS neural networks gradually reorganize in different ways in order to progressively increase their control over the behavioral performance.

Adaptive circuits for action and value information in rodent operant learning

Operant learning is a behavioral paradigm where animals learn to associate their actions with consequences, adapting their behavior accordingly. This review delves into the neural circuits that underpin operant learning in rodents, emphasizing the dynamic interplay between neural pathways, synaptic plasticity, and gene expression changes. We explore the cortico-basal ganglia circuits, highlighting the pivotal role of dopamine in modulating these pathways to reinforce behaviors that yield positive outcomes. We include insights from recent studies, which reveals the intricate roles of midbrain dopamine neurons in integrating action initiation and reward feedback, thereby enhancing movement-related activities in the dorsal striatum. Additionally, we discuss the molecular diversity of striatal neurons and their specific roles in reinforcement learning. The review also covers advances in transcriptome analysis techniques, such as single-cell RNA sequencing, which have provided deeper insights into the gene expression profiles associated with different neuronal populations during operant learning.

A dual-pathway architecture enables chronic stress to disrupt agency and promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted systems neuroscience approach in male and female mice, we reveal a dual pathway, amygdala-striatal neuronal circuit architecture by which a recent history of chronic stress disrupts the action-outcome learning underlying adaptive agency and promotes the formation of inflexible habits. We found that the basolateral amygdala projection to the dorsomedial striatum is activated by rewarding events to support the action-outcome learning needed for flexible, goal-directed decision making. Chronic stress attenuates this to disrupt action-outcome learning and, therefore, agency. Conversely, the central amygdala projection to the dorsomedial striatum mediates habit formation. Following stress this pathway is progressively recruited to learning to promote the premature formation of inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to disrupt agency and promote habit. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and mental health conditions.

Atypical effective connectivity from the frontal cortex to striatum in alcohol use disorder

Alcohol use disorder (AUD) is a profound psychiatric condition marked by disrupted connectivity among distributed brain regions, indicating impaired functional integration. Previous connectome studies utilizing functional magnetic resonance imaging (fMRI) have predominantly focused on undirected functional connectivity, while the specific alterations in directed effective connectivity (EC) associated with AUD remain unclear. To address this issue, this study utilized multivariate pattern analysis (MVPA) and spectral dynamic causal modeling (DCM). We recruited 32 abstinent men with AUD and 30 healthy controls (HCs) men, and collected their resting-state fMRI data. A regional homogeneity (ReHo)-based MVPA method was employed to classify AUD and HC groups, as well as predict the severity of addiction in AUD individuals. The most informative brain regions identified by the MVPA were further investigated using spectral DCM. Our results indicated that the ReHo-based support vector classification (SVC) exhibits the highest accuracy in distinguishing individuals with AUD from HCs (classification accuracy: 98.57%). Additionally, our results demonstrated that ReHo-based support vector regression (SVR) could be utilized to predict the addiction severity (alcohol use disorders identification test, AUDIT, R2 = 0.38; Michigan alcoholism screening test, MAST, R2 = 0.29) of patients with AUD. The most informative brain regions for the prediction include left pre-SMA, right dACC, right LOFC, right putamen, and right NACC. These findings were validated in an independent data set (35 patients with AUD and 36 HCs, Classification accuracy: 91.67%; AUDIT, R2 = 0.17; MAST, R2 = 0.20). The results of spectral DCM analysis indicated that individuals with AUD exhibited decreased EC from the left pre-SMA to the right putamen, from the right dACC to the right putamen, and from the right LOFC to the right NACC compared to HCs. Moreover, the EC strength from the right NACC to left pre-SMA and from the right dACC to right putamen mediated the relationship between addiction severity (MAST scores) and behavioral measures (impulsive and compulsive scores). These findings provide crucial evidence for the underlying mechanism of impaired self-control, risk assessment, and impulsive and compulsive alcohol consumption in individuals with AUD, providing novel causal insights into both diagnosis and treatment.

Multiple faces of anxiety: a frontal lobe perspective

Marked dysregulation of the human prefrontal cortex (PFC) and anterior cingulate cortex (ACC) characterises a variety of anxiety disorders, and its amelioration is a key feature of treatment success. Overall treatment response, however, is highly variable, and about a third of patients are resistant to treatment. In this review we hypothesise that a major contributor to this variation in treatment response are the multiple faces of anxiety induced by distinct forms of frontal cortex dysregulation. Comparison of findings from humans and non-human primates reveals marked similarity in the functional organisation of threat regulation across the frontal lobes. This organisation is discussed in relation to the 'predatory imminence continuum' model of threat and the differential engagement of executive functions at the core of both emotion generation and regulation strategies.

Attenuated responses to attention-modulating drugs in the neuroligin-3R451C mouse model of autism

Attention deficits are frequently reported within the clinical autism population. Despite not being a core diagnostic feature, some aetiological theories place atypical attention at the centre of autism development. Drugs used to treat attention dysfunction are therefore increasingly prescribed to autistic patients, though currently off-label with uncertain efficacy. We utilised a rodent-translated touchscreen test of sustained attention in mice carrying an autism-associated R451C mutation in the neuroligin-3 gene (Nlgn3R451C). In doing so, we replicated their cautious but accurate response profile and probed it using two widely prescribed attention-modulating drugs: methylphenidate (MPH) and atomoxetine (ATO). In wild-type mice, acute administration of MPH (3 mg/kg) promoted impulsive responding at the expense of accuracy, while ATO (3 mg/kg) broadly reduced impulsive responding. These drug effects were absent in Nlgn3R451C mice, other than a small reduction in blank touches to the screen following ATO administration. The absence of drug effects in Nlgn3R451C mice likely arises from their altered behavioural baseline and underlying neurobiology, highlighting caveats to the use of classic attention-modulating drugs across disorders and autism subsets. It further suggests that altered dopaminergic and/or norepinephrinergic systems may drive behavioural differences in the Nlgn3R451C mouse model of autism, supporting further targeted investigation.

Behavioral outputs and overlapping circuits between conditional fear and active avoidance

Aversive learning can produce a wide variety of defensive behavioral responses depending on the circumstances, ranging from reactive responses like freezing to proactive avoidance responses. While most of this initial learning is behaviorally supported by an expectancy of an aversive outcome and neurally supported by activity within the basolateral amygdala, activity in other brain regions become necessary for the execution of defensive strategies that emerge in other aversive learning paradigms such as active avoidance. Here, we review the neural circuits that support both reactive and proactive defensive behaviors that are motivated by aversive learning, and identify commonalities between the neural substrates of these distinct (and often exclusive) behavioral strategies.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Abnormal functional connectivity of the putamen in obsessive-compulsive disorder

The putamen has been proposed to play a critical role in the development of obsessive-compulsive disorder (OCD). The primary objective of this study was to examine the resting-state regional activity and functional connectivity patterns of the putamen in individuals diagnosed with OCD. To achieve this, we employed resting-state functional magnetic resonance imaging (rs-fMRI) to collect data from a sample of 45 OCD patients and 53 healthy control participants. We aimed to use the regional amplitude of low-frequency fluctuation (ALFF) analysis to generate the ROI masks of the putamen and then conduct the whole brain functional connectivity of the putamen in individuals with OCD. Compared to controls, the OCD group demonstrated decreased ALFF in bilateral putamen. The right putamen also displayed decreased FC with the left putamen extending to the inferior frontal gyrus (IFG), bilateral precuneus extending to calcarine, the right middle occipital cortex extending to the right middle temporal cortex, and the left middle occipital gyrus. The decreased connectivity between the right putamen and the left IFG was negatively correlated with Yale-Brown Obsessive Compulsive scale (Y-BOCS) Obsession Scores. This study aimed to reveal the putamen changes in resting-state activity and connectivity in OCD patients, highlighting the significance of aberrant ALFF/FC of the putamen is a key characteristic of OCD.

Enhanced "learning to learn" through a hierarchical dual-learning system: the case of action video game players

In contrast to conventional cognitive training paradigms, where learning effects are specific to trained parameters, playing action video games has been shown to produce broad enhancements in many cognitive functions. These remarkable generalizations challenge the conventional theory of generalization that learned knowledge can be immediately applied to novel situations (i.e., immediate generalization). Instead, a new "learning to learn" theory has recently been proposed, suggesting that these broad generalizations are attained because action video game players (AVGPs) can quickly acquire the statistical regularities of novel tasks in order to increase the learning rate and ultimately achieve better performance. Although enhanced learning rate has been found for several tasks, it remains unclear whether AVGPs efficiently learn task statistics and use learned task knowledge to guide learning. To address this question, we tested 34 AVGPs and 36 non-video game players (NVGPs) on a cue-response associative learning task. Importantly, unlike conventional cognitive tasks with fixed task statistics, in this task, cue-response associations either remain stable or change rapidly (i.e., are volatile) in different blocks. To complete the task, participants should not only learn the lower-level cue-response associations through explicit feedback but also actively estimate the high-level task statistics (i.e., volatility) to dynamically guide lower-level learning. Such a dual learning system is modelled using a hierarchical Bayesian learning framework, and we found that AVGPs indeed quickly extract the volatility information and use the estimated higher volatility to accelerate learning of the cue-response associations. These results provide strong evidence for the "learning to learn" theory of generalization in AVGPs. Taken together, our work highlights enhanced hierarchical learning of both task statistics and cognitive abilities as a mechanism underlying the broad enhancements associated with action video game play.

Functional near-infrared spectroscopy-based neurofeedback training targeting the dorsolateral prefrontal cortex induces changes in cortico-striatal functional connectivity

Due to its central role in cognitive control, the dorso-lateral prefrontal cortex (dlPFC) has been the target of multiple brain modulation studies. In the context of the present pilot study, the dlPFC was the target of eight repeated neurofeedback (NF) sessions with functional near infrared spectroscopy (fNIRS) to assess the brain responses during NF and with functional and resting state magnetic resonance imaging (task-based fMRI and rsMRI) scanning. Fifteen healthy participants were recruited. Cognitive task fMRI and rsMRI were performed during the 1st and the 8th NF sessions. During NF, our data revealed an increased activity in the dlPFC as well as in brain regions involved in cognitive control and self-regulation learning (pFWE < 0.05). Changes in functional connectivity between the 1st and the 8th session revealed increased connectivity between the posterior cingulate cortex and the dlPFC, and between the posterior cingulate cortex and the dorsal striatum (pFWE < 0.05). Decreased left dlPFC-left insula connectivity was also observed. Behavioural results revealed a significant effect of hunger and motivation on the participant control feeling and a lower control feeling when participants did not identify an effective mental strategy, providing new insights on the effects of behavioural factors that may affect the NF learning.

Mapping functional traces of opioid memories in the rat brain

Addiction to psychoactive substances is a maladaptive learned behaviour. Contexts surrounding drug use integrate this aberrant mnemonic process and hold strong relapse-triggering ability. Here, we asked where context and salience might be concurrently represented in the brain during retrieval of drug-context paired associations. For this, we developed a morphine-conditioned place preference protocol that allows contextual stimuli presentation inside a magnetic resonance imaging scanner and investigated differences in activity and connectivity at context recall. We found context-specific responses to stimulus onset in multiple brain regions, namely, limbic, sensory and striatal. Differences in functional interconnectivity were found among amygdala, lateral habenula, and lateral septum. We also investigated alterations to resting-state functional connectivity and found increased centrality of the lateral septum in a proposed limbic network, as well as increased functional connectivity of the lateral habenula and hippocampal 'cornu ammonis' 1 region, after a protocol of associative drug-context. Finally, we found that pre- conditioned place preference resting-state connectivity of the lateral habenula and amygdala was predictive of inter-individual conditioned place preference score differences. Overall, our findings show that drug and saline-paired contexts establish distinct memory traces in overlapping functional brain microcircuits and that intrinsic connectivity of the habenula, septum, and amygdala likely underlies the individual maladaptive contextual learning to opioid exposure. We have identified functional maps of acquisition and retrieval of drug-related memory that may support the relapse-triggering ability of opioid-associated sensory and contextual cues. These findings may clarify the inter-individual sensitivity and vulnerability seen in addiction to opioids found in humans.

Resource-rational psychopathology

Psychopathology is vast and diverse. Across distinct disease states, individuals exhibit symptoms that appear counter to the standard view of rationality (expected utility maximization). We argue that some aspects of psychopathology can be described as resource-rational, reflecting a rational trade-off between reward and cognitive resources. We review work on two theories of this kind: rational inattention, where a capacity limit applies to perceptual channels, and policy compression, where the capacity limit applies to action channels. We show how these theories can parsimoniously explain many forms of psychopathology, including affective, primary psychotic, and neurodevelopmental disorders, as well as many effects of psychoactive medications on these disorders. While there are important disorder-specific differences and the theories are by no means universal, we argue that resource rationality offers a useful new perspective on psychopathology. By emphasizing the role of cognitive resource constraints, this approach offers a more inclusive picture of rationality. Some aspects of psychopathology may reflect rational trade-offs rather than the breakdown of rationality. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity

The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.

Feedback timing-modulated weather prediction reveals relative deficits in both procedural and declarative learning in adults with dyslexia

A topic of recent debate is the hypothesis that deficits associated with developmental disorders of language, such as reading disability, can be explained by a selective weakness in procedural memory. Adults with (n = 29; RD) and without (n = 29; TD) reading disability completed a weather prediction task under immediate and delayed feedback conditions, that rely on the striatal (procedural) and hippocampal (declarative) circuits, respectively. We examined trial-by-trial accuracy by feedback condition (immediate vs. delayed) and group (RD vs. TD). In the immediate feedback condition, we found the TD group to have a higher learning rate than the RD group. In the delayed feedback condition, we found the TD group reach a high level of accuracy early, and outperform the RD group for the duration of the task. The TD group also made gains in reaction time under both conditions, while the RD group slowed in their responses. Taken together, it appears that while procedural memory is indeed impaired in adults with reading disability, to a lesser extent, declarative memory is also affected. This lends partial support to the PDH, and more broadly to the position that reading disability is associated with general (non-linguistic) difficulties in learning.

Cortico-striatal white-matter connectivity underlies the ability to exert goal-directed control

The balance between goal-directed and habitual control has been proposed to determine the flexibility of instrumental behaviour, in both humans and animals. This view is supported by neuroscientific studies that have implicated dissociable neural pathways in the ability to flexibly adjust behaviour when outcome values change. A previous Diffusion Tensor Imaging study provided preliminary evidence that flexible instrumental performance depends on the strength of parallel cortico-striatal white-matter pathways previously implicated in goal-directed and habitual control. Specifically, estimated white-matter strength between caudate and ventromedial prefrontal cortex correlated positively with behavioural flexibility, and posterior putamen-premotor cortex connectivity correlated negatively, in line with the notion that these pathways compete for control. However, the sample size of the original study was limited, and so far, there have been no attempts to replicate these findings. In the present study, we aimed to conceptually replicate these findings by testing a large sample of 205 young adults to relate cortico-striatal connectivity to performance on the slips-of-action task. In short, we found only positive neural correlates of goal-directed performance, including striatal connectivity (caudate and anterior putamen) with the dorsolateral prefrontal cortex. However, we failed to provide converging evidence for the existence of a neural habit system that puts limits on the capacity for flexible, goal-directed action. We discuss the implications of our findings for dual-process theories of instrumental action.

Disorders of compulsivity: Deficits in arbitrating learning strategies

While previous research has shown that compulsivity is related to an imbalance between goal-directed and habitual learning systems, very little is known about whether this effect is due to the impairment of a single system or the impairment of the arbitration mechanism that determines which system controls behaviour at any given moment; the current study aims to address this disagreement. Nineteen alcohol use disorder, 30 obsessive-compulsive disorder (OCD) and 20 major depressive disorder patients and corresponding sex- and age-matched controls performed two-choice, three-stage Markov decision-making paradigm. Model-based and mode-free reinforcement learning models were used to independently fitted their behavioural data. Alcohol use disorder and OCD patients showed less model-based strategy choice than healthy controls in task conditions where the model-based strategy was optimal. Only OCD patients showed higher behavioural control system switching in task conditions where model-free use was optimal. Major depressive disorder patients did not differ from the matched control in both. These findings suggest that dysfunction in arbitration control between dual systems may be the basis for diverse disorders involving compulsivity.

Subjective well-being can be predicted by caudate volume and promotion focus

It is well-known that the caudate nucleus is associated with motivational behaviors and subjective well-being. However, no longitudinal studies have examined the relationship between brain structure, behavioral orientations, and subjective well-being. This study analyzes data from our previous longitudinal study to examine whether future subjective well-being can be predicted by the volume of the caudate nucleus. We also examined whether behavioral orientation, based on the regulatory focus theory showing two orientations-promotion and prevention focus-was related to the volume of the caudate nucleus. Voxel-based morphometry analysis indicated that the left caudate volume was positively associated with rating scores for future subjective well-being and promotion orientation. Further, mediation analysis indicated that promotion orientation significantly mediated the relationship between future subjective well-being and left caudate volume. The findings indicate that future subjective well-being can be predicted by the volume of the left caudate nucleus, and that this relationship is mediated by promotion focus orientation.

Cocaine disrupts action flexibility via glucocorticoid receptors

Many addictive drugs increase stress hormone levels. They also alter the propensity of organisms to prospectively select actions based on long-term consequences. We hypothesized that cocaine causes inflexible action by increasing circulating stress hormone levels, activating the glucocorticoid receptor (GR). We trained mice to generate two nose pokes for food and then required them to update action-consequence associations when one response was no longer reinforced. Cocaine delivered in adolescence or adulthood impaired the capacity of mice to update action strategies, and inhibiting CORT synthesis rescued action flexibility. Next, we reduced Nr3c1, encoding GR, in the orbitofrontal cortex (OFC), a region of the brain responsible for interlacing new information into established routines. Nr3c1 silencing preserved action flexibility and dendritic spine abundance on excitatory neurons, despite cocaine. Spines are often considered substrates for learning and memory, leading to the discovery that cocaine degrades the representation of new action memories, obstructing action flexibility.

The Small G-Protein Rac1 in the Dorsomedial Striatum Promotes Alcohol-Dependent Structural Plasticity and Goal-Directed Learning in Mice

The small G-protein Ras-related C3 botulinum toxin substrate 1 (Rac1) promotes the formation of filamentous actin (F-actin). Actin is a major component of dendritic spines, and we previously found that alcohol alters actin composition and dendritic spine structure in the nucleus accumbens (NAc) and the dorsomedial striatum (DMS). To examine if Rac1 contributes to these alcohol-mediated adaptations, we measured the level of GTP-bound active Rac1 in the striatum of mice following 7 weeks of intermittent access to 20% alcohol. We found that chronic alcohol intake activates Rac1 in the DMS of male mice. In contrast, Rac1 is not activated by alcohol in the NAc and DLS of male mice or in the DMS of female mice. Similarly, closely related small G-proteins are not activated by alcohol in the DMS, and Rac1 activity is not increased in the DMS by moderate alcohol or natural reward. To determine the consequences of alcohol-dependent Rac1 activation in the DMS of male mice, we inhibited endogenous Rac1 by infecting the DMS of mice with an adeno-associated virus (AAV) expressing a dominant negative form of the small G-protein (Rac1-DN). We found that overexpression of AAV-Rac1-DN in the DMS inhibits alcohol-mediated Rac1 signaling and attenuates alcohol-mediated F-actin polymerization, which corresponded with a decrease in dendritic arborization and spine maturation. Finally, we provide evidence to suggest that Rac1 in the DMS plays a role in alcohol-associated goal-directed learning. Together, our data suggest that Rac1 in the DMS plays an important role in alcohol-dependent structural plasticity and aberrant learning.

Right thalamic volume mediates impact of the dopamine beta-hydroxylase gene on the endowment effect

The endowment effect is a tendency that individuals overvalue items belonging to them relative to those items that do not. Previous studies showed a strong relation between the dopamine beta-hydroxylase (DBH) gene and the endowment effect (EE), and a link between EE and task-based functional MRI activation in multiple brain regions. However, the role of brain structure on EE remains unclear. In this study, we have explored whether regional brain volume mediate the effect of the DBH gene on EE. Results showed that rs1611115, single-nucleotide polymorphisms (SNPs) at DBH loci, were significantly associated with right thalamus volume and the endowment effect in males but not in female participants. Specifically, male DBH rs1611115 T-carriers had larger right thalamus volume compared to carriers of CC genotype and exhibited a greater endowment effect. Importantly, we found that right thalamus volume mediated the effect of rs1611115 on the endowment effect in male participants. This study demonstrated how thalamic volume plays an important mediating role between genetics and decision-making in humans.

The human VGLUT3-pT8I mutation elicits uneven striatal DA signaling, food or drug maladaptive consumption in male mice

Cholinergic striatal interneurons (ChIs) express the vesicular glutamate transporter 3 (VGLUT3) which allows them to regulate the striatal network with glutamate and acetylcholine (ACh). In addition, VGLUT3-dependent glutamate increases ACh vesicular stores through vesicular synergy. A missense polymorphism, VGLUT3-p.T8I, was identified in patients with substance use disorders (SUDs) and eating disorders (EDs). A mouse line was generated to understand the neurochemical and behavioral impact of the p.T8I variant. In VGLUT3T8I/T8I male mice, glutamate signaling was unchanged but vesicular synergy and ACh release were blunted. Mutant male mice exhibited a reduced DA release in the dorsomedial striatum but not in the dorsolateral striatum, facilitating habit formation and exacerbating maladaptive use of drug or food. Increasing ACh tone with donepezil reversed the self-starvation phenotype observed in VGLUT3T8I/T8I male mice. Our study suggests that unbalanced dopaminergic transmission in the dorsal striatum could be a common mechanism between SUDs and EDs.

A molecularly defined orbitofrontal cortical neuron population controls compulsive-like behavior, but not inflexible choice or habit

Habits are familiar behaviors triggered by cues, not outcome predictability, and are insensitive to changes in the environment. They are adaptive under many circumstances but can be considered antecedent to compulsions and intrusive thoughts that drive persistent, potentially maladaptive behavior. Whether compulsive-like and habit-like behaviors share neural substrates is still being determined. Here, we investigated mice bred to display inflexible reward-seeking behaviors that are insensitive to action consequences. We found that these mice demonstrate habitual response biases and compulsive-like grooming behavior that was reversible by fluoxetine and ketamine. They also suffer dendritic spine attrition on excitatory neurons in the orbitofrontal cortex (OFC). Nevertheless, synaptic melanocortin 4 receptor (MC4R), a factor implicated in compulsive behavior, is preserved, leading to the hypothesis that Mc4r+ OFC neurons may drive aberrant behaviors. Repeated chemogenetic stimulation of Mc4r+ OFC neurons triggered compulsive and not inflexible or habitual response biases in otherwise typical mice. Thus, Mc4r+ neurons within the OFC appear to drive compulsive-like behavior that is dissociable from habitual behavior. Understanding which neuron populations trigger distinct behaviors may advance efforts to mitigate harmful compulsions.

Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease

To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

Cortico-striatal action control inherent of opponent cognitive-motivational styles

Turning on cue or stopping at a red light requires the detection of such cues to select action sequences, or suppress action, in accordance with cue-associated action rules. Cortico-striatal projections are an essential part of the brain's attention-motor interface. Glutamate-sensing microelectrode arrays were used to measure glutamate transients in the dorsomedial striatum (DMS) of male and female rats walking a treadmill and executing cued turns and stops. Prelimbic-DMS projections were chemogenetically inhibited to determine their behavioral necessity and the cortico-striatal origin of cue-evoked glutamate transients. Furthermore, we investigated rats exhibiting preferably goal-directed (goal trackers, GTs) versus cue-driven attention (sign trackers, STs), to determine the impact of such cognitive-motivational biases on cortico-striatal control. GTs executed more cued turns and initiated such turns more slowly than STs. During turns, but not missed turns or cued stops, cue-evoked glutamate concentrations were higher in GTs than in STs. In conjunction with turn cue-evoked glutamate spike levels, the presence of a single spike rendered GTs to be almost twice as likely to turn than STs. In contrast, multiple glutamate spikes predicted GTs to be less likely to turn than STs. In GTs, but not STs, inhibition of prelimbic-DMS projections attenuated turn rates, turn cue-evoked glutamate peaks, and increased the number of spikes. These findings suggest that turn cue-evoked glutamate release in GTs is tightly controlled by cortico-striatal neuronal activity. In contrast, in STs, glutamate release from DMS glutamatergic terminals may be regulated by other striatal circuitry, preferably mediating cued suppression of action and reward tracking.

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

The impact of abstinence from chronic alcohol consumption on the mouse striatal proteome: sex and subregion-specific differences

Alcohol misuse is the third leading preventable cause of death in the world. The World Health Organization currently estimates that 1 in 20 deaths are directly alcohol related. One of the ways in which consuming excessive levels of alcohol can both directly and indirectly affect human mortality and morbidity, is through chronic inflammation. Recently, studies have suggested a link between increased alcohol use and the incidence of neuroinflammatory-related diseases. However, the mechanism in which alcohol potentially influences neuroinflammatory processes is still being uncovered. We implemented an unbiased proteomics exploration of alcohol-induced changes in the striatum, with a specific emphasis on proteins related to inflammation. The striatum is a brain region that is critically involved with the progression of alcohol use disorder. Using mass spectrometry following voluntary alcohol self-administration in mice, we show that distinct protein abundances and signaling pathways in different subregions of the striatum are disrupted by chronic exposure to alcohol compared to water drinking control mice. Further, in mice that were allowed to experience abstinence from alcohol compared to mice that were non-abstinent, the overall proteome and signaling pathways showed additional differences, suggesting that the responses evoked by chronic alcohol exposure are dependent on alcohol use history. To our surprise we did not find that chronic alcohol drinking or abstinence altered protein abundance or pathways associated with inflammation, but rather affected proteins and pathways associated with neurodegeneration and metabolic, cellular organization, protein translation, and molecular transport processes. These outcomes suggest that in this drinking model, alcohol-induced neuroinflammation in the striatum is not a primary outcome controlling altered neurobehavioral function, but these changes are rather mediated by altered striatal neuronal structure and cellular health.

Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system

Despite a theory that an imbalance in goal-directed versus habitual systems serve as building blocks of compulsions, research has yet to delineate how this occurs during arbitration between the two systems in obsessive-compulsive disorder. Inspired by a brain model in which the inferior frontal cortex selectively gates the putamen to guide goal-directed or habitual actions, this study aimed to examine whether disruptions in the arbitration process via the fronto-striatal circuit would underlie imbalanced decision-making and compulsions in patients. Thirty patients with obsessive-compulsive disorder [mean (standard deviation) age = 26.93 (6.23) years, 12 females (40%)] and 30 healthy controls [mean (standard deviation) age = 24.97 (4.72) years, 17 females (57%)] underwent functional MRI scans while performing the two-step Markov decision task, which was designed to dissociate goal-directed behaviour from habitual behaviour. We employed a neurocomputational model to account for an uncertainty-based arbitration process, in which a prefrontal arbitrator (i.e. inferior frontal gyrus) allocates behavioural control to a more reliable strategy by selectively gating the putamen. We analysed group differences in the neural estimates of uncertainty of each strategy. We also compared the psychophysiological interaction effects of system preference (goal-directed versus habitual) on fronto-striatal coupling between groups. We examined the correlation between compulsivity score and the neural activity and connectivity involved in the arbitration process. The computational model captured the subjects' preferences between the strategies. Compared with healthy controls, patients had a stronger preference for the habitual system (t = -2.88, P = 0.006), which was attributed to a more uncertain goal-directed system (t = 2.72, P = 0.009). Before the allocation of controls, patients exhibited hypoactivity in the inferior frontal gyrus compared with healthy controls when this region tracked the inverse of uncertainty (i.e. reliability) of goal-directed behaviour (P = 0.001, family-wise error rate corrected). When reorienting behaviours to reach specific goals, patients exhibited weaker right ipsilateral ventrolateral prefronto-putamen coupling than healthy controls (P = 0.001, family-wise error rate corrected). This hypoconnectivity was correlated with more severe compulsivity (r = -0.57, P = 0.002). Our findings suggest that the attenuated top-down control of the putamen by the prefrontal arbitrator underlies compulsivity in obsessive-compulsive disorder. Enhancing fronto-striatal connectivity may be a potential neurotherapeutic approach for compulsivity and adaptive decision-making.

The brain's arbitration system and obsessive-compulsive disorder

This scientific commentary relates to ‘Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system’ by Kim et al. (https://doi.org/10.1093/brain/awae102).

Mesocorticolimbic system reactivity to alcohol use-related visual cues as a function of alcohol sensitivity phenotype: A pilot fMRI study

Low sensitivity (LS) to alcohol is a risk factor for alcohol use disorder (AUD). Compared to peers with high sensitivity (HS), LS individuals drink more, report more problems, and exhibit potentiated alcohol cue reactivity (ACR). Heightened ACR suggests LS confers AUD risk via incentive sensitization, which is thought to take place in the mesocorticolimbic system. This study examined neural ACR in LS and HS individuals. Young adults (N = 32, M age=20.3) were recruited based on the Alcohol Sensitivity Questionnaire (HS: n = 16; LS: n = 16; 9 females/group). Participants completed an event-related fMRI ACR task. Group LS had higher ACR in left ventrolateral prefrontal cortex than group HS. In group LS, ACR in left caudomedial orbitofrontal cortex or left putamen was low at low alcohol use levels and high at heavier or more problematic alcohol use levels, whereas the opposite was true in group HS. Alcohol use level also was associated with the level of ACR in left substantia nigra among males in group LS. Taken together, results suggest elevated mesocorticolimbic ACR among LS individuals, especially those using alcohol at hazardous levels. Future studies with larger samples are warranted to determine the neurobiological loci underlying LS-based amplified ACR and AUD risk.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Anti-Hebbian plasticity drives sequence learning in striatum

Spatio-temporal activity patterns have been observed in a variety of brain areas in spontaneous activity, prior to or during action, or in response to stimuli. Biological mechanisms endowing neurons with the ability to distinguish between different sequences remain largely unknown. Learning sequences of spikes raises multiple challenges, such as maintaining in memory spike history and discriminating partially overlapping sequences. Here, we show that anti-Hebbian spike-timing dependent plasticity (STDP), as observed at cortico-striatal synapses, can naturally lead to learning spike sequences. We design a spiking model of the striatal output neuron receiving spike patterns defined as sequential input from a fixed set of cortical neurons. We use a simple synaptic plasticity rule that combines anti-Hebbian STDP and non-associative potentiation for a subset of the presented patterns called rewarded patterns. We study the ability of striatal output neurons to discriminate rewarded from non-rewarded patterns by firing only after the presentation of a rewarded pattern. In particular, we show that two biological properties of striatal networks, spiking latency and collateral inhibition, contribute to an increase in accuracy, by allowing a better discrimination of partially overlapping sequences. These results suggest that anti-Hebbian STDP may serve as a biological substrate for learning sequences of spikes.

Action sequence learning, habits, and automaticity in obsessive-compulsive disorder

This study investigates the goal/habit imbalance theory of compulsion in obsessive-compulsive disorder (OCD), which postulates enhanced habit formation, increased automaticity, and impaired goal/habit arbitration. It directly tests these hypotheses using newly developed behavioral tasks. First, OCD patients and healthy participants were trained daily for a month using a smartphone app to perform chunked action sequences. Despite similar procedural learning and attainment of habitual performance (measured by an objective automaticity criterion) by both groups, OCD patients self-reported higher subjective habitual tendencies via a recently developed questionnaire. Subsequently, in a re-evaluation task assessing choices between established automatic and novel goal-directed actions, both groups were sensitive to re-evaluation based on monetary feedback. However, OCD patients, especially those with higher compulsive symptoms and habitual tendencies, showed a clear preference for trained/habitual sequences when choices were based on physical effort, possibly due to their higher attributed intrinsic value. These patients also used the habit-training app more extensively and reported symptom relief post-study. The tendency to attribute higher intrinsic value to familiar actions may be a potential mechanism leading to compulsions and an important addition to the goal/habit imbalance hypothesis in OCD. We also highlight the potential of smartphone app training as a habit reversal therapeutic tool.

Information flow between motor cortex and striatum reverses during skill learning

The coordination of neural activity across brain areas during a specific behavior is often interpreted as neural communication involved in controlling the behavior. However, whether information relevant to the behavior is actually transferred between areas is often untested. Here, we used information-theoretic tools to quantify how motor cortex and striatum encode and exchange behaviorally relevant information about specific reach-to-grasp movement features during skill learning in rats. We found a temporal shift in the encoding of behaviorally relevant information during skill learning, as well as a reversal in the primary direction of behaviorally relevant information flow, from cortex-to-striatum during naive movements to striatum-to-cortex during skilled movements. Standard analytical methods that quantify the evolution of overall neural activity during learning-such as changes in neural signal amplitude or the overall exchange of information between areas-failed to capture these behaviorally relevant information dynamics. Using these standard methods, we instead found a consistent coactivation of overall neural signals during movement production and a bidirectional increase in overall information propagation between areas during learning. Our results show that skill learning is achieved through a transformation in how behaviorally relevant information is routed across cortical and subcortical brain areas and that isolating the components of neural activity relevant to and informative about behavior is critical to uncover directional interactions within a coactive and coordinated network.

From compulsivity to compulsion: the neural basis of compulsive disorders

Compulsive behaviour, an apparently irrational perseveration in often maladaptive acts, is a potential transdiagnostic symptom of several neuropsychiatric disorders, including obsessive-compulsive disorder and addiction, and may reflect the severe manifestation of a dimensional trait termed compulsivity. In this Review, we examine the psychological basis of compulsions and compulsivity and their underlying neural circuitry using evidence from human neuroimaging and animal models. Several main elements of this circuitry are identified, focused on fronto-striatal systems implicated in goal-directed behaviour and habits. These systems include the orbitofrontal, prefrontal, anterior cingulate and insular cortices and their connections with the basal ganglia as well as sensoriomotor and parietal cortices and cerebellum. We also consider the implications for future classification of impulsive-compulsive disorders and their treatment.

Distributional coding of associative learning in discrete populations of midbrain dopamine neurons

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

The small G-protein Rac1 in the dorsomedial striatum promotes alcohol-dependent structural plasticity and goal-directed learning in mice

The small G-protein Rac1 promotes the formation of filamentous actin (F-Actin). Actin is a major component of dendritic spines, and we previously found that alcohol alters actin composition and dendritic spine structure in the nucleus accumbens (NAc) and the dorsomedial striatum (DMS). To examine if Rac1 contributes to these alcohol-mediated adaptations, we measured the level of GTP-bound active Rac1 in the striatum of mice following 7 weeks of intermittent access to 20% alcohol. We found that chronic alcohol intake activates Rac1 in the DMS of male mice. In contrast, Rac1 is not activated by alcohol in the NAc and DLS of male mice, or in the DMS of female mice. Similarly, closely related small G-proteins are not activated by alcohol in the DMS, and Rac1 activity is not increased in the DMS by moderate alcohol or natural reward. To determine the consequences of alcohol-dependent Rac1 activation in the DMS of male mice, we inhibited endogenous Rac1 by infecting the DMS of mice with an AAV expressing a dominant negative form of the small G-protein (Rac1-DN). We found that overexpression of AAV-Rac1-DN in the DMS inhibits alcohol-mediated Rac1 signaling and attenuates alcohol-mediated F-actin polymerization, which corresponded with a decrease in dendritic arborization and spine maturation. Finally, we provide evidence to suggest that Rac1 in the DMS plays a role in alcohol-associated goal-directed learning. Together, our data suggest that Rac1 in the DMS plays an important role in alcohol-dependent structural plasticity and aberrant learning.

Significance Statement

Addiction, including alcohol use disorder, is characterized by molecular and cellular adaptations that promote maladaptive behaviors. We found that Rac1 was activated by alcohol in the dorsomedial striatum (DMS) of male mice. We show that alcohol-mediated Rac1 signaling is responsible for alterations in actin dynamics and neuronal morphology. We also present data to suggest that Rac1 is important for alcohol-associated learning processes. These results suggest that Rac1 in the DMS is an important contributor to adaptations that promote alcohol use disorder.

Characterizing the dynamic learning process: Implications of a quantitative analysis

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.