Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

Shared and distinct aberrations in frontal-striatal system functional patterns among patients with irritable bowel syndrome and major depressive disorder

BACKGROUND

Considering the high comorbidity, shared risk factors, and genetic pathways between irritable bowel syndrome (IBS) and major depressive disorder (MDD), we hypothesized that there would be both shared and disorder-specific alterations in brain function.

METHODS

A total of 39 IBS patients, 39 MDD patients, and 40 healthy controls (HCs) were enrolled and matched for sex, age, and educational level. All subjects underwent resting-state functional MRI. The clinical variables of anxiety, depression, gastrointestinal symptoms and alexithymia were recorded. The 12 subregions of the striatum were employed as seeds to assess their functional connectivity (FC) with every voxel throughout the whole brain.

RESULTS

Compared to HC, IBS and MDD patients exhibited aberrant frontal-striatal circuitry. We observed a common decrease in FC between the dorsal striatum and regions of the hippocampus, sensorimotor cortex, and prefrontal cortex (PFC) in both IBS and MDD patients. Patients with IBS exhibited disorder-specific decreases in FC within the striatum, along with reduced connectivity between the ventral striatum and sensorimotor cortex. In contrast, MDD patients showed disorder-specific hyperconnectivity in the medial PFC-limbic system. Receiver operating characteristic curve analysis showed that frontal-striatal FC values could serve as transdiagnostic markers of IBS and MDD. Within the IBS group, striatal connectivity was not only negatively associated with weekly abdominal pain days but also negatively correlated with the levels of anxiety and alexithymia.

CONCLUSIONS

This exploratory analysis indicated that patients with IBS and MDD exhibited both shared and disorder-specific frontal-striatal circuit impairments, potentially explaining both comorbidity and distinct phenotypes.

Behavioral outputs and overlapping circuits between conditional fear and active avoidance

Aversive learning can produce a wide variety of defensive behavioral responses depending on the circumstances, ranging from reactive responses like freezing to proactive avoidance responses. While most of this initial learning is behaviorally supported by an expectancy of an aversive outcome and neurally supported by activity within the basolateral amygdala, activity in other brain regions become necessary for the execution of defensive strategies that emerge in other aversive learning paradigms such as active avoidance. Here, we review the neural circuits that support both reactive and proactive defensive behaviors that are motivated by aversive learning, and identify commonalities between the neural substrates of these distinct (and often exclusive) behavioral strategies.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Enhanced "learning to learn" through a hierarchical dual-learning system: the case of action video game players

In contrast to conventional cognitive training paradigms, where learning effects are specific to trained parameters, playing action video games has been shown to produce broad enhancements in many cognitive functions. These remarkable generalizations challenge the conventional theory of generalization that learned knowledge can be immediately applied to novel situations (i.e., immediate generalization). Instead, a new "learning to learn" theory has recently been proposed, suggesting that these broad generalizations are attained because action video game players (AVGPs) can quickly acquire the statistical regularities of novel tasks in order to increase the learning rate and ultimately achieve better performance. Although enhanced learning rate has been found for several tasks, it remains unclear whether AVGPs efficiently learn task statistics and use learned task knowledge to guide learning. To address this question, we tested 34 AVGPs and 36 non-video game players (NVGPs) on a cue-response associative learning task. Importantly, unlike conventional cognitive tasks with fixed task statistics, in this task, cue-response associations either remain stable or change rapidly (i.e., are volatile) in different blocks. To complete the task, participants should not only learn the lower-level cue-response associations through explicit feedback but also actively estimate the high-level task statistics (i.e., volatility) to dynamically guide lower-level learning. Such a dual learning system is modelled using a hierarchical Bayesian learning framework, and we found that AVGPs indeed quickly extract the volatility information and use the estimated higher volatility to accelerate learning of the cue-response associations. These results provide strong evidence for the "learning to learn" theory of generalization in AVGPs. Taken together, our work highlights enhanced hierarchical learning of both task statistics and cognitive abilities as a mechanism underlying the broad enhancements associated with action video game play.

Functional near-infrared spectroscopy-based neurofeedback training targeting the dorsolateral prefrontal cortex induces changes in cortico-striatal functional connectivity

Due to its central role in cognitive control, the dorso-lateral prefrontal cortex (dlPFC) has been the target of multiple brain modulation studies. In the context of the present pilot study, the dlPFC was the target of eight repeated neurofeedback (NF) sessions with functional near infrared spectroscopy (fNIRS) to assess the brain responses during NF and with functional and resting state magnetic resonance imaging (task-based fMRI and rsMRI) scanning. Fifteen healthy participants were recruited. Cognitive task fMRI and rsMRI were performed during the 1st and the 8th NF sessions. During NF, our data revealed an increased activity in the dlPFC as well as in brain regions involved in cognitive control and self-regulation learning (pFWE < 0.05). Changes in functional connectivity between the 1st and the 8th session revealed increased connectivity between the posterior cingulate cortex and the dlPFC, and between the posterior cingulate cortex and the dorsal striatum (pFWE < 0.05). Decreased left dlPFC-left insula connectivity was also observed. Behavioural results revealed a significant effect of hunger and motivation on the participant control feeling and a lower control feeling when participants did not identify an effective mental strategy, providing new insights on the effects of behavioural factors that may affect the NF learning.

Multiple faces of anxiety: a frontal lobe perspective

Marked dysregulation of the human prefrontal cortex (PFC) and anterior cingulate cortex (ACC) characterises a variety of anxiety disorders, and its amelioration is a key feature of treatment success. Overall treatment response, however, is highly variable, and about a third of patients are resistant to treatment. In this review we hypothesise that a major contributor to this variation in treatment response are the multiple faces of anxiety induced by distinct forms of frontal cortex dysregulation. Comparison of findings from humans and non-human primates reveals marked similarity in the functional organisation of threat regulation across the frontal lobes. This organisation is discussed in relation to the 'predatory imminence continuum' model of threat and the differential engagement of executive functions at the core of both emotion generation and regulation strategies.

Attenuated responses to attention-modulating drugs in the neuroligin-3R451C mouse model of autism

Attention deficits are frequently reported within the clinical autism population. Despite not being a core diagnostic feature, some aetiological theories place atypical attention at the centre of autism development. Drugs used to treat attention dysfunction are therefore increasingly prescribed to autistic patients, though currently off-label with uncertain efficacy. We utilised a rodent-translated touchscreen test of sustained attention in mice carrying an autism-associated R451C mutation in the neuroligin-3 gene (Nlgn3R451C). In doing so, we replicated their cautious but accurate response profile and probed it using two widely prescribed attention-modulating drugs: methylphenidate (MPH) and atomoxetine (ATO). In wild-type mice, acute administration of MPH (3 mg/kg) promoted impulsive responding at the expense of accuracy, while ATO (3 mg/kg) broadly reduced impulsive responding. These drug effects were absent in Nlgn3R451C mice, other than a small reduction in blank touches to the screen following ATO administration. The absence of drug effects in Nlgn3R451C mice likely arises from their altered behavioural baseline and underlying neurobiology, highlighting caveats to the use of classic attention-modulating drugs across disorders and autism subsets. It further suggests that altered dopaminergic and/or norepinephrinergic systems may drive behavioural differences in the Nlgn3R451C mouse model of autism, supporting further targeted investigation.

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity

The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.

Feedback timing-modulated weather prediction reveals relative deficits in both procedural and declarative learning in adults with dyslexia

A topic of recent debate is the hypothesis that deficits associated with developmental disorders of language, such as reading disability, can be explained by a selective weakness in procedural memory. Adults with (n = 29; RD) and without (n = 29; TD) reading disability completed a weather prediction task under immediate and delayed feedback conditions, that rely on the striatal (procedural) and hippocampal (declarative) circuits, respectively. We examined trial-by-trial accuracy by feedback condition (immediate vs. delayed) and group (RD vs. TD). In the immediate feedback condition, we found the TD group to have a higher learning rate than the RD group. In the delayed feedback condition, we found the TD group reach a high level of accuracy early, and outperform the RD group for the duration of the task. The TD group also made gains in reaction time under both conditions, while the RD group slowed in their responses. Taken together, it appears that while procedural memory is indeed impaired in adults with reading disability, to a lesser extent, declarative memory is also affected. This lends partial support to the PDH, and more broadly to the position that reading disability is associated with general (non-linguistic) difficulties in learning.

Cortico-striatal white-matter connectivity underlies the ability to exert goal-directed control

The balance between goal-directed and habitual control has been proposed to determine the flexibility of instrumental behaviour, in both humans and animals. This view is supported by neuroscientific studies that have implicated dissociable neural pathways in the ability to flexibly adjust behaviour when outcome values change. A previous Diffusion Tensor Imaging study provided preliminary evidence that flexible instrumental performance depends on the strength of parallel cortico-striatal white-matter pathways previously implicated in goal-directed and habitual control. Specifically, estimated white-matter strength between caudate and ventromedial prefrontal cortex correlated positively with behavioural flexibility, and posterior putamen-premotor cortex connectivity correlated negatively, in line with the notion that these pathways compete for control. However, the sample size of the original study was limited, and so far, there have been no attempts to replicate these findings. In the present study, we aimed to conceptually replicate these findings by testing a large sample of 205 young adults to relate cortico-striatal connectivity to performance on the slips-of-action task. In short, we found only positive neural correlates of goal-directed performance, including striatal connectivity (caudate and anterior putamen) with the dorsolateral prefrontal cortex. However, we failed to provide converging evidence for the existence of a neural habit system that puts limits on the capacity for flexible, goal-directed action. We discuss the implications of our findings for dual-process theories of instrumental action.

Disorders of compulsivity: Deficits in arbitrating learning strategies

While previous research has shown that compulsivity is related to an imbalance between goal-directed and habitual learning systems, very little is known about whether this effect is due to the impairment of a single system or the impairment of the arbitration mechanism that determines which system controls behaviour at any given moment; the current study aims to address this disagreement. Nineteen alcohol use disorder, 30 obsessive-compulsive disorder (OCD) and 20 major depressive disorder patients and corresponding sex- and age-matched controls performed two-choice, three-stage Markov decision-making paradigm. Model-based and mode-free reinforcement learning models were used to independently fitted their behavioural data. Alcohol use disorder and OCD patients showed less model-based strategy choice than healthy controls in task conditions where the model-based strategy was optimal. Only OCD patients showed higher behavioural control system switching in task conditions where model-free use was optimal. Major depressive disorder patients did not differ from the matched control in both. These findings suggest that dysfunction in arbitration control between dual systems may be the basis for diverse disorders involving compulsivity.

Subjective well-being can be predicted by caudate volume and promotion focus

It is well-known that the caudate nucleus is associated with motivational behaviors and subjective well-being. However, no longitudinal studies have examined the relationship between brain structure, behavioral orientations, and subjective well-being. This study analyzes data from our previous longitudinal study to examine whether future subjective well-being can be predicted by the volume of the caudate nucleus. We also examined whether behavioral orientation, based on the regulatory focus theory showing two orientations-promotion and prevention focus-was related to the volume of the caudate nucleus. Voxel-based morphometry analysis indicated that the left caudate volume was positively associated with rating scores for future subjective well-being and promotion orientation. Further, mediation analysis indicated that promotion orientation significantly mediated the relationship between future subjective well-being and left caudate volume. The findings indicate that future subjective well-being can be predicted by the volume of the left caudate nucleus, and that this relationship is mediated by promotion focus orientation.

Abnormal functional connectivity of the putamen in obsessive-compulsive disorder

The putamen has been proposed to play a critical role in the development of obsessive-compulsive disorder (OCD). The primary objective of this study was to examine the resting-state regional activity and functional connectivity patterns of the putamen in individuals diagnosed with OCD. To achieve this, we employed resting-state functional magnetic resonance imaging (rs-fMRI) to collect data from a sample of 45 OCD patients and 53 healthy control participants. We aimed to use the regional amplitude of low-frequency fluctuation (ALFF) analysis to generate the ROI masks of the putamen and then conduct the whole brain functional connectivity of the putamen in individuals with OCD. Compared to controls, the OCD group demonstrated decreased ALFF in bilateral putamen. The right putamen also displayed decreased FC with the left putamen extending to the inferior frontal gyrus (IFG), bilateral precuneus extending to calcarine, the right middle occipital cortex extending to the right middle temporal cortex, and the left middle occipital gyrus. The decreased connectivity between the right putamen and the left IFG was negatively correlated with Yale-Brown Obsessive Compulsive scale (Y-BOCS) Obsession Scores. This study aimed to reveal the putamen changes in resting-state activity and connectivity in OCD patients, highlighting the significance of aberrant ALFF/FC of the putamen is a key characteristic of OCD.

Cocaine disrupts action flexibility via glucocorticoid receptors

Many addictive drugs increase stress hormone levels. They also alter the propensity of organisms to prospectively select actions based on long-term consequences. We hypothesized that cocaine causes inflexible action by increasing circulating stress hormone levels, activating the glucocorticoid receptor (GR). We trained mice to generate two nose pokes for food and then required them to update action-consequence associations when one response was no longer reinforced. Cocaine delivered in adolescence or adulthood impaired the capacity of mice to update action strategies, and inhibiting CORT synthesis rescued action flexibility. Next, we reduced Nr3c1, encoding GR, in the orbitofrontal cortex (OFC), a region of the brain responsible for interlacing new information into established routines. Nr3c1 silencing preserved action flexibility and dendritic spine abundance on excitatory neurons, despite cocaine. Spines are often considered substrates for learning and memory, leading to the discovery that cocaine degrades the representation of new action memories, obstructing action flexibility.

The Small G-Protein Rac1 in the Dorsomedial Striatum Promotes Alcohol-Dependent Structural Plasticity and Goal-Directed Learning in Mice

The small G-protein Ras-related C3 botulinum toxin substrate 1 (Rac1) promotes the formation of filamentous actin (F-actin). Actin is a major component of dendritic spines, and we previously found that alcohol alters actin composition and dendritic spine structure in the nucleus accumbens (NAc) and the dorsomedial striatum (DMS). To examine if Rac1 contributes to these alcohol-mediated adaptations, we measured the level of GTP-bound active Rac1 in the striatum of mice following 7 weeks of intermittent access to 20% alcohol. We found that chronic alcohol intake activates Rac1 in the DMS of male mice. In contrast, Rac1 is not activated by alcohol in the NAc and DLS of male mice or in the DMS of female mice. Similarly, closely related small G-proteins are not activated by alcohol in the DMS, and Rac1 activity is not increased in the DMS by moderate alcohol or natural reward. To determine the consequences of alcohol-dependent Rac1 activation in the DMS of male mice, we inhibited endogenous Rac1 by infecting the DMS of mice with an adeno-associated virus (AAV) expressing a dominant negative form of the small G-protein (Rac1-DN). We found that overexpression of AAV-Rac1-DN in the DMS inhibits alcohol-mediated Rac1 signaling and attenuates alcohol-mediated F-actin polymerization, which corresponded with a decrease in dendritic arborization and spine maturation. Finally, we provide evidence to suggest that Rac1 in the DMS plays a role in alcohol-associated goal-directed learning. Together, our data suggest that Rac1 in the DMS plays an important role in alcohol-dependent structural plasticity and aberrant learning.

Right thalamic volume mediates impact of the dopamine beta-hydroxylase gene on the endowment effect

The endowment effect is a tendency that individuals overvalue items belonging to them relative to those items that do not. Previous studies showed a strong relation between the dopamine beta-hydroxylase (DBH) gene and the endowment effect (EE), and a link between EE and task-based functional MRI activation in multiple brain regions. However, the role of brain structure on EE remains unclear. In this study, we have explored whether regional brain volume mediate the effect of the DBH gene on EE. Results showed that rs1611115, single-nucleotide polymorphisms (SNPs) at DBH loci, were significantly associated with right thalamus volume and the endowment effect in males but not in female participants. Specifically, male DBH rs1611115 T-carriers had larger right thalamus volume compared to carriers of CC genotype and exhibited a greater endowment effect. Importantly, we found that right thalamus volume mediated the effect of rs1611115 on the endowment effect in male participants. This study demonstrated how thalamic volume plays an important mediating role between genetics and decision-making in humans.

The human VGLUT3-pT8I mutation elicits uneven striatal DA signaling, food or drug maladaptive consumption in male mice

Cholinergic striatal interneurons (ChIs) express the vesicular glutamate transporter 3 (VGLUT3) which allows them to regulate the striatal network with glutamate and acetylcholine (ACh). In addition, VGLUT3-dependent glutamate increases ACh vesicular stores through vesicular synergy. A missense polymorphism, VGLUT3-p.T8I, was identified in patients with substance use disorders (SUDs) and eating disorders (EDs). A mouse line was generated to understand the neurochemical and behavioral impact of the p.T8I variant. In VGLUT3T8I/T8I male mice, glutamate signaling was unchanged but vesicular synergy and ACh release were blunted. Mutant male mice exhibited a reduced DA release in the dorsomedial striatum but not in the dorsolateral striatum, facilitating habit formation and exacerbating maladaptive use of drug or food. Increasing ACh tone with donepezil reversed the self-starvation phenotype observed in VGLUT3T8I/T8I male mice. Our study suggests that unbalanced dopaminergic transmission in the dorsal striatum could be a common mechanism between SUDs and EDs.

A molecularly defined orbitofrontal cortical neuron population controls compulsive-like behavior, but not inflexible choice or habit

Habits are familiar behaviors triggered by cues, not outcome predictability, and are insensitive to changes in the environment. They are adaptive under many circumstances but can be considered antecedent to compulsions and intrusive thoughts that drive persistent, potentially maladaptive behavior. Whether compulsive-like and habit-like behaviors share neural substrates is still being determined. Here, we investigated mice bred to display inflexible reward-seeking behaviors that are insensitive to action consequences. We found that these mice demonstrate habitual response biases and compulsive-like grooming behavior that was reversible by fluoxetine and ketamine. They also suffer dendritic spine attrition on excitatory neurons in the orbitofrontal cortex (OFC). Nevertheless, synaptic melanocortin 4 receptor (MC4R), a factor implicated in compulsive behavior, is preserved, leading to the hypothesis that Mc4r+ OFC neurons may drive aberrant behaviors. Repeated chemogenetic stimulation of Mc4r+ OFC neurons triggered compulsive and not inflexible or habitual response biases in otherwise typical mice. Thus, Mc4r+ neurons within the OFC appear to drive compulsive-like behavior that is dissociable from habitual behavior. Understanding which neuron populations trigger distinct behaviors may advance efforts to mitigate harmful compulsions.

Implication of regional selectivity of dopamine deficits in impaired suppressing of involuntary movements in Parkinson's disease

To improve the initiation and speed of intended action, one of the crucial mechanisms is suppressing unwanted movements that interfere with goal-directed behavior, which is observed relatively aberrant in Parkinson's disease patients. Recent research has highlighted that dopamine deficits in Parkinson's disease predominantly occur in the caudal lateral part of the substantia nigra pars compacta (SNc) in human patients. We previously found two parallel circuits within the basal ganglia, primarily divided into circuits mediated by the rostral medial part and caudal lateral part of the SNc dopamine neurons. We have further discovered that the indirect pathway in caudal basal ganglia circuits, facilitated by the caudal lateral part of the SNc dopamine neurons, plays a critical role in suppressing unnecessary involuntary movements when animals perform voluntary goal-directed actions. We thus explored recent research in humans and non-human primates focusing on the distinct functions and networks of the caudal lateral part of the SNc dopamine neurons to elucidate the mechanisms involved in the impairment of suppressing involuntary movements in Parkinson's disease patients.

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

Cortico-striatal action control inherent of opponent cognitive-motivational styles

Turning on cue or stopping at a red light requires the detection of such cues to select action sequences, or suppress action, in accordance with cue-associated action rules. Cortico-striatal projections are an essential part of the brain's attention-motor interface. Glutamate-sensing microelectrode arrays were used to measure glutamate transients in the dorsomedial striatum (DMS) of male and female rats walking a treadmill and executing cued turns and stops. Prelimbic-DMS projections were chemogenetically inhibited to determine their behavioral necessity and the cortico-striatal origin of cue-evoked glutamate transients. Furthermore, we investigated rats exhibiting preferably goal-directed (goal trackers, GTs) versus cue-driven attention (sign trackers, STs), to determine the impact of such cognitive-motivational biases on cortico-striatal control. GTs executed more cued turns and initiated such turns more slowly than STs. During turns, but not missed turns or cued stops, cue-evoked glutamate concentrations were higher in GTs than in STs. In conjunction with turn cue-evoked glutamate spike levels, the presence of a single spike rendered GTs to be almost twice as likely to turn than STs. In contrast, multiple glutamate spikes predicted GTs to be less likely to turn than STs. In GTs, but not STs, inhibition of prelimbic-DMS projections attenuated turn rates, turn cue-evoked glutamate peaks, and increased the number of spikes. These findings suggest that turn cue-evoked glutamate release in GTs is tightly controlled by cortico-striatal neuronal activity. In contrast, in STs, glutamate release from DMS glutamatergic terminals may be regulated by other striatal circuitry, preferably mediating cued suppression of action and reward tracking.

Flexible gating between subspaces in a neural network model of internally guided task switching

Behavioral flexibility relies on the brain's ability to switch rapidly between multiple tasks, even when the task rule is not explicitly cued but must be inferred through trial and error. The underlying neural circuit mechanism remains poorly understood. We investigated recurrent neural networks (RNNs) trained to perform an analog of the classic Wisconsin Card Sorting Test. The networks consist of two modules responsible for rule representation and sensorimotor mapping, respectively, where each module is comprised of a circuit with excitatory neurons and three major types of inhibitory neurons. We found that rule representation by self-sustained persistent activity across trials, error monitoring and gated sensorimotor mapping emerged from training. Systematic dissection of trained RNNs revealed a detailed circuit mechanism that is consistent across networks trained with different hyperparameters. The networks' dynamical trajectories for different rules resided in separate subspaces of population activity; the subspaces collapsed and performance was reduced to chance level when dendrite-targeting somatostatin-expressing interneurons were silenced, illustrating how a phenomenological description of representational subspaces is explained by a specific circuit mechanism.

The impact of abstinence from chronic alcohol consumption on the mouse striatal proteome: sex and subregion-specific differences

Alcohol misuse is the third leading preventable cause of death in the world. The World Health Organization currently estimates that 1 in 20 deaths are directly alcohol related. One of the ways in which consuming excessive levels of alcohol can both directly and indirectly affect human mortality and morbidity, is through chronic inflammation. Recently, studies have suggested a link between increased alcohol use and the incidence of neuroinflammatory-related diseases. However, the mechanism in which alcohol potentially influences neuroinflammatory processes is still being uncovered. We implemented an unbiased proteomics exploration of alcohol-induced changes in the striatum, with a specific emphasis on proteins related to inflammation. The striatum is a brain region that is critically involved with the progression of alcohol use disorder. Using mass spectrometry following voluntary alcohol self-administration in mice, we show that distinct protein abundances and signaling pathways in different subregions of the striatum are disrupted by chronic exposure to alcohol compared to water drinking control mice. Further, in mice that were allowed to experience abstinence from alcohol compared to mice that were non-abstinent, the overall proteome and signaling pathways showed additional differences, suggesting that the responses evoked by chronic alcohol exposure are dependent on alcohol use history. To our surprise we did not find that chronic alcohol drinking or abstinence altered protein abundance or pathways associated with inflammation, but rather affected proteins and pathways associated with neurodegeneration and metabolic, cellular organization, protein translation, and molecular transport processes. These outcomes suggest that in this drinking model, alcohol-induced neuroinflammation in the striatum is not a primary outcome controlling altered neurobehavioral function, but these changes are rather mediated by altered striatal neuronal structure and cellular health.

Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system

Despite a theory that an imbalance in goal-directed versus habitual systems serve as building blocks of compulsions, research has yet to delineate how this occurs during arbitration between the two systems in obsessive-compulsive disorder. Inspired by a brain model in which the inferior frontal cortex selectively gates the putamen to guide goal-directed or habitual actions, this study aimed to examine whether disruptions in the arbitration process via the fronto-striatal circuit would underlie imbalanced decision-making and compulsions in patients. Thirty patients with obsessive-compulsive disorder [mean (standard deviation) age = 26.93 (6.23) years, 12 females (40%)] and 30 healthy controls [mean (standard deviation) age = 24.97 (4.72) years, 17 females (57%)] underwent functional MRI scans while performing the two-step Markov decision task, which was designed to dissociate goal-directed behaviour from habitual behaviour. We employed a neurocomputational model to account for an uncertainty-based arbitration process, in which a prefrontal arbitrator (i.e. inferior frontal gyrus) allocates behavioural control to a more reliable strategy by selectively gating the putamen. We analysed group differences in the neural estimates of uncertainty of each strategy. We also compared the psychophysiological interaction effects of system preference (goal-directed versus habitual) on fronto-striatal coupling between groups. We examined the correlation between compulsivity score and the neural activity and connectivity involved in the arbitration process. The computational model captured the subjects' preferences between the strategies. Compared with healthy controls, patients had a stronger preference for the habitual system (t = -2.88, P = 0.006), which was attributed to a more uncertain goal-directed system (t = 2.72, P = 0.009). Before the allocation of controls, patients exhibited hypoactivity in the inferior frontal gyrus compared with healthy controls when this region tracked the inverse of uncertainty (i.e. reliability) of goal-directed behaviour (P = 0.001, family-wise error rate corrected). When reorienting behaviours to reach specific goals, patients exhibited weaker right ipsilateral ventrolateral prefronto-putamen coupling than healthy controls (P = 0.001, family-wise error rate corrected). This hypoconnectivity was correlated with more severe compulsivity (r = -0.57, P = 0.002). Our findings suggest that the attenuated top-down control of the putamen by the prefrontal arbitrator underlies compulsivity in obsessive-compulsive disorder. Enhancing fronto-striatal connectivity may be a potential neurotherapeutic approach for compulsivity and adaptive decision-making.

The brain's arbitration system and obsessive-compulsive disorder

This scientific commentary relates to ‘Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system’ by Kim et al. (https://doi.org/10.1093/brain/awae102).

Ceftriaxone reverses diet-induced deficits in goal-directed control

RATIONALE

Obesity is associated with numerous health risks and ever-increasing rates are a significant global concern. However, despite weight loss attempts many people have difficulty maintaining weight loss. Previous studies in animals have shown that chronic access to an obesogenic diet can disrupt goal-directed behavior, impairing the ability of animals to flexibly adjust food-seeking behavior following changes in the value of earned outcomes. Changes in behavioral control have been linked to disruption of glutamate transmission in the dorsal medial striatum (DMS), a region critical for the acquisition and expression of goal-directed behavior.

OBJECTIVES

The goal of this study was to test whether ceftriaxone, a beta-lactam antibiotic shown elsewhere to upregulate the expression of the glutamate transporter GLT-1, would improve goal-directed control following long-term exposure to an obesogenic diet.

METHODS

Male and female rats were given access to either standard chow or chow plus sweetened condensed milk (SCM) for 6 weeks. Access to SCM was ended and rats received daily injections of either ceftriaxone or saline for 6 days. Rats were then trained to press a lever to earn a novel food reward and, finally, were assessed for sensitivity to outcome devaluation. Histological analyses examined changes to GLT-1 protein levels and morphological changes to astrocytes, within the DMS.

RESULTS

We found that ceftriaxone robustly restored goal-directed behavior in animals following long-term exposure to SCM. While we did not observe changes in protein levels of GLT-1 in the DMS, we observed that SCM induced changes in the morphology of astrocytes in the DMS, and that ceftriaxone mitigated these changes.

CONCLUSIONS

These results demonstrate that long-term access to a SCM diet impairs goal-directed behavior while also altering the morphology of astrocytes in the DMS. Furthermore, these results suggest that ceftriaxone administration can reverse the impairment of goal-directed behavior potentially through its actions on astrocytes in decision-making circuitry.

Mesocorticolimbic system reactivity to alcohol use-related visual cues as a function of alcohol sensitivity phenotype: A pilot fMRI study

Low sensitivity (LS) to alcohol is a risk factor for alcohol use disorder (AUD). Compared to peers with high sensitivity (HS), LS individuals drink more, report more problems, and exhibit potentiated alcohol cue reactivity (ACR). Heightened ACR suggests LS confers AUD risk via incentive sensitization, which is thought to take place in the mesocorticolimbic system. This study examined neural ACR in LS and HS individuals. Young adults (N = 32, M age=20.3) were recruited based on the Alcohol Sensitivity Questionnaire (HS: n = 16; LS: n = 16; 9 females/group). Participants completed an event-related fMRI ACR task. Group LS had higher ACR in left ventrolateral prefrontal cortex than group HS. In group LS, ACR in left caudomedial orbitofrontal cortex or left putamen was low at low alcohol use levels and high at heavier or more problematic alcohol use levels, whereas the opposite was true in group HS. Alcohol use level also was associated with the level of ACR in left substantia nigra among males in group LS. Taken together, results suggest elevated mesocorticolimbic ACR among LS individuals, especially those using alcohol at hazardous levels. Future studies with larger samples are warranted to determine the neurobiological loci underlying LS-based amplified ACR and AUD risk.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration. We then exposed them to four days of punishment testing in which footshock was delivered randomly on one-third of trials. Before and after punishment testing (four days pre-punishment and ≥ four days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance for cocaine was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment in males, although it was associated with habitual responding both pre- and post-punishment in females, indicating that punishment resistance was predicted by habitual responding in food-seeking females. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

What Role Does Striatal Dopamine Play in Goal-directed Action?

Evidence suggests that dopamine activity provides a US-related prediction error for Pavlovian conditioning and the reinforcement signal supporting the acquisition of habits. However, its role in goal-directed action is less clear. There are currently few studies that have assessed dopamine release as animals acquire and perform self-paced instrumental actions. Here we briefly review the literature documenting the psychological, behavioral and neural bases of goal-directed actions in rats and mice, before turning to describe recent studies investigating the role of dopamine in instrumental learning and performance. Plasticity in dorsomedial striatum, a central node in the network supporting goal-directed action, clearly requires dopamine release, the timing of which, relative to cortical and thalamic inputs, determines the degree and form of that plasticity. Beyond this, bilateral release appears to reflect reward prediction errors as animals experience the consequences of an action. Such signals feedforward to update the value of the specific action associated with that outcome during subsequent performance, with dopamine release at the time of action reflecting the updated predicted action value. More recently, evidence has also emerged for a hemispherically lateralised signal associated with the action; dopamine release is greater in the hemisphere contralateral to the spatial target of the action. This effect emerges over the course of acquisition and appears to reflect the strength of the action-outcome association. Thus, during goal-directed action, dopamine release signals the action, the outcome and their association to shape the learning and performance processes necessary to support this form of behavioral control.

Anti-Hebbian plasticity drives sequence learning in striatum

Spatio-temporal activity patterns have been observed in a variety of brain areas in spontaneous activity, prior to or during action, or in response to stimuli. Biological mechanisms endowing neurons with the ability to distinguish between different sequences remain largely unknown. Learning sequences of spikes raises multiple challenges, such as maintaining in memory spike history and discriminating partially overlapping sequences. Here, we show that anti-Hebbian spike-timing dependent plasticity (STDP), as observed at cortico-striatal synapses, can naturally lead to learning spike sequences. We design a spiking model of the striatal output neuron receiving spike patterns defined as sequential input from a fixed set of cortical neurons. We use a simple synaptic plasticity rule that combines anti-Hebbian STDP and non-associative potentiation for a subset of the presented patterns called rewarded patterns. We study the ability of striatal output neurons to discriminate rewarded from non-rewarded patterns by firing only after the presentation of a rewarded pattern. In particular, we show that two biological properties of striatal networks, spiking latency and collateral inhibition, contribute to an increase in accuracy, by allowing a better discrimination of partially overlapping sequences. These results suggest that anti-Hebbian STDP may serve as a biological substrate for learning sequences of spikes.

Action sequence learning, habits, and automaticity in obsessive-compulsive disorder

This study investigates the goal/habit imbalance theory of compulsion in obsessive-compulsive disorder (OCD), which postulates enhanced habit formation, increased automaticity, and impaired goal/habit arbitration. It directly tests these hypotheses using newly developed behavioral tasks. First, OCD patients and healthy participants were trained daily for a month using a smartphone app to perform chunked action sequences. Despite similar procedural learning and attainment of habitual performance (measured by an objective automaticity criterion) by both groups, OCD patients self-reported higher subjective habitual tendencies via a recently developed questionnaire. Subsequently, in a re-evaluation task assessing choices between established automatic and novel goal-directed actions, both groups were sensitive to re-evaluation based on monetary feedback. However, OCD patients, especially those with higher compulsive symptoms and habitual tendencies, showed a clear preference for trained/habitual sequences when choices were based on physical effort, possibly due to their higher attributed intrinsic value. These patients also used the habit-training app more extensively and reported symptom relief post-study. The tendency to attribute higher intrinsic value to familiar actions may be a potential mechanism leading to compulsions and an important addition to the goal/habit imbalance hypothesis in OCD. We also highlight the potential of smartphone app training as a habit reversal therapeutic tool.

Information flow between motor cortex and striatum reverses during skill learning

The coordination of neural activity across brain areas during a specific behavior is often interpreted as neural communication involved in controlling the behavior. However, whether information relevant to the behavior is actually transferred between areas is often untested. Here, we used information-theoretic tools to quantify how motor cortex and striatum encode and exchange behaviorally relevant information about specific reach-to-grasp movement features during skill learning in rats. We found a temporal shift in the encoding of behaviorally relevant information during skill learning, as well as a reversal in the primary direction of behaviorally relevant information flow, from cortex-to-striatum during naive movements to striatum-to-cortex during skilled movements. Standard analytical methods that quantify the evolution of overall neural activity during learning-such as changes in neural signal amplitude or the overall exchange of information between areas-failed to capture these behaviorally relevant information dynamics. Using these standard methods, we instead found a consistent coactivation of overall neural signals during movement production and a bidirectional increase in overall information propagation between areas during learning. Our results show that skill learning is achieved through a transformation in how behaviorally relevant information is routed across cortical and subcortical brain areas and that isolating the components of neural activity relevant to and informative about behavior is critical to uncover directional interactions within a coactive and coordinated network.

From compulsivity to compulsion: the neural basis of compulsive disorders

Compulsive behaviour, an apparently irrational perseveration in often maladaptive acts, is a potential transdiagnostic symptom of several neuropsychiatric disorders, including obsessive-compulsive disorder and addiction, and may reflect the severe manifestation of a dimensional trait termed compulsivity. In this Review, we examine the psychological basis of compulsions and compulsivity and their underlying neural circuitry using evidence from human neuroimaging and animal models. Several main elements of this circuitry are identified, focused on fronto-striatal systems implicated in goal-directed behaviour and habits. These systems include the orbitofrontal, prefrontal, anterior cingulate and insular cortices and their connections with the basal ganglia as well as sensoriomotor and parietal cortices and cerebellum. We also consider the implications for future classification of impulsive-compulsive disorders and their treatment.

Distributional coding of associative learning in discrete populations of midbrain dopamine neurons

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

The small G-protein Rac1 in the dorsomedial striatum promotes alcohol-dependent structural plasticity and goal-directed learning in mice

The small G-protein Rac1 promotes the formation of filamentous actin (F-Actin). Actin is a major component of dendritic spines, and we previously found that alcohol alters actin composition and dendritic spine structure in the nucleus accumbens (NAc) and the dorsomedial striatum (DMS). To examine if Rac1 contributes to these alcohol-mediated adaptations, we measured the level of GTP-bound active Rac1 in the striatum of mice following 7 weeks of intermittent access to 20% alcohol. We found that chronic alcohol intake activates Rac1 in the DMS of male mice. In contrast, Rac1 is not activated by alcohol in the NAc and DLS of male mice, or in the DMS of female mice. Similarly, closely related small G-proteins are not activated by alcohol in the DMS, and Rac1 activity is not increased in the DMS by moderate alcohol or natural reward. To determine the consequences of alcohol-dependent Rac1 activation in the DMS of male mice, we inhibited endogenous Rac1 by infecting the DMS of mice with an AAV expressing a dominant negative form of the small G-protein (Rac1-DN). We found that overexpression of AAV-Rac1-DN in the DMS inhibits alcohol-mediated Rac1 signaling and attenuates alcohol-mediated F-actin polymerization, which corresponded with a decrease in dendritic arborization and spine maturation. Finally, we provide evidence to suggest that Rac1 in the DMS plays a role in alcohol-associated goal-directed learning. Together, our data suggest that Rac1 in the DMS plays an important role in alcohol-dependent structural plasticity and aberrant learning.

Significance Statement

Addiction, including alcohol use disorder, is characterized by molecular and cellular adaptations that promote maladaptive behaviors. We found that Rac1 was activated by alcohol in the dorsomedial striatum (DMS) of male mice. We show that alcohol-mediated Rac1 signaling is responsible for alterations in actin dynamics and neuronal morphology. We also present data to suggest that Rac1 is important for alcohol-associated learning processes. These results suggest that Rac1 in the DMS is an important contributor to adaptations that promote alcohol use disorder.

Characterizing the dynamic learning process: Implications of a quantitative analysis

Understanding the neural mechanisms involved in learning processes is crucial for unraveling the complexities of behavior and cognition. Sudden change from the untrained level to the fully-learned level is a pivotal feature of instrumental learning. However, the concept of change point and suitable methods to conveniently analyze the characteristics of sudden change in groups remain elusive, which might hinder a fuller understanding of the neural mechanism underlying dynamic leaning process. In the current study, we investigated the learning processes of mice that were trained in an aversive instrumental learning task, and introduced a novel strategy to analyze behavioral variations in instrumental learning, leading to improved clarity on the concept of sudden change and enabling comprehensive group analysis. By applying this novel strategy, we examined the effects of cocaine and a cannabinoid receptor agonist on instrumental learning. Intriguingly, our analysis revealed significant differences in timing and occurrence of sudden changes that were previously overlooked using traditional analysis. Overall, our research advances understanding of behavioral variation during instrumental learning and the interplay between learning behaviors and neurotransmitter systems, contributing to a deeper comprehension of learning processes and informing future investigations and therapeutic interventions.

Lateral Orbitofrontal Cortex Encodes Presence of Risk and Subjective Risk Preference During Decision-Making

Adaptive decision-making requires consideration of objective risks and rewards associated with each option, as well as subjective preference for risky/safe alternatives. Inaccurate risk/reward estimations can engender excessive risk-taking, a central trait in many psychiatric disorders. The lateral orbitofrontal cortex (lOFC) has been linked to many disorders associated with excessively risky behavior and is ideally situated to mediate risky decision-making. Here, we used single-unit electrophysiology to measure neuronal activity from lOFC of freely moving rats performing in a punishment-based risky decision-making task. Subjects chose between a small, safe reward and a large reward associated with either 0% or 50% risk of concurrent punishment. lOFC activity repeatedly encoded current risk in the environment throughout the decision-making sequence, signaling risk before, during, and after a choice. In addition, lOFC encoded reward magnitude, although this information was only evident during action selection. A Random Forest classifier successfully used neural data accurately to predict the risk of punishment in any given trial, and the ability to predict choice via lOFC activity differentiated between and risk-preferring and risk-averse rats. Finally, risk preferring subjects demonstrated reduced lOFC encoding of risk and increased encoding of reward magnitude. These findings suggest lOFC may serve as a central decision-making hub in which external, environmental information converges with internal, subjective information to guide decision-making in the face of punishment risk.

A novel free-operant framework enables experimental habit induction in humans

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

Dopamine transients follow a striatal gradient of reward time horizons

Animals make predictions to guide their behavior and update those predictions through experience. Transient increases in dopamine (DA) are thought to be critical signals for updating predictions. However, it is unclear how this mechanism handles a wide range of behavioral timescales-from seconds or less (for example, if singing a song) to potentially hours or more (for example, if hunting for food). Here we report that DA transients in distinct rat striatal subregions convey prediction errors based on distinct time horizons. DA dynamics systematically accelerated from ventral to dorsomedial to dorsolateral striatum, in the tempo of spontaneous fluctuations, the temporal integration of prior rewards and the discounting of future rewards. This spectrum of timescales for evaluative computations can help achieve efficient learning and adaptive motivation for a broad range of behaviors.

Optogenetic stimulation of corticostriatal circuits improves behavioral flexibility in mice with prenatal alcohol exposure

Fetal alcohol spectrum disorder (FASD) is the most common preventable form of developmental and neurobehavioral disability. Animal models have demonstrated that even low to moderate prenatal alcohol exposure (PAE) is sufficient to impair behavioral flexibility in multiple domains. Previously, utilizing a moderate limited access drinking in the dark paradigm, we have shown that PAE 1) impairs touchscreen pairwise visual reversal in male adult offspring 2) leads to small but significant decreases in orbitofrontal (OFC) firing rates 3) significantly increases dorsal striatum (dS) activity and 4) aberrantly sustains OFC-dS synchrony across early reversal. In the current study, we examined whether optogenetic stimulation of OFC-dS projection neurons would be sufficient to rescue the behavioral inflexibility induced by PAE in male C57BL/6J mice. Following discrimination learning, we targeted OFC-dS projections using a retrograde adeno-associated virus (AAV) delivered to the dS which expressed channel rhodopsin (ChR2). During the first four sessions of reversal learning, we delivered high frequency optogenetic stimulation to the OFC via optic fibers immediately following correct choice responses. Our results show that optogenetic stimulation significantly reduced the number of sessions, incorrect responses, and correction errors required to move past the early perseverative phase for both PAE and control mice. In addition, OFC-dS stimulation during early reversal learning reduced the increased sessions, correct and incorrect responding seen in PAE mice during the later learning phase of reversal but did not significantly alter later performance in control ChR2 mice. Taken together these results suggest that stimulation of OFC-dS projections can improve early reversal learning in PAE and control mice, and these improvements can persist even into later stages of the task days later. These studies provide an important foundation for future clinical approaches to improve executive control in those with FASD. This article is part of the Special Issue on "PFC circuit function in psychiatric disease and relevant models".

Combining physical & cognitive training with iPSC-derived dopaminergic neuron transplantation promotes graft integration & better functional outcome in parkinsonian marmosets

Parkinson's disease (PD) is a relentlessly progressive and currently incurable neurodegenerative disease with significant unmet medical needs. Since PD stems from the degeneration of midbrain dopaminergic (DA) neurons in a defined brain location, PD patients are considered optimal candidates for cell replacement therapy. Clinical trials for cell transplantation in PD are beginning to re-emerge worldwide with a new focus on induced pluripotent stem cells (iPSCs) as a source of DA neurons since they can be derived from adult somatic cells and produced in large quantities under current good manufacturing practices. However, for this therapeutic strategy to be realized as a viable clinical option, fundamental translational challenges need to be addressed including the manufacturing process, purity and efficacy of the cells, the method of delivery, the extent of host reinnervation and the impact of patient-centered adjunctive interventions. In this study we report on the impact of physical and cognitive training (PCT) on functional recovery in the nonhuman primate (NHP) model of PD after cell transplantation. We observed that at 6 months post-transplant, the PCT group returned to normal baseline in their daily activity measured by actigraphy, significantly improved in their sensorimotor and cognitive tasks, and showed enhanced synapse formation between grafted cells and host cells. We also describe a robust, simple, efficient, scalable, and cost-effective manufacturing process of engraftable DA neurons derived from iPSCs. This study suggests that integrating PCT with cell transplantation therapy could promote optimal graft functional integration and better outcome for patients with PD.

Striatal insights: a cellular and molecular perspective on repetitive behaviors in pathology

Animals often behave repetitively and predictably. These repetitive behaviors can have a component that is learned and ingrained as habits, which can be evolutionarily advantageous as they reduce cognitive load and the expenditure of attentional resources. Repetitive behaviors can also be conscious and deliberate, and may occur in the absence of habit formation, typically when they are a feature of normal development in children, or neuropsychiatric disorders. They can be considered pathological when they interfere with social relationships and daily activities. For instance, people affected by obsessive-compulsive disorder, autism spectrum disorder, Huntington's disease and Gilles de la Tourette syndrome can display a wide range of symptoms like compulsive, stereotyped and ritualistic behaviors. The striatum nucleus of the basal ganglia is proposed to act as a master regulator of these repetitive behaviors through its circuit connections with sensorimotor, associative, and limbic areas of the cortex. However, the precise mechanisms within the striatum, detailing its compartmental organization, cellular specificity, and the intricacies of its downstream connections, remain an area of active research. In this review, we summarize evidence across multiple scales, including circuit-level, cellular, and molecular dimensions, to elucidate the striatal mechanisms underpinning repetitive behaviors and offer perspectives on the implicated disorders. We consider the close relationship between behavioral output and transcriptional changes, and thereby structural and circuit alterations, including those occurring through epigenetic processes.

Striatal dopamine release tracks the relationship between actions and their consequences

The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.

Neural inhibition as implemented by an actor-critic model involves the human dorsal striatum and ventral tegmental area

Inhibition is implicated across virtually all human experiences. As a trade-off of being very efficient, this executive function is also prone to many errors. Rodent and computational studies show that midbrain regions play crucial roles during errors by sending dopaminergic learning signals to the basal ganglia for behavioural adjustment. However, the parallels between animal and human neural anatomy and function are not determined. We scanned human adults while they performed an fMRI inhibitory task requiring trial-and-error learning. Guided by an actor-critic model, our results implicate the dorsal striatum and the ventral tegmental area as the actor and the critic, respectively. Using a multilevel and dimensional approach, we also demonstrate a link between midbrain and striatum circuit activity, inhibitory performance, and self-reported autistic and obsessive-compulsive subclinical traits.

Adaptation of sequential action benefits from timing variability related to lateral basal ganglia circuitry

Streamlined action sequences must remain flexible should stable contingencies in the environment change. By combining analyses of behavioral structure with a circuit-specific manipulation in mice, we report on a relationship between action timing variability and successful adaptation that relates to post-synaptic targets of primary motor cortical (M1) projections to dorsolateral striatum (DLS). In a two-lever instrumental task, mice formed successful action sequences by, first, establishing action scaffolds and, second, smoothly extending action duration to adapt to increased task requirements. Interruption of DLS neurons in M1 projection territories altered this process, evoking higher-rate actions that were more stereotyped in their timing, reducing opportunities for success. Based on evidence from neuronal tracing experiments, we propose that DLS neurons in M1 projection territories supply action timing variability to facilitate adaptation, a function that may involve additional downstream subcortical processing relating to collateralization of descending motor pathways to multiple basal ganglia centers.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Stimulus awareness is necessary for both instrumental learning and instrumental responding to previously learned stimuli

Instrumental conditioning is a crucial part of adaptive behaviour, allowing agents to selectively interact with stimuli in their environment. Recent evidence suggests that instrumental conditioning cannot proceed without stimulus awareness. However, whether accurate unconscious instrumental responding can emerge from consciously acquired knowledge of the stimulus-action-outcome contingencies is unknown. We studied this question using instrumental trace conditioning, where participants learned to make approach/avoid decisions in two within-subject modes: conscious (stimuli in plain view) and unconscious (visually masked). Both tasks were followed by an unconscious-only instrumental performance task. We show that even when the contingencies are reliably learned in the conscious mode, participants fail to act upon them in the unconscious responding task. We also replicate the previous results that no instrumental learning occurs in the unconscious mode. Consequently, the absence of stimulus awareness not only precludes instrumental conditioning, but also precludes any kind of instrumental responding to already known stimuli. This suggests that instrumental behaviour is entirely supported by conscious awareness of the world, and corroborates the proposals that consciousness may be necessary for adaptive behaviours requiring selective action.

The influence of pitcher handedness on pitch-calling behavior: Insights from fMRI study on baseball umpires

This functional magnetic resonance imaging study delves into the impact of experience and pitcher handedness on the pitch-calling behavior of baseball umpires. Expert and intermediate umpires were asked to make ball/strike calls on videotaped pitches of left- and right-handed pitchers and rate their certainty for the call while undergoing scanning. Behavioral results replicated previous findings that expert umpires were more certain but not more accurate or quicker than intermediate umpires, suggesting that, as sports officials, umpires may learn to project confidence to maintain control of the game. At the neural level, expert umpires exhibited more extensive and pronounced activations within the action observation network, dorsal striatum, and cerebellum. These heightened neural responses were probably associated with their enhanced visual processing abilities for pitching action and ball trajectory, honed over years of officiating. Notably, both expert and intermediate umpires exhibited decreased accuracy when judging pitches from left-handed pitchers compared to right-handed ones. These challenges in accuracy corresponded with weaker neural activations in the aforementioned brain regions, implying difficulties in processing specific visual details of the rarely encountered left-handed pitchers. Moreover, slightly longer reaction times and reduced uncertainty were observed particularly for left-handed ball pitches, as revealed by lower activation in the right premotor cortex, highlighting issues with predictive processing. In summary, our findings shed light on the influence of pitcher handedness on the pitch-calling behavior of baseball umpires and extend the current understanding of the perceptual and decision-making behavior of sports officials.

Comparative Roles of the Caudate and Putamen in the Serial Order of Behavior: Effects of Striatal Glutamate Receptor Blockade on Variable versus Fixed Spatial Self-Ordered Sequencing in Marmosets

Self-ordered sequencing is an important executive function involving planning and executing a series of steps to achieve goal-directed outcomes. The lateral frontal cortex is implicated in this behavior, but downstream striatal outputs remain relatively unexplored. We trained marmosets on a three-stimulus self-ordered spatial sequencing task using a touch-sensitive screen to explore the role of the caudate nucleus and putamen in random and fixed response arrays. By transiently blocking glutamatergic inputs to these regions, using intrastriatal CNQX microinfusions, we demonstrate that the caudate and putamen are both required for, but contribute differently to, flexible and fixed sequencing. CNQX into either the caudate or putamen impaired variable array accuracy, and infusions into both simultaneously elicited greater impairment. We demonstrated that continuous perseverative errors in variable array were caused by putamen infusions, likely due to interference with the putamen's established role in monitoring motor feedback. Caudate infusions, however, did not affect continuous errors, but did cause an upward trend in recurrent perseveration, possibly reflecting interference with the caudate's established role in spatial working memory and goal-directed planning. In contrast to variable array performance, while both caudate and putamen infusions impaired fixed array responding, the combined effects were not additive, suggesting possible competing roles. Infusions into either region individually, but not simultaneously, led to continuous perseveration. Recurrent perseveration in fixed arrays was caused by putamen, but not caudate, infusions. These results are consistent overall with a role of caudate in planning and flexible responding and the putamen in more rigid habitual or automatic responding.