Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes

Neural interactions in the human frontal cortex dissociate reward and punishment learning

How human prefrontal and insular regions interact while maximizing rewards and minimizing punishments is unknown. Capitalizing on human intracranial recordings, we demonstrate that the functional specificity toward reward or punishment learning is better disentangled by interactions compared to local representations. Prefrontal and insular cortices display non-selective neural populations to rewards and punishments. Non-selective responses, however, give rise to context-specific interareal interactions. We identify a reward subsystem with redundant interactions between the orbitofrontal and ventromedial prefrontal cortices, with a driving role of the latter. In addition, we find a punishment subsystem with redundant interactions between the insular and dorsolateral cortices, with a driving role of the insula. Finally, switching between reward and punishment learning is mediated by synergistic interactions between the two subsystems. These results provide a unifying explanation of distributed cortical representations and interactions supporting reward and punishment learning.

Human mutations in high-confidence Tourette disorder genes affect sensorimotor behavior, reward learning, and striatal dopamine in mice

Tourette disorder (TD) is poorly understood, despite affecting 1/160 children. A lack of animal models possessing construct, face, and predictive validity hinders progress in the field. We used CRISPR/Cas9 genome editing to generate mice with mutations orthologous to human de novo variants in two high-confidence Tourette genes, CELSR3 and WWC1. Mice with human mutations in Celsr3 and Wwc1 exhibit cognitive and/or sensorimotor behavioral phenotypes consistent with TD. Sensorimotor gating deficits, as measured by acoustic prepulse inhibition, occur in both male and female Celsr3 TD models. Wwc1 mice show reduced prepulse inhibition only in females. Repetitive motor behaviors, common to Celsr3 mice and more pronounced in females, include vertical rearing and grooming. Sensorimotor gating deficits and rearing are attenuated by aripiprazole, a partial agonist at dopamine type II receptors. Unsupervised machine learning reveals numerous changes to spontaneous motor behavior and less predictable patterns of movement. Continuous fixed-ratio reinforcement shows that Celsr3 TD mice have enhanced motor responding and reward learning. Electrically evoked striatal dopamine release, tested in one model, is greater. Brain development is otherwise grossly normal without signs of striatal interneuron loss. Altogether, mice expressing human mutations in high-confidence TD genes exhibit face and predictive validity. Reduced prepulse inhibition and repetitive motor behaviors are core behavioral phenotypes and are responsive to aripiprazole. Enhanced reward learning and motor responding occur alongside greater evoked dopamine release. Phenotypes can also vary by sex and show stronger affection in females, an unexpected finding considering males are more frequently affected in TD.

The roots of polarization in the individual reward system

Polarization raises concerns for democracy and society, which have expanded in the internet era where (mis)information has become ubiquitous, its transmission faster than ever, and the freedom and means of opinion expressions are expanding. The origin of polarization however remains unclear, with multiple social and emotional factors and individual reasoning biases likely to explain its current forms. In the present work, we adopt a principled approach and show that polarization tendencies can take root in biased reward processing of new information in favour of choice confirmatory evidence. Through agent-based simulations, we show that confirmation bias in individual learning is an independent mechanism and could be sufficient for creating polarization at group level independently of any additional assumptions about the opinions themselves, a priori beliefs about them, information transmission mechanisms or the structure of social relationship between individuals. This generative process can interact with polarization mechanisms described elsewhere, but constitutes an entrenched biological tendency that helps explain the extraordinary resilience of polarization against mitigating efforts such as dramatic informational change in the environment.

Human mutations in high-confidence Tourette disorder genes affect sensorimotor behavior, reward learning, and striatal dopamine in mice

Tourette disorder (TD) is poorly understood, despite affecting 1/160 children. A lack of animal models possessing construct, face, and predictive validity hinders progress in the field. We used CRISPR/Cas9 genome editing to generate mice with mutations orthologous to human de novo variants in two high-confidence Tourette genes, CELSR3 and WWC1 . Mice with human mutations in Celsr3 and Wwc1 exhibit cognitive and/or sensorimotor behavioral phenotypes consistent with TD. Sensorimotor gating deficits, as measured by acoustic prepulse inhibition, occur in both male and female Celsr3 TD models. Wwc1 mice show reduced prepulse inhibition only in females. Repetitive motor behaviors, common to Celsr3 mice and more pronounced in females, include vertical rearing and grooming. Sensorimotor gating deficits and rearing are attenuated by aripiprazole, a partial agonist at dopamine type II receptors. Unsupervised machine learning reveals numerous changes to spontaneous motor behavior and less predictable patterns of movement. Continuous fixed-ratio reinforcement shows Celsr3 TD mice have enhanced motor responding and reward learning. Electrically evoked striatal dopamine release, tested in one model, is greater. Brain development is otherwise grossly normal without signs of striatal interneuron loss. Altogether, mice expressing human mutations in high-confidence TD genes exhibit face and predictive validity. Reduced prepulse inhibition and repetitive motor behaviors are core behavioral phenotypes and are responsive to aripiprazole. Enhanced reward learning and motor responding occurs alongside greater evoked dopamine release. Phenotypes can also vary by sex and show stronger affection in females, an unexpected finding considering males are more frequently affected in TD.

Significance Statement

We generated mouse models that express mutations in high-confidence genes linked to Tourette disorder (TD). These models show sensorimotor and cognitive behavioral phenotypes resembling TD-like behaviors. Sensorimotor gating deficits and repetitive motor behaviors are attenuated by drugs that act on dopamine. Reward learning and striatal dopamine is enhanced. Brain development is grossly normal, including cortical layering and patterning of major axon tracts. Further, no signs of striatal interneuron loss are detected. Interestingly, behavioral phenotypes in affected females can be more pronounced than in males, despite male sex bias in the diagnosis of TD. These novel mouse models with construct, face, and predictive validity provide a new resource to study neural substrates that cause tics and related behavioral phenotypes in TD.

Reward and punishment learning deficits among bipolar disorder subtypes

BACKGROUND

Reward sensitivity is an essential dimension related to mood fluctuations in bipolar disorder (BD), but there is currently a debate around hypersensitivity or hyposensitivity hypotheses to reward in BD during remission, probably related to a heterogeneous population within the BD spectrum and a lack of reward bias evaluation. Here, we examine reward maximization vs. punishment avoidance learning within the BD spectrum during remission.

METHODS

Patients with BD-I (n = 45), BD-II (n = 34) and matched (n = 30) healthy controls (HC) were included. They performed an instrumental learning task designed to dissociate reward-based from punishment-based reinforcement learning. Computational modeling was used to identify the mechanisms underlying reinforcement learning performances.

RESULTS

Behavioral results showed a significant reward learning deficit across BD subtypes compared to HC, captured at the computational level by a lower sensitivity to rewards compared to punishments in both BD subtypes. Computational modeling also revealed a higher choice randomness in BD-II compared to BD-I that reflected a tendency of BD-I to perform better during punishment avoidance learning than BD-II.

LIMITATIONS

Our patients were not naive to antipsychotic treatment and were not euthymic (but in syndromic remission) according to the International Society for Bipolar Disorder definition.

CONCLUSIONS

Our results are consistent with the reward hyposensitivity theory in BD. Computational modeling suggests distinct underlying mechanisms that produce similar observable behaviors, making it a useful tool for distinguishing how symptoms interact in BD versus other disorders. In the long run, a better understanding of these processes could contribute to better prevention and management of BD.

Impulse control disorder in Parkinson's disease is associated with abnormal frontal value signalling

Dopaminergic medication is well established to boost reward- versus punishment-based learning in Parkinson's disease. However, there is tremendous variability in dopaminergic medication effects across different individuals, with some patients exhibiting much greater cognitive sensitivity to medication than others. We aimed to unravel the mechanisms underlying this individual variability in a large heterogeneous sample of early-stage patients with Parkinson's disease as a function of comorbid neuropsychiatric symptomatology, in particular impulse control disorders and depression. One hundred and ninety-nine patients with Parkinson's disease (138 ON medication and 61 OFF medication) and 59 healthy controls were scanned with functional MRI while they performed an established probabilistic instrumental learning task. Reinforcement learning model-based analyses revealed medication group differences in learning from gains versus losses, but only in patients with impulse control disorders. Furthermore, expected-value related brain signalling in the ventromedial prefrontal cortex was increased in patients with impulse control disorders ON medication compared with those OFF medication, while striatal reward prediction error signalling remained unaltered. These data substantiate the hypothesis that dopamine's effects on reinforcement learning in Parkinson's disease vary with individual differences in comorbid impulse control disorder and suggest they reflect deficient computation of value in medial frontal cortex, rather than deficient reward prediction error signalling in striatum. See Michael Browning (https://doi.org/10.1093/brain/awad248) for a scientific commentary on this article.

Convergent imaging-transcriptomic evidence for disturbed iron homeostasis in Gilles de la Tourette syndrome

Gilles de la Tourette syndrome (GTS) is a neuropsychiatric movement disorder with reported abnormalities in various neurotransmitter systems. Considering the integral role of iron in neurotransmitter synthesis and transport, it is hypothesized that iron exhibits a role in GTS pathophysiology. As a surrogate measure of brain iron, quantitative susceptibility mapping (QSM) was performed in 28 patients with GTS and 26 matched controls. Significant susceptibility reductions in the patients, consistent with reduced local iron content, were obtained in subcortical regions known to be implicated in GTS. Regression analysis revealed a significant negative association of tic scores and striatal susceptibility. To interrogate genetic mechanisms that may drive these reductions, spatially specific relationships between susceptibility and gene-expression patterns from the Allen Human Brain Atlas were assessed. Correlations in the striatum were enriched for excitatory, inhibitory, and modulatory neurochemical signaling mechanisms in the motor regions, mitochondrial processes driving ATP production and iron‑sulfur cluster biogenesis in the executive subdivision, and phosphorylation-related mechanisms affecting receptor expression and long-term potentiation in the limbic subdivision. This link between susceptibility reductions and normative transcriptional profiles suggests that disruptions in iron regulatory mechanisms are involved in GTS pathophysiology and may lead to pervasive abnormalities in mechanisms regulated by iron-containing enzymes.

Decision-making under risk and ambiguity in adults with Tourette syndrome

BACKGROUND

Tourette syndrome (TS) as well as its most common comorbidities are associated with a higher propensity for risky behaviour in everyday life. However, it is unclear whether this increased risk propensity in real-life contexts translates into a generally increased attitude towards risk. We aimed to assess decision-making under risk and ambiguity based on prospect theory by considering the effects of comorbidities and medication.

METHODS

Fifty-four individuals with TS and 32 healthy controls performed risk and ambiguity decision-making tasks under both gains and losses conditions. Behavioural and computational parameters were evaluated using (i) univariate analysis to determine parameters difference taking independently; (ii) supervised multivariate analysis to evaluate whether our parameters could jointly account for between-group differences (iii) unsupervised multivariate analysis to explore the potential presence of sub-groups.

RESULTS

Except for general 'noisier' (less consistent) decisions in TS, we showed no specific risk-taking behaviour in TS or any relation with tics severity or antipsychotic medication. However, the presence of comorbidities was associated with distortion of decision-making. Specifically, TS with obsessive-compulsive disorder comorbidity was associated with a higher risk-taking profile to increase gain and a higher risk-averse profile to decrease loss. TS with attention-deficit hyperactivity disorder comorbidity was associated with risk-seeking in the ambiguity context to reduce a potential loss.

CONCLUSIONS

Impaired valuation of risk and ambiguity was not related to TS per se. Our findings are important for clinical practice: the involvement of individuals with TS in real-life risky situations may actually rather result from other factors such as psychiatric comorbidities.

Convergent imaging-transcriptomic evidence for disturbed iron homeostasis in Gilles de la Tourette syndrome

Gilles de la Tourette syndrome (GTS) is a neuropsychiatric movement disorder with reported abnormalities in various neurotransmitter systems. Considering the integral role of iron in neurotransmitter synthesis and transport, it is hypothesized that iron exhibits a role in GTS pathophysiology. As a surrogate measure of brain iron, quantitative susceptibility mapping (QSM) was performed in 28 patients with GTS and 26 matched controls. Significant susceptibility reductions in the patient cohort, consistent with reduced local iron content, were obtained in subcortical regions known to be implicated in GTS. Regression analysis revealed a significant negative association of tic scores and striatal susceptibility. To interrogate genetic mechanisms that may drive these reductions, spatially specific relationships between susceptibility and gene-expression patterns extracted from the Allen Human Brain Atlas were assessed. Correlations in the striatum were enriched for excitatory, inhibitory, and modulatory neurochemical signaling mechanisms in the motor regions, mitochondrial processes driving ATP production and iron-sulfur cluster biogenesis in the executive subdivision, and phosphorylation-related mechanisms that affect receptor expression and long-term potentiation. This link between susceptibility reductions and normative transcriptional profiles suggests that disruptions in iron regulatory mechanisms are involved in GTS pathophysiology and may lead to pervasive abnormalities in mechanisms regulated by iron-containing enzymes.

Unconscious integration: Current evidence for integrative processing under subliminal conditions

Integrative processing is traditionally believed to be dependent on consciousness. While earlier studies within the last decade reported many types of integration under subliminal conditions (i.e. without perceptual awareness), these findings are widely challenged recently. This review evaluates the current evidence for 10 types of subliminal integration that are widely studied: arithmetic processing, object-context integration, multi-word processing, same-different processing, multisensory integration and 5 different types of associative learning. Potential methodological issues concerning awareness measures are also taken into account. It is concluded that while there is currently no reliable evidence for subliminal integration, this does not necessarily refute 'unconscious' integration defined through non-subliminal (e.g. implicit) approaches.

Serotonin modulates asymmetric learning from reward and punishment in healthy human volunteers

Instrumental learning is driven by a history of outcome success and failure. Here, we examined the impact of serotonin on learning from positive and negative outcomes. Healthy human volunteers were assessed twice, once after acute (single-dose), and once after prolonged (week-long) daily administration of the SSRI citalopram or placebo. Using computational modelling, we show that prolonged boosting of serotonin enhances learning from punishment and reduces learning from reward. This valence-dependent learning asymmetry increases subjects’ tendency to avoid actions as a function of cumulative failure without leading to detrimental, or advantageous, outcomes. By contrast, no significant modulation of learning was observed following acute SSRI administration. However, differences between the effects of acute and prolonged administration were not significant. Overall, these findings may help explain how serotonergic agents impact on mood disorders.

Two factors can drive learning: punishment of failures and reward of successes. Serotonin induces a valence-dependent learning asymmetry, as revealed by prolonged administering of SSRIs to healthy participants in a gambling task.

The computational roots of positivity and confirmation biases in reinforcement learning

Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.

Role of dopamine and clinical heterogeneity in cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) is commonly treated with dopaminergic medication, which enhances some, while impairing other cognitive functions. It can even contribute to impulse control disorder and addiction. We describe the history of research supporting the dopamine overdose hypothesis, which accounts for the large within-patient variability in dopaminergic medication effects across different tasks by referring to the spatially non-uniform pattern of dopamine depletion in dorsal versus ventral striatum. However, there is tremendous variability in dopaminergic medication effects not just within patients across distinct tasks, but also across different patients. In the second part of this chapter we review recent studies addressing the large individual variability in the negative side effects of dopaminergic medication on functions that implicate dopamine, such as value-based learning and choice. These studies begin to unravel the mechanisms of dopamine overdosing, thus revising the strict version of the overdose hypothesis. For example, the work shows that the canonical boosting of reward-versus punishment-based choice by medication is greater in patients with depression and a non-tremor phenotype, which both implicate, among other pathology, more rather than less severe dysregulation of the mesolimbic dopamine system. Future longitudinal cohort studies are needed to identify how to optimally combine different clinical, personality, cognitive, neural, genetic and molecular predictors of detrimental medication effects in order to account for as much of the relevant variability as possible. This will provide a useful tool for precision neurology, allowing individual and contextual tailoring of (the dose of) dopaminergic medication in order to maximize its cognitive benefits, yet minimize its side effects.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

The sooner the better: clinical and neural correlates of impulsive choice in Tourette disorder

Reward sensitivity has been suggested as one of the central pathophysiological mechanisms in Tourette disorder. However, the subjective valuation of a reward by introduction of delay has received little attention in Tourette disorder, even though it has been suggested as a trans-diagnostic feature of numerous neuropsychiatric disorders. We aimed to assess delay discounting in Tourette disorder and to identify its brain functional correlates. We evaluated delayed discounting and its brain functional correlates in a large group of 54 Tourette disorder patients and 31 healthy controls using a data-driven approach. We identified a subgroup of 29 patients with steeper reward discounting, characterised by a higher burden of impulse-control disorders and a higher level of general impulsivity compared to patients with normal behavioural performance or to controls. Reward discounting was underpinned by resting-state activity of a network comprising the orbito-frontal, cingulate, pre-supplementary motor area, temporal and insular cortices, as well as ventral striatum and hippocampus. Within this network, (i) lower connectivity of pre-supplementary motor area with ventral striatum predicted a higher impulsivity and a steeper reward discounting and (ii) a greater connectivity of pre-supplementary motor area with anterior insular cortex predicted steeper reward discounting and more severe tics. Overall, our results highlight the heterogeneity of the delayed reward processing in Tourette disorder, with steeper reward discounting being a marker of burden in impulsivity and impulse control disorders, and the pre-supplementary motor area being a hub region for the delay discounting, impulsivity and tic severity.

Exploring reward-related attention selectivity deficits in Parkinson's disease

An important aspect of managing a limited cognitive resource like attention is to use the reward value of stimuli to prioritize the allocation of attention to higher-value over lower-value stimuli. Recent evidence suggests this depends on dopaminergic signaling of reward. In Parkinson's disease, both reward sensitivity and attention are impaired, but whether these deficits are directly related to one another is unknown. We tested whether Parkinson's patients use reward information when automatically allocating their attention and whether this is modulated by dopamine replacement. We compared patients, tested both ON and OFF dopamine replacement medication, to older controls using a standard attention capture task. First, participants learned the different reward values of stimuli. Then, these reward-associated stimuli were used as distractors in a visual search task. We found that patients were generally distracted by the presence of the distractors but that the degree of distraction caused by the high-value and low-value distractors was similar. Furthermore, we found no evidence to support the possibility that dopamine replacement modulates the effect of reward on automatic attention allocation. Our results suggest a possible inability in Parkinson's patients to use the reward value of stimuli when automatically allocating their attention, and raise the possibility that reward-driven allocation of resources may affect the adaptive modulation of other cognitive processes.

Recovering Reliable Idiographic Biological Parameters from Noisy Behavioral Data: the Case of Basal Ganglia Indices in the Probabilistic Selection Task

Behavioral data, despite being a common index of cognitive activity, is under scrutiny for having poor reliability as a result of noise or lacking replications of reliable effects. Here, we argue that cognitive modeling can be used to enhance the test-retest reliability of the behavioral measures by recovering individual-level parameters from behavioral data. We tested this empirically with the Probabilistic Stimulus Selection (PSS) task, which is used to measure a participant’s sensitivity to positive or negative reinforcement. An analysis of 400,000 simulations from an Adaptive Control of Thought-Rational (ACT-R) model of this task showed that the poor reliability of the task is due to the instability of the end-estimates: because of the way the task works, the same participants might sometimes end up having apparently opposite scores. To recover the underlying interpretable parameters and enhance reliability, we used a Bayesian Maximum A Posteriori (MAP) procedure. We were able to obtain reliable parameters across sessions (intraclass correlation coefficient ≈ 0.5). A follow-up study on a modified version of the task also found the same pattern of results, with very poor test-retest reliability in behavior but moderate reliability in recovered parameters (intraclass correlation coefficient ≈ 0.4). Collectively, these results imply that this approach can further be used to provide superior measures in terms of reliability, and bring greater insights into individual differences.

Neuropsychiatric and Cognitive Deficits in Parkinson's Disease and Their Modeling in Rodents

Parkinson’s disease (PD) is associated with a large burden of non-motor symptoms including olfactory and autonomic dysfunction, as well as neuropsychiatric (depression, anxiety, apathy) and cognitive disorders (executive dysfunctions, memory and learning impairments). Some of these non-motor symptoms may precede the onset of motor symptoms by several years, and they significantly worsen during the course of the disease. The lack of systematic improvement of these non-motor features by dopamine replacement therapy underlines their multifactorial origin, with an involvement of monoaminergic and cholinergic systems, as well as alpha-synuclein pathology in frontal and limbic cortical circuits. Here we describe mood and neuropsychiatric disorders in PD and review their occurrence in rodent models of PD. Altogether, toxin-based rodent models of PD indicate a significant but non-exclusive contribution of mesencephalic dopaminergic loss in anxiety, apathy, and depressive-like behaviors, as well as in learning and memory deficits. Gene-based models display significant deficits in learning and memory, as well as executive functions, highlighting the contribution of alpha-synuclein pathology to these non-motor deficits. Collectively, neuropsychiatric and cognitive deficits are recapitulated to some extent in rodent models, providing partial but nevertheless useful options to understand the pathophysiology of non-motor symptoms and develop therapeutic options for these debilitating symptoms of PD.

Unconscious preparation: Effects of prime visibility on semantic generalization of task priming

Studies found that subliminal primes can be associated with specific tasks to facilitate task performance, and such learning is highly adaptive and generalizable. Meanwhile, conditioning studies suggest that aversive/reward learning and generalization actually occur at the semantic level. The current study shows that prime-task associations can also be generalized to novel word/neighbour primes from the same semantic category, and this occurs without contingency awareness. Previous studies have counterintuitively suggested that both the learning of task priming and the semantic priming of word neighbours depend on the lack of visibility. Here, we show that semantic generalization indeed depends on reduced visibility, but cannot occur subliminally. The current study shows for the first time that semantic learning and generalization can occur without any emotional or motivational factors, and that semantic priming can occur for arbitrary-linked stimuli in a context completely devoid of semantics.

The Phenomenon of Exquisite Motor Control in Tic Disorders and its Pathophysiological Implications

The unifying characteristic of movement disorders is the phenotypic presentation of abnormal motor outputs, either as isolated phenomena or in association with further clinical, often neuropsychiatric, features. However, the possibility of a movement disorder also characterized by supranormal or enhanced volitional motor control has not received attention. Based on clinical observations and cases collected over a number of years, we here describe the intriguing clinical phenomenon that people with tic disorders are often able to control specific muscle contractions as part of their tic behaviors to a degree that most humans typically cannot. Examples are given in accompanying video documentation. We explore medical literature on this topic and draw analogies with early research of fine motor control physiology in healthy humans. By systematically analyzing the probable sources of this unusual capacity, and focusing on neuroscientific accounts of voluntary motor control, sensory feedback, and the role of motor learning in tic disorders, we provide a novel pathophysiological account explaining both the presence of exquisite control over motor output and that of overall tic behaviors. We finally comment on key questions for future research on the topic and provide concluding remarks on the complex movement disorder of tic behaviors. © 2021 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

A Causal Role for the Pedunculopontine Nucleus in Human Instrumental Learning

A critical mechanism for maximizing reward is instrumental learning. In standard instrumental learning models, action values are updated on the basis of reward prediction errors (RPEs), defined as the discrepancy between expectations and outcomes. A wealth of evidence across species and experimental techniques has established that RPEs are signaled by midbrain dopamine neurons. However, the way dopamine neurons receive information about reward outcomes remains poorly understood. Recent animal studies suggest that the pedunculopontine nucleus (PPN), a small brainstem structure considered as a locomotor center, is sensitive to reward and sends excitatory projection to dopaminergic nuclei. Here, we examined the hypothesis that the PPN could contribute to reward learning in humans. To this aim, we leveraged a clinical protocol that assessed the therapeutic impact of PPN deep-brain stimulation (DBS) in three patients with Parkinson disease. PPN local field potentials (LFPs), recorded while patients performed an instrumental learning task, showed a specific response to reward outcomes in a low-frequency (alpha-beta) band. Moreover, PPN DBS selectively improved learning from rewards but not from punishments, a pattern that is typically observed following dopaminergic treatment. Computational analyses indicated that the effect of PPN DBS on instrumental learning was best captured by an increase in subjective reward sensitivity. Taken together, these results support a causal role for PPN-mediated reward signals in human instrumental learning.

Toward a Computational Neuropsychology of Cognitive Flexibility

Cognitive inflexibility is a well-documented, yet non-specific corollary of many neurological diseases. Computational modeling of covert cognitive processes supporting cognitive flexibility may provide progress toward nosologically specific aspects of cognitive inflexibility. We review computational models of the Wisconsin Card Sorting Test (WCST), which represents a gold standard for the clinical assessment of cognitive flexibility. A parallel reinforcement-learning (RL) model provides the best conceptualization of individual trial-by-trial WCST responses among all models considered. Clinical applications of the parallel RL model suggest that patients with Parkinson's disease (PD) and patients with amyotrophic lateral sclerosis (ALS) share a non-specific covert cognitive symptom: bradyphrenia. Impaired stimulus-response learning appears to occur specifically in patients with PD, whereas haphazard responding seems to occur specifically in patients with ALS. Computational modeling hence possesses the potential to reveal nosologically specific profiles of covert cognitive symptoms, which remain undetectable by traditionally applied behavioral methods. The present review exemplifies how computational neuropsychology may advance the assessment of cognitive flexibility. We discuss implications for neuropsychological assessment and directions for future research.

Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype

Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.

Parallel model-based and model-free reinforcement learning for card sorting performance

The Wisconsin Card Sorting Test (WCST) is considered a gold standard for the assessment of cognitive flexibility. On the WCST, repeating a sorting category following negative feedback is typically treated as indicating reduced cognitive flexibility. Therefore such responses are referred to as 'perseveration' errors. Recent research suggests that the propensity for perseveration errors is modulated by response demands: They occur less frequently when their commitment repeats the previously executed response. Here, we propose parallel reinforcement-learning models of card sorting performance, which assume that card sorting performance can be conceptualized as resulting from model-free reinforcement learning at the level of responses that occurs in parallel with model-based reinforcement learning at the categorical level. We compared parallel reinforcement-learning models with purely model-based reinforcement learning, and with the state-of-the-art attentional-updating model. We analyzed data from 375 participants who completed a computerized WCST. Parallel reinforcement-learning models showed best predictive accuracies for the majority of participants. Only parallel reinforcement-learning models accounted for the modulation of perseveration propensity by response demands. In conclusion, parallel reinforcement-learning models provide a new theoretical perspective on card sorting and it offers a suitable framework for discerning individual differences in latent processes that subserve behavioral flexibility.

A Computational Study of Executive Dysfunction in Amyotrophic Lateral Sclerosis

Executive dysfunction is a well-documented, yet nonspecific corollary of various neurological diseases and psychiatric disorders. Here, we applied computational modeling of latent cognition for executive control in amyotrophic lateral sclerosis (ALS) patients. We utilized a parallel reinforcement learning model of trial-by-trial Wisconsin Card Sorting Test (WCST) behavior. Eighteen ALS patients and 21 matched healthy control participants were assessed on a computerized variant of the WCST (cWCST). ALS patients showed latent cognitive symptoms, which can be characterized as bradyphrenia and haphazard responding. A comparison with results from a recent computational Parkinson's disease (PD) study (Steinke et al., 2020, J Clin Med) suggests that bradyphrenia represents a disease-nonspecific latent cognitive symptom of ALS and PD patients alike. Haphazard responding seems to be a disease-specific latent cognitive symptom of ALS, whereas impaired stimulus-response learning seems to be a disease-specific latent cognitive symptom of PD. These data were obtained from the careful modeling of trial-by-trial behavior on the cWCST, and they suggest that computational cognitive neuropsychology provides nosologically specific indicators of latent facets of executive dysfunction in ALS (and PD) patients, which remain undiscoverable for traditional behavioral cognitive neuropsychology. We discuss implications for neuropsychological assessment, and we discuss opportunities for confirmatory computational brain imaging studies.

Computational Modeling for Neuropsychological Assessment of Bradyphrenia in Parkinson's Disease

The neural mechanisms of cognitive dysfunctions in neurological diseases remain poorly understood. Here, we conjecture that this unsatisfying state-of-the-art is in part due to the non-specificity of the typical behavioral indicators for cognitive dysfunctions. Our study addresses the topic by advancing the assessment of cognitive dysfunctions through computational modeling. We investigate bradyphrenia in Parkinson's disease (PD) as an exemplary case of cognitive dysfunctions in neurological diseases. Our computational model conceptualizes trial-by-trial behavioral data as resulting from parallel cognitive and sensorimotor reinforcement learning. We assessed PD patients 'on' and 'off' their dopaminergic medication and matched healthy control (HC) participants on a computerized version of the Wisconsin Card Sorting Test. PD patients showed increased retention of learned cognitive information and decreased retention of learned sensorimotor information from previous trials in comparison to HC participants. Systemic dopamine replacement therapy did not remedy these cognitive dysfunctions in PD patients but incurred non-desirable side effects such as decreasing cognitive learning from positive feedback. Our results reveal novel insights into facets of bradyphrenia that are indiscernible by observable behavioral indicators of cognitive dysfunctions. We discuss how computational modeling may contribute to the advancement of future research on brain-behavior relationships and neuropsychological assessment.

Maladaptive avoidance patterns in Parkinson's disease are exacerbated by symptoms of depression

Parkinson's disease (PD) is a chronic, progressive neurodegenerative disorder, characterized by a loss of dopaminergic neurons in the substantia nigra pars compacta. Given that dopamine is critically involved in learning and other cognitive processes, such as working memory, dopamine loss in PD has been linked both to learning abnormalities and to cognitive dysfunction more generally in the disease. It is unclear, however, whether avoidance behavior is impacted in PD. This is significant, as this type of instrumental behavior plays an important role in both decision-making and emotional (dys) function. Consequently, the aim of the present study was to examine avoidance learning and operant extinction in PD using a computer-based task. On this task, participants control a spaceship and attempt to shoot an enemy spaceship to gain points. They also learn to hide in safe areas to protect from (i.e., avoid) aversive events (on-screen explosions and point loss). The results showed that patients with PD (N = 25) acquired an avoidance response during aversive periods to the same extent as healthy age-matched controls (N = 19); however, patients demonstrated greater hiding during safe periods not associated with aversive events, which could represent maladaptive generalization of the avoidance response. Furthermore, this impairment was more pronounced during the extinction phase, and in patients who reported higher levels of depression. These results demonstrate for the first time that PD is associated with maladaptive avoidance patterns, which could possibly contribute to the emergence of depression in the disease.

Decreased transfer of value to action in Tourette syndrome

OBJECTIVE

Tourette syndrome is a neurodevelopmental disorder putatively associated with a hyperdopaminergic state. Therefore, it seems plausible that excessive dopamine transmission in Tourette syndrome alters the ability to learn based on rewards and punishments. We tested whether Tourette syndrome patients exhibited altered reinforcement learning and corresponding feedback-related EEG deflections.

METHODS

We used a reinforcement learning task providing factual and counterfactual feedback in a sample of 15 Tourette syndrome patients and matched healthy controls whilst recording EEG. The paradigm presented various reward probabilities to enforce adaptive adjustments. We employed a computational model to derive estimates of the prediction error, which we used for single-trial regression analysis of the EEG data.

RESULTS

We found that Tourette syndrome patients showed increased choice stochasticity compared to controls. The feedback-related negativity represented an axiomatic prediction error for factual feedback and did not differ between groups. We observed attenuated P3a modulation specifically for factual feedback in Tourette syndrome patients, representing impaired coding of attention allocation.

CONCLUSION

Our findings indicate that cortical prediction error coding is unaffected by Tourette syndrome. Nonetheless, the transfer of learned values into choice formation is degraded, in line with a hyperdopaminergic state.

Dopaminergic and serotonergic modulation of social reward appraisal in zebrafish (Danio rerio) under circumstances of motivational conflict: Towards a screening test for anti-compulsive drug action

Cognitive flexibility, shown to be impaired in patients presenting with compulsions, is dependent on balanced dopaminergic and serotonergic interaction. Towards the development of a zebrafish (Danio rerio) screening test for anti-compulsive drug action, we manipulated social reward appraisal under different contexts by means of dopaminergic (apomorphine) and serotonergic (escitalopram) intervention. Seven groups of zebrafish (n = 6 per group) were exposed for 24 days (1 h per day) to either control (normal tank water), apomorphine (50 or 100 μg/L), escitalopram (500 or 1000 μg/L) or a combination (A100/E500 or A100/E1000 μg/L). Contextual reward appraisal was assessed over three phases i.e. Phase 1 (contingency association), Phase 2 (dissociative testing), and Phase 3 (re-associative testing). We demonstrate that 1) sight of social conspecifics is an inadequate motivational reinforcer under circumstances of motivational conflict, 2) dopaminergic and serotonergic intervention lessens the importance of an aversive stimulus, increasing the motivational valence of social reward, 3) while serotoninergic intervention maintains reward directed behavior, high-dose dopaminergic intervention bolsters cue-directed responses and 4) high-dose escitalopram reversed apomorphine-induced behavioral inflexibility. The results reported here are supportive of current dopamine-serotonin opponency theories and confirm the zebrafish as a potentially useful species in which to investigate compulsive-like behaviors.

Dopaminergic medication reduces striatal sensitivity to negative outcomes in Parkinson's disease

Reduced levels of dopamine in Parkinson's disease contribute to changes in learning, resulting from the loss of midbrain neurons that transmit a dopaminergic teaching signal to the striatum. Dopamine medication used by patients with Parkinson's disease has previously been linked to behavioural changes during learning as well as to adjustments in value-based decision-making after learning. To date, however, little is known about the specific relationship between dopaminergic medication-driven differences during learning and subsequent changes in approach/avoidance tendencies in individual patients. Twenty-four Parkinson's disease patients ON and OFF dopaminergic medication and 24 healthy controls subjects underwent functional MRI while performing a probabilistic reinforcement learning experiment. During learning, dopaminergic medication reduced an overemphasis on negative outcomes. Medication reduced negative (but not positive) outcome learning rates, while concurrent striatal blood oxygen level-dependent responses showed reduced prediction error sensitivity. Medication-induced shifts in negative learning rates were predictive of changes in approach/avoidance choice patterns after learning, and these changes were accompanied by systematic striatal blood oxygen level-dependent response alterations. These findings elucidate the role of dopamine-driven learning differences in Parkinson's disease, and show how these changes during learning impact subsequent value-based decision-making.

Neural reward processing in paediatric Tourette syndrome and/or attention-deficit/hyperactivity disorder

Attention-deficit/hyperactivity disorder (ADHD) is the most common comorbidity in individuals with Tourette syndrome (TS). Yet, it is unclear to what extent TS and ADHD show overlapping or distinct neural abnormalities. ADHD has been associated with altered reward processing, but there are very few studies on reward processing in TS. This study assessed neural activation of basal ganglia and thalamus during reward anticipation and receipt in children with TS and/or ADHD. We analysed mean activations of a priori specified regions of interest during an fMRI monetary incentive delay task. Data was used from 124 children aged 8-12 years (TS n = 47, of which 29 had comorbid ADHD; ADHD n = 29; healthy controls n = 48). ADHD severity across ADHD and TS groups and healthy controls was marginally related to hypoactivation of the right nucleus accumbens during reward anticipation; this effect was not moderated by TS diagnosis. We detected no associations of neural activation with TS. The association between ADHD severity and hypoactivation of the right nucleus accumbens during reward anticipation, independent of the presence or absence of TS, is in line with the view of nucleus accumbens hypoactivation as a dimensional, neurofunctional marker of ADHD severity, transcending the boundaries of primary diagnosis.

A superior ability to suppress fast inappropriate responses in children with Tourette syndrome is further improved by prospect of reward

In children with Tourette syndrome (TS), tics are often attributed to deficient self-control by health-care professionals, parents, and peers. In this behavioural study, we examined response inhibition in TS using a modified Simon task which probes the ability to solve the response conflict between a new non-spatial rule and a highly-overlearned spatial stimulus-response mapping rule. We applied a distributional analysis to the behavioural data, which grouped the trials according to the individual distribution of reaction times in four time bins. Distributional analyses enabled us to probe the children's ability to control fast, impulsive, responses, which corresponded to the trials in the fastest time bin. Additionally, we tested whether the ability to suppress inappropriate action tendencies can be improved further by the prospect of a reward. Forty-one clinically well-characterized medication-naïve children with TS, 20 children with attention-deficit/hyperactivity disorder (ADHD), and 43 typically developing children performed a Simon task during alternating epochs with and without a prospect of reward. We applied repeated measures ANCOVAs to estimate how the prospect of reward modulated reaction times and response accuracy, while taking into account the distribution of the reaction times across trials. We found between-group differences in accuracy when subjects responded relatively fast. The TS group responded more accurately than typically developing control children when resolving the response conflict introduced by the Simon task. The opposite pattern was found in children with ADHD. Prospect of reward improved accuracy rates in all groups. Although the Tourette group performed with superior accuracy in the fast trials, it was still possible for them to benefit from prospect of reward in fast trials. The findings corroborate the notion that children with TS have an enhanced capacity to inhibit fast inappropriate response tendencies. This ability can be improved further by offering a prospect of reward which might be useful during non-pharmacological therapeutic interventions.

Socially Learned Attitude Change is not reduced in Medicated Patients with Schizophrenia

Schizophrenia is often associated with distinctive or odd social behaviours. Previous work suggests this could be due to a general reduction in conformity; however, this work only assessed the tendency to publicly agree with others, which may involve a number of different mechanisms. In this study, we specifically investigated whether patients display a reduced tendency to adopt other people’s opinions (socially learned attitude change). We administered a computerized conformity task, assumed to rely on reinforcement learning circuits, to 32 patients with schizophrenia or schizo-affective disorder and 39 matched controls. Each participant rated 153 faces for trustworthiness. After each rating, they were immediately shown the opinion of a group. After approximately 1 hour, participants were unexpectedly asked to rate all the faces again. We compared the degree of attitude change towards group opinion in patients and controls. Patients presented equal or more social influence on attitudes than controls. This effect may have been medication induced, as increased conformity was seen with higher antipsychotic dose. The results suggest that there is not a general decline in conformity in medicated patients with schizophrenia and that previous findings of reduced conformity are likely related to mechanisms other than reinforcement based social influence on attitudes.

Left-shifting prism adaptation boosts reward-based learning

Visuospatial cognition has an inherent lateralized bias. Individual differences in the direction and magnitude of this bias are associated with asymmetrical D2/3 dopamine binding and dopamine system genotypes. Dopamine level affects feedback-based learning and dopamine signaling asymmetry is related to differential learning from reward and punishment. High D2 binding in the left hemisphere is associated with preference for reward. Prism adaptation (PA) is a simple sensorimotor technique, which modulates visuospatial bias according to the direction of the deviation. Left-deviating prism adaptation (LPA) induces rightward bias in healthy subjects. It is therefore possible that the right side of space increases in saliency along with left hemisphere dopaminergic activity. Right-deviating prism adaptation (RPA) has been used mainly as a control condition because it does not modulate behavior in healthy individuals. Since LPA induces a rightward visuospatial bias as a result of left hemisphere modulation, and higher dopaminergic activity in the left hemisphere is associated with preference for rewarding events we hypothesized that LPA would increase the preference for learning with reward. Healthy volunteers performed a computer-based probabilistic classification task before and after LPA or RPA. Consistent with our predictions, PA altered the preference for rewarded versus punished learning, with the LPA group exhibiting increased learning from reward. These results suggest that PA modulates dopaminergic activity in a lateralized fashion.

Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences

In economics and perceptual decision-making contextual effects are well documented, where decision weights are adjusted as a function of the distribution of stimuli. Yet, in reinforcement learning literature whether and how contextual information pertaining to decision states is integrated in learning algorithms has received comparably little attention. Here, we investigate reinforcement learning behavior and its computational substrates in a task where we orthogonally manipulate outcome valence and magnitude, resulting in systematic variations in state-values. Model comparison indicates that subjects' behavior is best accounted for by an algorithm which includes both reference point-dependence and range-adaptation-two crucial features of state-dependent valuation. In addition, we find that state-dependent outcome valuation progressively emerges, is favored by increasing outcome information and correlated with explicit understanding of the task structure. Finally, our data clearly show that, while being locally adaptive (for instance in negative valence and small magnitude contexts), state-dependent valuation comes at the cost of seemingly irrational choices, when options are extrapolated out from their original contexts.

Associative plasticity in supplementary motor area - motor cortex pathways in Tourette syndrome

The important role of the supplementary motor area (SMA) in the generation of tics and urges in Gilles de la Tourette syndrome (GTS) is underscored by an increased SMA-motor cortex (M1) connectivity. However, whether plasticity is also altered in SMA-M1 pathways is unclear. We explored whether SMA-M1 plasticity is altered in patients with Tourette syndrome. 15 patients with GTS (mean age of 33.4 years, SD = 9.9) and 19 age and sex matched healthy controls were investigated with a paired association stimulation (PAS) protocol using three transcranial magnetic stimulation (TMS) coils stimulating both M1 and the SMA. Standard clinical measures for GTS symptoms were collected. There was a significant PAS effect showing that MEP amplitudes measured in blocks during and after PAS were significantly higher compared to those in the first block. However, the degree of PAS was not differentially modulated between patients and controls as shown by a Bayesian data analysis. PAS effects in GTS correlated positively with the YGTSS motor tic severity. Plasticity previously reported to be altered in sensorimotor pathways in GTS is normal in SMA-M1 projections suggesting that the dysfunction of the SMA in GTS is not primarily related to altered plasticity in SMA-M1 connections.

What's in a word? How instructions, suggestions, and social information change pain and emotion

Instructions, suggestions, and other types of social information can have powerful effects on pain and emotion. Prominent examples include observational learning, social influence, placebo, and hypnosis. These different phenomena and their underlying brain mechanisms have been studied in partially separate literatures, which we discuss, compare, and integrate in this review. Converging findings from these literatures suggest that (1) instructions and social information affect brain systems associated with the generation of pain and emotion, and with reinforcement learning, and that (2) these changes are mediated by alterations in prefrontal systems responsible for top-down control and the generation of affective meaning. We argue that changes in expectation and appraisal, a process of assessing personal meaning and implications for wellbeing, are two potential key mediators of the effects of instructions and social information on affective experience. Finally, we propose a tentative model of how prefrontal regions, especially dorsolateral and ventromedial prefrontal cortex may regulate affective processing based on instructions and socially transmitted expectations more broadly.

Implicit sequence learning in young people with Tourette syndrome with and without co-occurring attention-deficit/hyperactivity disorder

Impaired habit-learning has been proposed to underlie the tic symptoms of Tourette syndrome (TS). However, accounts differ in terms of how habit-learning is altered in TS, with some authors proposing habit formation is impaired due to a deficient 'chunking' mechanism, and others proposing habit-learning is overactive and tics reflect hyperlearned behaviours. Attention-deficit/hyperactivity disorder (ADHD) frequently co-occurs with TS and is known to affect cognitive function in young people with co-occurring TS and ADHD (TS + ADHD). It is unclear, however, how co-occurring ADHD symptoms affect habit-learning in TS. In this study, we investigated whether young people with TS would show deficient or hyperactive habit-learning, and assessed the effects of co-occurring ADHD symptoms on habit-learning in TS. Participants aged 9-17 years with TS (n = 18), TS + ADHD (n = 17), ADHD (n = 13), and typical development (n = 20) completed a motor sequence learning task to assess habit-learning. We used a 2 (TS-yes, TS-no) × 2 (ADHD-yes, ADHD-no) factorial analysis to test the effects of TS, ADHD, and their interaction on accuracy and reaction time indices of sequence learning. TS was associated with intact sequence learning, but a tendency for difficulty transitioning from sequenced to non-sequenced performance was suggestive of hyper-learning. ADHD was associated with significantly poorer accuracy during acquisition of the sequence, indicative of impaired habit-learning. There were no interactions between the TS and ADHD factors, indicating young people with TS + ADHD showed both TS- and ADHD-related atypicalities in habit-learning.

Peromyscus maniculatus bairdii as a naturalistic mammalian model of obsessive-compulsive disorder: current status and future challenges

Obsessive-compulsive disorder (OCD) is a prevalent and debilitating condition, characterized by intrusive thoughts and repetitive behavior. Animal models of OCD arguably have the potential to contribute to our understanding of the condition. Deer mice (Permomyscus maniculatus bairdii) are characterized by stereotypic behavior which is reminiscent of OCD symptomology, and which may serve as a naturalistic animal model of this disorder. Moreover, a range of deer mouse repetitive behaviors may be representative of different compulsive-like phenotypes. This paper will review work on deer mouse behavior, and evaluate the extent to which this serves as a valid and useful model of OCD. We argue that findings over the past decade indicate that the deer mouse model has face, construct and predictive validity.

Separating the effect of reward from corrective feedback during learning in patients with Parkinson's disease

Parkinson's disease (PD) is associated with procedural learning deficits. Nonetheless, studies have demonstrated that reward-related learning is comparable between patients with PD and controls (Bódi et al., Brain, 132(9), 2385-2395, 2009; Frank, Seeberger, & O'Reilly, Science, 306(5703), 1940-1943, 2004; Palminteri et al., Proceedings of the National Academy of Sciences of the United States of America, 106(45), 19179-19184, 2009). However, because these studies do not separate the effect of reward from the effect of practice, it is difficult to determine whether the effect of reward on learning is distinct from the effect of corrective feedback on learning. Thus, it is unknown whether these group differences in learning are due to reward processing or learning in general. Here, we compared the performance of medicated PD patients to demographically matched healthy controls (HCs) on a task where the effect of reward can be examined separately from the effect of practice. We found that patients with PD showed significantly less reward-related learning improvements compared to HCs. In addition, stronger learning of rewarded associations over unrewarded associations was significantly correlated with smaller skin-conductance responses for HCs but not PD patients. These results demonstrate that when separating the effect of reward from the effect of corrective feedback, PD patients do not benefit from reward.

Premonitory urges and tics in Tourette syndrome: computational mechanisms and neural correlates

Tourette syndrome is characterized by open motor behaviors - tics - but another crucial aspect of the disorder is the presence of premonitory urges: uncomfortable sensations that typically precede tics and are temporarily alleviated by tics. We review the evidence implicating the somatosensory cortices and the insula in premonitory urges and the motor cortico-basal ganglia-thalamo-cortical loop in tics. We consider how these regions interact during tic execution, suggesting that the insula plays an important role as a nexus linking the sensory and emotional character of premonitory urges with their translation into tics. We also consider how these regions interact during tic learning, integrating the neural evidence with a computational perspective on how premonitory-urge alleviation reinforces tics.

Reward learning deficits in Parkinson's disease depend on depression

BACKGROUND

Depression is one of the most common and debilitating non-motor symptoms of Parkinson's disease (PD). The neurocognitive mechanisms underlying depression in PD are unclear and treatment is often suboptimal.

METHODS

We investigated the role of striatal dopamine in reversal learning from reward and punishment by combining a controlled medication withdrawal procedure with functional magnetic resonance imaging in 22 non-depressed PD patients and 19 PD patients with past or present depression.

RESULTS

PD patients with a depression (history) exhibited impaired reward v. punishment reversal learning as well as reduced reward v. punishment-related BOLD signal in the striatum (putamen) compared with non-depressed PD patients. No effects of dopaminergic medication were observed.

CONCLUSIONS

The present findings demonstrate that impairments in reversal learning from reward v. punishment and associated striatal signalling depend on the presence of (a history of) depression in PD.

The Roles of Phasic and Tonic Dopamine in Tic Learning and Expression

Tourette syndrome (TS) prominently involves dopaminergic disturbances, but the precise nature of those disturbances has remained elusive. A substantial body of empirical work and recent computational models have characterized the specific roles of phasic and tonic dopamine (DA) in action learning and selection, respectively. Using insights from this work and models, we suggest that TS involves increases in both phasic and tonic DA, which produce increased propensities for tic learning and expression, respectively. We review the evidence from reinforcement-learning and habit-learning studies in TS, which supports the idea that TS involves increased phasic DA responses; we also review the evidence that tics engage the habit-learning circuitry. On the basis of these findings, we suggest that tics are exaggerated, maladaptive, and persistent motor habits reinforced by aberrant, increased phasic DA responses. Increased tonic DA amplifies the tendency to execute learned tics and also provides a fertile ground of motor hyperactivity for tic learning. We review evidence suggesting that antipsychotics may counter both the increased propensity for tic expression, by increasing excitability in the indirect pathway, and the increased propensity for tic learning, by shifting plasticity in the indirect pathway toward long-term potentiation (and possibly also through more complex mechanisms). Finally, we review evidence suggesting that low doses of DA agonists that effectively treat TS decrease both phasic and tonic DA, thereby also reducing the propensity for both tic learning and tic expression, respectively.