Neuroelectric signatures of reward learning and decision-making in the human nucleus accumbens

Structural and functional correlates of olfactory reward processing in behavioral variant frontotemporal dementia

The behavioral variant of frontotemporal dementia (bvFTD) includes symptoms that reflect altered pursuit of rewards, including food, alcohol, and money. Little is known, however, about how these reward changes relate to atrophy and functional connectivity within reward-related regions. The goal of this study was to examine the structural and functional correlates of valence perception for olfactory rewards in 24 patients with bvFTD. Regression analysis of resting-state brain functional connectivity indicated that more positive valence ratings of olfactory stimuli were predicted by ventral pallidum connectivity to other reward circuit regions, particularly functional connectivity between ventral pallidum and bilateral anterior cingulate cortex/ventromedial prefrontal cortex. Structural analysis showed that atrophy of the anterior cingulate cortex was also significantly associated with perceiving stimuli as more rewarding. Finally, there was a significant interaction between ventral pallidum connectivity and atrophy of the anterior cingulate cortex. More specifically, the ventral pallidum connectivity had a greater effect on the positive perception of olfactory stimuli in the setting of low anterior cingulate cortex volume. These findings indicate that atrophy and functional connectivity within reward-relevant regions exert independent and interacting effects on the perception of pleasantness in bvFTD, potentially due to changes in hedonic "liking" signals.

Linking connectivity of deep brain stimulation of nucleus accumbens area with clinical depression improvements: a retrospective longitudinal case series

Treatment-resistant depression is a severe form of major depressive disorder and deep brain stimulation is currently an investigational treatment. The stimulation's therapeutic effect may be explained through the functional and structural connectivities between the stimulated area and other brain regions, or to depression-associated networks. In this longitudinal, retrospective study, four female patients with treatment-resistant depression were implanted for stimulation in the nucleus accumbens area at our center. We analyzed the structural and functional connectivity of the stimulation area: the structural connectivity was investigated with probabilistic tractography; the functional connectivity was estimated by combining patient-specific stimulation volumes and a normative functional connectome. These structural and functional connectivity profiles were then related to four clinical outcome scores. At 1-year follow-up, the remission rate was 66%. We observed a consistent structural connectivity to Brodmann area 25 in the patient with the longest remission phase. The functional connectivity analysis resulted in patient-specific R-maps describing brain areas significantly correlated with symptom improvement in this patient, notably the prefrontal cortex. But the connectivity analysis was mixed across patients, calling for confirmation in a larger cohort and over longer time periods.

Melanocortin agonism in a social context selectively activates nucleus accumbens in an oxytocin-dependent manner

Social deficits are debilitating features of many psychiatric disorders, including autism. While time-intensive behavioral therapy is moderately effective, there are no pharmacological interventions for social deficits in autism. Many studies have attempted to treat social deficits using the neuropeptide oxytocin for its powerful neuromodulatory abilities and influence on social behaviors and cognition. However, clinical trials utilizing supplementation paradigms in which exogenous oxytocin is chronically administered independent of context have failed. An alternative treatment paradigm suggests pharmacologically activating the endogenous oxytocin system during behavioral therapy to enhance the efficacy of therapy by facilitating social learning. To this end, melanocortin receptor agonists like Melanotan II (MTII), which induces central oxytocin release and accelerates formation of partner preference, a form of social learning, in prairie voles, are promising pharmacological tools. To model pharmacological activation of the endogenous oxytocin system during behavioral therapy, we administered MTII prior to social interactions between male and female voles. We assessed its effect on oxytocin-dependent activity in brain regions subserving social learning using Fos expression as a proxy for neuronal activation. In non-social contexts, MTII only activated hypothalamic paraventricular nucleus, a primary site of oxytocin synthesis. However, during social interactions, MTII selectively increased oxytocin-dependent activation of nucleus accumbens, a site critical for social learning. These results suggest a mechanism for the MTII-induced acceleration of partner preference formation observed in previous studies. Moreover, they are consistent with the hypothesis that pharmacologically activating the endogenous oxytocin system with a melanocortin agonist during behavioral therapy has potential to facilitate social learning.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

AoA-L2 and Usage-L2 modulate the functional neuroplasticity of the subcortex

Previous studies revealed structural differences in subcortical regions between monolinguals and bilinguals; however, whether the functional neuroplasticity of the subcortex is modulated by different bilingual experiences remains unclear. Here, we examined the effect of age of second language acquisition (AoA-L2) and usage of L2 (Usage-L2) on subcorto-cortical and intra-subcortical functional connectivity (FC) in bilinguals by using resting-state fMRI data. The relations between brain measurements and bilingual experiences were revealed by using multiple regression analysis. We found that increased AoA-L2 was mainly related to decreased subcortical FC involving the anterior thalamus, basal ganglia, and hippocampus. Increased Usage-L2 at home was mainly associated with decreased subcortical FC of the amygdala, globus pallidus, hippocampus, and nucleus accumbens. The FC of these subcortical regions displayed a positive relation with Usage-L2 in social settings. These findings reveal that bilingual experiences modulate the functional neuroplasticity of the subcortex in different ways.

Negative psychological and physiological effects of social networking site use: The example of Facebook

Social networking sites (SNS), with Facebook as a prominent example, have become an integral part of our daily lives and more than four billion people worldwide use SNS. However, the (over-)use of SNS also poses both psychological and physiological risks. In the present article, we review the scientific literature on the risk of Facebook (over-)use. Addressing this topic is critical because evidence indicates the development of problematic Facebook use ("Facebook addiction") due to excessive and uncontrolled use behavior with various psychological and physiological effects. We conducted a review to examine the scope, range, and nature of prior empirical research on the negative psychological and physiological effects of Facebook use. Our literature search process revealed a total of 232 papers showing that Facebook use is associated with eight major psychological effects (perceived anxiety, perceived depression, perceived loneliness, perceived eating disorders, perceived self-esteem, perceived life satisfaction, perceived insomnia, and perceived stress) and three physiological effects (physiological stress, human brain alteration, and affective experience state). The review also describes how Facebook use is associated with these effects and provides additional details on the reviewed literature, including research design, sample, age, and measures. Please note that the term "Facebook use" represents an umbrella term in the present work, and in the respective sections it will be made clear what kind of Facebook use is associated with a myriad of investigated psychological variables. Overall, findings indicate that certain kinds of Facebook use may come along with significant risks, both psychologically and physiologically. Based on our review, we also identify potential avenues for future research.

Performance monitoring in obsessive-compulsive disorder: Insights from internal capsule/nucleus accumbens deep brain stimulation

•

Theta phase coherence is increased following negative performance feedback.

•

Deep brain stimulation globally modulates theta phase coherence.

•

Fronto-striatal connectivity is related to OCD symptom severity.

Background

Symptoms of obsessive–compulsive disorder (OCD) are partly related to impaired cognitive control processes and theta modulations constitute an important electrophysiological marker for cognitive control processes such as signaling negative performance feedback in a fronto-striatal network. Deep brain stimulation (DBS) targeting the anterior limb of the internal capsule (ALIC)/nucleus accumbens (NAc) shows clinical efficacy in OCD, while the exact influence on the performance monitoring system remains largely unknown.

Methods

Seventeen patients with treatment-refractory OCD performed a probabilistic reinforcement learning task. Analyses were focused on 4–8 Hz (theta) power, intertrial phase coherence (ITPC) and debiased weighted Phase-Lag Index (dwPLI) in response to negative performance feedback. Combined EEG and local field potential (LFP) recordings were obtained shortly after DBS electrode implantation to investigate fronto-striatal network modulations. To assess the impact of clinically effective DBS on negative performance feedback modulations, EEG recordings were obtained pre-surgery and at follow-up with DBS on and off.

Results

Medial frontal cortex ITPC, striatal ITPC and striato-frontal dwPLI were increased following negative performance feedback. Decreased right-lateralized dwPLI was associated with pre-surgery symptom severity. ITPC was globally decreased during DBS-off.

Conclusion

We observed a theta phase coherence mediated fronto-striatal performance monitoring network. Within this network, decreased connectivity was related to increased OCD symptomatology, consistent with the idea of impaired cognitive control in OCD. While ALIC/NAc DBS decreased theta network activity globally, this effect was unrelated to clinical efficacy and performance monitoring.

Reward Processing and Circuit Dysregulation in Posttraumatic Stress Disorder

Past decades have witnessed substantial progress in understanding of neurobiological mechanisms that contribute to generation of various PTSD symptoms, including intrusive memories, physiological arousal and avoidance of trauma reminders. However, the neurobiology of anhedonia and emotional numbing in PTSD, that have been conceptualized as reward processing deficits - reward wanting (anticipation of reward) and reward liking (satisfaction with reward outcome), respectively, remains largely unexplored. Empirical evidence on reward processing in PTSD is rather limited, and no studies have examined association of reward processing abnormalities and neurocircuitry-based models of PTSD pathophysiology. The manuscript briefly summarizes "state of the science" of both human reward processing, and of PTSD implicated neurocircuitry, as well as empirical evidence of reward processing deficits in PTSD. We then summarize current gaps in the literature and outline key future directions, further illustrating it by the example of two alternative explanations of PTSD pathophysiology potentially affecting reward processing via different neurobiological pathways. Studying reward processing in PTSD will not only advance the understanding of their link, but also could enhance current treatment approaches by specifically targeting anhedonia and emotional symptoms in PTSD patients.

Error-Related Activity in Striatal Local Field Potentials and Medial Frontal Cortex: Evidence From Patients With Severe Opioid Abuse Disorder

For successful goal-directed behavior, a performance monitoring system is essential. It detects behavioral errors and initiates behavioral adaptations to improve performance. Two electrophysiological potentials are known to follow errors in reaction time tasks: the error-related negativity (ERN), which is linked to error processing, and the error positivity (Pe), which is associated with subjective error awareness. Furthermore, the correct-related negativity (CRN) is linked to uncertainty about the response outcome. Here we attempted to identify the involvement of the nucleus accumbens (NAc) in the aforementioned performance monitoring processes. To this end, we simultaneously recorded cortical activity (EEG) and local field potentials (LFP) during a flanker task performed by four patients with severe opioid abuse disorder who underwent electrode implantation in the NAc for deep brain stimulation. We observed significant accuracy-related modulations in the LFPs at the time of the ERN/CRN in two patients and at the time of Pe in three patients. These modulations correlated with the ERN in 2/8, with CRN in 5/8 and with Pe in 6/8, recorded channels, respectively. Our results demonstrate the functional interrelation of striatal and cortical processes in performance monitoring specifically related to error processing and subjective error awareness.

Local Field Potentials Reflect Dopaminergic and Non-Dopaminergic Activities within the Primate Midbrain

Midbrain dopamine neurons are thought to play a crucial role in motivating behaviors toward desired goals. While the activity of dopamine single-units is known to adhere closely to the reward prediction error (RPE) signal hypothesized by learning theory, much less is known about the dynamic coordination of population-level neuronal activities in the midbrain. Local field potentials (LFPs) are thought to reflect the changes in membrane potential synchronized across a population of neurons nearby a recording electrode. These changes involve complex combinations of local spiking activity with synaptic processing that are difficult to interpret. Here we sampled LFPs from the substantia nigra pars compacta (SNc) of behaving monkeys to determine if local population-level synchrony encodes specific aspects of a reward/effort instrumental task and whether dopamine single-units participate in that signal. We found that reward-correlated information is encoded in a low-frequency signal (<32-Hz; delta and beta bands) that is synchronized across a neural population that includes dopamine neurons. Conversely, high-frequency power (>33-Hz; gamma band) was anticorrelated with predicted reward value and dopamine single-units were never phase-locked to those frequencies. This high-frequency signal may reflect inhibitory processes that were not otherwise observable. LFP encoding of movement-related parameters was negligible. Together, LFPs provide novel insights into the multidimensional processing of reward information subserved by dopaminergic and other components of the midbrain.

Nucleus accumbens deep brain stimulation for a patient with self-injurious behavior and autism spectrum disorder: functional and structural changes of the brain: report of a case and review of literature

The aim of this report was to investigate the clinical outcome of deep brain stimulation (DBS) for autism spectrum disorder (ASD) and the functional and structural changes in the brain after DBS. We present a 14-year-old boy with ASD and self-injurious behavior (SIB) refractory with medical and behavioral therapy. He was treated by bilateral nucleus accumbens (NAc) DBS. Remarkable clinical improvement was observed following NAc DBS. Brain fluorodeoxyglucose-positron emission tomography (FDG-PET) and magnetic resonance imaging (MRI) volumetric studies revealed that the metabolism in the prefrontal and the frontal cortex as well as the occipital cortex was markedly decreased in association with the decreased cortical volumes in those areas 2 years after NAc DBS. The therapeutic potential of NAc DBS is suggested for the clinical improvement of patients with ASD and SIB with structural and functional changes after DBS.

Invasive seizure monitoring in the critically-Ill brain injury patient: Current practices and a review of the literature

Seizures commonly occur in a variety of serious neurological illnesses, and lead to additional morbidity and worsened outcomes. Recently, it has become clear that not all seizures in the acute brain injury setting are evident on scalp EEG. To address this, we have developed a protocol for depth electrode placement in the neuro-intensive care unit for patients in whom the clinical suspicion of occult seizures is high. In the current manuscript, we review the literature on depth EEG monitoring for ictal events in critically-ill, unconscious patients, focusing on the incidence of seizures not detected with scalp EEG in various conditions. We critically discuss evidence in support of and against treating these events that are only detectable on depth recordings. We describe additional specific scenarios in which depth EEG recordings may be helpful, including for the detection of delayed cerebral ischemia following subarachnoid hemorrhage. We then describe current techniques for bedside electrode placement. Finally, we outline potential avenues for future investigations, including the use of depth electrodes to describe circuit abnormalities in acute brain injury.

Perimovement decrease of alpha/beta oscillations in the human nucleus accumbens

The present work clarifies how the nucleus accumbens contributes to action. This region is often assumed to influence behavior “off-line” by evaluating outcomes. Studying rare recordings of local field potentials from the human nucleus accumbens, we observe a perimovement decrease of alpha and beta oscillations in seven of eight individuals, a signal that, in the motor system, is directly related to action preparation. Our results support the idea of an online role of this region for imminent action.

The human nucleus accumbens is thought to play an important role in guiding future action selection via an evaluation of current action outcomes. Here we provide electrophysiological evidence for a more direct, i.e., online, role during action preparation. We recorded local field potentials from the nucleus accumbens in patients with epilepsy undergoing surgery for deep brain stimulation. We found a consistent decrease in the power of alpha/beta oscillations (10–30 Hz) before and around the time of movements. This perimovement alpha/beta desynchronization was observed in seven of eight patients and was present both before instructed movements in a serial reaction time task as well as before self-paced, deliberate choices in a decision making task. A similar beta decrease over sensorimotor cortex and in the subthalamic nucleus has been directly related to movement preparation and execution. Our results support the idea of a direct role of the human nucleus accumbens in action preparation and execution.

Olanzapine Treatment of Adolescent Rats Alters Adult D2 Modulation of Cortical Inputs to the Ventral Striatum

BACKGROUND

The striatal dopamine system undergoes vast ontogenetic changes during adolescence, making the brain vulnerable to drug treatments that target this class of neurotransmitters. Atypical antipsychotic drugs are often prescribed to children and adolescents for off-label treatment of neuropsychiatric disorders, yet the long-term impact this treatment has on brain development remains largely unknown.

METHODS

Adolescent male rats were treated with olanzapine or vehicle for 3 weeks (during postnatal day 28-49) using a dosing condition designed to approximate closely D2 receptor occupancies in the human therapeutic range. We assessed D2 receptor modulation of corticostriatal inputs onto medium spiny neurons in the adult ventral striatum using in vitro whole-cell current clamp recordings.

RESULTS

The D2/D3 agonist quinpirole (5 µM) enhanced cortically driven medium spiny neuron synaptic responses in slices taken from adult rats treated with vehicle during adolescence, as in untreated adult rats. However, in slices from mature rats treated with olanzapine during adolescence, quinpirole reduced medium spiny neuron activation. The magnitude of decrease was similar to previous observations in untreated, prepubertal rats. These changes may reflect alterations in local inhibitory circuitry, as the GABA-A antagonist picrotoxin (100 µM) reversed the effects of quinpirole in vehicle-treated slices but had no impact on cortically evoked responses in olanzapine-treated slices.

CONCLUSIONS

These data suggest that adolescent atypical antipsychotic drug treatment leads to enduring changes in dopamine modulation of corticostriatal synaptic function.

Assessing anhedonia in depression: Potentials and pitfalls

The resurgence of interest in anhedonia within major depression has been fuelled by clinical trials demonstrating its utility in predicting antidepressant response as well as recent conceptualizations focused on the role and manifestation of anhedonia in depression. Historically, anhedonia has been understood as a "loss of pleasure", yet neuropsychological and neurobiological studies reveal a multifaceted reconceptualization that emphasizes different facets of hedonic function, including desire, effort/motivation, anticipation and consummatory pleasure. To ensure generalizability across studies, evaluation of the available subjective and objective methods to assess anhedonia is necessary. The majority of research regarding anhedonia and its neurobiological underpinnings comes from preclinical research, which uses primary reward (e.g. food) to probe hedonic responding. In contrast, behavioural studies in humans primarily use secondary reward (e.g. money) to measure many aspects of reward responding, including delay discounting, response bias, prediction error, probabilistic reversal learning, effort, anticipation and consummatory pleasure. The development of subjective scales to measure anhedonia has also increased in the last decade. This review will assess the current methodology to measure anhedonia, with a focus on scales and behavioural tasks in humans. Limitations of current work and recommendations for future studies are discussed.

A Short Review on the Current Understanding of Autism Spectrum Disorders

Autism spectrum disorder (ASD) is a set of neurodevelopmental disorders characterized by a deficit in social behaviors and nonverbal interactions such as reduced eye contact, facial expression, and body gestures in the first 3 years of life. It is not a single disorder, and it is broadly considered to be a multi-factorial disorder resulting from genetic and non-genetic risk factors and their interaction. Genetic studies of ASD have identified mutations that interfere with typical neurodevelopment in utero through childhood. These complexes of genes have been involved in synaptogenesis and axon motility. Recent developments in neuroimaging studies have provided many important insights into the pathological changes that occur in the brain of patients with ASD in vivo. Especially, the role of amygdala, a major component of the limbic system and the affective loop of the cortico-striatothalamo-cortical circuit, in cognition and ASD has been proved in numerous neuropathological and neuroimaging studies. Besides the amygdala, the nucleus accumbens is also considered as the key structure which is related with the social reward response in ASD. Although educational and behavioral treatments have been the mainstay of the management of ASD, pharmacological and interventional treatments have also shown some benefit in subjects with ASD. Also, there have been reports about few patients who experienced improvement after deep brain stimulation, one of the interventional treatments. The key architecture of ASD development which could be a target for treatment is still an uncharted territory. Further work is needed to broaden the horizons on the understanding of ASD.

The neural dynamics of reward value and risk coding in the human orbitofrontal cortex

The orbitofrontal cortex is known to carry information regarding expected reward, risk and experienced outcome. Yet, due to inherent limitations in lesion and neuroimaging methods, the neural dynamics of these computations has remained elusive in humans. Here, taking advantage of the high temporal definition of intracranial recordings, we characterize the neurophysiological signatures of the intact orbitofrontal cortex in processing information relevant for risky decisions. Local field potentials were recorded from the intact orbitofrontal cortex of patients suffering from drug-refractory partial epilepsy with implanted depth electrodes as they performed a probabilistic reward learning task that required them to associate visual cues with distinct reward probabilities. We observed three successive signals: (i) around 400 ms after cue presentation, the amplitudes of the local field potentials increased with reward probability; (ii) a risk signal emerged during the late phase of reward anticipation and during the outcome phase; and (iii) an experienced value signal appeared at the time of reward delivery. Both the medial and lateral orbitofrontal cortex encoded risk and reward probability while the lateral orbitofrontal cortex played a dominant role in coding experienced value. The present study provides the first evidence from intracranial recordings that the human orbitofrontal cortex codes reward risk both during late reward anticipation and during the outcome phase at a time scale of milliseconds. Our findings offer insights into the rapid mechanisms underlying the ability to learn structural relationships from the environment.

Parallel Representation of Value-Based and Finite State-Based Strategies in the Ventral and Dorsal Striatum

Previous theoretical studies of animal and human behavioral learning have focused on the dichotomy of the value-based strategy using action value functions to predict rewards and the model-based strategy using internal models to predict environmental states. However, animals and humans often take simple procedural behaviors, such as the “win-stay, lose-switch” strategy without explicit prediction of rewards or states. Here we consider another strategy, the finite state-based strategy, in which a subject selects an action depending on its discrete internal state and updates the state depending on the action chosen and the reward outcome. By analyzing choice behavior of rats in a free-choice task, we found that the finite state-based strategy fitted their behavioral choices more accurately than value-based and model-based strategies did. When fitted models were run autonomously with the same task, only the finite state-based strategy could reproduce the key feature of choice sequences. Analyses of neural activity recorded from the dorsolateral striatum (DLS), the dorsomedial striatum (DMS), and the ventral striatum (VS) identified significant fractions of neurons in all three subareas for which activities were correlated with individual states of the finite state-based strategy. The signal of internal states at the time of choice was found in DMS, and for clusters of states was found in VS. In addition, action values and state values of the value-based strategy were encoded in DMS and VS, respectively. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.

The neural mechanism of decision-making, a cognitive process to select one action among multiple possibilities, is a fundamental issue in neuroscience. Previous studies have revealed the roles of the cerebral cortex and the basal ganglia in decision-making, by assuming that subjects take a value-based reinforcement learning strategy, in which the expected reward for each action candidate is updated. However, animals and humans often use simple procedural strategies, such as “win-stay, lose-switch.” In this study, we consider a finite state-based strategy, in which a subject acts depending on its discrete internal state and updates the state based on reward feedback. We found that the finite state-based strategy could reproduce the choice behavior of rats in a binary choice task with higher accuracy than the value-based strategy. Interestingly, neuronal activity in the striatum, a crucial brain region for reward-based learning, encoded information regarding both strategies. These results suggest that both the value-based strategy and the finite state-based strategy are implemented in the striatum.

Dynamic risk control by human nucleus accumbens

The nucleus accumbens is a key node in the network linking reward to action. Studying a rare series of patients with bilaterally implanted electrodes in the nucleus accumbens, Nachev et al. show that external electrical stimulation of the accumbens dynamically shifts behaviour towards more risky decision making.

Real-world decisions about reward often involve a complex counterbalance of risk and value. Although the nucleus accumbens has been implicated in the underlying neural substrate, its criticality to human behaviour remains an open question, best addressed with interventional methodology that probes the behavioural consequences of focal neural modulation. Combining a psychometric index of risky decision-making with transient electrical modulation of the nucleus accumbens, here we reveal profound, highly dynamic alteration of the relation between probability of reward and choice during therapeutic deep brain stimulation in four patients with treatment-resistant psychiatric disease. Short-lived phasic electrical stimulation of the region of the nucleus accumbens dynamically altered risk behaviour, transiently shifting the psychometric function towards more risky decisions only for the duration of stimulation. A critical, on-line role of human nucleus accumbens in dynamic risk control is thereby established.

Directed Communication between Nucleus Accumbens and Neocortex in Humans Is Differentially Supported by Synchronization in the Theta and Alpha Band

Here, we report evidence for oscillatory bi-directional interactions between the nucleus accumbens and the neocortex in humans. Six patients performed a demanding covert visual attention task while we simultaneously recorded brain activity from deep-brain electrodes implanted in the nucleus accumbens and the surface electroencephalogram (EEG). Both theta and alpha oscillations were strongly coherent with the frontal and parietal EEG during the task. Theta-band coherence increased during processing of the visual stimuli. Granger causality analysis revealed that the nucleus accumbens was communicating with the neocortex primarily in the theta-band, while the cortex was communicating the nucleus accumbens in the alpha-band. These data are consistent with a model, in which theta- and alpha-band oscillations serve dissociable roles: Prior to stimulus processing, the cortex might suppress ongoing processing in the nucleus accumbens by modulating alpha-band activity. Subsequently, upon stimulus presentation, theta oscillations might facilitate the active exchange of stimulus information from the nucleus accumbens to the cortex.

Multimodal approaches to define network oscillations in depression

The renaissance in the use of encephalography-based research methods to probe the pathophysiology of neuropsychiatric disorders is well afoot and continues to advance. Building on the platform of neuroimaging evidence on brain circuit models, magnetoencephalography, scalp electroencephalography, and even invasive electroencephalography are now being used to characterize brain network dysfunctions that underlie major depressive disorder using brain oscillation measurements and associated treatment responses. Such multiple encephalography modalities provide avenues to study pathologic network dynamics with high temporal resolution and over long time courses, opportunities to complement neuroimaging methods and findings, and new approaches to identify quantitative biomarkers that indicate critical targets for brain therapy. Such goals have been facilitated by the ongoing testing of novel invasive neuromodulation therapies, notably, deep brain stimulation, where clinically relevant treatment effects can be monitored at multiple brain sites in a time-locked causal manner. We review key brain rhythms identified in major depressive disorder as foundation for development of putative biomarkers for objectively evaluating neuromodulation success and for guiding deep brain stimulation or other target-based neuromodulation strategies for treatment-resistant depression patients.

No unified reward prediction error in local field potentials from the human nucleus accumbens: evidence from epilepsy patients

Functional magnetic resonance imaging (fMRI), cyclic voltammetry, and single-unit electrophysiology studies suggest that signals measured in the nucleus accumbens (Nacc) during value-based decision making represent reward prediction errors (RPEs), the difference between actual and predicted rewards. Here, we studied the precise temporal and spectral pattern of reward-related signals in the human Nacc. We recorded local field potentials (LFPs) from the Nacc of six epilepsy patients during an economic decision-making task. On each trial, patients decided whether to accept or reject a gamble with equal probabilities of a monetary gain or loss. The behavior of four patients was consistent with choices being guided by value expectations. Expected value signals before outcome onset were observed in three of those patients, at varying latencies and with nonoverlapping spectral patterns. Signals after outcome onset were correlated with RPE regressors in all subjects. However, further analysis revealed that these signals were better explained as outcome valence rather than RPE signals, with gamble gains and losses differing in the power of beta oscillations and in evoked response amplitudes. Taken together, our results do not support the idea that postsynaptic potentials in the Nacc represent a RPE that unifies outcome magnitude and prior value expectation. We discuss the generalizability of our findings to healthy individuals and the relation of our results to measurements of RPE signals obtained from the Nacc with other methods.

A general role for medial prefrontal cortex in event prediction

A recent computational neural model of medial prefrontal cortex (mPFC), namely the predicted response-outcome (PRO) model (Alexander and Brown, 2011), suggests that mPFC learns to predict the outcomes of actions. The model accounted for a wide range of data on the mPFC. Nevertheless, numerous recent findings suggest that mPFC may signal predictions and prediction errors even when the predicted outcomes are not contingent on prior actions. Here we show that the existing PRO model can learn to predict outcomes in a general sense, and not only when the outcomes are contingent on actions. A series of simulations show how this generalized PRO model can account for an even broader range of findings in the mPFC, including human ERP, fMRI, and macaque single-unit data. The results suggest that the mPFC learns to predict salient events in general and provides a theoretical framework that links mPFC function to model-based reinforcement learning, Bayesian learning, and theories of cognitive control.

A universal role of the ventral striatum in reward-based learning: evidence from human studies

Reinforcement learning enables organisms to adjust their behavior in order to maximize rewards. Electrophysiological recordings of dopaminergic midbrain neurons have shown that they code the difference between actual and predicted rewards, i.e., the reward prediction error, in many species. This error signal is conveyed to both the striatum and cortical areas and is thought to play a central role in learning to optimize behavior. However, in human daily life rewards are diverse and often only indirect feedback is available. Here we explore the range of rewards that are processed by the dopaminergic system in human participants, and examine whether it is also involved in learning in the absence of explicit rewards. While results from electrophysiological recordings in humans are sparse, evidence linking dopaminergic activity to the metabolic signal recorded from the midbrain and striatum with functional magnetic resonance imaging (fMRI) is available. Results from fMRI studies suggest that the human ventral striatum (VS) receives valuation information for a diverse set of rewarding stimuli. These range from simple primary reinforcers such as juice rewards over abstract social rewards to internally generated signals on perceived correctness, suggesting that the VS is involved in learning from trial-and-error irrespective of the specific nature of provided rewards. In addition, we summarize evidence that the VS can also be implicated when learning from observing others, and in tasks that go beyond simple stimulus-action-outcome learning, indicating that the reward system is also recruited in more complex learning tasks.

Simultaneous EEG and fMRI reveals a causally connected subcortical-cortical network during reward anticipation

Electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been used to study the neural correlates of reward anticipation, but the interrelation of EEG and fMRI measures remains unknown. The goal of the present study was to investigate this relationship in response to a well established reward anticipation paradigm using simultaneous EEG-fMRI recording in healthy human subjects. Analysis of causal interactions between the thalamus (THAL), ventral-striatum (VS), and supplementary motor area (SMA), using both mediator analysis and dynamic causal modeling, revealed that (1) THAL fMRI blood oxygenation level-dependent (BOLD) activity is mediating intermodal correlations between the EEG contingent negative variation (CNV) signal and the fMRI BOLD signal in SMA and VS, (2) the underlying causal connectivity network consists of top-down regulation from SMA to VS and SMA to THAL along with an excitatory information flow through a THAL→VS→SMA route during reward anticipation, and (3) the EEG CNV signal is best predicted by a combination of THAL fMRI BOLD response and strength of top-down regulation from SMA to VS and SMA to THAL. Collectively, these findings represent a likely neurobiological mechanism mapping a primarily subcortical process, i.e., reward anticipation, onto a cortical signature.

Neuromodulation for treatment resistant depression: state of the art and recommendations for clinical and scientific conduct

Research of Deep Brain Stimulation as a putative treatment for resistant psychiatric disorders might very well lead to the most significant development in clinical psychiatry of the last 40 years-possibly offering a rise of hope for patients to whom medicine had hitherto little to offer. Furthermore, translational research on neuromodulation will allow us to glean something about the underlying cause of patient's illnesses before figuring out a treatment that addresses the source of the problem. Major depression offers perhaps the best example of the rapid progress being made in understanding the biology of mental illness. Studies on the underlying neurobiology of major depression have typically focused on the description of biological differences between patients and healthy subjects such as alterations of monoaminergic or endocrine systems. Psychotropic drugs work by altering neurochemistry to a large extent in widespread regions of the brain, many of which may be unrelated to depression. We believe that more focused, targeted treatment approaches that modulate specific networks in the brain will prove a more effective approach to help treatment-resistant patients. In other words, whereas existing depression treatments approach this disease as a general brain dysfunction, a more complete and appropriate treatment will arise from thinking of depression as a dysfunction of specific brain networks that mediate mood and reward signals (Berton and Nestler, Nat Rev Neurosci 7 (2):137-151, 2006; Krishnan and Nestler, Nature 455(7215):894-902, 2008). A better understanding of defined dysfunctions in these networks will invariably lead to a better understanding of patients afflicted with depression and perhaps contribute to a de-stigmatization of psychiatric patients and the medical specialty treating them.

The stability of NR2B in the nucleus accumbens controls behavioral and synaptic adaptations to chronic stress

BACKGROUND

The nucleus accumbens (NAc) is closely correlated with depression. It has been demonstrated that the glutamatergic system in NAc plays an important role in the reward pathway, dysfunction of which would cause anhedonia, a core symptom of depression. We therefore tested whether N-methyl-D-aspartate receptors and the synaptic plasticity in the NAc are regulated by chronic stress and the relevance to depression.

METHODS

We applied behavioral tests (n = 12, each group) of social interaction and sucrose preference tests to identify the susceptibility of mice to chronic social defeat stress. We then tested N-methyl-D-aspartate receptor-long-term depression at cortico-accumbal synapse to determine the relationship between the susceptibility and changes in synaptic plasticity (n = 8, each group). We further investigated whether restoration of these changes could produce antidepressant effects (n = 10).

RESULTS

We found that chronic stress induced selective downregulation of N-methyl-D-aspartate receptor NR2B subunits in the confined surface membrane pool of NAc neurons. Remarkably, the loss of synaptic NR2B was a long-lived event and further translated to the significant modulation of synaptic plasticity in the form of long-term depression. We further observed that the stress-induced changes were restored by fluoxetine and that resilient mice-those resistant to chronic stress-showed patterns of molecular regulation in the NAc that overlapped dramatically with those seen with fluoxetine treatment. Behaviorally, restoration of NR2B loss prevented the behavioral sensitization of mice to chronic stress.

CONCLUSIONS

Our results identify NR2B in the NAc as a key regulator in the modulation of persistent psychomotor plasticity in response to chronic stress.

The amygdalostriatal and corticostriatal effective connectivity in anticipation and evaluation of facial attractiveness

Decision-making consists of several stages of information processing, including an anticipation stage and an outcome evaluation stage. Previous studies showed that the ventral striatum (VS) is pivotal to both stages, bridging motivation and action, and it works in concert with the ventral medial prefrontal cortex (vmPFC) and the amygdala. However, evidence concerning how the VS works together with the vmPFC and the amygdala came mainly from neuropathology and animal studies; little is known about the dynamics of this network in the functioning human brain. Here we used fMRI combined with dynamic causal modeling (DCM) to investigate the information flow along amygdalostriatal and corticostriatal pathways in a facial attractiveness guessing task. Specifically, we asked participants to guess whether a blurred photo of female face was attractive and to wait for a few seconds ("anticipation stage") until an unblurred photo of feedback face, which was either attractive or unattractive, was presented ("outcome evaluation stage"). At the anticipation stage, the bilateral amygdala and VS showed higher activation for the "attractive" than for the "unattractive" guess. At the outcome evaluation stage, the vmPFC and the bilateral VS were more activated by feedback faces whose attractiveness was congruent with the initial guess than by incongruent faces; however, this effect was only significant for attractive faces, not for unattractive ones. DCM showed that at the anticipation stage, the choice-related information entered the amygdalostriatal pathway through the amygdala and was projected to the VS. At the evaluation stage, the outcome-related information entered the corticostriatal pathway through the vmPFC. Bidirectional connectivities existed between the vmPFC and VS, with the VS-to-vmPFC connectivity weakened by unattractive faces. These findings advanced our understanding of the reward circuitry by demonstrating the pattern of information flow along the amygdalostriatal and corticostriatal pathways at different stages of decision-making.

Closing the gate in the limbic striatum: prefrontal suppression of hippocampal and thalamic inputs

Many brain circuits control behavior by integrating information arising from separate inputs onto a common target neuron. Neurons in the ventral striatum (VS) receive converging excitatory afferents from the prefrontal cortex (PFC), hippocampus (HP), and thalamus, among other structures, and the integration of these inputs is critical for goal-directed behaviors. Although HP inputs have been described as gating PFC throughput in the VS, recent data reveal that the VS desynchronizes from the HP during epochs of burst-like PFC activity related to decision making. It is therefore possible that PFC inputs locally attenuate responses to other glutamatergic inputs to the VS. Here, we found that delivering trains of stimuli to the PFC suppresses HP- and thalamus-evoked synaptic responses in the VS, in part through activation of inhibitory processes. This interaction may enable the PFC to exert influence on basal ganglia loops during decision-making instances with minimal disturbance from ongoing contextual inputs.

Deficient modulation of pain by a positive emotional context in fibromyalgia patients

This study aimed to investigate the modulating effects of emotional context on pain perception in 16 patients with fibromyalgia syndrome (FMS) and 16 healthy control (HC) subjects. An infrared laser was used to apply individually adapted painful stimuli to the dorsum of the left hand. The emotional background of the painful stimuli was modulated by concurrent presentations of negative, neutral, and positive picture stimuli selected from the International Affective Picture System. As control conditions, painful stimuli and the pictures were also presented by themselves. During each of the 5 laser-picture trials, subjects received 10 painful stimuli and were asked to rate the average intensity and unpleasantness of the experienced pain. Functional magnetic resonance images were obtained, using a T2(∗) sensitive echo planar sequence. HC subjects showed a linear increase in pain intensity and unpleasantness ratings when painful stimuli were presented during positive, neutral, and negative pictures. In contrast, FMS patients showed a quadratic trend for pain intensity ratings indicating a lack of pain reduction by the positive pictures. In addition, the FMS patients showed less activation in secondary somatosensory cortex, insula, orbitofrontal cortex, and anterior cingulate cortex during the positive picture pain trials. Our results suggest that fibromyalgia patients are less efficient in modulating pain by positive affect and may benefit less from appetitive events than healthy control subjects.

Reinforcement learning and Tourette syndrome

In this chapter, we report the first experimental explorations of reinforcement learning in Tourette syndrome, realized by our team in the last few years. This report will be preceded by an introduction aimed to provide the reader with the state of the art of the knowledge concerning the neural bases of reinforcement learning at the moment of these studies and the scientific rationale beyond them. In short, reinforcement learning is learning by trial and error to maximize rewards and minimize punishments. This decision-making and learning process implicates the dopaminergic system projecting to the frontal cortex-basal ganglia circuits. A large body of evidence suggests that the dysfunction of the same neural systems is implicated in the pathophysiology of Tourette syndrome. Our results show that Tourette condition, as well as the most common pharmacological treatments (dopamine antagonists), affects reinforcement learning performance in these patients. Specifically, the results suggest a deficit in negative reinforcement learning, possibly underpinned by a functional hyperdopaminergia, which could explain the persistence of tics, despite their evident inadaptive (negative) value. This idea, together with the implications of these results in Tourette therapy and the future perspectives, is discussed in Section 4 of this chapter.

Deep brain stimulation of nucleus accumbens region in alcoholism affects reward processing

The influence of bilateral deep brain stimulation (DBS) of the nucleus nucleus (NAcc) on the processing of reward in a gambling paradigm was investigated using H₂[¹⁵O]-PET (positron emission tomography) in a 38-year-old man treated for severe alcohol addiction. Behavioral data analysis revealed a less risky, more careful choice behavior under active DBS compared to DBS switched off. PET showed win- and loss-related activations in the paracingulate cortex, temporal poles, precuneus and hippocampus under active DBS, brain areas that have been implicated in action monitoring and behavioral control. Except for the temporal pole these activations were not seen when DBS was deactivated. These findings suggest that DBS of the NAcc may act partially by improving behavioral control.

Disclosing information about the self is intrinsically rewarding

Humans devote 30-40% of speech output solely to informing others of their own subjective experiences. What drives this propensity for disclosure? Here, we test recent theories that individuals place high subjective value on opportunities to communicate their thoughts and feelings to others and that doing so engages neural and cognitive mechanisms associated with reward. Five studies provided support for this hypothesis. Self-disclosure was strongly associated with increased activation in brain regions that form the mesolimbic dopamine system, including the nucleus accumbens and ventral tegmental area. Moreover, individuals were willing to forgo money to disclose about the self. Two additional studies demonstrated that these effects stemmed from the independent value that individuals placed on self-referential thought and on simply sharing information with others. Together, these findings suggest that the human tendency to convey information about personal experience may arise from the intrinsic value associated with self-disclosure.

Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: response to commentary

We recently demonstrated that the feedback negativity may be better understood as a reward-related positivity that is absent on nonreward trials, and source localization revealed that this reward response may reflect activity in the striatum. In a commentary on our report, Cohen et al. argue against this latter finding, claiming it is unlikely that the striatum contributes to the scalp-recorded event-related potential. We disagree with the line of reasoning presented by Cohen et al., and we respond here to each of their points. Based on all the available evidence, we argue that the striatum is a plausible generator of a reward-related response observed at the scalp, and this possibility warrants further investigation.

Contextual novelty modulates the neural dynamics of reward anticipation

We investigated how rapidly the reward-predicting properties of visual cues are signaled in the human brain and the extent these reward prediction signals are contextually modifiable. In a magnetoencephalography study, we presented participants with fractal visual cues that predicted monetary rewards with different probabilities. These cues were presented in the temporal context of a preceding novel or familiar image of a natural scene. Starting at ∼100 ms after cue onset, reward probability was signaled in the event-related fields (ERFs) over temporo-occipital sensors and in the power of theta (5-8 Hz) and beta (20-30 Hz) band oscillations over frontal sensors. While theta decreased with reward probability beta power showed the opposite effect. Thus, in humans anticipatory reward responses are generated rapidly, within 100 ms after the onset of reward-predicting cues, which is similar to the timing established in non-human primates. Contextual novelty enhanced the reward anticipation responses in both ERFs and in beta oscillations starting at ∼100 ms after cue onset. This very early context effect is compatible with a physiological model that invokes the mediation of a hippocampal-VTA loop according to which novelty modulates neural response properties within the reward circuitry. We conclude that the neural processing of cues that predict future rewards is temporally highly efficient and contextually modifiable.

Event-related potential activity in the basal ganglia differentiates rewards from nonrewards: temporospatial principal components analysis and source localization of the feedback negativity: commentary

Foti et al. propose that a reward-related brain potential component recorded from scalp EEG is generated by deep brain basal ganglia structures. Previous work, cited in their original article, provides only speculative and theoretical support for this interpretation. Based on empirical and anatomical evidence, we argue that this scalp-recorded ERP component is highly unlikely to be generated by the basal ganglia.

Modulating affect, cognition, and behavior - prospects of deep brain stimulation for treatment-resistant psychiatric disorders

Most patients suffering from psychiatric disorders respond to combinations of psycho- and psychopharmacotherapy; however there are patients who profit little if anything even after many years of treatment. Since about a decade different modalities of targeted neuromodulation – among them most prominently – deep brain stimulation (DBS) – are being actively researched as putative approaches to very treatment-resistant forms of those disorders. Recently, promising pilot data have been reported both for major depression (MD) and obsessive–compulsive disorder (OCD). Given the fact that patients included in DBS studies had been treated unsuccessfully for many years with conventional treatment methods, renders these findings remarkable. Remarkable is the fact, that in case of the long-term studies underway for MD, patients show a stable response. This gives hope to a substantial percentage of therapy–resistant psychiatric patients requiring new therapy approaches. There are no fundamental ethic objections to its use in psychiatric disorders, but until substantial clinical data is available, mandatory standards are needed. DBS is a unique and very promising method for the treatment of therapy–resistant psychiatric patients. The method allows manipulating pathological neuronal networks in a very precise way.

Human hypothalamus shows differential responses to basic motivational stimuli--an invasive electrophysiology study

The hypothalamus supports basic motivational behaviours such as mating and feeding. Recording directly from the posterior inferior hypothalamus in a male patient receiving a deep brain stimulation (DBS) electrode for the alleviation of cluster headache, we tested the hypothalamic response to different classes of motivational stimuli (sexually relevant: pictures of dressed and undressed women; pictures of food) and pictures of common objects as control. Averaged local field potentials (LFP) to sexually relevant stimuli were characterized by a biphasic significantly enhanced response (relative to objects; bootstrapping statistics) with a first phase starting at around 200 ms and a second phase peaking at around 600 ms. Sexually relevant stimuli also showed a greatly enhanced positivity relative to other stimulus classes in surface event-related potentials in a group of 11 male control participants. It is suggested that the hypothalamus is involved in the recruitment of attentional resources by sexually relevant stimuli reflected in this surface positivity. In a second session, the response to food stimuli relative to objects was tested in two states: after fasting for 14 h, LFPs to food and object stimuli showed significant differences in between 300 and 850 ms, which disappeared after a full high-calorie meal, thus replicating classic studies in monkeys [Rolls et al., Brain Res (1976) 111:53-66]. The current data are the first to demonstrate hypothalamic responses to the sight of motivational stimuli in man and thus shows that recording from DBS electrodes might provide important information about the cognitive functions of subcortical structures.

Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit

Accumulating evidence shows that the neural network of the cerebral cortex and the basal ganglia is critically involved in reinforcement learning. Recent studies found functional heterogeneity within the cortico-basal ganglia circuit, especially in its ventromedial to dorsolateral axis. Here we review computational issues in reinforcement learning and propose a working hypothesis on how multiple reinforcement learning algorithms are implemented in the cortico-basal ganglia circuit using different representations of states, values, and actions.

Error-related medial frontal theta activity predicts cingulate-related structural connectivity

Studies on electrophysiological signatures of error processing have focused on the medial frontal cortex, although widespread neuroanatomical networks support error/action monitoring. Here, electrophysiological responses to errors were combined with structural white matter diffusion tensor imaging (DTI) to investigate the long-range anatomical networks that support error processing. The approach taken here was to link individual differences in error-related EEG responses to individual differences in white matter connectional anatomy. Twenty subjects performed a speeded instructed choice task (a variant of the Simon task) designed to elicit response errors, and also underwent DTI scanning in a separate session. In the EEG data, significantly enhanced theta (4-8 Hz) oscillations were observed over medial frontal electrodes (centered on FCz) during response errors. Mid-frontal scalp sites (likely reflecting medial frontal cortex activity) also functioned as a strong "hub" for information flow, measured through theta-band phase synchronization degree. Next, a dipole source of the error-related theta-band activity was localized for each subject, accounting for approximately 80% of the topographical variance. Correlating individual differences in medial frontal theta dynamics with white matter tracts linking these dipole sources to the rest of the brain revealed that subjects with stronger error-related theta also had stronger white matter connectivity with the ventral striatum and inferior frontal gyrus. Further, subjects in whom medial frontal regions acted as a stronger synchronization "hub" had stronger connectivity between the dipole source location and the corpus callosum and dorsomedial prefrontal white matter pathways. These findings provide novel evidence for the role of widespread fronto-striatal networks in monitoring actions and signaling behavioral errors.

Cross-species affective functions of the medial forebrain bundle-implications for the treatment of affective pain and depression in humans

Major depression (MD) might be conceptualized as pathological under-arousal of positive affective systems as parts of a network of brain regions assessing, reconciling and storing emotional stimuli versus an over-arousal of parts of the same network promoting separation-distress/GRIEF. In this context depression can be explained as an emotional pain state that is the result of a disregulation of several sub-systems that under physiological conditions are concerned with bodily or emotional homeostasis of the human organism in a social context. Physiologically, homeostasis is maintained by influences of the SEEKING system represented - amongst others - by the medial forebrain bundle (MFB). Neuroimaging studies show that the MFB has a proven access to the GRIEF/Sadness system. A functional decoupling of these systems with a dysfunctional GRIEF pathway might result in MD. Therewith GRIEF and SEEKING/PLEASURE systems play important roles as opponents in maintenance of emotional homeostasis. Chronic electrical modulation of the reward SEEKING pathways with deep brain stimulation might show anti-depressive effects in humans suffering from MD by re-initiating an emotional equilibrium (of higher or lower activity) between these opposing systems.