Electrophysiological signatures of reward learning in the rodent touchscreen-based Probabilistic Reward Task

Cortico-striatal beta oscillations as a reward-related signal

The value associated with reward is sensitive to external factors, such as the time between the choice and reward delivery as classically manipulated in temporal discounting tasks. Subjective preference for two reward options is dependent on objective variables of reward magnitude and reward delay. Single neuron correlates of reward value have been observed in regions, including ventral striatum, orbital, and medial prefrontal cortex. Brain imaging studies show cortico-striatal-limbic network activity related to subjective preferences. To explore how oscillatory dynamics represent reward processing across brain regions, we measured local field potentials of rats performing a temporal discounting task. Our goal was to use a data-driven approach to identify an electrophysiological marker that correlates with reward preference. We found that reward-locked oscillations at beta frequencies signaled the magnitude of reward and decayed with longer temporal delays. Electrodes in orbitofrontal/medial prefrontal cortex, anterior insula, ventral striatum, and amygdala individually increased power and were functionally connected at beta frequencies during reward outcome. Beta power during reward outcome correlated with subjective value as defined by a computational model fit to the discounting behavior. These data suggest that cortico-striatal beta oscillations are a reward signal correlated, which may represent subjective value and hold potential to serve as a biomarker and potential therapeutic target.

Beta activity in human anterior cingulate cortex mediates reward biases

The rewards that we get from our choices and actions can have a major influence on our future behavior. Understanding how reward biasing of behavior is implemented in the brain is important for many reasons, including the fact that diminution in reward biasing is a hallmark of clinical depression. We hypothesized that reward biasing is mediated by the anterior cingulate cortex (ACC), a cortical hub region associated with the integration of reward and executive control and with the etiology of depression. To test this hypothesis, we recorded neural activity during a biased judgment task in patients undergoing intracranial monitoring for either epilepsy or major depressive disorder. We found that beta (12-30 Hz) oscillations in the ACC predicted both associated reward and the size of the choice bias, and also tracked reward receipt, thereby predicting bias on future trials. We found reduced magnitude of bias in depressed patients, in whom the beta-specific effects were correspondingly reduced. Our findings suggest that ACC beta oscillations may orchestrate the learning of reward information to guide adaptive choice, and, more broadly, suggest a potential biomarker for anhedonia and point to future development of interventions to enhance reward impact for therapeutic benefit.

Reward-Related Brain Activity Mediates the Relationship Between Decision-Making Deficits and Pediatric Depression Symptom Severity

BACKGROUND

The mechanisms that link neural and behavioral indices of reduced reward sensitivity in depression, particularly in children, remain unclear. Reward positivity (RewP), a neural index of reward processing, has been consistently associated with depression. Separately, recent studies using the drift-diffusion model on behavioral data have delineated computational indices of reward sensitivity. Therefore, in the current study, we examined whether RewP is a neural mediator of drift-diffusion model-based indices of reward processing in predicting pediatric depression across varying levels of symptom severity.

METHODS

A community sample of 166 girls, ages 8 to 14 years, completed 2 tasks. The first was a reward guessing task from which RewP was computed using electroencephalography; the second was a probabilistic reward-based decision-making task. On this second task, drift-diffusion model analysis was applied to behavioral data to quantify the efficiency of accumulating reward-related evidence (drift rate) and potential baseline bias (starting point) toward the differently rewarded choices. Depression severity was measured using the self-report Children's Depression Inventory.

RESULTS

RewP was correlated with drift rate, but not starting point bias, toward the more rewarded choice. Furthermore, RewP completely mediated the association between a slower drift rate toward the more rewarded option and higher depression symptom severity.

CONCLUSIONS

Our findings suggest that reduced neural sensitivity to reward feedback may be a neural mechanism that underlies behavioral insensitivity to reward in children and adolescents with higher depression symptom severity, offering novel insights into the relationship between neural and computational indices of reward processing in this context.

Human Substantia Nigra Neurons Encode Reward Expectations

Dopamine (DA) signals originating from substantia nigra (SN) neurons are centrally involved in the regulation of motor and reward processing. DA signals behaviorally relevant events where reward outcomes differ from expectations (reward prediction errors, RPEs). RPEs play a crucial role in learning optimal courses of action and in determining response vigor when an agent expects rewards. Nevertheless, how reward expectations, crucial for RPE calculations, are conveyed to and represented in the dopaminergic system is not fully understood, especially in the human brain where the activity of DA neurons is difficult to study. One possibility, suggested by evidence from animal models, is that DA neurons explicitly encode reward expectations. Alternatively, they may receive RPE information directly from upstream brain regions. To address whether SN neuron activity directly reflects reward expectation information, we directly examined the encoding of reward expectation signals in human putative DA neurons by performing single-unit recordings from the SN of patients undergoing neurosurgery. Patients played a two-armed bandit decision-making task in which they attempted to maximize reward. We show that neuronal firing rates (FR) of putative DA neurons during the reward expectation period explicitly encode reward expectations. First, activity in these neurons was modulated by previous trial outcomes, such that FR were greater after positive outcomes than after neutral or negative outcome trials. Second, this increase in FR was associated with shorter reaction times, consistent with an invigorating effect of DA neuron activity during expectation. These results suggest that human DA neurons explicitly encode reward expectations, providing a neurophysiological substrate for a signal critical for reward learning.

Using Drift Diffusion and RL Models to Disentangle Effects of Depression On Decision-Making vs. Learning in the Probabilistic Reward Task

The Probabilistic Reward Task (PRT) is widely used to investigate the impact of Major Depressive Disorder (MDD) on reinforcement learning (RL), and recent studies have used it to provide insight into decision-making mechanisms affected by MDD. The current project used PRT data from unmedicated, treatment-seeking adults with MDD to extend these efforts by: (1) providing a more detailed analysis of standard PRT metrics-response bias and discriminability-to better understand how the task is performed; (2) analyzing the data with two computational models and providing psychometric analyses of both; and (3) determining whether response bias, discriminability, or model parameters predicted responses to treatment with placebo or the atypical antidepressant bupropion. Analysis of standard metrics replicated recent work by demonstrating a dependency between response bias and response time (RT), and by showing that reward totals in the PRT are governed by discriminability. Behavior was well-captured by the Hierarchical Drift Diffusion Model (HDDM), which models decision-making processes; the HDDM showed excellent internal consistency and acceptable retest reliability. A separate "belief" model reproduced the evolution of response bias over time better than the HDDM, but its psychometric properties were weaker. Finally, the predictive utility of the PRT was limited by small samples; nevertheless, depressed adults who responded to bupropion showed larger pre-treatment starting point biases in the HDDM than non-responders, indicating greater sensitivity to the PRT's asymmetric reinforcement contingencies. Together, these findings enhance our understanding of reward and decision-making mechanisms that are implicated in MDD and probed by the PRT.

Hypoactivation of ventromedial frontal cortex in major depressive disorder: an MEG study of the Reward Positivity

Background

The Reward Positivity (RewP) is sensitive and specific electrophysiological marker of reward receipt. These characteristics make it a compelling candidate marker of dysfunctional reward processing in major depressive disorder. We previously proposed that the RewP is a nexus of multiple aspects of reward variance, and that a diminished RewP in depression might only reflect a deficit in some of this variance. Specifically, we predicted a diminished ventromedial contribution in depression in the context of maintained reward learning.

Methods

Here we collected magnetoencephalographic (MEG) recordings of reward receipt in 43 individuals with major depressive disorder (MDD group) and 38 healthy controls (CTL group). MEG allows effective source estimation due to the absence of volume conduction that compromises electroencephalographic recordings.

Results

The MEG RewP analogue was generated by a broad set of cortical areas, yet only right ventromedial and right ventral temporal areas were diminished in MDD. These areas correlated with a principal component of anhedonia derived from multiple questionnaires. Compellingly, BA25 was the frontal region with the largest representation in both of these effects.

Conclusions

These findings not only advance our understanding underlying the computation of the RewP, but they also dovetail with convergent findings from other types of functional source imaging in depression, as well as from deep brain stimulation treatments. Together, these discoveries suggest that the RewP may be a valuable marker for objective assessment of reward affect and its disruption in major depression.

Rodent tests of depression and anxiety: Construct validity and translational relevance

Behavioral testing constitutes the primary method to measure the emotional states of nonhuman animals in preclinical research. Emerging as the characteristic tool of the behaviorist school of psychology, behavioral testing of animals, particularly rodents, is employed to understand the complex cognitive and affective symptoms of neuropsychiatric disorders. Following the symptom-based diagnosis model of the DSM, rodent models and tests of depression and anxiety focus on behavioral patterns that resemble the superficial symptoms of these disorders. While these practices provided researchers with a platform to screen novel antidepressant and anxiolytic drug candidates, their construct validity-involving relevant underlying mechanisms-has been questioned. In this review, we present the laboratory procedures used to assess depressive- and anxiety-like behaviors in rats and mice. These include constructs that rely on stress-triggered responses, such as behavioral despair, and those that emerge with nonaversive training, such as cognitive bias. We describe the specific behavioral tests that are used to assess these constructs and discuss the criticisms on their theoretical background. We review specific concerns about the construct validity and translational relevance of individual behavioral tests, outline the limitations of the traditional, symptom-based interpretation, and introduce novel, ethologically relevant frameworks that emphasize simple behavioral patterns. Finally, we explore behavioral monitoring and morphological analysis methods that can be integrated into behavioral testing and discuss how they can enhance the construct validity of these tests.

Validation of a touchscreen probabilistic reward task for mice: A reverse-translated assay with cross-species continuity

The Probabilistic Reward Task (PRT) is a laboratory-based technique used to objectively quantify responsivity to reward. The PRT was initially designed to identify reinforcement learning deficits in clinical populations and subsequently was reverse-translated for use in preclinical studies with rats and monkeys. In this task, subjects make visual discriminations and asymmetric probabilistic contingencies are arranged such that correct responses to one stimulus (rich) are reinforced more often than correct responses to the other (lean). Numerous studies have demonstrated that healthy subjects reliably develop a response bias toward the richly rewarded stimulus, whereas humans with anhedonia and laboratory animals with a history of chronic stress exhibit a blunted response bias. This is important because anhedonia, the loss of responsivity to previously rewarding stimuli, is a behavioral phenotype that is a cardinal feature of multiple neuropsychiatric conditions and is without approved pharmacotherapeutic options. To aid in addressing this critical treatment gap, this report describes validation of the first PRT designed for mice, which are a commonly utilized species in preclinical research toward neuropsychiatric medications development. Results reveal orderly psychophysical functions in response to asymmetric probabilistic contingencies in mice, with signal detection outcomes comparable to previous PRT findings in humans, rats, and monkeys. Taken together, such robust cross-species continuity in task performance confirms that the mouse is well-positioned to serve in bidirectional research efforts between human and animal laboratories. These efforts may accelerate the development of treatment options for anhedonia in the different neuropsychiatric conditions in which it is prominent.

Depth recordings of the mouse homologue of the Reward Positivity

We recently advanced a rodent homologue for the reward-specific, event-related potential component observed in humans known as the Reward Positivity. We sought to determine the cortical source of this signal in mice to further test the nature of this homology. While similar reward-related cortical signals have been identified in rats, these recordings were all performed in cingulate gyrus. Given the value-dependent nature of this event, we hypothesized that more ventral prelimbic and infralimbic areas also contribute important variance to this signal. Depth probes assessed local field activity in 29 mice (15 males) while they completed multiple sessions of a probabilistic reinforcement learning task. Using a priori regions of interest, we demonstrated that the depth of recording in the cortical midline significantly correlated with the size of reward-evoked delta band spectral activity as well as the single trial correlation between delta power and reward prediction error. These findings provide important verification of the validity of this translational biomarker of reward responsiveness, learning, and valuation.

Translational In Vivo Assays in Behavioral Biology

The failure of preclinical research to advance successful candidate medications in psychiatry has created a paradigmatic crisis in psychiatry. The Research Domain Criteria (RDoC) initiative was designed to remedy this situation with a neuroscience-based approach that employs multimodal and cross-species in vivo methodology to increase the probability of translational findings and, consequently, drug discovery. The present review underscores the feasibility of this methodological approach by briefly reviewing, first, the use of multidimensional and cross-species methodologies in traditional behavioral pharmacology and, subsequently, the utility of this approach in contemporary neuroimaging and electrophysiology research-with a focus on the value of functionally homologous studies in nonhuman and human subjects. The final section provides a brief review of the RDoC, with a focus on the potential strengths and weaknesses of its domain-based underpinnings. Optimistically, this mechanistic and multidimensional approach in neuropsychiatric research will lead to novel therapeutics for the management of neuropsychiatric disorders.

Distinct profiles of anhedonia and reward processing and their prospective associations with quality of life among individuals with mood disorders

Leading professional health bodies have called for the wider adoption of Patient Reported Outcome Measures, such as quality of life, in research and clinical practice as a means for understanding why the global burden of depression continues to climb despite increased rates of treatment use. Here, we examined whether anhedonia-an often recalcitrant and impairing symptom of depression-along with its neural correlates, was associated with longitudinal changes in patient-reported quality of life among individuals seeking treatment for mood disorders. We recruited 112 participants, including n = 80 individuals with mood disorders (58 unipolar, 22 bipolar) and n = 32 healthy controls (63.4% female). We assessed anhedonia severity along with two electroencephalographic markers of neural reward responsiveness (scalp-level 'Reward Positivity' amplitude and source-localized reward-related activation in the dorsal anterior cingulate cortex), and assessed quality of life at baseline, 3- and 6-month follow-up. Anhedonia emerged as a robust correlate of quality of life cross-sectionally and longitudinally among individuals with mood disorders. Furthermore, increased neural reward responsiveness at baseline was associated with greater improvements in quality of life over time, and this improvement was mediated by longitudinal improvements in anhedonia severity. Finally, differences in quality of life observed between individuals with unipolar and bipolar mood disorders were mediated by differences in anhedonia severity. Our findings indicate that anhedonia and its reward-related neural correlates are linked to variability in quality of life over time in individuals with mood disorders. Treatments capable of improving anhedonia and normalizing brain reward function may be necessary for improving broader health outcomes for individuals seeking treatment for depression.ClinicalTrials.gov identifier: NCT01976975.