Reward expectation alters learning and memory: the impact of the amygdala on appetitive-driven behaviors

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Expectations are often dynamic: sports fans know that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here we present behavioral and electrophysiological evidence of sub-second changes in expectations using slot machines as a case study. In Study 1, we demonstrate that EEG signal before the slot machine stops varies based on proximity to winning. Study 2 introduces a behavioral paradigm to measure dynamic expectations via betting, and shows that expectation trajectories vary as a function of winning proximity. Notably, these expectation trajectories parallel Study 1's EEG activity. Studies 3 (EEG) and 4 (behavioral) replicate these findings in the loss domain. These four studies provide compelling evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our research opens promising avenues for understanding the dynamic nature of reward expectations and their impact on cognitive processes.

The neuroanatomical hallmarks of chronic tinnitus in comorbidity with pure-tone hearing loss

Tinnitus is one of the main hearing impairments often associated with pure-tone hearing loss, and typically manifested in the perception of phantom sounds. Nevertheless, tinnitus has traditionally been studied in isolation without necessarily considering auditory ghosting and hearing loss as part of the same syndrome. Hence, in the present neuroanatomical study, we attempted to pave the way toward a better understanding of the tinnitus syndrome, and compared two groups of almost perfectly matched individuals with (TIHL) and without (NTHL) pure-tone tinnitus, but both characterized by pure-tone hearing loss. The two groups were homogenized in terms of sample size, age, gender, handedness, education, and hearing loss. Furthermore, since the assessment of pure-tone hearing thresholds alone is not sufficient to describe the full spectrum of hearing abilities, the two groups were also harmonized for supra-threshold hearing estimates which were collected using temporal compression, frequency selectivity und speech-in-noise tasks. Regions-of-interest (ROI) analyses based on key brain structures identified in previous neuroimaging studies showed that the TIHL group exhibited increased cortical volume (CV) and surface area (CSA) of the right supramarginal gyrus and posterior planum temporale (PT) as well as CSA of the left middle-anterior part of the superior temporal sulcus (STS). The TIHL group also demonstrated larger volumes of the left amygdala and of the left head and body of the hippocampus. Notably, vertex-wise multiple linear regression analyses additionally brought to light that CSA of a specific cluster, which was located in the left middle-anterior part of the STS and overlapped with the one found to be significant in the between-group analyses, was positively associated with tinnitus distress level. Furthermore, distress also positively correlated with CSA of gray matter vertices in the right dorsal prefrontal cortex and the right posterior STS, whereas tinnitus duration was positively associated with CSA and CV of the right angular gyrus (AG) and posterior part of the STS. These results provide new insights into the critical gray matter architecture of the tinnitus syndrome matrix responsible for the emergence, maintenance and distress of auditory phantom sensations.

Dynamic expectations: Behavioral and electrophysiological evidence of sub-second updates in reward predictions

Expectations are often dynamic: any sports fan knows that expectations are rapidly updated as games unfold. Yet expectations have traditionally been studied as static. Here, using slot machines as a case study, we provide parallel behavioral and electrophysiological evidence of sub-second moment-to-moment changes in expectations. In Study 1, we show that the dynamics of the EEG signal before the slot machine stopped differed depending on the nature of the outcome, including not only whether the participant won or lost, but also how close they came to winning. In line with our predictions, Near Win Before outcomes (the slot machine stops one item before a match) were similar to Wins, but different than Near Win After (the machine stops one item after a match) and Full Miss (the machine stops two or three items from a match). In Study 2, we designed a novel behavioral paradigm to measure moment-to-moment changes in expectations via dynamic betting. We found that different outcomes also elicited unique expectation trajectories in the deceleration phase. Notably, these behavioral expectation trajectories paralleled Study 1's EEG activity in the last second prior to the machine's stop. In Studies 3 (EEG) and 4 (behavior) we replicated these findings in the loss domain where a match entails a loss. Again, we found a significant correlation between behavioral and EEG results. These four studies provide the first evidence that dynamic sub-second updates in expectations can be behaviorally and electrophysiologically measured. Our findings open up new avenues for studying the ongoing dynamics of reward expectations and their role in healthy and unhealthy cognition.

Wolbachia manipulate fitness benefits of olfactory associative learning in a parasitoid wasp

Upon encountering a host, a female parasitoid wasp has to decide whether to learn positive or negative cues related to the host. The optimal female decision will depend on the fitness costs and benefits of learned stimuli. Reward quality is positively related to the rate of behavioral acquisition in processes such as associative learning. Wolbachia, an endosymbiotic bacterium, often plays an impressive role in the manipulation of its arthropod host's biology. Here, we studied the responses of two natural Wolbachia infected/uninfected Trichogramma brassicae wasp populations to theoretically high- and low-reward values during a conditioning process and the consequences of their responses in terms of memory duration. According to our results, uninfected wasps showed an attraction response to high-value rewards, but showed aversive learning in response to low-value rewards. The memory span of uninfected wasps after conditioning by low-value rewards was significantly shorter than that for high-value rewards. As our results revealed, responses to high-quality hosts will bring more benefits (bigger size, increased fecundity and enhanced survival) than those to low-quality hosts for uninfected wasps. Infected wasps were attracted to conditioned stimuli with the same memory duration after conditioning by both types of hosts. This was linked to the fact that parasitoids emerging from both types of hosts present the same life-history traits. Therefore, these hosts represent the same quality reward for infected wasps. According to the obtained results, it can be concluded that Wolbachia manipulates the learning ability of its host, resulting in the wasp responding to all reward values similarly.

Endocannabinoid and dopaminergic system: the pas de deux underlying human motivation and behaviors

Endocannabinoid system (ECS) has been identified ever since cannabinoid, an active substance of Cannabis, was known to interact with endogenous cannabinoid (endocannabinoid/eCB) receptors. It later turned out that eCB was more intricate than previously thought. It has a pervasive role and exerts a multitude of cellular signaling mechanisms, regulating various physiological neurotransmission pathways in the human brain, including the dopaminergic (DA) system. eCB roles toward DA system were robust, clearly delineated, and reproducible with respect to physiological as well as pathological neurochemical and neurobehavioral manifestations of DA system, particularly those involving the nigrostriatal and mesocorticolimbic pathways. The eCB–DA system regulates the basics in the Maslow’s pyramid of hierarchy of needs required for individual survival such as food and sexual activity for reproductive purpose to those of higher needs in the pyramid, including self-actualization behaviors leading to achievement and reward (e.g., academic- and/or work-related performance and achievements). It is, thus, interesting to specifically discuss the eCB–DA system, not only on the molecular level, but also its tremendous potential to be developed as a future therapeutic strategy for various neuropsychiatric problems, including obesity, drug addiction and withdrawal, pathological hypersexuality, or low motivation behaviors.

Implicit outcomes expectancies shape memory process: Electrophysiological evidence

The simple manipulation of pairing specific outcomes with the sample stimuli strongly affects discriminative learning and memory processes. This procedure has been named the Differential Outcomes Procedure (DOP) and is usually compared to a control condition (the non-differential procedure, NOP) consisting in the random administration of the outcomes after each correct response. Recent research has revealed that the DOP effect arises even under unconscious conditions. In this study, we explored the temporal dynamics of short-term memory processes in both the DOP and the NOP in the absence of awareness of either the outcome (Experiment 1A) or the initial sample stimulus (Experiment 1B) through the evoked-related potentials technique. Results showed distinctive electrophysiological activation patterns in the DOP compared with the NOP at encoding, maintenance and retrieval phases. The present findings provide electrophysiological evidence of implicit-prospective processes involved in the DOP. They elucidate the processes that result in improved visual recognition memory.

Electrophysiological correlates of the differential outcomes effect in visual short-term memory

The differential outcomes procedure (DOP) consists in applying a specific outcome after each discriminative stimulus-correct response pairing, leading to improved performance in both memory and learning tasks (faster acquisition and/or higher response accuracy), compared to the non-differential outcomes procedure (NOP). The main aim of this study was to explore the electrophysiological correlates (ERPs) of the DOP in a visual short-term memory task, and to test whether a differential activation pattern would be observed depending on the outcomes condition (DOP vs. NOP). The ERP signals showed differences between both outcomes condition in all three phases of the short-term memory task: encoding, maintenance and retrieval. Our results are in accordance with the view that in the DOP condition the probe stimulus triggers a representation of the unique outcome, which remains active over the maintenance period (prospective process). In the NOP condition, in contrast, a representation of the probe stimulus is maintained (retrospective process). In addition, these results suggested that stimuli associated with unique outcomes captured attention involuntary at retrieval, decreasing the interference from distractor stimuli in the retrieval phase.

Discriminative learning and associative memory under the differential outcomes procedure is modulated by cognitive load

Working memory (WM) has been thought to be the cause of associative memory deficits in older adults. Previous research has demonstrated the benefits of a discriminative learning procedure, the differential outcomes procedure (DOP), to ameliorate such associative-memory maintenance deficits in situations that simulate adherence to medical prescriptions in both healthy and pathological ageing. Specifically, the DOP involves rewarding each correct response to each stimulus-stimulus association with a distinct and unique outcome (reinforcer). The aim of the present study was to explore the limits of this procedure by testing the amount of cognitive load at which the DOP improves discriminative learning and associative memory in a task that simulates adherence to medical treatment in undergraduate students. During the training phase, participants were asked to learn three pill/name (low-load condition) or four pill/name associations (high-load conditions) under the DOP in comparison with a control condition (the non-differential outcomes condition, NOP). Long-term retention of such learned associations was tested 1h and 1week after completion of the training phase. Participants showed a better accuracy and long-term retention of the learned associations when the DOP was used, but just in the high-load condition. These results suggest that when WM is overtaxed, the DOP plays a fundamental role in the long-term maintenance of the learned stimulus-stimulus associations, rendering such learning procedure as a useful technique to enhance people's discriminative learning and associative memory.

Enhanced learning and retention of medical information in Alzheimer's disease after differential outcomes training

BACKGROUND

Adherence to treatment is a crucial factor for patients who have chronic illnesses or multiple morbidities and polypharmacy, which is frequently found in older adults. The non-adherence to medications has important economic and social consequences as well as impacts on the health of the patients. One of the reasons that can explain the low adherence to treatment, is the memory deficits that are characteristics of this population and that are even more evident in cases that involve neurodegenerative diseases.

METHODS AND FINDINGS

In this study, we explore whether the differential outcomes procedure (DOP), which has been shown to be useful in improving discriminative learning and memory in different populations, may facilitate learning and retention of medical recommendations in older adults who have been diagnosed with Alzheimer's disease. The results demonstrate that when this procedure was applied, the patients showed improvements in learning and long-term retention of two pill/time of day associations in a situation that simulates adherence to medical prescriptions.

CONCLUSIONS

These findings contribute new data about the potential benefits of the DOP in patients with neurodegenerative disorders, highlighting the important role that this procedure could play in addressing important issues related to the health and quality of life of older adults, with or without neurodegenerative diseases, such as low adherence to medical treatments.

Lipopolysaccharide Increases Cortical Kynurenic Acid and Deficits in Reference Memory in Mice

Kynurenic acid (KYNA), a glial-derived metabolite of tryptophan metabolism, is an antagonist of the alpha 7 nicotinic acetylcholine receptor and the glycine-binding site of N-methyl-d-aspartate (NMDA) receptors. Kynurenic acid levels are increased in both the brain and cerebrospinal fluid of several psychiatric disorders including bipolar disorder, schizophrenia, and Alzheimer disease. In addition, pro-inflammatory cytokines have been found to be elevated in the blood of schizophrenic patients suggesting inflammation may play a role in psychiatric illness. As both pro-inflammatory cytokines and KYNA can be elevated in the brain by peripheral lipopolysaccharide (LPS) injection, we therefore sought to characterize the role of neuroinflammation on learning and memory using a well-described dual-LPS injection model. Mice were injected with an initial injection (0.25 mg/kg LPS, 0.50 mg/kg, or saline) of LPS and then administrated a second injection 16 hours later. Our results indicate both 0.25 and 0.50 mg/kg dual-LPS treatment increased l-kynurenine and KYNA levels in the medial pre-frontal cortex (mPFC). Mice exhibited impaired acquisition of CS+ (conditioned stimulus) Pavlovian conditioning. Notably, mice showed impairment in reference memory while working memory was normal in an 8-arm maze. Taken together, our findings suggest that neuroinflammation induced by peripheral LPS administration contributes to cognitive dysfunction.

The role of differential outcomes-based feedback on procedural memory

In categorization tasks, two memory systems may be involved in the learning of categories: one explicit and rule-based system and another implicit and procedure-based system. Learning of rule-based categories relies on some form of explicit reasoning, whereas procedural memory underlies information-integration category-learning tasks, in which performance is maximized only if information of two (or more) dimensions is integrated. The present study aimed at investigating the role of how feedback is administered, whether differential or non-differential, in procedural learning. An information-integration category-learning task was designed, where the to-be-categorized stimuli differed in two dimensions. Participants were randomly assigned to two groups: one group received the reinforcers for correct categorizations differentially, one for each category (the differential outcomes procedure, DOP), and the other group received the reinforcers randomly (the non-differential outcomes procedure, NOP). The participants of the DOP group showed better procedural learning in the categorization task, compared to the NOP group. Moreover, the analysis of learning strategies revealed that more participants developed more optimal strategies in the DOP group than in the NOP group. These results extend the benefits of the differential outcomes-based feedback to non-declarative memory tasks and help better understand the role of feedback in procedural learning.

Does the implicit outcomes expectancies shape learning and memory processes?

Does the explicit or implicit knowledge about the consequences of our choices shape learning and memory processes? This seems to be the case according to previous studies demonstrating improvements in learning and retention of symbolic relations and in visuospatial recognition memory when each correct choice is reinforced with its own unique and explicit outcome (the differential outcomes procedure, DOP). In the present study, we aim to extend these findings by exploring the impact of the DOP under conditions of non-conscious processing. To test for this, both the outcomes (Experiment 1A) and the sample stimuli (Experiment 1B) were presented under subliminal (non-conscious) and supraliminal conditions in a delayed visual recognition memory task. Results from both experiments showed a better visual recognition memory when participants were trained with the DOP regardless the awareness of the outcomes or even of the stimuli used for training. To our knowledge, this is the first demonstration that the DOP can be effective under unconscious conditions. This finding is discussed in the light of the two-memory systems model developed by Savage and colleagues to explain the beneficial effects observed on learning and memory when differential outcomes are applied.

Differential Outcomes Training Ameliorates Visual Memory Impairments in Patients With Alzheimer's Disease: A Pilot Study

It is well known that Alzheimer's disease (AD), the most common form of dementia, is associated with deficits in cognitive processes including visual memory impairments. One technique that might be used to ameliorate these impairments is the differential outcomes procedure (DOP) that involves associating each to-be-remembered stimulus with a specific outcome. Objective: Previous research has demonstrated that the DOP can be used to reduce or eliminate the learning and memory deficits associated with animal models of amnesia and dementia. Furthermore, this procedure has been shown to improve delayed facial recognition in healthy older adults as well as in patients diagnosed with AD. The main aim of the present study is twofold: to extend these findings to other types of visual stimulus and to investigate the effect of the DOP in memory retention in AD patients. Method: Ten patients diagnosed with AD and 10 healthy controls participated in this study. The experiment included two phases. In the first one, they had to perform a delayed matching-to-sample task. In the second phase, participants performed a recognition memory task, designed to assess long-term retention, 1 h and 1 week after the training. Results: Participants showed a better memory-based performance as well as a higher long-term retention of the information when trained under the differential outcomes condition, relative to the non-differential outcomes condition. Conclusions: The DOP seems to be an effective, easy-to-implement, technique to enhance visual memory in AD patients.

An Analysis of the Basic Processes of Formation and Change of Placebo Expectations

Placebo effects are the measurable psychological, biological, and behavioral changes that can result from expecting a treatment to be effective. Here we argue that not all expectations are created equally and there is much to learn by clarifying the psychological processes that underlie the expectations that cause placebo effects. It is proposed that the formation and change of placebo expectations can be understood from the standpoint of a general psychological model describing the basic processes of mental change. Specifically, we use the Elaboration Likelihood Model to explain how placebo-relevant variables (e.g., doctor’s lab coat, drug price, number and color of pills, patient’s mood) can work to affect expectations. Clarifying the antecedent processes responsible for placebo expectations leads to new insights regarding placebo effects, including their durability, directionality, and ability to alter subsequent behaviors (e.g., treatment adherence). A key point from our approach is that expectations causing placebo effects can be formed under high or low thought. High-thought expectations should be more likely to resist change, last over time, predict placebo effects better than low-thought expectations, and have a greater likelihood to alter subsequent behaviors. We conclude by describing a variety of theoretical innovations that this new conceptualization raises and suggest novel paths for research and application.

Enhancement of Visuospatial Working Memory by the Differential Outcomes Procedure in Mild Cognitive Impairment and Alzheimer's Disease

In the present study we investigated the efficacy of the differential outcomes procedure (DOP) to improve visuospatial working memory in patients with Alzheimer’s disease and mild cognitive impairment (MCI). The DOP associates correct responses to the to-be-remember stimulus with unique outcomes. Eleven patients diagnosed with Alzheimer’s disease, 11 participants with MCI, and 17 healthy matched controls performed a spatial delayed memory task under the DOP and a control condition (non-differential outcomes –NOP-). We found that performance (terminal accuracy) was significantly better in the DOP condition relative to the NOP condition in all three groups of participants. AD patients performed worse, and took longer to benefit from the DOP. In line with previous animal and human research, we propose that the DOP activates brain structures and cognitive mechanisms that are less affected by healthy and pathological aging, optimizing in this way the function of the cognitive system.

Learning and Recall of Medical Treatment-Related Information in Older Adults Using the Differential Outcomes Procedure

It has recently been reported that the differential outcomes procedure (DOP) might be one of the therapeutical techniques focused at promoting autonomy in the elderly to deal with their medical issues. Molina et al. (2015) found that a group of healthy young adults improved their learning and long-term retention of six disorder/pill associations when each relationship to be learned was associated with a particular reinforcer (the differential outcomes condition) compared to when they were randomly administered (the non-differential outcomes condition). In the present study, we extend these findings to older adults who usually show difficulties to remember to take their medications as prescribed. Participants were asked to learn the association between three pills and the specific time at the day when they had to take each medication. Two memory tests were also performed 1 h and 1 week after completing the training phase. Results showed a faster learning of the task and long-term retention of the previously learned associations (pill/time of day) when differential outcomes were used. Furthermore, the older adults’ performance in the learning and memory phases did not differ from that of the younger adults in the DOP condition. These findings demonstrate that this procedure can help elderly people to ameliorate not only their learning, but also their long-term memory difficulties, suggesting the potential for the DOP to promote adherence to treatment in this population.

Proprioceptive Feedback Facilitates Motor Imagery-Related Operant Learning of Sensorimotor β-Band Modulation

Motor imagery (MI) activates the sensorimotor system independent of actual movements and might be facilitated by neurofeedback. Knowledge on the interaction between feedback modality and the involved frequency bands during MI-related brain self-regulation is still scarce. Previous studies compared the cortical activity during the MI task with concurrent feedback (MI with feedback condition) to cortical activity during the relaxation task where no feedback was provided (relaxation without feedback condition). The observed differences might, therefore, be related to either the task or the feedback. A proper comparison would necessitate studying a relaxation condition with feedback and a MI task condition without feedback as well. Right-handed healthy subjects performed two tasks, i.e., MI and relaxation, in alternating order. Each of the tasks (MI vs. relaxation) was studied with and without feedback. The respective event-driven oscillatory activity, i.e., sensorimotor desynchronization (during MI) or synchronization (during relaxation), was rewarded with contingent feedback. Importantly, feedback onset was delayed to study the task-related cortical activity in the absence of feedback provision during the delay period. The reward modality was alternated every 15 trials between proprioceptive and visual feedback. Proprioceptive input was superior to visual input to increase the range of task-related spectral perturbations in the α- and β-band, and was necessary to consistently achieve MI-related sensorimotor desynchronization (ERD) significantly below baseline. These effects occurred in task periods without feedback as well. The increased accuracy and duration of learned brain self-regulation achieved in the proprioceptive condition was specific to the β-band. MI-related operant learning of brain self-regulation is facilitated by proprioceptive feedback and mediated in the sensorimotor β-band.

The differential outcomes procedure can overcome self-bias in perceptual matching

There are biases in perceptual matching between shapes and labels referring to familiar others, compared with when the labels refer to unfamiliar people. We assessed whether these biases could be affected by differential feedback (using the differential outcomes procedure [DOP]) compared with when feedback is provided using a nondifferential outcomes procedure (NOP). Participants formed associations between simple geometric shapes and labels referring to people the participant did or did not know (self, best friend, other). Subsequently, the task was to match a label to one of two shapes shown on a trial. When feedback for correct responses was given following the NOP condition, matches were faster to known people (self and friend) compared with those to an unknown person (stranger). However, this advantage for known personal relations was eliminated when participants were given feedback for correct responses following the DOP condition. The data are consistent with prior work showing that the DOP can facilitate the learning of taxing associations (for the stranger stimuli relative to the familiar self and friend stimuli). In addition, the results suggest that the facilitated perceptual matching for stimuli associated to individuals known personally may reflect better individuation of the association between the shape stimulus and the label, a process enhanced by using a DOP for associations with unfamiliar people.

Goal-directed, habitual and Pavlovian prosocial behavior

Although prosocial behaviors have been widely studied across disciplines, the mechanisms underlying them are not fully understood. Evidence from psychology, biology and economics suggests that prosocial behaviors can be driven by a variety of seemingly opposing factors: altruism or egoism, intuition or deliberation, inborn instincts or learned dispositions, and utility derived from actions or their outcomes. Here we propose a framework inspired by research on reinforcement learning and decision making that links these processes and explains characteristics of prosocial behaviors in different contexts. More specifically, we suggest that prosocial behaviors inherit features of up to three decision-making systems employed to choose between self- and other- regarding acts: a goal-directed system that selects actions based on their predicted consequences, a habitual system that selects actions based on their reinforcement history, and a Pavlovian system that emits reflexive responses based on evolutionarily prescribed priors. This framework, initially described in the field of cognitive neuroscience and machine learning, provides insight into the potential neural circuits and computations shaping prosocial behaviors. Furthermore, it identifies specific conditions in which each of these three systems should dominate and promote other- or self- regarding behavior.

Visual recognition memory enhancement in children through differential outcomes

OBJECTIVE

The use of differential outcomes has been shown to enhance discriminative learning and face recognition in children and adults. In this study, we further investigated whether the differential outcome procedure (DOP) would also be effective in improving recognition memory for a wide range of stimuli with varying visual complexity (familiar objects, abstract stimuli, and complex scenes) in 5- and 7-year-old children.

METHOD

Participants viewed a sample stimulus and, after a short (5s) or a long (15s) delay, they had to identify the previously seen stimulus among four choice alternatives. In the differential outcomes condition, each sample stimulus was paired with a specific outcome; whereas in the non-differential conditions outcomes were administered randomly. In Experiment 2, we replicated Experiment 1 but in addition we asked participants to perform an articulatory suppression task to prevent verbal rehearsal.

RESULTS

Children showed a greater overall visual delayed recognition when differential outcomes were arranged in both experiments. The type of stimulus being used modulated this effect; a beneficial effect of the differential outcomes training was evident with abstract objects in Experiment 1 and with both, abstract objects and scenes in Experiment 2.

Differential outcomes training improves face recognition memory in children and in adults with Down syndrome

Previous studies have demonstrated that the differential outcomes procedure (DOP), which involves paring a unique reward with a specific stimulus, enhances discriminative learning and memory performance in several populations. The present study aimed to further investigate whether this procedure would improve face recognition memory in 5- and 7-year-old children (Experiment 1) and adults with Down syndrome (Experiment 2). In a delayed matching-to-sample task, participants had to select the previously shown face (sample stimulus) among six alternatives faces (comparison stimuli) in four different delays (1, 5, 10, or 15s). Participants were tested in two conditions: differential, where each sample stimulus was paired with a specific outcome; and non-differential outcomes, where reinforcers were administered randomly. The results showed a significantly better face recognition in the differential outcomes condition relative to the non-differential in both experiments. Implications for memory training programs and future research are discussed.

The effects of differential outcomes and different types of consequential stimuli on 7-year-old children's discriminative learning and memory

Researchers have demonstrated that discriminative learning is facilitated when a particular outcome is associated with each relation to be learned. Our primary purpose in the two experiments reported here was to assess whether the differential outcomes procedure (DOP) would enhance 7-year-old children's learning of symbolic discriminations using three different forms of consequences in which (1) reinforcers are given when correct choices are made ("+"), (2) reinforcers are withdrawn when errors are made ("-"), or (3) children receive a reinforcer following a correct choice and lose one following an incorrect choice ("+/-"), as well as different types of reinforcers (secondary and primary reinforcers, Experiment 1; primary reinforcers alone, Experiment 2). Participants learned the task faster and showed significantly better performance whenever differential outcomes were arranged independently of (1) the way of providing consequences (+, -, or +/-) and (2) the type of reinforcers being used. Interestingly, as in a previous study with 5-year-old children (Martínez, Estévez, Fuentes, & Overmier, The Quarterly Journal of Experimental Psychology 62(8):1617-1630, 2009), the use of the DOP also enhanced long-term persistence of learning.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

Cocaine cue-induced dopamine release in amygdala and hippocampus: a high-resolution PET [¹⁸F]fallypride study in cocaine dependent participants

Drug-related cues are potent triggers for relapse in people with cocaine dependence. Dopamine (DA) release within a limbic network of striatum, amygdala and hippocampus has been implicated in animal studies, but in humans it has only been possible to measure effects in the striatum. The objective here was to measure drug cue-induced DA release in the amygdala and hippocampus using high-resolution PET with [(18)F]fallypride. Twelve cocaine-dependent volunteers (mean age: 39.6 ± 8.0 years; years of cocaine use: 15.9 ± 7.4) underwent two [(18)F]fallypride high-resolution research tomography-PET scans, one with exposure to neutral cues and one with cocaine cues. [(18)F]Fallypride non-displaceable-binding potential (BPND) values were derived for five regions of interest (ROI; amygdala, hippocampus, ventral limbic striatum, associative striatum, and sensorimotor striatum). Subjective responses to the cues were measured with visual analog scales and grouped using principal component analysis. Drug cue exposure significantly decreased BPND values in all five ROI in subjects who had a high-, but not low-, craving response (limbic striatum: p=0.019, associative striatum: p=0.008, sensorimotor striatum: p=0.004, amygdala: p=0.040, and right hippocampus: p=0.025). Individual differences in the cue-induced craving response predicted the magnitude of [(18)F]fallypride responses within the striatum (ventral limbic: r=0.581, p=0.048; associative: r=0.589, p=0.044; sensorimotor: r=0.675, p=0.016). To our knowledge this study provides the first evidence of drug cue-induced DA release in the amygdala and hippocampus in humans. The preferential induction of DA release among high-craving responders suggests that these aspects of the limbic reward network might contribute to drug-seeking behavior.

Egr-1 induction provides a genetic response to food aversion in zebrafish

As soon as zebrafish larvae start eating, they exhibit a marked aversion for bitter and acidic substances, as revealed by a consumption assay, in which fluorescent Tetrahymena serve as a feeding basis, to which various stimuli can be added. Bitter and acidic substances elicited an increase in mRNA accumulation of the immediate-early response gene egr-1, as revealed by in situ hybridization. Conversely, chemostimulants that did not induce aversion did not induce egr-1 response. Maximum labeling was observed in cells located in the oropharyngeal cavity and on the gill rakers. Gustatory areas of the brain were also labeled. Interestingly, when bitter tastants were repeatedly associated with food reward, zebrafish juveniles learned to ingest food in the presence of the bitter compound. After habituation, the acquisition of acceptance for bitterness was accompanied by a loss of egr-1 labeling. Altogether, our data indicate that egr-1 participates specifically in food aversion. The existence of reward-coupled changes in taste sensitivity in humans suggests that our results are relevant to situations in humans.

Schizophrenia in translation: dissecting motivation in schizophrenia and rodents

The negative symptoms of schizophrenia include deficits in motivation, for which there is currently no treatment available. Animal models provide a powerful tool for identifying the potential pathophysiological mechanisms underlying the motivation deficits of schizophrenia with the aim of discovering novel treatment targets. The success of such an approach critically depends on meticulously detailed analysis of motivational phenotypes in patients and in animal models. Here, we review the results of recent human behavioral and imaging studies of motivation, and we relate those findings to the results from animal studies, including a mouse model of striatal dopamine D2 receptor hyperfunction. The motivational deficit in patients with schizophrenia is not due to an inability to experience pleasure in the moment as hedonic reaction appears intact in patients. Instead, the motivation deficit represents a reduced capacity for anticipating future pleasure resulting from goal-directed action. The diminished anticipation appears to be a consequence of an inability to accurately represent the expected reward values of actions. A strikingly similar phenotype in incentive motivation has also been described in mice with striatal dopamine D2 receptor hyperfunction. These convergent findings identify potential pathophysiological mechanisms that underlie the deficit in anticipatory motivation, and importantly, the mouse model provides a tool for investigating novel treatment strategies, which we discuss here.

Translational rodent models of Korsakoff syndrome reveal the critical neuroanatomical substrates of memory dysfunction and recovery

Investigation of the amnesic disorder Korsakoff Syndrome (KS) has been vital in elucidating the critical brain regions involved in learning and memory. Although the thalamus and mammillary bodies are the primary sites of neuropathology in KS, functional deactivation of the hippocampus and certain cortical regions also contributes to the chronic cognitive dysfunction reported in KS. The rodent pyrithiamine-induced thiamine deficiency (PTD) model has been used to study the extent of hippocampal and cortical neuroadaptations in KS. In the PTD model, the hippocampus, frontal and retrosplenial cortical regions display loss of cholinergic innervation, decreases in behaviorally stimulated acetylcholine release and reductions in neurotrophins. While PTD treatment results in significant impairment in measures of spatial learning and memory, other cognitive processes are left intact and may be recruited to improve cognitive outcome. In addition, behavioral recovery can be stimulated in the PTD model by increasing acetylcholine levels in the medial septum, hippocampus and frontal cortex, but not in the retrosplenial cortex. These data indicate that although the hippocampus and frontal cortex are involved in the pathogenesis of KS, these regions retain neuroplasticity and may be critical targets for improving cognitive outcome in KS.

Visual working memory in deaf children with diverse communication modes: improvement by differential outcomes

Although visual functions have been proposed to be enhanced in deaf individuals, empirical studies have not yet established clear evidence on this issue. The present study aimed to determine whether deaf children with diverse communication modes had superior visual memory and whether their performance was improved by the use of differential outcomes. Severely or profoundly deaf children who employed spoken Spanish, Spanish Sign Language (SSL), and both spoken Spanish and SSL modes of communication were tested in a delayed matching-to-sample task for visual working memory assessment. Hearing controls were used to compare performance. Participants were tested in two conditions, differential outcome and non-differential outcome conditions. Deaf groups with either oral or SSL modes of communication completed the task with less accuracy than bilingual and control hearing children. In addition, the performances of all groups improved through the use of differential outcomes.

Minimizing sleep deprivation effects in healthy adults by differential outcomes

Sleep deprivation reduces vigilance or arousal levels, affecting the efficiency of certain cognitive functions such as learning and memory. Here we assessed whether the differential outcomes procedure (DOP), a learning procedure that has proved useful to ameliorate episodic memory deficits, can also improve memory performance in sleep-deprived participants. Photographs were presented as sample faces. A probe face was then presented for recognition after either short or long delays. In the differential outcomes condition a unique reinforcer followed correct responses. In the non-differential outcomes condition reinforcers were provided in a random manner. The results indicated that the DOP prevented the recognition memory to decrement during the long delay in the control group, replicating previous findings. The sleep-deprived group showed DOP benefits mainly with the short delay, when working memory could be affected by low arousal. These findings confirm that the DOP can overcome impaired recognition memory due to sleep deprivation conditions.

Brain and behavioral pathology in an animal model of Wernicke's encephalopathy and Wernicke-Korsakoff Syndrome

Animal models provide the opportunity for in-depth and experimental investigation into the anatomical and physiological underpinnings of human neurological disorders. Rodent models of thiamine deficiency have yielded significant insight into the structural, neurochemical and cognitive deficits associated with thiamine deficiency as well as proven useful toward greater understanding of memory function in the intact brain. In this review, we discuss the anatomical, neurochemical and behavioral changes that occur during the acute and chronic phases of thiamine deficiency and describe how rodent models of Wernicke-Korsakoff Syndrome aid in developing a more detailed picture of brain structures involved in learning and memory.

Training with differential outcomes enhances discriminative learning and visuospatial recognition memory in children born prematurely

Previous studies have demonstrated that discriminative learning is facilitated when a particular outcome is associated with each relation to be learned. When this training procedure is applied (the differential outcome procedure; DOP), learning is faster and more accurate than when the more common non-differential outcome procedure is used. This enhancement of accuracy and acquisition has been called the differential outcome effect (DOE). Our primary purpose in the present study was to explore the DOE in children born with great prematurity performing a discriminative learning task (Experiment 1) or a delayed visuospatial recognition task (Experiment 2). In Experiment 1, participants showed a faster learning and a better performance when differential outcomes were used. In Experiment 2, a significant DOE was also observed. That is, premature children performed the visuospatial recognition task better when they received differential outcomes following their correct responses. By contrast, the overall performance of full-term children was similar in both differential and non-differential conditions. These results are first to show that the DOP can enhance learning of conditional discriminations and recognition memory in children born prematurely with very low birth-weight.

Vagal control of satiety and hormonal regulation of appetite

The paradigm for the control of feeding behavior has changed significantly. In this review, we present evidence that the separation of function in which cholecystokinin (CCK) controls short-term food intake and leptin regulate long-term eating behavior and body weight become less clear. In addition to the hypothalamus, the vagus nerve is critically involved in the control of feeding by transmitting signals arising from the upper gut to the nucleus of the solitary tract. Among the peripheral mediators, CCK is the key peptide involved in generating the satiety signal via the vagus. Leptin receptors have also been identified in the vagus nerve. Studies in the rodents clearly indicate that leptin and CCK interact synergistically to induce short-term inhibition of food intake and long-term reduction of body weight. The synergistic interaction between vagal CCK-A receptor and leptin is mediated by the phosphorylation of signal transducer and activator of transcription3 (STAT3), which in turn, activates closure of K(+) channels, leading to membrane depolarization and neuronal firing. This involves the interaction between CCK/SRC/phosphoinositide 3-kinase cascades and leptin/Janus kinase-2/phosphoinositide 3-kinase/STAT3 signaling pathways. It is conceivable that malfunctioning of these signaling molecules may result in eating disorders.

Enhancing recognition memory in adults through differential outcomes

It has been widely demonstrated that the differential outcomes procedure (DOP) facilitates both the learning of conditional relationships and the memory for the conditional stimuli in animal subjects. For conditional discriminations in humans, the DOP also produces an increase in the speed of acquisition and/or final accuracy. However, the potential facilitative effects of differential outcomes in human memory have not been fully assessed. In the present study, we aimed to test whether this procedure improves performance on a recognition memory task in healthy adults. Participants showed significantly better delayed face recognition when differential outcomes were used. This novel finding is discussed in the light of other studies on the differential outcomes effect (DOE) in both animals and humans, and implications for future research are presented.

Appetitive operant conditioning in mice: heritability and dissociability of training stages

To study the heritability of different training stages of appetitive operant conditioning, we carried out behavioral screening of 5 standard inbred mouse strains, 28 recombinant-inbred (BxD) mouse lines and their progenitor strains C57BL/6J and DBA/2J. We also computed correlations between successive training stages to study whether learning deficits at an advanced stage of operant conditioning may be dissociated from normal performance in preceding phases of training. The training consisted of two phases: an operant nose poking (NP) phase, in which mice learned to collect a sucrose pellet from a food magazine by NP, and an operant lever press and NP phase, in which mice had to execute a sequence of these two actions to collect a food pellet. As a measure of magazine oriented exploration, we also studied the nose poke entries in the food magazine during the intertrial intervals at the beginning of the first session of the nose poke training phase. We found significantly heritable components in initial magazine checking behavior, operant NP and lever press-NP. Performance levels in these phases were positively correlated, but several individual strains were identified that showed poor lever press-NP while performing well in preceding training stages. Quantitative trait loci mapping revealed suggestive likelihood ratio statistic peaks for initial magazine checking behavior and lever press-NP. These findings indicate that consecutive stages toward more complex operant behavior show significant heritable components, as well as dissociability between stages in specific mouse strains. These heritable components may reside in different chromosomal areas.

Risky decisions and their consequences: neural processing by boys with Antisocial Substance Disorder

BACKGROUND

Adolescents with conduct and substance problems ("Antisocial Substance Disorder" (ASD)) repeatedly engage in risky antisocial and drug-using behaviors. We hypothesized that, during processing of risky decisions and resulting rewards and punishments, brain activation would differ between abstinent ASD boys and comparison boys.

METHODOLOGY/PRINCIPAL FINDINGS

We compared 20 abstinent adolescent male patients in treatment for ASD with 20 community controls, examining rapid event-related blood-oxygen-level-dependent (BOLD) responses during functional magnetic resonance imaging. In 90 decision trials participants chose to make either a cautious response that earned one cent, or a risky response that would either gain 5 cents or lose 10 cents; odds of losing increased as the game progressed. We also examined those times when subjects experienced wins, or separately losses, from their risky choices. We contrasted decision trials against very similar comparison trials requiring no decisions, using whole-brain BOLD-response analyses of group differences, corrected for multiple comparisons. During decision-making ASD boys showed hypoactivation in numerous brain regions robustly activated by controls, including orbitofrontal and dorsolateral prefrontal cortices, anterior cingulate, basal ganglia, insula, amygdala, hippocampus, and cerebellum. While experiencing wins, ASD boys had significantly less activity than controls in anterior cingulate, temporal regions, and cerebellum, with more activity nowhere. During losses ASD boys had significantly more activity than controls in orbitofrontal cortex, dorsolateral prefrontal cortex, brain stem, and cerebellum, with less activity nowhere.

CONCLUSIONS/SIGNIFICANCE

Adolescent boys with ASD had extensive neural hypoactivity during risky decision-making, coupled with decreased activity during reward and increased activity during loss. These neural patterns may underlie the dangerous, excessive, sustained risk-taking of such boys. The findings suggest that the dysphoria, reward insensitivity, and suppressed neural activity observed among older addicted persons also characterize youths early in the development of substance use disorders.

Memory for reward location is enhanced even though acetylcholine efflux within the amygdala is impaired in rats with damage to the diencephalon produced by thiamine deficiency

A rodent model of diencephalic amnesia produced by thiamine deficiency (pyrithiamine-induced thiamine deficiency [PTD]) was implemented to assess both changes in behavior and acetylcholine (ACh) efflux in the amygdala across four training sessions of a delayed alternation task. Two versions of the delayed alternation task were used. In one version, when a correct alternation was made a unique reward was paired with each spatial location ([left arm-chocolate milk] or [right arm-rat chow]). This paradigm is called the differential outcomes procedure (DOP). In the second version of the task, correct delayed alternation resulted in the same rewards but randomized across location (Nondifferential Outcomes Procedure [NOP]). The PTD rats were impaired on the first session of delayed alternation testing. However, both control and PTD rats using the DOP performed significantly better on delayed alternation than rats trained with the NOP.This effect was driven primarily by the PTD rats in the DOP condition outperforming all other groups on sessions 2-4. Although ACh efflux in the amygdala increased during delayed alternation testing in all groups, the NOP-trained rats had a greater rise in training-related ACh release in the post-training period. This suggests that increased amygdalar cholinergic activation is more critical for processing spatial information than episodic reward information. These data correspond with the idea that cholinergic activation of the amygdala promotes processing in other neural systems.