How instructed knowledge modulates the neural systems of reward learning

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Galanin receptor 1 expressing neurons in hippocampal-prefrontal circuitry modulate goal directed attention and impulse control

While amino acid neurotransmitters are the main chemical messengers in the brain, they are co-expressed with neuropeptides which are increasingly recognized as modulators of cognitive pathways. For example, the neuropeptide galanin has been implicated in a wide range of pathological conditions in which frontal and temporal structures are compromised. In a recent study in rats, we discovered that direct pharmacological stimulation of galanin receptor type 1 (GalR1) in the ventral prefrontal cortex (vPFC) and ventral hippocampus (vHC) led to opposing effects on attention and impulse control behavior. In the present study, we investigate how subtypes of neurons expressing GalR1 in these two areas differentially contribute to these behaviors. We first establish that GalR1 is predominantly expressed in glutamatergic neurons in both the vPFC and HC. We develop a novel viral approach to gain genetic access to GalR1-expressing neurons and demonstrate that optogenetic excitation of GalR1 expressing neurons in the vPFC, but not vHC, selectively disrupts attention in a complex behavioral task. Finally, using fiber photometry, we measure the bulk calcium dynamics in GalR1-expressing neurons during the same task to demonstrate opposing activity in vPFC and vHC. These results are consistent with our previous work demonstrating differential behavioral effects induced by GalR1 activating in vPFC and vHC. These results indicate the distinct neuromodulatory and behavioral contributions of galanin mediated by subclasses of neurons in the hippocampal and prefrontal circuitry.

Comparing experience- and description-based economic preferences across 11 countries

Recent evidence indicates that reward value encoding in humans is highly context dependent, leading to suboptimal decisions in some cases, but whether this computational constraint on valuation is a shared feature of human cognition remains unknown. Here we studied the behaviour of n = 561 individuals from 11 countries of markedly different socioeconomic and cultural makeup. Our findings show that context sensitivity was present in all 11 countries. Suboptimal decisions generated by context manipulation were not explained by risk aversion, as estimated through a separate description-based choice task (that is, lotteries) consisting of matched decision offers. Conversely, risk aversion significantly differed across countries. Overall, our findings suggest that context-dependent reward value encoding is a feature of human cognition that remains consistently present across different countries, as opposed to description-based decision-making, which is more permeable to cultural factors.

Disentangling the contribution of individual and social learning processes in human advice-taking behavior

The study of social learning examines how individuals learn from others by means of observation, imitation, or compliance with advice. However, it still remains largely unknown whether social learning processes have a distinct contribution to behavior, independent from non-social trial-and-error learning that often occurs simultaneously. 153 participants completed a reinforcement learning task, where they were asked to make choices to gain rewards. Advice from an artificial teacher was presented in 60% of the trials, allowing us to compare choice behavior with and without advice. Results showed a strong and reliable tendency to follow advice (test-retest reliability ~0.73). Computational modeling suggested a unique contribution of three distinct learning strategies: (a) individual learning (i.e., learning the value of actions, independent of advice), (b) informed advice-taking (i.e., learning the value of following advice), and (c) non-informed advice-taking (i.e., a constant bias to follow advice regardless of outcome history). Comparing artificial and empirical data provided specific behavioral regression signatures to both informed and non-informed advice taking processes. We discuss the theoretical implications of integrating internal and external information during the learning process.

Human ventromedial prefrontal cortex lesions enhance the effect of expectations on pain perception

Pain is strongly modulated by expectations and beliefs. Across species, subregions of ventromedial prefrontal cortex (VMPFC) are implicated in a variety of functions germane to pain, predictions, and learning. Human fMRI studies show that VMPFC activity tracks expectations about pain and mediates expectancy effects on pain-related activity in other brain regions. Prior lesion studies suggest that VMPFC may instead play a more general role in generating affective responses to painful stimuli. To test whether VMPFC is required to generate affective responses to pain or is more specifically involved in expectancy-based pain modulation, we studied responses to heat stimuli in five adults with bilateral surgical lesions of VMPFC and twenty healthy adults without brain damage. All participants underwent a quantitative sensory testing procedure followed by a pain expectancy task in which cues predicting either low or high pain were followed by intermittent medium intensity heat stimuli. Compared to adults without brain damage, individuals with VMPFC lesions reported larger differences in expected pain based on predictive cues and failed to update expectations following the covert introduction of unexpected medium temperature stimuli. Consistent with observed expectancy differences, subjective pain unpleasantness ratings in the VMPFC lesion group were more strongly modulated by cue during thermal stimulation. We found no group differences in overall pain sensitivity, nor in relationships between pain and autonomic arousal, suggesting that VMPFC damage specifically enhances the effect of expectations on pain processing, likely driven by impaired integration of new sensory feedback to update expectations about pain. These results provide essential new data regarding the specific functional contribution of VMPFC to pain modulation.

Teachers recruit mentalizing regions to represent learners' beliefs

Teaching enables humans to impart vast stores of culturally specific knowledge and skills. However, little is known about the neural computations that guide teachers' decisions about what information to communicate. Participants (N = 28) played the role of teachers while being scanned using fMRI; their task was to select examples that would teach learners how to answer abstract multiple-choice questions. Participants' examples were best described by a model that selects evidence that maximizes the learner's belief in the correct answer. Consistent with this idea, participants' predictions about how well learners would do closely tracked the performance of an independent sample of learners (N = 140) who were tested on the examples they had provided. In addition, regions that play specialized roles in processing social information, namely the bilateral temporoparietal junction and middle and dorsal medial prefrontal cortex, tracked learners' posterior belief in the correct answer. Our results shed light on the computational and neural architectures that support our extraordinary abilities as teachers.

Statistical information about reward timing is insufficient for promoting optimal persistence decisions

When deciding how long to keep waiting for delayed rewards that will arrive at an uncertain time, different distributions of possible reward times dictate different optimal strategies for maximizing reward. When reward timing distributions are heavy-tailed (e.g., waiting on hold) there is a point at which waiting is no longer advantageous because the opportunity cost of waiting is too high. Alternatively, when reward timing distributions have more predictable timing (e.g., uniform), it is advantageous to wait as long as necessary for the reward. Although people learn to approximate optimal strategies, little is known about how this learning occurs. One possibility is that people learn a general cognitive representation of the probability distribution that governs reward timing and then infer a strategy from that model of the environment. Another possibility is that they learn an action policy in a way that depends more narrowly on direct task experience, such that general knowledge of the reward timing distribution is insufficient for expressing the optimal strategy. Here, in a series of studies in which participants decided how long to persist for delayed rewards before quitting, we provided participants with information about the reward timing distribution in several ways. Whether the information was provided through counterfactual feedback (Study 1), previous exposure (Studies 2a and 2b), or description (Studies 3a and 3b), it did not obviate the need for direct, feedback-driven learning in a decision context. Therefore, learning when to quit waiting for delayed rewards might depend on task-specific experience, not solely on probabilistic reasoning.

Evaluating instrumental learning and striatal-cortical functional connectivity in adolescent alcohol and cannabis use

Adolescence is a vulnerable time for the acquisition of substance use disorders, potentially relating to ongoing development of neural circuits supporting instrumental learning. Striatal-cortical circuits undergo dynamic changes during instrumental learning and are implicated in contemporary addiction theory. Human studies have not yet investigated these dynamic changes in relation to adolescent substance use. Here, functional magnetic resonance imaging was used while 135 adolescents without (AUD-CUDLow ) and with significant alcohol (AUDHigh ) or cannabis use disorder symptoms (CUDHigh ) performed an instrumental learning task. We assessed how cumulative experience with instrumental cues altered cue selection preferences and functional connectivity strength between reward-sensitive striatal and cortical regions. Adolescents in AUDHigh and CUDHigh groups were slower in learning to select optimal instrumental cues relative to AUD-CUDLow adolescents. The relatively fast learning observed for AUD-CUDLow adolescents coincided with stronger functional connectivity between striatal and frontoparietal regions during early relative to later periods of task experience, whereas the slower learning for the CUDHigh group coincided with the opposite pattern. The AUDHigh group not only exhibited slower learning but also produced more instrumental choice errors relative to AUD-CUDLow adolescents. For the AUDHigh group, Bayesian analyses evidenced moderate support for no experience-related changes in striatal-frontoparietal connectivity strength during the task. Findings suggest that adolescent cannabis use is related to slowed instrumental learning and delays in peak functional connectivity strength between the striatal-frontoparietal regions that support this learning, whereas adolescent alcohol use may be more closely linked to broader impairments in instrumental learning and a general depression of the neural circuits supporting it.

Instructions and experiential learning have similar impacts on pain and pain-related brain responses but produce dissociations in value-based reversal learning

Recent data suggest that interactions between systems involved in higher order knowledge and associative learning drive responses during value-based learning. However, it is unknown how these systems impact subjective responses, such as pain. We tested how instructions and reversal learning influence pain and pain-evoked brain activation. Healthy volunteers (n=40) were either instructed about contingencies between cues and aversive outcomes or learned through experience in a paradigm where contingencies reversed three times. We measured predictive cue effects on pain and heat-evoked brain responses using functional magnetic resonance imaging. Predictive cues dynamically modulated pain perception as contingencies changed, regardless of whether participants received contingency instructions. Heat-evoked responses in the insula, anterior cingulate, and other regions updated as contingencies changed, and responses in the prefrontal cortex mediated dynamic cue effects on pain, whereas responses in the brainstem’s rostroventral medulla (RVM) were shaped by initial contingencies throughout the task. Quantitative modeling revealed that expected value was shaped purely by instructions in the Instructed Group, whereas expected value updated dynamically in the Uninstructed Group as a function of error-based learning. These differences were accompanied by dissociations in the neural correlates of value-based learning in the rostral anterior cingulate, thalamus, and posterior insula, among other regions. These results show how predictions dynamically impact subjective pain. Moreover, imaging data delineate three types of networks involved in pain generation and value-based learning: those that respond to initial contingencies, those that update dynamically during feedback-driven learning as contingencies change, and those that are sensitive to instruction. Together, these findings provide multiple points of entry for therapies designs to impact pain.

Explicit knowledge of task structure is a primary determinant of human model-based action

Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and in individuals suffering from obsessive-compulsive or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of participants, and less in those with obsessive-compulsive disorder. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, in humans, explicit task structural knowledge is a primary determinant of model-based reinforcement learning and is most readily acquired from instruction rather than experience.

Dopaminergic challenge dissociates learning from primary versus secondary sources of information

Some theories of human cultural evolution posit that humans have social-specific learning mechanisms that are adaptive specialisations moulded by natural selection to cope with the pressures of group living. However, the existence of neurochemical pathways that are specialised for learning from social information and individual experience is widely debated. Cognitive neuroscientific studies present mixed evidence for social-specific learning mechanisms: some studies find dissociable neural correlates for social and individual learning, whereas others find the same brain areas and, dopamine-mediated, computations involved in both. Here, we demonstrate that, like individual learning, social learning is modulated by the dopamine D2 receptor antagonist haloperidol when social information is the primary learning source, but not when it comprises a secondary, additional element. Two groups (total N = 43) completed a decision-making task which required primary learning, from own experience, and secondary learning from an additional source. For one group, the primary source was social, and secondary was individual; for the other group this was reversed. Haloperidol affected primary learning irrespective of social/individual nature, with no effect on learning from the secondary source. Thus, we illustrate that dopaminergic mechanisms underpinning learning can be dissociated along a primary-secondary but not a social-individual axis. These results resolve conflict in the literature and support an expanding field showing that, rather than being specialised for particular inputs, neurochemical pathways in the human brain can process both social and non-social cues and arbitrate between the two depending upon which cue is primarily relevant for the task at hand.

Confirmation Bias in the Course of Instructed Reinforcement Learning in Schizophrenia-Spectrum Disorders

A large body of research attributes learning deficits in schizophrenia (SZ) to the systems involved in value representation (prefrontal cortex, PFC) and reinforcement learning (basal ganglia, BG) as well as to the compromised connectivity of these regions. In this study, we employed learning tasks hypothesized to probe the function and interaction of the PFC and BG in patients with SZ-spectrum disorders in comparison to healthy control (HC) subjects. In the Instructed Probabilistic Selection task (IPST), participants received false instruction about one of the stimuli used in the course of probabilistic learning which creates confirmation bias, whereby the instructed stimulus is overvalued in comparison to its real experienced value. The IPST was administered to 102 patients with SZ and 120 HC subjects. We have shown that SZ patients and HC subjects were equally influenced by false instruction in reinforcement learning (RL) probabilistic task (IPST) (p-value = 0.441); however, HC subjects had significantly higher learning rates associated with the process of overcoming cognitive bias in comparison to SZ patients (p-value = 0.018). The behavioral results of our study could be hypothesized to provide further evidence for impairments in the SZ-BG circuitry; however, this should be verified by neurofunctional imaging studies.

Rating expectations can slow aversive reversal learning

The process of learning allows organisms to develop predictions about outcomes in the environment, and learning is sensitive to both simple associations and higher order knowledge. However, it is unknown whether consciously attending to expectations shapes the learning process itself. Here, we directly tested whether rating expectations shapes arousal during classical conditioning. Participants performed an aversive learning paradigm wherein one image (CS+) was paired with shock on 50% of trials, while a second image (CS-) was never paired with shock. Halfway through the task, contingencies reversed. One group of participants rated the probability of upcoming shock on each trial, while the other group made no online ratings. We measured skin conductance response (SCR) evoked in response to the CS and used traditional analyses as well as quantitative models of reinforcement learning to test whether rating expectations influenced arousal and aversive reversal learning. Participants who provided online expectancy ratings displayed slower learning based on a hybrid model of adaptive learning and reduced reversal of SCR relative to those who did not rate expectations. Mediation analysis revealed that the effect of associative learning on SCR could be fully explained through its effects on subjective expectancy within the group who provided ratings. This suggests that the act of rating expectations reduces the speed of learning, likely through changes in attention, and that expectations directly influence arousal. Our findings indicate that higher order expectancy judgments can alter associative learning.

Executive Function Assigns Value to Novel Goal-Congruent Outcomes

People often learn from the outcomes of their actions, even when these outcomes do not involve material rewards or punishments. How does our brain provide this flexibility? We combined behavior, computational modeling, and functional neuroimaging to probe whether learning from abstract novel outcomes harnesses the same circuitry that supports learning from familiar secondary reinforcers. Behavior and neuroimaging revealed that novel images can act as a substitute for rewards during instrumental learning, producing reliable reward-like signals in dopaminergic circuits. Moreover, we found evidence that prefrontal correlates of executive control may play a role in shaping flexible responses in reward circuits. These results suggest that learning from novel outcomes is supported by an interplay between high-level representations in prefrontal cortex and low-level responses in subcortical reward circuits. This interaction may allow for human reinforcement learning over arbitrarily abstract reward functions.

How social learning amplifies moral outrage expression in online social networks

Moral outrage shapes fundamental aspects of social life and is now widespread in online social networks. Here, we show how social learning processes amplify online moral outrage expressions over time. In two preregistered observational studies on Twitter (7331 users and 12.7 million total tweets) and two preregistered behavioral experiments (N = 240), we find that positive social feedback for outrage expressions increases the likelihood of future outrage expressions, consistent with principles of reinforcement learning. In addition, users conform their outrage expressions to the expressive norms of their social networks, suggesting norm learning also guides online outrage expressions. Norm learning overshadows reinforcement learning when normative information is readily observable: in ideologically extreme networks, where outrage expression is more common, users are less sensitive to social feedback when deciding whether to express outrage. Our findings highlight how platform design interacts with human learning mechanisms to affect moral discourse in digital public spaces.

Effects of category learning strategies on recognition memory

Extant research has shown that previously acquired categorical knowledge affects recognition memory, and that differences in category learning strategies impact classification accuracy. However, it is unknown whether different learning strategies also have downstream effects on subsequent recognition memory. The present study investigates the effect of two unidimensional rule-based category learning strategies - learning (a) with or without explicit instruction, and (b) with or without supervision - on subsequent recognition memory. Our findings suggest that acquiring categorical knowledge increased both hits (Experiments 1 and 2) and false-alarms (Experiment 1) for category-congruent items regardless of the particular strategy employed in initially learning these categories. There were, however, small processing speed advantages during recognition memory for both explicit instruction and supervised practice relative to neither (Experiment 2). We discuss these findings in the context of how prior knowledge influences recognition memory, and in relation to similar findings showing schematic effects on episodic memory.

Effects of advice on experienced-based learning in adolescents and adults

Recent studies that compared effects of pre-learning advice on experience-based learning in adolescents and adults have yielded mixed results. Previous studies on this topic used choice tasks in which age-related differences in advice-related learning bias and exploratory choice behavior are difficult to dissociate. Moreover, these studies did not examine whether effects of advice depend on working memory load. In this preregistered study (in adolescents [13-15 years old] and adults [18-31 years old]), we addressed these issues by factorially combining advice and working memory load manipulations in an estimation task that does not require choices and hence eliminates the influence of known age-related differences in exploration. We found that advice guided participants' initial estimates in both age groups. When advice was correct, this improved estimation performance, especially in adolescents when working memory load was high. When advice was incorrect, it had a longer-lasting effect on adolescents' performance than on adults' performance. In contrast to previous findings in choice tasks, we found no evidence that advice biased learning in either age group. Taken together, our results suggest that learning in an estimation task improves between adolescence and adulthood but that the effects of advice on learning do not differ substantially between adolescents and adults.

Probabilistic value learning in medial temporal lobe amnesia

A prevailing view in cognitive neuroscience suggests that different forms of learning are mediated by dissociable memory systems, with a mesolimbic (i.e., midbrain and basal ganglia) system supporting incremental trial-and-error reinforcement learning and a hippocampal-based system supporting episodic memory. Yet, growing evidence suggests that the hippocampus may also be important for trial-and-error learning, particularly value or reward-based learning. In the present report, we use a lesion-based neuropsychological approach to clarify hippocampal contributions to such learning. Six amnesic patients with medial temporal lobe damage and a group of healthy controls were administered a simple value-based learning task involving probabilistic trial-and-error acquisition of stimulus-response-outcome (reward or none) contingencies modeled after Li et al. (Proceedings of the National Academy of Sciences , 2011, 108 (1), 55-60). As predicted, patients were significantly impaired on the task, demonstrating reduced learning of the contingencies. Our results provide further supportive evidence that the hippocampus' role in cognition extends beyond episodic memory tasks and call for further refinement of theoretical models of hippocampal functioning.

Reward history impacts attentional orienting and inhibitory control on untrained tasks

It has been robustly shown that stimuli with reward history receive attentional priority. However, the majority of this research tests reward history effects on attentional bias using similar tasks for both the reward learning phase and the unrewarded testing phase, which limits our understanding of how the effects of reward history generalize beyond the trained tasks and mental sets. Across two new experiments, the current study addresses these issues by first associating reward with a stimulus in a visual search paradigm, and then testing value-driven effects of that stimulus in untrained and unrewarded tasks, including a cueing paradigm, a go/no-go task, and a delay discounting task. Results of Experiment 1 demonstrate that history of reward association in a visual search task generalizes to value-driven attentional bias in a different attention paradigm (i.e., cueing), indicating these effects are indeed attributable to imbued value that can transfer to other tasks beyond that in which the reward was trained. The results of Experiment 2 demonstrate that in addition to eliciting attentional orienting on untrained tasks, reward history can lead to better inhibitory control in the go/no-go task. We find no evidence for reward history effects in the delay discounting task. Together, these experiments demonstrate that when the reward association task is in the attention domain, reward history modulates attentional priority, and this effect generalizes to untrained and unrewarded tasks that utilize both spatial and nonspatial attention.

Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia

Recent evidence suggests that the human hippocampus-known primarily for its involvement in episodic memory-plays a role in a host of motivationally relevant behaviors, including some forms of value-based decision-making. However, less is known about the role of the hippocampus in value-based learning. Such learning is typically associated with a striatal system, yet a small number of studies, both in human and nonhuman species, suggest hippocampal engagement. It is not clear, however, whether this engagement is necessary for such learning. In the present study, we used both functional MRI (fMRI) and lesion-based neuropsychological methods to clarify hippocampal contributions to value-based learning. In Experiment 1, healthy participants were scanned while learning value-based contingencies (whether players in a "game" win money) in the context of a probabilistic learning task. Here, we observed recruitment of the hippocampus, in addition to the expected ventral striatal (nucleus accumbens) activation that typically accompanies such learning. In Experiment 2, we administered this task to amnesic patients with medial temporal lobe damage and to healthy controls. Amnesic patients, including those with damage circumscribed to the hippocampus, failed to acquire value-based contingencies, thus confirming that hippocampal engagement is necessary for task performance. Control experiments established that this impairment was not due to perceptual demands or memory load. Future research is needed to clarify the mechanisms by which the hippocampus contributes to value-based learning, but these findings point to a broader role for the hippocampus in goal-directed behaviors than previously appreciated.

The neural and computational systems of social learning

Learning the value of stimuli and actions from others - social learning - adaptively contributes to individual survival and plays a key role in cultural evolution. We review research across species targeting the neural and computational systems of social learning in both the aversive and appetitive domains. Social learning generally follows the same principles as self-experienced value-based learning, including computations of prediction errors and is implemented in brain circuits activated across task domains together with regions processing social information. We integrate neural and computational perspectives of social learning with an understanding of behaviour of varying complexity, from basic threat avoidance to complex social learning strategies and cultural phenomena.

The neural processes of acquiring placebo effects through observation

Learning through social observation is critical for humans. The present study investigates the neural processes underlying the acquisition of placebo effects through observational learning. We created a new functional magnetic resonance imaging (fMRI) paradigm where participants (n ​= ​38, healthy, both sexes) observed a demonstrator experiencing pain relief by a placebo treatment cream and experiencing pain without a treatment (control cream), and subsequently performed the same procedure themselves. Participants demonstrated placebo hypoalgesia while they performed the procedure themselves, confirming that observational learning can lead to placebo effects. During the observational learning phase, fMRI analysis showed a modulation of the amygdalae, periaqueductal grey, temporoparietal junctions (TPJ), and dorsolateral prefrontal cortex (DLPFC). Connectivity between the DLPFC and TPJ during the observational learning task was modulated by the placebo treatment and predicted subsequent placebo effects. Mediation analysis further confirmed that the DLPFC-TPJ connectivity formally mediated the effect of the observed treatment condition on subsequent placebo effects. Additionally, pre-recorded resting state connectivity between the DLPFC and TPJ also predicted observationally-learned placebo effects. Our findings provide an understanding of the neural processes during the acquisition of placebo effects through observation and indicate a critical role for DLPFC-TPJ integration processes during observational learning of therapeutic outcomes.

How We Know What Not To Think

Humans often represent and reason about unrealized possible actions - the vast infinity of things that were not (or have not yet been) chosen. This capacity is central to the most impressive of human abilities: causal reasoning, planning, linguistic communication, moral judgment, etc. Nevertheless, how do we select possible actions that are worth considering from the infinity of unrealized actions that are better left ignored? We review research across the cognitive sciences, and find that the possible actions considered by default are those that are both likely to occur and generally valuable. We then offer a unified theory of why. We propose that (i) across diverse cognitive tasks, the possible actions we consider are biased towards those of general practical utility, and (ii) a plausible primary function for this mechanism resides in decision making.

Different brain networks mediate the effects of social and conditioned expectations on pain

Information about others' experiences can strongly influence our own feelings and decisions. But how does such social information affect the neural generation of affective experience, and are the brain mechanisms involved distinct from those that mediate other types of expectation effects? Here, we used fMRI to dissociate the brain mediators of social influence and associative learning effects on pain. Participants viewed symbolic depictions of other participants' pain ratings (social information) and classically conditioned pain-predictive cues before experiencing painful heat. Social information and conditioned stimuli each had significant effects on pain ratings, and both effects were mediated by self-reported expectations. Yet, these effects were mediated by largely separable brain activity patterns, involving different large-scale functional networks. These results show that learned versus socially instructed expectations modulate pain via partially different mechanisms-a distinction that should be accounted for by theories of predictive coding and related top-down influences.

Differential effects of instructed and objective feedback reliability on feedback-related brain activity

Feedback reliability refers to the probability that the same decision leads to the same positive or negative feedback in the future. Previous research has shown that unreliable feedback is associated with attenuated feedback-related brain activity in ERPs, represented by a reduced fronto-central valence effect (feedback-related negativity or reward positivity) and a reduced feedback-related P3. Here, we asked whether these effects reflect top-down mechanisms or whether they can be explained by implicit feedback-outcome contingency learning. In two experiments, participants performed a trial-and-error learning task while subjective or objective feedback reliability was varied across blocks. In Experiment 1, we manipulated instructed feedback reliability while holding objective feedback reliability constant. Low instructed feedback reliability led to an attenuation of the fronto-central valence effect and the P3. In Experiment 2, we manipulated objective feedback reliability while holding instructed feedback reliability constant. Here, no modulation of feedback-related brain activity was observed. These results suggest that effects of feedback reliability are driven by top-down mechanisms based on explicit knowledge. Specifically, effects on the fronto-central valence effect could indicate a devaluation of unreliable feedback or a bias on the generation or utilization of reward prediction errors.

Effects of feedback reliability on feedback-related brain activity: A feedback valuation account

Adaptive decision making relies on learning from feedback. Because feedback sometimes can be misleading, optimal learning requires that knowledge about the feedback's reliability be utilized to adjust feedback processing. Although previous research has shown that feedback reliability indeed influences feedback processing, the underlying mechanisms through which this is accomplished remain unclear. Here we propose that feedback processing is adjusted by the adaptive, top-down valuation of feedback. We assume that unreliable feedback is devalued relative to reliable feedback, thus reducing the reward prediction errors that underlie feedback-related brain activity and learning. A crucial prediction of this account is that the effects of feedback reliability are susceptible to contrast effects. That is, the effects of feedback reliability should be enhanced when both reliable and unreliable feedback are experienced within the same context, as compared to when only one level of feedback reliability is experienced. To evaluate this prediction, we measured the event-related potentials elicited by feedback in two experiments in which feedback reliability was varied either within or between blocks. We found that the fronto-central valence effect, a correlate of reward prediction errors during reinforcement learning, was reduced for unreliable feedback. But this result was obtained only when feedback reliability was varied within blocks, thus indicating a contrast effect. This suggests that the adaptive valuation of feedback is one mechanism underlying the effects of feedback reliability on feedback processing.

In Cocaine Dependence, Neural Prediction Errors During Loss Avoidance Are Increased With Cocaine Deprivation and Predict Drug Use

BACKGROUND

In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state-dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility.

METHODS

In a randomized crossover design, 22 participants with current cocaine use disorder completed a probabilistic loss-learning task during functional magnetic resonance imaging while on and off cocaine (44 sessions). Another 54 participants without Axis I psychopathology served as a secondary reference group. Within-drug state and paired-subjects' learning effects were assessed with computational model-derived individual learning parameters. Model-based neuroimaging analyses evaluated effects of drug use state on neural learning signals. Relationships among model-derived behavioral learning rates (α+, α-), neural prediction error signals (δ+, δ-), cocaine use, and desire to use were assessed.

RESULTS

During cocaine deprivation, cocaine-dependent individuals exhibited heightened positive learning rates (α+), heightened neural positive prediction error (δ+) responses, and heightened association of α+ with neural δ+ responses. The deprivation-enhanced neural learning signals were specific to successful loss avoidance, comparable to participants without psychiatric conditions, and mediated a relationship between chronicity of drug use and desire to use cocaine.

CONCLUSIONS

Neurocomputational learning signals are sensitive to drug use status and suggest that heightened reinforcement by successful avoidance of negative outcomes may contribute to drug seeking during deprivation. More generally, attention to drug use state is important for delineating substrates of addiction.

Social threat learning transfers to decision making in humans

In today’s world, indirect exposure to threatening situations is more common than ever, as illustrated by footage of terror and disaster in social media. How do such social threat learning experiences shape our decisions? We found that learning about threats from both observation and verbal information strongly influenced decision making. As with learning from our own experience, this influence could be either adaptive or maladaptive depending on whether the social information provided accurate expectations about the environment. Our findings can help explain both adaptive and pathological behaviors resulting from the indirect exposure to threatening events.

In today’s world, mass-media and online social networks present us with unprecedented exposure to second-hand, vicarious experiences and thereby the chance of forming associations between previously innocuous events (e.g., being in a subway station) and aversive outcomes (e.g., footage or verbal reports from a violent terrorist attack) without direct experience. Such social threat, or fear, learning can have dramatic consequences, as manifested in acute stress symptoms and maladaptive fears. However, most research has so far focused on socially acquired threat responses that are expressed as increased arousal rather than active behavior. In three experiments (n = 120), we examined the effect of indirect experiences on behaviors by establishing a link between social threat learning and instrumental decision making. We contrasted learning from direct experience (i.e., Pavlovian conditioning) (experiment 1) against two common forms of social threat learning—social observation (experiment 2) and verbal instruction (experiment 3)—and how this learning transferred to subsequent instrumental decision making using behavioral experiments and computational modeling. We found that both types of social threat learning transfer to decision making in a strong and surprisingly inflexible manner. Notably, computational modeling indicated that the transfer of observational and instructed threat learning involved different computational mechanisms. Our results demonstrate the strong influence of others’ expressions of fear on one’s own decisions and have important implications for understanding both healthy and pathological human behaviors resulting from the indirect exposure to threatening events.

The Role of Frontostriatal Systems in Instructed Reinforcement Learning: Evidence From Genetic and Experimentally-Induced Variation

Instructions have a powerful effect on learning and decision-making, biasing choice even in the face of disconfirming feedback. Detrimental biasing effects have been reported in a number of studies in which instruction was given prior to trial-and-error learning. Previous work has attributed individual differences in instructional bias to variations in prefrontal and striatal dopaminergic genes, suggesting a role for prefrontally-mediated cognitive control processes in biasing learning. The current study replicates and extends these findings. Human subjects performed a probabilistic reinforcement learning task after receiving inaccurate instructions about the quality of one of the options. In order to establish a causal relationship between prefrontal cortical mechanisms and instructional bias, we applied transcranial direct current stimulation over dorsolateral prefrontal cortex (anodal, cathodal, or sham) while subjects performed the task. We additionally genotyped subjects for the COMT Val158Met genetic polymorphism, which influences the breakdown of prefrontal dopamine, and for the DAT1/SLC6A3 variable number tandem repeat, which affects expression of striatal dopamine transporter. We replicated the finding that the COMT Met allele is associated with increased instructional bias and further demonstrated that variation in DAT1 has similar effects to variation in COMT, with 9-repeat carriers demonstrating increased bias relative to 10-repeat homozygotes. Consistent with increased top-down regulation of reinforcement learning, anodal subjects demonstrated greater bias relative to sham, though this effect was present only early in training. In contrast, there was no effect of cathodal stimulation. Finally, we fit computational models to subjects' data to better characterize the mechanisms underlying instruction bias. A novel choice bias model, in which instructions influence decision-making rather than learning, was found to best account for subjects' behavior. Overall, these data provide further evidence for the role of frontostriatal interactions in biasing instructed reinforcement learning, which adds to the growing literature documenting both costs and benefits of cognitive control.

Behavioural and neural evidence for self-reinforcing expectancy effects on pain

Beliefs and expectations often persist despite evidence to the contrary. Here we examine two potential mechanisms underlying such 'self-reinforcing' expectancy effects in the pain domain: modulation of perception and biased learning. In two experiments, cues previously associated with symbolic representations of high or low temperatures preceded painful heat. We examined trial-to-trial dynamics in participants' expected pain, reported pain and brain activity. Subjective and neural pain responses assimilated towards cue-based expectations, and pain responses in turn predicted subsequent expectations, creating a positive dynamic feedback loop. Furthermore, we found evidence for a confirmation bias in learning: higher- and lower-than-expected pain triggered greater expectation updating for high- and low-pain cues, respectively. Individual differences in this bias were reflected in the updating of pain-anticipatory brain activity. Computational modelling provided converging evidence that expectations influence both perception and learning. Together, perceptual assimilation and biased learning promote self-reinforcing expectations, helping to explain why beliefs can be resistant to change.

A Motivational Model of BCI-Controlled Heuristic Search

Several researchers have proposed a new application for human augmentation, which is to provide human supervision to autonomous artificial intelligence (AI) systems. In this paper, we introduce a framework to implement this proposal, which consists of using Brain⁻Computer Interfaces (BCI) to influence AI computation via some of their core algorithmic components, such as heuristic search. Our framework is based on a joint analysis of philosophical proposals characterising the behaviour of autonomous AI systems and recent research in cognitive neuroscience that support the design of appropriate BCI. Our framework is defined as a motivational approach, which, on the AI side, influences the shape of the solution produced by heuristic search using a BCI motivational signal reflecting the user's disposition towards the anticipated result. The actual mapping is based on a measure of prefrontal asymmetry, which is translated into a non-admissible variant of the heuristic function. Finally, we discuss results from a proof-of-concept experiment using functional near-infrared spectroscopy (fNIRS) to capture prefrontal asymmetry and control the progression of AI computation of traditional heuristic search problems.

Testing the Possibility of Model-based Pavlovian Control of Attention to Threat

Signals for reward or punishment attract attention preferentially, a principle termed value-modulated attention capture (VMAC). The mechanisms that govern the allocation of attention can be described with a terminology that is more often applied to the control of overt behaviors, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. Although instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible, we measured steady-state visual evoked potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage, participants underwent aversive threat conditioning with two conditioned stimuli (CSs): one that predicted pain (CS+) and one that predicted safety (CS-). Instructions given before the test stage allowed participants to infer whether novel, ambiguous CSs (new_CS+/new_CS-) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new_CS+ than new_CS-. This result is potentially indicative of model-based Pavlovian control of VMAC, but additional controls are necessary to verify this conclusively. This result underlines the potential transformative role of information and inference in emotion regulation.

What's in a word? How instructions, suggestions, and social information change pain and emotion

Instructions, suggestions, and other types of social information can have powerful effects on pain and emotion. Prominent examples include observational learning, social influence, placebo, and hypnosis. These different phenomena and their underlying brain mechanisms have been studied in partially separate literatures, which we discuss, compare, and integrate in this review. Converging findings from these literatures suggest that (1) instructions and social information affect brain systems associated with the generation of pain and emotion, and with reinforcement learning, and that (2) these changes are mediated by alterations in prefrontal systems responsible for top-down control and the generation of affective meaning. We argue that changes in expectation and appraisal, a process of assessing personal meaning and implications for wellbeing, are two potential key mediators of the effects of instructions and social information on affective experience. Finally, we propose a tentative model of how prefrontal regions, especially dorsolateral and ventromedial prefrontal cortex may regulate affective processing based on instructions and socially transmitted expectations more broadly.

Credit Assignment in a Motor Decision Making Task Is Influenced by Agency and Not Sensory Prediction Errors

Failures to obtain reward can occur from errors in action selection or action execution. Recently, we observed marked differences in choice behavior when the failure to obtain a reward was attributed to errors in action execution compared with errors in action selection (McDougle et al., 2016). Specifically, participants appeared to solve this credit assignment problem by discounting outcomes in which the absence of reward was attributed to errors in action execution. Building on recent evidence indicating relatively direct communication between the cerebellum and basal ganglia, we hypothesized that cerebellar-dependent sensory prediction errors (SPEs), a signal indicating execution failure, could attenuate value updating within a basal ganglia-dependent reinforcement learning system. Here we compared the SPE hypothesis to an alternative, "top-down" hypothesis in which changes in choice behavior reflect participants' sense of agency. In two experiments with male and female human participants, we manipulated the strength of SPEs, along with the participants' sense of agency in the second experiment. The results showed that, whereas the strength of SPE had no effect on choice behavior, participants were much more likely to discount the absence of rewards under conditions in which they believed the reward outcome depended on their ability to produce accurate movements. These results provide strong evidence that SPEs do not directly influence reinforcement learning. Instead, a participant's sense of agency appears to play a significant role in modulating choice behavior when unexpected outcomes can arise from errors in action execution.SIGNIFICANCE STATEMENT When learning from the outcome of actions, the brain faces a credit assignment problem: Failures of reward can be attributed to poor choice selection or poor action execution. Here, we test a specific hypothesis that execution errors are implicitly signaled by cerebellar-based sensory prediction errors. We evaluate this hypothesis and compare it with a more "top-down" hypothesis in which the modulation of choice behavior from execution errors reflects participants' sense of agency. We find that sensory prediction errors have no significant effect on reinforcement learning. Instead, instructions influencing participants' belief of causal outcomes appear to be the main factor influencing their choice behavior.

The Neural Basis of Changing Social Norms through Persuasion

Social norms regulate behavior, and changes in norms have a great impact on society. In most modern societies, norms change through interpersonal communication and persuasive messages found in media. Here, we examined the neural basis of persuasion-induced changes in attitude toward and away from norms using fMRI. We measured brain activity while human participants were exposed to persuasive messages directed toward specific norms. Persuasion directed toward social norms specifically activated a set of brain regions including temporal poles, temporo-parietal junction, and medial prefrontal cortex. Beyond these regions, when successful, persuasion away from an accepted norm specifically recruited the left middle temporal and supramarginal gyri. Furthermore, in combination with data from a separate attitude-rating task, we found that left supramarginal gyrus activity represented participant attitude toward norms and tracked the persuasion-induced attitude changes that were away from agreement.

Developmental Changes in Learning: Computational Mechanisms and Social Influences

Our ability to learn from the outcomes of our actions and to adapt our decisions accordingly changes over the course of the human lifespan. In recent years, there has been an increasing interest in using computational models to understand developmental changes in learning and decision-making. Moreover, extensions of these models are currently applied to study socio-emotional influences on learning in different age groups, a topic that is of great relevance for applications in education and health psychology. In this article, we aim to provide an introduction to basic ideas underlying computational models of reinforcement learning and focus on parameters and model variants that might be of interest to developmental scientists. We then highlight recent attempts to use reinforcement learning models to study the influence of social information on learning across development. The aim of this review is to illustrate how computational models can be applied in developmental science, what they can add to our understanding of developmental mechanisms and how they can be used to bridge the gap between psychological and neurobiological theories of development.

Suppression of Striatal Prediction Errors by the Prefrontal Cortex in Placebo Hypoalgesia

Classical learning theories predict extinction after the discontinuation of reinforcement through prediction errors. However, placebo hypoalgesia, although mediated by associative learning, has been shown to be resistant to extinction. We tested the hypothesis that this is mediated by the suppression of prediction error processing through the prefrontal cortex (PFC). We compared pain modulation through treatment cues (placebo hypoalgesia, treatment context) with pain modulation through stimulus intensity cues (stimulus context) during functional magnetic resonance imaging in 48 male and female healthy volunteers. During acquisition, our data show that expectations are correctly learned and that this is associated with prediction error signals in the ventral striatum (VS) in both contexts. However, in the nonreinforced test phase, pain modulation and expectations of pain relief persisted to a larger degree in the treatment context, indicating that the expectations were not correctly updated in the treatment context. Consistently, we observed significantly stronger neural prediction error signals in the VS in the stimulus context compared with the treatment context. A connectivity analysis revealed negative coupling between the anterior PFC and the VS in the treatment context, suggesting that the PFC can suppress the expression of prediction errors in the VS. Consistent with this, a participant's conceptual views and beliefs about treatments influenced the pain modulation only in the treatment context. Our results indicate that in placebo hypoalgesia contextual treatment information engages prefrontal conceptual processes, which can suppress prediction error processing in the VS and lead to reduced updating of treatment expectancies, resulting in less extinction of placebo hypoalgesia.SIGNIFICANCE STATEMENT In aversive and appetitive reinforcement learning, learned effects show extinction when reinforcement is discontinued. This is thought to be mediated by prediction errors (i.e., the difference between expectations and outcome). Although reinforcement learning has been central in explaining placebo hypoalgesia, placebo hypoalgesic effects show little extinction and persist after the discontinuation of reinforcement. Our results support the idea that conceptual treatment beliefs bias the neural processing of expectations in a treatment context compared with a more stimulus-driven processing of expectations with stimulus intensity cues. We provide evidence that this is associated with the suppression of prediction error processing in the ventral striatum by the prefrontal cortex. This provides a neural basis for persisting effects in reinforcement learning and placebo hypoalgesia.

Neural substrates of updating the prediction through prediction error during decision making

Learning of prediction error (PE), including reward PE and risk PE, is crucial for updating the prediction in reinforcement learning (RL). Neurobiological and computational models of RL have reported extensive brain activations related to PE. However, the occurrence of PE does not necessarily predict updating the prediction, e.g., in a probability-known event. Therefore, the brain regions specifically engaged in updating the prediction remain unknown. Here, we conducted two functional magnetic resonance imaging (fMRI) experiments, the probability-unknown Iowa Gambling Task (IGT) and the probability-known risk decision task (RDT). Behavioral analyses confirmed that PEs occurred in both tasks but were only used for updating the prediction in the IGT. By comparing PE-related brain activations between the two tasks, we found that the rostral anterior cingulate cortex/ventral medial prefrontal cortex (rACC/vmPFC) and the posterior cingulate cortex (PCC) activated only during the IGT and were related to both reward and risk PE. Moreover, the responses in the rACC/vmPFC and the PCC were modulated by uncertainty and were associated with reward prediction-related brain regions. Electric brain stimulation over these regions lowered the performance in the IGT but not in the RDT. Our findings of a distributed neural circuit of PE processing suggest that the rACC/vmPFC and the PCC play a key role in updating the prediction through PE processing during decision making.

Top-down control over feedback processing: The probability of valid feedback affects feedback-related brain activity

Adaptive decision-making requires that feedback about decision outcomes is adequately processed. Recent studies have shown that fronto-central event-related potentials (ERPs) are sensitive to feedback valence and can be used as an index of feedback processing. The present study investigated whether the processes involved in feedback evaluation are affected by top-down mechanisms driven by knowledge about feedback validity. In a simple decision task, participants had to make use of feedback to learn which one of two stimuli was associated with a reward in a later test phase. Feedback stimuli were followed by a cue indicating whether feedback was valid or invalid. Prior to each block, participants were informed about the frequency of valid feedback in this block. An effect of feedback validity was obtained not only for learning but also for fronto-central ERPs. While high-validity feedback was associated with a fronto-central valence effect, this effect was absent for low-validity feedback. This indicates that processes involved in feedback evaluation are affected by prior knowledge about feedback validity via top-down processes.

Neural Activities Underlying the Feedback Express Salience Prediction Errors for Appetitive and Aversive Stimuli

Feedback information is essential for us to adapt appropriately to the environment. The feedback-related negativity (FRN), a frontocentral negative deflection after the delivery of feedback, has been found to be larger for outcomes that are worse than expected, and it reflects a reward prediction error derived from the midbrain dopaminergic projections to the anterior cingulate cortex (ACC), as stated in reinforcement learning theory. In contrast, the prediction of response-outcome (PRO) model claims that the neural activity in the mediofrontal cortex (mPFC), especially the ACC, is sensitive to the violation of expectancy, irrespective of the valence of feedback. Additionally, increasing evidence has demonstrated significant activities in the striatum, anterior insula and occipital lobe for unexpected outcomes independently of their valence. Thus, the neural mechanism of the feedback remains under dispute. Here, we investigated the feedback with monetary reward and electrical pain shock in one task via functional magnetic resonance imaging. The results revealed significant prediction-error-related activities in the bilateral fusiform gyrus, right middle frontal gyrus and left cingulate gyrus for both money and pain. This implies that some regions underlying the feedback may signal a salience prediction error rather than a reward prediction error.

Dopamine dependence in aggregate feedback learning: A computational cognitive neuroscience approach

Procedural learning of skills depends on dopamine-mediated striatal plasticity. Most prior work investigated single stimulus-response procedural learning followed by feedback. However, many skills include several actions that must be performed before feedback is available. A new procedural-learning task is developed in which three independent and successive unsupervised categorization responses receive aggregate feedback indicating either that all three responses were correct, or at least one response was incorrect. Experiment 1 showed superior learning of stimuli in position 3, and that learning in the first two positions was initially compromised, and then recovered. An extensive theoretical analysis that used parameter space partitioning found that a large class of procedural-learning models, which predict propagation of dopamine release from feedback to stimuli, and/or an eligibility trace, fail to fully account for these data. The analysis also suggested that any dopamine released to the second or third stimulus impaired categorization learning in the first and second positions. A second experiment tested and confirmed a novel prediction of this large class of procedural-learning models that if the to-be-learned actions are introduced one-by-one in succession then learning is much better if training begins with the first action (and works forwards) than if it begins with the last action (and works backwards).

Adaptive behaviour and feedback processing integrate experience and instruction in reinforcement learning

In any non-deterministic environment, unexpected events can indicate true changes in the world (and require behavioural adaptation) or reflect chance occurrence (and must be discounted). Adaptive behaviour requires distinguishing these possibilities. We investigated how humans achieve this by integrating high-level information from instruction and experience. In a series of EEG experiments, instructions modulated the perceived informativeness of feedback: Participants performed a novel probabilistic reinforcement learning task, receiving instructions about reliability of feedback or volatility of the environment. Importantly, our designs de-confound informativeness from surprise, which typically co-vary. Behavioural results indicate that participants used instructions to adapt their behaviour faster to changes in the environment when instructions indicated that negative feedback was more informative, even if it was simultaneously less surprising. This study is the first to show that neural markers of feedback anticipation (stimulus-preceding negativity) and of feedback processing (feedback-related negativity; FRN) reflect informativeness of unexpected feedback. Meanwhile, changes in P3 amplitude indicated imminent adjustments in behaviour. Collectively, our findings provide new evidence that high-level information interacts with experience-driven learning in a flexible manner, enabling human learners to make informed decisions about whether to persevere or explore new options, a pivotal ability in our complex environment.

•

Study used instructions to modulate beliefs about informativeness of feedback.

•

Reversal learning performance improved with perceived informativeness.

•

Instruction-sensitive Hidden Markov Model provides good fit of behaviour.

•

EEG recordings of feedback-related negativity (FRN) show modulation by instructions.

•

Findings suggest reinforcement learning integrates experience with high-level beliefs.

Contributions of the hippocampus to feedback learning

Humans learn about the world in a variety of manners, including by observation, by associating cues in the environment, and via feedback. Across species, two brain structures have been predominantly involved in these learning processes: the hippocampus--supporting learning via observation and paired association--and the striatum--critical for feedback learning. This simple dichotomy, however, has recently been challenged by reports of hippocampal engagement in feedback learning, although the role of the hippocampus is not fully understood. The purpose of this experiment was to characterize the hippocampal response during feedback learning by manipulating varying levels of memory interference. Consistent with prior reports, feedback learning recruited the striatum and midbrain. Notably, feedback learning also engaged the hippocampus. The level of activity in these regions was modulated by the degree of memory interference, such that the greatest activation occurred during the highest level of memory interference. Importantly, the accuracy of information learned via feedback correlated with hippocampal activation and was reduced by the presence of high memory interference. Taken together, these findings provide evidence of hippocampal involvement in feedback learning by demonstrating both its relevance for the accuracy of information learned via feedback and its susceptibility to interference.

Instructed knowledge shapes feedback-driven aversive learning in striatum and orbitofrontal cortex, but not the amygdala

Socially-conveyed rules and instructions strongly shape expectations and emotions. Yet most neuroscientific studies of learning consider reinforcement history alone, irrespective of knowledge acquired through other means. We examined fear conditioning and reversal in humans to test whether instructed knowledge modulates the neural mechanisms of feedback-driven learning. One group was informed about contingencies and reversals. A second group learned only from reinforcement. We combined quantitative models with functional magnetic resonance imaging and found that instructions induced dissociations in the neural systems of aversive learning. Responses in striatum and orbitofrontal cortex updated with instructions and correlated with prefrontal responses to instructions. Amygdala responses were influenced by reinforcement similarly in both groups and did not update with instructions. Results extend work on instructed reward learning and reveal novel dissociations that have not been observed with punishments or rewards. Findings support theories of specialized threat-detection and may have implications for fear maintenance in anxiety.

DOI:http://dx.doi.org/10.7554/eLife.15192.001

Around the start of the twentieth century, Pavlov discovered that dogs salivate upon hearing a bell that has previously signaled that food is available. This phenomenon, in which a neutral stimulus (the bell) becomes associated with a particular outcome (such as food), is known as classical conditioning. The network of brain regions that supports this process – which includes the striatum, the amygdala and the prefrontal cortex – seems to work in a similar way across most animal species, including humans.

However, humans don’t learn only through experience or trial-and-error. We do not need to burn our hands to learn not to touch a hot stove: a verbal warning from others is usually sufficient. Experiments have shown that giving people verbal instructions on how to obtain rewards alters the activity of the striatum and prefrontal cortex. That is, the instructions interact with the circuit that also supports learning through experience. But is this the case for learning how to avoid punishments? That process depends largely on the amygdala, and it is possible that systems designed to detect threats may be less sensitive to verbal warnings.

To address this question, Atlas et al. taught people to associate one image with a mild electric shock, and another with the absence of a shock. After a number of trials, the relationships were reversed so that the previously neutral picture now predicted a shock and vice versa. Telling the participants about the reversal in advance triggered changes in the activity of the striatum and part of the prefrontal cortex. By contrast, such warnings had no effect on the amygdala. Instead, the activity of the amygdala changed only after the volunteers had experienced for themselves the new relationship between the pictures and the shocks.

A key next step is to find out whether this distinction between the two types of learning signals (those that can be updated by instructions and those that cannot) is specific to humans. While the current study relied upon language, there are other methods that could be used to explore this issue in animals. Furthermore, knowing that the human brain has a specialized threat detection system that is less sensitive to instructions could help us to understand and treat anxiety disorders. Atlas et al. hope to test this possibility directly in the future.

DOI:http://dx.doi.org/10.7554/eLife.15192.002